JP4519900B2 - Objective sound extraction device, objective sound extraction program, objective sound extraction method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To perform object sound extraction in which a high object sound extraction performance (noise removal performance) can be ensured by a small apparatus in acoustic environments where the object sound and other noises (non-object sounds) are mixed in acoustic signals obtained via a plurality of microphones and mixing conditions can vary. <P>SOLUTION: The object sound extraction apparatus includes sound source separation sections 10 for separating and generating an object sound separation signal corresponding to an object sound and other reference sound separation signals based on each combination of a main acoustic signal obtained through a main microphone receiving mainly input of the object sound and a plurality of other sub acoustic signals; an object sound separation signal synthesis section 20 for synthesizing the plurality of the separated and generated object sound separation signals; and a spectrum subtraction processing section 31 for extracting an acoustic signal corresponding to the object sound from the synthesis signal by performing a spectrum subtraction processing between the synthesis signal and the reference sound separation signals. <P>COPYRIGHT: (C)2009,JPO&amp;INPIT

Description

  The present invention relates to a target sound extraction apparatus that extracts and outputs an acoustic signal corresponding to a target sound from a predetermined target sound source based on an acoustic signal obtained through a microphone, a program thereof, and a method thereof.

In a device with a function to input sound emitted from a sound source such as a speaker, such as a telephone conference system, video conference system, ticket vending machine, car navigation system, etc., the sound can be emitted from a specific sound source (hereinafter referred to as a target sound source) by a microphone. Sound (hereinafter referred to as the target sound) is collected, but depending on the environment in which the sound source exists, the sound signal obtained through the microphone includes a noise component other than the sound signal component corresponding to the target sound. . In the acoustic signal obtained through the microphone, if the ratio of the noise component is large, the clarity of the target sound is impaired, and problems such as deterioration in call quality and automatic speech recognition rate occur.
Conventionally, as shown in Non-Patent Document 1, for example, a main microphone (speech microphone) that mainly inputs a voice uttered by a speaker (an example of a target sound) and noise around the speaker are mainly input (speaker). A two-input spectral subtraction process that uses a secondary microphone (noise microphone) and removes a noise signal based on the acoustic signal obtained through the secondary microphone from the acoustic signal obtained through the primary microphone. It has been. Here, the two-input spectrum subtraction process is an acoustic signal corresponding to the voice (the target sound) uttered by the speaker by the subtraction process of the time series feature vectors of the input signal from the main microphone and the input signal from the sub microphone. Is extracted (that is, noise components are removed).

By the way, as the sub microphone, a microphone disposed at a position different from the main microphone or a microphone having directivity in a direction different from the main microphone is adopted so that the target sound is not mixed therein. For this reason, when different noises arrive at each microphone from a plurality of directions, a situation may occur in which noise mainly picked up by the sub-microphone and noise mainly mixed in the main microphone are different. When such a situation occurs, the noise removal performance by the two-input spectrum subtraction process deteriorates.
On the other hand, Patent Document 1 uses a plurality of sub-microphones (noise microphones), and for each of the sound signals input through each of the sub-microphones (noise microphones), a signal obtained by selecting one of them according to the situation or predetermined There is shown a noise removal apparatus that performs the two-input spectrum subtraction process based on a synthesized signal weighted and averaged by weights and an acoustic signal input through the main microphone. As a result, it is said that it is possible to remove noise effectively even in an acoustic space where non-stationary noise whose properties change temporally and spatially occurs.
Further, in Patent Document 2, in a camera-integrated VTR apparatus, a correlation coefficient of a plurality of audio signals obtained by collecting sounds from a plurality of directions in a shooting range is obtained, and based on the correlation coefficient, the center of the shooting range is obtained. A technique for enhancing an audio signal from a person in a direction is shown.
In Patent Documents 3 to 5, a reference sound other than the target sound (non-target sound) is obtained from an acoustic signal (hereinafter referred to as a main acoustic signal) obtained through a microphone that mainly inputs the target sound (corresponding to the main microphone). The target sound extraction signal is obtained by removing the signal obtained by processing the acoustic signal obtained through the microphone (equivalent to the sub-microphone) mainly processed by the adaptive filter and the power of the extraction signal is minimized. Techniques for adjusting the filter are shown.

On the other hand, when there are a plurality of sound sources and a plurality of microphones (acoustic input means) in a predetermined acoustic space, individual sound signals (hereinafter referred to as sound source signals) from the plurality of sound sources are provided for each of the plurality of microphones. A superimposed acoustic signal (hereinafter referred to as a mixed acoustic signal) is input. A sound source separation processing method for identifying (separating) each of the sound source signals based only on the plurality of mixed sound signals input in this manner is a blind source separation method (hereinafter referred to as a BSS method). ).
Further, as one of the BSS sound source separation processes, there is a BSS sound source separation process based on an independent component analysis method (hereinafter referred to as ICA method). The BSS method based on the ICA method uses a fact that the sound source signals are statistically independent among a plurality of the mixed acoustic signals input through a plurality of microphones to generate a predetermined separation matrix (inverse mixing matrix). In this processing method, the sound source signal is identified (sound source separation) by performing a filtering process using an optimized separation matrix on the plurality of input mixed sound signals. At that time, the optimization of the separation matrix is used later by sequential calculation (learning calculation) based on the signal (separated signal) identified (separated) by the filter processing using the separation matrix set at a certain time. This is done by calculating the separation matrix.
Here, according to the sound source separation processing of the BSS method based on the ICA method, each separated signal has the same number of output terminals (also called output channels) as the number of mixed acoustic signals input (= the number of microphones). Is output through. Such BSS sound source separation processing based on the ICA method is described in detail in, for example, Non-Patent Document 2 and Non-Patent Document 3.
As sound source separation processing, sound source separation processing by binary masking processing (an example of binaural signal processing) is also known. The binary masking process is performed mainly for each mixed audio signal by comparing the level (power) of each divided frequency component (frequency bin) between the mixed audio signals input through a plurality of directional microphones. Is a sound source separation process that can be realized with a relatively low calculation load. For example, Non-Patent Document 4 and Non-Patent Document 5 are described in detail.
JP-A-6-67691 JP 2001-8285 A JP-A-6-83372 JP-A-6-90493 JP-A-6-165286 Kashimura et al., “Voice recognition using two-input noise reduction method”, IEICE Technical Report, SP-81, pp.41-48, 1989 Hiroshi Saruwatari, “Basics of Blind Sound Source Separation Using Array Signal Processing”, IEICE Technical Report, vol.EA2001-7, pp.49-56, April 2001. Tomoya Takatani et al., “High fidelity blind source separation using ICA based on SIMO model”, IEICE technical report, vol.US2002-87, EA2002-108, January 2003. RFLyon, "A computational model of binaural localization and separation", In Proc. ICASSP, 1983. M. Bodden, "Modeling human sound-source localization and the cocktail-party-effect", Acta Acoustica, vol.1, pp.43--55, 1993.

However, in the technique shown in Non-Patent Document 1 and the techniques shown in Patent Documents 3 to 5, when the target sound is mixed with the sub-microphone at a relatively large volume, the component of the acoustic signal corresponding to the target sound There is a problem that high noise removal performance cannot be obtained due to erroneous removal of noise as a noise component.
Further, as shown in Patent Document 1, a composite signal obtained by weighted averaging a plurality of audio signals input through a plurality of sub-microphones (noise microphones) with a predetermined weight is obtained as the two-input spectrum substructure. When employed as an input signal for processing, there is a problem in that noise removal performance deteriorates due to a mismatch between the weighted average weight and the degree of mixing of the target sound with respect to each of the plurality of sub-microphones due to changes in the acoustic environment. there were. Moreover, as shown in Patent Document 1, a signal selected from among a plurality of acoustic signals input through a plurality of sub-microphones (noise microphones) is employed as an input signal for the two-input spectrum subtraction process. In this case, under the situation where different noises arrive at each microphone from a plurality of directions, the noise component based on the acoustic signal leaked to the selection is not removed, and the noise removal performance is deteriorated.
In the technique disclosed in Patent Document 2, although the audio signal from the person in the center of the shooting range is emphasized, other audio signals remain and the target sound signal is not extracted.

In addition, if the BSS sound source separation processing based on the ICA method and the binary masking processing are executed based on the main acoustic signal and the sub acoustic signal, a separation signal corresponding to the target sound can be obtained. Depending on the acoustic environment, there is a problem in that the separated signal may contain a signal component of noise other than the target sound at a relatively high rate. For example, in the BSS sound source separation processing based on the ICA method, the sound source separation performance is deteriorated in an environment where the target sound and other noise sound sources are present more than the number of microphones, or the noise is reflected / reflected. To do.
As an acoustic input device that realizes a sharp directional characteristic, for example, an acoustic input device including a microphone array and a delay-and-sum type filter is known. However, the larger the directivity, the larger the size of the device. There was a point.
Therefore, the present invention has been made in view of the above circumstances, and the object of the present invention is that the target sound and other noise (non-target sound) are mixed in the acoustic signal obtained through a plurality of microphones, and An object of the present invention is to provide a target sound extraction device, a target sound extraction program, and a target sound extraction method that can ensure high target sound extraction performance (noise removal performance) with a small device in an acoustic environment in which the mixing state can change.

In order to achieve the above object, a target sound extraction apparatus according to the present invention (corresponding to a first invention described later) is a single target microphone obtained through one main microphone that mainly inputs a target sound output from a predetermined target sound source . a main audio signal, and a plurality of sub-acoustic signals obtained through the respective plurality of sub microphones having directivity in a plurality of different directions from the front Kinushi microphone, based on the extracted audio signal corresponding to the target sound Thus, the target sound extraction apparatus that outputs the extraction signal includes the following components (1-1) to (1-3).
(1-1) provided separately for each combination of the main acoustic signal and two acoustic signals consisting of said plurality of sub-acoustic signal, target sound based on the two acoustic signals, corresponding to the target sound Sound source separation means for separating and generating a separation signal and a reference sound separation signal corresponding to a reference sound other than the target sound by sound source separation processing by a blind sound source separation method based on an independent component analysis method .
(1-2) A target sound separation signal synthesizing unit that synthesizes a plurality of target sound separation signals separated and generated by the sound source separation unit.
(1-3) The target sound is obtained by performing spectral subtraction processing between the synthesized signal obtained by the target sound separation signal synthesizing means and the plurality of reference sound separation signals separated and generated by the sound source separation means. spectrum subtraction processing means to output an extraction signal to extract an acoustic signal from the obtained combined signal corresponding to the target sound by separating the signal combining means.
In the onset Ming, a plurality of target sound separation signal generated separated by the sound source separation unit is mainly including signal a signal component of the target sound. Similarly, the plurality of reference sound separation signals separated and generated by the sound source separation means are noise sound sources (sounds other than the target sound ( This signal mainly includes the signal component of the reference sound)).
However, depending on the position of the target sound source and the occurrence of noise for a plurality of microphones (the main microphone and the sub microphone), a relatively large amount of noise signal components other than the target sound may remain in the target sound separation signal. is there. Therefore, the synthesized signal obtained by synthesizing them is basically a signal mainly including the signal component of the target sound, but a relatively large amount of noise signal components may remain depending on the situation.
On the other hand, even when the synthesized signal includes a component of a noise sound (reference sound) other than the target sound, the signal obtained by extracting the signal component of the target sound from the synthesized signal by spectral subtraction processing is The signal from which the signal component of the reference sound separation signal is removed. In addition, the signal extracted by the spectrum subtraction processing means includes all signal components of the reference sound separation signal corresponding to each of the plurality of noises even when different noises (reference sounds) arrive at the main microphone from a plurality of directions. Is a signal that has been removed.
Therefore, a situation in which relatively strong specific noise arrives at the main microphone by performing the spectral subtraction process for removing the signal component of each of the reference sound separation signals on a composite signal of the plurality of target sound separation signals. Even in a situation where different noises arrive at the main microphone from a plurality of directions, high noise removal performance can be ensured.

By the way, generally, in the sound source separation process by the BSS method based on the ICA method, in order to obtain a high sound source separation performance, the number of sequential calculations (learning calculations) for obtaining a separation matrix used for the separation process (filter process) is increased. Alternatively, it is necessary to increase the number of samples of the acoustic signal (digital signal) used for the sequential calculation, which increases the calculation load. For example, when the sequential calculation is performed by a practical processor, it may take several times the time length of the input acoustic signal, and is not suitable for real-time processing.
On the other hand, spectral subtraction processing has a relatively small calculation load, and real-time processing is possible even with a practical processor.
Therefore, in the target sound extraction apparatus according to the present invention, the sound source separation process executed by the sound source separation means is a process shown in either of the following (1-1-1) or (1-1-2). Can be considered.
(1-1-1) the sound the sound source separation processing Oite sense the separating means is executed, sequential execution to separation signal filter processing based on a predetermined separating matrix to audio signals inputted in time series through the microphone And sequentially calculating the separation matrix used for the subsequent filter processing using all the section signals for each section signal divided by a predetermined period in the acoustic signal input in time series. And the number of sequential calculations is limited to a predetermined number.
(1-1-2) the sound the sound source separation processing Oite sense the separating means is executed, sequential execution to separation signal filter processing based on a predetermined separating matrix to audio signals inputted in time series through the microphone For each of the signals in the partial time zone on the head side of the section signal divided by a predetermined period in the acoustic signal input in time series, the subsequent filter processing using the signal The sequential calculation for obtaining the separation matrix used in the above is executed.
In the sound source separation process shown in (1-1-1) or (1-1-2), the filter process is a process with a small calculation load, and is executed together with the spectrum subtraction process by a practical processor. However, real-time processing can be realized with a relatively large margin.
In addition, the sequential calculation (learning calculation) in the sound source separation processing shown in (1-1-1) or (1-1-2) is the same as the number of sequential calculations and the acoustic signal (digital signal) used for the sequential calculation. This is a processing with a small calculation load with a limited number of samples (time zone). Therefore, even if the sequential calculation (learning calculation) is executed by a practical processor in combination with the filter processing and the spectral subtraction processing (real-time processing), the processing (the separation matrix used later) is performed in a relatively short time. Calculation) is completed. As a result, the separation matrix used for the filtering process is quickly updated to a state adapted to the change in the acoustic environment, and the adaptability of the target sound extraction to the change in the acoustic environment is increased. Further, due to the simplification of the sequential calculation (learning calculation), even if some noise is included in the separated signal obtained by the sound source separation process, the combination of the sound source separation process and the spectrum subtraction process is combined. Therefore, the target sound extraction performance can be sufficiently secured as a whole.

It is further preferable that the target sound extraction apparatus according to the present invention further includes the constituent elements shown in the following (1-4) and (1-5).
(1-4) its Re respectively based on three or more input audio signals obtained through directional three or more microphones in different directions, and one of the main sound among the three or more input audio signals Main / sub-acoustic signal specifying means for specifying a signal and a plurality of the sub-acoustic signals;
(1-5) Signal path switching means for switching the transmission path of the acoustic signal from the three or more microphones to the sound source separation means according to the identification result by the main / sub acoustic signal identification means.
For example, the main / sub-acoustic signal specifying means may determine, for example, a predetermined frequency component in each of the three or more input sound signals based on a comparison of signal strengths of the three or more input sound signals. It may be possible to specify one main acoustic signal and a plurality of sub-acoustic signals based on comparison of the proportions occupied.
By providing these components, the target sound extraction apparatus according to the present invention can change the position of the target sound source, so that a predetermined one of a plurality of microphones cannot be fixed as the main microphone. It can also be applied to.

The present invention can also be understood as a target sound extraction program that causes a computer to execute the processing executed by each means in the target sound extraction apparatus described above.
That is, target sound extraction program according to the present invention, one of the main sound signal obtained through one of the primary microphone for inputting a target sound to be outputted from a predetermined target source mainly before several different from the Kinushi microphone An object of causing a computer to execute a process of extracting an acoustic signal corresponding to the target sound and outputting an extracted signal based on a plurality of sub-acoustic signals obtained through a plurality of sub-microphones each having directivity in each direction It is a sound extraction program, and is a program that causes a computer to execute the following processes (2-1) to (2-3).
(2-1) individually for each combination of the main acoustic signal and two acoustic signals consisting of said plurality of sub-acoustic signal, based on the two acoustic signals, target sound separation signals corresponding to the target sound Sound source separation processing for separating and generating a reference sound separation signal corresponding to a reference sound other than the target sound by a blind sound source separation method based on an independent component analysis method .
(2-2) A target sound separation signal synthesis process for synthesizing a plurality of target sound separation signals separated and generated by the sound source separation process.
(2-3) Spectral subtraction processing is performed between the synthesized signal obtained by the target sound separation signal synthesis processing and the plurality of reference sound separation signals separated and generated by the sound source separation processing, whereby the target sound is obtained. A process of extracting an acoustic signal corresponding to the target sound from the synthesized signal obtained by the separated signal synthesizing process and outputting the extracted signal.
The same effect as that of the above-described target sound extraction apparatus according to the present invention can be obtained by a computer that executes the target sound extraction program described above.
The present invention can also be understood as a target sound extraction method in which each process in the target sound extraction program according to the present invention described above is executed by a computer.

According to the present invention (corresponding to the first invention described later), the target sound is mixed at a relatively large volume with respect to any of the sub-microphones in an acoustic environment where different noises arrive at each microphone from a plurality of directions. High noise removal performance can be ensured under such an acoustic environment, and even when such an acoustic environment changes.
Further, according to the present invention, as will be described later, even if the directivity of the main microphone itself is moderate, the target sound extraction device according to the present invention is an acoustic input device having a very steep directivity. Function. Moreover, the position of the sound source of the sound treated (removed) as noise by adjusting (approaching or moving away) the position or directionality of the sub microphone relative to the position or directionality of the main microphone Since the directivity of the target sound extraction apparatus according to the present invention can be adjusted, the convenience is high. Further, as will be described later, a device that functions as an acoustic input device having such steep or flexible directivity can be realized as a very small device.

Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings so that the present invention can be understood. The following embodiment is an example embodying the present invention, and does not limit the technical scope of the present invention.
FIG. 1 is a block diagram showing a schematic configuration of the target sound extraction device X1 according to the embodiment of the first invention, FIG. 2 is a conceptual diagram showing a process of target sound extraction processing in the target sound extraction device X1, and FIG. FIG. 4 is a block diagram showing a schematic configuration of the target sound extraction device X2 according to the embodiment of the second invention, FIG. 4 is a conceptual diagram showing a process of target sound extraction processing in the target sound extraction device X2, and FIG. 5 is an embodiment of the third invention. FIG. 6 is a conceptual diagram showing the process of target sound extraction processing in the target sound extraction device X3, and FIG. 7 is the target sound extraction performance of the target sound extraction devices X1 to X3. FIG. 8 is a diagram showing a second experimental condition for evaluating the target sound extraction performance of the target sound extraction devices X1 to X3, and FIG. 9 is a graph showing the first experimental condition for evaluating the target sound extraction device X1 to X3. Target sound extraction devices X1 to X3 and conventional target sound extraction FIG. 10 shows the target sound extraction performance of the target sound extraction devices X1 to X3 and the conventional target sound extraction processing under the second experimental conditions, and FIG. 11 shows the target sound extraction performance. FIG. 12 is a diagram showing the third experimental condition for evaluating the directivity of the extracting device X1, FIG. 12 is a diagram showing the directivity of the target sound extracting device X1 under the third experimental condition, and FIG. 13 is the target sound extracting device X1. FIG. 14 is a block diagram showing a schematic configuration of a sound source separation device Z that performs BSS type sound source separation processing based on the FDICA method, and FIG. 15 is a target sound. FIG. 16 is a time chart showing a first example of a processing sequence excluding learning calculation in the sound source separation processing of the extraction devices X1 to X3. FIG. 16 shows a first processing sequence excluding learning calculation in the sound source separation processing of the target sound extraction devices X1 to X3. 2 examples FIG. 17 is a time chart showing a learning calculation sequence according to the first embodiment in the sound source separation processing of the target sound extraction devices X1 to X3, and FIG. It is a time chart showing the sequence of the learning calculation which concerns on 2 Example.

[First invention]
First, the target sound extraction device X1 according to the embodiment of the first invention will be described with reference to the block diagram shown in FIG.
As shown in FIG. 1, the target sound extraction device X1 includes a sound input device V1 including a plurality of microphones, a plurality (three in FIG. 1) of sound source separation processing units 10 (10-1 to 10-3), a target sound. A separated signal synthesis processing unit 20 and a spectrum subtraction processing unit 31 are provided. Here, the acoustic input device V1 includes one main microphone 101 and a plurality of (three in FIG. 1) sub microphones 102 (102-1 to 102-3). The main microphone 101 and the plurality of sub microphones 102 are arranged at a plurality of different positions, respectively, or have directivity in a plurality of different directions.
The main microphone 101 is sound input means for mainly inputting sound (hereinafter referred to as target sound) emitted from a predetermined target sound source (for example, a speaker that can move within a predetermined range).
The plurality of sub-microphones 102-1 to 102-3 are arranged at a plurality of positions different from the main microphone 101, or have directivity in a plurality of different directions, respectively. It is an acoustic input means for inputting a reference sound (noise) other than sound. Note that the description of the sub microphone 102 is a general term for the plurality of sub microphones 102-1 to 102-3.
The main microphone 101 and the sub microphone 102 shown in FIG. 1 are microphones having directivity, and the sub microphones 102 are arranged so as to have directivities in a plurality of directions different from the main microphone 102, respectively. Yes.

When each of the main microphone 101 and the sub microphone 102 is a directional microphone, the direction (less than + 180 °) on one side with respect to the center direction (front direction) of the main microphone 101 (0 °) (for example, , + 90 ° direction) and a direction of less than −180 ° on the other side (for example, −90 ° direction), it is desirable that the pointing center direction (front direction) of the sub microphone 102 is set. .
It is also conceivable that the directivity directions of the microphones 101 and 102 are set in different directions in the same plane and in three-dimensionally different directions.

Then, the target sound extraction device X1 generates an acoustic signal corresponding to the target sound based on the main acoustic signal obtained through the main microphone 101 and the sub acoustic signals obtained through the other plurality of sub microphones 102. The extracted signal (hereinafter referred to as the target sound extraction signal) is output.
In the target sound extraction apparatus X1, the sound source separation processing unit 10, the target sound separation signal synthesis processing unit 20, and the spectrum subtraction processing unit 31 are executed by, for example, a DSP (Digital Signal Processor) which is an example of a computer and the DSP. It is embodied by a ROM in which a program is stored, an ASIC, or the like. In this case, the ROM stores in advance a program for causing the DSP to execute processing (described later) performed by the sound source separation processing unit 10, the target sound separation signal synthesis processing unit 20, and the spectrum subtraction processing unit 31. ing.

The sound source separation processing unit 10 (10-1 to 10-3) is provided for each combination of the main acoustic signal and each of the plurality of sub-acoustic signals. Based on the target sound separation signal corresponding to the target sound (identification signal of the target sound) and the reference sound separation corresponding to the reference sound (may be referred to as noise) other than the target sound. A sound source separation process for separating and generating a signal (identification signal of a reference sound) is executed (an example of the sound source separation means).
An A / D converter (not shown) is provided between each of the microphones 101 and 102 and the sound source separation processing unit 10, and an acoustic signal converted into a digital signal by the A / D converter is It is transmitted to the sound source separation processing unit 10. For example, if the target sound is a human voice, it may be digitized with a sampling period of about 8 kHz.
Here, the sound source separation processing unit 10 (10-1 to 10-3) is, for example, a sound source separation process by a blind sound source separation method based on an independent component analysis method shown in Non-Patent Document 2 or Non-Patent Document 3, or The sound source separation process such as the binary masking process shown in Non-Patent Document 4 and Non-Patent Document 5 is executed.

Hereinafter, a sound source separation device Z that is an example of a device that can be employed as the sound source separation processing unit 10 will be described with reference to the block diagram shown in FIG.
The sound source separation device Z shown below is a state in which a plurality of sound sources and a plurality of microphones 101 and 102 exist in a predetermined acoustic space, and through the microphones 101 and 102, individual audio signals (hereinafter referred to as sound sources). When a plurality of mixed sound signals, which are signals superimposed on each other, are sequentially input, the mixed sound signals are subjected to BSS sound source separation processing based on the ICA method, thereby supporting the sound source signals. A process for sequentially generating a plurality of separated signals (signals identifying sound source signals) is performed.
The sound source separation device Z shown in FIG. 14 performs sound source separation processing based on the FDICA method (Frequency-Domain ICA) which is a kind of ICA-BSS method.

この（１）式からわかるように，分離演算処理（フィルタ処理）は，周波数ビンごとに行われる。
ここで，分離フィルタＷ(ｆ)の更新式は，例えば次の（２）式のように表すことができる。

このＦＤＩＣＡ方式によれば，音源分離処理が各狭帯域における瞬時混合問題として取り扱われ，比較的簡単かつ安定に分離フィルタ（分離行列）Ｗ(ｆ)を更新することができる。
図１４において，主マイクロホン１０１に対応する分離信号ｙ1(ｆ)が前記目的音分離信号である。また，副マイクロホン１０２に対応する分離信号ｙ2(ｆ)が前記参照音分離信号である。
なお，図１４においては，入力される混合音声信号ｘ1，ｘ2のチャンネル数（即ち，マイクロホンの数）が２つである例について示しているが，（チャンネル数ｎ）≧（音源の数ｍ）であれば，３チャンネル以上であっても同様の構成により実現できる。 In the FDICA method, first, a short time discrete Fourier transform (Short Time Discrete Fourier Transform) is performed for each frame, which is a signal divided by the ST-DFT processing unit 13 for each predetermined period, with respect to the input mixed audio signal x (t). , Hereinafter referred to as ST-DFT processing), and a short time analysis of the observation signal is performed. Then, the signal of each channel (signal of each frequency component) after the ST-DFT processing is subjected to separation calculation processing (filter processing) based on the separation matrix W (f) by the separation calculation processing unit 11f, thereby separating sound sources ( Sound source signal identification). Here, if f is a frequency bin and m is an analysis frame number, the separated signal (identification signal) y (f, m) can be expressed as the following equation (1).

As can be seen from the equation (1), the separation calculation process (filter process) is performed for each frequency bin.
Here, the update formula of the separation filter W (f) can be expressed as the following formula (2), for example.

According to this FDICA method, sound source separation processing is handled as an instantaneous mixing problem in each narrow band, and the separation filter (separation matrix) W (f) can be updated relatively easily and stably.
In FIG. 14, a separation signal y1 (f) corresponding to the main microphone 101 is the target sound separation signal. Further, the separation signal y2 (f) corresponding to the sub microphone 102 is the reference sound separation signal.
FIG. 14 shows an example in which the number of channels (that is, the number of microphones) of the input mixed audio signals x1 and x2 is two, but (number of channels n) ≧ (number of sound sources m). If so, it can be realized with the same configuration even if there are three or more channels.

ここで，目的音信号と雑音信号との間に相関がないものと仮定し，さらに，雑音信号のスペクトル値Ｎ(ｆ，ｍ)を前記参照音信号のスペクトル値で近似できるとすると，前記スペクトル減算処理部３１は，目的音信号のスペクトル推定値（即ち，前記目的音抽出信号のスペクトル値）を，次の（４）式に基づき算出できる。

Further, in the target sound extraction device X1, the target sound separation signal synthesis processing unit 20 executes a synthesis process of the plurality of target sound separation signals separated and generated by each of the sound source separation processing units 10 and is obtained thereby. A synthesized signal is output (an example of the target sound separation signal synthesizing means).
For example, the target sound separation signal synthesis processing unit 20 performs an average process or a weighted average process for each of the plurality of target sound separation signals for each of the frequency components (frequency bins) divided into a plurality of the target sound separation signals. Synthesize a sound separation signal.
Further, in the target sound extraction device X1, the spectrum subtraction processing unit 31 includes a plurality of the reference signals separated and generated by the synthesized signal obtained by the target sound separation signal synthesis processing unit 20 and the sound source separation processing unit 10, respectively. By performing spectral subtraction processing with the sound separation signal, an acoustic signal corresponding to the target sound is extracted from the synthesized signal, and the extracted signal (the target sound extraction signal) is output (the spectrum An example of subtraction processing means).
The spectrum subtraction processing unit 31 removes the signal component of each of the reference sound separation signals from the synthesized signal by a known spectrum subtraction process (target sound extraction process based on a spectrum difference method) to obtain the target sound extraction signal. The process to extract is performed.
In the spectrum subtraction process, the spectrum subtraction processing unit 31 performs a discrete Fourier transform process (DFT) for each frame of a predetermined time length for each of the synthesized signal and the reference sound separation signal, and an observation signal (here, , The synthesized signal) is analyzed for a short time. Here, the frequency bin is f, the analysis frame number is m, the spectrum value of the synthesized signal (signal value after DFT) is Y (f, m), and the spectrum value of the target sound signal is S (f , M) and the spectrum value of the noise signal (sound signal other than the target sound) is N (f, m), the spectrum value Y (f, m) of the synthesized signal is expressed by the following equation (3): Is represented by

Here, assuming that there is no correlation between the target sound signal and the noise signal, and further assuming that the spectrum value N (f, m) of the noise signal can be approximated by the spectrum value of the reference sound signal, the spectrum The subtraction processing unit 31 can calculate the spectrum estimation value of the target sound signal (that is, the spectrum value of the target sound extraction signal) based on the following equation (4).

Next, the process of target sound extraction processing in the target sound extraction device X1 will be described with reference to FIG. For simplification of explanation, FIG. 2 shows an example in which there are two sub-acoustic signals (that is, there are two sub-microphones 102).
The plurality of target sound separation signals separated and generated by the sound source separation processing unit 10 are signals mainly including a signal component of the target sound. Similarly, the plurality of reference sound separation signals (Y _B1 and Y _B2 in FIG. 2) separated and generated by the sound source separation processing unit 10 are collected by the sub microphones 102 having different positions and directivity directions. It is a signal mainly including signal components (components indicated by bar graphs other than the shaded bar graph in FIG. 2) of noise sound source (reference sound) in the range.
However, depending on the position of the target sound source and the occurrence of noise, a relatively large amount of reference sound signal components other than the target sound may remain in the target sound separation signal. Therefore, the synthesized signal obtained by synthesizing them (Y _C in FIG. 2) is basically a signal mainly including the signal component of the target sound (the component indicated by the hatched bar graph in FIG. 2). In some cases, a relatively large amount of noise signal components may remain.
On the other hand, even if the target sound separation signal includes a noise sound (reference sound) component other than the target sound, the spectrum subtraction processing unit 31 converts the signal component of the target sound from the synthesized signal. The target sound extraction signal (Y _O in FIG. 2), which is the result of extraction, is a signal from which the signal component of the reference sound separation signal has been removed. Moreover, the target sound extraction signal removes all signal components of the reference sound separation signal corresponding to each of the plurality of noises even in a situation where different noises (reference sounds) arrive at the main microphone 101 from a plurality of directions. Signal.
Therefore, according to the target sound extraction device Y1, high noise removal performance can be obtained even in a situation where a relatively strong specific noise arrives at the main microphone 101 or a situation where different noises arrive at the main microphone 101 from a plurality of directions. Can be secured.
Further, only the spectral subtraction process, which is a non-linear process, tends to generate musical noise peculiar to the non-linear process in its output signal (target sound extraction signal). However, in the target sound extraction apparatus X1, the sound source separation processing unit 10 Since the spectral subtraction process is performed based on the signal after the linear filter process is performed, it is possible to prevent an unpleasant musical noise from being included in the target sound extraction signal. In particular, when the number of sound sources including the target sound and noise is a small number (about three or less), the sound source separation processing contributes to the target sound extraction particularly effectively, and the effect of suppressing musical noise is enhanced.

[Second invention]
Next, the target sound extraction device X2 according to the embodiment of the second invention will be described with reference to the block diagram shown in FIG. In FIG. 3, among the constituent elements included in the target sound extraction device X2, the same reference numerals as those in FIG. 1 are assigned to the constituent elements that perform the same processing as that included in the target sound extraction device X1.
As shown in FIG. 3, the target sound extraction device X2 includes an acoustic input device V1 including a plurality of microphones, a plurality (three in FIG. 3) of sound source separation processing units 10 (10-1 to 10-3), and a spectrum approximation. A signal extraction processing unit 32 is provided. Here, the sound input device V1 is the same as the sound input device V1 in the target sound extraction device X1.
The target sound extraction device X2 also obtains an acoustic signal corresponding to the target sound based on the main acoustic signal obtained through the main microphone 101 and the sub acoustic signals obtained through the other plurality of sub microphones 102. The extracted signal (the target sound extraction signal) is output.
In the target sound extraction apparatus X2, the sound source separation processing unit 10 and the spectrum approximate signal extraction processing unit 32 are implemented by, for example, a DSP that is an example of a computer and a ROM that stores a program executed by the DSP, an ASIC, or the like. It becomes. In this case, the ROM stores in advance a program for causing the DSP to execute processing (described later) performed by the sound source separation processing unit 10 and the spectrum approximate signal extraction processing unit 32.

The sound source separation processing unit 10 (10-1 to 10-3) is provided for each combination of the main sound signal and each of the plurality of sub sound signals, and based on the main sound signal and the sub sound signal. , A sound source separation process for separating and generating a target sound separation signal which is a separation signal (identification signal) corresponding to the target sound is executed.
Note that an A / D converter (not shown) is provided between the microphones 101 and 102 and the sound source separation processing unit 10 as in the target sound extraction device X1.
Here, the sound source separation processing unit 10 (10-1 to 10-3) is, for example, the independent component analysis method shown in Non-Patent Document 2 or Non-Patent Document 3, as in the case of the target sound extraction device X1. The sound source separation process based on the blind sound source separation method based on the above, or the sound source separation process such as the binary masking process shown in Non-Patent Document 4 or Non-Patent Document 5 is executed.

In addition, the spectrum approximate signal extraction processing unit 32 includes a plurality of target sound separation signals separated and generated by the sound source separation processing unit 10 among signal components for each of frequency bands (frequency bins) divided into a plurality of parts. By extracting the signal component that satisfies a predetermined approximation condition between the target sound separation signals, an acoustic signal corresponding to the target sound is extracted from the plurality of target sound separation signals, and the extracted signal (the Target sound extraction signal).
For example, the spectrum approximation signal extraction processing unit 32 compares the levels (power) of the signal components for each frequency bin for the plurality of target sound separation signals, and the ratio or difference between the levels is predetermined. If the approximate condition of being within is satisfied, the target sound extraction is performed by selecting any one of those signal components or synthesizing those signal components (for example, calculating an average value or a minimum value). Extract the signal.

Next, the process of target sound extraction processing in the target sound extraction device X2 will be described with reference to FIG. For simplification of explanation, FIG. 4 shows an example in which there are two sub-acoustic signals (that is, there are two sub-microphones 102).
A plurality of the target sound separation signals (Y _A1 and Y _A2 in FIG. 4) separated and generated by the sound source separation processing unit 10 each have a signal component of the target sound (components indicated by hatched bar graphs in FIG. 4). This signal is mainly included.
However, depending on the position of the target sound source and the state of noise generation, the target sound separation signal may include a signal component of a reference sound other than the target sound (a component indicated by a bar graph other than the shaded bar graph in FIG. 4). There may be many remaining.
On the other hand, even if the target sound separation signal includes a noise sound (reference sound) component other than the target sound, the position or directivity direction of each of the plurality of microphones 101 and 102 is different. The target sound separation signal containing many components is usually a part of all of them, or the types of noise components included in the target sound separation signals are usually different.
Accordingly, the target sound extraction signal (Y in FIG. 4) is obtained as a result of extracting approximate signal components in the plurality of target sound separation signals (Y _A1 and Y _A2 in FIG. 4) by the spectrum approximate signal extraction processing unit 32. _O ) is a signal from which various noise signal components have been removed.
Therefore, according to the target sound extraction device Y2, high noise removal performance can be obtained even in a situation where a relatively strong specific noise arrives at the main microphone 101 or a situation where different noises arrive at the main microphone 101 from a plurality of directions. Can be secured.

[Third invention]
Next, the target sound extraction device X3 according to the embodiment of the third invention will be described with reference to the block diagram shown in FIG. In FIG. 5, among the constituent elements included in the target sound extracting device X3, constituent elements that execute the same processes as those included in the target sound extracting device X1 are denoted by the same reference numerals as those in FIG.
As shown in FIG. 5, the target sound extraction device X3 includes an acoustic input device V1 including a plurality of microphones, a plurality (three in FIG. 3) of sound source separation processing units 10 (10-1 to 10-3), and a spectral subtraction. A processing unit 31 ′ is provided. Here, the sound input device V1 is the same as the sound input device V1 in the target sound extraction device X1.
The target sound extraction device X3 also generates an acoustic signal corresponding to the target sound based on the main acoustic signal obtained through the main microphone 101 and the sub acoustic signals obtained through the other plurality of sub microphones 102. The extracted signal (the target sound extraction signal) is output.
In the target sound extraction device X3, the sound source separation processing unit 10 and the spectral subtraction processing unit 31 ′ are realized by a DSP that is an example of a computer and a ROM that stores a program executed by the DSP, an ASIC, or the like. Is done. In this case, the ROM stores in advance a program for causing the DSP to execute processing (described later) performed by the sound source separation processing unit 10 and the spectrum subtraction processing unit 31 ′.

The sound source separation processing unit 10 (10-1 to 10-3) is provided for each combination of the main sound signal and each of the plurality of sub sound signals, and based on the main sound signal and the sub sound signal. The sound source separation process for separating and generating a reference sound separation signal that is a separation signal (identification signal) corresponding to noise (reference sound) other than the target sound is executed.
Note that an A / D converter (not shown) is provided between the microphones 101 and 102 and the sound source separation processing unit 10 as in the target sound extraction device X1.
Here, the sound source separation processing unit 10 (10-1 to 10-3) is, for example, the independent component analysis method shown in Non-Patent Document 2 or Non-Patent Document 3, as in the case of the target sound extraction device X1. The sound source separation process based on the blind sound source separation method based on the above, or the sound source separation process such as the binary masking process shown in Non-Patent Document 4 or Non-Patent Document 5 is executed.

  Further, the spectral subtraction processing unit 31 ′ performs the spectral subtraction described above between the main acoustic signal obtained through the main microphone 101 and the plurality of reference sound separation signals separated and generated by the sound source separation processing unit 10. By performing processing, an acoustic signal corresponding to the target sound is extracted from the main acoustic signal, and the extracted signal (the target sound extraction signal) is output. The spectrum subtraction processing unit 31 ′ performs the same processing as the spectrum subtraction processing unit 31 in the target sound extraction device X1 except that the processing target (observation signal) is switched from the synthesized signal to the main acoustic signal. It is.

Next, the process of target sound extraction processing in the target sound extraction device X1 will be described with reference to FIG. For simplification of explanation, FIG. 6 shows an example in which there are two sub-acoustic signals (that is, there are two sub-microphones 102).
A plurality of the reference sound separation signals (Y _B1 and Y _B2 in FIG. 6) separated and generated by the sound source separation processing unit 10 are noise sound sources in the sound collection ranges of the sub microphones 102 having different positions and directivity directions. Is a signal mainly including a signal component (a component indicated by a bar graph other than the shaded bar graph in FIG. 6).
On the other hand, there may be a case where a relatively large amount of the reference sound signal component other than the target sound remains in the main sound signal. As described above, even if the main sound signal includes a noise sound (reference sound) component other than the target sound, the spectrum subtraction processing unit 31 ′ extracts the signal component of the target sound from the main sound signal. The target sound extraction signal (Y _O in FIG. 6), which is the result of the above, is a signal from which the signal component of the reference sound separation signal has been removed. Moreover, the target sound extraction signal removes all signal components of the reference sound separation signal corresponding to each of the plurality of noises even in a situation where different noises (reference sounds) arrive at the main microphone 101 from a plurality of directions. Signal.
Therefore, according to the target sound extraction device Y3, high noise removal performance can be obtained even in a situation where a relatively strong specific noise arrives at the main microphone 101 or a situation where different noises arrive at the main microphone 101 from a plurality of directions. Can be secured.
Further, only the spectral subtraction process, which is a non-linear process, tends to generate musical noise peculiar to the non-linear process in its output signal (target sound extraction signal). However, in the target sound extraction apparatus X3, the sound source separation processing unit 10 Since the spectral subtraction process is performed based on the signal after the linear filter process is performed, it is possible to prevent an unpleasant musical noise from being included in the target sound extraction signal. In particular, when the number of sound sources including the target sound and noise is a small number (about three or less), the sound source separation process contributes to noise extraction particularly effectively, and the effect of suppressing musical noise is enhanced.
The reference sound separation signal, the target sound separation signal and their combined signal, which are the processing results of the sound source separation processing unit 10 that executes the sound source separation processing of the FDICA method, and the spectrum subtraction process and the spectrum approximation signal The target extraction signals obtained by the extraction process are all acoustic signals in the frequency domain. Therefore, although not shown in FIGS. 1, 3, and 5, the target sound extraction devices Y1, Y2, and Y3 further include an IDFT processing unit and a sound output processing unit.
The IDFT processing unit executes a process of converting the target sound extraction signal in the frequency domain into a signal in the time domain, that is, a process of performing an inverse discrete Fourier transform (IDFT) process and outputting it to a predetermined buffer memory.
The sound output processing unit sequentially outputs the target sound extraction signal in the time domain obtained by the IDFT processing unit to the outside (for example, outputs in real time).

[Evaluation of target sound extraction performance]
Hereinafter, the evaluation results of the target sound extraction performance of each of the target sound extraction devices X1 to X3 described above will be described with reference to FIGS.
7 and 8 show the first experimental condition and the second experimental condition for evaluating the target sound extraction performance of the target sound extraction devices X1 to X3.
The first experimental condition is that the target sound source exists in the front direction of the main microphone 101 having directivity, and other noise sound sources (reference sound sources) exist in the front direction of the sub microphones 102 having directivity. The condition is relatively close to the state.
The second experimental condition is that the target sound source exists in the front direction of the main microphone 101 having directivity, while other noise sound sources (reference sound sources) do not necessarily correspond to the sub microphones 102, respectively. The conditions are relatively close to the actual usage environment.
The target sound extraction performance of the target sound extraction devices X1 to X3 and the conventional target sound extraction device under each of the first experimental condition and the second experimental condition is expressed as NRR (Noise Reduction) in the target sound extraction signal. FIG. 9 and FIG. 10 show those expressed by (Rate). 9 and 10, the target sound extraction devices X1 to X3 are referred to as devices X1 to X3, and the conventional target sound extraction device is referred to as a conventional device. The conventional target sound extraction device here extracts a signal component corresponding to the target sound from the main sound signal by the spectral subtraction process based on the sub-acoustic signal.
As can be seen from FIGS. 9 and 10, regardless of the experimental conditions, any of the target sound extraction devices X1 to X3 can obtain extremely high target sound extraction performance as compared with the conventional device.
Among the target sound extraction devices X1 to X3, in particular, the target sound extraction performance by the target sound extraction device X1 is high, followed by the target sound extraction device X3 and the target sound extraction device X2. It can be seen that high target sound extraction performance can be obtained.
As described above, according to the target sound extraction devices X1 to X3, higher target sound extraction performance (noise removal performance) than the conventional one can be ensured under various acoustic environments.

[Evaluation of directivity]
The directivity evaluation results of the target sound extraction device X1 will be described below with reference to FIGS.
FIG. 11 shows a third experimental condition for evaluating the directivity of the target sound extraction device X1. This third experimental condition is an experimental condition for evaluating to what extent the target sound can be extracted by moving the target sound source with reference to the front direction of the main microphone 101.
The directional characteristic of the target sound extraction device X1 and the directional main microphone 101 itself under the third experimental condition, that is, the microphone sensitivity (unit dB) with respect to the sound source from all 360 degrees. Is FIG.

As can be seen from FIG. 12, although the directivity of the main microphone 101 itself is very gradual, the target sound extraction device X1 is very narrow with the front direction of the main microphone 101 as the center. While a high NRR is obtained in the range, the NRR rapidly decreases when the target sound source is out of the range.
Thus, even if the directivity of the main microphone 101 itself is very gentle, the target sound extracting device X1 functions as an acoustic input device having a very steep directivity.

Further, in the result shown in FIG. 12, directions of + 45 ° and −45 ° with the front direction (center direction of the directivity range) of the main microphone 101 as the center (0 ° direction) form the boundary of the directivity range. It has become a direction.
On the other hand, in the third experimental condition, the main microphone 101 and the sub microphone 102, which are symmetric with respect to each other and have substantially the same directivity characteristics, are divided into two sub-directories with respect to the central direction (0 °) of the main microphone 101. The directivity center directions of the microphones 102 are set to + 90 ° and −90 °, respectively. From this, in the target sound extraction devices X1 to X3, when the main microphone 101 and the sub microphone 102 are symmetrical and have substantially the same directivity characteristics, the direction in which the boundary of the directivity range is formed is the main sound extraction device X1 to X3. It can be seen that the directional center direction of the microphone 101 is an intermediate direction between the directional center directions of the sub microphones 102.
Further, the example shown in FIG. 12 is an example in which the directivity directions of the microphones 101 and 102 are set in different directions in the same plane, but when they are set in three-dimensionally different directions, The boundary of the directivity range can be set in a desired direction three-dimensionally.
For example, in a certain plane, the front direction of the main microphone 101 and the front directions of the two sub microphones 102-1 and 102-2 are directed in the directions of 0 ° and ± 90 °, and the other sub microphone is directed. For example, the front direction of the microphone 102-3 may be oriented in a direction perpendicular to the one plane. Thereby, the directivity characteristic of the target sound extraction device X1 can be set to a desired characteristic three-dimensionally.
Therefore, the target sound extraction device X1 has a switch, a dial, etc. for adjusting the position or directivity direction of the sub microphone 102 with respect to the position or directivity direction of the main microphone 101 (approaching or moving away). By providing the operation unit, the directivity of the target sound extraction device X1 can be easily adjusted, which is highly convenient.
Further, the target sound extraction devices X2 and X3 have the directivity performance of the target sound extraction device X1 described above.

By the way, as an acoustic input device that realizes a sharp directional characteristic, for example, an acoustic input device including a microphone array and a delay sum filter is known. However, in such a conventional acoustic input device, in order to realize the sharp directivity as shown in FIG. 12, the number of microphones constituting the microphone array is increased, and the microphones are arranged over several meters. The equipment becomes so large that it cannot be easily transported by people.
On the other hand, the target sound extraction devices X1 to X3 are small devices including about 3 to 5 microphones arranged at intervals of several centimeters and a very small processor such as a DSP or ASIC for performing signal processing. A sharp directivity as shown in FIG. 12 can be realized by (an apparatus having a size of a general handy microphone).

Next, an acoustic input device V2 that is an example of a device that can be used in place of the acoustic input device V1 in the target sound extraction devices X1 to X3 will be described with reference to a block diagram shown in FIG.
In the sound input device V1, the main microphone 101 for obtaining the main sound signal and the plurality of sub microphones 102 for obtaining the sub sound signals are determined in advance. A plurality of microphones are provided, and which one of them functions as the main microphone 101 and the sub microphone 102 is switched according to the situation.
As shown in FIG. 13, the acoustic input device V2 includes three or more (four in FIG. 13) microphones 100-1 to 100-4, a main / sub acoustic signal specifying unit 41, a signal switch 42, It has.
The three or more microphones 100-1 to 100-4 are microphones having different arrangement positions or different directivity directions. These microphones 100-1 to 100-4 function as the main microphone 101 or the sub microphone 102 depending on the situation.
For example, each of the microphones 100-1 to 100-4 is a microphone having the same directivity, and as shown in FIG. 13, on the predetermined circumference (center PO) toward the outside in the radiation direction in the circle, etc. They are arranged at intervals (so that the central angles when the microphone position and the center PO of the circle are connected are equal).

The main / sub-acoustic signal specifying unit 41 selects one of the input acoustic signals based on three or more input acoustic signals obtained through the three or more microphones 100-1 to 100-4. A process of specifying the main sound signal and the plurality of sub sound signals is executed (an example of the main / sub sound signal specifying means). Further, the main / sub-acoustic signal specifying unit 41 outputs a control signal corresponding to the specification result of the main and sub-acoustic signals to the signal switch 42.
The main / sub-acoustic signal specifying unit 41 compares, for example, the signal strengths (sound pressures) of three or more input acoustic signals, and specifies the input acoustic signal having the maximum signal strength as the main acoustic signal. Then, all or some (two or more) of the other input sound signals are specified as the sub sound signals. As a method of specifying a part of the other input sound signals as the sub-acoustic signal, for example, through the two microphones whose arrangement positions or directing directions are adjacent to both sides with respect to the microphone that obtains the main sound signal. It is conceivable to specify the obtained acoustic signal as the sub acoustic signal.
Further, the main / sub-acoustic signal specifying unit 41 compares the proportions of predetermined frequency components in each of the three or more input acoustic signals, and specifies the one having the largest proportion as the main acoustic signal. However, it may be possible to specify all or part (two or more) of the other input acoustic signals as the sub-acoustic signal. This is effective when the frequency characteristics of the sound emitted from the target sound source are known to some extent.
The main / sub-acoustic signal specifying unit 41 is embodied by, for example, a DSP that is an example of a computer and a ROM that stores a program executed by the DSP, or an ASIC. In this case, the ROM stores in advance a program for causing the DSP to execute processing (to be described later) performed by the main / sub-acoustic signal specifying unit 41.

Further, the signal switching unit 42 is connected to the three or more microphones 100-1 to 100-4 in accordance with a control signal (a signal corresponding to a signal identification result) output from the main / sub-acoustic signal identifying unit 41. It is a device that switches a transmission path of an acoustic signal to the sound source separation processing unit 10 (an example of the signal path switching unit).
The signal switch 42 includes signal input terminals In1 to In4 connected to the microphones 100-1 to 100-4, one signal output terminal Ot1 for outputting the main acoustic signal, and the sub acoustic signal. A plurality of (three in FIG. 13) signal output terminals Ot2 to Ot4 for output are provided. Further, the signal switch 42 has a signal path for connecting the signal input terminals In1 to In4 and the signal output terminals Ot1 to Ot4 in accordance with the control signal output from the main / sub acoustic signal specifying unit 41. , Selectively switch from a plurality of predetermined switching patterns. Thereby, the acoustic signal specified as the main acoustic signal by the main / sub-acoustic signal specifying unit 41 is output from the output end Ot1, and specified as the sub-acoustic signal by the main / sub-acoustic signal specifying unit 41. An acoustic signal is output from the output terminals Ot2 to Ot4.
Since the target sound extraction devices X1 to X3 include a sound input device V2 as shown in FIG. 13, the position of the target sound source can be changed, so that a predetermined one of a plurality of microphones is selected as the main sound source. It can also be applied to a target that cannot be fixed as the microphone 101.

Next, with reference to the time charts shown in FIGS. 15 to 18, a description will be given of a sequence of the sound source separation processing when the sound source separation processing unit 10 performs sound source separation processing based on the FDICA method. As described above, the sound source separation process based on the FDICA method is an example of the sound source separation process based on the blind sound source separation method based on the independent component analysis method. In the following description, the processing of the target sound separation signal synthesis processing unit 20 and the spectral subtraction processing unit 31 in the target sound extraction device X1 and the processing of the spectrum approximate signal extraction processing unit 32 in the target sound extraction device X2 will be described. The processing and the processing of the spectrum subtraction processing unit 31 ′ in the target sound extraction device X3 are collectively referred to as post processing.
In the sound source separation processing based on the FDICA method, sound signals (hereinafter referred to as input sound signals) input in time series through a plurality of microphones (the main microphone 101 and the sub microphone 102 in the target sound extraction devices X1 to X3). On the other hand, after converting this into a frequency domain signal, filter processing (matrix operation) based on the separation matrix W (f) is sequentially performed to generate a separation signal (the reference sound separation signal and the target sound separation signal). Is executed. Here, the input sound signal corresponds to the mixed sound signals x1 (t) and x2 (t) in FIG. 14, and corresponds to the main sound signal and the sub sound signal in FIGS. .
Further, as described above, the filtering process is performed for each frame signal of a predetermined time length (for example, a signal obtained by dividing the mixed audio signal with a period of about several tens of ms to several hundreds of ms). This filter process is a process with a small calculation load, and even if it is executed together with the post process by a practical processor, it is possible to realize a real-time process with a relatively large margin.
Further, as described above, in the sound source separation processing based on the FDICA method, learning calculation (sequential calculation) for obtaining the separation matrix W (f) used for the filter processing is performed using the input acoustic signals input in time series. ) Is also performed. This learning calculation is computationally intensive and is generally not suitable for real-time processing.

FIG. 15 is a time chart showing a first example of a processing sequence excluding the learning calculation in the target sound extraction devices X1 to X3. St1, St2,... Shown below represent identification codes of processing procedures (steps).
As shown in FIG. 15, in the target sound extraction devices X1 to X3, the sound source separation processing unit 10 performs frame signal {Frame (i-1), Frame (i) for a predetermined time length for the input acoustic signal. , Frame (i + 1)...}, A discrete Fourier transform (DFT) process (St1) is performed, and a frequency domain frame signal as a result of the process is temporarily stored in the memory. In the first example, the sound source separation processing unit 10 executes the discrete Fourier transform process (St1) at the same cycle as the time length of the frame signal. As a result, two consecutive frame signals become signals without overlapping time zones.
Further, the sound source separation processing unit 10 sequentially performs filter processing (St2: matrix operation) based on the separation matrix W (f) for each frame signal in the frequency domain obtained by the DFT processing to generate a separation signal.
Next, another processing unit (the target sound separation signal synthesis processing unit 20 and the spectral subtraction processing unit 31, or the spectral approximate signal extraction processing unit 32, or the spectral subtraction processing unit 31 ′) performs the filtering process ( The post processing (St3) is executed based on the separation signal obtained in St2). Thus, the target sound extraction signal in the frequency domain corresponding to each of the frame signals in the input acoustic signal is obtained.
Further, the IDFT processing unit (not shown) executes an inverse discrete Fourier transform (IDFT) process (St4) to convert the target sound extraction signal in the frequency domain into a time domain signal, and the sound output processing unit The target sound extraction signal (output acoustic signal) in the time domain is sequentially output to the outside (St5).
The processing of steps St1 to St4 described above is processing with a small calculation load, and even if executed by a practical processor, the processing can be completed within a range of the time length of the frame signal with a comparative margin. Therefore, although the output sound signal has a slight delay time td (several tens of ms to less than several hundred ms) with respect to the input sound signal, the sound output in real time according to the input sound signal is input. Signal.

FIG. 16 is a time chart showing a second example of a processing sequence excluding the learning calculation in the target sound extraction devices X1 to X3.
Also in the example shown in FIG. 16, the sound source separation processing unit 10 performs discrete Fourier transform on the input acoustic signal for each frame signal {Frame (i−1), Frame (i), Frame (i + 1). (DFT) processing (St1) is performed, and the frame signal in the frequency domain, which is the processing result, is temporarily stored in the memory. However, in the second example, the sound source separation processing unit 10 executes the discrete Fourier transform process (St1) at a cycle shorter than the time length of the frame signal. As a result, two consecutive frame signals become signals with some overlapping time zones.
Further, the sound source separation processing unit 10 sequentially performs filter processing (St2: matrix operation) based on the separation matrix W (f) for each frame signal in the frequency domain obtained by the DFT processing to generate a separation signal. At this time, the separated signals for two consecutive frames generated by the sound source separation processing unit 10 are also signals having overlapping portions of time zones (time zones within a wavy circle in FIG. 16). For this reason, the sound source separation processing unit 10 generates a separated signal to be output by performing a synthesis process (weighted average process or the like) on overlapping time zone portions in the separated signals for two consecutive frames.
Next, as in the first example (FIG. 15), another processing unit executes the post processing (St3) based on the separated signal obtained by the filter processing (St2).
Further, as in the first example (FIG. 15), the IDFT processing unit (not shown) executes an inverse discrete Fourier transform (IDFT) process (St4) to convert the target sound extraction signal in the frequency domain into the time domain. The sound output processing unit sequentially outputs the target sound extraction signal (output sound signal) in the time domain to the outside (St5).
Also in the processing of the second example described above, the output acoustic signal has a slight delay time td (several tens of ms to less than several hundred ms) with respect to the input acoustic signal, but the input acoustic signal is input. The acoustic signal is output in real time according to

On the other hand, the learning calculation in the sound source separation processing based on the FDICA method is performed every time a plurality of continuous frame signals are input, and a new separation matrix W (f) is obtained by sequential calculation using the plurality of frame signals. This is a process for calculating (separation matrix used in the subsequent filter process), and is executed in parallel with each process (St1 to St5) shown in FIG. The newly calculated separation matrix W (f) is used for the filter processing to be executed later.
Hereinafter, a set of a predetermined number (several) consecutive frame signals used every time a new separation matrix W (f) is calculated in the learning calculation is hereinafter referred to as a metaframe signal. This metaframe signal is a signal (corresponding to the section signal) divided in a predetermined cycle in the input acoustic signal input in time series, and is directly converted to a frequency domain signal. A metaframe signal (which has been subjected to inverse discrete Fourier transform processing) is used for the learning calculation. Whereas the time length of the frame signal (period of signal division) is several tens of milliseconds to several hundred milliseconds, the time length of the metaframe signal (period of signal division) is the ability of the processor to execute processing. Although it depends, it is the time (for example, about several seconds) allowed as the adaptation time to the change of the acoustic environment.

FIG. 17 is a time chart of the first embodiment of the learning calculation executed by the sound source separation processing unit 10 that performs sound source separation processing based on the FDICA method.
The example (first embodiment) of the learning calculation (sequential calculation) shown in FIG. 17 is that each metaframe signal {Mframe (1), Mframe (2), Mframe (3),. This is an example in which the separation matrix W (f) used for the subsequent filter processing is obtained using all of them. However, in this case, the number of sequential calculations in the learning calculation is limited to be equal to or less than a predetermined upper limit number (so that the sequential calculation is completed when the upper limit number is reached).
In the learning calculation of the first embodiment shown in FIG. 17, the metaframe signal Mframe (i) corresponding to the input acoustic signal input during the period of time Ti to Ti + 1 (period: Ti + 1−Ti). All are used to calculate (learn) the separation matrix W (f). Then, the separation matrix W (f) used in the subsequent filtering process is updated to a new separation matrix W (f) obtained by the learning calculation. At this time, the separation matrix W (f) calculated (learned) using a certain metaframe signal Mframe (i) is used as the separation matrix W (f) using the next metaframe signal Mframe (i + 1). Is used as the initial value (initial separation matrix) when calculating (sequential calculation) (inheritance of the initial matrix), it is preferable that convergence of the sequential calculation (learning) is accelerated.
Here, when the learning calculation with a high calculation load is executed without particular limitation, the learning calculation time ts for each metaframe signal becomes larger than the time length (Ti + 1−Ti) of the metaframe signal, A situation occurs in which it is difficult to quickly adapt to changes in the acoustic environment.
Therefore, the number of sequential calculations in the learning calculation is limited by the upper limit number so that the learning calculation time ts for each metaframe signal is always shorter than the time length (Ti + 1−Ti) of the metaframe signal. This will enable quick adaptation to changes in the acoustic environment.
In addition, due to the limitation of the number of times of sequential calculation (simplification of learning calculation), even if some noise is included in the separated signal obtained by the sound source separation process, the sound source separation process and the post process ( By combining with the spectral subtraction processing and spectral approximate signal extraction processing), the target sound extraction performance can be sufficiently secured as a whole.
In the first filter processing at the start of the processing of the target sound extraction devices X1 to X3 (when the device is turned on), for example, an initial matrix prepared in advance or at the end of the previous processing (the power of the device) It is conceivable to use a separation matrix or the like stored in the memory at the time of OFF) as the separation matrix.
In addition, the upper limit number of times is determined in advance by experiments or calculations in accordance with the ability of a processor (DSP, ASIC, etc.) that executes this processing.

FIG. 18 is a time chart of the second embodiment of the learning calculation executed by the sound source separation processing unit 10 that performs sound source separation processing based on the FDICA method.
The learning calculation (sequential calculation) example (second embodiment) shown in FIG. 18 is a part of the top side of the metaframe signal {Mframe (1), Mframe (2), Mframe (3),. This is an example of obtaining the separation matrix W (f) used for the subsequent filter processing by using a part of the time zone signals for each time zone signal.
In the learning calculation of the second embodiment shown in FIG. 17, the metaframe signal Mframe (i) corresponding to the input acoustic signal input during the period of time Ti to Ti + 1 (period: Ti + 1−Ti). The separation matrix W (f) is calculated (learned) using a part of the head side. Then, the separation matrix W (f) used in the subsequent filtering process is updated to a new separation matrix W (f) obtained by the learning calculation. Also at this time, a separation matrix W (f) calculated (learned) using a part of a certain metaframe signal Mframe (i) is used as a part of the next metaframe signal Mframe (i + 1). If the separation matrix W (f) is used as an initial value (initial separation matrix) when calculating (sequential calculation) (inheritance of the initial matrix), the convergence of the sequential calculation (learning) is accelerated.
In the second embodiment, a part of the metaframe signal is set so that the learning calculation time ts for each metaframe signal is always shorter than the time length (Ti + 1−Ti) of the metaframe signal. By thinning and using for the learning calculation, it is possible to quickly adapt to changes in the acoustic environment.
In addition, even if some noise is included in the separated signal obtained by the sound source separation processing due to the thinning of the signals used for the learning calculation (simplification of the learning calculation), the sound source separation processing and the post processing are performed. By combining with processing (spectral subtraction processing and spectral approximate signal extraction processing), it is possible to ensure sufficient target sound extraction performance as a whole.
Note that the time length (number of digital signal samples) used for the learning calculation in the metaframe signal is determined in advance by experiment or calculation according to the ability of the processor (DSP, ASIC, etc.) that executes this processing. .

  The present invention is applicable to a target sound extraction apparatus that extracts and outputs an acoustic signal corresponding to a target sound from an acoustic signal including the target sound component and a noise component.

The block diagram showing the schematic structure of the target sound extraction device X1 which concerns on embodiment of 1st invention. The conceptual diagram showing the process of the target sound extraction process in the target sound extraction apparatus X1. The block diagram showing schematic structure of the target sound extraction device X2 which concerns on embodiment of 2nd invention. The conceptual diagram showing the process of the target sound extraction process in the target sound extraction apparatus X2. The block diagram showing schematic structure of the target sound extraction device X3 which concerns on embodiment of 3rd invention. The conceptual diagram showing the process of the target sound extraction process in the target sound extraction apparatus X3. The figure showing the 1st experimental condition which evaluates the target sound extraction performance of the target sound extraction apparatuses X1-X3. The figure showing the 2nd experimental condition which evaluates the target sound extraction performance of the target sound extraction apparatuses X1-X3. The figure showing the target sound extraction performance of the target sound extraction apparatuses X1-X3 and the conventional target sound extraction process under 1st experimental conditions. The figure showing the target sound extraction performance of the target sound extraction apparatuses X1-X3 and the conventional target sound extraction process under 2nd experimental conditions. The figure showing the 3rd experimental condition which evaluates the directivity of target sound extraction device X1. The figure showing the directivity of the target sound extraction apparatus X1 under 3rd experiment conditions. The block diagram showing schematic structure of the acoustic input device V2 which can be employ | adopted for the target sound extraction apparatus X1-X3. The block diagram showing the schematic structure of the sound source separation apparatus Z which performs the sound source separation process of the BSS system based on the FDICA method. The time chart showing the 1st example of the sequence of the process except the learning calculation in the sound source separation process of the target sound extraction apparatuses X1 to X3. The time chart showing the 2nd example of the sequence of the process except the learning calculation in the sound source separation process of the target sound extraction apparatuses X1-X3. The time chart showing the sequence of the learning calculation which concerns on 1st Example in the sound source separation process of the target sound extraction apparatuses X1-X3. The time chart showing the sequence of the learning calculation which concerns on 2nd Example in the sound source separation process of the target sound extraction apparatuses X1-X3.

Explanation of symbols

X1: target sound extraction device X2 according to the embodiment of the first invention X2: target sound extraction device X3 according to the embodiment of the second invention X: target sound extraction devices V1, V2 according to the embodiment of the third invention: sound input device 10 (10-1 to 10-3): sound source separation processing unit 20: target sound separation signal synthesis processing unit 31, 31 ′: spectrum subtraction processing unit 32: spectrum approximate signal extraction processing unit 41: main / sub acoustic signal specifying unit 42 : Signal switch 101: Main microphone 102: Sub microphone

Claims (7)

And one of the main sound signal obtained through one of the primary microphone mainly enter the target sound to be outputted from a predetermined target source, a plurality of sub microphones each having directivity in a plurality of different directions from the front Kinushi microphone A target sound extraction device that extracts an acoustic signal corresponding to the target sound and outputs an extraction signal based on a plurality of sub-acoustic signals obtained through
Provided individually for each combination of the two acoustic signals consisting of each of the main acoustic signal and the plurality of sub-acoustic signal, based on the two acoustic signals, wherein the objective sound separation signals corresponding to the target sound object Sound source separation means for separating and generating a reference sound separation signal corresponding to a reference sound other than sound by sound source separation processing by a blind sound source separation method based on an independent component analysis method ;
Target sound separation signal synthesis means for synthesizing a plurality of target sound separation signals separated and generated by the sound source separation means;
Spectral subtraction processing is performed between the synthesized signal obtained by the target sound separation signal synthesizing means and the plurality of reference sound separation signals separated and generated by the sound source separation means, thereby the target sound separation signal synthesizing means. Spectral subtraction processing means for extracting an acoustic signal corresponding to the target sound from the obtained synthesized signal and outputting the extracted signal;
A target sound extraction apparatus comprising: Oite to the sound source separation processing for the sound source separation unit executes, with the filtering process sequentially executes to generate a separated signal based on a predetermined separating matrix to audio signals inputted in a time series through a microphone, the time For each section signal segmented at a predetermined period in the acoustic signal input to the series, perform the sequential calculation to obtain the separation matrix used for the subsequent filter processing using all the section signals, and the number of times of the sequential calculation The target sound extraction device according to claim 1, wherein the number is limited to a predetermined number. Oite to the sound source separation processing for the sound source separation unit executes, with the filtering process sequentially executes to generate a separated signal based on a predetermined separating matrix to audio signals inputted in a time series through a microphone, the time For each signal in a part of the time zone on the head side of the section signal divided by a predetermined period in the acoustic signal input to the series, the separation matrix used for the subsequent filter processing is obtained using the signal. The target sound extraction apparatus according to claim 1, wherein sequential sound calculation is executed. Three or more, based on the input audio signal their Re respective directivity direction is obtained through different three or more microphones, one of the main acoustic signal and a plurality of the out of the three or more input audio signals Main / sub-acoustic signal specifying means for specifying sub-acoustic signals;
Signal path switching means for switching the transmission path of the acoustic signal from the three or more microphones to the sound source separation means according to the identification result by the main / sub acoustic signal identification means;
Target sound extraction apparatus according to any one of claims 1 to 3 and comprising comprises a. The main / sub-acoustic signal specifying means is based on a comparison of signal intensities of the three or more input acoustic signals, or a comparison of a proportion of a predetermined frequency component in each of the three or more input acoustic signals. The target sound extraction device according to claim 4 , wherein the main sound signal and the plurality of sub-acoustic signals are specified based on the above. And one of the main sound signal obtained through one of the primary microphone mainly enter the target sound to be outputted from a predetermined target source, a plurality of sub microphones each having directivity in a plurality of different directions from the front Kinushi microphone A target sound extraction program for causing a computer to execute a process of extracting an acoustic signal corresponding to the target sound and outputting the extracted signal based on a plurality of sub-acoustic signals obtained through
Computer
Individually for each combination of the two acoustic signals consisting of each of the main acoustic signal and the plurality of sub-acoustic signal, based on the two acoustic signals, target sound separation signal corresponding to the target sound and other than the target sound A sound source separation process for separating and generating a reference sound separation signal corresponding to the reference sound of the sound by a blind sound source separation method based on an independent component analysis method ;
A target sound separation signal synthesis process for synthesizing a plurality of target sound separation signals separated and generated by the sound source separation process;
By performing spectral subtraction processing between the synthesized signal obtained by the target sound separation signal synthesis processing and the plurality of reference sound separation signals separated and generated by the sound source separation processing, the target sound separation signal synthesis processing is performed. A process of extracting an acoustic signal corresponding to the target sound from the resultant synthesized signal and outputting an extracted signal;
The target sound extraction program characterized by running. And one of the main sound signal obtained through one of the primary microphone mainly enter the target sound to be outputted from a predetermined target source, a plurality of sub microphones each having directivity in a plurality of different directions from the front Kinushi microphone A target sound extraction method in which a computer executes a process of extracting an acoustic signal corresponding to the target sound and outputting the extracted signal based on a plurality of sub-acoustic signals obtained through
By computer
Individually for each combination of the two acoustic signals consisting of each of the main acoustic signal and the plurality of sub-acoustic signal, based on the two acoustic signals, target sound separation signal corresponding to the target sound and other than the target sound A sound source separation process for separating and generating a reference sound separation signal corresponding to the reference sound of the sound by a blind sound source separation method based on an independent component analysis method ;
A target sound separation signal synthesis process for synthesizing a plurality of target sound separation signals separated and generated by the sound source separation process;
By performing spectral subtraction processing between the synthesized signal obtained by the target sound separation signal synthesis processing and the plurality of reference sound separation signals separated and generated by the sound source separation processing, the target sound separation signal synthesis processing is performed. A process of extracting an acoustic signal corresponding to the target sound from the resultant synthesized signal and outputting an extracted signal;
The target sound extraction method characterized by performing.

Images