JP2008092363A - Signal separation apparatus and method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a signal separation apparatus and method in which signals can be correspondingly separated without performing post-processing after separation, when separating a mixed signal in which a plurality of signals are mixed, for each signal. <P>SOLUTION: The signal separation apparatus comprises a plurality of separation systems 1<SB>j</SB>each for generating a separated signal from an observation signal in a time domain or a time/frequency domain in which a plurality of signals are mixed and a separation matrix and a mapping unit 6 for generating a signal set of corresponding separated signals among a plurality of separated signals generated from different and multiple separation systems 1<SB>j</SB>, wherein the separation matrix is updated to maximize independency among signal sets and the separated signals are generated. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a signal separation apparatus and method for separating a signal using independent component analysis (ICA).

  Independent component analysis (ICA), which uses only statistical independence to separate and restore the original signal when multiple original signals are linearly mixed by unknown coefficients, It is attracting attention in the field. By applying this independent component analysis, voice signals can be separated and restored even in situations where the speaker and the microphone are separated and the microphone picks up sounds other than the speaker's voice. It becomes possible.

As shown in FIG. 20, it is assumed that different sounds are generated from N sound sources and these are observed by n microphones. Assuming that the signal (observation signal) x _k (t) observed by the k-th (1 ≦ k ≦ n) microphone k is a linear combination of the respective original signals s _j (t), x _k (t ) Is represented by the following formula (1). Further, if the observation signals for all the microphones are expressed by one equation, it can be expressed by the following equation (2). In these formulas (1) and (2), x (t) and s (t) represent column vectors each having x _k (t) and s _k (t) as elements, and A represents a _ij (t). This represents an n-row N-column matrix as an element. In the following, N = n.

  The ICA is formulated as a problem of estimating an original signal s and a matrix (separation matrix) that generates s from x from the observed signal x. That is, it is a problem of obtaining y and W such that y = Wx, where W is a separation matrix and y is an estimated original signal (ie, separation result).

JP 2003-263189 A JP 2006-238409 A JP 2004-145172 A “Blind sound source separation using multiple microphone pairs with different intervals”, Hiroshi Sawada, Akiko Araki, Ryo Mukai, Shoji Makino, Acoustical Society of Japan Spring Meeting, pp.621-622, March 2002 “Multimodal Independent Component Analysis -Common Feature Extraction Method from Multiple Sources-” Shotaro Akaho, Shinji Umeyama, IEICE Transactions, VOL.J83-A No.6 June 2000, pp.669-676

  The ICA has a problem that it is indeterminate which original signal is separated into which output channel. This is called “permutation problem between channels”. In the ICA in the time-frequency domain, which is called “permutation problem between frequency bins”, there is a problem that which original signal is output to which channel is inconsistent depending on the frequency bin. The permutation problem between bins "has been solved by the present inventors (see Patent Document 2).

  Regarding the “permutation problem between channels”, for example, in FIG. 20, two sound sources are picked up by two microphones and separated into two components, that is, N = 2 and n = 2. Think about the case. In this case, there are the following two types of sound sources that are output to two output channels.

Case 1: channel 1 = sound source 1, channel 2 = sound source 2
Case 2: channel 1 = sound source 2, channel 2 = sound source 1
Which case the output of the separation system becomes depends on various factors as described below, and it is difficult to predict in advance.

-Type of sound source-Positional relationship between sound source and microphone-ICA algorithm-Acoustic processing (Fourier transform, etc.), pre-processing, post-processing-Initial value of separation matrix W Next, consider the case where there are multiple separation systems. For example, consider a case where two 2-input / 2-output separation systems are operated in the above-described environment where the number of sound sources = 2. The second separation system may use a microphone at a position different from the first (that is, the positional relationship between the sound source and the microphone is different between the two separation systems), or a separate microphone may be used. An algorithm, an initial value, or the like may be used. In this case, which sound source is output to the output channel can be the following four types.

Case 1:
Separation system 1: channel 1 = sound source 1, channel 2 = sound source 2
Separation system 2: channel 1 = sound source 1, channel 2 = sound source 2
Case 2:
Separation system 1: channel 1 = sound source 1, channel 2 = sound source 2
Separation system 2: channel 1 = sound source 2, channel 2 = sound source 1
Case 3:
Separation system 1: channel 1 = sound source 2, channel 2 = sound source 1
Separation system 2: channel 1 = sound source 1, channel 2 = sound source 2
Case 4:
Separation system 1: channel 1 = sound source 2, channel 2 = sound source 1
Separation system 2: channel 1 = sound source 2, channel 2 = sound source 1
In the following, the case where the same sound source is output to the same channel between separation systems (case 1 and case 4) will be referred to as “the correspondence between the channels is taken”, otherwise ( Cases 2 and 3) are referred to as “channels are not compatible”.

  Here, in the case where only the output corresponding to the channel is generated, that is, only cases 1 and 4 are generated and cases 2 and 3 are prevented, the signal separation system using the above-mentioned ICA (hereinafter referred to as separation system). Let us consider a case where a plurality of output channels are associated with separation results. Specific examples include the following variations.

  Each separation system handles different but related types of signals. For example, when one separation system separates sound generated in the body and the other separation system separates electrical signals generated in the body, the sound signal derived from the heart (from the output of each separation system ( When associating a heart sound with an electrical signal (electrocardiogram) of the heart.

  When each separation system handles signals (same type) observed under different conditions. For example, when one separation system handles sound signals recorded with a wide interval microphone and the other separation system handles signals recorded with a narrow interval microphone, the output from the same sound source is separated into two When mapping between systems. In other cases, one system uses a plurality of microphones arranged in a horizontal row, and the other system uses microphones arranged in a perpendicular direction (that is, arranged in a vertical row). It is the same.

  The signal type and observation conditions are the same, but the initial conditions for ICA differ for each separation system. For example, when each separation system is a separation system in the time-frequency domain and the window length and shift width of the Fourier transform are different from each other, or when the initial value of the separation matrix is different, outputs derived from the same sound source When mapping between two separation systems.

  However, in general, ICA has a problem called “permutation (replacement) indefiniteness”, and it is indeterminate which original signal is output to which channel. Therefore, a plurality of separation systems are operated separately. However, there is a possibility that separation results that cannot be dealt with will be output, that is, Case 2 or Case 3 described above.

  Some of the prior arts related to ICA overlap with the above problem areas. For example, in Patent Document 1 and Non-Patent Document 1, a separation system using a wide-space microphone and a separation system using a narrow-space microphone are prepared, and the low frequency adopts the separation result of the wide-space microphone system, A method has been proposed for improving the signal separation performance by adopting the separation result of the narrow interval microphone for high frequencies. However, in this method, it is necessary to associate output channels between both separation systems, but this method is not mentioned.

  One method for performing the association is rearrangement by post-processing. For example, the separation system itself generates the case 2 and the case 3 described above, but thereafter, the output is exchanged between channels based on some scale, and finally the case 1 and the case 4 are obtained. However, since the scale used in the post-processing depends on the type of signal, there is no method corresponding to all the above problem areas. For example, when dealing with sound data, by using the scales used to solve permutation problems between frequency bins, such as the similarity of the temporal direction envelope and the estimated angle of the sound source direction, to some extent the channel Can be associated with each other, but application to other types of signals is difficult. For canceling permutation between frequency bins using a time direction envelope or a sound source direction, see, for example, Patent Document 3.

  In addition, even for sound signals, if microphones are installed at different positions in a plurality of separation systems, the sound source directions estimated by the respective systems may be different, which may make association based on the sound source directions difficult. .

Furthermore, as the number of output channels and the number of separation systems increase, the number of combinations increases rapidly, making it difficult to find an optimal association in post-processing. For example, when the number of output channels = 4 and the number of separation systems = 3, there are (4!) ³ = 13824 possible combinations. Therefore, it is more computationally preferable that the separation system directly outputs the associated result than the association in post-processing.

  Next, let us consider a case where the separation system directly outputs the associated signal instead of realizing the association by post-processing of ICA. For example, there are two separation systems for separating sound signals, and one separation system can obtain a separation result of “channel 1 = sound, channel 2 = music”, and the other separation system similarly. This is a case where a separation result “channel 1 = sound, channel 2 = music” is obtained. In this case, there are two original signals, one is voice and the other is music.

FIG. 21 is a schematic diagram illustrating a signal separation result when _m separation systems 100 _{1 to} 100 _m using ICA are provided. Each separation system outputs n independent components Y as separation results. In this figure, Y _i ^[j] is time series data (or similar data) and represents the entire output of the j th channel of the i th separation system. _Further, denoted ^{Y i [j]} at time t in the (or t-th sample) _Y ^{i [j]} and (t). Y _i ^[j] (t) may be a scalar quantity such as a speech waveform or a multi-dimensional quantity (multivariate) such as a spectrum. However, since the scalar quantity is considered to be a special case of a multi-dimensional quantity, Consider a case where Y _i ^[j] (t) is a multidimensional quantity (vector), particularly a spectrum. Note that the number of dimensions needs to be the same in the same separation system, but the number of dimensions may be different between different separation systems. For example, Y ₁ ^[1] (t) and Y ₂ ^[1] (t) need to have the same dimension, but Y ₁ ^[1] (t) and Y ₁ ^[2] (t) have dimensions. The number may be different.

Here, it is considered that the signals are separated between different output channels in the same system, while signals are separated (= dependence) between the same channels of different systems. For example, Y ₁ ^{[1] to} Y _n ^[1] are independent from each other, but Y ₁ ^{[1] to} Y ₁ ^[m] are dependent. In other words, when made k-th Y _{k ^[1]} in the output channel ^{to Y k _[m]} that the signal pairs, inside the pair-dependent, is between a pair separates the signal to be independently.

As one method for obtaining such independent components in pairs, a technique called “multimodal independent component analysis” has been proposed (see, for example, Non-Patent Document 2). This “multimodal independent component analysis” maximizes the independence between outputs in the same separation system when there are two separation systems (m = 2) (Y ₁ ^{[1 ]} And Y ₂ ^[1] are independent of each other, and the independence between the same channels between different separation systems is minimized (Y ₁ ^[1] and Y ₁ ^[2] are the least independent). Is like separating.

However, in multimodal independent component analysis, for example, when m> 2 as shown in FIG. 21 or when each Y _i ^[j] is a vector, it is difficult to separate signals for the following reason.

  First, in order to integrate the two measures of “independence in the same separation system (maximization)” and “independence between different systems (minimization)”, the weighted sum of the two is used. The weight had to be adjusted heuristically.

Next, in order to calculate “independence between different channels”, entropy is approximated using a two-dimensional Gram-Charlier expansion, but the Gram-Charlier expansion formula is already complex in the two-dimensional case. Therefore, it is difficult to increase the number of separation systems to 3 or more, or to extend Y _i ^[j] to multiple dimensions.

  The present invention has been proposed in view of such a conventional situation, and when separating a mixed signal in which a plurality of signals are mixed for each signal using independent component analysis, post-processing after separation is performed. It is an object of the present invention to provide a signal separation device and method capable of associating and separating signals without performing them.

  In order to achieve the above-described object, a signal separation device according to the present invention includes a plurality of separation means for generating a separation signal from an observation signal and a separation matrix in a time domain or a time frequency domain in which a plurality of signals are mixed, Using a signal set generation means for generating a signal set of corresponding separation signals among a plurality of separation signals generated from different separation means, and a multivariate probability density function or score function taking the signal set as an argument An independence calculation means for calculating the independence between the signal sets or a change in a measure of independence, and controlling the plurality of separation means so that the independence between the signal sets is maximized, Separation means generates a separation signal from the observed signal and a separation matrix into which initial values are substituted, and the separation matrix substantially converges based on the independence between the signal sets computed by the independence computation means. The separation matrix until Correct, and generates the separated signal by using the substantially converged separation matrix.

  In order to achieve the above-described object, a signal separation apparatus that separates an observation signal in a time domain or a time frequency domain, in which a plurality of signals are mixed, into a separated signal using independent component analysis, is separated from the observed signal. Separation means for generating a separation signal from the matrix, signal set generation means for generating a signal set of a corresponding separation signal among a plurality of separation signals generated by the separation means, and a multiplicity that takes the signal set as an argument. The independence of calculating the independence between the signal pairs using the random probability density function or the score function, or controlling the separation means so that the independence between the signal sets is maximized. The separation means generates a separation signal from the observation signal and a separation matrix into which the initial values are substituted, and based on the independence between the signal sets computed by the independence computation means Separation Column modifies the separation matrix until substantially converge, it generates the separated signal by using the substantially converged separation matrix.

  In addition, in order to achieve the above-described object, the signal separation method according to the present invention is a signal for separating an observation signal in a time domain or a time frequency domain in which a plurality of signals are mixed into a separated signal using independent component analysis. In the separation method, a signal set of a corresponding separation signal is generated between the step of generating a separation signal from the observation signal and the separation matrix by a plurality of separation means and a plurality of separation signals generated from a plurality of different separation means. A step of calculating an independence between the signal sets or a change in a measure of independence using a multivariate probability density function or a score function taking the signal set as an argument, and the observed signal and the initial value are A separation signal is generated from the substituted separation matrix and the separation matrix is substantially converged until the separation matrix is substantially converged so that the independence between the signal sets calculated using the probability density function or the score function is maximized. And a step of modifying a column.

  In addition, in order to achieve the above-described object, the signal separation method according to the present invention is a signal for separating an observation signal in a time domain or a time frequency domain in which a plurality of signals are mixed into a separated signal using independent component analysis. In the separation method, a step of generating a separation signal from the observed signal and the separation matrix, a step of generating a signal pair of a corresponding separation signal between the separation signals, and a multivariate probability density taking the signal pair as an argument A separation signal is generated from the step of calculating the independence between the signal pairs using a function or a score function, or a change in the measure of independence, and the separation matrix into which the observed signal and the initial value are substituted, and the probability Modifying the separation matrix until the separation matrix is substantially converged so that the independence between the signal sets calculated using the density function or the score function is maximized.

  According to the speech signal separation device and method therefor according to the present invention, a signal set of separated signals is generated, and the independence or independence between the signal sets using a multivariate probability density function or a score function that takes the signal set as an argument. The change of the gender measure is calculated, the separation matrix is corrected until the separation matrix is substantially converged so that the independence between the calculated signal pairs is maximized, and the separation is performed using the substantially converged separation matrix. By generating the signals, the signals can be associated and separated without performing post-processing after separation.

  Hereinafter, specific embodiments to which the present invention is applied will be described in detail with reference to the drawings. A “pair” used in the following description represents more than one set of signals and is not limited to two sets of signals.

FIG. 1 is a block diagram showing a configuration of a signal separation device according to an embodiment of the present invention. This signal separation device has m separation systems 1 ₁ to 1 _m using Independent Component Analysis (ICA). Details of the separation system will be described later.

  The signal integration unit 2 integrates the outputs (= separation results) of each separation system into several signals as necessary.

  The output unit 3 is a device such as a speaker or a display, and outputs voice, music, waveforms, and the like. For example, in the case of a sound signal separation system using an ICA in the time frequency domain, a time frequency domain signal (= spectrogram) is converted into a time domain signal (= waveform) and output from a speaker or the like.

  The control unit 4 comprehensively controls each component, and controls, for example, the integration and output of the separated signals separated by each separation system based on the correspondence between the signals.

Independence calculation unit 5, the separation system 1 ₁ to 1 independence between signals corresponding pair of signals calculated between a plurality of signals output from the _m, the separation system 1 ₁ to 1 _m based on independence To control. Instead of calculating the independence itself, any amount corresponding to the amount of change in independence (for example, a score function described later) may be calculated.

The mapping unit 6 generates a signal pair of corresponding signals among a plurality of signals output from the separation systems 1 ₁ to 1 _m , and the independence calculation unit 5 calculates the independence between the signal pairs. Information indicating whether a pair is formed between signals is given. That is, association between signals is performed. Specifically, one or more selected from the interval and position of the microphone, the sampling frequency, the window length of the short-time Fourier transform, the shift width of the short-time Fourier transform, the initial value of the separation matrix, and the normalization method of the observation signal spectrogram Configure pairs based on conditions. Details will be described later.

Next, the separation systems 1 ₁ to 1 _m will be described in detail. FIG. 2 is a block diagram showing the configuration of the jth separation system. The separation system 1 _j has a plurality of sensors 11 _{j1 to} 11 _jn and a separation matrix update unit 12 _j, and is observed by an observation signal X _k ^[j] (X _k ^[j] (kth sensor). A separation result (or “component”) Y ₁ ^{[j] to} Y _n ^[j] is generated by multiplying a vector composed of the signal) by a matrix W ^[j] for separation.

Various sensors 11 _{j1 to} 11 _jn can be used in accordance with the type of signal to be handled. For example, when a sound signal is handled, a microphone can be used. Further, the number of sensors may be different for each separation system 1 _j , and the sensors may be shared among a plurality of separation systems. Signals X ₁ ^{[j] to} X _n ^[j] observed by the sensors 11 _{j1 to} 11 _jn may be scalar quantities or multidimensional quantities (vector quantities). In addition, the number of elements of X _k ^[j] (the number of vector dimensions) may be different for each separation system.

The separation matrix update unit 12 _j updates the separation matrix W ^[j] based on the independence calculated by the independence calculation unit 5.

The separation system 1 _j sends the separated components Y ₁ ^{[j] to} Y _n ^[j] to the mapping unit 6 in order to form a pair between the separation systems or within the separation system. Then, the independence calculation unit 5 calculates the independence between the pairs configured by the mapping unit 6 (or an amount corresponding to the change amount of independence). Measure of the calculated independence at isolation calculation unit 5 is sent to the separation matrix updating section 12 _j, a separation matrix updating section 12 _j calculates the value of a new separation matrix W ^[j]. Detailed processing of the independence calculation unit 5 and the mapping unit 6 will be described for each specific example described later.

Next, the outline of the signal separation processing will be described with reference to the flowchart shown in FIG. First, signals are observed using the sensors 11 _{j1 to} 11 _jn (step S11). In the case of sound ICA, sound is recorded with a microphone. In step S12, the observation signals acquired by the sensors 11 _{j1 to} 11 _jn are processed as preprocessing.

  Steps S13 to S15 are a learning loop, and the separation process (step S14) is repeatedly executed until the separation matrix and the separation result converge. Repeating the process until the separation matrix and the separation result converge in this manner is hereinafter referred to as “learning”.

  When learning is completed in step S15, the signal is processed again as post-processing (step S16). The post-processed signal is processed at a later stage as a separation result (step S17). The subsequent processing is, for example, voice recognition processing.

  Next, the pre-processing in step S12 will be described using the flowchart shown in FIG. Here, a case where a sound signal is used as input and a time-frequency domain ICA is used as separation processing will be described.

  The observation signal converted into digital by the AD conversion in step S121 is converted into a signal in the time-frequency domain by short-time Fourier transform (step S122). Such a signal in the time frequency domain is called a spectrogram. By this operation, the observation signal which is a time series scalar quantity is converted into a time series vector quantity (multivariate).

  Next, in step S123, normalization is performed on the observed signal spectrogram as necessary. As normalization, for example, a method of processing a signal so that the average in the time direction becomes 0 and the variance becomes 1 for each frequency bin of the spectrogram can be used. Further, normalization may be performed for each separation system, each channel, or each frequency bin.

  The normalized observation signal is subjected to decorrelation as will be described later if necessary (step S124). In step S125, the initial value of the separation matrix is set. As an initial value, a unit matrix may be used, but a separation matrix obtained in the previous learning may be used.

  Next, the separation processing in step S14 will be described using the flowchart shown in FIG. The outline of this separation process is to repeat the process of “updating the separation matrix in a direction in which the independence between pairs increases” until the separation matrix converges.

  First, in order to calculate the independence between pairs, a pair is constituted by components between separation systems or between a plurality of channels in the separation system (step S141). Next, independence between the pairs is calculated (step S142), and ΔW, which is a modified value of the separation matrix, is calculated so that the independence increases (step S143). Finally, the separation matrix is updated based on the correction value (step S144).

  In an actual system, step S142 to step S144 may be collectively expressed as one expression, and expressions (15) and (17) described later correspond thereto. In this case, what corresponds to the calculation of the independence between the pairs in step S142 is to calculate the value of the score function appearing in the equation (17). Also, the processing that forms the pair in step S141 corresponds to “which component each score function takes as an argument” and is performed by the mapping unit 6.

  Next, the post-processing in step S16 will be described using the flowchart shown in FIG. The outline of the post-processing is to return the signal (spectrogram) converted to the time-frequency domain in the pre-processing to the time-domain signal (waveform), and is the same as the post-processing of the conventional time-frequency domain ICA.

  The rescaling in step S161 is a process of adjusting the scale between frequency bins. Thereafter, the separation result is returned to the waveform by the inverse Fourier transform in step S162. These processes can be omitted depending on the contents of the process subsequent to step S16.

  Next, the signal separation processing described above will be described in detail using a speech spectrum as a signal example.

First, a short-time Fourier transform (STFT) is applied to the signal observed by the microphone to obtain a spectrum for each frame. Assuming that the sampling frequency is Fs, the STFT window width is L, and M = L / 2 + 1, the spectrum obtained by the STFT is a component corresponding to a frequency (frequency bin) obtained by dividing the frequency from 0 to Fs / 2 into M equal parts. is there. These are set as X _k ^[j] (1, t),..., X _k ^[j] (M, t), and a vector obtained by arranging them vertically is set as X _k ^[j] (t).

  Next, a matrix for generating a separation result from the observation signal will be described. This matrix representation method can have several stages, from one corresponding to a specific frequency bin to one that separates the signals of all separation systems together.

In the separation system 1 _j , the separation results Y ₁ ^[j] (ω, t),..., Y _n ^[j] (ω, t) corresponding to the ωth frequency bin correspond to the ωth frequency bin. observed signals _X ^{1 [j]} that _{(ω, t), ···,} X n [j] (ω, t) to the separation matrix ^{W [j] (ω)} can be produced by applying a. This is represented by the following formula (3).

  Next, when Expression (3) is expanded for all frequency bins (ω = 1 to M) and the elements are rearranged, the following Expression (4) is obtained. This is equivalent to the following formula (5) and the following formula (6), and is a formula that separates observation signals input to the system at one time in one separation system.

Next, consider an equation for simultaneously separating the observation signals of all the separation systems 1 ₁ to 1 _m . For that purpose, the above equation (4) is expanded for all j (j = 1 to m) and the elements are rearranged, and as a result, the following equation (7) is obtained. In Equation (7), if the separation result, separation matrix, and observation signal are Y (t), W, and X (t), respectively, Y (t) = WX (t) as shown in Equation (8) below. Can be written. That is, even in an apparatus having a plurality of separation systems 1 ₁ to 1 _m , simultaneous separation of all observation signals can be expressed by one equation.

By using a matrix that collectively separates the signals of all the separation systems 1 ₁ to 1 _m in this way, “independence of pair units” described later can be introduced.

  Next, the Kullback-Leiblar (KL) information amount calculated from the multivariate (multidimensional) probability density function (Probability Density Function: PDF) will be described as one of the measures representing “independence of pair units”. .

  As preparation for introducing the Kullback-Leiblar information amount, two types of PDF are introduced. One is a function that takes the separation results (outputs) of all separation systems as an argument, that is, a function that gives the joint probability of a certain separation result. This is shown in the following formula (9). In the following description, all the arguments such as the left end of Expression (9) are expressed as vertical vectors, and all the arguments such as the center arranged horizontally are treated as equivalent. In addition, since PDF of Formula (9) disappears in the middle of formula deformation, it is not necessary to apply a specific formula.

  Another PDF is a function that takes an element of the k-th pair as an argument, that is, a function that gives a probability that a certain pair will occur. This is shown in the following formula (10). Specific examples of functions will be described later.

Here, consider an equation obtained by multiplying the above equation (10) for all k (k = 1 to n) (the right side of the following equation (11)). According to the definition of independence, the left side of Equation (11) (= Equation (9)) and the right side are equal only when the pair-wise separation results Y _{1 to} Y _n are independent of each other. Therefore, when a scale such as “distance” is used on both sides of equation (11), the scale is minimum (ideally 0) when Y ₁ to Y _n are independent.

There is a KL information amount as a measure representing “distance” between PDFs. The distance between the PDF of both sides of the equation (11), i.e., KL information amount, and Y co-entropy H of _(Y), Y 1 to Y _n of the respective entropy _H (Y 1) to H _(Y n) And is defined as in the following formula (12). Expression (12) is transformed into the following expression (13) using the definition of entropy and the relationship of Y = WX (the above expression (5)). Here, P (Yk (t)) shown in Equation (13) is the PDF of Equation (10).

In order to make the pair-wise separation results Y _{1 to} Y _n independent of each other, the separation matrix W that minimizes the KL information amount I (Y) calculated by the equation (13) may be obtained by some algorithm. . In the following, as an example of an algorithm, an equation by a natural gradient method is derived.

  In order to obtain W that minimizes I (Y) by the natural gradient method, Expressions (14) to (16) may be repeated until W converges (learning). However, Expression (14) means that Y (t) = WX (t) is performed for all frames. In addition, η in equation (16) is called a learning rate and is a small positive real number (for example, 0.1). η may be a constant or may be dynamically changed according to the degree of learning. For example, at the initial stage of learning, η is set to a small value (for example, about 0.05) to prevent W overflow, and when learning has converged, η is set to a large value (for example, 0.5) to accelerate convergence. Also good.

  Note that the specific expression of the partial differentiation of equation (15) varies depending on the type of signal and the pair configuration method.

  Hereinafter, an application example of the above-described signal separation processing method will be specifically described.

(Specific example 1)
As specific example 1, cooperative separation of different microphone intervals will be described. For example, as shown in FIG. 7, there may be a case where microphones (mic1, 4) with wide intervals and microphones (mic2, 3) with narrow intervals are provided.

Specifically, the observed signal spectrograms obtained from the signals observed with the wide interval microphone are X ₁ ^[1] , X ₂ ^[1] , and the separation results are Y ₁ ^[1] , Y ₂ ^[1] , narrow. Similarly, the interval microphones are set as X ₁ ^[2] , X ₂ ^[2] , Y ₁ ^[2] , Y ₂ ^[2], between Y ₁ ^[1] and Y ₁ ^[2] , and Y ₂ ^{[ 1]} and Y ₂ ^[2] are generated as a result of separation. In addition, although the separation system shown in FIG. 7 is equipped with four microphones, the present invention is not limited to this, and as shown in FIG. 8, a microphone at the same position is shared among a plurality of separation systems. It doesn't matter. In this figure, for example, X ₁ ^[1] and X ₁ ^[2] surrounded by a frame indicate that signals observed by the same microphone are used as observation signals of two systems.

When the separation matrix corresponding to the ω-th frequency bin of the separation system j is W ^[j] (ω) and the increment is ΔW ^[j] (ω), ΔW ^[j] (ω) is expressed by the following equation (17). ). In this equation, φ is a function having as an element a logarithmically differentiated PDF of the pair Y _k , and is called a score function or an activation function.

As a specific example of PDF, a kind of multivariate (multidimensional) PDF called a spherical distribution represented by the following formula (18) can be applied. This is expressed as a value obtained by substituting the LN norm (formula (19)) of the pair Y _k (t) into an arbitrary scalar function f (x).

Here, N is an arbitrary positive number, for example, N = 2 is used. The coefficient h is a term for adjusting the total sum of probabilities to 1, but it disappears when the score function is derived, so it is not necessary to obtain a specific value. Further, instead of the norm of Y _k (t), Expression (20) in which a weight is given for each element may be used. As the scalar function f (x), any non-negative function can be used. For example, Expression (21) or Expression (22) is used. In these equations, K and l are arbitrary positive values.

  In addition, as PDF, the sum of norms for each channel is substituted into f (x) (see equation (23)), or the product of norms is also substituted into f (x) (equation ( See 24). Moreover, you may use multivariate PDF other than spherical distribution.

A specific example of the score function is shown below. Expression (25) is a score function derived from the PDF of Expression (20) and f (x) of Expression (21). Expression (26) is obtained from the PDF of Expression (20) and Expression (22). This is a derived score function. In these equations, K _k ^[j] (ω) corresponds to K in Equation (21) or Equation (22), and different values may be used for each pair, each channel, and each frequency bin. Thereby, the scale of Y or W at the time of convergence can be controlled.

  As described above, all the signals to be matched (desired to be paired), such as not only between the frequency bins but also between the separation systems and between the plurality of channels in the system, are substituted into the argument of the score function. As a result, when the signals assigned to the same score function are paired, the KL information amount of the separation result is lower, that is, the independence of the entire signal is higher, so that the channels are associated with each other. A separation result is generated.

  Next, processing of the independence calculation unit 5 and the mapping unit 6 will be described with reference to FIG.

  The mapping unit 6 takes a correspondence relationship between the component (separation result) and the argument of the score function in order to express the score function as described above. For example, the independence calculation unit 5 calculates a score function when using an update expression based on a gradient method such as Expression (17).

  Here, different score functions are prepared for each separation system, each channel, and each frequency bin. That is, when the separation result in the entire system is represented by a vector Y (t), the number of elements of Y (t) and the number of score functions are the same.

That is, the score functions are grouped by those having the same argument. For example, the score function of group 1 represents taking the same argument. Φ ₁₁ ^{[1] to} φ _1M ^[1] represents a score function corresponding to the first channel of the separation system 1, and φ ₁₁ ^[1] represents a single score corresponding to the first frequency bin. Represents a function.

  The mapping unit 6 extracts corresponding components from each separation system and configures a pair. For example, pair 1 is a pair composed of the first channel of each separation system. Similarly, pair k is a pair composed of the kth channel of each separation system.

  The pair configured by the mapping unit 6 is supplied to the score function that takes the same argument by the independence calculation unit 5. For example, pair 1 is supplied to the score function of group 1, and similarly pair k is supplied to group k.

Note that the process of forming a pair does not necessarily need to be accompanied by a copy of data, and may be referred to by a pointer or the like. For example, group 1 refers to the pair 1, component Y ₁ ^[1] of the pair 1 may be a method of referring to the components Y ₁ ^[1] of the separation system 1.

(Specific example 2)
Next, analysis of different sounds will be described. For example, the microphone pair is shared between the separation systems, but the acoustic analysis may be different between the separation systems. The acoustic analysis here refers to sampling, discrete Fourier transform (DFT), and the like. For example, two separation systems share two microphones, one with 16 kHz sampling and the other with 48 kHz sampling. In addition, even when the same sampling frequency is used, the DFT window length is different between the two (for example, 512 and 2048), the window length is the same but the shift width is different (for example, 128 and 256), or the window type is different. Different cases (for example, Hanning window and square root Hanning window) correspond to different acoustic analysis.

Even in such a case, by using the same formula as in the first specific example, it is possible to generate a separation result that is associated with each other. However, if the acoustic analysis is different, the frame interval may be different. For example, when a DFT with a shift width of 128 is applied to a signal sampled at 16 kHz, the length per frame is 8 ms, and when a DFT with a shift width of 160 is applied, the length per frame is 10 ms. is there. In such a case, a column vector like Y _k (t) in equation (10) cannot be directly constructed, but for example, by using some interpolation method such as linear interpolation, the frame interval between them is unified. be able to. Further, the frame interval can be adjusted by duplicating the frame data as necessary. For example, as shown in FIG. 10, frame _{Y 1} ^[1] is 8ms _intervals, if ^{Y 1 [2]} is a frame 10ms _interval, ^{Y 1} with every other data to four frames of ^[2] If they are overlapped, the interval between Y ₁ ^[1] and Y ₁ ^[2] can be matched. Or conversely, Y ₁ ^[1] may be thinned out at a rate of once every 5 frames.

  As described above, in the specific example 1 and the specific example 2, different microphone intervals and different acoustic analysis have been introduced as different examples. Of course, a combination of the specific example 1 and the specific example 2 is also possible. For example, even when different acoustic analysis is used between a separation system with a wide microphone interval and a separation system with a narrow microphone interval, it is possible to generate an associated separation result by the same method as in the second specific example.

(Specific example 3)
Next, a simple method for realizing the present invention will be described. This is a method for realizing “separation of pair units” by diverting a system for performing separation for each channel as in Patent Document 2, in other words, when there is only one separation system in FIG. It is about the method of realizing “separation”.

FIG. 11 is a schematic diagram showing processing for realizing signal separation simply. Here, _n of the observation signals X ₁ ^{[1] to} X _n ^[m] represents a separation system, and m represents a signal pair. First, the observation signals are rearranged. That is, in this figure, the pairs of observation signals X _k ^{[1] to} X _k ^[m] arranged in the horizontal direction are rearranged in the vertical direction, which is _denoted as X ′ _k . Next, X _'k is regarded as one of the spectrogram, X' is separated into ₁ to X 'independent components Y for each channel _{n' 1 ~Y 'n.} In the separation of X ′ _{1 to} X ′ _n , only one separation system is required. The separation results Y ′ _{1 to} Y ′ _n are arranged in the same manner as X ′ _{1 to} X ′ _n . A desired separation result can be obtained by rearranging the separation result in reverse order to the rearrangement with respect to the observation signal.

  Similarly, as in the specific example shown in FIG. 7, when there are two separation systems having the frequency bin number = M and the channel number = 2, for example, by assuming that the frequency bin number = 2M and the channel number = 2. Thus, signal separation can be realized easily.

(Specific example 4)
Also, the number of channels may be different between separation systems. FIG. 12 is a diagram illustrating an example of signal correspondence in the separation system 1 with three inputs and three outputs and the separation system 2 with two inputs and two outputs. In this configuration example, the first and second channels are associated with each other in pairs. For the separation system 2, Equations (17) to (26) are applied with n = 2, m = 2, and j = 2. And about the separation system 1, a change is made to the score function corresponding to the 3rd channel.

  The score function used in the separation system 1 is expressed as the following equation (27). Here, the reason why the argument of the score function differs only for the third channel is that the corresponding component does not exist in the separation system 2. Therefore, the third channel uses a score function different from those in Expression (25) and Expression (26), and uses, for example, Expression (29) derived from Expression (28). In this equation, j ≠ 2 attached to the sigma symbol means the sum for the other separation systems. In the example of FIG. 12, since the number of separation systems = 2, j ≠ 2 has the same meaning as j = 1.

In the above equation (27), equation (29) is also used for the score function of the second channel (in the right side of equation (27), k = 2 in equation (29) instead of the second element from the top) And the second pair cannot be associated with each other. That is, it is undefined whether the component corresponding to Y ₂ ^[2] appears in Y ₂ ^[1] or Y ₃ ^[1] . In other words, if the number of arguments of the score function is made different for each channel, the presence / absence of association can be controlled for each channel.

  In other words, components that are to be associated are included in the argument of the score function, and components that do not need to be associated or cannot be associated are excluded from the argument of the score function. The association can be controlled.

(Specific example 5)
Further, the separation system may include a fixed output. As an example of application, in a system equipped with a microphone and a speaker, echo cancellation for removing the sound derived from the speaker from the sound recorded by the microphone can be mentioned. In the following, a system having a fixed output is also expressed as a “separation system”.

  FIG. 13 is a schematic diagram for explaining signal separation when there are two signal sources, one is sound, and the other is sound (music) from a speaker.

Here, the mixed sound from any number of sound sources is recorded by a plurality of microphones equal to or more than the number of sound sources. In addition, sound output from the speaker is directly input to one side of the separation system through a line output terminal or the like. Let X ₁ ^[2] be a directly input signal. When there are a plurality of speakers, signals that are not mixed with other sound sources are directly input from these speakers, and these are assumed to be X ₂ ^[2] , X ₃ ^[2] ,.

In addition, two types of separation systems are prepared. One is a separation system that separates a signal observed by a microphone into independent components, and the other is a separation system that outputs a directly input signal X ₁ ^[2] as it is. This can be regarded as a special case in which the separation matrix of the second separation system is fixed to the unit matrix in the specific example 4. Therefore, the same formula as in specific example 4 can be used.

That is, the separation system 1 is updated using the equations (17) and (27) to (29) similarly to the specific example 4, and the separation matrix 2 is not updated for the separation system 2. , Y _k ^[2] = X _k ^[2] .

Thereby, the signal corresponding to each channel of the separation system 2 is separated into each channel of the separation system 1 so as to be drawn. And the sound from which the sound derived from a speaker was canceled is output to the remaining channels of the separation system 1. In the example of FIG. 13, a component corresponding to a speaker-derived sound is output to Y ₁ ^[1] , and a component corresponding to a human voice is output to Y ₂ ^[1] .

(Specific example 6)
In the above-described specific example, it is assumed that the signals are not mixed between the separation systems (= the signals in the pair are not mixed), but the present invention can also be applied to the case where the signals are mixed in the pair. .

FIG. 14 is a schematic diagram for explaining a process of recording three sound sources with three microphones and separating them into three components. Here, it is assumed that two of the sound sources generate highly correlated signals. For example, a case where the same program is broadcast from two radios or a case where the left and right channels of a stereo speaker are used. Therefore, when the sound recorded by three microphones is separated into three components, there is a dependency between the two components, for example, depending on the pair of Y ₁ ^[1] and Y ₂ ^[1]. There is a relationship, and the pair and Y ₃ ^[1] alone are separated so as to be independent.

The fact that the pair of Y ₁ ^[1] and Y ₂ ^[1] and Y ₃ ^[1] alone are independent can be expressed as in Expression (30) using PDF. Therefore, if the distance between both sides of the expression (30) is expressed by the KL information amount and the separation matrix W is updated with an algorithm that minimizes the distance, the separation result as described above can be obtained.

  Note that the update of the separation matrix may be performed using Expression (17). However, equation (31) is used for the score function. That is, a component to be given a dependency is included in the argument of the score function.

  In order to express such a score function, the mapping unit 6 and the independent calculation unit 5 perform processing different from the processing shown in FIG. FIG. 15 is a schematic diagram for explaining the processing of the independence calculation unit 5 and the mapping unit 6.

  The independence calculation unit 5 groups score functions that take the same argument. Here, since a pair is composed of channel 1 and channel 2, one group 1 is composed of score functions corresponding to these channels, and another group 2 is composed of only the score function corresponding to channel 3. Composed.

  The mapping unit 6 forms a pair 1 with the channel 1 and the channel 2 and supplies it to the score function in the group 1 of the independence calculation unit 5. Further, another pair is formed by only channel 3 and is supplied to the score function of group 2 of the independence calculation unit 5.

(Specific example 7)
Further, in the conventional ICA, algorithms that converge at high speed under the restriction that the separation matrix W is an orthonormal matrix are known, but it is also possible to apply these algorithms. In the following description, the gradient method with an orthonormal constraint will be described as an example of an algorithm based on the orthonormal constraint, but other algorithms such as the Newton method and the fixed point method are also applicable.

  The orthonormal matrix is a matrix that satisfies the following equation (32). Considering the sparsity of W, this equation is equivalent to the equation (33) indicating that the separation matrix for each separation system and each frequency bin is orthonormal.

  In order to apply the orthonormal constraint to the separation matrix, the observed signal needs to be decorrelated in advance. The decorrelation is also called whitening or sphering, and here is a process of setting the correlation coefficient between channels to zero. Since equation (32) is equivalent to equation (33), decorrelation may be performed for each separation system and for each frequency bin. That is, processing such as “simultaneous decorrelation in the entire separation system” is unnecessary. The process of decorrelation will be described using the following equation.

Σ ^[j] (ω) is the variance-covariance matrix of the observed signal in the separation system j and frequency bin = ω (formula (34)). Further, Σ ^[j] (ω) can be expressed as Equation (35) using the eigenvalue λ _k ^[j] (ω) and the eigenvector p _k ^[j] (ω). A diagonal matrix composed of eigenvalues λ _k ^[j] (ω) is Λ ^[j] (ω) (formula (38)), a matrix composed of eigenvectors is P ^[j] (ω) (formula (37)), and X ^{When [j]} (ω, t) is transformed according to the equation (36), the resulting components of X ′ ^[j] (ω, t) are uncorrelated with each other. That is, Et [X ′ ^[j] (ω, t) X ′ ^[j] (ω, t) ^H ] = I is satisfied.

  In addition, learning is performed using equation (40) instead of equation (17). This equation is obtained by applying an orthonormal constraint to the equation (17). The score function φ in this equation is the same as that used in equation (17).

Also, some ICA algorithms incorporate both decorrelation and separation into the ΔW equation, which is equivalent to “Equivariant Adaptive Separation via Independence: EASI). Applying EASI to equation (40) yields equation (41). If this equation (41) is used, the term I−Y ^[j] (ω, t) Y ^ ^[j] (ω, t) ^H is decorrelated, so that prior decorrelation is unnecessary. .

  For EASI, reference is made to the literature (“Detailed Independent Component Analysis – New World of Signal Analysis”, Aapo Hyvarinen et al., Tokyo Denki University Press, pp.269272, 12.5 Equally Distributed Adaptive Separation by Independence) I want.

(Specific example 8)
When a plurality of separation systems having different microphone intervals are used and a specific frequency region is extracted and synthesized later, there are frequency bins that are not used. By removing these unused frequency bins from X _k ^{[j] in} advance, the amount of ICA calculation can be reduced.

(Specific example 9)
In the second specific example, as a method of integrating systems having different frame intervals, as shown in FIG. 10, a method in which the rough side (Y _k ^[2] ) of the frame interval is adjusted to the fine side (Y _k ^[1] ). However, as another example, learning may be performed using only frames with the same time.

For example, in FIG. 10, after the first frame, the frame times coincide again after 40 ms. That is, the Y _k ^[1] side is every 5 frames (1, 6, 11, 16,...), And the Y _k ^[2] side is every 4 frames (1, 5, 9, 13,... Therefore, only the frames with the same time are extracted from the observation signal, and the entire separation matrix is learned using the data. Although the separation matrix at this stage has correspondence between the outputs between the systems, it is learned with the thinned data, so the accuracy is lower than when learning with all the data, so this separation matrix is the initial value, Further learning is performed in each separation system. In that case, learning is performed using all frames.

(Specific Example 10)
Next, cooperative separation when the microphone installation positions are different between the separation systems will be described with reference to FIG.

  Before describing the cooperative separation, the relationship between ICA and beamforming will be described. It is known that the separation matrix obtained by ICA in the time-frequency domain is equivalent to the blind spot forming type among various beam forming methods using a microphone array. For the equivalence of the two, refer to the following documents, for example.

  Akiko Araki, Shoji Makino, Ryo Mukai, Hiroshi Saruwatari, “Relationship between Frequency Domain Blind Source Separation and Frequency Domain Adaptive Beamformer” Proceedings of the 2001 Acoustical Society of Japan Annual Meeting 2-6-12, pp. 613- 614

  Therefore, the separation result of ICA can be interpreted as follows. For example, when the i-th sound source is output to the channel k, a dead angle (a low sensitivity direction) is formed in the direction of the sound source other than the i-th by the microphone array, and the sound of the sound source other than the i-th sound is suppressed. As a result, it is considered that only the sound from the i-th sound source remains as a result.

  Since the sharpness of the formed blind spot is affected by the microphone installation position, the ICA separation performance is also affected by the microphone arrangement. For example, when forming a blind spot using microphones arranged in a straight line, a narrow (sharp) blind spot can be formed in a direction orthogonal to the microphone array, but a width is formed on the extension line of the microphone array. Only a wide blind spot can be formed. Therefore, in ICA using microphones arranged in a straight line, it is easy to separate a plurality of sound sources located in a direction orthogonal to the microphone row with high accuracy, but a plurality of sound sources located on an extension line of the microphone row are arranged. Is difficult to separate with high accuracy.

  Here, the microphone arrangement shown in FIG. 16 will be described. In this figure, there are two separation systems, one (separation system 1) uses a microphone arranged in the horizontal direction, and the other (separation system 2) uses a microphone arranged in the vertical direction. (In this figure, the central microphone is shared by both systems, but you may prepare them separately. Each system has five microphones, but if there are more than the number of sound sources, how many. However, it is assumed that there are four sound sources and they are playing simultaneously.

  For the microphone used in the separation system 1, the sound source 1 and the sound source 2 are positioned in a direction substantially orthogonal to the microphone row, so that the sound from each sound source can be pinpointly masked by a sharp blind spot. Therefore, the separation system 1 can separate the sound source 1 and the sound source 2 with high accuracy. On the other hand, since the sound source 3 and the sound source 4 are located almost on the extension line of the microphone row, it is easy to mask both sound sources together with a wide blind spot, but only one sound source is masked and the other is left. It ’s difficult. In summary, the separation result of the separation system 1 has the following characteristics.

Channel to which sound source 1 is mainly output:
Separated with high accuracy. Sound source 2 is masked with a sharp blind spot, and sound sources 3 and 4 are masked together with a wide blind spot.

Channel to which sound source 2 is mainly output:
Separated with high accuracy. Sound source 1 is masked with a sharp blind spot, and sound sources 3 and 4 are masked together with a wide blind spot.

Channel to which sound source 3 is mainly output:
Sounds like sound source 4 remains. The sound sources 1 and 2 are each masked with a sharp blind spot, whereas the sound source 4 is masked with a wide blind spot, and the sound source 3 is suppressed to some extent.

Channel to which sound source 4 is mainly output:
Sounds like sound source 3 remains. The sound sources 1 and 2 are each masked with a sharp blind spot, whereas the sound source 3 is masked with a wide blind spot, and the sound source 4 is suppressed to some extent.

  On the other hand, since the microphone used by the separation system 2 is orthogonal to the microphone of the separation system 1, the positional relationship between the sound source and the microphone row is opposite to that of the separation system 1. Therefore, the separation result of the separation system 2 has the following characteristics.

Channel to which sound source 1 is mainly output:
Sounds like sound source 2 remains.

Channel to which sound source 2 is mainly output:
Sounds like sound source 1 remains.

Channel to which sound source 3 is mainly output:
Separated with high accuracy.

Channel to which sound source 4 is mainly output:
Separated with high accuracy.

  In summary, when the microphone array and the sound source are in the relationship shown in FIG. 16, it is better to adopt the separation system 1 for the separation results corresponding to the sound source 1 and the sound source 2, and the sound source 3 and the sound source 4 are separated. It is better to adopt the system 2. In this way, by preparing a plurality of separation systems with different microphone arrangements, it is possible to employ only the channel corresponding to the sound source in the direction in which each microphone arrangement is good, among the separation results.

  In order to perform channel selection between separation systems as described above, it is necessary to associate channels between separation systems before that. By using the same method and formula as in the first specific example, it is possible to obtain an output in which the channels are associated between the separation systems. For example, when the sound source 4 is output to the first channel of the separation system, the sound source 4 is also output to the first channel of the separation system 2.

  As described above, by using the score function that takes the same argument in the output channel to be associated, a separation result in which the output channels are associated with each other can be generated between the plurality of separation systems. . Furthermore, it is possible to generate a separation result that “the signals in the pair are dependent but the signals between the pairs are independent of each other”.

  FIG. 17 is a schematic diagram for explaining a specific example of cooperative separation between a narrow interval microphone and a wide interval microphone. As shown in FIG. 17A, the closely spaced microphones 1 to 3 are installed at equal intervals of 7.5 cm. Widely spaced microphones 1, 3, 4 are installed at 15 cm and 30 cm intervals. That is, the number of channels n = 3 and the number of separation systems m = 2. Since there are microphones shared by both parties, the number of microphones in the entire system is four instead of six.

  Also, as shown in FIG. 17B, three speakers are placed at a distance of 1 m from the center of the microphone group, the sound source 1 is music from the 45 ° right front direction, the sound source 2 is the sound from the front, and the sound source 3 is the left It was music from the side.

  These sounds were recorded with four microphones and sampled at 16 kHz. The signal was subjected to FFT with a window length of 512 and a shift width of 128 to generate spectrograms of observation signals (six images). That is, M = (512/2) + 1 = 257. In order to calculate a ratio called SIR (Signal-to-Interference Ratio), which will be described later, the original signal that was sounded independently was observed with a plurality of microphones, and the waveforms were mixed by a computer.

  The observation signal spectrogram generated in this way was separated using Equation (17) and Equation (25). However, in the equation (25), the coefficient a is all 1 and K is common, and sqrt (M) ≈16.03 and N = 2. The number of iterations is 300.

18 and 19 show the separation results obtained from the narrow interval microphone and the wide interval microphone, respectively. It can be seen from the spectrograms shown in FIGS. 18A and 19A that corresponding signals are output to the same channel. That is, Y ₁ ^[1] and Y ₁ ^[2] are components corresponding to the sound source 3 of music, Y ₂ ^[1] and Y ₂ ^[2] are components corresponding to the sound source 2 of sound, and Y ₃ ^[1] and Y ₃ ^[2] is a component corresponding to the music source 1. In addition, when two separation systems (a system with a narrow interval microphone and a system with a wide interval microphone) are operated separately as in the past, a separation result in which the channel correspondence between both systems can be output is output. There is no guarantee. Therefore, it is an advantage of the present invention that a separation result that can be dealt with between systems in this way is generated.

  The right side of FIGS. 18 and 19 is the SIR for each frequency bin. The SIR is a scale representing how much power ratio the three original signals are included, and the unit in this figure is decibel. In a frequency bin in this graph, if the ratio of only one original signal is high (projecting to the right), it means that the separation is successful in that frequency bin, and conversely all three are as much If it is included in the ratio, it means that the separation has failed. Comparing the graph of FIG. 18B and the graph of FIG. 19B, it can be seen that the frequency bins separated with high accuracy are different between the two. Which frequency bin is likely (or difficult to) to succeed in separation depends to some extent on the distance between the microphones. Therefore, it is also possible to extract the frequency bin components separated with high accuracy from the results (spectrogram) of each separation system, and then mix them together to generate a higher accuracy separation result. is there.

  Now, regarding the theoretical difference between the present invention and the time frequency domain ICA before JP 2006-238409 (hereinafter, “conventional time frequency domain ICA”), the KL information amount is disclosed. This will be explained with a focus on. The amount of KL information is a measure such as the distance between PDFs, but the difference in which PDF is calculated differs among the three. In the following, the number of channels n = 3 and the number of separation systems m = 1, but the numbers may be arbitrary. Here, the following four PDFs are considered.

Expression (42) is a PDF having all the elements of the output as arguments, and represents the joint probability of all the arguments. Formula (43) represents that the pair of Y ₁ ^[1] and Y ₂ ^[1] and Y ₃ ^[1] alone are independent, and are used in Specific Example 6. Formula (44) represents that it is independent in spectrogram units. Expression (45) indicates that all arguments are independent.

  The conventional ICA in the time-frequency domain is equivalent to minimizing the KL information amount of Equation (45) and Equation (42). Since the expression (45) includes an extra condition that “the frequency bins in the same channel are also independent”, when the separation matrix and the separation result that minimize the amount of KL information are obtained, the same channel is obtained. A phenomenon in which different original signals appear between different frequency bins, that is, a permutation problem between frequency bins has occurred.

  On the other hand, in the technique described in Japanese Patent Application Laid-Open No. 2006-238409, the KL information amount of Expression (44) and Expression (42) is minimized. Since the equation (44) represents a condition that “different channels are independent but frequency bins in the same channel are not independent (with a dependency relationship)”, the separation result obtained by minimizing this KL information amount Then, the permutation problem between frequency bins is solved.

  Here, when Equation (42) is regarded as “PDF that has not been factorized at all” and Equation (45) is regarded as “PDF that is completely decomposed element by element”, Equation (44) is an intermediate stage between the two. Can be considered PDF. That is, the expression (44) shows that the decomposition of the PDF is stopped in the state of being decomposed for each channel.

  On the other hand, the present invention can be regarded as stopping the decomposition of the PDF in a “decomposed pairwise” state. For example, Equation (43) used in Specific Example 6 can be regarded as an intermediate PDF between Equation (42) and Equation (44).

  The method of configuring the pair is arbitrary, and the number of elements in the pair may be different for each pair. Therefore, any PDF that can exist between Expression (42) and Expression (45) “Intermediate stage PDF”) can be expressed. In other words, all ICAs that minimize the amount of KL information between the intermediate stage PDF and the equation (42) are included in the scope of the present invention.

It is a block diagram which shows the structure of the signal separation apparatus in one Embodiment of this invention. It is a block diagram which shows the structure of the j-th separation system. It is a flowchart explaining the outline of a signal separation process. It is a flowchart explaining the pre-process in a signal separation process. It is a flowchart explaining the separation process in a signal separation process. It is a flowchart explaining the post-process in a signal separation process. It is a schematic diagram for demonstrating the isolation | separation process in the case of providing a microphone of a wide space | interval and a microphone of a narrow space | interval. It is a schematic diagram which shows the case where the microphone in the same position is shared between several isolation | separation systems. It is a schematic diagram for demonstrating the process of an independence calculation part and a mapping part. Y ₁ ^[1] is a schematic diagram for frame 8ms _intervals, ^{Y 1 [2]} is for explaining a process performed when a frame of 10ms intervals. It is a schematic diagram which shows the process for implement | achieving signal separation simply. It is a figure which shows the example of the matching of the signal in the separation system of 3 inputs and 3 outputs, and the separation system of 2 inputs and 2 outputs. It is a schematic diagram for demonstrating signal separation in case there are two signal sources, one is sound, and the other is sound (music) coming out of a speaker. It is a mimetic diagram for explaining processing which records three sound sources with three microphones, and separates them into three components. It is a schematic diagram for demonstrating the process of an independence calculation part and a mapping part. It is a schematic diagram for demonstrating cooperative separation in case the installation position of a microphone differs between separation systems. It is a schematic diagram for demonstrating the specific example of the cooperative separation of a narrow interval microphone and a wide interval microphone. It is a figure which shows the isolation | separation result obtained from the narrow space | interval microphone. It is a figure which shows the isolation | separation result obtained from the wide space | interval microphone. It is a figure which shows the condition which observes the original signal output from N sound sources with n microphones. It is a schematic diagram which shows the separation result of the signal at the time of providing the m pieces of separation system using ICA.

Explanation of symbols

1 ₁ to 1 _m separation system, 2 signal integration unit, 3 output unit, 4 control unit, 5 independence calculation unit, 6 mapping unit

Claims (8)

In a signal separation device that separates an observation signal in a time domain or a time frequency domain in which a plurality of signals are mixed into a separated signal using independent component analysis,
A plurality of separation means for generating a separation signal from the observed signal and the separation matrix;
A signal set generation means for generating a signal set of corresponding separation signals among a plurality of separation signals generated from different separation means;
Using the multivariate probability density function or score function that takes the signal pair as an argument, calculate the independence between the signal pairs or the change in the independence measure, and so that the independence between the signal pairs is maximized An independence calculating means for controlling a plurality of separating means,
Each separation means generates a separation signal from the observation signal and the separation matrix into which the initial value is substituted, and the separation matrix is approximately based on the independence between the signal sets computed by the independence computation means. A signal separation device, wherein the separation matrix is corrected until convergence, and the separation signal is generated using a substantially converged separation matrix.   2. The signal separation device according to claim 1, wherein a Kullback-Leibler information amount calculated from the multivariate probability density function is used as a measure representing independence between the signal sets.   The signal set generation means is selected from microphone interval or installation position, sampling frequency, short-time Fourier transform window length, short-time Fourier transform shift width, initial value of separation matrix, and observation signal spectrogram normalization method 2. The signal separation device according to claim 1, wherein a signal set of corresponding separation signals is generated among a plurality of separation signals generated from a plurality of separation means having at least one or more different conditions. In a signal separation device that separates an observation signal in a time domain or a time frequency domain in which a plurality of signals are mixed into a separated signal using independent component analysis,
Separation means for generating a separation signal from the observed signal and the separation matrix;
Signal set generation means for generating a signal set of corresponding separation signals between the channels of the separation means;
Using the multivariate probability density function or score function that takes the signal pair as an argument, calculate the independence between the signal pairs or the change in the independence measure, and so that the independence between the signal pairs is maximized An independence calculating means for controlling the separating means,
The separation means generates a separation signal from the observation signal and a separation matrix into which initial values are substituted, and the separation matrix is approximately based on the independence between the signal sets calculated by the independence calculation means. A signal separation device, wherein the separation matrix is corrected until convergence, and the separation signal is generated using a substantially converged separation matrix. In a signal separation method for separating a time-domain or time-frequency domain observation signal in which a plurality of signals are mixed into a separated signal using independent component analysis,
Generating a separation signal from the observed signal and the separation matrix by a plurality of separation means;
Generating a signal set of corresponding separated signals among a plurality of separated signals generated from different separating means;
Calculating an independence between the signal pairs or a change in an independence measure using a multivariate probability density function or a score function taking the signal pairs as arguments;
A separation signal is generated from the observation signal and a separation matrix into which initial values are substituted, and the separation matrix is converged so that the independence between signal sets calculated using the score function is maximized. Modifying the separation matrix.   6. The signal separation method according to claim 5, wherein a Kullback-Leibler information amount calculated from the score function is used as a measure representing independence between the signal sets.   In the process of generating the above signal set, select from microphone spacing or installation position, sampling frequency, short Fourier transform window length, short Fourier transform shift width, initial value of separation matrix, and observation signal spectrogram normalization method 6. The signal separation method according to claim 5, wherein a signal set of corresponding separation signals is generated among a plurality of separation signals generated from a plurality of separation means having different one or more conditions. In a signal separation method for separating a time-domain or time-frequency domain observation signal in which a plurality of signals are mixed into a separated signal using independent component analysis,
Generating a separation signal from the observed signal and the separation matrix;
Generating a signal set of corresponding separated signals between the channels;
Calculating an independence between the signal pairs or a change in an independence measure using a multivariate probability density function or a score function taking the signal pairs as arguments;
A separation signal is generated from the observation signal and a separation matrix into which initial values are substituted, and the separation matrix is converged so that the independence between signal sets calculated using the score function is maximized. Modifying the separation matrix.

Images