JP2014137389A - Acoustic analyzer - Google Patents

Abstract

PROBLEM TO BE SOLVED: To precisely separate a specific sound component of sound signals.SOLUTION: A storage 24 stores a known instructor matrix F which includes plural base vectors each representing a spectrum of an instructor sound component in which acoustic property is close to a target sound component. A matrix analyzing section 34 performs a non-negative value matrix factorization on a sound signal SA(t) to calculate: a basis matrix B including plural base vectors each of which is a matrix in which the instructor matrix F is added with a compensation matrix D and which represents a spectrum of the target sound component of the sound signal SA(t); a coefficient matrix G including plural coefficient vectors each representing a time change of weight with respect to the base vectors of the basis matrix B; a basis matrix H including plural base vectors each representing a spectrum of a non-target sound component other than the target sound component of the sound signal SA(t); and a coefficient matrix U including plural coefficient vectors each representing a time change of weight with respect to the respective base vectors of the basis matrix H.

Description

  The present invention relates to a technique for separating (for example, extracting or suppressing) a specific acoustic component from an acoustic signal of a mixed sound of a plurality of acoustic components.

  Conventionally, a sound source separation technique for extracting or suppressing a specific acoustic component from a mixed sound of a plurality of acoustic components having different acoustic characteristics has been proposed. For example, Non-Patent Document 1 and Non-Patent Document 2 disclose unsupervised sound source separation using non-negative matrix factorization (NMF).

  In the techniques of Non-Patent Document 1 and Non-Patent Document 2, as shown in FIG. 9, an observation matrix Y indicating an amplitude spectrogram of an observation sound in which a plurality of acoustic components is mixed is converted into a base matrix H and a coefficient by non-negative matrix factorization. It is decomposed into a matrix (activation matrix) U. The basis matrix H is composed of a plurality of basis vectors h indicating the spectrum of each acoustic component included in the observation sound, and the coefficient matrix U is composed of a plurality of coefficient vectors u indicating the time variation of the weight value for each basis vector. The A plurality of basis vectors h of the basis matrix H and a plurality of coefficient vectors u of the coefficient matrix U are selected for each acoustic component (for each sound source), and a basis vector h and a coefficient vector u of a desired acoustic component are extracted and multiplied. Thus, an amplitude spectrogram of the acoustic component is generated.

A. CICHOCKI, et. Al., "NEW ALGORITHMS FOR NON-NEGATIVE MATRIX FACTORIZATION IN APPLICATIONS TO BLIND SOURCE SEPARATION," ICASSP 2006 Tuomas Virtanen, "Monaural Sound Source Separation by Nonnegative Matrix Factorization With Temporal Continuity and Sparseness Criteria", IEEE Trans. Aurio, Speech and Language Processing, volume 15, pp.1066-1074, 2007

  However, in the techniques of Non-Patent Document 1 and Non-Patent Document 2, a plurality of basis vectors h of the basis matrix H and a plurality of coefficient vectors u of the coefficient matrix U are accurately selected (clustered) for each acoustic component of the acoustic signal. Is difficult. Therefore, in reality, it is not easy to extract or suppress only a specific acoustic component of the acoustic signal with high accuracy. In view of the above circumstances, an object of the present invention is to separate a specific acoustic component of an acoustic signal with high accuracy.

  In order to solve the above problems, the acoustic analysis device of the present invention adds a compensation matrix (eg, compensation matrix D) to a known teacher matrix (eg, teacher matrix F) including a plurality of basis vectors indicating the spectrum of the teacher acoustic component. A first base matrix (for example, base matrix B) including a plurality of base vectors indicating the spectrum of the target sound component of the acoustic signal, and a time change of a weight value for each base vector of the first base matrix. A first coefficient matrix (for example, coefficient matrix G) including a plurality of coefficient vectors, and a second basis matrix (for example, base matrix H) including a plurality of basis vectors indicating spectra of non-target sound components other than the target sound component of the acoustic signal. And a second coefficient matrix (for example, coefficient matrix U) including a plurality of coefficient vectors indicating temporal changes in weight values for the respective basis vectors of the second basis matrix, and a non-negative value for the acoustic signal It comprises a matrix analysis means for calculating the column factorization. In the above configuration, since the non-negative matrix factorization using the unknown compensation matrix having the first coefficient matrix in common with the known teacher matrix indicating the teacher acoustic component is performed, the target sound component of the acoustic signal is There is an advantage that the target sound component can be separated with high accuracy even when the acoustic characteristic does not completely match the teacher acoustic component.

  The acoustic analysis device according to a preferred aspect of the present invention includes at least a target sound component according to the first basis matrix and the first coefficient matrix, and a non-target sound component according to the second basis matrix and the second coefficient matrix. Sound source separation means for generating one is provided. In the above aspect, the target sound component and the non-target sound component of the acoustic signal can be separated (extracted or suppressed).

  In a preferred aspect of the present invention, the matrix analysis means performs non-negative matrix factorization under a constraint condition (for example, constraint condition C1 described later) that the degree of similarity between the teacher matrix and the compensation matrix decreases. In the above aspect, since the non-negative matrix factorization is performed so that the similarity between the teacher matrix and the compensation matrix decreases, it is possible to avoid the teacher matrix and the compensation matrix being in common (the teacher among the target sound components). There is an advantage that an acoustic component different from the acoustic component can be extracted as a compensation matrix.

  In a preferred aspect of the present invention, the matrix analysis means performs non-negative matrix factorization under a constraint condition (for example, constraint condition C2 described later) that the similarity between the teacher matrix and the second basis matrix decreases. . In the above aspect, there is an advantage that it is possible to avoid that the teacher matrix and the second basis matrix are common (a component that approximates the teacher acoustic component among the target sound components of the acoustic signal is included in the second basis matrix). is there.

  In a preferred aspect of the present invention, the matrix analysis means performs non-negative matrix factorization under a constraint condition (for example, constraint condition C3 described later) that the degree of similarity between the compensation matrix and the second basis matrix decreases. . In the above aspect, it is possible to avoid that the compensation matrix and the second basis matrix are common (a component different from the teacher acoustic component among the target sound components of the acoustic signal is included in the second basis matrix). There is.

  In a preferred aspect of the present invention, the matrix analysis means performs non-negative matrix factorization under a constraint condition (for example, constraint condition C4 described later) that the similarity between the first basis matrix and the second basis matrix decreases. Run. The above aspect has an advantage that the first base matrix and the second base matrix can be prevented from being shared (the target sound component and the non-target sound component can be clearly distinguished).

  Note that “the similarity between matrices decreases” in the constraint conditions of each aspect described above means that the correlation of each matrix decreases (ideally minimizes), and the distance of each matrix increases (ideal It implies both maximization).

  In a preferred aspect of the present invention, the compensation matrix includes a negative number element, and the matrix analysis means is a non-negative matrix under a constraint condition that the first basis matrix is a non-negative matrix (for example, a constraint condition C5 described later). Perform factorization. In the above aspect, since the compensation matrix may include a negative element, the target sound component can be separated with high accuracy even when the spectrum intensity (for example, amplitude) of the target sound component is lower than the spectrum intensity of the teacher sound component. There is. In addition, since the constraint that the first basis matrix is a non-negative matrix is taken into account, it is possible to appropriately execute non-negative matrix factorization of the observation matrix. In addition, the specific example of the above aspect is later mentioned, for example as 2nd Embodiment.

  The acoustic analysis apparatus according to each aspect described above is realized by hardware (electronic circuit) such as DSP (Digital Signal Processor) dedicated to the analysis of acoustic signals, and general-purpose computation such as CPU (Central Processing Unit). This is also realized by cooperation between the processing device and the program. The program of the present invention is a matrix obtained by adding a compensation matrix to a non-negative teacher matrix including a plurality of basis vectors indicating the spectrum of the teacher acoustic component, and includes a plurality of basis vectors indicating the spectrum of the target sound component of the acoustic signal. 1 basis matrix, a first coefficient matrix including a plurality of coefficient vectors indicating temporal changes in weight values for each basis vector of the first basis matrix, and a plurality of spectra indicating non-target sound components other than the target sound component of the acoustic signal A non-negative matrix factorization for the acoustic signal, and a second coefficient matrix including a plurality of coefficient vectors indicating a time change of a weight value for each basis vector of the second basis matrix Causes the computer to execute matrix analysis processing. The program of the present invention can be provided in a form stored in a computer-readable recording medium and installed in the computer. The recording medium is, for example, a non-transitory recording medium, and an optical recording medium (optical disk) such as a CD-ROM is a good example, but a known arbitrary one such as a semiconductor recording medium or a magnetic recording medium This type of recording medium can be included. For example, the program of the present invention can be provided in the form of distribution via a communication network and installed in a computer.

1 is a block diagram of an acoustic analysis device according to a first embodiment of the present invention. It is explanatory drawing of a teacher matrix. It is explanatory drawing of operation | movement of a matrix analysis part. It is explanatory drawing of the relationship between a teacher matrix, a compensation matrix, and a base matrix. It is explanatory drawing of the subject which 2nd Embodiment solves. It is a chart of an experimental result of a 2nd embodiment. It is explanatory drawing of the effect of 2nd Embodiment. It is explanatory drawing of the teacher matrix and compensation matrix which concern on a modification. It is explanatory drawing of the nonnegative matrix factorization in background art.

<First Embodiment>
FIG. 1 is a block diagram of an acoustic analysis apparatus 100 according to the first embodiment of the present invention. As shown in FIG. 1, a signal supply device 12 and a sound emission device 14 are connected to the acoustic analysis device 100. The signal supply device 12 supplies the acoustic signal SA (t) to the acoustic analysis device 100. The acoustic signal SA (t) is a time domain signal indicating a waveform of a mixed sound of a plurality of acoustic components (for example, musical sound and voice) having different acoustic characteristics (t: time). For example, an acoustic signal SA (t) indicating a mixed sound of performance sounds (including sounds such as singing sounds) of a plurality of types of musical instruments is supplied to the acoustic analysis device 100. A sound collection device that collects ambient sounds and generates an acoustic signal SA (t), or a playback device that acquires the acoustic signal SA (t) from a portable or built-in recording medium and supplies it to the acoustic analysis device 100 Alternatively, a communication device that receives the acoustic signal SA (t) from the communication network and supplies the acoustic signal SA (t) to the acoustic analysis device 100 can be employed as the signal supply device 12.

  The acoustic analysis device 100 of the first embodiment is an acoustic processing device (sound source separation device) that generates an acoustic signal SB (t) by acoustic processing on the acoustic signal SA (t) supplied from the signal supply device 12. The acoustic signal SB (t) is a sound obtained by extracting a specific acoustic component (hereinafter referred to as “target sound component”) from among a plurality of acoustic components included in the acoustic signal SA (t) (that is, a non-purpose other than the target sound component). It is a time domain signal showing a waveform of a sound with suppressed sound components. The sound emitting device 14 (for example, a speaker or headphones) radiates a sound wave corresponding to the acoustic signal SB (t) supplied from the acoustic analysis device 100. The D / A converter that converts the acoustic signal SB (t) from digital to analog is not shown for convenience.

  As shown in FIG. 1, the acoustic analysis device 100 is realized by a computer system including an arithmetic processing device 22 and a storage device 24. The storage device 24 stores a program PGM executed by the arithmetic processing device 22 and various data used by the arithmetic processing device 22. A known recording medium such as a semiconductor recording medium or a magnetic recording medium or a combination of a plurality of types of recording media can be arbitrarily employed as the storage device 24. A configuration in which the acoustic signal SA (t) is stored in the storage device 24 (therefore, the signal supply device 12 can be omitted) is also suitable.

  The storage device 24 of the first embodiment stores a base matrix (hereinafter referred to as “teacher matrix”) F that represents acoustic characteristics corresponding to (for example, approximate or match) the target sound component. The acoustic analysis device 100 generates the acoustic signal SB (t) from the acoustic signal SA (t) by supervised sound source separation using the teacher matrix F stored in the storage device 24 as prior information (teacher information). .

  The teacher matrix F is generated in advance from an acoustic component generated by a known sound source (hereinafter referred to as “teacher acoustic component”) and stored in the storage device 24. The teacher acoustic component is an acoustic component whose acoustic characteristics (for example, timbre) are similar to or in common with the target sound component. For example, the teacher sound component and the target sound component correspond to performance sounds of different musical instruments (for example, performance sounds of different types of flutes) although the types of musical instruments (for example, the principle of pronunciation) are common to each other. . Therefore, the acoustic characteristics of the teacher acoustic component and the acoustic characteristics of the target sound component of the acoustic signal SA (t) are approximated to each other to the extent that the listener can perceive that they are performance sounds of the same type of instrument, but are completely matched. do not do. In other words, it can be said that the teacher sound component expressed by the teacher matrix F prepared in advance and the sound component whose sound characteristics are approximated correspond to the target sound component. On the other hand, the non-target sound component and the teacher sound component have different acoustic characteristics. For example, the teacher sound component and the non-target sound component correspond to performance sounds of different types of musical instruments, and the acoustic characteristics are different to the extent that the listener can perceive that they are performance sounds of different types of musical instruments. . For example, the target sound component and the teacher sound component correspond to a flute performance sound, and the non-target sound component corresponds to a clarinet performance sound.

  FIG. 2 is an explanatory diagram of processing for generating the teacher matrix F from the teacher acoustic component. The observation matrix X in FIG. 2 is an M-row N-column non-negative matrix representing a time series (amplitude spectrogram) of amplitude spectra of N frames obtained by dividing pre-recorded teacher acoustic components on the time axis. (M and N are natural numbers). That is, the nth column (n = 1 to N) of the observation matrix X corresponds to the amplitude spectrum x [n] of the nth frame among the teacher acoustic components. The element of the m-th row (m = 1 to M) of the amplitude spectrum x [n] means an amplitude value at the m-th frequency among M frequencies set on the frequency axis.

The observation matrix X in FIG. 2 is decomposed into a teacher matrix F and a coefficient matrix (activation matrix) Q by non-negative matrix factorization (NMF) as expressed by the following equation (1). Is done.

  As shown in FIG. 2, the teacher matrix F of Equation (1) has M rows in which K basis vectors f [1] to f [K] corresponding to the components constituting the teacher acoustic component are arranged in the horizontal direction. It is a non-negative matrix (base matrix) of K columns. The basis vector f [k] of the k-th column (k = 1 to K) in the teacher matrix F corresponds to the amplitude spectrum of the k-th component among the K components (bases) constituting the teacher acoustic component. . That is, the element of the m-th row (m-th row and k-th column of the teacher matrix F) of the basis vector f [k] is the m-th component on the frequency axis in the amplitude spectrum of the k-th component of the teacher acoustic component. It means the amplitude value at frequency.

  As shown in FIG. 2, the coefficient matrix Q of Equation (1) arranges K coefficient vectors q [1] to q [K] corresponding to the respective base vectors f [k] of the teacher matrix F in the vertical direction. This is a non-negative matrix of K rows and N columns. The coefficient vector q [k] in the k-th row of the coefficient matrix Q corresponds to a time series of weight values (activity) for the base vector f [k] of the teacher matrix F. As understood from the above description, one amplitude spectrum x [n] of the observation matrix X is expressed as a weighted sum of the K basis vectors f [1] to f [K] of the teacher matrix F.

  The teacher matrix F and the coefficient matrix Q are calculated so that the matrix FQ obtained by multiplying the teacher matrix F and the coefficient matrix Q approximates the observation matrix X (that is, the similarity between the matrix FQ and the observation matrix X increases). In addition, the teacher matrix F is stored in the storage device 24. Each of the K basis vectors f [1] to f [K] of the teacher matrix F generally corresponds to different pitches. That is, the teacher acoustic component used for generating the teacher matrix F is generated so as to include all possible pitches of the target sound component of the acoustic signal SA (t), and the basis vector f [k of the teacher matrix F is generated. ] (Base number) K is set to a value equal to or greater than the total number of pitches that can be assumed as the target sound component of the acoustic signal SA (t). The above is the procedure for generating the teacher matrix F.

  The arithmetic processing unit 22 in FIG. 1 executes a program PGM stored in the storage device 24 to thereby generate a plurality of functions (frequency analysis unit 32) for generating the acoustic signal SB (t) from the acoustic signal SA (t). , Matrix analysis unit 34, and sound source separation unit 36). Processing by each element of the arithmetic processing unit 22 is sequentially repeated in units of N frames obtained by dividing the acoustic signal SA (t) on the time axis. A configuration in which each function of the arithmetic processing unit 22 is distributed over a plurality of integrated circuits, or a configuration in which a dedicated electronic circuit (for example, a DSP) realizes a part of the functions may be employed.

  FIG. 3 is an explanatory diagram of processing by the frequency analysis unit 32 and the matrix analysis unit 34. The frequency analysis unit 32 sequentially generates the observation matrix Y in FIG. 3 in units of N frames of the acoustic signal SA (t). As shown in FIG. 3, the observation matrix Y is a time series (amplitude spectrogram) of amplitude spectra y [1] to y [N] of N frames obtained by dividing the acoustic signal SA (t) on the time axis. Is a non-negative matrix with M rows and N columns. That is, the nth column of the observation matrix Y corresponds to the amplitude spectrum y [n] (sequence of amplitude values at each of the M frequencies) of the nth frame of the acoustic signal SA (t). For the generation of the observation matrix Y, a known frequency analysis such as a short-time Fourier transform is used. The time series of the power spectrum of each frame of the acoustic signal SA (t) can be used as the observation matrix Y.

The matrix analysis unit 34 in FIG. 1 performs non-negative matrix factorization (NMF) on the observation matrix Y using the known teacher matrix F stored in the storage device 24 as prior information. The matrix analysis unit 34 according to the first embodiment uses the observation matrix Y generated by the frequency analysis unit 32 as a basis matrix B, a coefficient matrix G, a basis matrix H, and a coefficient matrix, as expressed by the following formula (2). Disassemble into U.

  As shown in FIG. 3, the basis matrix B of the equation (2) is a non-negative matrix of M rows and K columns in which K basis vectors b [1] to b [K] are arranged in the horizontal direction. In addition, the coefficient matrix G of the equation (2) has K coefficient vectors g [1] to g [K] corresponding to the basis vectors b [k] of the basis matrix B in the vertical direction, as shown in FIG. 2 is a non-negative matrix of K rows and N columns arranged in a matrix. The coefficient vector g [k] in the k-th row of the coefficient matrix G corresponds to a time series of weight values (activity) for the base vector b [k] of the base matrix B. That is, the element in the nth column of the coefficient vector g [k] means a weight value of the base vector f [k] in the nth frame among the N frames of the acoustic signal SA (t).

  As expressed by Equation (2), the base matrix B is expressed by adding the known teacher matrix F and the unknown compensation matrix D stored in the storage device 24. As described above, the teacher matrix F is a non-negative matrix of M rows and K columns in which K basis vectors f [1] to f [K] corresponding to the components constituting the teacher acoustic component are arranged in the horizontal direction. . On the other hand, the compensation matrix D is a non-negative matrix of M rows and K columns in which K basis vectors d [1] to d [K] are arranged in the horizontal direction, as shown in FIG. As shown in Equation (2) and FIG. 3, the basis vector b [k] of the kth column of the basis matrix B is the basis vector f [k] of the kth column of the teacher matrix F and the kth column of the compensation matrix D. This corresponds to addition with the basis vector d [k].

  As understood from Equation (2), the teacher matrix F and the compensation matrix D correspond to a common coefficient matrix G. That is, the basis vector d [k] of the compensation matrix D is excited with the same degree (one weight value in the coefficient vector g [k]) at the same time as the basis vector f [k] of the teacher matrix F. Corresponds to the amplitude spectrum to be applied. As understood from the above relationship, the basis vector f [k] of the teacher matrix F and the basis vector d [k] of the compensation matrix D correspond to the amplitude spectrum of the acoustic component common to the sound sources. As described above, the teacher matrix F is generated using a teacher acoustic component whose acoustic characteristics approximate to the target sound component (for example, an acoustic component in which the type of sound source is common to the target sound component). Therefore, as shown in FIG. 4, the base vector b [k] of the base matrix B obtained by adding the base vector f [k] of the teacher matrix F and the base vector d [k] of the compensation matrix D is the acoustic signal SA ( Among the plurality of acoustic components of t), the acoustic characteristic corresponds to the amplitude spectrum Ab of the target sound component whose acoustic characteristics approximate to the teacher acoustic component (for example, the acoustic component having the same sound source type as the teacher acoustic component). That is, as can be understood from FIG. 4, the basis vector d [k] of the compensation matrix D is changed to the amplitude spectrum Ad of the difference (residual) between the amplitude spectrum Ab of the target sound component and the amplitude spectrum Af of the teacher sound component. It can be understood that it corresponds. Therefore, when the acoustic characteristic of the target sound component in the acoustic signal SA (t) completely matches the acoustic characteristic of the teacher acoustic component, the compensation matrix D is a zero matrix. In addition, the base vector d [k] of the compensation matrix D is approximated (ideally matched) to the amplitude spectrum Ab of the target sound component in the acoustic signal SA (t). The amplitude spectrum Af in FIG. 11 can be transformed into an element for compensating (absorbing) the difference between the amplitude spectrum Ab of the target sound component and the amplitude spectrum Af of the teacher sound component.

  As described above, since each base vector b [k] of the base matrix B corresponds to the amplitude spectrum Ab of the target sound component, the matrix BG of the formula (2) obtained by multiplying the base matrix B and the coefficient matrix G is An amplitude spectrogram of the target sound component in the acoustic signal SA (t) is expressed. Therefore, the matrix HU obtained by multiplying the base matrix H and the coefficient matrix U in Equation (2) is M rows and N columns representing the amplitude spectrogram of the non-target sound component other than the target sound component in the acoustic signal SA (t). Is a non-negative matrix.

  As shown in FIG. 3, the base matrix H arranges R basis vectors h [1] to h [R] corresponding to the components constituting the non-target sound components of the acoustic signal SA (t) in the horizontal direction. This is a non-negative matrix of M rows and R columns. The base vector h [r] of the r-th column (r = 1 to R) of the base matrix H is the amplitude of the r-th component among the R components constituting the non-target sound component of the acoustic signal SA (t). Corresponds to the spectrum. That is, the mth row element of the basis vector h [r] is the mth frequency on the frequency axis in the amplitude spectrum of the rth component constituting the non-target sound component of the acoustic signal SA (t). Means the amplitude value. The difference between the number of columns K of the base matrix B and the number of columns R of the base matrix H is not questioned.

  As shown in FIG. 3, the coefficient matrix U of Equation (2) arranges R coefficient vectors u [1] to u [R] corresponding to each base vector h [r] of the base matrix H in the vertical direction. This is a non-negative matrix of R rows and N columns. The coefficient vector u [r] in the r-th row of the coefficient matrix G corresponds to a time series of weight values for the base vector h [r] of the base matrix H. That is, the element in the nth column of the coefficient vector u [r] means the magnitude (weight value) of the basis vector h [r] in the nth frame among the N frames of the acoustic signal SA (t). To do. Therefore, as described above, the matrix HU of the mathematical formula (2) obtained by multiplying the base matrix H and the coefficient matrix G represents the amplitude spectrogram of the non-target sound component in the acoustic signal SA (t). Note that the non-target sound component can include a plurality of acoustic components that are generated from different sound sources (a sound source different from the sound source of the target sound component).

The matrix analysis unit 34 shown in FIG. 1 adds the target sound component matrix (F + D) G and the non-target sound component matrix HU to the observation matrix Y of the acoustic signal SA (t) as shown in Equation (2). Non-negative matrix factorization using the observation matrix Y generated by the frequency analysis unit 32 from the acoustic signal SA (t) and the known teacher matrix F so as to approximate (that is, the difference between the two is minimized). A compensation matrix D, a coefficient matrix G, a base matrix H, and a coefficient matrix U are calculated. In the non-negative matrix factorization of the first embodiment, the following constraint conditions C1 to C4 are introduced that the degree of similarity between the matrices decreases (ideally minimized).
C1: The similarity between the teacher matrix F and the compensation matrix D decreases.
C2: The similarity between the teacher matrix F and the base matrix H decreases.
C3: The degree of similarity between the compensation matrix D and the basis matrix H decreases.
C4: The similarity between the base matrix B (B = F + D) and the base matrix H decreases.

The constraint condition C1 is a condition for extracting, as the compensation matrix D, an acoustic component having an acoustic characteristic different from that of the teacher acoustic component (teacher matrix F) among the target sound components. That is, the introduction of the constraint condition C1 avoids the situation in which Equation (2) is established in a state where the teacher matrix F and the compensation matrix D are common (the basis matrix B reflects only the teacher acoustic component). In the first embodiment, the minimization of the correlation λ (F | D) between the teacher matrix F and the compensation matrix D (minimize λ (F | D)) is exemplified as the constraint condition C1. It is suitable as | (D F) correlation matrix F ^T D of a teacher matrix F and the compensation matrix D (symbol T is meant the matrix transpose) Frobenius norm ‖F ^T D‖ _Fr of correlation lambda.

The constraint condition C2 is a condition for extracting a non-target sound component having an acoustic characteristic different from that of the teacher acoustic component expressed by the teacher matrix F as the base matrix H. In other words, the introduction of the constraint condition C2 avoids the situation where the mathematical formula (2) is established in a state where the teacher matrix F and the base matrix H are in common. In the first embodiment, the correlation lambda of teacher matrix F and the basis matrix H illustrated as minimizing the constraint C2 (F | | H) ( H) = ‖F T H‖ Fr example lambda (F).

The constraint condition C3 is a condition for making the acoustic characteristics different between the acoustic component expressed by the compensation matrix D (difference between the target sound component and the teacher acoustic component) and the non-target sound component expressed by the base matrix H. . That is, by introducing the constraint condition C3, a situation in which the formula (2) is established in a state where the compensation matrix D and the base matrix H are common is avoided. In the first embodiment, the correlation between the compensation matrix D and the basis matrix H lambda illustrate the minimization as constraints C3 (D | | H) ( H) = ‖F T H‖ Fr example lambda (D).

The constraint condition C4 is a condition for making the acoustic characteristics different between the target sound component expressed by the base matrix B (B = F + D) and the non-target sound component expressed by the base matrix H. That is, by introducing the constraint condition C4, a situation in which Equation (2) is established in a state where the base matrix B and the base matrix H are in common (a state where the target sound component and the non-target sound component cannot be distinguished) is avoided. In the first embodiment, the correlation lambda between base matrix B (B = F + D) and the basis matrix H smallest H) = ‖ _{^{(F + D) T H‖ Fr}} | (F + D | H) ( e.g., lambda (F + D Is exemplified as the constraint condition C4.

In order to evaluate the success or failure of the formula (2) under the constraint conditions C1 to C4 described above, an evaluation function Z of the following formula (3) is introduced.
The symbol δ (Y | (F + D) G + HU) in Equation (3) means the distance between the observation matrix Y and the matrix {(F + D) G + HU}. For example, a generalized KL (Kullback-Leibler) pseudorange (I-divergence) is suitable as the distance δ (Y | (F + D) G + HU). The coefficients μ1 to μ4 in the formula (3) are coefficients for mutually matching the value ranges of the respective terms (correlation λ) in the formula (3), and are predetermined non-negative values (μ1, μ2, μ3, μ4 ≧ 0). Set to

The non-negative matrix factorization by the matrix analysis unit 34 of the first embodiment is a process for minimizing the evaluation function Z of the expression (3) (that is, a process for establishing the expression (2) under the constraint conditions C1 to C4). ). From the condition that the evaluation function Z is minimized, the following equation (7) is derived from the following equation (4).

Equation (4) is an update equation that sequentially updates the element D _mk (D _mk ≧ 0) of the m-th row and the k-th column of the compensation matrix D (M rows and K columns). This is an update formula for sequentially updating the element G _kn (G _kn ≧ 0) of the k-th row and the n-th column of the matrix G (K rows and N columns). Equation (6) is an update equation that sequentially updates the element H _mr (H _mr ≧ 0) of the m-th row and the r-th column of the base matrix H (M rows and R columns). This is an update equation for sequentially updating the element U _rn (U _rn ≧ 0) of the r-th row and the n-th column of the matrix U (R rows and N columns).

1 sets the initial value of the unknown matrix (D, G, H, U) in the evaluation function Z to a random number, and then repeats the operations of Equation (4) to Equation (7). The calculation results (D _mk , G _kn , H _mr , U _rn ) when the number of iterations reaches a predetermined number are determined as a compensation matrix D, a coefficient matrix G, a base matrix H, and a coefficient matrix U. The number of iterations of the update formula is selected experimentally or statistically so that the evaluation function Z converges to a predetermined value (for example, zero). As can be understood from the above description, the matrix analysis unit 34 of the first embodiment uses the mathematical expression (1) under the constraint conditions C1 to C4 with respect to the observation matrix Y of the acoustic signal SA (t) and the known teacher matrix F. A compensation matrix D, a coefficient matrix G, a base matrix H, and a coefficient matrix U are generated so that the relationship 2) is established.

  The sound source separation unit 36 in FIG. 1 generates an acoustic signal SB (t) using the analysis result (D, G, H, U) by the matrix analysis unit 34. Specifically, the sound source separation unit 36 adds the coefficient matrix G calculated by the matrix analysis unit 34 to the base matrix B obtained by adding the teacher matrix F stored in the storage device 24 and the compensation matrix D calculated by the matrix analysis unit 34. To calculate the amplitude spectrogram ((F + D) G) of the target sound component in the acoustic signal SA (t), and the amplitude spectrum of each frame and the phase spectrum of the acoustic signal SA (t) in that frame The time domain acoustic signal SB (t) is generated by inverse Fourier transform using the above. The acoustic signal SB (t) generated by the sound source separation unit 36 is supplied to the sound emitting device 14 and reproduced as a sound wave.

  In the first embodiment described above, non-negative matrix factorization is performed by applying an unknown compensation matrix D having a common coefficient matrix G with a known teacher matrix F that expresses a teacher acoustic component. There is an advantage that the target sound component can be separated with high accuracy even when the acoustic characteristics of the target sound component of the acoustic signal SA (t) do not completely match the teacher acoustic component. In the first embodiment, since non-negative matrix factorization is performed under the constraint conditions C1 to C4 regarding the teacher matrix F, the compensation matrix D, and the base matrix H, the target sound component and the non-target sound component are increased. The effect that it can be separated into precision is particularly remarkable.

Second Embodiment
A second embodiment of the present invention will be described below. In addition, about the element in which an effect | action and a function are the same as that of 1st Embodiment in each aspect illustrated below, the detailed description of each is abbreviate | omitted suitably using the code | symbol referred by description of 1st Embodiment.

In the first embodiment, the compensation matrix D is assumed to be a non-negative matrix (D _mk ≧ 0). As described with reference to FIG. 4, the amplitude spectrum Ab (basic vector y [n]) of the target sound component expressed by the base vector b [k] of the base matrix B is the base vector f [k] of the teacher matrix F. ] Is a relationship obtained by adding a positive number (basic vector d [f]) to the amplitude spectrum Af of the teacher acoustic component expressed by the above equation (that is, the amplitude value of the amplitude spectrum Ab exceeds the amplitude spectrum Af over all frequencies). In the first case, the target sound component can be separated with high accuracy even in the first embodiment in which the compensation matrix D is a non-negative matrix. However, as illustrated in FIG. 5, when the amplitude value of the amplitude spectrum Ab of the target sound component of the acoustic signal SA (t) is lower than the amplitude spectrum Af of the teacher matrix F, the base vector f [ k] and the basis vector d [k] of the non-negative compensation matrix D cannot appropriately represent the amplitude spectrum Ab of the target sound component.

In order to solve the above problems, in the second embodiment, the condition of the first embodiment that the compensation matrix D is a non-negative matrix is canceled. That is, the compensation matrix D is not limited to a non-negative matrix, and each element D _mk of the compensation matrix D can be set to a negative number as well as a non-negative number (positive number and zero). Therefore, at the frequency where the amplitude value of the amplitude spectrum Ab of the target sound component of the acoustic signal SA (t) is lower than the amplitude spectrum Af of the teacher acoustic component, the amplitude value of the amplitude spectrum Ad of the compensation matrix D (the basis vector d [k] If each element) is set to a negative number, it is possible to appropriately represent the amplitude spectrum Ab of the target sound component by adding the base vector f [k] of the teacher matrix F and the base vector d [k] of the compensation matrix D. It is. As understood from the above description, the positive number in the basis vector d [k] of the compensation matrix D is added to the amplitude spectrum Af indicated by the basis vector f [k] of the teacher matrix F to add the target sound component. The negative number in the basis vector d [k] functions as an element approximated to the amplitude spectrum Ab by removing the component of the amplitude spectrum Af.

However, the non-negative matrix factorization executed by the matrix analysis unit 34 is premised on the condition that the base matrix (B, H) and the coefficient matrix (G, U) are non-negative matrices. Therefore, in the second embodiment, a constraint condition that the element D _mk of the compensation matrix D is allowed to be set to a negative number, while the base matrix B obtained by adding the teacher matrix F and the compensation matrix D is a non-negative matrix. C5 is added to the same constraint conditions C1 to C4 as in the first embodiment. Specifically, the addition value (B) of the element F _mk (non-negative value) in the m-th row and k-th column of the teacher matrix F and the element D _mk (non-negative number or negative number) in the m-th row and k-th column of the compensation matrix D. A constraint C5 is introduced that _mk ) is non-negative. For example, in the second embodiment, the constraint condition C5 expressed by the following formula (8) is applied.

The coefficient η in Equation (8) is a coefficient for adjusting a range of values allowed for the element B _mk of the base matrix B (the degree to which the element B _mk approaches a negative number), and is a positive number of 1 or less (0 <η ≦ 1). For example, the coefficient η is preferably set to about 0.3. From the condition that the evaluation function Z of Equation (3) is minimized under the constraint condition C5 of Equation (8), the unknown elements (D _mk , G _kn , H _mr , U _rn ) Equation (12) is derived from Equation (9) below for updating sequentially. Note that the symbol E in the equations (9) to (12) means a matrix of M rows and N columns in which all elements are 1. The operator () p means an operator that maintains positive elements of the matrix in parentheses and replaces negative elements with zero, and the operator () n An operator that maintains negative elements and replaces positive elements with zeros. The operator .- means division for each element of the matrix.

  The matrix analysis unit 34 repeats the operations of the formulas (9) to (12) instead of the formulas (4) to (7) of the first embodiment, thereby performing the compensation matrix D, the coefficient matrix G, and the basis matrix H. And the coefficient matrix U are calculated. In other words, the matrix analysis unit 34 of the second embodiment establishes the relationship of the formula (2) with respect to the observation matrix Y of the acoustic signal SA (t) and the known teacher matrix F under the constraint conditions C1 to C5. Thus, the compensation matrix D, the coefficient matrix G, the base matrix H, and the coefficient matrix U are generated.

In the second embodiment, the same effect as in the first embodiment is realized. In the second embodiment, the compensation matrix D is not limited to a non-negative matrix (the element D _mk of the compensation matrix D can be set not only to a non-negative number but also to a negative number), so that the target sound of the acoustic signal SA (t) is obtained. There is an advantage that the target sound component can be separated with high accuracy even when the amplitude value of the amplitude spectrum Ab of the component is lower than the amplitude spectrum Af of the teacher acoustic component. On the other hand, since the base matrix B obtained by adding the teacher matrix F and the compensation matrix D is limited to a non-negative matrix (constraint condition C5), the matrix analysis unit is not limited to the non-negative matrix. The non-negative matrix factorization by 34 is appropriately executed as in the first embodiment.

  FIG. 6 is a result of an experiment of sound source separation according to the second embodiment. In the experiment of FIG. 6, a teacher matrix F is generated using a mixed sound of flute and clarinet musical sounds generated by a MIDI (Musical Instrument Digital Interface) sound source as a teacher acoustic component, and a mixed sound of natural musical instrument flute and clarinet musical sounds is generated. The flute music was extracted as the target sound component from the recorded acoustic signal SA (t). 6 is a configuration in which the compensation matrix D is not used (the compensation matrix D is excluded from the mathematical expression (2) of the first embodiment and the constraint condition C1, the constraint condition C3, and the constraint condition C4 are not considered. Configuration). In FIG. 6, a signal to distortion ratio (SDR), a signal to interference ratio (SIR), and a non-linear distortion (SAR: Sources to Artifacts Ratio) are the second evaluation criteria for separation results. It describes about each of embodiment and contrast. The signal-to-distortion ratio is a measure of the accuracy of sound source separation and the quality of the separated signal, the signal-to-interference ratio is a measure of only the accuracy of sound source separation, and the nonlinear distortion is an evaluation of signal distortion before and after sound source separation. It is a scale. It can be confirmed from the experimental results of FIG. 6 that good sound source separation can be realized by the second embodiment in terms of both the accuracy of sound source separation and the quality of the separated signal.

  FIG. 7 is a spectrogram showing the result of sound source separation according to the second embodiment and comparison. The flute musical sound was extracted as the target sound component from the acoustic signal SA (t) (spectrogram of part (A) in FIG. 7) containing the mixed sound of the flute and clarinet musical sounds. Part (B) of FIG. 7 shows a spectrogram (that is, an ideal result of sound source separation) of an acoustic signal SA (t) of a musical tone played by a flute alone. A part (C) of FIG. 7 is a spectrogram of the acoustic signal SB (t) generated in the second embodiment, and a part (D) of FIG. 7 is an acoustic signal SB (t) generated in proportion. Spectrogram. The part (C) in FIG. 7 approximates the part (B) as compared with the part (D) in FIG. That is, according to the second embodiment, it can be confirmed from FIG. 7 that sound source separation with higher accuracy can be realized as compared with the comparative example.

<Modification>
Each of the above forms can be variously modified. Specific modifications are exemplified below. Two or more aspects arbitrarily selected from the following examples can be appropriately combined.

(1) In each of the above embodiments, the teacher matrix F is generated by non-negative matrix factorization with respect to the observation matrix X of the teacher acoustic component (FIG. 2), but the method of generating the teacher matrix F is arbitrary. Since the teacher matrix F is composed of K amplitude spectra assumed for the teacher sound component, for example, an average amplitude spectrum is calculated for each of the K pitches of the teacher sound component, and each average after the average is calculated. It is also possible to generate the teacher matrix F by arranging K amplitudes with the amplitude spectrum as the basis vector f [k]. That is, any technique for specifying the acoustic amplitude spectrum is applied to the generation of the teacher matrix F.

(2) In each of the above-mentioned forms, paying attention to the correlation λ ((λ (F | D), λ (F | H), λ (D | H), λ (F + D | H)) between the matrices. Constraint conditions C1 to C4 were defined, but focusing on the distance δ (δ (F | D), δ (F | H), δ (D | H), δ (F + D | H)) between the matrices. It is also possible to define the constraint conditions C1 to C4, for example, the constraint condition C1 corresponds to maximization of the distance δ (F | D) between the teacher matrix F and the compensation matrix D. This corresponds to maximization of the distance δ (F | H) between the teacher matrix F and the base matrix H, and the constraint condition C3 corresponds to maximization of the distance δ (D | H) between the compensation matrix D and the base matrix H. The constraint condition C4 corresponds to the maximization of the distance δ (B | H) between the base matrix B and the base matrix H. For example, the generalized KL pseudorange, the Itakura-Saito distance, and β- A well-known distance criterion such as divergence is arbitrarily adopted as the distance δ.When attention is paid to the distance δ between the matrices, the evaluation function Z in the following equation (13) is the highest. Of the elements of each unknown matrix as _{(D mk, G kn, H} mr, U rn) is calculated.

  As can be understood from the above description, the constraint condition that the similarity between the matrices decreases (ideally minimized) is that the correlation λ between the matrices decreases (ideally minimizes). And the condition that the distance δ between the matrices is increased (ideally maximized).

(3) In each of the embodiments described above, the acoustic signal SB (t) is generated by extracting the target sound component of the acoustic signal SA (t) (suppressing the non-target sound component), but the target sound of the acoustic signal SA (t) is generated. It is also possible to generate an acoustic signal SB (t) with components suppressed (extracting non-target sound components). For example, the sound source separation unit 36 multiplies the base matrix H calculated by the matrix analysis unit 34 and the coefficient matrix U to generate the acoustic signal SB (t) from which the non-target sound component of the acoustic signal SA (t) is extracted. Is done. As understood from the above description, the sound source separation unit 36 is included as an element that separates (extracts or suppresses) one of the target sound component and the non-target sound component using the analysis result of the matrix analysis unit 34.

  The method by which the sound source separation unit 36 generates the acoustic signal SB (t) using the analysis result by the matrix analysis unit 34 is not limited to the above example. For example, by multiplying the base matrix H and the coefficient matrix U, a matrix E (hereinafter referred to as “estimated noise matrix”) E representing the amplitude spectrogram of the non-target sound component (noise component) is calculated, and the acoustic signal SA (t It is also possible to generate the acoustic signal SB (t) by suppressing the estimated noise matrix E from the observation matrix Y of FIG. For the suppression of the estimated noise matrix E, a known technique (for example, spectral subtraction, Wiener filter, MMSE-STSA, etc.) for suppressing the noise component (estimated noise matrix E) from the acoustic signal SA (t) (observation matrix Y) is arbitrary. Adopted.

(4) In each of the above-described embodiments, the teacher matrix F of the teacher sound component generated by one type of sound source is illustrated. However, the mixed sound of sounds generated by a plurality of different types (J) of sound sources is used as the teacher sound component. It is also possible to generate a teacher matrix F as follows. As shown in FIG. 8, the teacher matrix F is a large matrix in which J matrices F [1] to F [J] corresponding to sound sources having different teacher acoustic components are arranged in the horizontal direction. One matrix F [j] (j = 1 to J) horizontally represents one or more basis vectors representing the amplitude spectrum of each component constituting the acoustic component of the jth sound source among the J sound sources. It is a matrix of M rows (arbitrary number of columns) arranged in the direction. The compensation matrix D is a matrix in which J matrices D [1] to D [J] corresponding to different matrices F [j] are arranged in the horizontal direction. The number of rows and columns of the matrix D [j] is the same as that of the matrix F [j]. The method (update formula and constraint condition) for calculating the elements (D _mk , G _kn , H _mr , U _rn ) of each unknown matrix is the same as in each of the above-described embodiments.

  According to the above configuration, the target sound component (matrix BG) obtained by mixing the J acoustic components whose acoustic characteristics (sound source type) approximate the teacher acoustic component among the plurality of acoustic components of the acoustic signal SA (t). Non-target sound components (matrix HU) other than the target sound component are separated. In addition, the coefficient vector g [k] corresponding to the matrix of the coefficient matrix G is multiplied by a matrix obtained by adding the matrix F [j] in the teacher matrix F and the matrix D [j] in the compensation matrix D. Thus, the amplitude spectrogram of the acoustic component of the jth sound source among the J sound sources is calculated. As understood from the above examples, the total number of sound sources of the teacher sound component or the target sound component is arbitrary.

(5) It is also possible to merge acoustic separation using spatial information (acoustic component arrival direction) into the above-described embodiments. For example, an acoustic signal SA (t) of an incoming sound from a specific direction is extracted by analyzing acoustic signals of a plurality of channels recorded by sound collecting devices (microphone arrays) separated from each other, and the extracted acoustic signal SA The sound source separation of each form described above is executed for (t). Known techniques can be arbitrarily employed for sound source separation using spatial information. For example, Shigeki Miyabe, et.al., "Temporal quantization of spatial information using directional clustering for multichannel audio coding", IEEE WASPAA2009, p. 261-264, 2009 is preferred. According to the above structure, there exists an advantage that the target sound component which arrives from a predetermined direction with respect to a sound receiving point (several sound collection apparatus) can be isolate | separated with high precision.

(6) The contents of the calculation that the matrix analysis unit 34 repeatedly executes are not limited to the examples ((4) to (7) and (9) to (12)) in the above embodiments. For example, the above formula (9) can be replaced with the following formula (9A).
That is, in the above formula (9), the sign of each element is determined for each term of the matrix V (2μ ₁ (FF ^T D), 2μ ₃ (HH ^T D)), whereas in the formula (9A) The sign of each element is determined as a whole of the matrix V (that is, after addition of each term).

(7) The constraint condition applied to the non-negative matrix factorization is not limited to the examples of the above-described embodiments. For example, as in the above-described embodiments, the constraint condition C2 (decrease in similarity between the teacher matrix F and the base matrix H) and the constraint condition C3 (decrease in similarity between the compensation matrix D and the base matrix H) are taken into account. In this case, the constraint condition C4 can also be established, so that the constraint condition C4 can be omitted.

(8) In the above-described embodiments, the entire band of the acoustic signal SA (t) is the processing target. However, a specific band of the acoustic signal SA (t) can be selectively processed. If only the band component assumed for the desired sound source in the acoustic signal SA (t) is processed, the separation accuracy of the sound source can be improved.

(9) Generation of the acoustic signal SB (t) using the analysis result by the matrix analysis unit 34 (sound source separation unit 36) can be omitted. For example, the present invention is also implemented as an acoustic analysis apparatus 100 (matrix analysis unit 34) that calculates an unknown matrix (D, G, H, U) by non-negative matrix factorization with respect to the observation matrix Y of the acoustic signal SA (t). Can be done. In addition, the acoustic analysis device 100 can be realized by a server device that communicates with a terminal device such as a mobile phone. For example, the acoustic analysis device 100 generates an acoustic signal SB (t) from the acoustic signal SA (t) received from the terminal device and transmits the acoustic signal SB (t) to the terminal device. In the configuration in which the observation matrix Y of the acoustic signal SA (t) is received from the terminal device (for example, the configuration in which the terminal device includes the frequency analysis unit 32), the frequency analysis unit 32 is omitted from the acoustic analysis device 100, and the matrix analysis unit In the configuration in which the analysis result of 34 is transmitted to the terminal device (for example, the configuration in which the terminal device includes the sound source separation unit 36), the sound source separation unit 36 is omitted from the acoustic analysis device 100.

DESCRIPTION OF SYMBOLS 100 ... Acoustic analysis device, 12 ... Signal supply device, 14 ... Sound emission device, 22 ... Arithmetic processing device, 24 ... Memory | storage device, 32 ... Frequency analysis part, 34 ... Matrix analysis part, 36 ... ... sound source separation unit, F ... teacher matrix, D ... compensation matrix, B, H ... basis matrix, G, U ... coefficient matrix, X, Y ... observation matrix.

Claims (5)

A matrix obtained by adding a compensation matrix to a known teacher matrix including a plurality of basis vectors indicating a spectrum of a teacher acoustic component, the first basis matrix including a plurality of basis vectors indicating a spectrum of a target sound component of an acoustic signal; A first coefficient matrix including a plurality of coefficient vectors indicating a temporal change in a weight value for each basis vector of the first basis matrix; and a plurality of basis vectors indicating a spectrum of a non-target sound component other than the target sound component of the acoustic signal. A matrix that calculates a second basis matrix including the second basis matrix and a second coefficient matrix including a plurality of coefficient vectors indicating a time change of a weight value for each basis vector of the second basis matrix by non-negative matrix factorization for the acoustic signal An acoustic analysis device comprising analysis means. Sound source separation for generating at least one of the target sound component according to the first base matrix and the first coefficient matrix and the non-target sound component according to the second base matrix and the second coefficient matrix The acoustic analysis device according to claim 1, further comprising: The acoustic analysis apparatus according to claim 1, wherein the matrix analysis unit performs the non-negative matrix factorization under a constraint that a similarity between the teacher matrix and the compensation matrix decreases. The acoustic analysis according to any one of claims 1 to 3, wherein the matrix analysis unit performs the non-negative matrix factorization under a constraint that a similarity between the compensation matrix and the second basis matrix decreases. Analysis device. The compensation matrix includes negative elements;
The acoustic analysis apparatus according to claim 1, wherein the matrix analysis unit performs the non-negative matrix factorization under a constraint that the first basis matrix is a non-negative matrix.