JP5516169B2 - Sound processing apparatus and program - Google Patents

Description

  The present invention relates to a technique for suppressing a noise component included in an acoustic signal.

  Conventionally, a technique for suppressing a noise component from an acoustic signal of a mixed sound of a target sound component and a noise component has been proposed. For example, in Patent Document 1, a component having the minimum intensity is selected from a low-frequency component and an average component of each low-frequency component in each of a plurality of acoustic signals, and synthesized with a high-frequency component of each acoustic signal. A technique for generating a noise suppression signal in which wind noise is suppressed is disclosed.

Japanese Patent No. 4356670

  However, in the technique of Patent Document 1, since the component used for generating the noise suppression signal is selected based on the intensity only, for example, when the intensity of the target sound component is small compared to the wind noise, the target sound component May be removed. In addition, when the average component of multiple acoustic signals is selected as the low-frequency component of the noise suppression signal, the waveform of the target sound component changes significantly during the generation of the noise suppression signal. There is also a problem that it is not reproduced. In view of the above circumstances, an object of the present invention is to suppress a noise component of an acoustic signal with high accuracy.

  Means employed by the present invention to solve the above problems will be described. In order to facilitate understanding of the present invention, in the following description, the correspondence between each element of the present invention and the element of each of the embodiments described later is indicated in parentheses, but the scope of the present invention is not limited to the embodiment. It is not intended to limit the example.

  The acoustic processing apparatus according to the present invention provides an observation matrix (for example, an observation matrix Vi) for each of the first acoustic signal and the second acoustic signal collected in parallel as elements of a time series of component values for each frequency of the acoustic signal. ) In a non-negative matrix factorization, a base matrix (for example, base matrix Wi) including a plurality of bases (for example, base Ci [1] to Ci [K]) indicating component values for different frequencies of the acoustic signal, Matrix decomposition means for generating a coefficient matrix (for example, coefficient matrix Hi) including a plurality of weight series (for example, weight series Ei [1] to Ei [K]) each indicating a time series of the weight values of the respective bases ( For example, a base having a high correlation with the base of the base matrix of the second acoustic signal among a plurality of bases of the base matrix of the first acoustic signal is selected as a noise base corresponding to the noise component of the first acoustic signal. Noise specifying means (for example, noise base Ci_noise) A noise specifying unit 46) and an estimated target sound component (for example, a spectrum YTi of the estimated target sound signal qTi) obtained by suppressing the noise component from the first acoustic signal. Target sound extraction means (for example, target sound extraction unit 52) generated using each of the corresponding weight series, and an estimated noise component (for example, spectrum of the estimated noise signal qNi) obtained by suppressing the target sound component from the first acoustic signal. YNi) using a noise base and a weight sequence (for example, a weight sequence Ei_noise) corresponding to the noise base in the coefficient matrix, and a harmonic structure of the target sound component A harmonic component extracting means (for example, harmonic component extracting unit 64) for extracting a residual component corresponding to (for example, the spectrum Ri of the residual component) from the estimated noise component, and combining the estimated target sound component and the residual component That includes the target sound synthesizing means (e.g. target sound synthesizing unit 66).

  In the above configuration, each observation matrix of the first acoustic signal and the second acoustic signal is decomposed into a base matrix and a coefficient matrix, and the base matrix of the second acoustic signal among the plurality of bases of the base matrix of the first acoustic signal. The noise target having a high correlation with the base of the sound is excluded and the estimated target sound component is extracted. Therefore, even when the intensity of the target sound component of the first acoustic signal is lower than the noise component, the noise component can be suppressed with high accuracy. Also, since the residual component corresponding to the harmonic structure of the target sound component is extracted from the estimated noise component and synthesized with the estimated target sound component, a part of the target sound component (residual component) remains in the estimated noise component However, there is an advantage that the loss of the target sound component can be effectively prevented. In addition, since the residual component is extracted from the estimated noise component using the harmonic structure as a clue, there is an advantage that the residual component can be extracted with high accuracy even when the strength of the residual component is lower than the noise component. The scope of application of the present invention is not limited to a configuration that processes two systems of acoustic signals. That is, even in a configuration for processing three or more systems of acoustic signals, a configuration that satisfies the requirements of the present invention when focusing on two specific systems of acoustic signals is naturally included in the scope of the present invention.

In a preferred aspect of the present invention, the harmonic component extracting means includes frequency estimating means (for example, the frequency estimating unit 72) for estimating the fundamental frequency of the target sound component, and adjusting the frequency of an integral multiple of the fundamental frequency among the estimated noise components. Harmonic coefficient string generating means (for example, harmonic coefficient string generating unit 74) for generating a harmonic coefficient string in which each coefficient value is set so that the wave component is emphasized, and the harmonic coefficient string acting on the estimated noise component And harmonic extraction means (for example, harmonic extraction unit 78) for extracting residual components. In the above aspect, the residual component is extracted from the estimated noise component by applying the harmonic coefficient sequence generated according to the fundamental frequency of the target sound component estimated by the frequency estimating means. Therefore, there is an advantage that an appropriate residual component corresponding to the frequency characteristic (harmonic structure) of the acoustic signal can be extracted.
In a further preferred aspect, the frequency estimation means estimates the fundamental frequency of the estimated target sound component generated by the target sound extraction means. In the above aspect, since the fundamental frequency is estimated for the estimated target sound component in which the noise component is suppressed, the fundamental frequency of the target sound component is increased compared to the case where the fundamental frequency is estimated in a state where the noise component is mixed. There is an advantage that accuracy can be estimated. However, a method of estimating the fundamental frequency of the target sound component for the first sound signal or the second sound signal in which the target sound component and the noise component are mixed may be employed. Note that the method of estimating the fundamental frequency (for example, processing in the frequency domain or time domain) is arbitrary.

  The acoustic processing apparatus according to a preferred aspect of the present invention generates a time series of spectra of the first acoustic signal and the second acoustic signal as an observation matrix based on a first analysis parameter (for example, window width ωA, movement amount δA). First estimated frequency analysis means (for example, frequency analysis unit 42) and second analysis parameters (for example, window width ωB, movement amount δB) that are different from the first analysis parameters. Second frequency analysis means (for example, frequency analysis unit 62) that sequentially generates a spectrum and analysis points (for example, analysis point p2) arranged on the time axis and frequency axis at intervals according to the second analysis parameter. Coefficient sequence correction means (for example, coefficient sequence correction section 76) for generating a correction coefficient sequence (for example, correction coefficient sequence GBi) in which numerical values are set, and the noise extraction means is time-dependent at intervals according to the first analysis parameter. axis And a noise coefficient sequence (for example, noise coefficient sequence GNi) in which coefficient values are set for each analysis point (for example, analysis point p1) arranged on the frequency axis, using a noise base and a weight sequence corresponding to the noise base. And generating the estimated noise component by applying the noise coefficient sequence to the observation matrix, the coefficient sequence correcting means generates a correction coefficient sequence from the noise coefficient sequence, and the harmonic coefficient sequence generating means from the correction coefficient sequence A harmonic coefficient sequence is generated by extracting a component having a frequency that is an integral multiple of the fundamental frequency. In the above aspect, since the noise coefficient sequence used for extracting the estimated noise component is diverted for generation of the harmonic coefficient sequence, it remains as compared with the configuration in which the noise coefficient sequence is not used for generation of the harmonic coefficient sequence. The processing load necessary to extract the components is reduced. Since the noise coefficient sequence corresponding to the first analysis parameter is corrected to the correction coefficient sequence corresponding to the second analysis parameter and applied to the generation of the harmonic coefficient sequence (and the residual component extraction), Even when the first analysis parameter is different from the second analysis parameter, it is possible to generate an appropriate harmonic coefficient sequence used for extraction of residual components. Therefore, for example, the first analysis parameter can be set to a value that is optimal for non-negative matrix factorization, and the second analysis parameter can be set to a value that is optimal for estimation of the fundamental frequency and synthesis of the residual component and the estimated target sound component. It is.

  The acoustic processing apparatus according to a preferred aspect of the present invention is a phase difference calculating means (for example, a phase difference calculating unit 582) that calculates a phase difference (for example, a phase difference ΔP [nA]) between the first acoustic signal and the second acoustic signal. The target sound extraction means includes a target sound coefficient sequence in which each coefficient value is variably set according to the phase difference calculated by the phase difference calculation means, and a base matrix other than the noise base and a coefficient matrix. Are generated from weight sequences corresponding to those other than the noise basis, and are applied to the observation matrix. For example, each coefficient value of the target sound coefficient sequence increases so that the effect of noise suppression by the target sound coefficient sequence increases as the phase difference between the first sound signal and the second sound signal increases (the noise component is dominant). It is set according to the phase difference. In the above aspect, since the phase difference between the first acoustic signal and the second acoustic signal is reflected in the target sound coefficient sequence, the noise component is sufficiently compared with the configuration in which the phase difference is not reflected in the target sound coefficient sequence. There is an advantage that a suppressed estimated target sound component can be generated. A configuration in which the phase difference calculated by the phase difference calculating means is reflected in the noise coefficient sequence can also be employed. That is, the noise extraction means calculates a noise coefficient sequence in which each coefficient value is variably set according to the phase difference between the first acoustic signal and the second acoustic signal from the noise base and the weight sequence corresponding to the noise base. Generate and act on the observation matrix.

  The sound processing apparatus according to a preferred aspect of the present invention is an intensity difference calculating means (for example, an intensity difference calculating unit 584) for calculating an intensity difference (for example, an intensity difference ΔA [nA]) between the first acoustic signal and the second acoustic signal. And the target sound extraction means outputs the target sound coefficient sequence in which each coefficient value is variably set according to the intensity difference (for example, amplitude difference or power difference) calculated by the intensity difference calculation means, from the base matrix. It is generated from each basis other than the basis and a weight sequence other than the noise basis among the coefficient matrices and is applied to the observation matrix. For example, each coefficient value of the target sound coefficient sequence increases so that the effect of noise suppression by the target sound coefficient sequence increases as the intensity difference between the first acoustic signal and the second acoustic signal increases (the noise component is dominant). It is set according to the intensity difference. In the above embodiment, since the difference in intensity between the first acoustic signal and the second acoustic signal is reflected in the target sound coefficient sequence, the noise component is sufficiently compared with the configuration in which the intensity difference is not reflected in the target sound coefficient sequence. There is an advantage that a suppressed estimated target sound component can be generated. Note that a configuration in which the intensity difference calculated by the intensity difference calculating means is reflected in the noise coefficient sequence may be employed. That is, the noise extraction means calculates a noise coefficient sequence in which each coefficient value is variably set according to the intensity difference between the first acoustic signal and the second acoustic signal from the noise base and the weight sequence corresponding to the noise base. Generate and act on the observation matrix.

  The acoustic processing device according to each of the above aspects is realized by hardware (electronic circuit) such as a DSP (Digital Signal Processor) dedicated to processing of an acoustic signal, or a general-purpose calculation such as a CPU (Central Processing Unit). This is also realized by cooperation between the processing device and the program. The program according to the present invention is a non-negative matrix factorization of an observation matrix having a time series of component values for each frequency of the acoustic signal for each of the first acoustic signal and the second acoustic signal collected in parallel. Generating a base matrix including a plurality of bases indicating component values for different frequencies of different components of the acoustic signal, and a coefficient matrix including a plurality of weight sequences each indicating a time series of weight values of the respective bases A base having a high correlation between the matrix decomposition process and the base of the base matrix of the second acoustic signal among the plurality of bases of the base matrix of the first acoustic signal is specified as a noise base corresponding to the noise component of the first acoustic signal. Generates estimated target sound component with noise component suppressed from first acoustic signal using noise identification processing and each weight sequence corresponding to non-noise basis among coefficient matrix and coefficient matrix Target sound extraction processing , A noise extraction process for generating an estimated noise component obtained by suppressing the target sound component from the first acoustic signal using a noise base and a weight sequence corresponding to the noise base in the coefficient matrix, and a harmonic structure of the target sound component The computer executes a harmonic component extraction process for extracting a residual component corresponding to the estimated noise component from the estimated noise component and a target sound synthesis process for synthesizing the estimated target sound component and the residual component. According to the above program, the same operation and effect as the sound processing apparatus according to the present invention are realized. The program of the present invention is provided to the user in a form stored in a computer-readable recording medium and installed in the computer, or is provided from the server device in the form of distribution via a communication network. Installed on.

1 is a block diagram of a sound processing apparatus according to a first embodiment of the present invention. It is a block diagram of a 1st processing part. It is explanatory drawing of an observation matrix. It is explanatory drawing of a base matrix and a coefficient matrix. It is a block diagram of a target sound extraction part and a noise extraction part. It is a block diagram of a 2nd processing part. It is explanatory drawing of the analysis point assumed by the process of a 2nd process part. It is a block diagram of a harmonic component extraction unit. It is explanatory drawing of operation | movement of a harmonic component extraction part. It is a block diagram of the 1st processing part in a 2nd embodiment. It is a block diagram of the 2nd processing part in a 3rd embodiment. It is a block diagram of the harmonic component extraction part in 4th Embodiment.

<A: First Embodiment>
FIG. 1 is a block diagram of a sound processing apparatus 100 according to the first embodiment of the present invention. As shown in FIG. 1, a signal supply device 12 and a sound emitting device 14 are connected to the sound processing device 100. The signal supply device 12 supplies the sound processing device 100 with the stereo-type sound signal s1 and sound signal s2 collected in parallel (simultaneously) at different positions. Each acoustic signal si (i = 1, 2) is a time domain signal representing a sound pressure waveform of a mixed sound of a target sound component and a noise component. In FIG. 1, a plurality of sound collection devices 122 (for example, omnidirectional microphones) that are arranged apart from each other are illustrated as the signal supply device 12. However, a playback device that acquires each acoustic signal si from a portable or built-in recording medium and supplies it to the sound processing device 100, or a communication device that receives each acoustic signal si from a communication network and supplies it to the sound processing device 100. Can also be employed as the signal supply device 12.

  The sound processing apparatus 100 generates a stereo sound signal q1 and a sound signal q2 from the sound signal s1 and the sound signal s2. Each acoustic signal qi is a time domain signal obtained by suppressing a noise component (emphasizing a target sound component) from the acoustic signal si. The sound emitting device 14 (for example, a stereo speaker or stereo headphones) radiates sound waves corresponding to the acoustic signal q1 and the acoustic signal q2 generated by the acoustic processing device 100. Note that an A / D converter that converts the acoustic signal si from analog to digital and a D / A converter that converts the acoustic signal qi from digital to analog are not shown for convenience.

  As shown in FIG. 1, the sound processing device 100 is realized by a computer system including an arithmetic processing device 22 and a storage device 24. The storage device 24 stores a program executed by the arithmetic processing device 22 and various data used by the arithmetic processing device 22. A known recording medium such as a semiconductor recording medium or a magnetic recording medium or a combination of a plurality of types of recording media can be arbitrarily employed as the storage device 24. A configuration in which the acoustic signal s1 and the acoustic signal s2 are stored in the storage device 24 (therefore, the signal supply device 12 can be omitted) is also suitable.

  The arithmetic processing unit 22 implements a plurality of functions (a first processing unit 31 and a second processing unit 32) for generating the acoustic signal qi from the acoustic signal si by executing a program stored in the storage device 24. A configuration in which each function of the arithmetic processing unit 22 is distributed over a plurality of integrated circuits, or a configuration in which a dedicated electronic circuit (DSP) realizes each function may be employed.

  The first processing unit 31 in FIG. 1 emphasizes the target sound component (the noise component is suppressed) in the stereo form of the estimated target sound signal qT1 and the estimated target sound signal qT2 (T: target) and the noise component (the target sound). A stereo-type estimated noise signal qN1 and an estimated noise signal qN2 (N: noise) with components suppressed) are generated from the acoustic signal s1 and the acoustic signal s2. That is, the acoustic signal si is separated into a target sound component (estimated target sound signal qTi) and a noise component (estimated noise signal qNi). However, since it is difficult to completely separate the target sound component and the noise component, a part of the target sound component (hereinafter referred to as “residual component”) to be originally selected for the estimated target sound signal qTi is estimated noise. The signal qNi may be mixed. Therefore, the second processing unit 32 generates the acoustic signal qi (q1, q2) by extracting the residual component from the estimated noise signal qNi and synthesizing it with the estimated target sound signal qTi.

  FIG. 2 is a block diagram of the first processing unit 31. As shown in FIG. 2, the first processing unit 31 includes a frequency analysis unit 42, a matrix decomposition unit 44, a noise identification unit 46, a target sound extraction unit 52, a noise extraction unit 54, and a waveform synthesis unit 56. The

  As shown in FIG. 3, the frequency analysis unit 42 sequentially generates a spectrum Si (S1, S2) of each acoustic signal si for each unit period (frame) on the time axis. The spectrum Si of each unit period is a power spectrum in which a plurality of component values (power) xi corresponding to different frequencies (f1, f2,..., FMA,...) On the frequency axis are arranged. That is, as shown in FIG. 3, at time points t (t1, t2,...) Arranged at intervals ΔtA on the time axis and frequencies f (f1, f2,...) Arranged at intervals ΔfA on the frequency axis. Correspondingly, component values xi are calculated for each analysis point (grid) p1 arranged in a matrix on the time-frequency plane.

  The generation of each spectrum Si employs short-time Fourier transform in which the window width (frame length) ωA and movement amount (shift amount on the time axis) δA of the unit period are applied as analysis parameters. The interval ΔtA on the time axis and the interval ΔfA on the frequency axis of each analysis point p1 are variably set according to the analysis parameters (window width ωA, movement amount δA) of frequency analysis by the frequency analysis unit 42.

  As shown in FIG. 3, the spectrum Si of each acoustic signal si is divided into a spectrum Xi in the band BLa and a spectrum XHi in the band BHa. The band BLa is set so as to include the frequency of the noise component. In the present embodiment, wind noise is assumed as a noise component. Wind noise is a noise component generated when air itself flows and directly collides with the diaphragm of the sound collecting device 122. The vibration frequency of the diaphragm due to the air collision is lower than the frequency of the sound wave propagating to the diaphragm as air vibration (sound pressure change). Specifically, the frequency of the wind noise is predominantly a low frequency component of 1 kHz or less, for example. Considering the above tendency, the band BLa is set to a range of 1 kHz or less including MA (MA is a natural number) frequencies f1 to fMA. The band BHa is a band on the high frequency side (for example, 1 kHz or more) compared to the band BLa.

  As shown in FIG. 3, the time series (spectrogram) of the spectrum Si generated by the frequency analysis unit 42 is divided on the time axis for every analysis period T0 including NA time points t1 to tNA. The analysis period T0 is set to a long time of about several tens of seconds, for example. As shown in FIG. 3, the component values xi [1,1] to xi [at the analysis point p1 corresponding to the MA frequencies f1 to fMA in the band BLa and the NA time points t1 to tNA within the analysis period T0. MA, NA] is defined for each analysis period T0 for each of the acoustic signal s1 and acoustic signal s2. The component value xi [mA, nA] is the mAth (mA = 1 to MA) frequency fmA of the MA frequencies f1 to fMA in the band BLa and the NA time points t1 to tNA within the analysis period T0. Means the component value xi of the analysis point p1 corresponding to the nAth (nA = 1 to NA) time point tnA.

  As can be understood from the above description, the nAth column of the observation matrix Vi represents the MA component values xi [1, nA] to xi [MA, of the spectrum Xi at the nAth time point tnA within the analysis period T0. nA] series, and the mAth row of the observation matrix Vi represents the component values xi [mA, t1] to xi [mA, NA] of the frequency fmA over the NA time points t1 to tNA in the analysis period T0. Corresponds to the series. Since the component value xi [mA, nA] of the spectrum Xi means power (non-negative value), the observation matrix Vi is a non-negative matrix (matrix not including negative numbers). A configuration in which the spectrum Si (Xi) is an amplitude spectrum can also be employed.

  The matrix decomposition unit 44 in FIG. 2 generates a base matrix Wi (W1, W2) and a coefficient matrix Hi (H1, H2) by non-negative matrix factorization (NMF) of each observation matrix Vi. As shown in FIG. 4, the base matrix Wi is a non-negative matrix of MA rows × K columns in which component values wi [1,1] to wi [MA, K] are arranged, and the coefficient matrix Hi has a weight value hi [ 1,1] to hi [K, NA] is a non-negative matrix of K rows × NA columns (K is a natural number). The base matrix Wi and the coefficient matrix Hi are generated so that the product of the base matrix Wi and the coefficient matrix Hi approximates the observation matrix Vi (Vi≈Wi · Hi). The analysis parameters (window width ωA, movement amount δA) applied by the frequency analysis unit 42 are set to numerical values at which the non-negative matrix factorization of the observation matrix Vi can be appropriately performed.

  As shown in FIG. 4, the base matrix Wi is composed of K codebooks Ci [1] to Ci [K]. The basis Ci [k] of the k-th column (k = 1 to K) has frequencies f1 to fMA for one type of acoustic component selected from the K types of acoustic components constituting the acoustic signal si within the analysis period T0. Corresponds to a power spectrum in which component values wi [1, k] to wi [MA, k] are arranged. On the other hand, the coefficient matrix Hi is composed of K weight sequences (excitation) Ei [1] to Ei [K] as shown in FIG. The weight sequence Ei [k] in the k-th row is a time series of weight values hi [k, 1] to hi [k, NA] per unit period for the acoustic component indicated by the basis Ci [k] of the basis matrix Wi ( Corresponding to each component value wi [mA, k] of the base Ci [k]. As understood from the above definition, the spectrum Xi of the acoustic signal si at the time point tnA is the K weight values hi [1, nA] to hi [K, nA corresponding to the time point tnA in the coefficient matrix Hi. ] Is approximated by a weighted sum of K bases Ci [1] to Ci [K] (Xi≈hi [1, nA] × Ci [1] + hi [2, nA] × Ci [2] + ...... + hi [K, nA] × Ci [K]).

  A known method is arbitrarily employed for non-negative matrix factorization of the observation matrix Vi. For example, a method of sequentially updating (iteratively calculating) the base matrix Wi and the coefficient matrix Hi so as to minimize the difference (for example, distance) between the product of the base matrix Wi and the coefficient matrix Hi and the observation matrix Vi is preferable. Adopted. The initial value of the base matrix Wi applied to the iterative calculation (the initial value of the component value w i [mA, k]) is set to a random number, for example. For example, the initial values of the MA component values wi [1, k] to wi [MA, k] of each basis Ci [k] are simulated so as to simulate the spectrum of wind noise (frequency characteristics that attenuate as the frequency increases). The set configuration is also suitable.

  The noise specifying unit 46 shown in FIG. 2 has one basis Ci [k] (hereinafter referred to as “wind noise”) corresponding to a noise component (wind noise) among the K bases Ci [1] to Ci [K] of each basis matrix Wi. Specified as “noise base Ci_noise”. Since wind noise is generated due to the turbulent flow of air colliding with the sound collecting device 122, the instantaneous frequency of the wind noise included in each of the acoustic signals s1 and s2 collected at different positions. The characteristics are statistically independent of each other. However, the long-term frequency characteristics of wind noise are more likely to be maintained at the same characteristics regardless of the position of sound collection, as compared to voice and the like. That is, the frequency characteristics of wind noise over a long period such as the analysis period T0 tend to be similar between the acoustic signal s1 and the acoustic signal s2.

  In consideration of the above tendency, the noise specifying unit 46 determines the basis matrix W1 (K basis C1 [1] to C1 [K]) of the acoustic signal s1 and the basis matrix W2 (K basis C2) of the acoustic signal s2. [1] to C2 [K]), each base Ci [k] (C1 [k1], C2 [k2]) having a high correlation with each other is used as a noise base Ci_noise from each of the base matrix W1 and the base matrix W2. Identify. For example, the correlation between the basis C1 [k] and the basis C2 [k] for all combinations of selecting one basis C1 [k] of the basis matrix W1 and one basis C2 [k] of the basis matrix W2 An index (correlation index) indicating the degree of the correlation is calculated, and each of the combinations of the base C1 [k1] and the base C2 [k2] (the values of the variable k1 and the variable k2) having the maximum degree of correlation indicated by the correlation index The noise base Ci_noise (C1_noise, C2_noise) is extracted. As a correlation index between the base C1 [k] and the base C2 [k], for example, a distance (Euclidean distance) or an inner product is preferably employed.

  The target sound extraction unit 52 in FIG. 2 sequentially generates the spectrum YTi (YT1, YT2) of the estimated target sound signal qTi obtained by extracting the target sound component from the acoustic signal si. The noise extraction unit 54 generates a spectrum YNi (YN1, YN2) of the estimated noise signal qNi obtained by extracting a noise component from the acoustic signal si. FIG. 5 is a block diagram of the target sound extraction unit 52 and the noise extraction unit 54.

  As shown in FIG. 5, the target sound extraction unit 52 includes a coefficient sequence generation unit 522 and an extraction processing unit 524. The coefficient sequence generator 522 generates the target sound coefficient sequence GTi (GT1, GT2) for each analysis period T0. The target sound coefficient sequence GTi is a matrix of MA rows × NA columns in which coefficient values gTi [1,1] to gTi [MA, NA] are arranged. The coefficient value gTi [mA, nA] located in the nAth column of the mAth row of the target sound coefficient sequence GTi is the gain (spectrum) for the component value xi [mA, nA] at the frequency fmA of the spectrum Xi at the time tnA. Is variably set in accordance with the characteristics of the acoustic signal si (wind noise intensity) within a range of 0 or more and 1 or less. That is, the coefficient value gTi [mA, nA] is set to a smaller numerical value as the wind noise becomes dominant in the acoustic component of the frequency fmA of the acoustic signal si at the time point tnA.

  As shown in FIG. 4, the coefficient sequence generator 522 of the first embodiment includes a matrix WTi of MA rows × (K−1) columns excluding the noise basis Ci_noise from the basis matrix Wi of the acoustic signal si, and the noise basis Ci_noise. The target sound coefficient sequence GTi (G1, G2) is generated from the matrix HTi of (K-1) rows × NA columns excluding the weight sequence Ei_noise corresponding to.

  Specifically, the coefficient sequence generation unit 522 first calculates the matrix VTi by multiplying the matrix WTi after the noise basis Ci_noise is excluded and the matrix HTi after the weight sequence Ei_noise is excluded. As shown in FIG. 4, the matrix VTi is a matrix in which element values vTi [1,1] to vTi [MA, NA] are arranged in MA rows × NA columns. As understood from the above description, the MA element values vTi [1, nA] to vTi [MA, nA] located in the nA-th column of the matrix VTi suppress wind noise from the spectrum Xi at the time point tnA. It corresponds to the estimated value of the power spectrum.

Second, the coefficient sequence generation unit 522 calculates the coefficient value gTi [mA, nA] of the target sound coefficient sequence GTi by the calculation of the following formula (A). The symbol v [mA, nA] in equation (A) is the element value of the nth column of the mth row (ie, the spectrum Xi) of the matrix of MA rows × NA columns obtained by multiplying the base matrix Wi and the coefficient matrix Hi. Equivalent value of the component value x i [mA, nA]). The reason why the element value vTi [mA, nA] is divided by the element value v [mA, nA] is to normalize the coefficient value gTi [mA, nA] to a numerical value between 0 and 1. As described above, since the target sound coefficient sequence GTi is generated from the matrix VTi excluding the noise base Ci_noise and the weight sequence Ei_noise of the wind noise, the coefficient value gTi [mA, nA] becomes smaller as the wind noise becomes more dominant. Is set.
gTi [mA, nA] = vTi [mA, nA] / v [mA, nA] (A)

  The extraction processing unit 524 in FIG. 5 applies each of the target sound coefficient sequence GTi generated by the coefficient sequence generation unit 522 to the observation matrix Vi of the acoustic signal si, so that each of the NA points in time t1 to tNA within the analysis period T0. A time series (spectrogram within the analysis period T0) of NA spectra YTi corresponding to is sequentially generated for each analysis period T0. The spectrum YTi at the time tnA is a power spectrum composed of MA component values yTi [1, nA] to yTi [MA, nA]. Specifically, the component value yTi [mA, nA] is set to a product of the coefficient value gTi [mA, nA] of the target sound coefficient sequence GTi and the component value xi [mA, nA] of the observation matrix Vi. (YTi [mA, nA] = gTi [mA, nA] × xi [mA, nA]). As described above, the coefficient value gTi [mA, nA] is set to a smaller value as the wind noise becomes more dominant. Therefore, the spectrum YTi generated by the extraction processing unit 524 suppresses the wind noise from the spectrum Xi of the acoustic signal si. It corresponds to the spectrum.

  The noise extraction unit 54 in FIG. 5, similar to the target sound extraction unit 52, has a noise coefficient sequence GNi (GN 1, GN) of MA rows × NA columns composed of coefficient values gNi [1,1] to gNi [MA, NA]. GN2) is generated every analysis period T0, and extraction processing for generating a time series of NA spectra YNi (a spectrogram within the analysis period T0) by applying the noise coefficient sequence GNi to the observation matrix Vi. Part 544.

As shown in FIG. 4, the coefficient sequence generation unit 542 first multiplies the noise base Ci_noise specified by the noise specification unit 46 and the weight sequence Ei_noise corresponding to the noise base Ci_noise to obtain an element value vNi [1, 1] to vNi [MA, NA] are calculated as a matrix VNi arranged in MA rows × NA columns. The matrix VNi corresponds to the spectrogram of the noise component of the acoustic signal si within the analysis period T0. Second, the coefficient sequence generation unit 542 calculates a coefficient value gNi [mA, nA] that is greater than or equal to 0 and less than or equal to 1 by the calculation of the formula (B) similar to the formula (A) described above. As understood from the above description, the coefficient value gNi [mA, nA] is set to a larger numerical value as the wind noise becomes dominant in the acoustic component of the frequency fmA of the acoustic signal si at the time tnA.
gNi [mA, nA] = vNi [mA, nA] / v [mA, nA] (B)

  The extraction processing unit 544 analyzes the time series (spectrogram) of the NA spectra YNi within the analysis period T0 by applying the noise coefficient sequence GNi generated by the coefficient sequence generation unit 542 to the observation matrix Vi of the acoustic signal si. It generates sequentially for every period T0. The spectrum YNi is a power spectrum composed of MA component values yNi [1, nA] to yNi [MA, nA]. Specifically, the component value yNi [mA, nA] is set to a product of the coefficient value gNi [mA, nA] of the noise coefficient sequence GNi and the component value xi [mA, nA] of the observation matrix Vi ( yNi [mA, nA] = gNi [mA, nA] × xi [mA, nA]). As described above, the coefficient value gNi [mA, nA] is set to a larger value as the noise component (wind noise) becomes more dominant. Therefore, the spectrum YNi generated by the extraction processing unit 544 is derived from the spectrum Xi of the acoustic signal si. It corresponds to the spectrum from which wind noise is extracted.

  As described above, the target sound extraction unit 52 extracts a target sound component from the acoustic signal si, and the noise extraction unit 54 extracts a noise component from the acoustic signal si. That is, the target sound extraction unit 52 and the noise extraction unit 54 function as elements that separate the acoustic signal si into the target sound component (YT1, YT2) and the noise component (YN1, YN2).

  The waveform synthesis unit 56 in FIG. 2 uses the spectrum YTi (band BLa) generated by the target sound extraction unit 52 for each unit period and the spectrum XHi (band BHa) generated by the frequency analysis unit 42 to estimate the target sound signal in the time domain. qTi (qT1, qT2) is generated. Specifically, the waveform synthesizer 56 generates a time domain signal by inverse Fourier transform using the amplitude spectrum of the added value of the spectrum YTi and the spectrum XHi and the phase spectrum of the acoustic signal si, and at the same time before and after the unit period. To generate the estimated target sound signal qTi. Further, the waveform synthesizer 56 generates a time-domain estimated noise signal qNi (qN1, qN2) from the spectrum YNi generated by the noise extraction unit 54 for each unit period.

  As described above, the second processing unit 32 in FIG. 1 extracts a residual component from the estimated noise signal qNi generated by the above procedure and synthesizes it to the estimated target sound signal qTi. In the first embodiment, assuming that the target sound component is a harmonic structure sound (typically speech), the harmonic components (fundamental sound component and harmonic component) constituting the harmonic structure of the estimated noise signal qNi. ) As a residual component from the estimated noise signal qNi. FIG. 6 is a block diagram of the second processing unit 32. As shown in FIG. 6, the second processing unit 32 includes a frequency analysis unit 62, a harmonic component extraction unit 64, a target sound synthesis unit 66, and a waveform synthesis unit 68.

  The frequency analysis unit 62 sequentially generates a spectrum STi (ST1, ST2) of the estimated target sound signal qTi and a spectrum SNi (SN1, SN2) of the estimated noise signal qNi for each unit period. The spectrum STi of the estimated target sound signal qTi is a power spectrum in which a plurality of component values (power) sTi are arranged. Similarly, the spectrum SNi of the estimated noise signal qNi is a power spectrum in which a plurality of component values sNi are arranged. As shown in FIG. 7, the component value xTi and the component value xNi are calculated for each analysis point p2 corresponding to each time point t arranged at intervals ΔtB on the time axis and each frequency f arranged at intervals ΔfB on the frequency axis. Is done. As shown in FIG. 6, the spectrum STi of the estimated target sound signal qTi is divided into a spectrum Zi in the band BLb and a spectrum ZHi in the band BHb. The band BLb is set to a range (for example, a band from 0.1 kHz to 4.4 kHz) including MB (MB is a natural number) frequencies f1 to fMB, and the band BHb is a high band side of the band BLb (for example, 4. 4 kHz or higher band).

  For the calculation of the spectrum STi and the spectrum SNi, a short-time Fourier transform using the window width ωB and the movement amount (shift amount on the time axis) δB in the unit period as analysis parameters is adopted. The analysis parameters (window width ωA, movement amount δA) of the frequency analysis unit 42 are selected from the viewpoint of numerical values suitable for non-negative matrix factorization, whereas the analysis parameters (window width ωB, movement amount δA) of the frequency analysis unit 62 are selected. ) Is selected from the viewpoint of a numerical value suitable for extraction and synthesis of residual components (harmonic components) in the second processing unit 32. Due to the above differences, the analysis parameters (window width ωA, movement amount δA) of the frequency analysis unit 42 and the analysis parameters (window width ωB, movement amount δB) of the frequency analysis unit 62 are different. That is, the interval ΔtA on the time axis of each analysis point p1 assumed by the first processing unit 31 is different from the interval ΔtB of each analysis point p2 assumed by the second processing unit 32. The interval ΔfA on the frequency axis of each analysis point p1 assumed is different from the interval ΔfB of each analysis point p2 assumed in the second processing unit 32. Specifically, each analysis parameter is set so that the frequency analysis unit 62 exceeds the frequency analysis unit 42 (ΔtB <ΔtA) and the frequency resolution exceeds the frequency analysis unit 62 (ΔfA <ΔfB). Is selected. For example, for the frequency analysis unit 42, the window width ωA is set to 512 ms and the movement amount δA is set to 64 ms, whereas for the frequency analysis unit 62, the window width ωB is set to 25 ms and the movement amount δB is 5 ms. Set to

  6 sequentially extracts the residual component spectrum Ri from the noise component estimated noise signal qNi. A time series (spectrogram of residual components) of NB spectra Ri is sequentially generated every analysis period T0.

  FIG. 8 is a block diagram of the harmonic component extraction unit 64. As shown in FIG. 8, the harmonic component extraction unit 64 includes a frequency estimation unit 72, a harmonic coefficient sequence generation unit 74, a coefficient sequence correction unit 76, and a harmonic extraction unit 78. The frequency estimator 72 analyzes the spectrum Zi of the estimated target sound signal qTi and analyzes the fundamental frequency Fi [of the target sound component (estimated target sound signal qTi) of the acoustic signal si for each of the NB unit periods within each analysis period T0. nB] (Fi [1] to Fi [NB]) is estimated. For estimating the fundamental frequency Fi [nB], a known technique (for example, analysis of harmonic structure or calculation of cepstrum) is arbitrarily employed.

  The harmonic coefficient sequence generation unit 74 uses the fundamental frequency Fi [nB] estimated by the frequency estimation unit 72 and the noise coefficient sequence GNi generated by the coefficient sequence generation unit 542 of the first processing unit 31 to generate a harmonic coefficient sequence. GHi (H: harmonics) is generated. As shown in FIG. 9, the harmonic coefficient sequence GHi has an MB row × NB in which coefficient values gHi [1,1] to gHi [MB, NB] corresponding to different analysis points p2 in the time-frequency plane are arranged. A matrix of columns. The MB coefficient values gHi [1, nB] to gHi [MB, nB] constituting the nBth row of the harmonic coefficient row GHi are target sounds in the nBth unit period (time point tnB) in the analysis period T0. It is a coefficient sequence which shows the harmonic structure of a component.

  The generation of the harmonic coefficient string GHi by the harmonic coefficient string generation unit 74 uses the noise coefficient string GNi. The analysis point p1 corresponding to each coefficient value gNi [mA, nA] of the noise coefficient string GNi, This is different from the analysis point p2 corresponding to each coefficient value gHi [mB, nB] of the wave coefficient sequence GHi. In order to compensate for the above difference, the coefficient sequence correction unit 76 corrects the noise coefficient sequence GNi generated by the coefficient sequence generation unit 542 of the first processing unit 31 for each analysis period T0 as shown in FIG. A coefficient sequence GBi is generated. The correction coefficient sequence GBi is a matrix of MB rows × NB columns in which coefficient values gBi [1,1] to gBi [MB, NB] are arranged.

  Specifically, the coefficient sequence correction unit 76 generates each coefficient value gBi [mB, nB] of the correction coefficient sequence GBi by interpolation or thinning of each coefficient value gNi [mA, nA] of the noise coefficient sequence GNi. For example, when the number of rows MA of the noise coefficient sequence GNi exceeds the target number of rows MB (MA> MB), the coefficient sequence correction unit 76 sets the MA coefficient values gNi [1, MB coefficient sequences gBi [1, nA] to gBi [MB, nA] are generated by thinning out (interpolating) nA] to gHi [MA, nA]. When the number NA of noise coefficient columns GNi is less than the target number of columns NB (NA <NB), the coefficient column correction unit 76 uses NA coefficient values gNi [mA, mA constituting each row of the noise coefficient column GNi. NB coefficient sequences gBi [mA, 1] to gBi [mA, NB] are generated by interpolation of 1] to gNi [mA, NA]. A known technique (for example, linear interpolation) is arbitrarily employed for interpolation and thinning.

  As shown in FIG. 8, the harmonic coefficient sequence generation unit 74 includes a harmonic structure specifying unit 742 and a coefficient sequence synthesis unit 744. The harmonic structure specifying unit 742 generates a harmonic coefficient sequence Di indicating the harmonic structure of the target sound component (residual component) of the acoustic signal si. As shown in FIG. 9, the harmonic coefficient sequence Di is a matrix of MB rows × NB columns in which coefficient values di [1,1] to di [MB, NB] are arranged. The numerical sequence of MB coefficient values di [1, nB] to di [MB, nB] constituting the nBth column of the harmonic coefficient sequence Di is a target sound component in the nBth unit period in the analysis period T0. Specifies the harmonic structure of. Specifically, as shown in FIG. 9, among the MB coefficient values di [1, nB] to di [MB, nB], a frequency that is an integral multiple of the fundamental frequency Fi [nB] estimated by the frequency estimation unit 72. The coefficient value di [mB, nB] corresponding to (F0 [nB], 2F0 [nB], 3F0 [nB],...) Is set to 1 and the other coefficient values di [mB, nB] are set to zero. Is set.

  8 generates a harmonic coefficient sequence GHi by combining the correction coefficient sequence GBi generated by the coefficient sequence correction unit 76 and the harmonic coefficient sequence Di generated by the harmonic structure specifying unit 742. Specifically, the coefficient value gHi [mB, nB] of the harmonic coefficient sequence GHi is the coefficient value gBi [mB, nB] of the correction coefficient sequence GBi and the coefficient value of the harmonic coefficient sequence Di as shown in FIG. It is set to the product of di [mB, nB] (gHi [mB, nB] = gBi [mB, nB] × di [mB, nB]). Accordingly, the coefficient value gHi [mB, nB] is set to a larger value as the residual component becomes dominant at a frequency that is an integral multiple of the fundamental frequency Fi [nB].

  The harmonic extraction unit 78 of FIG. 8 applies the harmonic coefficient sequence GHi generated by the coefficient sequence synthesis unit 744 to the spectrum SNi of the estimated noise signal qNi, so that the time series of the NB spectra Ri in the analysis period T0 is obtained. (Spectrogram of residual components) is generated. The spectrum Ri at the time point tnB is a power spectrum composed of MB component values ri [1, nB] to ri [MB, nB] corresponding to the frequencies f1 to fMB. The harmonic extraction unit 78 calculates a product of the component value sNi [mB, nB] of the frequency fmB and the coefficient value gHi [mB, nB] of the harmonic coefficient sequence GHi in the spectrum SNi at the time tnB of the estimated noise signal qNi. It is calculated as the component value ri [mB, nB] of the spectrum Ri (ri [mB, nB] = gHi [mB, nB] × sNi [mB, nB]). Therefore, the spectrum Ri corresponds to the estimated value of the spectrum of the residual component (harmonic component having the frequency Fi [nB] as the fundamental frequency) mixed in the estimated noise signal qNi. The above is the configuration and operation of the harmonic component extraction unit 64.

  The target sound synthesizer 66 shown in FIG. 6 is unitized by combining (spectrum addition) the spectrum Zi (band BLb) generated from the estimated target sound signal qTi by the frequency analyzer 62 and the spectrum Ri generated by the harmonic component extractor 64. A spectrum ZRi is generated sequentially for each period. That is, the spectrum ZRi corresponds to the power spectrum of the mixed sound of the target sound component in the band BLb and the residual component remaining in the estimated noise signal qNi in the estimated target sound signal qTi.

  The waveform synthesizer 68 generates a time-domain acoustic signal qi (q1, q2) from the spectrum ZRi (band BLb) generated by the target sound synthesizer 66 for each unit period and the spectrum ZHi (band BHb) generated by the frequency analyzer 62. ) Is generated. For the generation of the acoustic signal qi by the waveform synthesizer 68, the same method as the generation of the estimated target sound signal qTi by the waveform synthesizer 56 is employed. As can be understood from the above description, the reproduced sound of the acoustic signal qi corresponds to a mixed sound of the target sound component of the estimated target sound signal qTi and the residual component of the estimated noise signal qNi.

  As described above, in the first embodiment, the observation matrix Vi of the acoustic signal si is decomposed into the base matrix Wi and the coefficient matrix Hi, and the base matrix Wi (matrix WTi) excluding the noise base Ci_noise and the weight sequence Ei_noise. The target sound coefficient sequence GTi is generated using the coefficient matrix Hi (matrix HTi) excluding the. Therefore, even when the intensity of the target sound component of the acoustic signal si is lower than that of the noise component, it is possible to suppress wind noise with high accuracy. Further, since each basis Ci [k] other than the noise basis Ci_noise in the basis matrix Wi and each weight series Ei [k] other than the weight series Ei_noise in the coefficient matrix Hi are maintained, the target sound component of the acoustic signal si is maintained. There is also an advantage that an acoustic signal qi can be generated in which the waveform of is maintained faithfully.

  As a method for identifying the noise base Ci_noise from the base matrix Wi, for example, a configuration in which a model created in advance so as to simulate the frequency characteristics of wind noise is compared with each base Ci [k] of the base matrix Wi. Can be employed. However, in a configuration using a wind noise model, wind noise having a frequency characteristic different from that of a model prepared in advance may not be sufficiently suppressed. On the other hand, in the first embodiment, each base Ci [k] having a high correlation between the base matrix W1 and the base matrix W2 is specified as a noise base Ci_noise, so that it is compared with a configuration using a wind noise model. There is an advantage that wind noise with various characteristics can be sufficiently suppressed.

  In the first embodiment, since the target sound component (residual component) remaining in the estimated noise signal qNi is extracted and synthesized with the estimated target sound signal qTi (spectrum Zi), for example, the estimated target sound signal qTi is released. Compared with the case of reproducing from the sound device 14, it is possible to prevent the target sound component from being lost in the reproduced sound. In addition, since the residual component is separated from the noise component in the analysis of the harmonic structure, there is an advantage that the residual component can be extracted with high accuracy even when the strength of the residual component is lower than the noise component.

  In the first embodiment, the noise coefficient sequence GNi corresponding to the analysis point p1 is corrected so as to correspond to the analysis point p2 in the second processing unit 32, and then used for extracting residual components. Therefore, the analysis parameters (window width ωA, movement amount δA) by the frequency analysis unit 42 of the first processing unit 31 and the analysis parameters (window width ωB, movement amount δB) by the frequency analysis unit 62 of the second processing unit 32 are individually set. There is an advantage that can be selected. Specifically, as described above, the analysis parameters (window width ωA, movement amount δA) of the frequency analysis unit 42 are selected from the viewpoint of numerical values suitable for the non-negative matrix factorization by the matrix decomposition unit 44, and the residual components are extracted. In addition, it is possible to select an analysis parameter of the frequency analysis unit 62 from the viewpoint of a numerical value appropriate for synthesis.

<B: Second Embodiment>
Next, a second embodiment of the present invention will be described. In addition, about the element which an effect | action and function are equivalent to 1st Embodiment in each following illustration, the code | symbol referred by the above description is diverted and each detailed description is abbreviate | omitted suitably.

  The target sound component that arrives from the front direction with respect to the two sound collecting devices 122 of the signal supply device 12 reaches each sound collecting device 122 with substantially the same intensity (amplitude) without generating a phase difference. On the other hand, since wind noise is caused by air turbulence as described above, the possibility of reaching each sound collecting device 122 with the same phase and the same amplitude is low. Therefore, the more the wind noise becomes dominant in the acoustic signal s1 and the acoustic signal s2, the more the phase difference and the intensity difference between the two tend to increase. In consideration of the above tendency, in the present embodiment, each coefficient value gTi [mA, nA] of the target sound coefficient sequence GTi and the noise coefficient sequence GNi according to the phase difference or intensity difference between the acoustic signal s1 and the acoustic signal s2. Each coefficient value gNi [mA, nA] is variably set.

  As shown in FIG. 10, the acoustic processing apparatus 100 of the second embodiment has a configuration in which a phase difference calculation unit 582 and an intensity difference calculation unit 584 are added to the first processing unit 31 of the first embodiment. The components of the bands BM of the acoustic signal s1 and the acoustic signal s2 are supplied to the phase difference calculation unit 582 and the intensity difference calculation unit 584. The band BM is set so as to include the frequency of the wind noise and the frequency of the main target sound component. For example, the band BM is set to a range of 4 kHz or less (that is, a band including the band BLa).

  The phase difference calculation unit 582 in FIG. 10 sequentially calculates the phase difference ΔP [nA] between the acoustic signal s1 and the acoustic signal s2 for each unit period (every time tnA). The phase difference ΔP [nA] is, for example, a representative value (for example, an average value) of the phase difference at each frequency in the band BM. Similarly, the intensity difference calculation unit 584 sequentially calculates the intensity difference (for example, amplitude difference or power difference) ΔA [nA] between the acoustic signal s1 and the acoustic signal s2 for each unit period.

  The coefficient sequence generation unit 522 of the target sound extraction unit 52 determines the target sound coefficient sequence according to the phase difference ΔP [nA] calculated by the phase difference calculation unit 582 and the intensity difference ΔA [nA] calculated by the intensity difference calculation unit 584. The coefficient value gTi [mA, nA] of GTi is variably set. Specifically, the coefficient sequence generation unit 522 calculates the above equation (A) as the phase difference ΔP [nA] or the intensity difference ΔA [nA] is larger (wind noise is dominant at the time tnA). The coefficient value gTi [mA, nA] is corrected to a small value. Therefore, according to the second embodiment, there is an advantage that it is possible to generate the estimated target sound signal qTi in which the wind noise is sufficiently suppressed as compared with the first embodiment.

  On the other hand, the coefficient sequence generation unit 542 of the noise extraction unit 54 variably sets the coefficient value gNi [mA, nA] of the noise coefficient sequence GNi according to the phase difference ΔP [nA] and the intensity difference ΔA [nA]. Specifically, the coefficient sequence generation unit 542 is calculated by the above formula (B) as the phase difference ΔP [nA] or the intensity difference ΔA [nA] is larger (wind noise is dominant at the time tnA). The coefficient value gNi [mA, nA] is corrected to a large value. Therefore, according to the second embodiment, there is an advantage that an estimated noise signal qNi in which wind noise is sufficiently emphasized can be generated as compared with the first embodiment.

<C: Third Embodiment>
The sound processing apparatus 100 according to the third embodiment has a configuration in which an adjustment unit 65 is added to the second processing unit 32 as illustrated in FIG. 11. The adjustment unit 65 is an amplifier (for example, a multiplier that multiplies a numerical value less than 1) that decreases the intensity (power) of the spectrum SNi of the estimated noise signal qNi. The target sound synthesizer 66 combines the spectrum Zi (band BLb) and spectrum Ri similar to those in the first embodiment and the spectrum SNi after processing (after attenuation) by the adjustment unit 65 to sequentially produce the spectrum ZRi for each unit period. Is generated. That is, the noise component of the acoustic signal si is added to the reproduced sound at a low volume.

  In the third embodiment, the same effect as in the first embodiment is realized. In the configuration of the first embodiment in which the spectrum ZRi is generated by synthesizing only the spectrum Zi of the estimated target sound signal qTi and the residual component spectrum Ri, the noise component can be highly excluded. The sound may be audibly unnatural. In the third embodiment, since the spectrum SNi of the estimated noise signal qNi is applied to the synthesis of the spectrum ZRi, there is an advantage that it is possible to generate a reproduced sound with an audibly natural impression.

<D: Fourth Embodiment>
In the first embodiment, the target sound extraction unit 52 applies the target sound coefficient sequence GTi generated from the base matrix Wi and the coefficient matrix Hi to the observation matrix Vi to generate a time series of the spectrum YTi, and the noise coefficient sequence GNi. Is applied to the observation matrix Vi by the noise extraction unit 54 to generate a time series of the spectrum YNi. In the fourth embodiment, the operations of the target sound extraction unit 52 and the noise extraction unit 54 are simplified.

  As described above with reference to FIG. 4, the matrix VTi obtained by multiplying the matrix WTi excluding the noise basis Ci_noise from the base matrix Wi and the matrix HTi excluding the weight sequence Ei_noise from the coefficient matrix Hi is a noise component from the acoustic signal si. Approximate to the spectrogram when. Therefore, the target sound extraction unit 52 of the fourth embodiment sequentially generates the matrix VTi as the time series (spectrogram) of the spectrum YTi after suppressing the noise component for each analysis period T0. A sequence of MA element values vTi [1, nA] to vTi [MA, nA] located in the nAth column of the matrix VTi is supplied to the waveform synthesis unit 56 as a spectrum YTi.

  The matrix VNi obtained by multiplying the noise base Ci_noise and the weight sequence Ei_noise approximates a spectrogram when the target sound component is suppressed from the acoustic signal si. Therefore, the noise extraction unit 54 of the fourth embodiment sequentially generates the matrix VNi as the time series (spectrogram) of the spectrum YNi after suppression of the target sound component for each analysis period T0. A sequence of MA element values vNi [1, nA] to vNi [MA, nA] located in the nA-th column of the matrix VNi is used as the spectrum YNi.

  FIG. 12 is a block diagram of the harmonic component extraction unit 64 in the fourth embodiment. As illustrated in FIG. 12, the harmonic component extraction unit 64 of the fourth embodiment has a configuration in which the coefficient string synthesis unit 744 of the first embodiment is omitted. The coefficient sequence correction unit 76 is supplied with the matrix VNi generated by the noise extraction unit 54.

  The matrix VNi is a matrix of MA rows × NA columns composed of element values vNi [1,1] to vNi [MA, NA] corresponding to each analysis point p1. The coefficient sequence correction unit 76 generates the matrix VBi by correcting (interpolating, thinning out) each element value vNi [mA, nA] of the matrix VNi. The matrix VBi is composed of element values vBi [1,1] to vBi [MB, NB] corresponding to the analysis points p2 corresponding to the analysis parameters (window width ωB, movement amount δB) of the frequency analysis unit 62.

  The harmonic extraction unit 78 applies the harmonic coefficient sequence Di generated by the harmonic structure specifying unit 742 to the matrix VBi after correction by the coefficient sequence correction unit 76, so that the NB spectra Ri within the analysis period T0 can be obtained. Generate time series (residual component spectrogram). Specifically, the harmonic extraction unit 78 multiplies the element value vBi [mB, nB] of the matrix VBi and the coefficient value di [mB, nB] of the harmonic coefficient sequence Di by the component value ri [ mB, nB]. Since the matrix VBi approximates the estimated value of the spectrum of the noise component after separation in the first processing unit 31, the spectrum Ri is the residual component of the target sound component (ie, the fundamental frequency Fi [nB]) from the noise component after separation. The harmonic component having a frequency that is an integral multiple of () is equivalent to the estimated value of the extracted spectrum. Therefore, the fourth embodiment can achieve the same effect as the first embodiment.

<E: Modification>
Each of the above forms can be variously modified. Specific modifications are exemplified below. Two or more aspects arbitrarily selected from the following examples can be appropriately combined.

(1) Modification 1
In each of the above embodiments, the fundamental frequency Fi [nB] of the target sound component of the acoustic signal si is estimated from the spectrum Zi of the estimated target sound signal qTi, but the method of estimating the fundamental frequency Fi [nB] is arbitrary. For example, the fundamental frequency Fi [nB] of the estimated target sound signal qTi can be estimated by time domain processing (for example, a method using an autocorrelation function). In addition, a configuration in which the fundamental frequency Fi [nB] is estimated by analyzing the acoustic signal si (or spectrum Xi) before extracting the target sound component may be employed. However, since the estimation accuracy of the fundamental frequency Fi [nB] decreases at the stage where noise components are mixed, from the viewpoint of high-precision estimation of the fundamental frequency Fi [nB], the fundamental frequency Fi is suppressed after the noise component is suppressed. The configuration of the first embodiment for estimating [nB] is advantageous.

  For example, even when the fundamental frequency Fi is estimated from the acoustic signal si in which the target sound component and the noise component are mixed, if the noise component is attenuated by removing the component below the cutoff frequency (low cut filter processing), the fundamental frequency Fi [nB] can also be estimated with high accuracy. However, assuming a situation where the frequency of the noise component changes every moment, it is very difficult to select the cut-off frequency as an optimum value. In each of the above-described embodiments, the noise component is effectively suppressed even when the frequency of the noise component changes, and the fundamental frequency Fi [nB] is estimated for the estimated target sound signal qTi after the suppression. There is an advantage that the fundamental frequency Fi [nB] can be estimated with high accuracy without causing the problem of selection of the cut-off frequency when the frequency changes.

(2) Modification 2
The configuration for limiting the target sound component and the noise component (first processing unit 31) and the residual component extraction (second processing unit 32) to the lower band (BLa, BLb) may be omitted. For example, a configuration in which the entire band of the acoustic signal si is a target of processing by the matrix decomposing unit 44 or the noise specifying unit 46 may be employed. However, since the intensity of the wind noise decreases in the higher band (for example, the band BHa), in the configuration in which the band division of the acoustic signal si is omitted, an independent basis Ci [k] of the wind noise is obtained by non-negative matrix factorization. It becomes difficult to extract with high accuracy. Therefore, when the frequency band of the noise component to be suppressed is known in advance, only the frequency band (band BLa) including the noise component is processed by the matrix decomposing unit 44 and the noise specifying unit 46. The above-described configuration is particularly suitable.

(3) Modification 3
In each of the above embodiments, the target sound coefficient sequence GTi and the noise coefficient sequence GNi are generated every analysis period T0 of the acoustic signal si, but the division of the analysis period T0 is omitted. For example, a configuration in which the time series of the spectrum Xi for each unit period over the entire section of the acoustic signal si is one observation matrix Vi can be adopted.

(4) Modification 4
In each of the above embodiments, the estimated target sound signal qTi is generated by multiplying each component value xi [mA, nA] of the acoustic signal si by each coefficient value gTi [mA, nA] of the target sound coefficient sequence GTi. The method of applying the target sound coefficient sequence GTi to the acoustic signal si is appropriately changed. For example, a configuration in which the coefficient value gTi [mA, nA] is added to each component value xi [mA, nA] of the acoustic signal si may be employed. Contrary to the examples in the above embodiments, in the configuration in which the target sound coefficient sequence GTi is generated so that the coefficient value gTi [mA, nA] becomes larger as the wind noise becomes more dominant, the component value xi A configuration in which [mA, nB] is divided or subtracted by the coefficient value gTi [mA, nA] may be employed. Similarly, for the noise coefficient string GNi for emphasizing the noise component, the method of application to the acoustic signal si and the relationship with the superiority or inferiority of wind noise are appropriately changed.

(5) Modification 5
In the above embodiments, two systems of acoustic signals qi (q1, q2) are generated, but the above embodiments can be similarly applied when only one system (monaural format) of acoustic signals q1 is generated. For example, separation of target sound components and noise components and extraction and addition of residual components are executed for only one system corresponding to the acoustic signal s1. In the above configuration, the acoustic signal s2 is used to specify the noise base C1_noise from the basis matrix W1 of the acoustic signal s1.

(6) Modification 6
The processing of the arithmetic processing unit 22 is executed in real time in parallel with the supply of the acoustic signal si, and the acoustic signal qi can be reproduced sequentially for each processing. However, a configuration (batch processing) in which the reproduction of the acoustic signal qi is started after the processing for the acoustic signal si prepared in advance is completed is also suitable.

DESCRIPTION OF SYMBOLS 100 ... Sound processing device, 12 ... Signal supply device, 14 ... Sound emission device, 22 ... Arithmetic processing device, 24 ... Memory | storage device, 31 ... 1st processing part, 32 ... 2nd processing part, 42 …… Frequency analysis unit, 44 …… Matrix decomposition unit, 46 …… Noise identification unit, 52 …… Target sound extraction unit, 54 …… Noise extraction unit, 56 …… Waveform synthesis unit, 522 …… Coefficient sequence generation unit 524 ... Extraction processing unit, 542 ... Coefficient sequence generation unit, 544 ... Extraction processing unit, 582 ... Phase difference calculation unit, 584 ... Intensity difference calculation unit, 62 ... Frequency analysis unit, 64 ... Adjustment Wave component extraction unit, 65 ... adjustment unit, 66 ... target sound synthesis unit, 68 ... waveform synthesis unit, 72 ... frequency estimation unit, 74 ... harmonic coefficient sequence generation unit, 742 ... harmonic structure identification Unit 744... Coefficient sequence combining unit 76... Coefficient sequence correction unit 78.

Claims (6)

For each of the first acoustic signal and the second acoustic signal collected in parallel, the acoustic signals are different by non-negative matrix factorization of an observation matrix having a time series of component values for each frequency of the acoustic signal as elements. Matrix decomposition means for generating a base matrix including a plurality of bases indicating component values for each frequency of the component, and a coefficient matrix including a plurality of weight sequences each indicating a time series of weight values of the respective bases;
A base having a high correlation with the base of the base matrix of the second acoustic signal among the plurality of bases of the base matrix of the first acoustic signal is identified as a noise base corresponding to a noise component of the first acoustic signal Noise identification means to
An estimated target sound component obtained by suppressing the noise component from the first acoustic signal is obtained using each basis other than the noise basis in the basis matrix and each weight sequence corresponding to the other than the noise basis in the coefficient matrix. A target sound extraction means to generate;
Noise extraction means for generating an estimated noise component obtained by suppressing a target sound component from the first acoustic signal using the noise base and a weight sequence corresponding to the noise base in the coefficient matrix;
Harmonic component extraction means for extracting a residual component corresponding to the harmonic structure of the target sound component from the estimated noise component;
A sound processing apparatus comprising: target sound synthesis means for synthesizing the estimated target sound component and the residual component. The harmonic component extraction means includes
Frequency estimating means for estimating a fundamental frequency of the target sound component;
Harmonic coefficient string generation means for generating a harmonic coefficient string in which each coefficient value is set so as to emphasize a harmonic component of a frequency that is an integral multiple of the fundamental frequency among the estimated noise components;
The acoustic processing apparatus according to claim 1, further comprising: a harmonic extraction unit that causes the harmonic coefficient sequence to act on the estimated noise component to extract the residual component. The sound processing apparatus according to claim 2, wherein the frequency estimation unit estimates a fundamental frequency of the estimated target sound component generated by the target sound extraction unit. A time series of spectra for each unit section of the first acoustic signal and the second acoustic signal is generated as the observation matrix based on a first analysis parameter including a window width and a movement amount of each unit section . Frequency analysis means;
A second frequency analysis for sequentially generating a spectrum of the estimated target sound component and the estimated noise component for each unit section based on a second analysis parameter including a window width and a movement amount different from the first analysis parameter. Means,
Coefficient sequence correction means for generating a correction coefficient sequence in which coefficient values are set for each analysis point arranged on the time axis and the frequency axis at intervals according to the second analysis parameter,
The noise extraction unit corresponds to the noise base and the noise base, a noise coefficient sequence in which coefficient values are set for each analysis point arranged on the time axis and the frequency axis at intervals according to the first analysis parameter. And generating the estimated noise component by applying the noise coefficient sequence to the observation matrix,
The coefficient sequence correction unit generates the correction coefficient sequence from the noise coefficient sequence,
4. The acoustic processing device according to claim 2, wherein the harmonic coefficient string generation unit generates a harmonic coefficient string by extracting a component having a frequency that is an integral multiple of the fundamental frequency from the correction coefficient string. Phase difference calculating means for calculating a phase difference between the first acoustic signal and the second acoustic signal;
An intensity difference calculating means for calculating an intensity difference between the first acoustic signal and the second acoustic signal;
The target sound extraction means uses a target sound coefficient sequence in which each coefficient value is variably set according to a phase difference and an intensity difference between the first acoustic signal and the second acoustic signal as the noise in the base matrix. Generating from each basis other than the basis and a weight sequence corresponding to the coefficient matrix other than the noise basis, and acting on the observation matrix;
The noise extraction unit corresponds to a noise coefficient sequence in which each coefficient value is variably set according to a phase difference and an intensity difference between the first acoustic signal and the second acoustic signal, and corresponds to the noise base and the noise base. The sound processing device according to claim 4, wherein the sound processing device is generated from a weight sequence to be applied to the observation matrix.   For each of the first acoustic signal and the second acoustic signal collected in parallel, the acoustic signals are different by non-negative matrix factorization of an observation matrix having a time series of component values for each frequency of the acoustic signal as elements. Matrix decomposition processing for generating a base matrix including a plurality of bases indicating component values for each frequency of the component, and a coefficient matrix including a plurality of weight sequences each indicating a time series of weight values of the respective bases;
  A base having a high correlation with the base of the base matrix of the second acoustic signal among the plurality of bases of the base matrix of the first acoustic signal is identified as a noise base corresponding to a noise component of the first acoustic signal Noise identification processing,
  An estimated target sound component obtained by suppressing the noise component from the first acoustic signal is obtained using each basis other than the noise basis in the basis matrix and each weight sequence corresponding to the other than the noise basis in the coefficient matrix. A target sound extraction process to be generated;
  A noise extraction process for generating an estimated noise component obtained by suppressing a target sound component from the first acoustic signal using the noise base and a weight sequence corresponding to the noise base in the coefficient matrix;
  A harmonic component extraction process for extracting a residual component corresponding to the harmonic structure of the target sound component from the estimated noise component;
  A target sound synthesis process for synthesizing the estimated target sound component and the residual component;
  A program that causes a computer to execute.

Images