JP2019078864A - Musical sound emphasis device, convolution auto encoder learning device, musical sound emphasis method, and program - Google Patents

Abstract

To provide a musical sound emphasis technique which enables sound source emphasis with high accuracy, using DNN which reflects characteristics of musical instrument sound.SOLUTION: The musical sound emphasis device includes a frequency domain conversion unit which converts a time domain musical signal into a frequency domain to generate a frequency domain musical signal, a Wiener filter estimation unit which estimates a Wiener filter used to emphasize a predetermined musical instrument sound from the frequency domain musical signal, a signal emphasizing unit for generating a frequency domain emphasized musical sound from the frequency domain musical signal and the Wiener filter, and a time domain converting unit for generating an emphasized musical sound in the time domain from the frequency domain emphasized musical sound. The Wiener filter estimation unit estimates the Wiener filter from the frequency domain musical signal, using a deep neural network including a 2n-layer convolutional denoising auto encoder which takes an amplitude spectrum of the music signal in a logarithmic frequency domain as input and outputs the amplitude spectrum of a part of the music signal in the logarithmic frequency domain.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a sound source emphasizing technique for emphasizing a sound source signal of a specific sound source from sound source signals mixed with sound source signals of various sound sources, and in particular to a target musical tone signal from music signals composed of a plurality of musical tone signals. Relates to a tone emphasizing technique that emphasizes

  Conventionally, a method of emphasizing a sound source signal of a specific sound source from a sound source signal in which sound source signals of various sound sources are mixed has been widely studied. Recently, for example, as described in Non-Patent Document 1, many approaches have been taken to obtain projections among a plurality of variables by supervised learning using Deep Neural Networks (DNN).

  Below, the sound source emphasizing device 900 of Non-Patent Document 1 will be described with reference to FIGS. 13 to 14 (see FIG. 1 of Non-Patent Document 1). The sound source emphasizing device 900 outputs an emphasized sound in which the target sound in the observation signal collected by the microphone is sound source-emphasized. FIG. 13 is a block diagram showing the configuration of the sound source emphasizing device 900. As shown in FIG. FIG. 14 is a flowchart showing the operation of the sound source enhancement apparatus 900. As shown in FIG. 13, the sound source emphasizing device 900 includes a frequency domain transforming unit 910, a Wiener filter estimating unit 920, a signal emphasizing unit 930, and a time domain transforming unit 940.

The sound source emphasizing device 900 is connected to the learning result recording unit 990. The learning result recording unit 990 records the values of DNN network parameters learned in advance. The network parameters of DNN are: weight matrix W _S ⁽²⁾ , W _S ⁽³⁾ and bias vector b _S ⁽²⁾ , b _S ⁽³⁾⁾ , weight matrix W _N ⁽²⁾ , W _N ⁽³⁾ and bias The vectors b _N ⁽²⁾ and b _N ^{(3) are} the weighting matrix W ⁽⁴⁾ .

  The frequency domain conversion unit 910 performs frequency domain conversion on the observation signal x (t) that is time domain mixed sound to generate a frequency domain observation signal X (ω, τ) (S 910). Here, t is an index of time, ω is a frequency bin number, and τ is a frame number.

  The Wiener filter estimation unit 920 estimates the Wiener filter ^ W (ω, τ) from the frequency domain observation signal X (ω, τ) using the network parameter read out from the learning result recording unit 990 (S920). Hereinafter, the Wiener filter estimation unit 920 will be described with reference to FIGS. FIG. 15 is a block diagram showing the configuration of the Wiener filter estimation unit 920. As shown in FIG. FIG. 16 is a flowchart showing the operation of the Wiener filter estimation unit 920. As shown in FIG. 15, the Wiener filter estimation unit 920 includes a beamforming output power calculation unit 921, a target sound nonnegative auto encoder unit 922, a noise nonnegative auto encoder unit 923, a complementary subtraction unit 924, and a Wiener filter calculation unit 925. including.

First, the beamforming output power calculator 921 calculates the target sound BF output power φ _{Y — S} (ω, τ) and noise BF output power φ _{Y — N} (ω, τ) from the frequency domain observation signal X (ω, τ) S921).

Next, the target sound nonnegative auto encoder unit 922 sets the target sound BF output power φ _{Y — S} (ω, τ) calculated in _{S 921 as} the input q _S ⁽¹⁾ of the first layer, and the target sound spectral characteristic vector q _S ^{(3 )} Is calculated (S922). The target sound non-negative auto encoder unit 922 includes the network parameters of DNN learned in advance (specifically, weighting matrices W _S ⁽²⁾ and W _S ⁽³⁾ and bias vectors b _S ⁽²⁾ and b _S ^{(3 ))} Is set.

Similarly, the noise non-negative auto encoder unit 923 calculates the noise spectrum characteristic vector q _N ⁽³⁾ with the noise BF output power φ _{Y — N} (ω, τ) calculated in S921 as the input q _N ⁽¹⁾ of the first layer. To do (S923). The noise non-negative auto encoder unit 923 includes the network parameters of DNN learned in advance (specifically, the weight matrices W _N ⁽²⁾ and W _N ⁽³⁾ and the bias vectors b _N ⁽²⁾ and b _N ⁽³⁾ ) Is set.

Next, the complementary subtraction unit 924 estimates the estimated target sound PSD q _S ⁽⁴⁾ , the estimated noise PSD from the target sound spectrum characteristic vector q _S ⁽³⁾ and the noise spectrum characteristic vector q _N ⁽³⁾ calculated in S922 and S923. q _N ⁽⁴⁾ is calculated (S924). In the complementary subtraction unit 924, network parameters (specifically, a weighting matrix W ⁽⁴⁾ ) of DNN learned in advance are set.

Finally, the Wiener filter calculator 925 calculates a Wiener filter ^ W (ω, τ) from the estimated target sound PSD q _S ⁽⁴⁾ and estimated noise PSD q _N ⁽⁴⁾ calculated in S924 (S925).

  The signal emphasizing unit 930 executes signal emphasizing (sound source emphasizing) according to the following equation from the frequency domain observation signal X (ω, τ) and the Wiener filter ^ W (ω, τ) estimated in S920, and the frequency domain emphasizing sound S '. (ω, τ) is generated (S930).

  The time domain transform unit 940 generates an enhanced sound s '(t) that is an estimated sound source signal in the time domain from the frequency domain enhanced sound S' (ω, τ) (S940).

  In the sound source emphasizing device 900, according to the description of Non-Patent Document 1, sounds other than the target sound are modeled as one noise sound source, but sounds other than the target sound may be modeled as a plurality of noise sources for each sound source. Good. In this case, the sound source enhancement apparatus 900 includes the same number of noise non-negative auto encoders 923 as the number of noise sources. For example, if it is desired to emphasize the sound of the guitar as the target sound among the three sound sources of guitar, bass and drums, two of the bass and the drums may be modeled as one noise source, or the bass may be a noise source 1, The drums may be modeled as two noise sources as the noise source 2.

K. Niwa, Y. Koizumi, T. Kawase, K. Kobayashi, and Y. Hioka, “Supervised source enhancement composed of non-autonomous auto-encoders and complementarity subtraction”, Proc. Of 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP 2017), pp. 266-270, 2017.

The sound source enhancement method of Non-Patent Document 1 is characterized by the Wiener filter estimation unit 920. That is, the Wiener filter estimation unit 920 models the target sound spectrum and the noise spectrum by calculating the target sound spectrum characteristic vector q _S ⁽³⁾ and the noise spectrum characteristic vector q _N ⁽³⁾ using an auto encoder. There is. Specifically, (i) constraining each element nonnegatively with respect to weighting matrices W _S ⁽²⁾ , W _S ⁽³⁾ , W _N ⁽²⁾ and W _N ⁽³⁾ which are network parameters of DNN, (ii) Weight matrices projected to the second and third layers are transposed (that is, W _S ⁽³⁾ = W _S ^{(2) T} , W _N ⁽³⁾ = W _N ^{(2) T} ) It is characterized by being restricted to These features are equivalent to decomposing into a matrix of a basis group (characteristic spectrum) constituting a target sound or noise and a matrix representing a component to be activated by the base. It comprises DNN reflecting the physical characteristics of spectrum and noise spectrum.

  However, in the method of Non-Patent Document 1, since all kinds of sounds are to be input, the configuration of the DNN is quite complicated. If the input is limited to a music signal composed of a plurality of musical instrument sounds such as the above-mentioned guitar, bass, and drums, and a DNN reflecting the characteristics of the musical instrument sound is formed, the target sound It is considered that spectrum and noise spectrum can be expressed as simpler and more efficient DNN. Moreover, it is expected that the accuracy of sound source emphasis can be improved by adopting such a DNN configuration.

  Therefore, it is an object of the present invention to provide a musical tone emphasizing technology capable of emphasizing sound source with high accuracy using DNN reflecting the characteristics of musical instrument sound.

  According to one aspect of the present invention, there is provided a frequency domain conversion unit that frequency domain converts a time domain music signal x (t), which is a monaural mixed music signal, to generate a frequency domain music signal X (ω, τ); A Wiener filter estimation unit for estimating a Wiener filter ^ W (ω, τ) used to emphasize a predetermined musical instrument sound from a signal X (ω, τ); the frequency domain music signal X (ω, τ) and the Wiener A signal emphasizing unit for generating a frequency domain emphasized tone S '(ω, τ) from the filter ^ W (ω, τ), and an emphasized tone s' in the time domain from the frequency domain emphasized tone S' (ω, τ) (t), and the Wiener filter estimation unit receives the amplitude spectrum of the music signal in the logarithmic frequency domain as an input, and the amplitude spectrum of a part of the music signal in the logarithmic frequency domain Output of 2n layers (n is an integer of 1 or more) The Wiener filter ^ W (ω, τ) is estimated from the frequency domain music signal X (ω, τ) using a deep neural network including a tone encoder.

  According to the present invention, it is possible to emphasize a specific musical tone of a music signal composed of a plurality of musical instrument sounds with high accuracy.

FIG. 1 is a block diagram showing a configuration of a tone emphasis device 100. 6 is a flowchart showing the operation of the tone emphasis device 100. FIG. 2 is a block diagram showing a configuration of a Wiener filter estimation unit 120. 6 is a flowchart showing the operation of the Wiener filter estimation unit 120. FIG. 2 is a block diagram showing the configuration of a Wiener filter estimation unit 220. 6 is a flowchart showing the operation of the Wiener filter estimation unit 220. FIG. 7 is a block diagram showing the configuration of a Wiener filter estimation unit 320. 6 is a flowchart showing the operation of the Wiener filter estimation unit 320. FIG. 7 is a block diagram showing the configuration of a Wiener filter estimation unit 420. 7 is a flowchart showing the operation of the Wiener filter estimation unit 420. FIG. 16 is a block diagram showing the configuration of a convolution auto encoder learning device 500. 8 is a flowchart showing the operation of the convolutional auto encoder learning device 500. FIG. 16 is a block diagram showing the configuration of a sound source emphasis device 900. 11 is a flowchart showing the operation of the sound source enhancement device 900. FIG. 16 is a block diagram showing a configuration of a Wiener filter estimation unit 920. The flowchart which shows the operation of the Wiener filter estimating part 920.

  Hereinafter, embodiments of the present invention will be described in detail. Note that components having the same function will be assigned the same reference numerals and redundant description will be omitted.

  Before describing each embodiment, a description method in this specification will be described.

_ (Underscore) represents a subscript subscript. For example, ^xy_z represents that _yz is a superscript for x, and _{xy_z} represents that _yz is a subscript for x.

<Technical background>
Hereinafter, a framework for estimating a target sound spectrum and a noise spectrum using characteristics of an instrument sound will be described, assuming that a monaural (1 ch) music signal is handled. First, a model (spectral model of musical instrument sound) regarding spectral characteristics of musical instrument sound will be described. Next, the structure of the DNN reflecting the spectral model of the musical instrument sound will be described. The structure of DNN described here corresponds to a structure in which a logarithmic frequency spectrum is input to a convolutional de-noising auto-encoder (CDAE).

It is assumed that the monaural mixed music signal X (ω, τ) in the frequency domain is represented by simple addition of N (N is an integer of 1 or more) sound source signals S _n (ω, τ).

  Here, n is a number representing each sound source, ω is a frequency bin number (index of frequency), and τ is a frame number (index of time frame).

  Each sound source is assumed to correspond to a predetermined musical instrument.

[Spectrum model of instrument sound]
The spectral characteristics S _n (ω ′, τ) of many instrumental sounds are represented by a weighted sum of spectral bases that vary depending on the fundamental frequency and the instrumental sound.

Here, I _n is the number of spectral bases that can be considered as different fundamental frequencies, w _{i, n} (ω ′, τ) and H _{i, n} (τ) are i in frame τ in the n-th musical instrument sound respectively The spectral base corresponding to the th fundamental frequency and the corresponding weight (gain) are shown. Also, ω 'is a logarithmic frequency bin number.

Here, a logarithmic frequency in the cent region is assumed. For example, an equation for converting the frequency f _k [Hz] to the logarithmic frequency f _cent [cents] on the cent scale is expressed as follows.

  The value of the constant c is often c = 1200.

  The logarithmic frequency bin numbers ω ′ are obtained by discretizing them as evenly as possible on the scale after conversion according to this equation.

Hereinafter, a method of converting Ω linear frequency bins {ω ₁ ,..., Ω _Ω } into D logarithmic frequency bands {ω ′ ₁ ,..., Ω ′ _D } will be described.

Assuming that the sampling frequency is F _s [Hz], and the central frequency of the k-th (1 ≦ k ≦ Ω) frequency bin is logarithmically converted, it is as follows.

  Similarly, when the central frequency of the d-th (1 ≦ d ≦ D) logarithmic frequency band is logarithmically converted, it is as follows.

  Next, when the correspondence relationship (d, k) of the indexes is obtained using the following equation so as to make them identical, conversion to a logarithmic frequency band is performed. Note that d and k are not in one-to-one correspondence, and a plurality of index groups correspond to one logarithmic frequency band.

The argument is returned to equation (2). The spectral base w _{i, n} (ω ′, τ) of the n-th instrumental sound is characterized by the harmonic structure component C _i (ω ′, τ) changing depending on the fundamental frequency as in the following equation and the characteristic for each instrument Can be decomposed into different envelope structure components F _n (ω ′, τ).

In addition, harmonic structure component C _i (ω ′, τ) and envelope structure component F _n (ω ′, τ) are functions having all peaks at the frequency corresponding to the fundamental frequency and its overtone as follows, all poles Expressed by type function.

Where R is the number of harmonic components of the harmonic structure component, f _{ω ′} [cents] is the logarithmic frequency corresponding to the logarithmic frequency bin ω ′, and f _o (i, τ) [cents] is the ith harmonic in frame τ Logarithmic fundamental frequency of wave structure component, a _{n, p} is autoregressive filter coefficient of all pole function of nth envelope structure component, ω _{i, r} (τ) [rad] is i th harmonic structure in frame τ It is a normalized angular frequency corresponding to the r-th harmonic of the component. Here, j represents an imaginary unit.

From harmonic structure component equation (4) it is seen that the fundamental frequency f _o (i, tau) the whole harmonic structure component change of can be expressed by shifting on the logarithmic frequency axis. For example, the harmonic structure component when the fundamental frequency is increased by f [cents] can be calculated as follows.

Here, for the harmonic structure component C f — _{o (i, τ)} (f _{ω ′} , τ), the fundamental frequency is fixed to an arbitrary value f _o [cents], and H _{i, n} (τ) is the fundamental frequency f _o ( Assuming that the function H _n (f _o (i, τ), τ) varies depending on i, τ), the equation (2) can be transformed as the equation (6).

  Here, * represents a convolution operation, δ (·) represents a unit impulse, and the product of variables is expressed as a convolution with an impulse.

In this equation variant, replace i with f _o (i, τ) so that it becomes clear that the i th harmonic structure component is determined by the fundamental frequency f _o (i, τ), and the logarithmic frequency bin number ω ' Rewritten to the corresponding logarithmic frequency f _{ω '} [cents].

From equation (6), the spectral characteristics of a certain instrumental sound are determined by the harmonic structure component corresponding to an arbitrary fundamental frequency f _o , the gain corresponding to each harmonic structure component, and the frequency f _{ω ′} of the instrument specific envelope structure component It can be seen that it is expressed by convolution of impulses having a magnitude corresponding to the gain of.

[Structure of DNN reflecting spectral model of musical instrument sound]
Next, the structure of the DNN reflecting the spectral model of the musical instrument sound will be described.

  The DNN is constructed using the convolutional decomposition model of the instrument sound spectrum of equation (6). As a result, it is expected that the spectrum of the target sound (that is, the target instrument sound as the target of sound source enhancement) can be efficiently expressed as DNN, and the spectrum of the target sound can be estimated with high accuracy.

  Specifically, the target instrument sound spectrum is represented as a DNN having a two-stage convolution product as follows.

  Here, g (·) represents an activation function.

  Here, since the musical instrument sound spectrum and its decomposition component are non-negative, for example, by using ReLU (ramp function) of the following equation as the activation function g (·), the equations (7) and (8) can be obtained. It is considered to be equivalent to equation (6).

  That is, the convolutional decomposition model of the instrument sound spectrum of Equation (6) is expressed as a CDAE for calculating Equation (7) and a CDAE for calculating Equation (8), that is, a DNN including a two-layer CDAE. Therefore, it is considered that by using this DNN, an instrument sound spectrum can be efficiently obtained from the mixed music spectrum.

  The CDAE included in DNN reflecting the spectral model of musical instrument sound does not have to be limited to two layers, and in general, it may be 2n layers (n is an integer of 1 or more).

  When learning DNN reflecting a spectral model of musical instrument sound as a DNN including a CDAE, even if an appropriate spectral model is obtained, the correspondence relationship to each layer is in random order, so the envelope structure component and the key It is also conceivable that the wave structure component is reversed and output.

  When learning is performed with DNN reflecting the spectral model of musical instrument sound as DNN including two-layer CDAE, the size (filter size) of the convolution filter in each layer in the frequency direction is the number of frequency bins of the input spectrum and It should be considered identical. In fact, although the experimental verification was made for the case where the filter size is smaller than the input spectrum, the result that the performance is the highest when matching with the input spectrum was obtained.

First Embodiment
The tone enhancing apparatus 100 will be described below with reference to FIGS. FIG. 1 is a block diagram showing the configuration of the tone enhancing apparatus 100. As shown in FIG. FIG. 2 is a flow chart showing the operation of the tone enhancing apparatus 100. As shown in FIG. 2, the musical tone emphasizing device 100 includes a frequency domain transforming unit 910, a Wiener filter estimating unit 120, a signal emphasizing unit 930, and a time domain transforming unit 940.

  The tone enhancement device 100 is connected to the learning result storage unit 990 as in the sound source enhancement device 900. The learning result recording unit 990 records the values of network parameters at the end of learning of the DNN. This network parameter is used by the Wiener filter estimation unit 120.

  The frequency domain conversion unit 910 frequency domain converts the time domain music signal x (t), which is a monaural mixed music signal, to generate a frequency domain music signal X (ω, τ) (S 910). Here, t is an index of time, ω is a frequency bin number, and τ is a frame number.

  The Wiener filter estimation unit 120 uses the network parameters read out from the learning result recording unit 990 to emphasize a predetermined instrumental sound (hereinafter referred to as the target sound) from the frequency domain music signal X (ω, τ). The Wiener filter ^ W (ω, τ) is estimated (S 120). The predetermined musical instrument sound is the sound of the musical instrument used to create the monaural mixed music signal. For example, if the monaural mixed music signal is a sound produced from a guitar, a bass, or a drum, the sound of the guitar can be a predetermined instrumental sound.

  Hereinafter, the Wiener filter estimation unit 120 will be described with reference to FIGS. 3 to 4. FIG. 3 is a block diagram showing the configuration of the Wiener filter estimation unit 120. As shown in FIG. FIG. 4 is a flowchart showing the operation of the Wiener filter estimation unit 120. As shown in FIG. 3, the Wiener filter estimation unit 120 includes a logarithmic amplitude spectrum calculation unit 121, a target sound convolution auto encoder unit 122, a logarithm-linear frequency conversion unit 123, and a Wiener filter calculation unit 124.

  The logarithmic amplitude spectrum calculation unit 121 performs logarithmic frequency domain conversion on the frequency domain music signal X (ω, τ) and calculates music signal logarithmic amplitude spectrum | X (ω ′, τ) | (S121). The music signal logarithmic amplitude spectrum | X (ω ′, τ) | is a logarithmic frequency domain music signal X (ω ′, which is a signal obtained by converting the frequency domain music signal X (ω, τ) from the frequency domain to the logarithmic frequency domain. amplitude spectrum of τ). The conversion from the frequency domain to the logarithmic frequency domain may be as described in <Technical background>.

  The target sound convolution auto encoder unit 122 estimates an estimated target sound logarithm that is an amplitude spectrum in a logarithmic frequency domain of a predetermined instrument sound (target sound) from the music signal log amplitude spectrum | X (ω ′, τ) | calculated in S121. The amplitude spectrum | ^ S (ω ', τ) | is calculated (S122). The target sound convolution auto-encoder unit 122 receives the amplitude spectrum of the music signal in the logarithmic frequency domain as an input, and outputs the amplitude spectrum of a predetermined musical instrument sound (which is a part of the music signal) in the logarithmic frequency domain. It is DNN containing 2 n layer CDAE. Further, in the target sound convolution auto encoder unit 122, network parameters at the end of DNN learning (that is, network parameters recorded in the learning result recording unit 990) are set. Specifically, the target sound convolution auto encoder unit 122 becomes a DNN including a 2n-layer CDAE for calculating the equations (7) and (8) as described in <Technical background>. The learning of the network parameters of DNN will be described later.

  The log-linear frequency conversion unit 123 performs frequency domain conversion on the estimated target sound log amplitude spectrum | ^ S (ω ′, τ) | calculated in S122, and estimates the estimated target sound amplitude spectrum | ^ S (ω, τ) | Calculate (S123). The estimated target sound amplitude spectrum | ^ S (ω, τ) | is an estimated frequency domain target sound ^ obtained by converting the estimated target sound log amplitude spectrum | ^ S (ω ′, τ) | from the logarithmic frequency domain to the frequency domain. It is an amplitude spectrum of S (ω, τ). The transformation from the logarithmic frequency domain to the frequency domain may be as described in <Technical background>.

  The Wiener filter calculation unit 124 calculates a music signal amplitude spectrum | X (ω, τ) | that is an amplitude spectrum of the frequency domain music signal X (ω, τ) generated in S910 and an estimated target sound amplitude spectrum | ^ calculated in S123. A Wiener filter ^ W (ω, τ) is calculated from S (ω, τ) | (S124). Specifically, the Wiener filter ^ W (ω, τ) is calculated by the following equation.

  The signal emphasizing unit 930 executes signal emphasizing (sound source emphasizing) according to the following equation from the frequency domain music signal X (ω, τ) generated in S 910 and the Wiener filter ^ W (ω, τ) estimated in S 120 An emphasized tone S '(. Omega.,. Tau.) Is generated (S930).

  The time domain conversion unit 940 generates an emphasized tone s '(t) in the time domain from the frequency domain emphasized tone S' (ω, τ) (S940).

  According to the invention of the present embodiment, it is possible to emphasize a specific musical tone of a music signal composed of a plurality of musical instrument sounds with high accuracy. The invention of the present embodiment uses the spectral characteristics of the instrument sound (that is, the spectrum is decomposed into harmonic-envelope components) by constructing a DNN including 2 n-layer CDAEs having a logarithmic frequency spectrum as an input. It is possible to estimate the target sound spectrum. Further, this makes it possible to estimate the Wiener filter with high accuracy even if the calculation scale of DNN is limited, and as a result, it is possible to enhance the musical tone with high accuracy.

(Modification)
Although the Wiener filter estimation unit 120 calculates the order of the logarithmic-linear frequency conversion and the Wiener filter calculation to estimate the Wiener filter, the calculation order may be reversed. Here, such a Wiener filter estimation part 220 is demonstrated. In this case, the musical tone emphasizing device 100 includes a frequency domain converting unit 910, a Wiener filter estimating unit 220, a signal emphasizing unit 930, and a time domain converting unit 940 (see FIGS. 1 and 2).

  Hereinafter, the Wiener filter estimation unit 220 will be described with reference to FIGS. 5 to 6. FIG. 5 is a block diagram showing the configuration of the Wiener filter estimation unit 220. As shown in FIG. FIG. 6 is a flowchart showing the operation of the Wiener filter estimation unit 220. As shown in FIG. 5, the Wiener filter estimation unit 220 includes a logarithmic amplitude spectrum calculation unit 121, a target sound convolution auto encoder unit 122, a logarithmic Wiener filter calculation unit 125, and a logarithmic-linear frequency conversion unit 123.

  The logarithmic amplitude spectrum calculation unit 121 performs logarithmic frequency domain conversion on the frequency domain music signal X (ω, τ) and calculates music signal logarithmic amplitude spectrum | X (ω ′, τ) | (S121).

  The target sound convolution auto encoder unit 122 estimates an estimated target sound logarithm that is an amplitude spectrum in a logarithmic frequency domain of a predetermined instrument sound (target sound) from the music signal log amplitude spectrum | X (ω ′, τ) | calculated in S121. The amplitude spectrum | ^ S (ω ', τ) | is calculated (S122).

  The logarithmic Wiener filter calculator 125 calculates a logarithmic Wiener from the music signal logarithmic amplitude spectrum | X (ω ′, τ) | calculated in S121 and the estimated target sound logarithmic amplitude spectrum | ^ S (ω ′, τ) | calculated in S122. The filter ^ W (ω ', τ) is calculated (S125). Specifically, the logarithmic Wiener filter ^ W (ω ', τ) is calculated by the following equation.

  The log-linear frequency conversion unit 123 performs frequency domain conversion on the logarithmic Wiener filter ^ W (ω ', τ) calculated in S125, and calculates the Wiener filter ^ W (ω, τ) (S123).

Second Embodiment
In the first embodiment, tone enhancement is performed using DNN including a 2n-layer CDAE that estimates a spectrum of a logarithmic frequency domain for a tone to be emphasized in a sound source. It is also possible to perform musical tone enhancement by using a DNN including a 2n-layer CDAE that estimates a spectrum in the logarithmic frequency domain. Here, such a Wiener filter estimation part 320 is demonstrated. In this case, the musical tone emphasizing device 100 includes a frequency domain converting unit 910, a Wiener filter estimating unit 320, a signal emphasizing unit 930, and a time domain converting unit 940 (see FIGS. 1 and 2).

  Hereinafter, the Wiener filter estimation unit 320 will be described with reference to FIGS. 7 to 8. FIG. 7 is a block diagram showing the configuration of the Wiener filter estimation unit 320. As shown in FIG. FIG. 8 is a flowchart showing the operation of the Wiener filter estimation unit 320. As shown in FIG. 7, the Wiener filter estimation unit 320 includes a logarithmic amplitude spectrum calculation unit 121, a target sound convolution auto encoder unit 122, a noise convolution auto encoder unit 222, a log-linear frequency conversion unit 123, and a Wiener filter calculation. Part 224 is included.

  The logarithmic amplitude spectrum calculation unit 121 performs logarithmic frequency domain conversion on the frequency domain music signal X (ω, τ) and calculates music signal logarithmic amplitude spectrum | X (ω ′, τ) | (S121).

  The target sound convolution auto encoder unit 122 estimates an estimated target sound logarithm that is an amplitude spectrum in a logarithmic frequency domain of a predetermined instrument sound (target sound) from the music signal log amplitude spectrum | X (ω ′, τ) | calculated in S121. The amplitude spectrum | ^ S (ω ', τ) | is calculated (S122).

  Similarly, from the music signal log amplitude spectrum | X (ω ′, τ) | calculated in S121, the noise convolution auto encoder unit 222 calculates the logarithmic frequency of noise that is all sounds other than a predetermined instrument sound (target sound). An estimated noise log amplitude spectrum | ^ N (ω ′, τ) |, which is an amplitude spectrum in the region, is calculated (S222). The noise convolution auto encoder unit 222 receives as input the amplitude spectrum of the music signal in the logarithmic frequency domain, and is noise that is all sounds other than a predetermined musical instrument sound (which is a part of the music signal) in the logarithmic frequency domain A DNN including a 2n-layer CDAE that outputs an amplitude spectrum of Further, in the noise convolution auto encoder unit 222, network parameters at the end of DNN learning (that is, network parameters recorded in the learning result recording unit 990) are set. Specifically, the noise convolution auto encoder unit 222 becomes a DNN including 2n-layer CDAEs for calculating the equations (7) and (8) as described in <Technical background>. The learning of the network parameters of DNN will be described later.

  The log-linear frequency conversion unit 123 performs frequency domain conversion on the estimated target sound log amplitude spectrum | ^ S (ω ′, τ) | calculated in S122, and estimates the estimated target sound amplitude spectrum | ^ S (ω, τ) | The estimated noise log amplitude spectrum | ^ N (ω ′, τ) | calculated in S222 is frequency domain transformed, and the estimated noise amplitude spectrum | ^ N (ω, τ) | is calculated (S123). The estimated noise amplitude spectrum | ^ N (ω, τ) | is an estimated frequency domain noise ^ N (ω) obtained by converting the estimated noise log amplitude spectrum | ^ N (ω ′, τ) | from the logarithmic frequency domain to the frequency domain. , τ) is an amplitude spectrum. Note that instead of configuring the Wiener filter estimation unit 320 to have two log-linear frequency conversion units 123 as shown in FIG. 7, the Wiener filter estimation unit 320 may have one log-linear frequency conversion unit 123. And the estimated noise amplitude spectrum | ^ N (ω, τ) | may be calculated by one log-linear frequency conversion unit 123. .

  The Wiener filter calculation unit 224 calculates the Wiener filter ^ W (ω, τ) from the estimated target sound amplitude spectrum | ^ S (ω, τ) | and the estimated noise amplitude spectrum | ^ N (ω, τ) | calculated in S123. Calculate (S124). Specifically, the Wiener filter ^ W (ω, τ) is calculated by the following equation.

  According to the invention of the present embodiment, it is possible to emphasize a specific musical tone of a music signal composed of a plurality of musical instrument sounds with high accuracy. The invention of the present embodiment uses the spectral characteristics of the instrument sound (that is, the spectrum is decomposed into harmonic-envelope components) by constructing a DNN including 2 n-layer CDAEs having a logarithmic frequency spectrum as an input. It is possible to estimate the target sound spectrum and noise spectrum. Further, this makes it possible to estimate the Wiener filter with high accuracy even if the calculation scale of DNN is limited, and as a result, it is possible to enhance the musical tone with high accuracy.

  In particular, since the complementarity in the amplitude domain holds among the target sound, the noise, and the observation sound, the estimated target sound amplitude spectrum | ^ S (ω, τ) | and the estimated noise amplitude spectrum | ^ N (ω, τ) | Even if one of the estimation accuracy is poor, the other estimation accuracy is considered to be good, and therefore, even in such a case, the Wiener filter ^ (ω, τ) can be estimated with high accuracy.

(Modification)
As described in the modification of the first embodiment, the Wiener filter estimation unit 320 calculates the order of the log-linear frequency conversion and the Wiener filter calculation to estimate the Wiener filter, but the calculation order may be reversed. Here, such a Wiener filter estimation unit 420 will be described. In this case, the musical tone emphasizing device 100 includes a frequency domain converting unit 910, a Wiener filter estimating unit 420, a signal emphasizing unit 930, and a time domain converting unit 940 (see FIGS. 1 and 2).

  Hereinafter, the Wiener filter estimation unit 420 will be described with reference to FIGS. 9 to 10. FIG. 9 is a block diagram showing the configuration of the Wiener filter estimation unit 420. As shown in FIG. FIG. 10 is a flowchart showing the operation of the Wiener filter estimation unit 420. As shown in FIG. 9, the Wiener filter estimation unit 420 includes a logarithmic amplitude spectrum calculation unit 121, a target sound convolution auto encoder unit 122, a noise convolution auto encoder unit 222, a logarithmic Wiener filter calculation unit 225, and log-linear frequency conversion. Part 123 is included.

  The logarithmic amplitude spectrum calculation unit 121 performs logarithmic frequency domain conversion on the frequency domain music signal X (ω, τ) and calculates music signal logarithmic amplitude spectrum | X (ω ′, τ) | (S121).

  The target sound convolution auto encoder unit 122 estimates an estimated target sound logarithm that is an amplitude spectrum in a logarithmic frequency domain of a predetermined instrument sound (target sound) from the music signal log amplitude spectrum | X (ω ′, τ) | calculated in S121. The amplitude spectrum | ^ S (ω ', τ) | is calculated (S122).

  Similarly, from the music signal log amplitude spectrum | X (ω ′, τ) | calculated in S121, the noise convolution auto encoder unit 222 calculates the logarithmic frequency of noise that is all sounds other than a predetermined instrument sound (target sound). An estimated noise log amplitude spectrum | ^ N (ω ′, τ) |, which is an amplitude spectrum in the region, is calculated (S222).

  The logarithmic Wiener filter calculation unit 225 calculates the estimated target sound log amplitude spectrum | ^ S (ω ′, τ) | calculated in S122 and the estimated noise log amplitude spectrum | ^ N (ω ′, τ) | calculated in S222 The Wiener filter ^ W (ω ', τ) is calculated (S225). Specifically, the logarithmic Wiener filter ^ W (ω ', τ) is calculated by the following equation.

  The log-linear frequency conversion unit 123 performs frequency domain conversion on the logarithmic Wiener filter ^ W (ω ', τ) calculated in S225, and calculates a Wiener filter ^ W (ω, τ) (S123).

(Modification 2)
In the above description, DNN is configured to have a noise other than the tone that is the purpose of sound source enhancement as one noise, but each tone other than the tone that is the purpose of sound source enhancement is noise. A DNN including 2 n-layer CDAEs may be configured to estimate a spectrum in the logarithmic frequency domain. In this case, the tone enhancing apparatus 100 includes the same number of noise convolution auto encoders 222 as the number of tones (number of musical instruments) other than the tone to be emphasized in the tone generator. The network parameters set by the noise convolution auto encoder unit 222 are generally different.

Third Embodiment
Here, a convolution auto encoder learning device 500 will be described which learns network parameters of DNNs constituting the target sound convolution auto encoder unit 122 and the noise convolution auto encoder unit 222.

  The convolutional auto encoder learning device 500 will be described below with reference to FIGS. FIG. 11 is a block diagram showing the configuration of the convolutional auto encoder learning device 500. As shown in FIG. FIG. 12 is a flowchart showing the operation of the convolutional auto encoder learning device 500. As shown in FIG. 11, the convolutional auto encoder learning device 500 includes a logarithmic amplitude spectrum calculator 121, a convolution auto encoder calculator 510, and a network parameter optimizer 520.

  The convolutional automatic encoder learning device 500 is connected to the learning data recording unit 590. The learning data recording unit 590 includes a frequency domain music signal X (ω, τ) which is a monaural mixed music signal in a frequency domain used for learning and a frequency domain partial music signal Y (ω, τ) which is a partial signal thereof. Are recorded as learning data. The frequency domain partial music signal Y (ω, τ) is, for example, a frequency domain target sound S (ω, τ) which is a musical tone for which the sound source is to be emphasized constituting the frequency domain music signal X (ω, τ). A frequency domain noise N (ω, τ) which is a sound other than a musical tone which is the purpose of sound source emphasis constituting the music signal X (ω, τ).

  The convolutional automatic encoder learning device 500 receives a set of frequency domain music signal X (ω, τ) and frequency domain partial music signal Y (ω, τ) as learning data, and uses the music signal logarithmic amplitude spectrum as input to estimate partial music Output network parameters of DNN including 2 n-layer CDAE that output signal logarithmic amplitude spectrum. That is, the convolutional automatic encoder learning device 500 learns the target sound by learning the frequency domain target sound S (ω, τ) and the frequency domain noise N (ω, τ) as the frequency domain partial music signal Y (ω, τ). The network parameters of DNNs constituting the convolution auto encoder unit 122 and the noise convolution auto encoder unit 222 can be learned.

  Logarithmic amplitude spectrum calculation unit 121 performs logarithmic frequency domain conversion of frequency domain music signal X (ω, τ), and calculates music signal logarithmic amplitude spectrum | X (ω ′, τ) | Logarithmic frequency domain conversion of ω, τ) and partial music signal logarithmic amplitude spectrum | Y (ω ′, τ) | is calculated (S121).

  The convolutional auto encoder calculation unit 510 calculates the music signal logarithmic amplitude spectrum | calculated in S121 using DNN including the CDAE of the 2n (n is an integer of 1 or more) layer in which the network parameters currently being learned (optimized) are set. The estimated partial music signal log amplitude spectrum | ^ Y (ω ′, τ) | is calculated from X (ω ′, τ) | (S510).

  The network parameter optimization unit 520 calculates the partial music signal log amplitude spectrum | Y (ω ′, τ) | calculated in S121 and the estimated partial music signal log amplitude spectrum | ^ Y (ω ′, τ) | calculated in S510. The network parameters used in S510 are optimized (S520). For optimization of the network parameters, for example, an error back propagation method using a sum of squares error of | Y (ω ′, τ) | and | ^ Y (ω ′, τ) | may be used.

  The processes of S121 to S520 are repeated by the number of learning data. Alternatively, the learning termination condition may be set, the learning may be stopped where appropriate, and the network parameter may be output.

  Thus, network parameters used in the target sound convolution auto encoder unit 122 and the noise convolution auto encoder unit 222 are learned.

  According to the invention of this embodiment, it is possible to learn network parameters of DNNs reflecting characteristics of musical instrument sounds, that is, DNNs including 2 n-layer CDAEs. According to the invention of the first embodiment and the second embodiment using the network parameters obtained by this learning, it becomes possible to emphasize a specific musical tone of a music signal composed of a plurality of musical instrument sounds with high accuracy.

<Supplementary Note>
The apparatus according to the present invention is, for example, an input unit to which a keyboard or the like can be connected, an output unit to which a liquid crystal display or the like can be connected as a single hardware entity, or a communication apparatus (eg, communication cable) capable of communicating outside the hardware entity. Communication unit that can be connected, CPU (central processing unit, cache memory, registers, etc. may be provided), RAM or ROM that is memory, external storage device that is hard disk, input unit for these, output unit, communication unit , CPU, RAM, ROM, and a bus connected so as to enable exchange of data between external storage devices. If necessary, the hardware entity may be provided with a device (drive) capable of reading and writing a recording medium such as a CD-ROM. Examples of physical entities provided with such hardware resources include general purpose computers.

  The external storage device of the hardware entity stores a program necessary for realizing the above-mentioned function, data required for processing the program, and the like (not limited to the external storage device, for example, the program is read) It may be stored in the ROM which is a dedicated storage device). In addition, data and the like obtained by the processing of these programs are appropriately stored in a RAM, an external storage device, and the like.

  In the hardware entity, each program stored in the external storage device (or ROM etc.) and data necessary for processing of each program are read into the memory as necessary, and interpreted and processed appropriately by the CPU . As a result, the CPU realizes predetermined functions (each component requirement expressed as the above-mentioned,...

  The present invention is not limited to the above-described embodiment, and various modifications can be made without departing from the spirit of the present invention. Further, the processing described in the above embodiment may be performed not only in chronological order according to the order of description but also may be performed in parallel or individually depending on the processing capability of the device that executes the processing or the necessity. .

  As described above, when the processing function in the hardware entity (the apparatus of the present invention) described in the above embodiment is implemented by a computer, the processing content of the function that the hardware entity should have is described by a program. Then, by executing this program on a computer, the processing function of the hardware entity is realized on the computer.

  The program describing the processing content can be recorded in a computer readable recording medium. As the computer readable recording medium, any medium such as a magnetic recording device, an optical disc, a magneto-optical recording medium, a semiconductor memory, etc. may be used. Specifically, for example, as a magnetic recording device, a hard disk device, a flexible disk, a magnetic tape or the like as an optical disk, a DVD (Digital Versatile Disc), a DVD-RAM (Random Access Memory), a CD-ROM (Compact Disc Read Only) Memory), CD-R (Recordable) / RW (Rewritable), etc. as magneto-optical recording medium, MO (Magneto-Optical disc) etc., as semiconductor memory EEP-ROM (Electronically Erasable and Programmable Only Read Memory) etc. Can be used.

  Further, this program is distributed, for example, by selling, transferring, lending, etc. a portable recording medium such as a DVD, a CD-ROM or the like in which the program is recorded. Furthermore, this program may be stored in a storage device of a server computer, and the program may be distributed by transferring the program from the server computer to another computer via a network.

  For example, a computer that executes such a program first temporarily stores a program recorded on a portable recording medium or a program transferred from a server computer in its own storage device. Then, at the time of execution of the process, the computer reads the program stored in its own recording medium and executes the process according to the read program. Further, as another execution form of this program, the computer may read the program directly from the portable recording medium and execute processing according to the program, and further, the program is transferred from the server computer to this computer Each time, processing according to the received program may be executed sequentially. In addition, a configuration in which the above-described processing is executed by a so-called ASP (Application Service Provider) type service that realizes processing functions only by executing instructions and acquiring results from the server computer without transferring the program to the computer It may be Note that the program in the present embodiment includes information provided for processing by a computer that conforms to the program (such as data that is not a direct command to the computer but has a property that defines the processing of the computer).

  Further, in this embodiment, the hardware entity is configured by executing a predetermined program on a computer, but at least a part of the processing content may be realized as hardware.

Claims (8)

A frequency domain conversion unit that frequency domain converts the time domain music signal x (t), which is a monaural mixed music signal, to generate a frequency domain music signal X (ω, τ);
A Wiener filter estimation unit for estimating a Wiener filter ^ W (ω, τ) used to emphasize a predetermined musical instrument sound from the frequency domain music signal X (ω, τ);
A signal emphasizing unit for generating a frequency domain emphasized tone S '(ω, τ) from the frequency domain music signal X (ω, τ) and the Wiener filter ^ W (ω, τ);
A time domain conversion unit for generating an enhanced tone s '(t) in the time domain from the frequency domain enhanced tone S' (. Omega., .Tau.);
The Wiener filter estimation unit
A 2 n-layer (n is an integer greater than or equal to 1) convolutional denoising auto encoder that receives the amplitude spectrum of a music signal in the logarithmic frequency domain and outputs the amplitude spectrum of a part of the music signal in the logarithmic frequency domain A tone emphasizing device for estimating the Wiener filter ^ W (ω, τ) from the frequency domain music signal X (ω, τ) using a deep neural network including The musical tone emphasizing device according to claim 1, wherein
The Wiener filter estimation unit
A logarithmic amplitude spectrum calculation unit that performs logarithmic frequency domain conversion on the frequency domain music signal X (ω, τ) and calculates music signal logarithmic amplitude spectrum | X (ω ′, τ) |
A deep including 2n layers (n is an integer greater than or equal to 1) convolutional denoising auto encoders that receive the amplitude spectrum of a music signal in the logarithmic frequency domain and output the amplitude spectrum of a predetermined instrument sound in the logarithmic frequency domain Estimated target sound logarithmic amplitude spectrum | ^ S (ω ′, τ) which is an amplitude spectrum in the logarithmic frequency domain of the predetermined musical instrument sound from the music signal logarithmic amplitude spectrum | X (ω ′, τ) | using a neural network A target sound convolution auto encoder unit that calculates |),
Logarithmic-linear frequency conversion unit that performs frequency domain conversion on the estimated target sound log amplitude spectrum | ^ S (ω ′, τ) | and calculates an estimated target sound amplitude spectrum | ^ S (ω, τ) |
The music signal amplitude spectrum | X (ω, τ) |, which is the amplitude spectrum of the frequency domain music signal X (ω, τ), and the estimated target sound amplitude spectrum | ^ S (ω, τ) | and a Wiener filter calculation unit for calculating (ω, τ). The musical tone emphasizing device according to claim 1, wherein
The Wiener filter estimation unit
A logarithmic amplitude spectrum calculation unit that performs logarithmic frequency domain conversion on the frequency domain music signal X (ω, τ) and calculates music signal logarithmic amplitude spectrum | X (ω ′, τ) |
A deep including 2n layers (n is an integer greater than or equal to 1) convolutional denoising auto encoders that receive the amplitude spectrum of a music signal in the logarithmic frequency domain and output the amplitude spectrum of a predetermined instrument sound in the logarithmic frequency domain Estimated target sound logarithmic amplitude spectrum | ^ S (ω ′, τ) which is an amplitude spectrum in the logarithmic frequency domain of the predetermined musical instrument sound from the music signal logarithmic amplitude spectrum | X (ω ′, τ) | using a neural network A target sound convolution auto encoder unit that calculates |),
A convolution of 2n layers (n is an integer greater than or equal to 1) that receives as input the amplitude spectrum of a music signal in the logarithmic frequency domain and outputs an amplitude spectrum of noise that is all sounds other than a predetermined musical instrument sound in the logarithmic frequency domain Amplitude spectrum in the logarithmic frequency domain of noise which is all sounds other than the predetermined musical instrument sound from the music signal logarithmic amplitude spectrum | X (ω ′, τ) | using a deep neural network including a denoising auto encoder A noise convolution auto encoder unit that calculates an estimated noise log amplitude spectrum | ^ N (ω ′, τ) |
Is frequency domain transformed to calculate the estimated target sound amplitude spectrum | ^ S (ω, τ) |, and the estimated noise log amplitude spectrum | ^ N a log-linear frequency conversion unit that performs frequency domain conversion of (ω ′, τ) | and calculates an estimated noise amplitude spectrum | ^ N (ω, τ) |
A Wiener filter calculator for calculating the Wiener filter ^ W (ω, τ) from the estimated target sound amplitude spectrum | ^ S (ω, τ) | and the estimated noise amplitude spectrum | ^ N (ω, τ) | Tone emphasis device including. The musical tone emphasizing device according to claim 1, wherein
The Wiener filter estimation unit
A logarithmic amplitude spectrum calculation unit that performs logarithmic frequency domain conversion on the frequency domain music signal X (ω, τ) and calculates music signal logarithmic amplitude spectrum | X (ω ′, τ) |
A deep including 2n layers (n is an integer greater than or equal to 1) convolutional denoising auto encoders that receive the amplitude spectrum of a music signal in the logarithmic frequency domain and output the amplitude spectrum of a predetermined instrument sound in the logarithmic frequency domain Estimated target sound logarithmic amplitude spectrum | ^ S (ω ′, τ) which is an amplitude spectrum in the logarithmic frequency domain of the predetermined musical instrument sound from the music signal logarithmic amplitude spectrum | X (ω ′, τ) | using a neural network A target sound convolution auto encoder unit that calculates |),
Log Wiener calculating log Wiener filter ^ W (ω ', τ) from the music signal log amplitude spectrum | X (ω', τ) | and the estimated target sound log amplitude spectrum | ^ S (ω ', τ) | A filter calculation unit,
A tone-to-linear frequency conversion unit that performs frequency domain conversion on the logarithmic Wiener filter ^ W (ω ', τ) and calculates the Wiener filter ^ W (ω, τ). The musical tone emphasizing device according to claim 1, wherein
The Wiener filter estimation unit
A logarithmic amplitude spectrum calculation unit that performs logarithmic frequency domain conversion on the frequency domain music signal X (ω, τ) and calculates music signal logarithmic amplitude spectrum | X (ω ′, τ) |
A deep including 2n layers (n is an integer greater than or equal to 1) convolutional denoising auto encoders that receive the amplitude spectrum of a music signal in the logarithmic frequency domain and output the amplitude spectrum of a predetermined instrument sound in the logarithmic frequency domain Estimated target sound logarithmic amplitude spectrum | ^ S (ω ′, τ) which is an amplitude spectrum in the logarithmic frequency domain of the predetermined musical instrument sound from the music signal logarithmic amplitude spectrum | X (ω ′, τ) | using a neural network A target sound convolution auto encoder unit that calculates |),
A convolution of 2n layers (n is an integer greater than or equal to 1) that receives as input the amplitude spectrum of a music signal in the logarithmic frequency domain and outputs an amplitude spectrum of noise that is all sounds other than a predetermined musical instrument sound in the logarithmic frequency domain Amplitude spectrum in the logarithmic frequency domain of noise which is all sounds other than the predetermined musical instrument sound from the music signal logarithmic amplitude spectrum | X (ω ′, τ) | using a deep neural network including a denoising auto encoder A noise convolution auto encoder unit that calculates an estimated noise log amplitude spectrum | ^ N (ω ′, τ) |
Log to calculate a logarithmic Wiener filter ^ W (ω ', τ) from the estimated target sound log amplitude spectrum | ^ S (ω', τ) | and the estimated noise log amplitude spectrum | ^ N (ω ', τ) | Wiener filter calculation unit,
A tone-to-linear frequency conversion unit that performs frequency domain conversion on the logarithmic Wiener filter ^ W (ω ', τ) and calculates the Wiener filter ^ W (ω, τ). The frequency domain music signal X (ω, τ), which is a monaural mixed music signal in the frequency domain, is logarithmic frequency domain transformed, and the music signal log amplitude spectrum | X (ω ′, τ) | is calculated. Logarithmic frequency domain conversion of frequency domain partial music signal Y (ω, τ), which is a part of X (ω, τ), to calculate partial music signal logarithmic amplitude spectrum | Y (ω ', τ) | An amplitude spectrum calculation unit,
The music signal log amplitude spectrum | X (ω ′, τ) using a deep neural network including a 2 n layer (n is an integer of 1 or more) convolutional denoising auto encoder in which the network parameter currently being optimized is set A convolutional auto-encoder calculating unit that calculates an estimated partial music signal log amplitude spectrum | ^ Y (ω ′, τ) | from |
Network parameter optimization for optimizing the network parameters using the partial music signal log amplitude spectrum | Y (ω ′, τ) | and the estimated partial music signal log amplitude spectrum | ^ Y (ω ′, τ) | A convolutional auto encoder learning device including a part and a. A frequency domain conversion step of the musical tone emphasizing device frequency domain converting the time domain music signal x (t) which is a monaural mixed music signal to generate a frequency domain music signal X (ω, τ);
A Wiener filter estimation step of estimating a Wiener filter ^ W (ω, τ) used by the musical tone emphasizing device to emphasize a predetermined musical instrument sound from the frequency domain music signal X (ω, τ);
A signal emphasizing step in which the musical tone emphasizing device generates a frequency domain emphasized musical tone S '(ω, τ) from the frequency domain music signal X (ω, τ) and the Wiener filter ^ W (ω, τ);
A tone emphasizing method comprising: a time domain conversion step of the tone emphasizing device generating an emphasized tone s '(t) in a time domain from the frequency domain emphasized tone S' (ω, τ),
The Wiener filter estimation step is
A 2 n-layer (n is an integer greater than or equal to 1) convolutional denoising auto encoder that receives the amplitude spectrum of a music signal in the logarithmic frequency domain and outputs the amplitude spectrum of a part of the music signal in the logarithmic frequency domain A musical tone emphasizing method for estimating the Wiener filter ^ W (ω, τ) from the frequency domain music signal X (ω, τ) using a deep neural network including   A program for causing a computer to function as the musical tone emphasizing device according to any one of claims 1 to 5 or the convolution auto encoder learning device according to claim 6.

Images