JP2019035888A - Sound signal generation device, sound signal generation method, and program - Google Patents

Abstract

To provide a sound signal generation technique for generating a sound having an arbitrary texture of general sound from white noise.SOLUTION: The sound signal generation device includes a first sound feature extraction unit for extracting a first sound feature which is a feature quantity not dependent on time from an input first sound, a second sound feature extraction unit for extracting a second sound feature which is a feature quantity dependent on time from an input second sound, a white signal generation unit for initializing the generated sound to be output with white noise, and a signal deformation unit for outputting a deformed generated sound using the first sound feature and the second sound feature. The signal deformation unit includes a generated sound feature extraction unit for extracting a first generated sound feature which is a feature quantity not dependent on time from the generated sound and a second generated sound feature which is a feature quantity dependent on time, and an error evaluation signal deformation unit which deforms the generated sound so that a feature error calculated from a first feature error which is the error between the first generated sound feature and the first sound feature and a second feature error which is the error between the second generated sound feature and the second sound feature is small.SELECTED DRAWING: Figure 2

Description

  The present invention relates to a technique for obtaining a sound signal having features of both input two sound signals.

  Research on technology for generating sound with a desired texture is underway. An example of such a technique is voice quality conversion as described in Non-Patent Document 1. Numerous studies have been conducted on this voice quality conversion.

  As another example, Non-Patent Document 2 proposes a method for texture synthesis of sound having a certain texture. In this method, a feature amount representing a texture is extracted from the input sound, and a new sound having the same texture is synthesized using the feature amount.

T. Toda, AW Black, and K. Tokuda, “Spectral Conversion Based on Maximum Likelihood Estimation Considering Global Variance of Converted Parameter”, IEEE International Conference on Acoustic, Speech, and Signal Processing 2005 (ICASSP '05) Proceedings, vol.1 , no.1, pp.9-12, 2005. J. H. McDermott and E. P. Simoncelli, “Sound texture perception via statistics of the auditory periphery: Evidence from sound synthesis”, Neuron, vol.71, no.5, pp.926-940, 2011.

  However, in the voice quality conversion technique of Non-Patent Document 1, the texture that is the conversion target is limited to the texture of a human voice that actually exists. Non-Patent Document 2 targets not only voice quality but general sound texture, but not texture conversion but texture synthesis.

  In other words, there has been no technique for texture conversion of general sound into an arbitrary texture such as a texture other than voice quality (for example, an environment in which the voice is uttered) and a sound texture other than voice.

  Therefore, an object of the present invention is to provide a sound signal generation technique for generating a sound having an arbitrary texture of a general sound from white noise.

  One aspect of the present invention includes a first sound feature extraction unit that extracts a first sound feature that is a feature quantity that does not depend on time from the input first sound, and a feature quantity that depends on time from the input second sound. A second sound feature extracting unit that extracts the second sound feature, a white signal generating unit that initializes a generated sound to be output with white noise, and a modification using the first sound feature and the second sound feature A signal generating unit that outputs the generated sound, and the signal deforming unit includes a first generated sound feature that is a feature quantity independent of time from the generated sound and a second generated sound that is a feature quantity dependent on time. A generated sound feature extracting unit for extracting features; a first characteristic error that is an error between the first generated sound feature and the first sound feature; and an error between the second generated sound feature and the second sound feature. An error that deforms the generated sound so that the characteristic error calculated from the second characteristic error is small. And a value signal modifying unit.

  According to the present invention, it is possible to generate a sound having an arbitrary texture of a general sound from white noise.

The table | surface which shows the parameter used with a sound signal generation algorithm. 1 is a block diagram showing an example of the configuration of a sound signal generation device 100. FIG. 5 is a flowchart showing an example of the operation of the sound signal generation device 100.

  Hereinafter, embodiments of the present invention will be described in detail. In addition, the same number is attached | subjected to the structure part which has the same function, and duplication description is abbreviate | omitted.

  Prior to the description of the embodiment, a description method in this specification will be described.

_ (Underscore) represents a subscript. For example, ^xy_z represents that _yz is a superscript to x, and _{xy_z} represents that _yz is a subscript to x.

<Technical background>
Here, the sound signal generation algorithm used in the embodiment of the present invention will be described. First, the outline will be described.

(Outline of sound signal generation algorithm)
(1) Input a sound whose content is to be converted (hereinafter referred to as a second sound) and a sound having a conversion target texture (hereinafter referred to as a first sound).
(2) A feature amount (first sound feature) representing the texture of the first sound is extracted from the first sound. As the first sound feature, the time periphery statistic of the output waveform of the auditory system model is used. Here, the time-peripheral statistic is a statistic that is marginalized in the dimension of time. For example, an average, variance, skewness, and correlation coefficient are used as the time-peripheral statistic. Therefore, the time periphery statistic is a feature quantity that does not depend on time.
(3) A feature amount (second sound feature) representing the content of the second sound is extracted from the second sound. As the second sound feature, the time pattern of the output waveform of the auditory system model is used. Here, as the time pattern, the output waveform itself, that is, the amplitude envelope and the band obtained by dividing the amplitude envelope are used. Therefore, the time pattern is a feature quantity dependent on time.
(4) The generated sound is initialized with white noise.
(5) The feature amount (first generated sound feature) representing the texture of the generated sound approaches the first sound feature, and the feature amount (second generated sound feature) representing the content of the generated sound approaches the second sound feature. , Transform the generated sound.
(6) Output the generated sound after deformation.

Here, the auditory system model is a process for simulating a human auditory system on an input sound. The auditory system model first divides a sound into a plurality of bands, and extracts its amplitude envelope and fine structure. Next, in order to simulate the nonlinearity of the response in the auditory system, the amplitude envelope is nonlinearly compressed. Finally, in order to reflect the fluctuation speed of the amplitude envelope for each band, a waveform obtained by dividing the amplitude envelope into bands is calculated. That is, the auditory system model is composed of the following four processes.
(a) Bandpass filter bank
(b) Envelope extraction
(c) Nonlinear compression
(d) Bandpass filter bank

  In addition, the auditory system model takes the amplitude value x (t) of the sound as input, the waveform s (i, t) for each band, the amplitude envelope e (i, t) of the waveform s (i, t), and the waveform s ( Four output waveforms of a waveform m (i, k, t) obtained by dividing a fine structure p (i, t) of i, t) and an amplitude envelope e (i, t) are output. From these output waveforms, the time periphery statistics and the time pattern are calculated in the processes (2) and (3).

  Next, the sound signal generation algorithm will be described.

(Sound signal generation algorithm)
Processing content: The first sound and the second sound are input, and a generated sound that is a sound having both characteristics is output.
Input: first sound amplitude value x _tex (t), second sound amplitude value x _con (t)
Output: Generated sound amplitude value x _gen (t)

Here, t is a parameter representing discrete time and is assumed to be t∈ {1, 2,..., T _{x_type} }. However, typeε {tex, con, gen}. That is, T _{x_tex} represents the length of the first sound (number of samples), T _{x_con} represents the length of the second sound (number of samples), and T _{x_gen} represents the length of the generated sound (number of samples). Further, T _{x_con} = T _{x_gen} .

Note that the calculation time of this algorithm increases linearly with respect to T _{x_con} .
(1) by the auditory system model, from the amplitude value of the first sound x _tex (t), for each band waveform s _tex (i, t), amplitude envelope e _tex waveform _{s tex (i, t) (} i, t ), A waveform m _tex (i, k, t) obtained by band-dividing the amplitude envelope e _tex (i, t) is calculated. The auditory system model will be described later. Here, i, k is a parameter representing a band number, i∈ {1, 2, ... , N i}, k∈ {1, 2, ..., N k} and.
(2) Time marginal statistics f _{mean, tex} (i), f _{var, tex} (i), f _{skew, tex} (i) of amplitude envelope e _tex (i, t) and waveform m _tex (i, k, t) ), F _{cce, tex} (i, j), f _{pow, tex} (i, k), and f _{ccm, tex} (i, j, k). A method for calculating the statistics around each time will be described later.
(3) The variance σ _{s_tex} (i) ² of the waveform s _tex (i, t) is calculated.

(4) Using the auditory system model, the amplitude envelope e _con (i, t) and the amplitude envelope e _con (i, t) of the waveform s _con (i, t) are _calculated from the amplitude value x _con (t) of the second sound. Calculate the band-divided waveform m _con (i, k, t).
(5) The generated sound x _gen (t) is initialized with white noise. That is, x _gen (t) ← white noise.
(6) Repeat (6.1) to (6.11) _Niter times. N _iter is an integer greater than or _equal to 1, for example 20 (see FIG. 1).
(6.1) From the amplitude value x _gen (t) of the generated sound, the amplitude envelope e _gen (i, t) of the waveform s _gen (i, t) and waveform s _gen (i, t) for each band is determined by the auditory system model , waveform s _gen (i, t) of the microstructure p _gen (i, t), amplitude envelope e _gen (i, t) to band divided waveform _{m gen (i, k, t} ) is calculated.
(6.2) Time marginal statistics f _{mean, gen} (i), f _{var, gen} (i), f _{skew, gen} (i) of amplitude envelope e _gen (i, t) and waveform m _gen (i, k, t) ), F _{cce, gen} (i, j), f _{pow, gen} (i, k), f _{ccm, gen} (i, j, k).
(6.3) The feature error is L (i) = αL _con (i) + (1-α) L _tex (i), and the second feature error L _con (i) and the first feature error L _tex (i) are respectively Calculate as follows. Here, α (0 ≦ α ≦ 1) is a parameter that determines the balance of the influence of the second sound and the first sound on the generated sound.

When α = 0, a sound having the first sound feature and not having the second sound feature is synthesized. Also, when α = 1, noise-driven speech is synthesized.
(6.3.1) The second characteristic error L _con (i) is divided into bands of the amplitude envelope e _con (i, t) and the amplitude envelope e _con (i, t) of the second sound expressed by the following equation: The waveform m _con (i, k, t) and the waveform m _gen (i, k, t) obtained by band-dividing the amplitude envelope e _gen (i, t) and the amplitude envelope e _gen (i, t) of the generated sound Let squared error.

(6.3.2) The first characteristic error L _tex (i) is a square error between the time-peripheral statistic of the first sound and the time-peripheral statistic of the generated sound expressed by the following equation.

However, Σ _{j, k} is executed only for stats that take j and k as arguments. Specifically, it is executed for statε {cce, pow, ccm}.
(6.4) The amplitude envelope e _gen (i, t) is modified by the stochastic gradient method so that the characteristic error L (i) becomes smaller in order from i with the largest variance σ _{s_tex} (i) ² . Let the deformed amplitude envelope be e ′ _gen (i, t).
(6.5) The inverse operation of nonlinear compression in the auditory system model is performed on the amplitude envelope e ′ _gen (i, t). That is, an amplitude envelope e ′ _gen (i, t) raised to the power of β is re- _{designated as} e ′ _gen (i, t).

(6.6) The amplitude envelope e ′ _gen (i, t) is up-sampled from the sampling frequency f _e of the amplitude envelope to the sampling frequency f _{x of the} amplitude value. For example, f _x = 20 kHz and f _e = 400 Hz are preferable (see FIG. 1). _Let e ′ _gen (i, t) be the _upsampled amplitude envelope.
(6.7) The waveform s ′ _gen (i, t) is restored from the amplitude envelope e ′ _gen (i, t) and the fine structure p _gen (i, t) by the following equation.

However, j is an imaginary unit.
(6.8) 'so that the variance of the _gen (i, t) is equal to the variance of the waveform s _tex (i, t), as in the following equation, the waveform s' waveform s _gen (i, t) to the sigma _{S_tex} ( i) _Multiply / σ _{s'_gen} (i). In other words, the waveform s ′ _gen (i, t) _multiplied by σ _{s_tex} (i) / σ _{s′_gen} (i) is changed to s ′ _gen (i, t).

Here, σ _{s′_gen} (i) is a standard deviation of the waveform s ′ _gen (i, t), and is expressed by the following equation.

(6.9) Apply the bandpass filter bank F _a (i) to the waveform s ′ _gen (i, t). _Let s' _gen (i, t) be the waveform after application. The bandpass filter F _a (i) will be described later.
(6.10) The waveform s ′ _gen (i, t) is added according to the following equation to calculate the amplitude value x ′ _gen (t).

(6.11) Let x _gen (t) ← x ' _gen (t).
(Hearing system model)
In the auditory system model, as described above, a process simulating the human auditory system is performed on the input sound.

In the following, for simplicity, type ∈ {tex, con, gen} and x _type (t), s _type (i, t), e _type (i, t), p _type (i, t), m _type ( i, k, t) are represented as x (t), s (i, t), e (i, t), p (i, t), m (i, k, t), respectively.
Input: Sound amplitude value x (t)
Output: waveform s (i, t) for each band, amplitude envelope e (i, t) of waveform s (i, t), fine structure p (i, t) of waveform s (i, t), amplitude envelope e Waveform m (i, k, t) obtained by band dividing (i, t)
(1) The amplitude value x (t) is divided into bands from the bandpass filter bank F _a (i). The center frequency c _a (i) of the filter is set to be equally spaced on the Equivalent rectangular bandwidth (ERB) scale. The ERB scale is a scale that simulates the arrangement of filters in the cochlea. Here, it is expressed as erb (fω). Where f is the sampling frequency and ω is the normalized frequency. The bandwidth of the i-th band is set so that the points at which the filter coefficient attenuates to 0 become c _a (i−1) and c _a (i + 1).
(1.1) Let X (ω) be the discrete Fourier transform of the amplitude value x (t). ω is a normalized frequency, and ω∈ {0, 1 / T _x , 2 / T _x , ..., 1/2}.
(1.2) Multiply X (ω) by the filter coefficient F _a (i, ω). The result is S (i, ω).

However, f _x is the sampling frequency of the amplitude value x (t), N _i is the number of bands of the band-pass filter bank F _a (i), ω _a0 bandpass filter bank F _a minimum bandwidth of the center frequency of (i) ( Ω _a1 is the center frequency of the highest band (maximum center frequency) of the bandpass filter bank F _a (i). For _{example, f x = 20kHz, N i} = 30, ω a0 = 20 / f x, or equal to ω _a1 = 10000 / f _x (see FIG. 1).
(1.3) S (i, ω) is a discrete inverse Fourier transform, which is a waveform s (i, t) for each band.
(2) The amplitude envelope e (i, t) and fine structure p (i, t) of the waveform s (i, t) are calculated by the Hilbert transform. That is, the amplitude envelope e (i, t) and the fine structure p (i, t) are set as the absolute value and the argument of the Hilbert transform of the waveform s (i, t), respectively.
(3) Non-linearly compress the amplitude envelope e (i, t). That is, the amplitude envelope e (i, t) raised to the power of β is re-designated as e (i, t).

However, β (0 <β ≦ 1) is a parameter that determines the degree of compression. For example, β = 0.3 is preferable (see FIG. 1).
(4) The amplitude envelope e (i, t) is down-sampled from the sampling frequency f _x of the amplitude value to the sampling frequency f _{e of the} amplitude envelope. By down-sampling, t∈ {1, 2,…, T _e }. However, T _e is the length of the amplitude envelope e (i, t) (the number of samples), a _{T e / f e = T x} / f x. Let the down-sampled amplitude envelope be e (i, t) again.
(5) The amplitude envelope e (i, t) is divided into bands by the bandpass filter bank F _m (k). This band pass filter bank F _m (k) is assumed to be a modulation filter bank that is considered to exist in the auditory peripheral system. The interval between the center frequencies of the filters is the log scale, and the bandwidth is set so that the sharpness (Q value) is 2.
(5.1) Let E (i, ω) be the discrete Fourier transform of the amplitude envelope e (i, t). ω is a normalized frequency, and ω∈ {0, 1 / T _e , 2 / T _e , ..., 1/2}.
(5.2) Multiply E (i, ω) by filter coefficient F _m (k, ω). The result is M (i, k, ω).

Where f _e is the sampling frequency of the amplitude envelope e (i, t), N _k is the number of bands in the bandpass filter bank F _m (k), ω _m0 is the center frequency of the bandpass filter bank F _m (k) lowest band (Minimum center frequency), ω _m1 is the center frequency (maximum center frequency) of the highest band of the bandpass filter bank F _m (k). For example, f _e = 400 Hz, N _k = 20, ω _m0 = 0.5 / f _e , and ω _m1 = 200 / f _e (see FIG. 1).
(5.3) Let M (i, k, t) be M (i, k, ω) obtained by discrete inverse Fourier transform.
(Calculation of statistics around time)
Calculate the time statistic from the output waveform of the auditory system model.

Hereinafter, for simplicity, s _type (i, t), e _type (i, t), and m _type (i, k, t) are set to s (i, t) and e, respectively, as type∈ {tex, gen}. (i, t) and m (i, k, t). In addition, the time marginal statistics f _{mean, type} (i), f _{var, type} (i), f _{skew, type} (i), f _{cce, type} (i, j), f _{pow, type} (i, k), f _{ccm, type} (i, j, k) is f _mean (i), f _var (i), f _skew (i), f _cce (i, j), f _pow (i, k), f _ccm ( i, j, k).
Input: Waveform m (i, k, t) obtained by band-dividing the amplitude envelope e (i, t) and amplitude envelope e (i, t) of the waveform s (i, t)
Output: Time marginal statistics f _mean (i), f _var (i), f _skew (i), f _cce (i, j), f _pow (i, k), f _ccm (i, j, k)
(1) Calculate the time marginal statistic of the amplitude envelope e (i, t).
(1.1) The mean μ _e (i) of e (i, t) is _defined as f _mean (i).

(1.2) A value obtained by dividing the variance σ _e (i) ² of e (i, t) by the square of the average μ _e (i) is defined as f _var (i).

(1.3) Let the skewness of e (i, t) be f _skew (i).

(1.4) Let the correlation coefficient between e (i, t) and e (j, t) be f _cce (i, j).

(2) Calculate the time marginal statistics of the waveform m (i, k, t).
(2.1) Let f _pow (i, k) be the value obtained by dividing the root mean square of m (i, k, t) by the variance σ _m (i, k) ² . Here, μ _m (i, k) is an average of m (i, k, t).

(2.2) Let the correlation coefficient of m (i, k, t) and m (j, k, t) be f _ccm (i, j, k).

  FIG. 1 is a table listing the parameters used in the sound signal generation algorithm. Note that the values in this table are merely examples.

(Modification of sound signal generation algorithm)
In order to generate sound with various complex textures with good performance, it is preferable to generate a sound signal after performing band division as in the above sound signal generation algorithm, but without performing band division. A sound signal can also be generated. Here, an algorithm that simplifies the sound signal generation algorithm will be described.

First, the simplified sound signal generation algorithm 1 will be described. The simplified sound signal generation algorithm 1 is different from the sound signal generation algorithm in that the waveform m (i, k, t) obtained by dividing the amplitude envelope e (i, t) is not calculated.
[Simplified sound signal generation algorithm 1]
(1) by the auditory system model, from the amplitude value of the first sound x _tex (t), for each band waveform s _tex (i, t), amplitude envelope e _tex waveform _{s tex (i, t) (} i, t ).
(2) Time envelope statistics f _{mean, tex} (i), f _{var, tex} (i), f _{skew, tex} (i), f _{cce, tex} (i, j) of amplitude envelope e _tex (i, t) Calculate
(3) The variance σ _{s_tex} (i) ² of the waveform s _tex (i, t) is calculated.
(4) The amplitude envelope e _con (i, t) of the waveform s _con (i, t) for each band is calculated from the amplitude value x _con (t) of the second sound using the auditory system model.
(5) The generated sound x _gen (t) is initialized with white noise. That is, x _gen (t) ← white noise.
(6) Repeat (6.1) to (6.11) _Niter times.
(6.1) From the amplitude value x _gen (t) of the generated sound, the amplitude envelope e _gen (i, t) of the waveform s _gen (i, t) and waveform s _gen (i, t) for each band is determined by the auditory system model Then, the fine structure p _gen (i, t) of the waveform s _gen (i, t) is calculated.
(6.2) Time envelope statistics f _{mean, gen} (i), f _{var, gen} (i), f _{skew, gen} (i), f _{cce, gen} (i, j) of amplitude envelope e _gen (i, t) Calculate
(6.3) The feature error is L (i) = αL _con (i) + (1-α) L _tex (i), and the second feature error L _con (i) and the first feature error L _tex (i) are respectively Calculate as follows.
(6.3.1) The second characteristic error L _con (i) is expressed by the following equation, the second sound amplitude envelope e _con (i, t) and the generated sound amplitude envelope e _gen (i, t): Is the square error.

(6.3.2) The first characteristic error L _tex (i) is a square error between the time-peripheral statistic of the first sound and the time-peripheral statistic of the generated sound expressed by the following equation.

However, Σ _j is executed only for a stat that takes j as an argument. Specifically, it is executed for statε {cce}.
(6.4) The amplitude envelope e _gen (i, t) is modified by the stochastic gradient method so that the characteristic error L (i) becomes smaller in order from i with the largest variance σ _{s_tex} (i) ² . Let the deformed amplitude envelope be e ′ _gen (i, t).
(6.5) The inverse operation of nonlinear compression in the auditory system model is performed on the amplitude envelope e ′ _gen (i, t). That is, an amplitude envelope e ′ _gen (i, t) raised to the power of β is re- _{designated as} e ′ _gen (i, t).

(6.6) The amplitude envelope e ′ _gen (i, t) is up-sampled from the sampling frequency f _e of the amplitude envelope to the sampling frequency f _{x of the} amplitude value. _Let e ′ _gen (i, t) be the _upsampled amplitude envelope.
(6.7) The waveform s ′ _gen (i, t) is restored from the amplitude envelope e ′ _gen (i, t) and the fine structure p _gen (i, t) by the following equation.

(6.8) 'so that the variance of the _gen (i, t) is equal to the variance of the waveform s _tex (i, t), as in the following equation, the waveform s' waveform s _gen (i, t) to the sigma _{S_tex} ( i) _Multiply / σ _{s'_gen} (i). In other words, the waveform s ′ _gen (i, t) _multiplied by σ _{s_tex} (i) / σ _{s′_gen} (i) is changed to s ′ _gen (i, t).

(6.9) Apply the bandpass filter bank F _a (i) to the waveform s ′ _gen (i, t). _Let s' _gen (i, t) be the waveform after application.
(6.10) The waveform s ′ _gen (i, t) is added according to the following equation to calculate the amplitude value x ′ _gen (t).

(6.11) Let x _gen (t) ← x ' _gen (t).

  Next, the simplified sound signal generation algorithm 2 will be described. The simplified sound signal generation algorithm 2 calculates the waveform s (i, t) for each band in addition to not calculating the waveform m (i, k, t) obtained by dividing the amplitude envelope e (i, t) into bands. It differs from the sound signal generation algorithm in that it is not.

[Simplified sound signal generation algorithm 2]
(1) from the amplitude value of the first sound x _tex (t), calculates the amplitude envelope e _tex (t) of the amplitude value x _tex (t). That is, the amplitude envelope e _tex (t) is the absolute value of the Hilbert transform of the amplitude value x _tex (t).
(2) Calculate the time marginal statistics f _{mean, tex} , f _{var, tex} , f _{skew, tex} of the amplitude envelope e _tex (t).

(3) The variance σ _{x_tex} ² of the amplitude value x _tex (t) is calculated.

(4) from the amplitude value x _con (t) of the second sound, it calculates the amplitude envelope e _con amplitude value _{x con (t) (i,} t). That is, the amplitude envelope e _tex (t) is the absolute value of the Hilbert transform of the amplitude value x _con (t).
(5) The generated sound x _gen (t) is initialized with white noise. That is, x _gen (t) ← white noise.
(6) Repeat (6.1) to (6.11) _Niter times.
(6.1) from the amplitude value x _gen product sound (t), calculates the amplitude envelope e _gen amplitude value x _gen (t) (t), the amplitude value x microstructure of _{gen (t) p gen (t} ). That is, the amplitude envelope e _gen (t) and the fine structure p _gen (t) are the absolute value and declination of the Hilbert transform of the amplitude value x _gen (t), respectively.
(6.2) Compute the time marginal statistics f _{mean, gen} , f _{var, gen} , f _{skew, gen} of the amplitude envelope e _gen (t).
(6.3) The feature error is L = αL _con + (1−α) L _tex, and the second feature error L _con and the first feature error L _tex are calculated as follows.
(6.3.1) The second characteristic error L _con is a square error between the amplitude envelope e _con (t) of the second sound and the amplitude envelope e _gen (t) of the generated sound, which is expressed by the following equation.

(6.3.2) Let the first characteristic error L _tex be the square error between the time-peripheral statistic of the first sound and the time-peripheral statistic of the generated sound, expressed by the following equation.

(6.4) The amplitude envelope e _gen (t) is transformed so that the feature error L is reduced by the probability gradient method. Let the deformed amplitude envelope be e ′ _gen (t).
(6.7) From the amplitude envelope e ′ _gen (t) and the fine structure p _gen (t), the amplitude value x ′ _gen (t) is restored by the following equation.

(6.8) 'so that the variance of the _gen (t) is equal to the variance of the amplitude value x _tex (t), as in the following equation, the amplitude value x' amplitude value x _gen (t) to _σ x_tex / σ _{x ' Multiply _gen} . That is, 're x a multiplied by _σ x_tex / σ x'_gen the _gen (t)' and _gen (t) the amplitude value x.

(6.11) Let x _gen (t) ← x ' _gen (t).

<First embodiment>
Hereinafter, the sound signal generation device 100 will be described with reference to FIGS. FIG. 2 is a block diagram illustrating a configuration of the sound signal generation device 100. FIG. 3 is a flowchart showing the operation of the sound signal generation device 100. As shown in FIG. 2, the sound signal generation device 100 includes a first sound feature extraction unit 110, a second sound feature extraction unit 120, a white signal generation unit 130, a signal transformation unit 140, and a recording unit 190. Further, the signal transformation unit 140 includes a generated sound feature extraction unit 141 and an error evaluation signal transformation unit 142. The recording unit 190 is a component that appropriately records information necessary for processing of the sound signal generation device 100.

The operation of the sound signal generation device 100 will be described with reference to FIG. The first sound feature extraction unit 110 extracts a first sound feature, which is a feature quantity independent of time, from the input first sound (S110). Specifically, first, by the sound signal generation algorithm (1), the waveform s _tex (i, t) and the waveform s _tex (i, t) for each band are obtained from the amplitude value x _tex (t) of the first sound. A waveform m _tex (i, k, t) obtained by band-dividing the amplitude envelope e _tex (i, t) and the amplitude envelope e _tex (i, t) is calculated. Next, by the sound signal generation algorithm (2), the time-peripheral statistics f _{mean, tex} (i), f _{var, tex are calculated} from the amplitude envelope e _tex (i, t) and the waveform m _tex (i, k, t). _{(i), f skew, tex} (i), f cce, tex (i, j), f pow, tex (i, k), f ccm, calculates the _{tex (i, j, k)} . In other words, time-peripheral statistics f _{mean, tex} (i), f _{var, tex} (i), f _{skew, tex} (i), f _{cce, tex} (i, j), f _{pow, tex} (i, k), f _{ccm, tex} (i, j, k) is the first sound feature. Also, the variance σ _{s_tex} (i) ² is calculated from the waveform s _tex (i, t) by the sound signal generation algorithm (3).

Note that it is not necessary to use all six time-peripheral statistics in the calculation of the first characteristic error L _tex (i) of the sound signal generation algorithm (6.3.2). That is, the first feature error L _tex (i) may be calculated by using at least one time marginal statistic. Therefore, it is not always necessary to calculate both the amplitude envelope e _tex (i, t) and the waveform m _tex (i, k, t), and it is not necessary to calculate all six time marginal statistics. That is, in S110, it is only necessary to calculate what is necessary for calculating the first feature error L _tex (i).

The second sound feature extraction unit 120 extracts a second sound feature, which is a feature amount dependent on time, from the input second sound (S120). Specifically, first, by the sound signal generation algorithm (4), from the amplitude value of the second sound x _con (t), amplitude envelope e _con waveform _{s con (i, t) (} i, t), amplitude envelope A waveform m _con (i, k, t) obtained by dividing the band of e _con (i, t) is calculated. The amplitude envelope e _con (i, t) and the waveform m _con (i, k, t), which are time patterns, are the second sound features.

Note that it is necessary to use both the amplitude envelope e _con (i, t) and the waveform m _con (i, k, t) in the calculation of the second characteristic error L _con (i) of the sound signal generation algorithm (6.3.1). There is no. That is, the second feature error L _con (i) may be calculated by using one of the amplitude envelope e _con (i, t) and the waveform m _con (i, k, t). Therefore, in some cases, it is not necessary to calculate the waveform m _con (i, k, t).

  The white signal generation unit 130 initializes the generated sound to be output with white noise (S130). Specifically, according to the sound signal generation algorithm (5).

The signal transformation unit 140 outputs a generated sound transformed using the first sound feature and the second sound feature (S140). The generated sound feature extraction unit 141 extracts, from the generated sound, a first generated sound feature that is a feature quantity independent of time and a second generated sound feature that is a time-dependent feature quantity (S141). The first generated sound feature and the second generated sound feature are referred to as generated sound features. The error evaluation signal transformation unit 142 calculates from a first feature error that is an error between the first generated sound feature and the first sound feature, and a second feature error that is an error between the second generated sound feature and the second sound feature. The generated sound is deformed so as to reduce the feature error (S142). When the predetermined condition is satisfied, the error evaluation signal deformation unit 142 outputs the deformed generated sound (S149). For example, the process of transforming the generated sound is repeated _Niter times and then output.

The operations of the generated sound feature extraction unit 141 and the error evaluation signal transformation unit 142 will be described below. First, the operation of the generated sound feature extraction unit 141 will be described. First, according to the sound signal generation algorithm (6.1), the amplitude envelope e _gen () of the waveform s _gen (i, t) and waveform s _gen (i, t) for each band is _calculated from the amplitude value x _gen (t) of the generated sound. i, t), waveform s _gen (i, t), fine structure p _gen (i, t), amplitude envelope e _gen (i, t), waveform m _gen (i, k, t) . The amplitude envelope e _gen (i, t) and the waveform m _gen (i, k, t), which are time patterns, are the second generated sound features. Next, the sound signal generation algorithm (6.2) uses the time envelope statistics f _{mean, gen} (i), f _{var, gen} () of the amplitude envelope e _gen (i, t) and the waveform m _gen (i, k, t). i), f _{skew, gen} (i), f _{cce, gen} (i, j), f _{pow, gen} (i, k), f _{ccm, gen} (i, j, k) are calculated. Time marginal statistics f _{mean, gen} (i), f _{var, gen} (i), f _{skew, gen} (i), f _{cce, gen} (i, j), f _{pow, gen} (i, k), f _{ccm , gen} (i, j, k) is the first generated sound feature.

  It should be noted that the generated sound features calculated here need only be those required in a form corresponding to S110 or S120.

Next, the operation of the error evaluation signal transformation unit 142 will be described. First, according to the sound signal generation algorithms (6.3) and (6.4), for the band number i, the feature error L (i) is reduced by the stochastic gradient method in order from i with the largest variance σ _{s_tex} (i) ^2. The amplitude envelope e _gen (i, t) is transformed. Thereby, a deformed amplitude envelope e ′ _gen (i, t) is obtained.

Incidentally, instead of calculating the first feature error L _tex (i) and the second characteristic error L _con (i) using a square error, the first feature using the absolute value of the error error L _tex (i) and a second The feature error L _con (i) may be calculated. In addition, when calculating the first feature error L _tex (i) and the second feature error L _con (i), each square sum (each absolute value sum) may be weighted and added.

Subsequently, the amplitude value x _{gen of the} generated sound is generated from the modified amplitude envelope e ′ _gen (i, t) (i∈ {1, 2,..., N _i }) by the sound signal generation algorithms (6.5) to (6.11). Calculate (t).

  The present invention is not limited to the above-described embodiment. For example, the processing of S110 to S130 may be executed in an appropriate manner instead of this order, or may be executed in parallel.

  According to the present invention, it is possible to generate a sound having an arbitrary texture of a general sound from white noise.

<Modification>
In the sound signal generation device 100, each component has been described as operating based on the sound signal generation algorithm. However, instead of the sound signal generation algorithm, the simplified sound signal generation algorithm 1 and the simplified sound signal generation algorithm 2 are used. It is good also as what operate | moves based.

  In this case, the first sound feature, the second sound feature, the first generated sound feature, and the second generated sound feature are not necessarily calculated by band division. In this respect, the sound signal generation device in which the first sound feature, the second sound feature, the first generated sound feature, and the second generated sound feature are calculated for each band divided in consideration of human auditory characteristics. Different from 100.

<Supplementary note>
The apparatus of the present invention includes, for example, a single hardware entity as an input unit to which a keyboard or the like can be connected, an output unit to which a liquid crystal display or the like can be connected, and a communication device (for example, a communication cable) capable of communicating outside the hardware entity Can be connected to a communication unit, a CPU (Central Processing Unit, may include a cache memory or a register), a RAM or ROM that is a memory, an external storage device that is a hard disk, and an input unit, an output unit, or a communication unit thereof , A CPU, a RAM, a ROM, and a bus connected so that data can be exchanged between the external storage devices. If necessary, the hardware entity may be provided with a device (drive) that can read and write a recording medium such as a CD-ROM. A physical entity having such hardware resources includes a general-purpose computer.

  The external storage device of the hardware entity stores a program necessary for realizing the above functions and data necessary for processing the program (not limited to the external storage device, for example, reading a program) It may be stored in a ROM that is a dedicated storage device). Data obtained by the processing of these programs is appropriately stored in a RAM or an external storage device.

  In the hardware entity, each program stored in an external storage device (or ROM or the like) and data necessary for processing each program are read into a memory as necessary, and are interpreted and executed by a CPU as appropriate. . As a result, the CPU realizes a predetermined function (respective component requirements expressed as the above-described unit, unit, etc.).

  The present invention is not limited to the above-described embodiment, and can be appropriately changed without departing from the spirit of the present invention. In addition, the processing described in the above embodiment may be executed not only in time series according to the order of description but also in parallel or individually as required by the processing capability of the apparatus that executes the processing. .

  As described above, when the processing functions in the hardware entity (the apparatus of the present invention) described in the above embodiments are realized by a computer, the processing contents of the functions that the hardware entity should have are described by a program. Then, by executing this program on a computer, the processing functions in the hardware entity are realized on the computer.

  The program describing the processing contents can be recorded on a computer-readable recording medium. As the computer-readable recording medium, for example, any recording medium such as a magnetic recording device, an optical disk, a magneto-optical recording medium, and a semiconductor memory may be used. Specifically, for example, as a magnetic recording device, a hard disk device, a flexible disk, a magnetic tape or the like, and as an optical disk, a DVD (Digital Versatile Disc), a DVD-RAM (Random Access Memory), a CD-ROM (Compact Disc Read Only). Memory), CD-R (Recordable) / RW (ReWritable), etc., magneto-optical recording medium, MO (Magneto-Optical disc), etc., semiconductor memory, EEP-ROM (Electronically Erasable and Programmable-Read Only Memory), etc. Can be used.

  The program is distributed by selling, transferring, or lending a portable recording medium such as a DVD or CD-ROM in which the program is recorded. Furthermore, the program may be distributed by storing the program in a storage device of the server computer and transferring the program from the server computer to another computer via a network.

  A computer that executes such a program first stores, for example, a program recorded on a portable recording medium or a program transferred from a server computer in its own storage device. When executing the process, the computer reads a program stored in its own recording medium and executes a process according to the read program. As another execution form of the program, the computer may directly read the program from a portable recording medium and execute processing according to the program, and the program is transferred from the server computer to the computer. Each time, the processing according to the received program may be executed sequentially. Also, the program is not transferred from the server computer to the computer, and the above-described processing is executed by a so-called ASP (Application Service Provider) type service that realizes the processing function only by the execution instruction and result acquisition. It is good. Note that the program in this embodiment includes information that is used for processing by an electronic computer and that conforms to the program (data that is not a direct command to the computer but has a property that defines the processing of the computer).

  In this embodiment, a hardware entity is configured by executing a predetermined program on a computer. However, at least a part of these processing contents may be realized by hardware.

Claims (8)

A first sound feature extraction unit that extracts a first sound feature that is a feature quantity independent of time from the input first sound;
A second sound feature extraction unit that extracts a second sound feature that is a time-dependent feature amount from the input second sound;
A white signal generator that initializes the output generated sound with white noise;
A sound signal generating device including: a signal deforming unit that outputs the generated sound deformed by using the first sound feature and the second sound feature;
The signal transformation unit is
A generated sound feature extracting unit that extracts a first generated sound feature that is a feature quantity independent of time and a second generated sound feature that is a time-dependent feature quantity from the generated sound;
Feature error calculated from a first feature error that is an error between the first generated sound feature and the first sound feature, and a second feature error that is an error between the second generated sound feature and the second sound feature A sound signal generating device including an error evaluation signal deforming unit that deforms the generated sound so that becomes smaller. The sound signal generating device according to claim 1,
The first sound feature is a time-peripheral statistic calculated from the first sound,
The first generated sound feature is a time-peripheral statistic calculated from the generated sound. The sound signal generating device according to claim 1 or 2,
The second sound feature is a time pattern calculated from the second sound,
The second generated sound feature is a time pattern calculated from the generated sound. The sound signal generation device according to any one of claims 1 to 3,
The sound signal generation device according to claim 1, wherein the first sound feature and the first generated sound feature are calculated for each band obtained by dividing a band in consideration of human auditory characteristics. The sound signal generation device according to claim 4,
x _tex (t) is the amplitude value of the first sound, s _tex (i, t) is a waveform for each band calculated from x _tex (t), and e _tex (i, t) is s _tex (i, t ) Amplitude envelope, m _tex (i, k, t) is a band-divided waveform of e _tex (i, t) (where t is a parameter representing discrete time, i and k are parameters representing band numbers),
x _gen (t) is the amplitude value of the generated sound, s _gen (i, t) is a waveform for each band calculated from x _gen (t), e _gen (i, t) is s _gen (i, t) The amplitude envelope of m _gen (i, k, t) is a band-divided waveform of e _gen (i, t) (where t is a parameter representing discrete time and i and k are parameters representing band numbers)
The first sound feature, e _tex (i, t) mean a is f _mean of _{a tex (i), e tex (} i, t) the variance of the average of the square of e _tex (i, t) is a value obtained by dividing f _{var, tex} (i) _{and, e tex (i, t)} f skew is skewness of _{a tex (i), e tex (} i, t) and e _tex (j, t) the correlation coefficient of a is _{f cce, tex (i, j} ) _{and, m tex (i, k,} t) root mean a m _tex of (i, k, t) f pow is divided by the _{dispersion, tex} (i, k), m _tex (i, k, t) and m _tex (j, k, t) are correlation coefficients f _{ccm, tex} (i, j, k),
Said first generating sound features, e _gen (i, t) mean a is f _{mean of} the _{gen (i), e gen (} i, t) 2 square of the average of the variance of e _gen (i, t) is divided by f _var, and _{gen (i), e gen (} i, t) a skewness is f _{skew of} the _{gen (i), e gen (} i, t) and e _gen (j, t f _cce a correlation _{coefficient), gen (i,} j) and, m _gen (i, k, root mean a m _gen (i of t), k, is divided by the variance of t) f _{pow , gen} (i, k) and m _gen (i, k, t) and m _gen (j, k, t) are correlation coefficients f _{ccm, gen} (i, j, k) A sound signal generator. The sound signal generation device according to any one of claims 1 to 5,
The sound signal generating device, wherein the second sound feature and the second generated sound feature are calculated for each band obtained by dividing a band in consideration of human auditory characteristics. A first sound feature extracting step in which the sound signal generating device extracts a first sound feature that is a feature quantity independent of time from the input first sound;
A second sound feature extracting step in which the sound signal generating device extracts a second sound feature which is a feature quantity dependent on time from the input second sound;
The sound signal generation device initializes a generated sound to be output with white noise, and a white signal generation step;
The sound signal generation device includes a signal modification step of outputting the generated sound deformed using the first sound feature and the second sound feature,
The signal transformation step includes
A generated sound feature extracting step of extracting a first generated sound feature that is a feature quantity independent of time and a second generated sound feature that is a time-dependent feature quantity from the generated sound;
Feature error calculated from a first feature error that is an error between the first generated sound feature and the first sound feature, and a second feature error that is an error between the second generated sound feature and the second sound feature A sound signal generation method comprising: an error evaluation signal deformation step of deforming the generated sound so as to decrease   A program for causing a computer to function as the sound signal generation device according to any one of claims 1 to 6.

Images