JP2019008206A - Voice band extension device, voice band extension statistical model learning device and program thereof - Google Patents

Abstract

To provide a voice band extension device capable of extending the band of a voice signal.SOLUTION: A voice band extension device 1 comprises sampling frequency converting means 11 for converting a broadband voice signal by sampling frequency conversion and generating a narrowband voice signal, narrowband acoustic feature quantity extracting means 13 for extracting a narrowband acoustic feature quantity from the narrowband voice signal, broadband acoustic feature quantity extracting means 12 for extracting a broadband acoustic feature quantity from the broadband voice signal, statistical model learning means 14 for learning a statistical model for inputting the narrowband acoustic feature quantity and outputting the broadband acoustic feature quantity, narrowband acoustic feature quantity extracting means 16 for extracting a narrowband acoustic feature quantity from a voice signal to be extended, broadband acoustic feature quantity generating means 17 for generating a broadband acoustic feature quantity from the extracted narrowband acoustic feature quantity by using the statistical model, and voice synthesizing means 18 for generating a band extended voice signal by using the generated broadband acoustic feature quantity.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a voice band extension device that extends a bandwidth of a voice signal, a voice band extension statistical model learning device, and a program thereof.

Conventionally, a method has been developed for expanding the bandwidth of a narrowband audio signal, which is an audio signal of a telephone, to a wideband audio signal. This bandwidth expansion method is generally divided into two methods.
One method is to upsample a narrowband audio signal (~ 4 kHz), process and shape the signal moved to the high frequency (4 kHz to 8 kHz), and add it to the narrowband audio signal to synthesize the wideband audio signal. This is a processing shaping type method to be generated.
Another method is an analysis and synthesis type method in which a narrowband speech signal is analyzed, divided into vocal tract information and a sound source signal, and each is expanded to a high frequency and re-synthesized.

Examples of the processing shaping method include the method described in Patent Document 1.
This method of Patent Document 1 divides a narrowband audio signal into frames, calculates a frequency domain spectrum by performing time-frequency conversion, and generates a high frequency component from a low frequency component of the spectrum. Ask for. And the method of patent document 1 produces | generates a high frequency component from the low frequency component of a spectrum with a mapping function, and produces | generates a broadband spectrum by integrating with a low frequency component. And the method of patent document 1 connects the time signal obtained by performing frequency time conversion to a broadband spectrum, and produces | generates a broadband audio | voice signal.

Examples of the analytical synthesis method include the method described in Patent Document 2.
This method of Patent Document 2 divides a wideband speech signal for learning and a narrowband speech signal in which the bandwidth of the broadband speech signal is limited into frames, and each speech signal is analyzed by acoustic analysis based on linear predictive coding. Separated into vocal tract characteristics and sound source signal (voice vocal cord waveform). Then, the method of Patent Document 2 performs vector quantization on the vocal tract characteristics and the sound source signal to generate a code book representing the vocal tract characteristics and the sound source signal, and generates the code book generated from the narrowband audio signal and the wideband audio signal. The generated code book is associated in advance.

  And the method of patent document 2 divides | segments the narrowband audio | voice signal which wants to expand a band into a frame unit, and calculates | requires a vocal tract characteristic and a sound source signal by acoustic analysis. After that, the method of Patent Document 2 is based on the synthesized speech obtained from the vocal tract characteristics and the sound source signal by the codebook of the wideband speech signal corresponding to the codebook representing the vocal tract characteristics and the sound source signal obtained from the narrowband speech signal. Are connected to generate a wideband audio signal.

Japanese Patent No. 5423684 Japanese Patent No. 4132154

  In the method described in Patent Document 1, a narrowband audio signal is time-frequency converted to generate a high-frequency component, and then a frequency signal is converted to obtain a wideband audio time signal. That is, this method uses only the frequency domain of the audio signal, that is, the amplitude, and does not consider the phase. Therefore, this method has a problem that the phase cannot be expanded to a high frequency component, and the sound quality of the generated wideband audio signal is deteriorated.

  In the method described in Patent Document 2, the voice signal is separated into the vocal tract characteristic and the sound source signal, and then the narrowband voice signal and the wideband voice signal are associated with each other by a code book having discrete values. Yes. Therefore, this method has a problem that the sound quality of the generated wideband audio signal is deteriorated due to the use of a discrete codebook.

  In addition, both of the above-described two methods are intended to expand the bandwidth of a narrow-band audio signal at a telephone audio level, and are intended for ease of listening to audio in a telephone. That is, by using conventional techniques, for example, a voice signal for learning a statistical model for voice synthesis or a general voice signal other than telephone voice such as a voice signal of a voice database used for voice synthesis is band-passed. Even if it is expanded, a high-quality audio signal cannot be generated due to the above-described problem.

  The present invention has been made in view of such a problem, and provides a voice band extension device, a voice band extension statistical model learning device, and a program thereof capable of extending a voice signal with high quality. Is an issue.

  In order to solve the above-mentioned problems, an audio band extending device according to the present invention is an audio band extending device that extends a band of an audio signal, and includes a sampling frequency converting unit, a narrow band acoustic feature amount extracting unit, and a wideband sound. A feature amount extraction unit, a statistical model learning unit, a second narrowband acoustic feature amount extraction unit, a wideband acoustic feature amount generation unit, and a speech synthesis unit are provided.

In such a configuration, the audio band expansion device performs sampling frequency conversion of the wideband audio signal having the target expanded band by downsampling by the sampling frequency conversion means, and converts the narrowband audio signal having the band before expansion to Generate. This wideband audio signal is a signal that becomes learning data to be input during prior learning. Thereby, the sampling frequency conversion means generates a narrowband audio signal corresponding to the wideband audio signal which is the learning data.
Then, the audio band expansion device extracts the first acoustic feature amount from the narrowband audio signal in units of frames by the narrowband acoustic feature amount extraction unit. In addition, the audio band extending device extracts the second acoustic feature amount in units of frames from the broadband audio signal by the broadband acoustic feature amount extraction unit. This acoustic feature amount is, for example, a spectrum parameter and a sound source parameter for each frame.
Then, the voice band extending device learns a statistical model that inputs the first acoustic feature quantity and outputs the second acoustic feature quantity by the statistical model learning means. A deep neural network can be used for this statistical model.
With the above configuration, the voice band extending device learns a statistical model as prior learning.

Then, the voice band extending device extracts the third acoustic feature quantity in units of frames from the extension target voice signal having the same band as the narrow band voice signal by the second narrow band acoustic feature quantity extraction unit.
Then, the speech band extending apparatus uses the statistical model learned in advance learning by the wideband acoustic feature amount generation unit to frame the fourth acoustic feature amount, which is the broadband acoustic feature amount, from the third acoustic feature amount. Generate in units.
Then, the voice band extending device generates a voice signal whose band has been extended by performing voice synthesis using the fourth acoustic feature amount by the voice synthesizing unit.
As a result, the audio band extending device can generate an audio signal whose band is extended to a wide band from the narrow band audio signal to be extended.

  In order to solve the above problem, a speech band extended statistical model learning device according to the present invention is a speech bandwidth extended statistical model learning device that learns a statistical model used for extending a bandwidth of a speech signal, and is a sampling Frequency conversion means, narrowband acoustic feature quantity extraction means, wideband acoustic feature quantity extraction means, and statistical model learning means are provided.

In such a configuration, the speech band extended statistical model learning device performs sampling frequency conversion by down-sampling a wideband speech signal having a target expanded band by a sampling frequency conversion unit, and narrowband having a band before expansion. Generate an audio signal.
Then, the speech band extended statistical model learning device extracts the first acoustic feature amount in units of frames from the narrowband speech signal by the narrowband acoustic feature amount extraction unit. Also, the speech band extended statistical model learning device extracts a second acoustic feature amount from the broadband speech signal in units of frames by the broadband acoustic feature amount extraction unit.
Then, the speech band extended statistical model learning device inputs a first acoustic feature amount and learns a statistical model that outputs a second acoustic feature amount by statistical model learning means.

  Further, in order to solve the above-mentioned problem, a voice band extending apparatus according to the present invention is a voice band extending apparatus that extends a band of a voice signal using a statistical model learned by a voice band extended statistical model learning apparatus, Narrowband acoustic feature quantity extraction means, wideband acoustic feature quantity generation means, and speech synthesis means.

In such a configuration, the speech band extending apparatus is configured to perform third frame-by-frame processing from the extension target speech signal having the same band as the narrowband speech signal from which the acoustic feature amount on the input side of the statistical model is extracted by the narrowband acoustic feature amount extraction unit. The acoustic feature amount is extracted.
Then, the audio band expanding device generates a fourth acoustic feature amount, which is a broadband acoustic feature amount, from the third acoustic feature amount in units of frames by using the statistical model by the broadband acoustic feature amount generation unit.
Then, the voice band extending device generates a voice signal whose band has been extended by performing voice synthesis using the fourth acoustic feature amount by the voice synthesizing unit.

The voice band extending device can be operated by a voice band extending program for causing a computer to function as each unit of the voice band extending device.
Further, the voice band extended statistical model learning apparatus can operate the computer with a voice band extended statistical model learning program for causing a computer to function as each unit of the voice band extended statistical model learning apparatus.

The present invention has the following excellent effects.
According to the present invention, an acoustic feature amount for each frame before and after band extension is used as a statistical model from an audio signal having a target band after extension and an audio signal having a band before extension generated from the voice signal. Can learn.
Therefore, the present invention can reflect the time-series acoustic feature amount of the audio signal before extension in the audio signal after the band extension, and can generate a high-quality audio signal.

It is a block block diagram which shows the structure of the audio | voice band expansion apparatus which concerns on embodiment of this invention. It is a block diagram of the neural network which shows the structural example of the statistical model which the audio | voice band extending apparatus which concerns on embodiment of this invention learns in advance. It is a flowchart which shows the operation | movement of the prior learning of the statistical model in the audio | voice band extending apparatus which concerns on embodiment of this invention. It is a flowchart which shows the operation | movement of the band expansion using the statistical model in the audio | voice band expansion apparatus which concerns on embodiment of this invention. It is a figure showing the spectrogram of an audio | voice signal, Comprising: (a) is a wideband audio | voice signal, (b) is the narrowband audio | voice signal converted from the wideband audio | voice signal of (a), (c) is the narrowband audio | voice signal of (b). Shows a spectrogram of a band-expanded voice signal obtained by extending the bandwidth of the signal with a voice band expansion device. It is a block block diagram which shows the structure of the audio | voice band expansion statistical model learning apparatus which concerns on the modification of this invention. It is a block block diagram which shows the structure of the audio | voice band expansion apparatus which concerns on the modification of this invention.

Embodiments of the present invention will be described below with reference to the drawings.
[Configuration of Voice Band Extender]
First, with reference to FIG. 1, the structure of the audio | voice band expansion apparatus 1 which concerns on embodiment of this invention is demonstrated.

  The voice band extending device 1 learns a statistical model for extending a voice signal band, and uses the learned statistical model to extend a voice signal band. In the following, the pre-expansion band to be expanded is called a narrow band, the target post-expansion band is called a wide band, the narrow band audio signal is called a narrow band audio signal, and the wide band audio signal is called a wide band audio signal. .

The voice band extending device 1 inputs a wide band voice signal as learning data at the time of prior learning for learning a statistical model. The learning data is a wideband voice signal that utters a plurality of sentences, and is, for example, a voice of about several thousand sentences (several hours long).
In addition, the audio band expansion device 1 inputs a narrowband audio signal to be subjected to band expansion when expanding the band of the audio signal.
Here, as an example, a wideband audio signal is sampled at a sampling frequency (sampling frequency) of 48 kHz, a narrowband audio signal is sampled at a sampling frequency of 16 kHz, and the number of conversion bits (number of quantization bits) is all sampled at 16 bits. To do.

  As shown in FIG. 1, the speech band extending apparatus 1 includes a switching unit 10, a sampling frequency converting unit 11, a wideband acoustic feature amount extracting unit 12, a narrowband acoustic feature amount extracting unit 13, and a statistical model learning unit. 14, a statistical model storage unit 15, a narrowband acoustic feature quantity extraction unit (second narrowband acoustic feature quantity extraction unit) 16, a broadband acoustic feature quantity generation unit 17, and a speech synthesis unit 18.

The switching means 10 switches the output path of the audio signal input from the outside between the mode for performing prior learning of the statistical model and the mode for expanding the bandwidth of the audio signal. The switching means 10 performs switching according to an instruction via an external input device (switch or the like) (not shown).
When the mode is instructed to perform “pre-learning”, the switching unit 10 outputs a wideband audio signal, which is an input audio signal, to the sampling frequency conversion unit 11 and the wideband acoustic feature amount extraction unit 12.
Further, when “band extension” is instructed as a mode, the switching unit 10 outputs a narrowband audio signal that is an input audio signal to the narrowband acoustic feature amount extraction unit 16.

The sampling frequency converting means 11 downsamples the sampling frequency of the wideband audio signal and converts it to the sampling frequency of the narrowband audio signal.
Here, the sampling frequency conversion means 11 down-samples a wideband audio signal with a sampling frequency of 48 kHz and converts it into a narrowband audio signal with a sampling frequency of 16 kHz.
The sampling frequency conversion unit 11 outputs the narrowband audio signal after the sampling frequency conversion to the narrowband acoustic feature amount extraction unit 13.

  The broadband acoustic feature amount extraction unit 12 extracts an acoustic feature amount (wideband acoustic feature amount) from a broadband speech signal input at the time of prior learning. The broadband acoustic feature extraction means 12 cuts out an audio signal (wideband audio signal) every predetermined frame length (for example, 25 ms) and every predetermined frame period (for example, 5 ms), and performs acoustic analysis, thereby performing acoustic analysis. Spectral parameters and sound source parameters are extracted as feature quantities.

The spectral parameter is a parameter representing the spectral envelope of speech, and is, for example, a mel cepstrum coefficient, a line spectral pair (LSP), or the like.
The sound source parameter is a parameter representing the strength of the voice, the pitch period, etc., and is, for example, a logarithmic pitch frequency, a band non-periodic component, or the like.

In addition, since the method of extracting the acoustic feature amount from the audio signal may be a general method,
Although the description is omitted here, for example, the technique (WORD) disclosed in Reference Document 1 below can be used.
(Reference 1) Masamasa Mori, Takanobu Nishiura, Hidenori Kawahara, “Proposal and Basic Evaluation of High-Quality Speech Analysis Conversion Synthesis System WORD—Effects of Fundamental Frequency and Spectrum Envelope Control on Quality Perception—” Hearing Society of Japan, Vol. 41, No. 7, pp. 555-560, Toyama, Oct. 2011.

The broadband acoustic feature amount extraction unit 12 extracts a parameter having a predetermined number of dimensions. For example, the wideband acoustic feature quantity extraction means 12 performs a 66-dimensional mel cepstrum coefficient, a one-dimensional logarithmic pitch frequency, and a five-dimensional band non-periodic component total 66-dimensional static identification (static feature quantity) for each frame. Extracted as broadband acoustic features.
The broadband acoustic feature quantity extracted by the broadband acoustic feature quantity extraction unit 12 becomes teacher data (correct data) when the statistical model learning unit 14 learns the statistical model.
The broadband acoustic feature quantity extraction unit 12 outputs the extracted 66-dimensional broadband acoustic feature quantity to the statistical model learning unit 14.

The narrowband acoustic feature amount extraction unit 13 extracts an acoustic feature amount (narrowband acoustic feature amount) from the narrowband audio signal converted by the sampling frequency conversion unit 11.
The narrow-band acoustic feature quantity extraction unit 13 extracts a parameter having a predetermined number of dimensions from the audio signal as the acoustic feature quantity, similarly to the broadband acoustic feature quantity extraction unit 12. However, the narrow-band acoustic feature amount extraction unit 13 extracts a dynamic characteristic (dynamic feature amount) as an acoustic feature amount in addition to a static specification (static feature amount) parameter.

For example, the narrow-band acoustic feature quantity extraction unit 13 determines a total of 62-dimensional static identification (static feature quantity) of 60-dimensional mel cepstrum coefficient, 1-dimensional logarithmic pitch frequency, and 1-dimensional band non-periodic component for each frame To extract.
Further, the narrowband acoustic feature quantity extraction unit 13 generates a dynamic characteristic that changes in the time direction from the static characteristic obtained for each frame. Here, the narrowband acoustic feature quantity extraction means 13 has a dynamic characteristic with a first-order difference (difference from a static characteristic after one frame) and a secondary difference (a static characteristic after two frames) as static characteristics. (Difference) 124-dimensional dynamic characteristics are obtained.

  The narrowband acoustic feature quantity extraction unit 13 outputs the extracted narrowband acoustic feature quantity to the statistical model learning unit 14. Here, the narrowband acoustic feature quantity extraction unit 13 outputs to the statistical model learning unit 14 a 186-dimensional narrowband acoustic feature quantity that is a combination of the 62-dimensional static identification and the 124-dimensional dynamic characteristics.

  Note that the narrowband acoustic feature quantity extraction unit 13 may use only the static characteristics as the narrowband acoustic feature quantity as in the wideband acoustic feature quantity extraction unit 12. However, the narrowband acoustic feature quantity extraction unit 13 can extract changes in the time direction as acoustic feature quantities by extracting dynamic characteristics, and can extract acoustic features more accurately.

  The statistical model learning means 14 is for learning a statistical model that receives a narrowband acoustic feature and outputs a broadband acoustic feature. The statistical model learning unit 14 uses the broadband acoustic feature extracted by the broadband acoustic feature extracting unit 12 as teacher data, and the narrowband acoustic feature extracted by the narrowband acoustic feature extracting unit 13 approximates the teacher data. To learn statistical models.

The statistical model learning means 14 learns a deep neural network (DNN) as a statistical model.
FIG. 2 shows an example of a statistical model composed of DNN. Specifically, the statistical model learning means 14 learns the statistical model M by using a forward forward neural network (FFNN) including the input layer L1, the hidden layer L2, and the output layer L3 shown in FIG. To do.
The structure of the statistical model M is not particularly specified, but here, the input layer L1 is 186 dimensions same as the narrowband acoustic feature, the hidden layer L2 is 256 dimensions × 3 layers, and the output layer L3 is wideband acoustic. The same 66 dimensions as the feature amount.

  The statistical model learning unit 14 inputs the narrowband acoustic feature amount (here, 186 dimensions) extracted by the narrowband acoustic feature amount extraction unit 13 to the input layer L1 in FIG. Then, the statistical model learning means 14 adds a weight to each element value of the narrowband acoustic feature value input to the input layer L1 and propagates it in the hidden layer L2, and transmits the broadband acoustic feature value (in the output layer L3). Here, the statistical model M for estimating 66 dimensions) is learned. For example, a normalized linear function is used as the activation function in the hidden layer L2.

  Here, the statistical model learning means 14 has an error of “0” between the broadband acoustic feature quantity that is the teacher data extracted by the broadband acoustic feature quantity extraction means 12 and the estimated broadband acoustic feature quantity output from the output layer L3. The weight of each layer is learned as a model parameter of the statistical model M in a direction approaching "." For example, a mean square error function is used as the loss error function for calculating the error. For learning the statistical model M, for example, a back propagation method is used.

The statistical model learning unit 14 repeatedly performs learning while the wideband acoustic feature quantity and the narrowband acoustic feature quantity are input in the pre-learning, but the learning is performed a predetermined number of times or the parameter error is within a predetermined error. The learning may be terminated at the stage of convergence.
The statistical model learning unit 14 writes and stores the learned statistical model M (model parameter) in the statistical model storage unit 15.

The statistical model storage unit 15 stores the statistical model learned by the statistical model learning unit 14. The statistical model storage unit 15 can be configured by a general storage device such as a hard disk or a semiconductor memory.
The statistical model (model parameter) stored in the statistical model storage unit 15 is referred to by the broadband acoustic feature generation unit 17.

  The narrowband acoustic feature quantity extraction unit (second narrowband acoustic feature quantity extraction unit) 16 extracts an acoustic feature quantity (narrowband acoustic feature quantity) from an extension target voice signal (narrowband voice signal) input at the time of band extension. To do. The narrowband acoustic feature quantity extraction unit 16 has the same function as the narrowband acoustic feature quantity extraction unit 13 and extracts the same type of narrowband acoustic feature quantity as the narrowband acoustic feature quantity extraction unit 13 from the narrowband audio signal. To do.

Here, the narrowband acoustic feature quantity extraction unit 16 extracts the 186-dimensional narrowband acoustic feature quantity that combines the same 62-dimensional static identification and 124-dimensional dynamic characteristics as the narrowband acoustic feature quantity extraction unit 13, This is output to the broadband acoustic feature quantity generation means 17.
Note that the narrowband acoustic feature quantity extraction unit 16 and the narrowband acoustic feature quantity extraction unit 13 have the same function, and thus may be configured by a single unit and switched only between input and output.

The broadband acoustic feature quantity generation unit 17 generates a broadband acoustic feature quantity from the narrowband acoustic feature quantity using a statistical model.
The broadband acoustic feature quantity generation means 17 refers to a statistical model stored in the statistical model storage means 15, specifically, a narrowband extracted by the narrowband acoustic feature quantity extraction means 16 with reference to the model parameter. A broadband acoustic feature is generated from the acoustic feature. For example, in the case of the statistical model M shown in FIG. 2, the broadband acoustic feature quantity generation unit 17 uses the narrowband acoustic feature quantity as an input to the input layer L1 and calculates the broadband acoustic feature quantity as the output of the output layer L3.
As a result, the broadband acoustic feature quantity generation unit 17 generates a broadband acoustic feature quantity corresponding to the narrowband acoustic feature quantity for each frame.
The broadband acoustic feature quantity generation unit 17 outputs the generated broadband acoustic feature quantity to the speech synthesis unit 18.

The voice synthesizer 18 performs voice synthesis using the broadband acoustic feature quantity generated by the broadband acoustic feature quantity generator 17 and generates a voice signal (band extended voice signal).
The speech synthesizer 18 expresses resonance characteristics in the vocal tract using a spectrum parameter (here, mel cepstrum coefficient) of a sound source waveform approximated by a sound source parameter (here, logarithmic pitch frequency, band non-periodic component). Speech synthesis is performed by synthesizing speech waveforms for each frame as input to the vocal tract filter.
As a method of performing speech synthesis using this acoustic feature amount, a general vocoder method may be used. For example, the method (WORD) disclosed in Reference Document 1 described above can be used.

  The configuration of the voice band extending apparatus 1 according to the embodiment of the present invention has been described above. The voice band extending apparatus 1 is operated by a program (voice band expanding program) for causing a computer to function as each of the above-described units. Can do.

By configuring the voice band extending apparatus 1 as described above, the voice band extending apparatus 1 learns a statistical model that converts a narrowband acoustic feature amount into a wideband acoustic feature amount from a wideband speech signal at the time of prior learning. can do.
Then, the voice band extension device 1 can generate a band extension voice signal (broadband voice signal) obtained by extending a band from a narrow band voice signal using a statistical model at the time of band extension.
As described above, the audio band extension device 1 can take into consideration continuous features for each frame by using the acoustic feature amount, and can generate a high-quality audio signal as compared with the conventional method.

[Operation of Voice Bandwidth Expansion Device]
Next, with reference to FIG. 3 and FIG. 4, the operation of the voice band extending apparatus 1 according to the embodiment of the present invention will be described. Here, the operation of the voice band expansion device 1 will be described separately for the time of prior learning for performing prior learning of a statistical model and the time of band expansion for performing band expansion of a voice signal.

(Learning in advance)
First, referring to FIG. 3 (refer to FIG. 1 as appropriate), the pre-learning operation in the voice band extending apparatus 1 will be described. Note that the switching path of the switching unit 10 is switched so that the input broadband audio signal is output to the sampling frequency conversion unit 11 and the broadband acoustic feature amount extraction unit 12 as a mode for performing pre-learning. .

  In step S1, the sampling frequency conversion means 11 performs sampling frequency conversion from a wide band (for example, 48 kHz) to a narrow band (for example, 16 kHz) with respect to the wideband audio signal input as learning data. Narrowband audio signal is generated.

  In step S2, the narrowband acoustic feature amount extraction unit 13 determines a parameter of a predetermined number of dimensions, for example, a 60-dimensional mel cepstrum coefficient and a one-dimensional logarithm for each frame from the narrowband speech signal generated in step S1. A 186-dimensional narrow-band acoustic feature quantity consisting of a pitch dimension and a 62-dimensional static characteristic of a one-dimensional band non-periodic component, and a 124-dimensional dynamic characteristic of a first-order difference and a second-order difference in the time direction of the static characteristics. Extract.

In step S3, the broadband acoustic feature amount extraction unit 12 determines a parameter of a predetermined dimension, for example, a 60-dimensional mel cepstrum coefficient, a one-dimensional logarithmic pitch for each frame, from the broadband speech signal input as learning data. A total 66-dimensional static identification of frequency and 5-dimensional band non-periodic components is extracted as a broadband acoustic feature.
This step S3 may be operated before steps S1 and S2 or in parallel with steps S1 and S2.
As a result, the narrowband acoustic feature quantity and the broadband acoustic feature quantity generated from the same broadband audio signal are generated corresponding to the frame.

In step S4, the statistical model learning unit 14 learns a statistical model that converts the narrowband acoustic feature amount into the broadband acoustic feature amount by using the broadband acoustic feature amount extracted in step S3 as teacher data. Here, the statistical model learning means 14 inputs the narrowband acoustic feature amount extracted in step S2 to the input layer L1 in the forward propagation neural network of FIG. Then, the statistical model learning means 14 adds a weight to the value of each element of the narrowband acoustic feature value input to the input layer L1 in the hidden layer L2 and propagates it, and extracts it in the output layer L3 in step S3. The statistical model M is learned so that an error from the set broadband acoustic feature amount approximates to “0”.
With the above operation, the audio band extension device 1 can generate a statistical model for estimating the acoustic feature amount of the wideband speech signal corresponding to the narrowband speech signal from the acoustic feature amount of the narrowband speech signal.

(Bandwidth expansion)
Next, with reference to FIG. 4 (refer to FIG. 1 as appropriate), the operation of band extension in the voice band extension apparatus 1 will be described. Note that the switching path of the switching unit 10 is switched so that the input narrowband audio signal is output to the narrowband acoustic feature quantity extraction unit 16 as a mode for performing band expansion.

  In step S10, the narrowband acoustic feature amount extraction unit 16 extracts a parameter of a predetermined number of dimensions, for example, a 60-dimensional mel cepstrum coefficient and a one-dimensional logarithm for each frame from the narrowband audio signal to be subjected to band expansion. A 186-dimensional narrow-band acoustic feature quantity consisting of a pitch dimension and a 62-dimensional static characteristic of a one-dimensional band non-periodic component, and a 124-dimensional dynamic characteristic of a first-order difference and a second-order difference in the time direction of the static characteristics. Extract. Note that the narrowband acoustic feature quantity extracted in step S10 is the same type and the same number of dimensions as the narrowband acoustic feature quantity extracted in step S2 of FIG.

  In step S11, the broadband acoustic feature quantity generation unit 17 generates a broadband acoustic feature quantity from the narrowband acoustic feature quantity extracted in step S10, using the statistical model learned in advance learning (see FIG. 3).

In step S12, the speech synthesizer 18 performs speech synthesis by synthesizing the speech waveform for each frame using the broadband acoustic feature generated in step S11, and generates a broadband (band extension) speech signal. To do.
With the above operation, the voice band extension device 1 can generate a wideband voice signal whose band is extended from a narrowband voice signal using a statistical model.

[Performance evaluation of voice bandwidth expansion equipment]
Next, with reference to FIG. 5, the performance evaluation of the voice band extending device 1 will be described. FIG. 5 shows a spectrogram of an audio signal, with the horizontal axis representing time [seconds] and the vertical axis representing frequency [Hz].
FIG. 5A is a spectrogram of a wideband audio signal having a sampling frequency of 48 kHz (the maximum frequency observable by the sampling theorem is 24 kHz). This wideband audio signal is not an audio signal used for prior learning.

FIG. 5B is a spectrogram of a narrowband audio signal obtained by down-sampling the wideband audio signal of FIG. 5A to a sampling frequency of 16 kHz.
FIG. 5C shows a band-expanded audio signal obtained as a result of band expansion to a sampling frequency of 48 kHz using the narrow-band audio signal of FIG. 5B as the input of the audio band expansion apparatus 1 that has completed preliminary learning in advance. This is the spectrogram.

As shown in FIG. 5B, this narrowband audio signal has only components up to 8 kHz due to downsampling.
However, as shown in FIG. 5C, the band-extended audio signal band-extended by the audio band-expanding device 1 has a component of 8 kHz or higher restored, and the original wide-band audio signal shown in FIG. You can see that it is well restored.
As described above, the audio band extending device 1 can easily and highly expand the audio signal band.

  Next, with reference to Table 1, another performance evaluation of the voice band expansion device 1 will be described. Table 1 is a table comparing the performance due to the difference in the technique for performing band expansion.

Table 1 shows a band extension audio signal obtained by up-sampling a narrowband audio signal generated by down-sampling a wideband audio signal to the same sampling frequency as that of the original wideband audio signal, and an audio band extension apparatus according to the present invention. The mean square error of each of the band expanded audio signals expanded in 1 and the original wideband audio signal is shown.
Specifically, the spectral parameters of the respective audio signals are calculated, and the corresponding spectral parameter is determined for each of the wideband audio signal and the band extended audio signal (upsampling), and the wideband audio signal and the band extended audio signal (the present invention). The mean square error is calculated. Also, here, static characteristics and dynamic characteristics are used as the narrow-band acoustic feature quantities when the voice band extension apparatus 1 performs band extension. Moreover, all measured using the audio | voice signal of the evaluation data 350 sentence unused in prior learning.
As shown in Table 1, the value of the mean square error is smaller in the band extended voice signal extended by the voice band extending apparatus 1 (the present invention) than in the upsampled band extended voice signal. It can be seen that the apparatus 1 can estimate the wideband audio signal with high accuracy.

  Furthermore, with reference to Table 2, another performance evaluation of the voice band extending apparatus 1 will be described. Table 2 is a table comparing the performance due to the difference in the narrowband acoustic feature amount extracted by the narrowband acoustic feature amount extraction means 13 and 16.

Table 2 shows that the narrowband acoustic feature amount extraction means 13 and 16 use the correct wideband audio signal and the audio band expansion device when only the static characteristic is used and when the static characteristic and the dynamic characteristic are used. 1 indicates the minimum value of the mean square error with the generated band extension voice signal. Here, similarly to the performance evaluation in Table 1, measurement was performed using an audio signal of 350 sentences of evaluation data unused for pre-learning.
As shown in Table 2, it can be seen that the use of a narrowband acoustic feature that combines the static characteristics and the dynamic characteristics can more accurately estimate the wideband audio signal than the static characteristics alone.

[Modification]
The configuration, operation, and evaluation of the voice band extending apparatus 1 according to the embodiment of the present invention have been described above. However, the present invention is not limited to this embodiment.
The voice band expansion device 1 performs two operations of pre-learning for learning a statistical model and band expansion for expanding a bandwidth of a voice signal by using the statistical model. However, these operations may be performed by separate devices.
Specifically, an apparatus that realizes pre-learning for learning a statistical model can be configured as the voice band extended statistical model learning apparatus 2 shown in FIG.

As shown in FIG. 6, the speech band extended statistical model learning device 2 includes a sampling frequency conversion unit 11, a wideband acoustic feature amount extraction unit 12, a narrowband acoustic feature amount extraction unit 13, and a statistical model learning unit 14. And a statistical model storage unit 15. In this configuration, the switching unit 10, the narrowband acoustic feature quantity extraction unit 16, the wideband acoustic feature quantity generation unit 17, and the voice synthesis unit 18 are deleted from the configuration of the voice band expansion device 1 described in FIG. 1. Is.
This voice band extended statistical model learning device 2 performs only a pre-learning operation for learning a statistical model.
The operation of the voice band extended statistical model learning device 2 is the same as the operation described in FIG.
The voice band extended statistical model learning device 2 can be operated by a program (voice band extended statistical model learning program) for causing a computer to function as each of the above-described means.

In addition, a device that realizes a bandwidth extension operation for extending the bandwidth of a voice signal using a statistical model can be configured as a voice bandwidth extension device 1B shown in FIG.
The voice band expansion device 1B includes a statistical model storage unit 15, a narrowband acoustic feature amount extraction unit 16, a wideband acoustic feature amount generation unit 17, and a voice synthesis unit 18. This configuration is different from the configuration of the voice band extending apparatus 1 described in FIG. 1 in that the switching unit 10, the sampling frequency converting unit 11, the wideband acoustic feature amount extracting unit 12, the narrowband acoustic feature amount extracting unit 13, The statistical model learning means 14 is deleted. The statistical model stored in the statistical model storage means 15 is learned by the voice band extended statistical model learning device 2 of FIG.
This voice band extending device 1B performs only a band extending operation for extending the band of the voice signal.
The operation of the voice band expansion device 1B is the same as the operation described in FIG.
The voice band extending device 1B can be operated by a program (voice band extending program) for causing a computer to function as each of the above-described means.

  In this way, the learning operation for learning the statistical model and the bandwidth expansion operation for expanding the bandwidth of the speech signal using the statistical model are performed by different devices (speech bandwidth extension statistical model learning device 2, speech bandwidth extension device 1B). By operating, a statistical model learned by one voice band expansion statistical model learning device 2 can be used by a plurality of voice band expansion devices 1B.

DESCRIPTION OF SYMBOLS 1,1B Voice band expansion apparatus 2 Voice band expansion statistical model learning apparatus 10 Switching means 11 Sampling frequency conversion means 12 Wideband acoustic feature amount extraction means 13 Narrowband acoustic feature amount extraction means 14 Statistical model learning means 15 Statistical model storage means 16 Narrowband acoustic feature extraction means (second narrowband acoustic feature extraction means)
17 Broadband acoustic feature generation means 18 Speech synthesis means

Claims (8)

An audio band extending device for extending an audio signal band,
Sampling frequency conversion means for performing sampling frequency conversion of a wideband audio signal having a target expanded band and generating a narrowband audio signal having a band before expansion;
Narrowband acoustic feature quantity extraction means for extracting a first acoustic feature quantity from the narrowband audio signal in units of frames;
A broadband acoustic feature quantity extracting means for extracting a second acoustic feature quantity from the broadband voice signal in units of frames;
Statistical model learning means for inputting a first acoustic feature and learning a statistical model for outputting the second acoustic feature;
Second narrowband acoustic feature quantity extraction means for extracting a third acoustic feature quantity in units of frames from the extension target voice signal having the same band as the narrowband voice signal;
Broadband acoustic feature value generation means for generating a fourth acoustic feature value, which is a broadband acoustic feature value, in units of frames from the third acoustic feature value using the statistical model;
Voice synthesis means for generating a voice signal with an extended band by performing voice synthesis using the fourth acoustic feature amount;
A voice band extending apparatus comprising:   The voice band extending apparatus according to claim 1, wherein the statistical model learning unit learns a deep neural network as the statistical model.   The audio band extending device according to claim 1 or 2, wherein the acoustic feature amount is a static characteristic of a spectrum parameter and a sound source parameter for each frame.   4. The narrowband acoustic feature quantity extraction unit and the second narrowband acoustic feature quantity extraction unit extract a difference between frames of the static characteristic as a dynamic characteristic in addition to the static characteristic. The voice bandwidth expansion device described. A speech bandwidth expansion statistical model learning device for learning a statistical model used for expanding a bandwidth of a speech signal,
Sampling frequency conversion means for performing sampling frequency conversion of a wideband audio signal having a target expanded band and generating a narrowband audio signal having a band before expansion;
Narrowband acoustic feature quantity extraction means for extracting a first acoustic feature quantity from the narrowband audio signal in units of frames;
A broadband acoustic feature quantity extracting means for extracting a second acoustic feature quantity from the broadband voice signal in units of frames;
Statistical model learning means for inputting a first acoustic feature and learning a statistical model for outputting the second acoustic feature;
A speech band extended statistical model learning device comprising: A voice band extension device for extending a voice signal band using a statistical model learned by the voice band extension statistical model learning device according to claim 5,
Narrowband acoustic feature quantity extraction means for extracting a third acoustic feature quantity in units of frames from an extension target voice signal having the same band as the narrowband voice signal from which the acoustic feature quantity on the input side of the statistical model is extracted;
Broadband acoustic feature value generation means for generating a fourth acoustic feature value, which is a broadband acoustic feature value, in units of frames from the third acoustic feature value using the statistical model;
Voice synthesis means for generating a voice signal with an extended band by performing voice synthesis using the fourth acoustic feature amount;
A voice band extending apparatus comprising:   A voice band expansion program for causing a computer to function as the voice band expansion apparatus according to any one of claims 1, 2, 3, 4, and 6.   A voice band extended statistical model learning program for causing a computer to function as the voice band extended statistical model learning device according to claim 5.