JP3342310B2 - Audio decoding device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention analyzes an input voice,
The present invention relates to an audio encoding device that compresses and encodes, and an audio decoding device that decodes and synthesizes audio from encoded data.

[0002]

2. Description of the Related Art In general, in a speech coding / decoding apparatus using speech coding, the compression rate of the coding is a major problem. That is, when transmitting coded audio, the higher the compression ratio, the more audio data can be transmitted with the same transmission capacity. Also, even when the encoded data is stored in the storage device and the audio is reproduced based on the stored encoded data, the higher the compression ratio of the encoding, the more the audio data is stored in the storage device having the same storage capacity. Data can be stored.

[0003] In general high-efficiency speech coding, speech is separated into spectral feature parameters and sound source parameters and encoded in consideration of a speech generation process. Among the sound source parameters, the pitch is an important parameter that characterizes the pitch of a sound, and is generally and widely used as a parameter in speech coding.

[0004] The dynamic range of the pitch frequency of voice is about 50 Hz to 250 including zero for general males.
Hz, for general women, about 100Hz-4 including zero
It is about 00 Hz. Therefore, when encoding the pitch, it is necessary to compress and encode a numerical value having a dynamic range of about 50 to 400 including zero.

Conventionally, for pitch compression encoding,
(1) Quantizing and compressing the logarithm of the pitch frequency using the human auditory characteristics, or (2) encoding the pitch difference. When considering the sound generation mechanism of a human sound source, the pitch does not become discontinuous in time.
For this reason, when the feature parameter of the voice is compression-coded at a time interval in a time series, if the analyzed time interval for compression is sufficiently short, the dynamic range of the difference value is smaller than the dynamic range of the pitch frequency itself. . The method (2) utilizes this fact.

[0006] In addition, when audio is reproduced by storing encoded audio data in a storage device, a part of generated contents is separately recorded and edited and connected for the purpose of reducing the storage capacity of the storage device. 2. Description of the Related Art A speech synthesis method using an edit synthesis method for creating a synthesized speech is known.

[0007]

At present, the compression coding of the pitch is based on the quantization on the logarithmic axis based on the auditory characteristics, or the reduction of the dynamic range by coding the difference. When the time interval for analyzing and compressing the audio waveform is sufficiently short, the dynamic range of the pitch difference value becomes smaller than the dynamic range of the pitch itself.
Therefore, it is possible to increase the compression ratio of the pitch by the difference processing. However, when the time interval for analysis is increased, the dynamic range of the difference value is not so small, and there is a problem that the compression ratio cannot be increased.

Accordingly, an object of the present invention is to solve such a problem of the prior art and to further increase the compression rate of compression encoding. In addition, when editing and synthesizing audio by storing and editing encoded audio data in a storage device, the pitch between recorded speech segments is not continuously connected, and the pitch of the sound changes suddenly, causing abnormal noise. And sounded unnatural. It is another object of the present invention to eliminate pitch discontinuity in an editing / synthesizing method for editing a compression-encoded phoneme unit and outputting a synthesized speech.

[0009]

SUMMARY OF THE INVENTION The object of the present invention is to model a pitch pattern and compression-encode speech using the modeled parameters, or further add information on a difference between an actual pitch pattern and a pitch pattern model. This is achieved by compression encoding. Modeling compresses pitch information by using a model based on a speech generation mechanism and modeling the pitch of sections such as accent units, phrase units, and phoneme units collectively.

[0010] The pitch of the input voice is analyzed and extracted for each analysis interval, and the extracted pitch is modeled by adding each pattern of accent, phrase, and phoneme, and the error between the pitch of the actual voice and the patterned pitch is calculated. And the respective patterns are transmitted, thereby generating compression-encoded audio code data. Then, the pitch is generated and decoded by adding each of the accent, phrase, and phoneme patterns and the error between the patterned pitch and the pitch of the actual voice from the voice encoded data.

The above-mentioned problem that the pitch between phonemes becomes discontinuous is considered to be caused by not modeling the pitch pattern. Therefore, by using a model based on the voice generation mechanism for pitch control,
Pitch discontinuities can be eliminated. By eliminating the discontinuity of the pitch, the above-mentioned sudden change in the pitch of the sound, the problem of abnormal noise and unnatural sound can be solved, and a more natural sound synthesis can be realized. be able to.

Further, the apparatus is provided with model parameter control means for controlling and changing the model parameter of the modeled and compressed pitch information based on the pitch model. By controlling the modeled parameters so that the pitch does not become too large, the pitch can be prevented from becoming too high or too low.

That is, the speech coding apparatus of the present invention comprises: a sound source extracting means for analyzing a natural speech to extract a pitch pattern in a predetermined section; a storage means storing a plurality of pitch pattern models in a predetermined section; Selecting means for selecting a pitch pattern model that best matches the pitch pattern extracted by the method, and means for encoding a pitch pattern of a predetermined section of natural speech using the selected pitch pattern model. I do. Encoding can be performed by a code corresponding to the pitch pattern model.

The predetermined section may be a section of an accent component, a section of an accent component and a section of a phrase component, or a section of an accent component, a section of a phrase component and a section of a phoneme component. A pitch error extracting means for extracting a pitch error between an actual pitch pattern and a pitch pattern model;
Alternatively, there may be provided a pitch error time series encoding means for encoding the pitch error extracted by the pitch error extracting means.

Further, a speech decoding apparatus according to the present invention stores a plurality of pitch pattern models of a predetermined section of natural speech, and decodes data encoded in association with the pitch pattern model using the pitch pattern model. And a pitch generating means for converting the pitch into a pitch. When the predetermined section is a section of an accent component unit, the pitch generating means includes an accent pattern generating means.

When the predetermined section is a section of an accent component unit and a section of a phrase component unit, the pitch generating means includes an accent pattern generating means, a phrase pattern generating means, and an accent pattern and a phrase pattern generated by the accent pattern generating means. Adding means for adding the phrase pattern generated by the generating means.

When the predetermined section is a section of an accent component unit, a section of a phrase component unit and a section of a phoneme component unit, the pitch generating means includes an accent pattern generating means,
Phrase pattern generation means, phoneme pattern generation means, and addition means for adding the accent pattern generated by the accent pattern generation means, the phrase pattern generated by the phrase pattern generation means, and the phoneme pattern generated by the phoneme pattern generation means And

When the data includes encoded data obtained by encoding a pitch error between a pitch pattern of a natural voice and a pitch pattern model, the pitch generating means includes pitch error restoring means for restoring the encoded data of the pitch error; The pitch error restored by the pitch error restoring means is added by the adding means.

When the data includes coded data obtained by time-sequentially coding pitch error data between a natural voice pitch pattern and a pitch pattern model, the pitch generating means restores the pitch error time-series coded data. Pitch error time series restoration means is provided, and pitch error data restored by the pitch error time series restoration means is input to the pitch error restoration means.

In the speech decoding apparatus, when synthesizing a speech by the edit synthesis method, a control for changing the data so that the pitch of the phoneme segment to be edited does not exceed the upper limit value and the lower limit value. It is characterized by comprising means.

[0021]

Embodiments of the present invention will be described below in detail with reference to the drawings. FIG. 1 is a block diagram schematically showing a configuration of a speech encoding / decoding apparatus according to the present invention. This speech coding / decoding device includes a speech compression / coding device 1 for compressing and coding speech, and a speech decoding device 2 for decoding a speech from compression-coded data and outputting a synthesized speech.

The voice compression encoding device 1 includes a voice input device 2
0, a spectrum analyzer 21, a sound source extractor 22, a pitch modeler 23, and an encoder 24. The speech decoding device 2 includes an encoded data storage (input unit) 10, a pitch generator 11, a sound source generator 12, a controller 13, a spectrum generator 14, and a speech synthesis output unit 15.

FIG. 2 shows a detailed example of the configuration of the pitch modeler 23 in the speech compression encoding apparatus 1, and FIG. 3 shows a detailed example of the pitch generator 11 in the speech decoding apparatus 2. The pitch modeler 23 is connected to the sound source extractor 22 and the encoder 24 as shown in FIG. 1, and as shown in FIG.
Pitch frequency input device 30, pattern matching device 31,
Pitch pattern output device 32, pitch pattern ROM 3
3. It is composed of a pitch error extractor 34 and the like.

The pitch generator 11 is connected to an encoded data accumulator (input unit) 10 and a sound source generator 12 as shown in FIG. 1, and as shown in FIG. Pattern generator 41, phrase pattern generator 42, phoneme pattern generator 43, adder 4
4, pitch error decoder 45, pitch error time series restoration unit 4
6 and so on.

The operation of the above components will be described below.
First, the operation of the audio compression encoding device 1 will be described. First, the speech input device 20 inputs speech, cuts out speech in units of analysis frames, and outputs the speech to the spectrum analyzer 21. The spectrum analyzer 21 analyzes the spectrum of the speech output from the speech input device 20, separates the speech into vocal tract information and sound source information, extracts spectrum information representing the vocal tract information, and outputs it to the encoder 24. The separated sound source information is output to the sound source extractor 22.

The sound source extractor 22 includes a spectrum analyzer 21
With the sound source information separated in step (1) as an input, a pitch frequency (interval) generally representing the pitch of a voice is extracted and output to the pitch modeler 23. Also, it outputs excitation information and power information other than the pitch frequency to the encoder 24. The pitch modeler 23 models the pitch frequency extracted from the sound source extractor 22, reduces and compresses the information amount, and outputs the modeled parameter to the encoder 24.

One of the pitch models performed by the pitch modeler 23 is a regularized and frequently used Japanese rule synthesis, a Fujisaki model based on a superposition model of an accent phrase component and a phrase component [Acoustic Society of Japan] Magazine 27,
pp. 445-453, (1971)]. This Fujisaki model is obtained by adding a phoneme pattern to the model by Takeda [“Basic frequency pattern generation model and speech synthesis rule considering changes due to phonemes”, IEICE (A)
J73-A, pp. 379-386 (1990)]. Also, a model by Abe et al. [“Two-layer control method of fundamental frequency pattern”, Sound Lecture Collection 1-2-11, pp. 227-228 (1992.
3)].

As an example of such modeling and use for speech recognition, a prosody recognition method by Shimohira et al. ["Phrase boundary detection of continuous speech by clustering of pitch patterns", Psychological Lectures 2-5-14, pp. . 81-82 (19
91)]. By applying these techniques, the pitch modeler 23 can extract a pitch pattern of each accent phrase, phrase, and phoneme. At this time, pitch modeling does not necessarily have to accurately correspond to speech phenomena and languages such as accent phrases and phrases, and may be modeled so that the compression ratio is high in terms of coding efficiency.

When the contents of the input speech are known, or when it is not necessary to perform the analysis and synthesis at the same time and it is sufficient to store the compression-encoded data in a ROM or the like, the pitch is automatically extracted when modeling the pitch. It is possible to obtain the type of each pattern obtained, modify the compression-encoded data, and create high-quality, natural and high-pitch compression data.

In the configuration example of the pitch modeler 23 shown in FIG. 2, the pitch frequency input unit 30 receives the pitch frequency extracted for each analysis frame by the sound source extractor 22 as an input, and outputs the accent phrase, phrase, sentence unit. Pitch frequency data sequence such as
Output to 1. The pitch pattern ROM 33 examines pitch patterns of voice accents, phrases, and phonemes in advance, and registers the patterns using clustering or the like. Here, the pitch pattern ROM 33 can have only an accent pattern, have an accent pattern and a phrase pattern, or have all (accent, phrase, phoneme) patterns.

The pattern matching unit 31 compares the pitch frequency data sequence input from the pitch frequency input unit 22 with the pitch pattern registered in the pitch pattern ROM 33, and determines the best matching pattern as the input pitch frequency. And outputs the type of pattern for each accent, phrase, and phoneme to the pitch pattern output unit 32.

Further, the pitch modeler 23 may include a pitch error extractor 34. In this case, the pitch error extractor 34 uses the pitch frequency train output from the pitch frequency input device 30 and the pattern matching device 31. The value sequence of the matched pattern is input and the error (difference) value sequence is output to the pitch pattern output device.

Further, a pitch error time series encoder 35 for compressing and encoding the pitch error extracted by the pitch error extractor 35 can be further provided. In this case, the pitch error time series encoder 35 The error value sequence output from the error extractor 34 is used as an input and ADPCM (Adaptive Dif
The compressed error data is output to the pitch pattern output device 32 by performing time-series coding using ferential pulse code modulation or the like.

The pitch pattern output unit 32 outputs a pattern for each accent, phrase, and phoneme output by the pattern matching unit 31. Also, pitch error extractor 3
4, the error (difference) between each pattern output by the pitch error extractor 34 and the actual pitch frequency sequence
Print more columns. If the apparatus further includes the pitch error time series encoder 35, the pitch pattern output unit 32
Is the error (difference) between each pattern and the actual pitch frequency sequence
Instead of the columns, the parameters output by the ADPCM or the like output from the pitch error time-series encoder 35 and subjected to time-series coding are output. These accents, phrases, patterns for each phoneme, an error (difference) sequence between each pattern and the actual pitch frequency sequence, and parameters that have been time-series coded by ADPCM or VQ coded are called modeled pitch information. .

Finally, the encoder 24 receives the spectrum information, the sound source information / power information, and the modeled pitch information which are analyzed and extracted by the spectrum analyzer 21, the sound source extractor 22, and the pitch modeler 23, respectively. , And outputs the encoded data to the audio decoding device 2.

Next, the speech decoding apparatus 2 will be described. The coded data storage (input unit) 10 provided in the audio decoding device 2 becomes an input device from a transmission path when transmitting audio in combination with the audio compression encoding device 1,
When the data stored in the ROM or the like is used as the voice encoded data, the voice data becomes a storage for the encoded data.

The coded data storage (input unit) 10 separates the coded coded data input in units of frames into excitation information, spectrum information, power information, and modeling pitch information, Information to the pitch generator 11, sound source information and power information to the sound source generator 12,
The spectrum information is output to the spectrum generator 14, respectively. The pitch generator 11 receives the modeled pitch information output from the encoded data storage (input unit) 10 as input, decodes the pitch frequency (interval), and outputs the decoded frequency to the excitation generator 12.

The speech decoding apparatus 2 edits the encoded stored and encoded speech data to perform speech synthesis (for editing and synthesis).
If the pitch frequency is too high or too low, the control unit 13 is provided, and if the pitch frequency does not exceed the upper limit or the lower limit, the modeled pitch information is controlled so as to limit the size of the accent pattern and the phrase pattern. Is output to the pitch generator 11. The pitch generator 11 changes the modeled pitch information based on the control signal from the controller 13 so that the pitch frequency does not exceed the lower limit and the upper limit, generates a pitch frequency (interval), and generates the pitch frequency (interval).
Output to 2. This can cause the pitch to be too high,
This solves the problem that the sound is too low and sounds unnatural, and allows speech synthesis with higher naturalness.

In addition to the case where speech synthesis is performed by editing and synthesizing, the pitch is modeled, and in the case of recording / editing / synthesis in which speech units (speech units) are connected to reproduce synthesized speech, the pitch is discontinuous. Eliminating the connection eliminates the problem of sudden changes in pitch, generation of abnormal noise, and unnatural sound, and enables speech synthesis with higher naturalness.

The sound source generator 12 receives the sound source information and power information input from the encoded data storage (input device) 10 and the pitch frequency (interval) output from the pitch generator 11 to generate a sound source. , Spectrum generator 14
Output to The spectrum generator 14 forms a synthesis filter based on the spectrum information output from the encoded data storage (input device) 10, and generates a synthesized speech signal by filtering the sound source generated by the sound source generator 12. I do. The synthesized voice output unit 15 generates a synthesized voice by performing D / A conversion on the synthesized voice signal generated by the spectrum generator 14.

FIG. 3 shows an example of the configuration of the pitch generator 11, and the pitch generator 11 will be described in more detail with reference to FIG. In the pitch pattern input device 40 of the pitch generator 11, an encoded data storage (input device) 1
The modeled pitch information compressed and encoded in units of analysis frames at 0 is input, and the accent pattern is output to the accent pattern generator 41, the phrase pattern is output to the phrase pattern generator 42, and the phoneme pattern is output to the phoneme pattern generator 43, respectively. I do. If there is data of an error (difference) sequence between each pattern and the actual pitch frequency sequence, the data is output to the pitch error reconstructor 45, and if there is a parameter that has been subjected to time-series coding by ADPCM or the like, It is output to the pitch error time series restoration unit 46, respectively.

Here, if the modeled pitch information targets only the accent pattern, the pitch generator 11
Is composed of only a pitch pattern input device 40 and an accent pattern generator 41. The accent pattern generator 41 generates and outputs a pitch frequency (interval) sequence of accent units from the input accent pattern.

When the modeled pitch information targets an accented phrase pattern, the pitch generator 11 includes a pitch pattern input unit 40, an accent pattern generator 41, a phrase pattern generator 42, and an adder 44. Is done. The phrase pattern generator 42
Generates and outputs a pitch frequency (interval) sequence of phrases in phrase units from the input phrase pattern. The adder 44 adds the input pitch frequencies (intervals) such as accent units and phrase units, and outputs the pitch frequency (interval) of the original voice.

When the modeled pitch information covers all (accent, phrase, phoneme) patterns, the pitch generator 11 includes a pitch pattern input device 40, an accent pattern generator 41, a phrase pattern generator 42, and a phoneme. It comprises a pattern generator 43 and an adder 44.

When the modeled pitch information includes an error (difference) sequence between the modeled pitch and the actual pitch, the pitch generator 11 also includes a pitch error reconstructor 45. The pitch error restoring unit 45 includes the pitch pattern input unit 4
The error (difference) sequence between each pattern output from 0 and the actual pitch frequency sequence is transferred to the adder 44 as an input.

Further, when the modeled pitch information includes parameters subjected to time series coding by ADPCM or the like, the pitch error time series
6 is also provided. The pitch error time series reconstructor 46 uses the ADPCM or the like output from the pitch pattern input device 40 to perform a time series encoding of the error (difference) sequence of the pitch frequency and the actual pitch frequency sequence by adding the respective patterns. As an input, an error (difference) sequence between the pitch frequency obtained by adding the patterns and the actual pitch frequency sequence is estimated and generated by ADPCM or the like. The generated error sequence is stored in the pitch restorer 4
5 is output.

As described above, when the pitch is encoded and decoded, by modeling the pitch, the pitch can be encoded with a smaller amount of data as compared with the encoded data of the conventional pitch. High-quality decoded speech can be generated with a transmission capacity and a small amount of memory. Hereinafter, the flow of processing according to the present invention will be described based on a specific example. 4
Shows an example of a process when “the next train is bound for Tenri” is input to the voice compression encoding device 1 as a voice input.

The sound source extractor 22 calculates the pitch pattern of the input natural speech and decomposes the pattern into phrases, accents, and phonemes. The decomposition into this component is HHM
(Hidden Markov Model) etc., automatic segmentation of phonemes by well-known speech recognition, the above-mentioned prosody recognition method by Shimohira et al. ["Phrase boundary detection of continuous speech by pitch pattern clustering", Jpn. pp. 81-82
(1991)].

In the pitch modeler 23, the pattern that matches the most among the patterns registered in advance in the pitch pattern ROM 33 for each component is determined by the pattern matching unit 3.
Select by 1. As a processing example, first, an accent is cut out, and the most matching accent pattern and phrase pattern are searched and extracted from the patterns registered in the ROM. Next, a phoneme pattern for each phoneme (consonant) is searched and extracted from the phoneme patterns registered in the ROM. In the encoder 24, the code number of each searched pattern is set as a compression pattern.

At the time of the pattern matching, the compression / expansion rate on the time axis, the start value, the end value, the maximum value of the pitch pattern,
Using a parameter such as the minimum value or all or a part of the compression / expansion rate, a function of modifying the registered pattern of the pitch pattern ROM 33 is provided, the ROM pattern is reduced, and the code number and the modification parameter of each pitch pattern are changed to the pattern. May be used as the compression parameter.

As a general method of creating a pattern to be registered in the pitch pattern ROM 33, clustering is performed after extracting a large number of natural voice pitch patterns.
Determine the pitch pattern to be stored in the ROM. At this time, if many pitch patterns are stored in the ROM 33,
On the other hand, the modeling accuracy is increased, but the compression ratio is reduced and the processing amount is increased.

The pitch generator 11 of the speech decoding device 2
Has a ROM that stores the same pitch pattern as the pitch pattern ROM held by the audio compression encoding device 1, and generates a pitch pattern from the code numbers encoded and transmitted by the encoder 24.

[0053]

As is clear from the above, according to the present invention,
Speech coding can be performed by modeling the pitch and compressing and coding with the modeled parameters, or by adding and compressing the difference between the actual pitch and the pitch model, or by estimating the difference in a time series. In this case, the compression ratio can be further increased.

Further, according to the present invention, in the editing / synthesizing system for editing a voice and outputting a synthesized voice by modeling a pitch, a model based on a voice generating mechanism is used to enable a speech unit to be connected. Since the pitch is connected continuously, a voice with higher naturalness can be synthesized.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a circuit configuration of an example of a speech synthesis device according to the present invention.

FIG. 2 is a block diagram showing a configuration example of a pitch modeler.

FIG. 3 is a block diagram showing a configuration example of a pitch generator.

FIG. 4 is a view for explaining an example of pattern extraction from natural speech.

[Explanation of symbols]

DESCRIPTION OF SYMBOLS 1 ... Audio compression coding apparatus, 2 ... Audio decoding apparatus, 10 ...
Encoded data storage (input device), 11 pitch generator,
12 ... sound source generator, 13 ... controller, 14 ... spectrum generator, 15 ... synthesized voice output device, 20 ... voice input device, 21
... spectrum analyzer, 22 ... sound source extractor, 23 ... pitch modeler, 24 ... encoder, 30 ... pitch frequency input device, 31 ... pattern matching device, 32 ... pitch pattern output device, 33 ... pitch pattern ROM, 34 pitch error extractor 35 pitch error time series encoder 40
Pitch pattern input device, 41: Accent pattern generator, 42: Phrase pattern scorer, 43: Phoneme pattern generator, 44: Adder, 45: Pitch error restorer, 4
6 Pitch error time series restorer

Claims (6)

(57) [Claims] Storage means for storing a plurality of pitch pattern models of a predetermined section of natural speech; control means for outputting a control signal for controlling a pitch frequency of a phoneme segment to be edited when synthesizing speech; Pitch generation means for decoding data encoded in association with the pitch pattern model using the pitch pattern model, so that the pitch frequency does not exceed an upper limit and a lower limit based on the control signal. And a pitch generating means having a pitch generator for generating a changed pitch frequency. 2. The speech decoding apparatus according to claim 1, wherein the predetermined section is a section of an accent component unit.
A speech decoding apparatus according to claim 1, wherein said pitch generating means includes an accent pattern generating means. 3. The speech decoding device according to claim 1, wherein the predetermined section is a section of an accent component unit and a section of a phrase component unit, and the pitch generation unit includes an accent pattern generation unit, a phrase pattern generation unit, A speech decoding apparatus comprising: an addition unit that adds the accent pattern generated by the accent pattern generation unit and the phrase pattern generated by the phrase pattern generation unit. 4. The speech decoding apparatus according to claim 1, wherein the predetermined section is a section of an accent component unit, a section of a phrase component unit, and a section of a phoneme component unit, and the pitch generation unit includes an accent pattern generation unit and an accent pattern generation unit. A phrase pattern generation unit, a phoneme pattern generation unit, an accent pattern generated by the accent pattern generation unit, a phrase pattern generated by the phrase pattern generation unit, and a phoneme pattern generated by the phoneme pattern generation unit. An audio decoding device comprising: an adding unit that performs addition. 5. The speech decoding apparatus according to claim 1, wherein the data includes encoded data obtained by encoding a pitch error between a natural speech pitch pattern and a pitch pattern model. Wherein the pitch generating means includes a pitch error restoring means for restoring data obtained by encoding the pitch error, and the pitch error restored by the pitch error restoring means is input to the adding means. Decryption device. 6. The speech decoding apparatus according to claim 5, wherein the data includes encoded data obtained by encoding pitch error data between a natural speech pitch pattern and a pitch pattern model in a time-series manner. The pitch generation means includes pitch error time series restoration means for restoring the pitch error time series encoded data, and inputs the pitch error data restored by the pitch error time series restoration means to the pitch error restoration means. Audio decoding device.