JP2931059B2 - Speech synthesis method and device used for the same - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION (Industrial application field) The present invention relates to a speech synthesis method in which speech data in a waveform area is stored in a storage area, and these speech data are read from the storage area to edit and synthesize speech. And its device.

(Prior Art) Research on rule synthesis that outputs an arbitrary voice according to rules has been active. PARCOR (Partial auto-correlation coefficient) is used as a speech synthesis method used for rule synthesis.
coefficient), LSP (line spectrum vs. line spectrum p)
LPC (linear predictive coding: linear predictive)
At present, the analysis / synthesis method of the coding type is most frequently used, but there is a problem in terms of clarity of the synthesized sound.
On the other hand, in an attempt to improve intelligibility, a method using a speech segment waveform extracted from a spectrum envelope of speech has been proposed (for example, Reference I: JP-A-60-22194 and Reference I).
I: Proceedings of the Acoustical Society of Japan (1-2-16), 1983
Mon., p. 73).

The method of obtaining speech unit waveforms disclosed in these documents will be briefly described with reference to FIGS. 2 (A) and 2 (B). FIG. 2A shows a flow for extracting a speech unit, particularly a speech unit of a voiced sound, and FIG. 2B shows a flow for extracting the speech unit.
The waveforms processed by this flow are shown next to the corresponding steps in that flow, respectively. In the figure, first, a sound generated by a speaker is captured as an original sound by a microphone or the like (step: S10). This original speech is cut out for a certain analysis section length (S20). Next, a spectrum envelope is extracted (S30). Next, an inverse FFT (FFT: Fast Fourier Transform) is performed on the spectral envelope with all phase components being zero, and a symmetric waveform is calculated (S4).
0). Next, a simple nonlinear conversion is performed to correct a sharp peak generated at the time origin of the symmetric waveform (S50). Next, a time window centering on the time origin is applied to the symmetrical waveform obtained by the above processing, the vicinity of the end point is cut, and the segment length is made constant (S6
0). Next, the unit power is normalized so that the power of all the extracted units becomes the same (S70).

In the case of rule synthesis, a method of preparing a large number of individual speech unit waveforms obtained in this way as basic units of speech and using them for synthesis has been considered, but in order to obtain a high-quality synthesized sound, It is necessary to faithfully represent the flow of the spectrum in the speech that is continuous to some extent. Therefore, a CV syllable or a VCV phoneme chain is generally selected as a basic unit of the speech. In that case, the speech unit waveform is extracted by CV or VC
The original speech such as V is subjected to the analysis processing shown in FIGS. 2 (A) and 2 (B) at regular time (frame) periods, and a series of obtained speech units are grouped into a basic speech. Treated as a unit and encoded (S8 in FIG. 2 (A))
0), stored in the storage device for the speech synthesizer.

(Problems to be Solved by the Invention) In the conventional speech synthesis technology using the encoded data of the waveform of the speech unit itself, the synthesis process is simple and a clear synthesized voice can be obtained. There is a problem that the amount of audio data to be stored is large because the waveform is directly handled.

SUMMARY OF THE INVENTION It is therefore an object of the present invention to provide a voice synthesizing method and apparatus for generating a clear synthesized voice with a small capacity by reducing the volume of voice data to be stored in a storage area.

(Means for Solving the Problems) In order to achieve this object, according to the method of the present invention, natural sound is analyzed every fixed frame period, and each sound is extracted from the spectral envelope of the sound in each frame. In a speech synthesis method in which speech segment data relating to a speech segment is stored in a storage device in advance, and speech is synthesized by synthesizing a speech segment using the speech segment data stored in advance in the storage device, The unit waveform is modified so that each speech unit has the same power, and all the frequency components of the speech unit waveform have the same phase characteristic between the speech units, and two adjacent frames are modified. The difference between the speech unit waveforms between the speech units extracted in step (a) is determined as a difference waveform, and each difference waveform is changed to encoded data to obtain the two adjacent frames. As speech unit data in the out subsequent frame, characterized in that stored in the storage device.

Further, in the embodiment of the present invention, preferably, the differential waveform is encoded by encoding the differential waveform encoded data of two adjacent frames with the number of encoded bits corresponding to the dynamic range of the amplitude of the differential waveform. The value and the number of coded bits are good.

In practicing the present invention, preferably, the encoded data of the differential waveform is obtained by extracting the dynamic range of the speech unit extracted in the succeeding frame of the two adjacent frames from the dynamic range of the differential waveform. Under the condition that it is smaller than the range, it is stored in the storage device, and under the condition that the magnitude of the dynamic range of the difference waveform is equal to or greater than the dynamic range of the speech unit, the It is preferable that encoded data of a speech unit waveform of the speech unit is stored in the storage device instead of encoded data of a difference waveform.

In the embodiment of the present invention, preferably, the encoded data of the difference waveform between two adjacent frames is a number of encoded bits that satisfies a threshold value of a predetermined quantization error when encoding the difference waveform. It can also be encoded.

In practicing the present invention, preferably, the encoded data of the differential waveform is obtained by extracting a speech unit from which a quantization error when encoding the differential waveform is extracted in a subsequent frame among the adjacent frames. It is stored in the storage device under the condition that it is smaller than the quantization error when encoding with the same encoding bit number, and the quantization error when encoding the differential waveform is Under the condition that the quantization error is equal to or larger than the quantization error at the time of encoding of the segment, the encoded data of the speech unit may be stored in the storage device instead of the encoded data of the difference waveform. it can.

In practicing the present invention, preferably, in the encoding of the differential waveform of two adjacent frames, the number of encoded bits of the differential waveform is adaptively determined for each differential waveform in accordance with the properties of the differential waveform, It is preferable that differential waveform encoded data encoded by the bit number and the encoded bit number are combined to form differential waveform encoded data.

Further, in the preferred embodiment of the present invention, for each frame, the speech segment waveform of the frame and the difference waveform of the speech segment waveform between the frame and the immediately preceding frame are encoded with the same encoding bit. In the case of encoding with numbers, it is better to determine which encoding accuracy is better, and store the encoded data with the better encoding accuracy in the storage device as encoded data of the speech unit waveform of the frame. good.

In practicing the present invention, preferably, the coded data stored in the storage device includes a flag for identifying whether the data is obtained by coding the difference waveform or the data obtained by coding the speech unit waveform itself. It is better to keep it.

Furthermore, according to the device of the present invention, a storage device in which speech unit data relating to speech units extracted from the spectral envelope of speech in each frame by analyzing natural speech at fixed frame periods is stored, By reading and playing back the speech unit data from the storage device, and synthesizing means for synthesizing a sound to be output to an external device.
Each speech unit data has the same power, and all the frequency components of the speech unit waveform are modified so as to have the same phase characteristic between the speech units. The encoded data of the difference waveform, which is the difference of the speech segment waveform between the speech segments extracted in each of the three frames, and the synthesizing means decodes the encoded data of the difference waveform from the storage device to perform speech segmentation. Is reproduced.

In practicing the invention, preferably, the encoded data is a value obtained by encoding the differential waveform with the number of encoded bits according to a dynamic range of the amplitude of the differential waveform, and the encoded bit number. Is good.

In practicing the present invention, the coded data is defined as a value obtained by coding the differential waveform with the number of coded bits satisfying a predetermined quantization error of the differential waveform and the coded bit number. Is good.

Further, according to the device of the present invention, a storage device in which speech unit data relating to speech units extracted from the spectral envelope of speech in each frame by analyzing natural speech at fixed frame periods, A synthesizing unit for synthesizing a voice to be output to an external device by reading and reproducing the voice unit data from the storage device, wherein a voice unit waveform is converted between the voice units. At the same time, the same power is obtained, and all the frequency components of the speech unit waveform are deformed so as to have the same phase characteristic between the speech units, and the speech components extracted between two adjacent frames are used. A speech element extracted in a subsequent frame of the two adjacent frames has a dynamic range of a difference waveform that is a difference between speech element waveforms. Under the condition that the difference is smaller than the dynamic range of the speech unit, the encoded data of the difference waveform is stored in the storage device as speech unit data, and the magnitude of the dynamic range of the difference waveform is Under the condition that it is equal to or larger than the dynamic range of the waveform, encoded data of the speech unit waveform of the speech unit is stored as speech unit data in the storage device, and the speech stored in the storage device is stored. The unit data includes a flag for identifying whether the data is data obtained by encoding the difference waveform or data obtained by encoding the speech unit waveform itself. Read the encoded data, and switch the reproduction of the speech unit between the reproduction based on the difference waveform and the reproduction based on the unit waveform according to the flag. It is better to do it.

Further, according to the device of the present invention, a storage device in which speech unit data relating to speech units extracted from the spectral envelope of speech in each frame by analyzing natural speech at fixed frame periods, A synthesizing unit for synthesizing a voice to be output to an external device by reading and reproducing the voice unit data from the storage device, wherein a voice unit waveform is converted between the voice units. At the same time, the same power is obtained, and all the frequency components of the speech unit waveform are deformed so as to have the same phase characteristic between the speech units, and the speech components extracted between two adjacent frames are used. A quantization error when encoding a difference waveform that is a difference between speech unit waveforms is the same as that of a speech unit extracted in a subsequent frame among the adjacent frames. Under the condition that less than the quantization errors after encoded by the encoding bits may be stored encoded data of the differential waveform as said speech unit data in the storage device, and,
Under the condition that the quantization error when encoding the difference waveform is equal to or larger than the quantization error when encoding the speech unit, the speech unit waveform is stored in the storage device as speech unit data. The speech unit waveform data stored in the storage device determines whether the data is data obtained by encoding the difference waveform or data obtained by encoding the speech unit waveform itself. The synthesizing unit reads the flag and the encoded data from the storage device, and switches the reproduction of the speech unit between the reproduction based on the difference waveform and the reproduction based on the unit waveform according to the flag. Good to do.

(Operation) In the method and apparatus of the present invention, the encoded data of the speech unit itself extracted for each frame from the spectrum envelope of the original speech is not stored in the storage device as speech unit data but is extracted in advance. The encoded data of the difference between the speech segment waveform of the adjacent frame and the speech segment waveform of the current frame is stored in the storage device as speech segment data. Then, in synthesizing the voice, the voice unit data is read from the storage device and decoded, and the voice unit waveform extracted from the spectral envelope of the voice is concatenated to synthesize the voice.

Focusing on these speech unit waveforms, the analysis frame period is usually selected to be about 5 to 20 ms in order to express the temporal change of the speech spectrum to some extent accurately. The change of the spectrum of the voice between is small. As shown in the above-described segment extraction process (FIGS. 2A and 2B), the speech segments of each sound have the same power and the same phase characteristics. Is a waveform expressing the difference in spectrum between adjacent frames as it is.

On the other hand, since speech unit waveforms between adjacent frames of continuous speech are very similar, the dynamic range of the difference waveform obtained by taking the difference is far greater than that of the original speech unit waveform. small. Therefore, the number of encoded bits when encoding the differential waveform can be clearly reduced as compared with the number of encoded bits when encoding the extracted segment waveform as it is. For this reason, by encoding and storing the difference waveform as speech unit data as compared with the case where the unit waveform is directly encoded and stored, the capacity of the speech unit data to be stored can be significantly reduced.

However, the difference waveform has a large variation in characteristics such as a dynamic range, unlike the speech unit waveform to which the power is normalized. That is, since the similarity of speech segment waveforms between adjacent frames has a considerable range depending on the type of speech or the stationary part and the transient part, it is efficient to always encode with the same number of encoding bits. is not. Therefore, when encoding the differential waveform, if the number of bits of the encoding is varied based on a certain evaluation criterion such as the dynamic range, the encoding can be performed even more efficiently, and the data amount can be reduced. .

As described above, in most speech sections, the unit waveforms between adjacent frames are very similar, and therefore, by encoding and storing the difference waveform obtained by taking the difference instead of the speech waveform, it is possible to greatly reduce the difference. However, the speech unit waveform between adjacent frames changes suddenly in a non-speech section or when a phoneme changes, and in some cases, the difference unit waveform can be encoded. There is also a possibility that the encoding accuracy is reduced rather than the case where the data is encoded.

In such a case, it is better to encode using the speech unit itself and store it. Therefore, in consideration of the overall information compression effect, both the encoding of the speech unit waveform itself between adjacent frames and the encoding of the difference between the speech unit waveforms, that is, the encoding of the difference waveform, are mixed and stored in the storage device. Good to put. Therefore, it is preferable that a flag for identifying whether the encoded data represents the differential waveform or the fragment waveform itself is included in the encoded data. Good. If you do this,
Since this flag can be stored in the storage device together with the speech unit data, it is possible to realize information compression of the speech unit and reduction of the coding error. In addition, since speech synthesis can be performed based on the flag, the quality of synthesized speech can be improved.

(Example) Hereinafter, an example of the present invention will be described with reference to the drawings.

FIG. 1A is a block diagram of a voice synthesizing apparatus for explaining a voice synthesizing method and apparatus according to the present invention, and FIG. 1B shows a basic process of voice unit waveform reproduction according to the present invention. FIGS. 3A and 3B are flowcharts showing a basic process of speech unit waveform encoding for explaining the present invention. FIGS. 4A and 4B are diagrams showing speech unit waveforms and difference waveforms. It is a figure showing the example of. Also, FIG.
FIG. 3 is a block diagram for explaining a state in which original speech is fetched and then speech unit data is stored in a storage device.

First, a speech synthesizer according to the present invention shown in FIG. 1 (A) has a storage device 10 for storing speech unit data of an original speech.
0 and a synthesizing means 102 for reading out voice unit data from the storage device 100, editing and synthesizing the voice unit data, and synthesizing voice for output to an external device. These storage devices 100
Control signals and the like necessary for the synthesizing means to function properly can be appropriately supplied from the control unit 104. Since the control unit 104 is a conventional means in this type of apparatus, a description thereof will be omitted. I do.

First, the basic process of speech unit waveform encoding will be described with reference to FIGS.

In the block diagram shown in FIG. 9, an original voice uttered by a speaker is captured by an original voice input device 10 and converted into an appropriate digital signal, and then a voice unit waveform is obtained by a voice unit creating device 20. , Necessary encoded data is obtained and stored in the storage device 100. The original voice input device 10 can be arbitrarily and appropriately configured with an acoustic-electric signal conversion device such as a microphone, a filter, and an A / D converter. Also,
The speech unit creation device 20 can also be arbitrarily and appropriately configured using a central processing unit (CPU) or the like. And these original voice input devices
10. The configuration is such that the speech unit creating device 20 and the storage device 100 can be operated while being controlled using a control signal or the like from the control unit 30, which is a conventional means.

The basic process of speech unit waveform encoding shown in FIG. 3 is performed in the speech unit creation device 20. Step (hereinafter, step is represented by S) 110 is a speech unit extraction process, and in the process of S110, the second
A speech unit waveform a _j ⁱ (j = 1, 2,... N) in a frame i is extracted according to a unit extraction process shown in FIG. Here, the subscript i indicates the frame number, the subscript j is the sample number of the unit, and the unit length is N. FIG. 4A shows an example of speech unit waveforms of frames 1 to 12 (i = 1,..., 12). Next, in S120, the current frame (hereinafter simply referred to as
Current frame hereinafter) frames preceding adjacent to unit waveform a _j ⁱ in i (hereinafter, simply referred to as previous frame) (the differential waveform b _j of unit waveforms A _j ^i-1 in the i-1) ⁱ
Is calculated. However, the reproduction waveform after encoding and decoding in the previous frame (i-1) is used as the unit waveform A _j ^i-1 here. However, in the first frame,
A _j ^i-1 is assumed to be all zero (0).

FIG. 4B shows an example of a difference waveform corresponding to the speech segment waveforms of frames 1 to 12 described above.

As can be understood from the example of the difference waveform between the speech segment waveform extracted from the actual speech and the segment waveform between adjacent frames shown in FIGS. 4 (A) and 4 (B), The difference between the dynamic ranges of the amplitudes of the waveforms is obvious, and it is clear that the storage capacity can be reduced by encoding and storing the difference waveforms instead of the fragment waveforms compared to the method of encoding the extracted fragment waveforms themselves. It is.

Next, in S130, the difference waveform calculated in the previous process of S120 is encoded, and the encoded data is stored in the storage device 100 as speech unit data in the frame i. Storage device 1
After accumulation of the encoded data finished to 00, in the processing of the next S140, it decodes the encoded difference waveform in S130, to obtain a decoded differential waveform B _j ^i. Next, in the processing of S150 adds the reproduced waveform A _j ^i-1 of the decoded differential waveform B _j ⁱ and the previous frame (i-1), calculates a reproduced waveform A _j ⁱ of the current frame i. In the process of S160, the frame is updated, and thereafter, the processes of the above-described steps S110 to S160 are repeated until the speech analysis section ends, and the encoded data of the differential waveform for all the frames is The storage of the segment data in the storage device 100 is completed.

In the example of FIG. 3 described above, the encoding method of the differential waveform encoding process in S130 is a method in which the quantization step width is fixedly determined based on a certain criterion in at least one speech unit such as PCM or logPCM. Here, an example of encoding by PCM encoding will be described below.

FIG. 5 shows an operation flow when the difference waveform is encoded by PCM encoding. Steps common to those in the flow shown in FIG. 3 are denoted by the same reference numerals, and detailed description thereof will be omitted. I do. In the flow shown in FIG. 5, S132 and
S134 is a process corresponding to S130 in FIG. This S132
In, the dynamic range of the differential waveform calculated in S120 is evaluated, and the number of bits n for encoding the differential waveform is optimally determined according to the magnitude of the dynamic range. The method of determining the number of encoded bits n will be described later. Next, in the process of S134, based on the coding bit number n defined in S132, the differential waveform b _j ⁱ to the coding, speech unit data in the frame i the encoded value with the number of coding bits n ( Storage device 100 as encoded data)
To be stored. After the storage of both the encoded value and the encoded bit number n in the storage device 100 is completed, the differential waveform encoded in S134 is decoded in the process of S140, and the decoded differential waveform B _j ^{i is} decoded. Get. Subsequent processing is performed in the same manner as the case described in the flow of FIG.

Here, the process of determining the number of coded bits in S132 in the flow of FIG. 5 will be described. 6 (A) and 6 (B) are explanatory diagrams of how to determine the number of coded bits. FIG. 6 (A) shows an example of a speech unit waveform, and FIG. 6 (B) shows an absolute difference waveform. An example of a digitized waveform is shown. Assume that the number of coded bits at which the required quantization precision is obtained when the speech unit waveform is directly coded is 8 bits (shown on the left side of FIG. 6B), and the dynamic range at that time is D ( 6B), the dynamic range of the differential waveform in the example shown here is in the range of D / 32, and 3-bit encoding is required to obtain the same quantization accuracy. It is understood that the number of bits is sufficient. As described above, in the process of determining the number of encoded bits in S132, the dynamic range of the differential waveform is evaluated to determine the number of encoded bits.

FIGS. 7 (A), (B) and (C) show speech unit waveforms extracted for each frame from actual speech, difference waveforms between adjacent frames, and FIGS. 5 and 6 (A) and (C). Each example of the number of quantization steps determined by the method described with reference to B) will be described. However, in order to make the display easier to see, the speech unit waveform is displayed on a reduced scale with respect to the difference waveform. The determined number of encoded bits is calculated assuming that the number of encoded bits in the case of directly encoding the speech unit waveform is eight. As shown in FIG. 7 (B), the dynamic range of the difference waveform is much smaller than the original speech unit waveform, and in the example shown in FIGS. 7 (A) to 7 (C), the number of bits is small. As shown in the graph, it can be seen that there is a large change from 1/16 to 1/64.

By the way, as described above, compression may not be performed as expected when an unvoiced sound or a phoneme is switched. Therefore, when storing the encoded data in the storage device 100, it is determined whether it is more advantageous to encode the differential waveform or to encode and accumulate the unit waveform itself to determine the encoding. It is preferable that the coded data include a flag for identifying the determination result.

 Hereinafter, this point will be described.

In most of the voice sections, as already illustrated in FIG. 7, the difference between the dynamic ranges of the amplitude of the unit waveform and the difference waveform is obvious, and in such a section,
It is clear that encoding the difference waveform is more efficient.

On the other hand, as shown in FIGS. 8 (A) and 8 (B) exemplarily showing the speech unit waveform and the difference waveform for each frame, the difference waveform in the phoneme change portion (frame 5) is particularly large. May be larger than the unit waveform. Therefore, in order to cope with such a case, the speech unit is stored as a difference according to the shape of the speech unit waveform, or
It is preferable to store or switch without taking the difference.

Therefore, to explain this, FIG. 10 shows an embodiment of an encoding sequence for obtaining encoded data without impairing the quantization accuracy. This encoding sequence is also performed by the speech unit creating apparatus 20 described above.

FIG. 10 is a diagram showing an encoding process in which a decision process of encoding a difference waveform or encoding a segment waveform itself in encoding a segment waveform is combined with a basic process of encoding in FIG. 3 is an operation flow showing a basic process of the first embodiment. In FIG. 10, steps common to those in FIG. 3 are denoted by the same reference numerals, and detailed description thereof will be omitted. In S112, S110
The dynamic range is evaluated for the speech unit extracted in step (1). As already described in S60 of FIG. 2A, the extracted speech unit has been subjected to power normalization, so that the dynamic range of the extracted speech unit is almost the same for each unit. However, some speech unit waveforms have sharp and sharp center segments depending on the spectrum shape. Therefore, it is more preferable to accurately evaluate the dynamic range of the speech unit waveform.

In performing such an evaluation, it is apparent that the amplitude is largest at the phase origin, that is, the center of the waveform, because of the nature of the speech unit waveform being a symmetrical waveform. . The evaluation value of the dynamic range of the speech unit waveform at this time is D _S. Next, the differential waveform is calculated in S120, and then the dynamic range of the differential waveform is evaluated in S122. In this case, since the center of the difference waveform does not always have the maximum amplitude, the entire difference waveform is examined. The evaluation value of the dynamic range of the differential waveform at this time is D _D. In S124, S112, the evaluation value of the dynamic range of the unit waveform obtained in S122 is compared with evaluation value D _D of the dynamic range of the D _S and the differential waveform, allocating an encoding method based on the results.

 Hereinafter, this point will be described.

If D _S > D _D, that is, if the evaluation value of the dynamic range of the segment is larger than the evaluation value of the dynamic range of the difference, the difference waveform is encoded in S130 as in the sequence of FIG. . However, at this time, an identification flag indicating that the differential waveform has been encoded is added to the encoded data together with the encoded data. The processing after S140 is the same as in FIG.

On the other hand, when D _S ≦ D _D, that is, when the evaluation value of the dynamic range of the difference is equal to or larger than the evaluation value of the dynamic range of the original unit, the unit waveform itself is encoded in S132. The encoded data is stored in the storage device 100 together with an identification flag indicating that the segment waveform itself has been encoded. Fragment reproduction processing in this case, the waveform addition process S150 for directly reproducing units A _j is obtained by the decoding of the unit waveform at S142 becomes unnecessary.

As described above, the method of providing a flag for identifying whether the difference waveform is encoded in the encoded data of the unit or the unit itself is added to the flow of FIG. As described above, the encoding shown in FIG. 5 can be substantially similarly added to the example in which the encoding is performed by the PCM method.

11 (A) to 11 (C) show the format of speech unit data stored in the storage device 100. FIG.

FIG. 11 (A) shows the format of data created by the basic process of speech unit waveform encoding of FIG. 3, and FIG.
FIG. 5 shows the format of data created in the process of performing the encoding by the PCM method, and FIG. 5C shows the format of data generated in the process of FIG.

In FIG. 11A, encoded data (here, a differential waveform) of a speech unit generated for each frame is sequentially stored.

In FIG. 11 (B), the data representing the unit of the i-th frame is composed of a pair of the coded bit number part of the frame and the coded data part of the differential waveform. The capacity of the encoded data part differs depending on the number of bits.

In FIG. 11 (C), the data representing the segment of the i-th frame includes a flag portion and a data portion. The flag portion indicates whether the data portion of the frame is obtained by encoding a differential waveform. A flag for identifying whether the one-side waveform itself is encoded is stored.

On the other hand, the waveform encoding method does not have a fixed quantization value such as PCM or logPCM, and furthermore, in order to perform more efficient encoding, a method of adaptively changing the quantization step width ( Adaptive PCM) or difference PCM that encodes the difference between adjacent samples (strictly speaking, the difference between the value after encoding and decoding of the previous sample and the value of the subsequent sample) instead of encoding the waveform itself. In the differential PCM, there is a method such as an adaptive differential PCM that adaptively changes the quantization step width. In such a scheme,
The encoding accuracy of a waveform does not always correspond to the dynamic range of the waveform. Therefore, the adaptive PC is used as the encoding method of the difference waveform or speech unit waveform in the embodiment.
When using a method such as M, differential PCM, or adaptive differential PCM, the evaluation criterion for determining the number of coded bits and the speech unit waveform used in the PCM encoding described above are directly encoded or the difference waveform is converted to a speech element. It is not appropriate to use the dynamic range of a waveform as a criterion for determining whether to encode in place of a single waveform. Therefore, each reference is replaced with the dynamic range of the waveform, and a quantization error representing the degree of error generated by encoding a waveform (speech unit waveform or difference waveform) of one frame. An example using the noise-to-noise ratio will be described below.

FIG. 12 shows a case where the determination of the number of S132 coded bits in the process of FIG. 5 for varying the number of coded bits of the differential waveform is performed by using the intra-frame signal-to-noise ratio instead of the dynamic range of the waveform. The processing flow of
The overall operation is almost the same as the flow in FIG. Twelfth
In the figure, the processing of S1320 to S1326 corresponds to the step of determining the number of coded bits in S132 of FIG. Further, in FIG. 12, steps denoted by the same reference numerals as those in FIG. 5 are the same processing as those in FIG. 5, and a description thereof will be omitted.

In the process of determining the number of coded bits, first, the minimum value n _{min of the} number of bits that can be coded is used as the initial value of the number n of coded bits.
(S1320). Next, this encoding bit number n, the differential waveform b _j ⁱ encodes, obtain coded data L _j ⁱ (S1321). Coded data L _j ⁱ is decoded value B _j ⁱ of the decoded differential waveform is calculated in the next step S1322 (S13
twenty two). In the next S1323, it is determined whether or not the currently given number of encoded bits n has reached a predetermined maximum value _{nmax of the} number of encoded bits.
If it reached n _max, and the coding bit number n at this time, the encoded data L _j ⁱ of the differential waveform is determined. on the other hand,
n calculates the coding accuracy in the next S1324 is less than the maximum value n _max. The encoding accuracy is expressed as an intra-frame signal-to-noise ratio as described above. If this is SN,
SN is given by the following equation.

On a frame in the signal-to-noise ratio SN calculated here, in step S1325, as compared to the threshold SN _th of a predetermined frame within the signal-to-noise ratio, if SN ≧ SN _th, encoding at this point n and coded data L _j ⁱ stored in the storage device 100 as the audio data at S141 as sufficient coding precision obtained in the number of bits n. On the other hand, SN <SN
If it is _th , the encoding precision is not sufficient, so the number of encoding bits is increased by one bit (S1326), and the processing from S1321 is repeated again. Thus, the processing of S1320 to S1326 is repeated until the desired encoding accuracy is obtained. However,
Since the object of the present invention is to reduce the capacity of audio data to be stored, in the process of S1322, the maximum value n _{max of the} number of encoded bits is determined, and the number n of encoded bits reaches n _max . If the encoding precision is the desired value SN
Whether or not _th is reached, no further increase in data is made. However, if n _max is the number of bits necessary to encode the speech unit waveform as it is, the further increase in the number of encoded bits in S1323 is terminated, and the quantization error is smaller than that of the conventional method. Does not increase.

FIG. 13 shows a process of FIG. 10 for performing encoding by determining whether to encode a speech segment waveform as it is or to encode a difference waveform in place of a speech segment waveform, according to the above-described criterion. 10 shows an operation flow when the dynamic range in FIG. 10 is changed to the intra-frame signal-to-noise ratio. here,
S1120 to S1122 correspond to S120 in FIG. 10, and S1220 to S1222 correspond to S122 in FIG. Also, in S1240,
Dynamic range D _S of speech unit waveforms in S124 of FIG. 10 is encoding the speech unit waveform, a frame in the signal-to-noise ratio SN _S when decrypted, the dynamic range D _D of the difference waveform is the difference coding waveforms, the frame within the signal-to-noise ratio SN _D for speech unit waveform when decrypted, replace, respectively. Further, S1300 and S1320 are respectively S1 of FIG.
30 and a part corresponding to S132, but since the decoding process has already been performed in S1121 and S1221,
In the processing of S130 and S132 in FIG.
It is a process that only stores it in 0.

Next, with reference to FIG. 1 (B), a basic process of the speech unit waveform reproduction in the speech synthesizer of the present invention will be described.
As described above, the storage device 100 stores, as speech unit data, only the encoded data of the difference waveform, or, in some cases, the encoded data of the speech unit waveform itself and the encoded data of the difference waveform. I have. Therefore, first, the synthesis means
In step S200, the coded difference waveform data is read from the storage device in step S200, and then, in step S210, it is determined whether the coded data of the difference waveform is coded data of the unit waveform itself by the coded data identification flag. I do. If it is determined that the difference waveform, calculates a decoded differential waveform B _j ⁱ decodes the coded data in the process of S220. Note that, of course, this decoding is performed by a method that matches the encoding. And the next
In S230, the sum of the speech unit reproduced waveform A _j ^i-1 in the previous frame decoded differential waveform B _j ⁱ obtained in S220 (i-1), the speech unit waveform in the frame i A _j ⁱ Get. On the other hand, if it is determined in the determination processing of S210 that the data is encoded data of the unit waveform itself, the speech unit itself is reproduced as in the related art, although not shown. Since then
While updating the frame in the process of S240, the above process is repeated for a necessary frame to reproduce a speech unit. The sound reproduced and synthesized in this way is subjected to a process for outputting it to a speaker, a computer, a display device, or another external device. Although not described in the description of FIGS. 3 and 1B, since the speech unit waveform extracted by the processing of FIG. 2 is symmetrical, it is not shown in FIGS. 3 and 1B. Needless to say, it is sufficient to handle half of the speech unit length in the actual process in the processing of (1).

(Effects of the Invention) As is clear from the above description, in the present invention, a segment waveform extracted from the spectrum envelope of speech is not encoded and stored as it is and used as speech segment data, but is extracted in an adjacent frame. Since the data obtained by encoding the difference waveform of the segment waveform to be obtained is stored as audio data, the storage capacity of the audio data can be significantly reduced. In other words, with the same storage capacity, much higher precision encoding is possible, and the quality of synthesized speech is improved.

In storing the difference waveform as audio data,
Because the number of bits for encoding is appropriately determined based on the dynamic range of the differential waveform or the criterion such as the quantization error at the time of encoding the differential waveform, encoding can be performed with an optimal amount of information without excess or deficiency. This makes it possible to obtain a high-quality synthesized sound with a small storage capacity.

Furthermore, by giving a flag indicating the method of encoding the speech unit waveform between frames to the encoded data,
The speech unit waveform can be represented with a small encoding error,
It is possible to further improve the quality of the synthesized sound.

[Brief description of the drawings]

FIG. 1 (A) is a block diagram of a main part of a speech synthesis apparatus for explaining a speech synthesis system and a speech synthesis apparatus of the present invention, and FIG. 1 (B) is a speech unit for explanation of the present invention. FIGS. 2A and 2B are flow charts showing the basic steps of waveform reproduction, FIGS. 2A and 2B are flow charts showing a speech unit extraction process for explanation of the prior art and the present invention, and FIG. 4 (a) and 4 (b) show an example of an actual speech unit waveform and a difference waveform of an actual speech by a plotter. FIG. 5 is an operation flow diagram showing a basic process of encoding a speech unit waveform by the PCM method. FIGS. 6A and 6B are diagrams for determining the number of encoded bits. FIGS. 7 (a) and (b) show the speech unit waveform and the difference of the actual speech. FIG. 8 is a diagram showing the relationship between the waveform and the number of coded bits, in which each waveform is drawn by a plotter. FIG. 8 is a waveform diagram drawn by a plotter, showing a speech unit waveform and a difference waveform of a phoneme change portion. FIG. 10 is a block diagram for explaining a state in which speech data is stored in a storage device after capturing an original speech. FIG. 11 (A), (B) and (C) are operation flowcharts showing a basic process of encoding including a process of determining whether or not to encode the data itself. FIG. 13 is an operation flow diagram showing a process of determining the number of coded bits by using a signal-to-noise ratio in a frame.
FIG. 9 is an operation flowchart of encoding when a criterion for encoding a differential waveform is used as an intra-frame signal-to-noise ratio. 100 storage device 102 synthesis means 104 control unit 106 data unit 108 flag unit

──────────────────────────────────────────────────続 き Continued on the front page (58) Field surveyed (Int.Cl. ⁶ , DB name) G10L 3/00-9/18 JICST file (JOIS)

Claims (13)

(57) [Claims] 1. Speech unit data relating to speech units extracted from a spectral envelope of speech in each frame by analyzing natural speech at regular frame intervals, and stored in a storage device in advance. In the speech synthesis method of synthesizing speech by synthesizing speech segments using the speech segment data stored in advance, the speech segment waveform is converted to the same power for each speech segment, For all frequency components of the waveform, each speech unit is deformed so as to have the same phase characteristic, and the difference of the speech unit waveform between the speech units extracted in two adjacent frames is used as a difference waveform, respectively. Then, each differential waveform is converted into encoded data and stored in a storage device as speech unit data in a subsequent frame of the two adjacent frames. A speech synthesis method characterized in that: 2. The speech synthesis system according to claim 1, wherein the encoded data of the difference waveform between two adjacent frames is
A speech synthesis method, wherein a value obtained by encoding the differential waveform and a number of encoded bits are respectively used as the number of encoded bits according to the dynamic range of the amplitude of the differential waveform. 3. The speech synthesis method according to claim 1, wherein encoded data of the differential waveform is extracted in a subsequent frame of the two adjacent frames whose dynamic range of the differential waveform is large. Under the condition that the dynamic range of the speech unit is smaller than the dynamic range of the speech unit, and that the magnitude of the dynamic range of the differential waveform is equal to or greater than the dynamic range of the speech unit. A speech synthesis method, characterized in that encoded data of a speech segment waveform of the speech segment is stored in the storage device instead of encoded data of the difference waveform under a condition. 4. The speech synthesis method according to claim 1, wherein the encoded data of the difference waveform between two adjacent frames is:
A speech synthesis method characterized in that the difference waveform is encoded with an encoding bit number that satisfies a predetermined threshold value of a quantization error. 5. The speech synthesis system according to claim 1, wherein the encoded data of the differential waveform is extracted from a subsequent frame among the adjacent frames, wherein a quantization error when encoding the differential waveform is extracted. Stored in the storage device under a condition that the quantization error is smaller than the quantization error when the encoded speech unit is encoded with the same encoding bit number, and the quantization at the time of encoding the differential waveform is performed. Under the condition that the error is equal to or larger than the quantization error at the time of encoding the speech unit, the encoded data of the speech unit is stored in the storage device instead of the encoded data of the difference waveform. A speech synthesis method characterized by being stored. 6. The speech synthesis system according to claim 3, wherein the encoded data stored in the storage device is data obtained by encoding the difference waveform or the speech unit waveform itself. A speech synthesis method characterized by including a flag for identifying encoded data. 7. The speech synthesis system according to claim 1, wherein in encoding a difference waveform between two adjacent frames,
The number of coded bits of the differential waveform is adaptively determined for each differential waveform according to the property of the differential waveform, and the differential waveform coded data coded by the coded bit number and the coded bit number are calculated. A speech synthesis method characterized in that the combined waveform data is differential waveform encoded data. 8. The speech synthesis system according to claim 1, wherein, for each frame, a speech segment waveform of the frame and a difference waveform of a speech segment waveform of a preceding frame adjacent to the frame are determined. When encoding is performed with the same encoding bit number, it is determined which encoding efficiency is higher, and the encoded data having the higher encoding efficiency is stored in the storage device as encoded data of the speech unit waveform of the frame. A speech synthesis method characterized by storing. 9. A storage device which stores speech unit data relating to speech units extracted from the spectral envelope of speech in each frame by analyzing natural speech at fixed frame periods, and And a synthesizing means for synthesizing a voice to be output to an external device by reading and reproducing the voice unit data. For all frequency components of the speech unit waveform, the speech units are deformed so as to have the same phase characteristic, and speech unit data is converted between the speech units extracted in two adjacent frames. And synthesizing means for decoding the encoded data of the difference waveform from the storage device to reproduce the speech unit. Speech synthesis apparatus and performing. 10. The speech synthesizer according to claim 9, wherein said encoded data is a value obtained by encoding said differential waveform with an encoding bit number corresponding to a dynamic range of the amplitude of said differential waveform, and said code. A speech synthesizer characterized in that the number of coded bits is set. 11. The speech synthesizer according to claim 9, wherein said encoded data is a value obtained by encoding each of said differential waveforms with the number of encoded bits satisfying a predetermined quantization error. And the number of coded bits. 12. A storage device storing speech unit data relating to speech units extracted from a spectral envelope of speech in each frame by analyzing natural speech at fixed frame periods, and A speech synthesizer including a synthesis unit for synthesizing a voice to be output to an external device by reading and reproducing the voice unit data. For all the frequency components of the speech segment waveform, the speech segments are deformed so as to have the same phase characteristic, and the speech segment waveform between the speech segments extracted in two adjacent frames is calculated. A speech unit waveform of a speech unit extracted from a subsequent frame of the two adjacent frames in which the magnitude of the dynamic range of the difference waveform is a difference. Under the condition that less than the dynamic range,
The storage device stores encoded data of the difference waveform as speech unit data, and a condition that a dynamic range of the difference waveform is equal to or larger than a dynamic range of the speech unit waveform. In the following, encoded data of the speech unit waveform is stored as speech unit data in the storage device, and speech unit data stored in the storage device is data obtained by encoding the difference waveform with the data. Or a flag for identifying whether the data is coded data of the speech unit waveform itself, and the synthesizing unit reads the flag and the coded data from the storage device, and according to the flag, A speech synthesizer characterized in that reproduction is performed by switching between reproduction based on a difference waveform and reproduction based on a segment waveform. 13. A storage device in which speech unit data relating to speech units extracted from a spectral envelope of speech in each frame by analyzing natural speech at fixed frame periods is stored, and A speech synthesizer including a synthesis unit for synthesizing a voice to be output to an external device by reading and reproducing the voice unit data. For all the frequency components of the speech segment waveform, the speech segments are deformed so as to have the same phase characteristic, and the speech segment waveform between the speech segments extracted in two adjacent frames is calculated. The quantization error when encoding the difference waveform that is the difference is the same as the speech unit waveform of the speech unit extracted in the subsequent frame of the adjacent frames. Under the condition that it is smaller than the quantization error when encoding with the number of encoded bits, the encoded data of the differential waveform is stored in the storage device as the speech unit data, and the differential waveform The quantization error when encoding
Under the condition that the quantization error at the time of encoding the speech unit is equal to or greater than the quantization error, encoded data of the speech unit waveform is stored as speech unit data in the storage device, The stored speech unit data includes a flag for identifying whether the data is data obtained by encoding the difference waveform or data obtained by encoding the speech unit waveform itself. A voice synthesis unit that reads out the flag and the encoded data from the device and reproduces a speech unit in accordance with the flag by switching between reproduction based on a difference waveform and reproduction based on a unit waveform.