JPH08110796A - Voice emphasizing method and device - Google Patents

Abstract

PURPOSE: To conduct the tone quality improvement, audition compensation, and voice emphasis of the deteriorated voice by emphasizing the temporal change of the acoustic feature of a voice. CONSTITUTION: A voice processing device is provided with a means inputting a voice, a means analyzing and processing the voice, a means reproducing and outputting the voice, a feature quantity calculation section calculating the acoustic feature quantity of the voice from the voice wave-form, a feature quantity change quantity calculation section calculating the temporal change quantity for a unit time of the acoustic feature quantity, a temporal change quantity change section changing the temporal change quantity, an acoustic feature quantity change section changing the acoustic feature quantity with the changed temporal change quantity, and a wave-form reconstitution section 133 reconstituting the voice wave-form from the changed acoustic feature quantity.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a sound processing device, and more particularly to a sound signal processing device for improving sound quality of deteriorated sound, compensating hearing for a hearing impaired person, and enhancing sound.

[0002]

2. Description of the Related Art The easiness of hearing a voice is affected by a voice organ of a speaker, a speech method, a hearing ability of a listener, a speech environment and a speech environment, a restriction of a transmission path, a change in sound quality due to information compression and the like.
If the speaker's vocal organs are damaged or if the vocal organs are not used properly during speech, the acoustic characteristics of the voice may be different or unclear from the speaker's intention. Further, when the hearing ability of the listener is deteriorated, it becomes difficult to sufficiently capture the acoustic features. If there is noise in the utterance environment, the receiving environment, the transmission system, etc., or if the sound quality is deteriorated due to the restriction of the pass band of the transmission path or information compression, the acoustic characteristics are distorted or masked by other sounds. Sometimes.
The deterioration of the audibility due to such factors can be recovered by using a technique for processing the acoustic characteristics of the voice.

In order to process acoustic characteristics for improving the audibility of speech, characteristics on the frequency axis are changed such as changing the frequency spectrum shape according to the audible level of the listener and emphasizing the formant peak. There is a method of increasing the clarity of sound quality by static processing.

On the other hand, as a conventional technique for emphasizing the temporal change that represents the intonation of speech, "sound quality control by emphasizing and suppressing formant changes" (Toki, Kuwahara, Onkyo, October 1986,
pp145-146), there is a method of emphasizing the temporal change of the vowel formants extracted by performing the linear prediction analysis. In addition, "Proposal and Evaluation of Speech Enhancement Method Based on Compensation of Continuous Masking" (Suzuki et al., IEEJ, SP91-135, Mar 199)
2, pp31-37), there is a method to emphasize the rising part of the sound pressure level of the speech waveform by multiplying the waveform by a window function of a specific shape.

For the purpose other than improving the sound quality, in order to improve the accuracy of machine recognition of speech, as a method of enhancing the phonological property by emphasizing the temporal change of the recognition parameter, for example, "word speech recognition by emphasizing spectrum change". (Furui, Voice Study Material, S85-77, Jan 1986, pp597-604)
There is. In this method, the time derivative of the LPC cepstrum envelope and power in the target time section is approximated by a quadratic regression coefficient, and the result obtained by multiplying the regression coefficient by an appropriate coefficient is added to the LPC cepstrum envelope and power in the target time section, respectively. Emphasize the change over time by adding.

As a method for improving the easiness of hearing by changing the utterance speed, for example, as described in "Voice storage / reproduction device" (Japanese Patent Laid-Open No. 3-48300), a periodical portion of a voiced sound portion having a relatively large power is periodically used. There is a conventional technique in which only the speaking rate is changed without changing the pitch of the voice by inserting or deleting the waveform.

[0007]

Of the above-mentioned conventional techniques,
The method of processing the characteristic on the frequency axis does not emphasize the change of the characteristic on the time axis.

A method for emphasizing the change in formant is as follows.
Although it is necessary to perform formant extraction prior to emphasizing, formants are generally not prominent in unvoiced sounds, so they are applicable only to voiced sounds for which formants can be extracted by conventional means. Therefore, there is a problem that it is not suitable for emphasizing continuous voice including unvoiced sound.

In the method of emphasizing the spectrum change, the LPC cepstrum envelope is mainly used as an emphasizing object, but this emphasizing is intended for acoustic parameters used for machine recognition, and speech restoration after emphasizing is not considered. There was a problem. Further, the conventional technique has a problem that the LPC cepstrum envelope and power of the original sound in the target time section are emphasized for each analysis frame, and the emphasis result of the previous frame is not accumulated.

A method for multiplying a waveform by a window function having a specific shape is as follows:
Although the sound pressure level at the rising edge of the voice section is emphasized, there is a problem that the temporal change of the power of the entire voice is not emphasized.

According to the method of changing only the utterance speed, the time structure of the short time section of the voiced part waveform is preserved, but the continuous time change is broken, so that the voice with the changed utterance speed is spoken by the human at that speed. There is a problem that the sound quality may be unnatural especially in the transition section, as compared with the above speech.

[0012]

In order to solve the above-mentioned problems, in a voice emphasizing method of the present invention and an apparatus using the same, a feature quantity calculating section for calculating an acoustic feature quantity of a voice from a voice waveform, A feature amount change amount calculation unit that calculates a time change amount of the acoustic feature amount per unit time, a time change amount change unit that changes the time change amount, and the acoustic amount using the changed time change amount. An acoustic feature amount changing unit that changes the feature amount and a waveform reconstructing unit that reconstructs a voice waveform from the changed acoustic feature amount are provided.

Further, among the acoustic feature quantities of speech, a fundamental frequency, a power, and a frequency spectrum, which have a relatively large contribution to the sound quality due to a remarkable change in characteristics over time, are used, and their temporal changes are used. Are provided at the same time, individually or in combination.

The result of the acoustic feature quantity changing unit adding the time change amount of the acoustic feature quantity per unit time changed by the time change amount changing unit to the acoustic feature quantity unit time before the target time section. A means for changing the acoustic characteristic amount of the target time section is provided by setting the acoustic characteristic amount of the target time section to.

As the acoustic feature amount before the unit time, the acoustic feature amount changed by the acoustic feature amount changing unit before the unit time is used.

A speech rate conversion section for changing the speech rate and a voice emphasis section for changing the time change are provided.

A frequency characteristic changing section for changing the frequency characteristic and a voice emphasizing section for changing the time change are provided.

A speech rate conversion section for changing the speech rate, a frequency characteristic changing section for changing the frequency characteristic, and a voice emphasizing section for changing the time change are provided.

[0019]

Operation: A feature amount calculation unit for calculating the acoustic feature amount of a voice from a voice waveform, a feature amount change amount calculation unit for calculating the time change amount of the acoustic feature amount per unit time, and the time change amount A time change amount changing unit for changing, an acoustic feature amount changing unit for changing the acoustic feature amount using the changed time change amount, and a waveform reconstructing unit for reconstructing a speech waveform from the changed acoustic feature amount. By providing the construction unit, it is possible to change the time change of the voice.

In the acoustic feature quantity of the voice, the fundamental frequency, the power, and the frequency spectrum, which have a relatively large contribution to the sound quality because the characteristic changes with time are remarkable, are used. It is possible to effectively change the time change of the voice by providing a means for changing simultaneously, alone or in combination.

The result of the acoustic feature quantity changing unit adding the time change amount of the acoustic feature quantity per unit time changed by the time change amount changing unit to the acoustic feature quantity unit time before the target time section. Is set as the acoustic feature amount of the target time section, and by providing a means for changing the acoustic feature amount of the target time section, it is possible to change the temporal change of the acoustic feature of the voice.

By using, as the acoustic feature amount before the unit time, the acoustic feature amount changed by the acoustic feature amount changing unit before the unit time, it is possible to accumulate the change in the time change of the voice. become.

By providing the speech speed conversion unit for changing the speech speed and the voice emphasizing unit for changing the time change, the speech speed and the time change can be changed at the same time.

By providing the frequency characteristic changing unit for changing the frequency characteristic and the voice emphasizing unit for emphasizing the time change, the frequency characteristic and the time change can be changed at the same time.

By providing a speech rate conversion section for changing the speech rate, a frequency characteristic changing section for changing the frequency characteristic, and a voice emphasizing section for emphasizing the time change, the speech rate, the frequency characteristic and the time change can be changed at the same time. It will be possible.

[0026]

Embodiments of the present invention will be described below with reference to the drawings.

FIG. 1 is a block diagram for explaining one embodiment of a voice emphasizing device according to the present invention. In FIG. 1, an input voice is converted into an electric signal 102 through a microphone 101 and then converted into a digital waveform signal 104 in an A / D converter 103. Frame processing unit 105
Is a frame waveform signal 106 obtained by cutting out a short-time section waveform from the digital waveform signal 104 using a time window of about several tens of milliseconds to hundreds of milliseconds at an analysis cycle of an appropriate time interval.

The frame power calculator 107 calculates the frame average power 108 of the frame waveform signal 106. The power change amount calculation unit 109 calculates a change amount 110 of the frame average power 108 per unit time. The amount of change in the frame average power per unit time is represented by the difference between the frame average power of the current analysis frame and the frame average power of the previous frame before enhancement. However, when the analysis cycle is short, a regression coefficient may be calculated from the frame average powers of several frames before and after it and used to reduce the variation. The power change emphasis unit 111
The post-enhancement frame average power 112 is obtained by multiplying the change amount 110 of the frame average power per unit time by an appropriate coefficient and adding it to the post-enhancement frame average power of the previous frame.

In addition, the fundamental frequency extraction unit 113 is, for example, "digital audio processing" (Furui, Tokai University Press, 19
85) from the frame waveform signal 106 to the fundamental frequency 1 for each analysis frame based on the autocorrelation analysis.
14 is extracted. The basic frequency change amount calculation unit 115 calculates the change amount 116 of the basic frequency 114 per unit time. The amount of change in the fundamental frequency per unit time is represented by the difference between the fundamental frequency of the current analysis frame and the fundamental frequency of the previous frame before enhancement. However, when the analysis cycle is short, a regression coefficient may be calculated from the fundamental frequencies of several frames before and after it in order to reduce the variation, and this may be used.
When the fundamental frequency cannot be extracted like unvoiced sound, no enhancement is performed or processing is performed using the fundamental frequency interpolated by linearly interpolating the fundamental frequencies of the voiced sounds before and after or with a spline function. .

On the other hand, the frame waveform signal 106 is converted into a frequency spectrum 118 by the Fourier transform unit 117. The spectrum normalization unit 119 normalizes the frequency spectrum 118 with the frame average power 108 to calculate a normalized frequency spectrum 120. The spectrum envelope calculation unit 121 calculates the spectrum envelope 122 from the normalized frequency spectrum 120. In this embodiment, the Fourier cepstrum envelope is used as the spectrum envelope.
The Fourier cepstrum envelope is obtained by calculating the peak envelope of the logarithmic power spectrum based on cepstrum analysis. The calculation method is described, for example, in "Digital Speech Processing" (Furui, Tokai University Press, 1985). The spectrum envelope change amount calculation unit 123 calculates a change amount 124 of the spectrum envelope 122 per unit time. The amount of change in the spectrum envelope per unit time is represented by the difference for each frequency component between the spectrum envelope of the current analysis frame and the spectrum envelope of the previous frame before enhancement. However, when the analysis cycle is short, the regression coefficient may be calculated from the spectral envelopes of several frames before and after it in order to reduce the variation, and this may be used. The spectrum change emphasizing unit 125 multiplies the amount of change 124 of the spectrum envelope per unit time by an appropriate coefficient and adds the result to the spectrum envelope after emphasis of the previous frame so that the result becomes the spectrum envelope after emphasis of the current analysis frame. Then, the normalized spectrum 120 of the current analysis frame is emphasized, and the normalized spectrum 126 after the spectrum change emphasis is obtained. At this time,
A value obtained by converting the difference between the spectrum envelope after emphasis and the spectrum envelope before emphasis into a complex spectrum from a logarithmic power spectrum is multiplied by the real part and the imaginary part of each frequency component of the normalized spectrum 120, and the spectrum is preserved while the phase is preserved. Emphasize.

The fundamental frequency change emphasizing section 127 multiplies the variation 116 of the fundamental frequency per unit time by an appropriate coefficient and adds it to the emphasized fundamental frequency of the previous frame to obtain the enhanced fundamental frequency. Pitch-synchro
nous waveform processing techniques for text-to-sp
eech synthesis using diphones "(Charpentier and Mo
ulines, Eurospeech 89, vol 2, Sep 1989, pp13-1
9) The fundamental frequency of the normalized spectrum 126 after emphasis of spectrum change is changed by using the method described above to obtain the normalized spectrum 128 after emphasis of change of fundamental frequency. The normalized spectrum 128 after emphasizing the fundamental frequency change is obtained by the inverse Fourier transform unit 12
At 9, the frame waveform signal 130 is inversely converted. The frame waveform recombining unit 131 multiplies the frame waveform signal 130 by the post-enhancement frame average power 112 to resynthesize the power change emphasized frame waveform signal 132. The frame waveform signal 132 after power change enhancement is reconstructed into a continuous waveform 134 by the waveform reconstructing unit 133, and the D / A converting unit 135.
After being converted into an analog electric signal 136 in, the sound is output from the speaker 137 as emphasized sound.

The fundamental frequency extraction method includes, in addition to the method based on the autocorrelation analysis described in the above embodiment, a method using a cepstrum analysis, a method by measuring the number of zero-crossings of a waveform, etc. The present speech emphasizing device can also be realized by using it.

Further, in the present embodiment, the average power is used for normalizing the frequency spectrum, but it is also possible to normalize with the maximum value of the frequency level.

Further, as the spectrum envelope, the Fourier cepstrum envelope is used.
It is also possible to use a Bark spectrum or a mel spectrum. Here, the Bark spectrum and the mel spectrum are based on the frequency resolution of the auditory sense, and by using the Bark spectrum or the Mel spectrum, it is possible to perform the enhancement in a method close to the frequency analysis of the human auditory sense. By the way, since the resolution of the Bark spectrum and the mel spectrum becomes coarser as the frequency becomes higher, a plurality of points of the Fourier spectrum are assigned to each order component corresponding to the high frequency component in these spectra. In such a case, a plurality of points of the Fourier spectrum assigned to one order are emphasized in equal proportions. The Bark spectrum and the Mel spectrum can be converted from the Fourier spectrum by using a conversion formula or conversion table of the frequency axis described in “Hearing Handbook” edited by Namba (Nakanishiya Publishing Co., Ltd., 1984). It is also possible to directly use the complex frequency spectrum without using the spectrum envelope to emphasize the time change of the frequency spectrum, but in this case, not only the frequency level but also the time change of the phase are emphasized at the same time.

As the coefficient value, an optimum value can be used for each characteristic amount to be changed, and it is not always necessary that the coefficient value is the same.

Since this embodiment uses only the audio signals in the temporal vicinity of the analysis frame, it is clear that the processing can be performed in real time within a range where the listener does not feel any delay.

FIG. 2 is a diagram for explaining a method of highlighting the time variation of the feature quantity.

In the figure, F is the feature amount of the original sound, and F'is the feature amount after emphasis. F and F ′ are functions of time and have discrete values for each unit time. Now, the feature amount of the original sound at time t is F (t), and the feature amount after emphasis is F ′ (t).
Then, at t + 1, the emphasized feature amount F ′ (t +
1) is the slope dF (t) / d of F (t) at time t
t

[0039]

[Equation 1]

Then, it can be expressed by the following equation using the emphasis coefficient a.

[0041]

[Equation 2]

Similarly, F '(t + 2) is expressed by the following equation.

[0043]

(Equation 3)

Therefore,

[0045]

[Equation 4]

Then, the emphasis of the feature amount is accumulated. In the present figure, the feature amount is a scalar for simplification, but it is obvious that the feature amount can be easily expanded even when it is a vector. Here, a is a real number larger than -1, and emphasizes a temporal change when a> 0 and suppresses it when -1 <a <0. a
When = 0, the time change of the original sound is not changed.

FIG. 3 is a conceptual diagram of temporal change emphasis when the feature amount is a scalar. The feature amount F is emphasized like F ′ by emphasizing the time change amount.

It should be noted that by changing the sound source and the vocal tract shape, the voice continuously utters voice segments having different articulation styles to form the intended language. Therefore, when the utterance is not accurate or the listening condition is restricted, the acoustic feature of each voice segment is dulled by continuous utterance, which makes it difficult to hear. In such a case, it is considered that the audibility is improved by emphasizing the continuously uttered speech so that the acoustic feature of each voice segment becomes the original feature of the phoneme. The boundary of each voice segment of the continuously uttered voice can be regarded as the time point when the temporal change of the acoustic feature amount is the maximum. Therefore, if the maximum value of the temporal change of the acoustic feature amount is detected and a = 0 is set only at this time, the temporal change is changed for each voice segment.

FIG. 4 is a block diagram of a time change amount changing unit having a time change maximum value detecting unit of the acoustic feature amount. In the figure, reference numeral 401 represents a time change amount changing unit. The time change amount 402 of the acoustic feature amount is stored in the time change amount holding unit 403, and the time change amount comparison unit 404 compares it with the time change amount 405 of the previous frame held in the time change amount holding unit. The difference is calculated, and the maximum value detection unit 407 determines whether the difference 406 is the maximum. Based on the determination result 408, the emphasis coefficient determination unit 409 sets the emphasis coefficient 410 to 0 if the amount of change with time is maximum, and otherwise sets a predetermined real number other than 0. The time change amount changing unit 411 changes the time change amount 402 using the emphasis coefficient 410 to obtain the changed time change amount 412.

On the other hand, when the voice is uttered sufficiently slowly, the stationary part of the voice segment having a relatively long duration such as a vowel may have the original acoustic characteristics of the voice segment. In such a case, it is possible to save the characteristics of the original sound by regarding the time point when the amount of time change is minimal as the stationary part start time and not changing the time change.

FIG. 5 is a block diagram of a time change amount changing unit having a time change minimum value detecting unit for the acoustic feature amount. In the figure, reference numeral 501 represents a time change amount changing unit. The time change amount 502 of the acoustic feature amount is stored in the time change amount holding unit 503, and the time change amount comparison unit 504
The difference from the time change amount 505 of the previous frame held in the time change amount holding unit is calculated, and the minimum value detection unit 507 determines whether the difference 506 is the minimum. Based on the determination result 508, the emphasis coefficient determination unit 509 sets the emphasis coefficient 510 to 0 when the time change amount is minimal, and otherwise sets a predetermined real number other than 0. The time change amount changing unit 511 changes the time change amount 502 using the emphasis coefficient 510 to obtain the changed time change amount 512.

Further, the absolute value of the time change amount is calculated, and when the absolute value is smaller than a predetermined value, the section is regarded as a steady section and the characteristic of the original sound is preserved by not changing the time change. It is possible to

FIG. 6 is a block diagram of a time change amount changing unit having a time change minimum value detecting unit for the acoustic feature amount. In the figure, 601 represents a time change amount changing unit. The time change amount absolute value calculation unit 603 calculates the time change amount 6 of the acoustic feature amount.
The absolute value 604 of 02 is calculated, and in the time change amount absolute value determination unit 605, the absolute value 604 is compared with a predetermined value, and if it is smaller than this, the emphasis coefficient 608 is set to 0, otherwise, It is a real number other than 0, which is determined in advance.
The time change amount changing unit 609 changes the time change amount 602 using the emphasis coefficient 608 to obtain the changed time change amount 610.

Normally, the temporal change of the acoustic feature amount is maximum at the boundary of the voice segment and minimum at the center of the voice segment. Therefore, by changing the emphasis coefficient according to the absolute value of the temporal change amount, it becomes possible to control the temporal change of the emphasized acoustic feature amount. When the system value is increased as the absolute value of the time change amount is increased, the change is emphasized more than the change near the voice segment boundary and the change near the center is less emphasized, so that the segment boundary becomes clearer. The duration of the stationary part of the feature is increased. Figure 7
A conceptual diagram of time change emphasis when the system numerical value is increased as the absolute value of the time change amount is increased is shown in FIG.

When the system value is increased as the absolute value of the time variation is smaller, the emphasis of the change near the voice segment boundary is reduced and the emphasis of the change near the center is increased to increase the original sound at the segment boundary. It is possible to save the change in the feature amount of. FIG. 8 shows a conceptual diagram of time variation emphasis when the system numerical value is increased as the absolute value of the time variation is smaller.

In general, among the various articulatory modal segments that make up a voice, it is considered that a segment that does not accompany vocal cord vibration, such as a voiceless consonant, has a small power and that the effect of articulatory coupling on the frequency spectrum is not significant. Further, in unvoiced sound, the fundamental frequency cannot be extracted because the vocal cords do not vibrate. Therefore, if the voice power is smaller than a threshold value set in advance, the portion with relatively small power is regarded as the unvoiced portion, and the processing amount is reduced by not performing the time change emphasis of the time change emphasis of the frequency spectrum or the fundamental frequency. Can be reduced. FIG. 9 shows a configuration example of a voice enhancement device having a power determination unit. In the figure, an input voice is an electric signal 90 through a microphone 901.
After being converted into 2, it is converted into a digital waveform signal 904 in the A / D converter 903. Frame processing unit 90
Reference numeral 5 is an analysis cycle at an appropriate time interval, and a short-time section waveform is cut out from the digital waveform signal 904 using a time window of about several tens of millimeters to hundreds of milliseconds to form a frame waveform signal 906. The frame power calculator 907 calculates the frame average power 908 of the frame waveform signal 906. The frame average power 908 is compared with a predetermined threshold value in the power determination unit 909, and when it is determined to be larger than the threshold value, the spectrum and fundamental frequency emphasis command 91
Give 0. The frame level time change emphasis unit 911 performs the spectrum and fundamental frequency emphasis of the frame waveform signal 906 when the spectrum and fundamental frequency emphasis instruction 910 is detected, and the frame waveform signal 906 when the spectrum and fundamental frequency emphasis instruction 910 is not detected. 906
Do not change the spectrum and fundamental frequency of. At this time, the average power emphasis processing is performed regardless of the spectrum and fundamental frequency emphasis instruction 910. The emphasized frame waveform signal 912 is converted into a continuous waveform 91 by the waveform reconstructing unit 913.
4 is reconstructed, converted into an analog electric signal 916 in the D / A conversion unit 915, and then output as emphasized sound from the speaker 917.

FIG. 10 is an example of a voice emphasizing a temporal change using the present invention. In the figure, (a) is a spectrogram of the original voice uttered as "baking bread", and (b) is a spectrogram of the time-change emphasized voice. It can be seen from FIG. 10 that the time change of the entire spectrum is emphasized.

FIG. 11 shows an example of the configuration of the voice emphasizing device according to the present invention. The analog electrical signal 1101 of the input voice is
Filter 1 after level adjustment by amplifier 1102
103 A band outside the required band is removed, and after being converted into a digital signal by the A / D conversion unit 1104, a voice enhancement process is performed by the DSP. The voice emphasis program and data are loaded from the program memory 1106 and the data memory 1107, respectively. The voice-enhanced digital signal is D
After being converted into an analog signal by the A / A conversion unit 1108, the outside of the required band is removed, the level is adjusted by the amplifier 1110, and the amplified analog audio electric signal 1111 is output.

FIG. 12 shows an embodiment in which the voice emphasizing device of the present invention is directly used. The voice is the microphone 120.
1 is input and converted into an analog electric signal, and the emphasized voice analog electric signal is converted into voice and output by the speaker 1202. If the speech impaired person uses this embodiment, it is possible to compensate the speech.

FIG. 13 shows an example in which the voice emphasizing device of the present invention is applied to an analog line telephone. Transmission line 130
The voice input to the analog telephone 1302 through 1 is
After being emphasized by the voice emphasizing device, the sound is output from the speaker 1304 of the handset 1303. The utterance of the listener is input from the microphone 1305 of the handset 1303 and directly transmitted to the analog telephone.

FIG. 14 shows an example in which the voice emphasizing device of the present invention is applied to a telephone for digital lines. In this embodiment, since the input voice is already a digital signal, A
The / D conversion process can be omitted.

FIG. 15 shows an embodiment in which the voice emphasizing device according to the present invention is used as a preprocessing unit of a transmitter such as a television or a radio.

FIG. 16 shows an embodiment in which the voice emphasizing device of the present invention is used as a post-processing unit of a receiver such as a television or a radio.

FIG. 17 shows an example of a configuration in which the emphasis amount of the time change amount of the voice emphasizing device according to the present invention can be adjusted from the outside. The emphasis amount is adjusted in the emphasis amount controller 1702 using the adjustment knob 1701.

FIG. 18 shows an example of the configuration of a voice emphasizing device having a plurality of adjustment knobs for each acoustic feature amount. The amount of emphasis of the plurality of acoustic feature values to be emphasized is adjusted by the adjustment knob 1801.
Is adjusted by the emphasis amount controller 1802 for each feature amount.

FIG. 19 shows an example of the configuration of a voice conversion device having a speech rate conversion unit and a time change emphasis unit. In this configuration example, the speech speed conversion unit is arranged in the front stage and the time change emphasis unit is arranged in the rear stage, but conversely, the time change emphasis unit is arranged in the front stage.
It is also possible to configure the speech rate conversion unit in the subsequent stage.

FIG. 20 shows an example of the configuration of a voice conversion device having a frequency characteristic changing section and a time change emphasizing section. In this configuration example, the frequency characteristic changing unit is arranged in the front stage and the time change emphasizing unit is arranged in the rear stage, but it is also possible to conversely configure the time change emphasizing unit in the front stage and the frequency characteristic changing unit in the rear stage. is there. Note that the frequency characteristic can be changed, for example, by using the method described in "Hearing compensation device" (Japanese Patent Application No. 4-254355) in accordance with the surrounding environment and the hearing ability of the listener.

FIG. 21 shows an example of the configuration of a voice conversion device having a frequency characteristic changing section, a time change emphasizing section, and a speech rate converting section. In this configuration example, the frequency characteristic changing unit, the time change emphasizing unit, and the speech speed converting unit are configured in this order.
It is also possible to change the order of emphasis.

In the configuration examples shown in FIGS. 19 and 21, if the conversion rate of the utterance speed is made equal to the emphasis rate of the time change amount in the time change emphasis section, the utterance speed is kept while the time change of the original sound is preserved. Can be converted.

Generally, it is said that the change pattern of the articulatory organ can be expressed by a critical damping secondary system. Therefore,
It is considered that the temporal change pattern of the acoustic feature amount based on the change of the articulatory organ can also be approximated by the model of the secondary system of critical braking. Therefore, the time change of the acoustic feature quantity is changed by approximating the time change of the acoustic feature quantity by using a critical braking quadratic system model, and setting the time change of the model as the time change of the acoustic feature quantity. Is possible. In order to realize this method, the feature amount change amount calculation unit of the speech enhancement apparatus according to the present invention includes a feature amount time change maximum point,
Also, a maximum point detection unit, a minimum point detection unit, and a critical braking secondary system model estimation unit that sequentially detect the minimum points are provided.

FIG. 22 shows an example of the configuration of a time change amount changing unit for changing the time change of the characteristic amount using the critical braking secondary system model. In the figure, 2201 represents a time change amount changing unit. The time change maximum point detection unit 2203 detects a maximum point from the time series of the time change amount 2202 of the acoustic feature amount, and outputs a maximum point detection signal 2204. The time change minimum point detection unit 2205 detects a minimum point from the time series of the time change amount 2202 of the acoustic feature amount, and outputs a minimum point detection signal 2206. The model estimating unit 2207 estimates the parameters of the field braking quadratic system model using a time series of a short section of several tens of milliseconds behind the maximum point, and removes the model up to a minimum value adjacent to the maximum point behind the maximum point. Also, the parameters of the field braking quadratic system model are estimated using a time series of a short section of several tens of milliseconds ahead of the maximum point, and the model is extrapolated to a minimum value adjacent in front of the maximum point. By doing so, the time change amount 2208 after the change of the feature amount is determined. Enhancement coefficient determination unit 2209
Calculates the emphasis coefficient 2210 by taking the difference between the changed time change amount 2208 and the time change amount of the feature amount of the original voice. The time change amount changing unit 2211 is configured to change the emphasis coefficient 22.
10 is used to change the time change amount 2202 to be the changed time change amount 2212.

FIG. 23 shows the time variation of the acoustic feature quantity by approximating the time series of the acoustic feature quantity in the vicinity of the time change maximum point by a critical damping quadratic system model and extrapolating the model. It is a conceptual diagram when it changes. Where 230
1, 2303 are local minimum points 2302, 2
Reference numeral 304 represents a local minimum change point of the feature amount. Now the maximum point 2
Focusing on 302, the critical braking quadratic system model estimated from several tens of milliseconds ahead of 2302 is extrapolated up to the minimum point 2303 adjacent to the front of 2302, and is set as the changed feature amount. Further, the minimum point 2301 adjacent to the rear of 2302
Up to 2302, a critical braking secondary system model estimated from several tens of milliseconds behind is extrapolated to obtain the changed feature amount. As a result, the change over time in the feature quantity is changed using the critical braking quadratic system model. A method of estimating the parameters of the critical braking secondary system model from the feature time series of the short section is described in, for example, "Spectrum target prediction model and its appl.
ication to speech recognition "(Akagi, Tohkura,
Computer Speech and Language (1990) 4, 325-344).

FIG. 24 is a conceptual diagram showing a voice adding method adapted to a target portion of a moving image. When associating voices recorded separately by dubbing or ateleco, etc., with the target such as the performer of the video, usually the voice is blown while watching the video, or the utterance is made so that the target time is specified and the target time is reached. adjust. In the present embodiment, the voice conversion method according to the present invention is used to change the utterance speed of the voice recorded separately from the moving image and associate it with the appearance portion of the moving image. In the figure, for example, the duration of the appearance part 1 of a certain object in the moving image is T1,
Assuming that the duration of the voice 1 recorded in advance for the purpose of assigning it in association with the appearance portion 1 is T1 ′, the duration T1 ′ of the addition voice 1 is calculated using the voice conversion device according to the present invention. By converting the duration T1 into the duration T1, it is possible to attach the added voice 1 in association with the appearance portion 1 of the target.

[0074]

EFFECTS OF THE INVENTION A feature amount calculation unit for calculating an acoustic feature amount of a voice from a voice waveform, a feature amount change amount calculation unit for calculating a time change amount of the acoustic feature amount per unit time, and the time change. Amount change unit for changing the amount, an acoustic feature amount change unit for changing the acoustic feature amount using the changed time change amount, and a voice waveform reconstructed from the changed acoustic feature amount By providing the waveform reconstructing unit, it became possible to change the time change of the voice.

Further, among the acoustic feature quantities of the voice, the fundamental frequency, power, and frequency spectrum, which have a relatively large contribution to the sound quality due to the remarkable temporal characteristic change, are used and the temporal change thereof is used. By providing a means for changing the voices simultaneously, alone or in combination, it becomes possible to effectively change the time change of the voice.

The result of the acoustic feature quantity changing unit adding the time change amount of the acoustic feature quantity per unit time changed by the time change amount changing unit to the acoustic feature quantity unit time before the target time section. By providing a means for changing the acoustic feature amount of the target time section by setting the above as the acoustic feature amount of the target time section, it is possible to change the temporal change of the acoustic feature of the voice.

As the acoustic feature amount before the unit time, the acoustic feature amount changed by the acoustic feature amount changing unit before the unit time is used, and thus the change in the time change of the voice can be accumulated. It became possible.

By providing the speech speed conversion unit for changing the speech speed and the voice emphasizing unit for changing the time change, it becomes possible to change the speech speed and the time change at the same time.

By providing the frequency characteristic changing unit for changing the frequency characteristic and the voice emphasizing unit for emphasizing the time change, it becomes possible to change the frequency characteristic and the time change at the same time.

By changing the utterance speed conversion unit for changing the utterance speed, the frequency characteristic changing unit for changing the frequency characteristic, and the voice emphasizing unit for emphasizing the time change, it is possible to change the utterance speed, the frequency characteristic, and the time change at the same time. Became possible.

[Brief description of drawings]

FIG. 1 is a block diagram illustrating an embodiment of a voice emphasizing device according to the present invention.

FIG. 2 is a conceptual diagram showing a method of time-varying emphasis processing of a feature amount.

FIG. 3 is a conceptual diagram of time change emphasis.

FIG. 4 is a configuration example of a time change amount changing unit having a time change maximum value detecting unit.

FIG. 5 is a configuration example of a time change amount changing unit having a time change minimum value detecting unit.

FIG. 6 is a configuration example of a time change amount changing unit having a time change amount absolute value determination unit.

FIG. 7 is a conceptual diagram of voice emphasis when the emphasis is increased as the time change is large.

FIG. 8 is a conceptual diagram of voice enhancement when the enhancement is increased as the time change is smaller.

FIG. 9 is a configuration example of a voice enhancement device having a power determination unit.

FIG. 10 is an example of a voice emphasizing a temporal change using the present invention.

FIG. 11 is a configuration example of a voice enhancement device.

FIG. 12 shows an example in which a voice emphasizing device is directly used.

FIG. 13 is an example in which the voice emphasizing device is applied to a telephone for an analog line.

FIG. 14 is an example in which the voice emphasizing device is applied to a telephone for a digital line.

FIG. 15 shows an example in which the voice enhancement device is used as a pre-processing unit of a transmitter.

FIG. 16 shows an example in which the voice enhancement device is used as a post-processing unit of a receiver.

FIG. 17 is a configuration example of a voice enhancement device with variable enhancement amount.

FIG. 18 is a configuration example of a voice enhancement device having a variable amount of enhancement having a plurality of adjustment knobs.

FIG. 19 is a configuration example of a voice conversion device having a speech rate conversion unit and a time change emphasis unit.

FIG. 20 is a configuration example of a voice conversion device having a frequency characteristic changing unit and a time change emphasizing unit.

FIG. 21 is a configuration example of a voice conversion device having a frequency characteristic changing unit, a time change emphasizing unit, and a speech rate converting unit.

FIG. 22 is a time change amount changing unit using a critical braking secondary system model.

FIG. 23 is a conceptual diagram of temporal change emphasis using a critical braking secondary system model.

[Fig. 24] Fig. 24 is a conceptual diagram of a sound adding method adapted to a target portion of a moving image.

[Explanation of symbols]

101 ... Microphone, 102 ... Electric signal, 104 ...
Digital waveform signal, 106 ... Frame waveform signal, 10
8 ... Frame average power, 110 ... Change amount of frame average power per unit time, 112 ... Frame average power after enhancement, 114 ... Basic frequency, 116 ... Change amount of basic frequency per unit time, 118 ... Frequency spectrum,
120 ... Normalized frequency spectrum, 122 ... Spectrum envelope, 124 ... Change amount of spectrum envelope per unit time, 126 ... Normalized spectrum after emphasis of spectrum change,
128 ... Normalized spectrum after enhancement of fundamental frequency, 13
0 ... Frame waveform signal, 132 ... Frame waveform signal after power change enhancement, 134 ... Continuous waveform, 136 ... Analog electric signal, 137 ... Speaker, 401 ... Time change amount changing unit, 402 ... Time change amount of acoustic feature amount, 405 ... Time change amount of previous frame, 406 ... Difference between time change amount of current analysis frame and previous frame, 408 ... Maximum value determination result,
410 ... emphasis coefficient, 412 ... post-change time change amount, 501
... Time change amount changing unit, 502 ... Time change amount of acoustic feature amount, 505 ... Time change amount of previous frame, 506 ... Difference between time change amount of current analysis frame and previous frame, 508 ...
Minimum value determination result Determination result 510 ... enhancement coefficient 512 ...
Time change amount after change, 601 ... Time change amount change unit, 602
... time change amount of acoustic feature amount, 604 ... absolute value of time change amount of acoustic feature amount, 608 ... enhancement coefficient, 610 ... changed time change amount, 901 ... microphone, 902 ... electrical signal, 904 ... digital waveform Signal, 906 ... Frame waveform signal, 908 ... Frame average power, 910 ... Spectrum and fundamental frequency enhancement command, 912 ... Frame waveform signal after enhancement, 914 ... Continuous waveform, 916 ... Analog electric signal, 917 ... Speaker, 1101 ... Input voice Analog electrical signal 1102 ... Amplifier, 1110 ... Amplifier, 1111 ... Voice analog electrical signal after emphasis, 1201
… Microphone, 1202… Speaker, 1301…
Transmission line 1303 ... Handset, 1304 ... Speaker, 1305 ... Microphone, 1701 ... Adjustment knob, 1801 ... Adjustment knob, 2201 ... Time change amount change unit, 2202 Acoustic feature amount time change amount, 2204 ... Maximum point detection signal, 2206 ... Minimum point detection signal, 2208 ...
Amount of time change after change of feature amount 2210 ... Enhancement coefficient, 22
12 ... Amount of time change after change 2301, 2303 ... Minimum point of time change of feature amount 2302, 2304 ... Minimum point of time change of feature amount.

Claims (36)

[Claims] 1. An acoustic processing device having at least a means for inputting a voice, a means for analyzing and processing the voice, and a means for reproducing and outputting the voice, and calculating an acoustic feature amount of the voice from a voice waveform. A feature amount calculation unit, a feature amount change amount calculation unit that calculates the time change amount of the acoustic feature amount per unit time, a time change amount change unit that changes the time change amount, and the changed time change amount. And a waveform reconstructing unit for reconstructing a speech waveform from the post-modification acoustic feature amount. The equipment used. 2. A speech enhancement method according to claim 1, and a device using the same, wherein the acoustic feature quantity is a fundamental frequency and / or power and / or a short-term frequency spectrum, and the speech enhancement method. Equipment using. 3. The speech enhancement method according to claim 2 and the apparatus using the same, wherein the speech enhancement method and the apparatus using the same are characterized by normalizing a frequency spectrum with an average power. 4. A speech enhancement method according to claim 2 and a device using the same, wherein a speech spectrum enhancement method and a device using the same are characterized by normalizing a frequency spectrum with a frequency level maximum value. 5. The speech enhancement method according to claim 2 and the apparatus using the same, wherein the acoustic feature quantity changing unit changes the frequency spectrum based on a time change of the logarithmic power spectrum. Speech enhancement method and apparatus using the same. 6. The voice emphasizing method according to claim 2 and a device using the same, wherein the acoustic feature amount changing unit changes the frequency spectrum based on a time change of the cepstrum envelope. Emphasizing method and apparatus using the same. 7. The voice enhancement method according to claim 2 and a device using the same, wherein the acoustic feature quantity changing unit changes the frequency spectrum based on a temporal change of the Bark spectrum. Emphasizing method and apparatus using the same. 8. The voice enhancement method according to claim 2 and a device using the same, wherein the acoustic feature quantity changing unit changes the frequency spectrum based on a temporal change of the mel spectrum. Emphasizing method and apparatus using the same. 9. The speech enhancement method according to claim 2, and the apparatus using the same, wherein the acoustic feature quantity changing unit does not change the phase of the frequency spectrum and the apparatus using the speech enhancement method. . 10. The acoustic feature amount changing unit adds the time change amount of the acoustic feature amount per unit time changed in the time change amount changing unit to the acoustic feature amount unit time before the target time section. The audio enhancement method according to claim 1, wherein the acoustic feature quantity of the target time section is changed by setting the result as the acoustic feature quantity of the target time section. 11. The acoustic feature quantity changing unit according to claim 10, wherein the acoustic feature quantity before the unit time is changed by the acoustic feature quantity changing unit according to claim 8 before the unit time. The speech enhancement method according to claim 1, and a device using the speech enhancement method. 12. The acoustic feature amount changing unit according to claim 10 or 11, further comprising a maximum value calculating unit for calculating a maximum value of a time change amount of the acoustic feature amount, and when the time change amount is a maximum value. The voice enhancement device according to claim 1, wherein the acoustic feature amount is not changed. 13. The acoustic feature quantity changing unit according to claim 10 or 11, further comprising a local minimum value calculating unit for calculating a local minimum value of the temporal change quantity of the acoustic feature quantity, and when the temporal change quantity is a minimum value. The voice enhancement device according to claim 1, wherein the acoustic feature amount is not changed. 14. The acoustic feature amount changing unit according to claim 10 or 11, further comprising an absolute value calculating unit for calculating an absolute value of a temporal change amount of the acoustic feature amount, wherein the absolute value is greater than a specified value. The voice enhancement device according to claim 1, wherein the acoustic feature amount is not changed when the voice feature amount is small. 15. The voice according to claim 1, wherein the time change amount changing unit changes the time change amount by multiplying the time change amount of the acoustic feature amount per unit time by a coefficient larger than 1. Emphasizing method and apparatus using the same. 16. The time change amount changing unit changes the time change amount by multiplying the time change amount of the acoustic feature amount per unit time by a positive coefficient smaller than 1. Speech enhancement method and apparatus using the same. 17. The time change amount changing unit according to claim 15 or 16, further comprising an absolute value calculating unit for calculating an absolute value of the time change amount of the acoustic feature amount per unit time, and multiplying by the absolute value. The voice enhancement method according to claim 1, wherein the system numerical value is changed, and an apparatus using the same. 18. The time change amount changing unit according to claim 17, wherein the larger the absolute value of the time change amount of the acoustic feature value per unit time, the larger the system numerical value to be multiplied. Speech enhancement method and apparatus using the same. 19. The time change amount changing unit according to claim 17, wherein the larger the absolute value of the time change amount of the acoustic feature amount per unit time, the smaller the system numerical value to be multiplied. Speech enhancement method and apparatus using the same. 20. The voice enhancement method according to claim 1, and the apparatus using the same, wherein the time change amount changing unit according to any one of claims 15 and 16 uses different system numerical values depending on acoustic feature values. 21. The acoustic feature quantity is not changed when the power is smaller than a predetermined threshold value.
The voice enhancement device described. 22. A voice emphasizing device characterized by externally adjusting the change of the temporal change amount of the acoustic feature amount of the voice by providing an adjusting unit adjustable from the outside. 23. A speech conversion method and a device using the same, comprising a speech rate conversion section for changing a speech rate and a speech enhancement section for enhancing speech by using the speech enhancement method according to claim 1. 24. A voice conversion method and a device using the same, comprising a frequency characteristic changing unit for changing frequency characteristics and a voice emphasizing unit for emphasizing voice by using the voice emphasizing method according to claim 1. 25. A speech rate conversion section for changing a speech rate, a frequency characteristic changing section for changing frequency characteristics, and a voice emphasizing section for emphasizing a voice by using the voice emphasizing method according to claim 1. Speech conversion method and device using the same. 26. The speech conversion method according to claim 23 or 25, and the apparatus using the same, wherein the conversion rate of the speech rate in the speech rate conversion section is equal to the enhancement rate of the temporal change in the speech enhancement section. 27. A voice enhancement method according to claim 1 or a voice conversion method according to any one of claims 23 to 25 is used to enhance a voice uttered in association with an entire image or a specific part. Voice registration device. 28. A voice characterized by using the voice enhancement method according to claim 1 or the voice conversion method according to any one of claims 23 to 25 to enhance a voice uttered in advance in association with a motion of a moving image. Registration device. 29. The voice registration device according to claim 28, wherein the utterance speed conversion unit changes the utterance speed of the uttered voice in association with the designated time period of the moving image. 30. A speech disorder compensating apparatus having a voice emphasizing unit for emphasizing a voice using the voice emphasizing method according to claim 1 or the voice converting method according to any one of claims 23 to 25. 31. A deafness compensating apparatus having a voice emphasizing unit for emphasizing a voice using the voice emphasizing method according to claim 1 or the voice converting method according to any one of claims 23 to 25. 32. A communication device having a voice emphasizing unit for emphasizing a voice using the voice emphasizing method according to claim 1 or the voice converting method according to any one of claims 23 to 25. 33. A broadcasting apparatus having a voice emphasizing unit for emphasizing a voice by using the voice emphasizing method according to claim 1 or the voice converting method according to any one of claims 23 to 25. 34. An acoustic feature quantity change section having a maximum point calculation section for detecting a maximum change point of the acoustic feature quantity over time and a minimum value calculation section for detecting a minimum point of the acoustic feature quantity over time. Calculates the function approximating the temporal change of the acoustic feature quantity in the vicinity of the local maximum point and extrapolates the function to calculate the following from the minimum point of the temporal change of the acoustic feature quantity before the local maximum point. 2. The speech enhancement method according to claim 1, wherein the acoustic feature up to the minimum point of the temporal change of the acoustic feature is changed, and a device using the method. 35. An acoustic feature amount change section having a maximum point calculation unit for detecting a maximum change point of the acoustic feature amount over time and a minimum point calculation unit for detecting a minimum point of the acoustic feature amount over time. Part calculates a function approximating the temporal change of the acoustic feature quantity in the vicinity after the local maximum point, and extrapolates the function to obtain a local minimum point of the temporal change of the next acoustic feature quantity from the local maximum point. Up to the maximum point, and a function approximating the time change of the acoustic feature in the vicinity before the maximum point is calculated, and by extrapolating the function, the sound before the maximum point is calculated. The speech enhancement method according to claim 1, wherein the acoustic feature up to the minimum point of the temporal change of the dynamic feature is changed, and a device using the method. 36. A speech emphasizing method according to claim 34 or 35, wherein a function approximating a time change of an acoustic feature is calculated by using a critical damping quadratic differential equation. apparatus.