JP2759646B2 - Sound waveform processing - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION   The U.S. Government has established the Department of the Air Forc
e) In accordance with Contract F19-028-80-C-0002, this invention
You have rights in. Technical field   The field of the invention is generally speech technology, and in particular
Analysis of voice or other acoustic waveforms and digital coding and repair
Methods and apparatus for positive and synthetic. Background of the Invention   Typically, the problem of representing audio signals is the glottis
The sound excitation waveform is filtered through a time-varying linear filter.
The sound is considered as a result and the resonance characteristics of the sound path are modeled.
It is started by using the speech generation model.
In various speech applications, glottal excitation can be voiced or
Being in one of two possible states corresponding to unvoiced sounds
It is sufficient to assume that you can. Smells voiced
Excitation is performed at the analysis frame rate (typically 10-20 m
For s), it can change slowly with respect to time.
Periodic with the allowed period. About unvoiced sound
Glottal excitation is random with a flat spectrum
Modeled as noise. In both cases,
The power level in Ki is also considered to change slowly and over time.
available.   This binary model uses a narrowband vocoder and speech synthesis system.
It has been used effectively to design stems,
The world is well known. For example, excitation can be voiced and
Often mixed with both voice components simultaneously
Often only a portion of the spectrum is actually harmonic (harmonic
Nick). In addition, the binary model has
If the frame is classified as either voiced or unvoiced
But the decision is also made if
Is particularly difficult to perform if typical acoustic noise is imposed.
It is difficult.   Rate compatible with conventional transmission lines
(Ie 2.4 to 9.6 kbit / s)
Will meet considerable needs. At this rate,
Binary models are unsuitable for coding applications. In addition, use
Modify various parameters when reconstructing waveform
Audio processing devices and methods that allow
It is for. For example, time scale (without pitch change)
File modification can be used for various audio applications (ie for translation purposes)
To slow down the audio and also to speed up the audio for scanning purposes
For music synthesis or analysis)
It will always be a useful feature. Unfortunately, the time scale
Modifications (and other parameters) also use the binary model
Some devices do not provide high quality.   Thus, a better way to process audible waveforms
And the need for equipment. In particular, speech perception products
To maintain quality and at the same time change the rate of articulation.
, Like a synthesizer (speech synthesizer)
Voice coder that can operate
Satisfies the need for immediate concerns and makes a significant contribution to the technical field
Let's offer. Summary of the Invention   Speech analysis and synthesis and coding and time scale
Is the time of the audio waveform that is independent of the audio state-
Easy and effective by using frequency expression
It was found to be realized. To be clear, the audio waveform
Sine model for advancing new analysis / synthesis techniques
Used for   The basic method of the present invention is: (A) Sample frame (ie, about 20-40 ms window)
From the waveform, (B) each of the samples to extract a set of frequency components
Analyze the frame, (C)-Track components from one frame to the next
(Tracking) (D) To obtain a parametric representation of the waveform,
Steps to interpolate component values from one frame to the next
Prepare.   Next, the synthesized waveform corresponds to the parametric representation.
It is configured by generating a series of sine waves.   In one simple embodiment of the present invention,
Using only the amplitude and frequency of the component sine wave
Is disclosed. This so-called “magnitude”
Phase), the phase continuity is the phase
By defining it as an integral or complex of the instantaneous frequency
Is maintained. In a more applicable embodiment, the
Clear use of constant phase and component amplitude and frequency
It is.   The invention is particularly useful for speech coding and time scale correction.
And has been validated in both of these applications.
Was. To operate in environments with additional acoustic noise,
A permanent device is manufactured according to the invention. The present invention
Single or multiple speaker signals or music sound or live
Even physical sound can be used to analyze. The present invention
Also, for example, reading machines and broadcast jars for the blind
For editing and transmission to remote performance devices
Find applications.   In one exemplary embodiment of the present invention, the groups summarized above
The method uses a periodogram of the signal under test independent of the audio state.
Corresponds to the largest peak in the gram (Periodgram)
Used to select the amplitude, frequency, and phase. one
The amplitude, frequency and position of the multiple sine waves evaluated in the frame
The phases are matched to reconstruct the speech waveform.
To the corresponding parameters set in the next frame
Continuous deployment is allowed. Multiple rated
Since the peaks are not constant and change slowly,
The matching process is not easy. For example, unvoiced / voiced
The rapidly changing area of the sound, such as changes, is the peak position
And numbers can result in large changes. Spectrum
To account for this kind of rapid movement in energy,
The sinusoidal components "birth" and "death"
h) is based on the frequency evaluated in each frame.
Used in the closest neighbor matching method. if,
If a new peak appears, a "berth" will occur
A new track is started. If old
If the peaks are not matched, a "death"
Declared and corresponding tracks are likely to decline to zero
Is done. Parameters consistently in successive frames
The phase continuity of each sinusoidal component evolves the phase
(Unwrapping). One preferred implementation
In the example, the phase is the measured phase and the
Parameter values selected to satisfy frequency constraints
While using the third-order phase interpolation function
As smooth as possible over a period of time
Su). Finally, the corresponding sinusoidal amplitude is
It is simply interpolated in a linear manner across the frames.   In speech coding applications, the pitch evaluation is
A set of harmonic (harmonic) frequency
Used to set the option. (Pitch is here to talk
Means the base rate where the vocal cords of the person are vibrating
Is used as ) Component amplitude crosses frequency
Directly using adaptive pulse code modulation (ADPCM)
Coded or indirect using linear predictive coding
Is encoded. At each harmonic frequency bin,
Peak with the highest amplitude is selected and the frequency in the center of the bin
Assigned to. This corresponds to the pitch period to be encoded.
Results in a harmonic series based on Then the phase is the end of the frame.
Instead, it is encoded using frequency to predict the phase.
And expand the measured phase with respect to this prediction and then
Encode the phase residual using the bit / phase peak. Also
Enough bits to encode all of the phase peaks
If there are no numbers (for example, a low pitch speaker)
Phase tracks for high frequency peaks are artificially generated.
It is. In a preferred embodiment, this is the baseband
The peak frequency of the uncoded phase peak
This is done by converting to frequency. This new code
The optimization plan is to assign bits to each speaker adaptively.
Important properties, hence low pitch and high
It is self-tuning to both pitchers. Pitch is a rhino
Used to provide code information to the encoding algorithm
However, standard voice excitation models for speech are not used.
Not used. This means that recourse is voiced / unvoiced
Means never to be done. As a result
Thus, the present invention is robust in noise and simply bit allocation.
Different data transmission levels by changing the rules for
It can be applied at the site.   The invention is also well adapted for time scale correction.
This is done so that the amplitude variation and
This is achieved by time-scaling the phases. voice
The time scale at which music is played or returned is
Is easily controlled by changing the rate at which the
It is. This is due to the fact that the time scale is
Means you can do it later or later. This rate is
Allows the author full flexibility to change the time scale
Is controlled by a panel knob. Time scale processing
There is no perceptual delay in performing.   The present invention is disclosed below in connection with certain exemplary embodiments.
Is done. However, do not depart from the technical idea of the present invention.
Various changes and modifications may occur to those skilled in the art.
It is clear that this is done by For example, another
Sampling technology uses variable frame length and hamming window
Replaced by the use of Furthermore, this kind of frame
And the length of the window can vary depending on the particular application. As well
In addition, frequency matching can be achieved by various means.
A variety of commercially available instruments are available to perform Fourier analysis
Noh. This type of analysis is also custom hardware or
Or by a specially designed program.   Various techniques are employed to extract pitch information.
For example, the pitch period is derived from a Fourier transform.
For example, Gold-Malpass technology
Also available. Generally, professionals by M.L.Malpass
Seedings of Airscon 1975 (Proc.of EASCON 1
975, September, 1975) “The Gold Pitch Detector in
a Real Time Environmet "and B. Gold
International Congress on Acoustic Tex
(Fourth International Congress on Acousticks,
Penhagen, August 21-28, 1962)
ion of a Computer Program for Pitch Detection "and
And B.Gold, Journal of the Acoustic
Cal Society of America (J.Acoust.Soc.A
mer.365, 1659-1661 (1964))
uzz-Hiss Detection ".
And is hereby incorporated by reference.   Various coding techniques interchanged with those disclosed below
It can also be used. Channel encoding technology
I.E.R.P.R.O.C. by N.HolmesIE
E PROC, Vol. 27, pp. 53-60, 1980).
el Vocoder ". Adaptive pulse code modulation.
By L.R.Rabiner and R.W.Shafer, "Digital Pro
Processing of Signal (Digital Processing of Si)
gnal, Prentice Hall, 1978). Linear
Measurement coding is described by J.D.Markel, "Linear Prediction
Linear Prediction of Speec
h), Springer-Verlog, 1967).
These techniques are also incorporated for reference.   The term "interpolation" is used in this application.
Complements data values between data values measured at frame boundaries.
Widely used to include various techniques for charging
You. In a magnitude-only system, linear interpolation is
And used to supplement frequency values. This easy
In such a system, the phase value is calculated from one frame to the next frame.
Series of instantaneous frequencies by interpolation of frequency components matched to
To define a number and then obtain a series of interpolated phase values
Obtained by integrating or combining a series of instantaneous frequency values.
Can be For more advanced systems, the phase value of each frame
Is directly derived and a third-order polynomial is
To get as large and smooth phase interpolation as possible into the frame
Used for   Other techniques to accomplish the same purpose are also described in this application as interpolation techniques.
Referred to as art. For example, if you want to replenish data values
"Overlap and add" me
thod) is also used. In this method, the weighted decimation function
Is added to the result of the sine wave generated during each frame.
Then the repetition values are between those measured at frame boundaries
Added to supplement the value. BRIEF DESCRIPTION OF THE FIGURES   Fig. 1 shows only the magnitude and frequency of the components
Is the book used to reconstruct the sampled waveform
FIG. 1 is a schematic block diagram of one embodiment of the present invention.   FIG. 2 shows the extracted waveforms sampled according to the present invention.
5 is an example of amplitude and frequency components.   FIG. 3 shows a general example of the frequency matching method of the present invention.
It is.   FIG. 4 shows details of the frequency matching method according to the present invention.
It is a typical example.   FIG. 5 shows a tracked track of an exemplary voice pattern.
It is an illustration of a wave number component.   Figure 6 shows the phase and magnitude of the frequency component.
Is used to reconstruct the sampled waveform
FIG. 4 is a schematic block diagram of another embodiment of the present invention.   FIG. 7 shows that the phase function “as large and smooth as possible”
A useful phase function in connection with the embodiment of FIG.
5 is an example of a set of third-order phase interpolation functions for smoothing.   FIG. 8 illustrates another embodiment of the invention particularly useful for time scale correction.
FIG. 6 is a schematic block diagram of the embodiment of FIG.   FIG. 9 shows an embodiment of the system evaluation function of FIG.
It is a schematic block diagram.   FIG. 10 is a block diagram of one real-time execution of the present invention.
You. Detailed description   In the present invention, the speech waveform is modeled as a sum of sine waves.
Be transformed into If s (n) represents the sampled speech waveform
If I do, s (n) = Σa_i(N) sin [φ_i(N)] (1) It is. Where a_i(N) and φ_i(N) is the i-th
Is the time-varying amplitude and phase of the tone.   In a simple embodiment, the phase is the instantaneous frequency f_i(N)
Is defined to be the integral or compound of φ_i(N) = φ_i(N-1) + 2πf_i(N) / f_s  (2) To be satisfied.   Where f_sIs the sampling frequency. If the tone
Are harmonically related, f_i(N) = i^*f₀(N) (3) It is. Where f₀(N) represents the fundamental frequency at time n
You. One particularly interesting property of the above model is the phase sequence.
Continuity and therefore continuity of the waveform is the phase from the instantaneous frequency point
Is the fact that it is guaranteed as a result of the definition.   This is because high-resolution spectral analysis is
The amplitude and frequency are revealed so that the waveform reconstruction
It is possible from only the magnitude (magnitude) spectrum.
It means there is.   The block diagram of the analysis / synthesis system according to the present invention is first.
It is shown in the figure. Discrete Fourier transform of windowed waveform
(DFT, discrete Fourier transform) size peak
By determining the location of the change in slope (concave down)
It is easily found. Furthermore, the total number of peaks is limited
This limit is consistent with the average pitch of the expected speaker.
It is.   In a simple embodiment, the audio waveform is a 10 kHz sampler.
Digitalization at low rate and low frequency filtering at 5kHz
20ms frame with 20ms Hamming window
Analyzed at the same time interval. The speech expression according to the invention is variable
It can also be obtained by employing a time analysis window. is there
For certain applications, the width of the analysis window may be
The pitch is set to 2.5 times the average pitch period along with the width.
It is preferable to have tsuchi adaptability.   The one plotted in Fig. 2 uses the previous procedure
For voice frames along the rated amplitude and frequency
Is a standard periodogram (periodic diagram). Discrete form
The Fourier transform DFT is a 512-point fast Fourier transform (FF
T, fast Fourier transform). This
Different sets of these parameters are obtained for each analysis frame.
Can be   FIG. 3 shows an example of basic processing of frequency component matching.
You. If the number of peaks is constant and the frame
Assuming a slow change to
The parameters to be evaluated are
The problem of matching is simply the frequency order of the peaks
Would require an assignment. However, in practice,
Spurious spin that fluctuates due to the influence of idrobe interaction
And the location of the peaks will change as the pitch changes.
And change the sound, such as in a voiced / unvoiced transition.
The location and number of peaks, corresponding to areas of rapid voice change
There are both rapid changes. This kind of spectral peak
In order to take into account the sudden movement in
As part of the binning process, the sine component "birt (birt
h) ”and“ death ”concepts.   The matching process is further explained by the idea of FIG.
You. The peak up to frame k is matched and
New parameter set for arm k + 1 occurs
Assume that Selected at frames k and k + 1
▲ ω₀ ^k, ▼ ▲ ω₁ ^k, ▼… ▲ ω^k _N-1▼ and
And ▲ ω₀ ^{k + 1}▼, ▲ ω₁ ^{k + 1}▼,… ▲ ω^{k + 1} _M-1▼ [N here
And M represent the total number of peaks selected in each frame.
(In general, N ≠ M)]. Frame k
Frequency (▲ ω^k _n▼) is a certain frequency at frame k + 1
Number (▲ ω^{k + 1} _mOne process to match on ▼) is as follows
Is given in three steps. Step 1   Frequency ▲ ω₀ ^k▼, ▲ ω₁ ^k▼,… ▲ ω^k _n-1About ▼
Suppose two matches are found. Right now
If the frequency ω_n ^kIs contemplated. FIG. 4 (a)
All frequencies ▲ ω at frame k + 1^{k + 1} _m▼, ▲ ω
_n ^kThe case where the position is outside the “matching interval” Δ of ▼ is shown.
That is, for all m It is. In this case, ▲ ω^k _n▼ frequency frequency related to
When entering frame k + 1, the "dead is declared.
Said ▲ ω^k _n▼ makes itself at frame k + 1
But matched with zero amplitude. Then, the frequency ▲ ω
^k _n▼ Step 1 without any further consideration
Is the next frequency in the list ▲ ω^k _{n + 1}Repeated for ▼
You.   On the other hand, if at frame k + 1, within the matching interval
▲ ω_n ^k▼ nearby, this kind of frequency-i.e.
For all unequal i, Frequency ω closest to^{k + 1} _mIf ▼ exists
If ▲ ω^{k + 1} _m▼ is ▲ ω^k _nCandidate matc
h) is declared. Definitive matching (definitive matc
h) has not been done yet. Because frequency ▲ ω^{k + 1} _m▼
Better match at frame k, in step 2
Because there may be uncertainties considered.
You. Step 2   In this step, the candidate match from step 1 is
It is confirmed. Frequency of frame k ▲ ω_n ^k▼ is frame k
+1 frequency ▲ ω^{k + 1} _mAssume experimentally matched to ▼
You. If ▲ ω_m ^{k + 1}▼ is the remaining mismatch frequency of frame k
If we do not have a better match for
Declared to be deterministic. This is illustrated in FIG. 4 (c).
The condition isGiven by When this occurs, the frequency ▲ ω
^k _n▼ and ▲ ω^{k + 1} _m▼ has been removed from further consideration
Step 1 is the next frequency in the list ▲ ω^k _{n + 1}About ▼
Repeated.   If the condition (6) is not satisfied, the frame k +
Frequency at 1 ▲ ω^{k + 1} _m▼ is the test frequency ▲ ω^k _nFor ▼
Than the frequency of frame k ▲ ω^k _{n + 1}▼ against
Good alignment. Two additional cases are considered next
You. In the first case illustrated in FIG. 4 (d),
Neighboring remaining lower frequency ▲ ω^{k + 1} _{m + 1}▼ (if it exists
) Is below the matching interval, so what
No matching is performed. As a result, ▲ ω^k _nRelated to ▼
The frequency track is "dead" when entering frame k + 1.
It is declared ▲ ω^k _n▼ on itself with zero amplitude
Be aligned. In the second case illustrated in FIG. 4 (c)
And the frequency ▲ ω^k _m-1▼ within the matching interval ▲
ω^k _n▼ It is near, and definite matching is performed. One of
After that, step 1 is the next frequency ω in frame k.
_n-1Is repeated using. In this step, many other
Many situations are possible, but there are alternatives to Tratka
For simplicity, only two cases are discussed.
You. Step 3   All frequencies in frame k are tested and continue
Assigned to trucks or dying tracks
No match is made to frame k + 1
Frequency may remain. ▲ ω^{k + 1} _m▼ is this kind of frequency
Assuming it is a number, ▲ ω^{k + 1} _m▼ is frame k
(Born) "
Number) ▲ ω^{k + 1} _m▼ starts at frame k with zero size
Be born. This is true for all such mismatch frequencies.
Done. This last step is illustrated in FIG. 4 (f).
Is done.   To apply Trakka to actual speech segments
The result is shown in FIG. 5, which is for example voiced / unvoiced
Temporary audio shaking such as transitions and mixed voiced / unvoiced areas
Demonstrate Tratka's ability to quickly adapt through dance
You.   For simple magnitude-only systems
And the composition is realized in a non-destructive way.
Each pair of matched frequencies (and their corresponding magnitudes
Ude) but linear interpolation across successive frame boundaries
Is done. As mentioned above, the magnitude-only system
In systems, the continuity of the phase is the phase from the instantaneous frequency point.
Is guaranteed by the definition of The interpolated values are then shown in FIG.
To drive a sine wave generator that produces the composite waveform shown
used. Performance is correlated at higher frequencies
Note that it can be improved by reducing the window size Δ
I want to be.   Illustrated in FIG. 1 (and discussed in detail below)
Yet another feature is that the present invention provides
To be adapted. From Fig. 3, the time scale is simply
Location and magnificence by expanding or compressing to
Fixed their time change rates while retaining their Eud
It will be understood that. Affects change rate b
For this purpose, the speech synthesizer interpolation rate R '(see FIG. 1)
) Is given by R '= bR. In addition,
Together with the stem, can introduce a time-varying rate of change.
Is easy. Because the frequency is interpolated
Stretched or compressed by changing the
Because it is.   FIG. 6 shows that the phase capacity is directly measured
FIG. 2 shows a block diagram of a system with the above. With this system
The frequency components and their amplitudes are described earlier
Same as the magnitude-only system shown in FIG.
Is determined in the following manner. However, the phase measurements are evaluated
By calculating the arc tangent at the frequency peak
Derived directly from the Fourier transform.   In the tolerable system of FIG. 6, a set of amplitudes
And frequency and phase are evaluated for each frame
Therefore, for kN <n (k + 1) N, the expression To generate the synthesized speech using
It may be reasonable to evaluate the original speech waveform at the frame
I think. However, the time variation of the parameters
This roundabout method severely degrades the quality of synthesized speech
Leads to discontinuities at frame boundaries. Therefore, one
Parameters measured from the frame
Meters need to find a way to interpolate smoothly
No.   Frequency matching algorithm described in the previous paragraph
As a result of
All parameters are the parameters for frame k + 1.
Associated with the corresponding set of data. [▲ A^k _l▼, ▲ ω^k _l▼, ▲ θ^k
_l▼) and [▲ A^{k + 1} _l▼, ▲ ω^{k + 1} _l▼, ▲ θ^{k + 1} _l▼)
Is the sequence of parameters for the l-th frequency track.
The solution to the amplitude interpolation problem is
Is (Where n = 1, 2,..., N is the k-th frame
Is a time sample). (Truck's
The subscript "l" has been omitted for convenience).   Unfortunately, this kind of simple method requires that the phase
θ^kIs obtained modulo 2π modulo 2π.
Cannot be used to interpolate wavenumber and phase. Soy sauce
For example, phase unwrapping (expansion)
"Maximum smoothness possible" across frame boundaries
Must be carried out to ensure that This question
The first step in solving the problem is a third order porous system, θ (t) = ξ + γt + αt^Two+ Βt^Three              (9) Is to assume a phase interpolation function Phase interpolation function
If the number is a frame like a function of a continuous time variable t
t = 0 for k and t for frame k + 1
It is convenient to handle the phase relationship with = T.
The porous parameter is the frequency obtained at the frame boundary
And must be selected to satisfy the phase measurements
No. Since the instantaneous frequency is a derivative of the phase, (T) = γ + 2αt + 3βt^Two                (Ten) And the starting point t = 0, θ (0) = ξ = θ^k (0) = γ = ω^k                           (11) And at the end point t = T θ (T) = θ^k+ Ω^kT+ ΑT^Two+ ΒT^Three= Θ^{k + 1}+ 2πM (T) = θ^k+ 2αT + 3βT^Two+ Ω^{k + 1}        (12) Becomes Here again, the track subscript "l" is
It is omitted for convenience.   End point phase θ^{k + 1}Is measured modulo 2π, so
Let the resulting frequency function be
To make it “clear,” we add it by the term 2πM (M is an integer)
It is necessary to increase. At this point, M is unknown
However, for each value of M, whatever the value,
(12) can be solved for α (M) and β (M).
(Dependence on M is clearly shown here). The solution is
Matrix type Is easily shown to satisfy.   Find solutions to M and unwrapping problems
"As big and smooth as possible" to make the ultimate decision
Additional constraints are imposed to quantify criteria
Need that. FIG. 7 shows the third order for each of a plurality of M values.
Exemplifies a standard set of phase interpolation functions. On an intuitive basis
The best phase function to choose based on
With no fluctuations. This should be as large as possible
What is meant by a smooth frequency track.
In fact, if the frequency is constant and the vocal tract (vocal
tract) is assumed to be immobile, the true phase is a line
It will be linear. Therefore, for "smoothness"
A reasonable standard of (Where θ (t; M) is 2 of θ (t; M) related to time variable t.
(Denoting the first derivative) is minimized by choosing M
is there.   M evaluates to an integer, but f (M)
Is quadratic, so the problem is that f
By minimizing (x) and also having next to x
Easy to choose by choosing M to be a large integer
Is solved. The number of awkward but boring algebras, the maximum of x
The small value is , And M^*Is also determined by the equation (1
3), use α (M^*) And β (M^*) And then calculate
The unwrapped phase interpolation function is θ (t) = θ^k+ Ω^k _t+ Α (M^*) T^Two+ Β (M^*) T^Three (16) Becomes This phase function is the measured phase and frequency
Not only satisfy all of the endpoint constraints of
Unwrap the phase in a way that is as smooth as possible
I do.   The above analysis shows that the frequency ω at the start of frame k^kCompatible with
Initial development phase θ^kStarted with the assumption of
It is necessary to specify the initialization of the frame interpolation procedure.
This resulted in the track under consideration at any point (bor
n) It is done by paying attention. This event occurs
The amplitude, frequency and phase are measured at frame k + 1
And these measured values are the parameters at the corresponding frame k.
The meter sets the amplitude to zero (ie, A^k= 0) and at the same time
Maintain a similar frequency (ie, ω^k= Ω^{k + 1})
More defined. The phase interpolation constraint is initially satisfied.
To ensure that the unfolded phase is the measured phase θ^{k + 1}
And the starting phase is θ^k= Θ^{k + 1}−ω^{k + 1}N (17) (Where N returns from frame k + 1 to frame k
Is the number of samples traversed in the case)
Is done.   As a result of the phase expansion procedure described above,
Tsuku shows the rapid phase change and voice due to the frequency of each sine component.
Slowness due to gated pulse and vocal track transfer function
Of the moment that takes into account both phase changes
It will have a phase associated with it. θ_l(T) is l
Denote the expanded phase function for the th track,
The final composite waveform is (Where kN <n (k + 1) N, A_l(N) becomes (8)
Given by θ_l(N) is the sample data of equation (16)
Deformation and L^(k)Is evaluated for the kth frame
Number of sine waves.   The invention described in conjunction with FIG.
/ Speech encoding
Used to develop the stem. At this rate,
High quality speech depends heavily on phase measurements, so phase
Encoding is a high priority. The sinusoidal representation also
And frequency identification, so multiple available bits
Considerably less peaks before all of the
It is clear that the loop is encoded. Therefore, the first
The steps in this section have several parameters that must be signed.
Data should be significantly reduced. One to do this
The method is to make all frequencies harmonic.   During voiced sounds, make sure that all peaks are harmonically related.
And by encoding the basics,
All frequency locations are available at the receiver. Nothing
During voice speech, the peak frequency position is not harmonic in this case.
No. However, from random process theory, noise-like waves
The shape is determined by the spacing between adjacent harmonics,
There is only a slight change in the rope (i.e.
If the interval is small enough, the harmonic extension of the sine wave
From a point (in the sense of overall mean squared error)
You. This representation implies that the amplitude and phase are
If it changes randomly to the
Retains metrological properties. Amplitude and phase are encoded
This random change inherent to the measured variable should be
Are held in a composite waveform.   In practice, the basics that characterize the set of frequencies in each frame
It is preferable to evaluate the frequency, which in turn
Related to out. For example, pitch extraction is a standard for perception
To produce the best fit to the input speech accordingly
By selecting the fundamental frequency of a set of harmonic sine waves,
Will be revealed. Other pitch extraction techniques can also be employed.   As an immediate consequence of using the harmonic frequency model
The number of sinusoidal components to be encoded is divided by the basic
To be encoded. Of the measured peak
There is no guarantee that the number will be equal to this harmonic number,
Measures are taken to adjust the number of peaks to be encoded.
Should be done. Based on the basics, a set of harmonic frequencies
A number of bins are set and the peaks in each bin
The numbers are checked. If one or more peaks are found
Only the amplitude and phase corresponding to the largest peak
Retained for encoding. If the peak in a given bin
If there is no short time at the frequency corresponding to the center of the bin
The amplitude and the amplitude obtained by sampling the Fourier transform
A virtual peak having a phase and a phase is generated.   The amplitude is then similar to that used in the channel vocoder
It is encoded by applying the technique of. That is,
For example, the first peak (ie, the first peak above 300 Hz)
2dB / level to encode the amplitude of the peak)
Gain level is set by using 5 bits for both
Is done. The next peak crosses the frequency and the delta modulation technique
It is encoded logarithmically by using the technique. One
3.6 kbps at 50 Hz frame rate
Assigned to encode amplitude at rate. Additional
The bit allocation rule is used to assign multiple bits to peaks.
Can be used for For example, if the pitch is high,
There are considerably fewer peaks to encode and
There are many bits. Conversely, if the pitch is low
There are considerably less bits per peak, but
As the peaks come closer together, their values are more correlated.
ADPCM coder therefore keeps them in good track
Should be able to.   To encode the phase, a fixed number of bits per peak
(Typically 4 or 5) is used. Sign phase
One way to achieve this is to change the measured phase by
2 ”(seconds) in the range of π to π
Where n = 4 or 5. Another method is
Predict the phase at the end of the current frame, expand the value, then
ADPCM technology with 4 or 5 bits per phase peak
To encode the rest of the phase using
Use the frequency track corresponding to the phase). phase
And encoding the base (7 bits are used)
50Hz frame rate since only 4.4kbps remains
Thus, at most 16 peaks can be encoded. 4kHz
Voice bandwidth and 4 bits per phase, pitch is
If greater than 250Hz, all phases are encoded
You. If the pitch is below 250Hz, uncoded high
To regenerate phase tracks due to frequency peaks
Measures must be taken. This is the instantaneous third phase
And the linear complement of the endpoint frequency for this track.
By calculating the difference frequency, which is the difference between
Will be The difference frequency is the phase to which it is encoded.
Add to linear interpolation of end frequency corresponding to rack
Is converted to a high frequency region. Instantaneous frequency obtained
The number function is then the instantaneous phase function added to the sine wave generator
Are integrated or combined to give Like this
The phase coherence and the unvoiced
Phase non-coherence properties are effective in uncoded frequency domain
Is converted to   FIG. 8 shows that it is particularly suitable for time scale correction.
Another embodiment of the present invention is illustrated. In this example
The sine wave on the display is the system contribution (ie
And excitation contribution (ie from vocal cords)
It is further defined as follows. Excitation phase contribution is based on cubic interpolation
Selected for The procedure is described above in relation to the other embodiments.
Almost follow what is said, but in another step
And the measured amplitude ▲ A^k _l▼ and phase ▲ θ^k _l▼ is the vocal tract
Decomposed into components and excited components. First, the method
In the analysis frame as a function of frequency (ie, M (ω, k
R) and φ (ω, kR)) vocal tract amplitude and phase evaluation
Is to form Selected frequency ▲ ω^k _lIn ▼
The system amplitude and phase evaluation is then and Given by   Finally, evaluation of excitation parameters at each analysis frame boundary
Is and Is obtained as   The problem of decomposition is then M (ω, kR) and φ (ω, k
R) from the high-resolution spectrum X (ω, kR)
It becomes a problem to evaluate as a number. (Of course in practice
Means that equally spaced frequency samples are available from the DFT
Noh. ) For example, all-pole modeling and homomorphic deco
System from high resolution spectrum such as evolution
Multiple established methods for identifying magnitude
There is. If the vocal tract transfer function is assumed to be minimum phase
System phase and system magnitude
Form the Hilbert transform pair. This
Under the condition, the phase estimation φ (ω, kR) is
Through the commutation, the magnitude evaluation of the system function M (ω, kR)
Derived from the logarithm. In addition, the resulting phase estimate is
Smooth and expanded as a function of number.   A method for evaluating system magnitude and Hilbe
The corresponding evaluation of the system phase through the use of the default transform is
It is illustrated in FIG. 9 and is based on a homomorphic transformation. With this technology
Identifies and identifies system amplitudes from high-resolution spectra.
The calculation of the Hilbert transform for evaluating the amplitude of
Will be performed. Logarithmic Fourier of high-resolution magnitude
Transformation is performed to obtain a "cepstrum"
Calculated to 1. Right window is proportional to average bite period
Along with time, it is added next. Inverse Fourier transformation obtained
The imaginary component of the permutation is the desired phase and the real part is
It is log magnitude. In fact, the Fourier transform
Evenly spaced samples are calculated together with the FFT. F
The length of the FT is sufficient to avoid aliasing in the cepstrum.
The size was selected at 512. Thus, the sine wave frequency
The high-resolution spectrum used to evaluate the number
Also used to evaluate road system functions.   Analysis of the residuals in the time scale correction system of FIG.
The steps correspond to those disclosed above in connection with other embodiments.
Is similar to As a result of the matching algorithm
The excitation component measured for any frame k and
And the magnitude and phase of all system components
Associated with a corresponding set of parameters for +1.
The next step in compositing is to align across frame boundaries
Interpolated excitation and system parameters
It is to be. The interpolation procedure depends on the excitation function and the system.
Function slowly changes across frame boundaries
Based on this assumption. This is because the model parameters are
It is said that it changes slowly compared to the time of the impulse response
Be consistent with assumptions. This slow change constraint is slow change
Map to the excitation and system amplitude of the
It satisfies to interpolate these functions linearly
You.   The vocal tract system is slow over successive frames
It is assumed that it is changing, so its phase also changes slowly
It is reasonable to assume that
Linear interpolation of the phase samples is also satisfactory. However, "slow
The characteristic of “change” is about system magnitude.
Is more difficult to achieve for system phases.
You. This does not impose any additional constraints on the phase being measured.
Must be-the phase is the frequency at each frame boundary
Is smooth and expanded as a function of
You. Here, if the system phase is obtained modulo 2π,
Linear interpolation between frame boundaries (unreasonably)
Be able to result in rapidly changing system phases
Is shown. Fig. 9 Homomorphic analyzer (analyzer)
The importance of the use is obvious here. From homomorphic analysis
Derived system phase estimates are unfolded or expanded in frequency
The system amplitude (which it is derived from)
It changes slowly when it changes slowly. Of this function
Linear interpolation of the samples then mirrors the underlying vocal tract movement
Result in phase trajectory. This phase function is φ_l(0) is the formula
▲ φ of (22)^k _lΦ corresponding to ▼_l(T)
Mentioned. Finally, as before, the third order polynomial is
Used to interpolate phase and frequency. This is Ω
_l(0) is ▲ Ω in equation (22)^k _lΩ corresponding to ▼
_l(T).   The purpose of the time scale correction is to adjust the apparent
Change) while maintaining the perceived quality of the original speech
It is to be. This is the frequency trajectory of the excitation (according to
The pitch contour (contour) is in time
Stretched or compressed and the vocal tract is loose or fast
It means changing at a rate. The previous synthesis method is clear
Vocal fold excitation for which a precise functional representation has been derived
And the sum of sinusoids composed of vocal tract system contributions
As it includes, it fits perfectly with this transformation.   Time t with new time scale₀Audio events that occur in
Is ρ on the original time scale^-1t₀Happens in. Upper sine
To apply the wave model to the time scale correction,
The scaled "event" is a system along each frequency track.
System amplitude and phase and excitation amplitude and phase
You. Parameter estimation for uncorrected synthesis is a continuous function of time.
Available, so in theory, any rate change
Is also possible. Together with Equations (19) and (22),
The synthesized waveform is (Where L (n) is the number of sine waves evaluated at time n
Can be expressed as: Value required by equation (23)
Is the time ρ^-1A at n_l(T) and Ω_l(T) and φ_l(T)
Scaling and the resulting excitation phase is ρ^-1By
It can be obtained only by sealing.   Time-varying with the proposed time correction system
It is also easy to add rate fluctuations. Where time
The distortion transformation is (Where ρ (T) is the desired time rate variation
). In this generalization, each time derivative dT
Is scaled by different factors ρ (T). new
Time scale time t₀The audio event that occurs in
Time t in kale₀= W^-1(T₀). If t₀Is t₀′
If we map back to t₁′ T₀'+ Ρ^-1(T₀') (twenty five) Given by   The parameters of the sine component can be used as a continuous function of time.
So they are always required t₁′
You.   t_nIs time t_n= N, the composite waveform is Given by here, Ω_l'(N) = Ω_l'(N-1) + ω_l(T_n') (27) and t_n'= T_n′_-1'+ Ρ^-1(T_n′_-1') (28) And ω_l(T) is a third-order phase function Ω_lFirst derivative of (t)
A quadratic function given by the function t₀= 0 (29) It is.   At the time when a particular track is born, the third phase
Function Ω_l'(N) is ρ (t_n') Ω_l(T_n') Initial by value
Be transformed into Where Ω_l(T_n') Is obtained using equation (17).
Initial excitation phase.   The present invention performs frequency and pitch scaling.
It should also be appreciated that it can be used to Synthetic wave
The short-term spectral envelope of the shape is at each frequency
Is changed by scaling the components and
Pitch scales frequency components contributed by excitation
Can be changed by   FIG. 10 shows the present invention operated and executed in real time.
A final example is illustrated. The illustrated embodiment has four resources.
Uses the Khan Digital Signal Processor (LDSP)
Then, it is executed by 16-bit fixed-point arithmetic. Huoag
The round program uses 100 input audio samples for 10 ms
It works on each input A / D sample. same
At a similar time, the 10ms buffer of the synthesized voice
Output through the barter. At the end of each frame,
Most of the voice is pushed down to 600ms buffer
(Last in, first out). Pitch adaptive humming (Hamm
ing) Data for windows is pulled out and also 512 points higher
The fast Courier transform (FFT) is applied to this buffer.
Because. Next, a set of magnitudes and phases are
Obtained by finding the peak of the magnitude.
Data generates pitch evaluation values that control the pitch adaptive window
Is supplied to the extracted pitch extraction module. This parame
Data also supplied to the encoding module for data compression applications
Is done. Once the pitch is evaluated, another pitch
Responsive Humming window is buffered for parallel computation
And transferred to another LDSP. Coding and speech correction
Another 512 point FFT to which the method is applied is amplitude and frequency
And used for the purpose of evaluating the phase. When
Once these peaks have been determined, frequency tracking and
And a phase interpolation method is performed. Depending on the application, these
The parameters are coded or encoded so that the audio conversion can be performed.
Or another pair of modified or sinusoidal summations are performed.
Will be forwarded to LDSP. The resulting composite waveform is then
Foreground program for D / A output
To be entered in the appropriate buffer to be accessed
It is transferred back to the master LDSP.

   ────────────────────────────────────────────────── ─── Continuation of front page    (72) Inventors Kuoteieri, Thomas F. , Jiyu               near               United States 02165 Masachiyu Se               Tutu, West Newton, Mossman                 Street 74                (56) References JP-A-57-197600 (JP, A)                 JP-A-60-88326 (JP, A)

Claims (1)

(57) [Claims] To sample the waveform to obtain a series of discrete samples, and then construct each series of frames spanning multiple samples to extract a set of variable frequency components corresponding to glottal excitation and having individual amplitudes An acoustic waveform processing method comprising the steps of analyzing each frame of a sample, wherein the variable frequency components are matched from one frame to the next frame, and components in one frame are matched with components in a subsequent frame Using a smooth phase interpolation function in phase expansion to match the phase and frequency of the variable frequency component extracted at the frame boundary and to realize the extracted variable frequency component phase at the frame boundary modulo 2π, A sound characterized by interpolating amplitude, frequency and phase match values from one frame to the next Waveform processing method. 2. The method of claim 1, wherein the step of sampling comprises constructing a frame having a variable length that varies with the pitch period and is at least twice the pitch period of the waveform. 3. The method of claim 1, wherein the step of sampling comprises sampling the waveform according to a Hamming window. 4. The method of claim 1, wherein the step of analyzing comprises analyzing each frame by Fourier analysis. 5. The method of claim 1, wherein the step of analyzing comprises selecting a harmonic series to approach the frequency component. 6. 6. The method according to claim 5, wherein the number of frequency components in the harmonic series changes according to the pitch period of the waveform. 7. The method of claim 1, wherein the step of tracking comprises matching frequency components from one frame with components in a next frame having similar values. 8. The method of claim 7, wherein the matching provides a berth of a new frequency component and a death of an old frequency component. 9. The step of interpolating the values comprises defining a series of instantaneous frequency values by interpolating frequency components that are matched from one frame to the next, and then a series of instantaneous frequency values to obtain a series of interpolated phase values. The method of claim 1 including combining values. 10. The method of claim 1, wherein the step of interpolating comprises deriving a phase value from the frequency and phase measurements obtained at each frame and then interpolating the phase measurement. 11. The method of claim 1 wherein the step of interpolating is accomplished by an overlap and add function. 12. The method of claim 1, comprising encoding frequency components for digital transmission. 13. 13. The method of claim 12, wherein the frequency components are limited to a predetermined number defined by a plurality of harmonic frequency bins. 14． 14. The method of claim 13, wherein the amplitude of only one of the components is encoded for gain and the amplitude of the other is encoded with respect to the next lowest frequency adjacent component. 15. 13. The method of claim 12, wherein the phase is encoded by applying a pulse code modulation technique to the predicted phase residual. 16. 13. The method according to claim 12, wherein high-frequency reproduction is added. 17． The method of claim 1 comprising constructing a composite waveform by generating a series of component sine waves corresponding in frequency and amplitude to the components to be extracted. 18. 18. The method of claim 17, wherein the time scale of the reconstructed waveform is changed by changing a rate at which the series of component sine waves is interpolated. 19. 19. The method of claim 18, wherein the time scale is continuously variable over a defined range. 20. The method of claim 1, comprising constructing a composite waveform by generating a series of component sine waves corresponding in frequency, amplitude, and phase to the components to be extracted. 21. 21. The method of claim 20, wherein the time scale of the reconstructed waveform is changed by changing a rate at which the series of component sine waves is interpolated. 22. 22. The method of claim 21 wherein the time scale is continuously variable over a defined range. 23. The component sine wave is further defined by the system contribution and the excitation contribution, wherein the time scale of the reconstructed waveform is determined by changing the rate at which the parameters defining the sine wave system contribution are interpolated. 21. The method of claim 20, wherein said method is varied. 24. 18. The method according to claim 17, wherein the short-time spectral envelope of the composite waveform is changed by scaling each frequency component. 25. 24. The method according to claim 23, wherein the pitch of the synthesized waveform is changed by scaling the excitation contributing frequency component. 26. An apparatus for processing an acoustic waveform, comprising: a. Sampling a waveform to obtain a series of discrete samples, and then constructing each series of frames spanning a plurality of samples; b. Individual amplitudes Analysis means for analyzing each frame of the sample to extract a set of frequency components having, c. Interpolating said component values from one frame to the next frame to obtain a waveform representation, wherein the composite waveform is A set of sine waves corresponding to the interpolated value using a smooth phase interpolation function in phase expansion that matches the phase and frequency of the variable frequency component extracted at the frame boundary and realizes the phase of the sample waveform at the frame boundary And an interpolation means configured to generate the sound waveform. 27. 27. The apparatus of claim 26, wherein the sampling means includes means for constructing a frame having a variable length that varies with the pitch period and is at least twice the pitch period of the waveform. 28. 27. The apparatus of claim 26, wherein said sampling means includes means for sampling according to a Hamming window. 29. 27. The apparatus according to claim 26, wherein the analyzing means includes means for analyzing each frame by Fourier analysis. 30. 27. The apparatus according to claim 26, wherein the analyzing means includes means for selecting a harmonic series to approach the frequency component. 31. 31. The apparatus according to claim 30, wherein the number of frequency components in the harmonic series changes according to a pitch period of the waveform. 32. 27. The apparatus of claim 26, wherein the tracking means includes means for matching frequency components from one frame with components in a next frame having similar values. 33. 33. The apparatus of claim 32, wherein said matching means provides a berth of a new frequency component and a death of an old frequency component. 34. The interpolating means combines the series of instantaneous frequency values to obtain a series of interpolated phase values with means for defining a series of instantaneous frequency values by interpolating the matched frequency components from one frame to the next. 27. The apparatus of claim 26 including means for: 35. 27. The apparatus according to claim 26, wherein the interpolation means includes means for deriving a phase value from the frequency and phase measurements obtained in each frame and then interpolating the phase measurement. 36. 27. The apparatus of claim 26, wherein said interpolating means includes means for performing an overlap and add function. 37. 27. The apparatus according to claim 26, including encoding means for encoding frequency components for digital transmission. 38. 32. The method of claim 32, wherein the frequency components are limited to a predetermined number defined by a plurality of harmonic frequency bins.
Item. 39. 39. The apparatus of claim 38, wherein the amplitude of only one of said components is encoded for gain and the amplitude of the other is encoded with respect to the next lowest frequency neighboring component. 40. Claim 37. The encoding means comprising means for applying a pulse code modulation technique to the predicted phase residual.
Item. 41. 38. The apparatus of claim 37, wherein the encoding means comprises means for generating a high frequency component. 42. 38. The apparatus of claim 37, further comprising means for constructing a composite waveform by generating a series of component sine waves corresponding in frequency and amplitude to the components to be extracted. 43. 43. The apparatus of claim 42, wherein the time scale of the reconstructed waveform is changed by changing a rate at which the series of component sine waves is interpolated. 44. 44. The apparatus of claim 43, wherein the time scale is continuously variable over a defined range. 45. 27. The apparatus of claim 26, further comprising means for composing a composite waveform by generating a series of component sine waves corresponding in frequency, amplitude, and phase to the components to be extracted. 46. 46. The apparatus of claim 45, wherein the time scale of the reconstructed waveform is changed by changing a rate at which the series of component sine waves is interpolated. 47. 47. The apparatus of claim 46, wherein the time scale is continuously variable over a defined range. 48. The component sine wave is further defined by the system contribution and the excitation contribution, wherein the time scale of the reconstructed waveform is determined by changing the rate at which the parameters defining the sine wave system contribution are interpolated. 43. The apparatus of claim 42, wherein said apparatus is varied. 49. 49. The apparatus according to claim 48, further comprising scale processing means for performing scale processing on the frequency component. 50. 49. The apparatus according to claim 48, further comprising a scale processing means for scaling an excitation contributing frequency component.