JPH04213500A - Method and device for encoding voice - Google Patents

Abstract

PURPOSE: To encode a voice to a form suitable for storing it in a storage device of a computer. CONSTITUTION: This device for encoding a voice consists of a sampling circuit 9 for sampling the voice and generating digital data expressing the sampled signal. An encoder 11 connected to the circuit 9 encodes digital data by linear predictive encoding and generates a variable expressing a sound signal. The variable is converted by a converter 12, a route of a polynomial for a sum and a difference is determined by a circuit 13 so as to generate linear spectral data and these data are stored. As an example, the linear spectral data are irregularly quantized in a frequency area and the route of the polynomial can be found out only by evaluating the polynomial by quantized frequency.

Description

DETAILED DESCRIPTION OF THE INVENTION [Field of Industrial Application] The present invention relates to a data compression method,
More particularly, it relates to encoding audio in a form suitable for storage in computer storage. BACKGROUND OF THE INVENTION It is desirable to facilitate audio recording of personal computers, particularly portable handheld or laptop type machines. For example, recorded audio may be used to annotate documents in a word processor running on a machine, or used to provide instructional messages in connection with a machine-controlled diary. It is known to equip personal computers with DSPs (digital signal processors) which are used to process audio consisting of human voices for storage by the computer. However, in order to store audio in acceptable quality, known devices require high data transfer rates, so in practice, audio of a few seconds or less is
Stored in typically available megabytes of RAM,
Moreover, the tens or even hundreds of megabytes available on mass storage devices, such as hard disks, are rapidly used up. To overcome this problem, data compression methods are used; however, known compression algorithms that allow very high compression ratios, which are desirable in this discussion,
Calculations are very intensive. Therefore, it has hitherto been impossible to record long-duration audio with this type of device due to the two limitations of personal computers: limited storage capacity and limited processing power. SUMMARY OF THE INVENTION According to a first aspect of the present invention, a method of audio encoding for storage in a computer comprises sampling an audio signal, encoding the sampled audio data with a linear predictive code, and Generating variables representing audio data, converting variables to obtain sum and difference polynomials, determining roots of polynomials and thereby generating line spectrum data, and storing line spectrum data in a computer, etc. It is configured. [0005] The inventors have developed an encoding that combines several features that give very superior performance to known devices,
This increased the compression ratio to about 100:1. This algorithm is computationally efficient enough to be implemented on a relatively low power handheld personal computer using a common portable DSP. [0006] Linear predictive coding is a well-known method used for speech analysis and synthesis as a prediction, and is described in, for example, the Journal of the Acoustical Society of America, 1971, vol. 50 No. 2
(Part 2), pp. 637-655, by B.S. Attar and others on "Speech Analysis and Synthesis by Linear Prediction of Speech Waves." The use of linear spectral pairs is described in IE, Special Issue on Acoustics, Speech and Signal Processing, December 1986, vol. A.S.P.-34, No. 6, P. Kabeil et al., “Calculation of line spectral frequencies using Tievisiev polynomials,” and IEE special issue on acoustics, speech and signal processing, April 1987, vol. A.S.P. No. 4, pp. 568-571, by George S. King et al. However, the root-calculating algorithms used to derive the line spectral pairs are computationally intensive, so it has never been practical to implement this form of method on relatively low-power computers. could not be considered, and the compression ratio obtained without using line spectrum pairs was not high enough. Preferably the stored line spectrum pairs are:
Quantized in the frequency domain with non-uniform quantization, the roots of the polynomial are found only by evaluating the polynomial at the quantized frequency. The compression ratio and the computational efficiency of the algorithm are found by evaluating the relevant function at the frequency at which the line spectral pair is quantized, and by quantizing the line spectral pair in the frequency domain by appropriate quantization of the frequency range. It has been shown that this can be improved. The sampled audio data is suitably divided into frames of a predetermined period, the consecutive frames are compared, and if the difference between the new frame and the previous frame is less than a predetermined degree, the new frame is divided into frames of a predetermined period. Encoded by an instruction to repeat the frame. [0009] If the new frame differs from the previous frame by more than a predetermined degree, the compression ratio can only be further improved by storing the new frame. If the frames are the same, the new frame is encoded one bit at a time in a certain area, which is recognized by data restoration as an instruction to repeat the previous frame. As discussed in more detail below,
It has been shown that with appropriate frame-to-frame comparison criteria, the number of stored frames can be reduced by as much as 50% without any significant loss of quality. This method is
The graphical display of the representation of the audio signal and the editing of the stored variable audio signal in response to actions of the graphical representation of the signal performed by the user are more suitably arranged. Using these compression methods set out in accordance with the first aspect of the invention, it is possible to store and edit audio using a computer in the same way that text is stored and edited using a word processor. is now possible. but,
The problem arises from setting the proper user interface. It is known in personal computers incorporating simple digital samplers to form a display of sampled data in the form of a plot of amplitude versus time. While such displays are somewhat useful for processing relatively uniform signals such as music, they are known to have more difficulty processing complex signals such as voice. The inventor has shown that if the signal is stored in a compressed and variable form, a representation can be generated from the stored variables which greatly facilitates the editing of complex signals such as speech. . Thus, the user can perform many of the operations commonly performed on text processors, including cutting, pasting, copying, and deleting. The method also consists of selecting points in the audio signal for editing by moving a cursor with respect to a graphical representation of the signal, and at the same time regenerating the part of the signal marked by the cursor. Properly configured. According to a second aspect of the invention, there is provided an apparatus for encoding speech comprising the following means. That is,
means for sampling the audio signal to thereby generate digital data representing the audio signal; and a means for sampling the audio signal and arranged to encode the digital data by linear predictive coding; means for transforming said variables and computing corresponding sum and difference polynomials; and determining the roots of said sum and difference polynomials thereby generating a series of lines. They are means for generating spectrum data, and storage means for storing the line spectrum data. According to a third aspect of the invention, there is provided a computer comprising means for encoding and decoding audio, and said means for encoding and decoding comprising the following means. namely, means for sampling an audio signal to thereby generate digital data representative of said audio signal; connected to said means for sampling and arranged for encoding said digital data by linear predictive coding; an encoder for generating variables thereby representing said audio signal; means for transforming said variables and computing corresponding sum-difference polynomials; and determining roots of said sum-difference polynomials thereby means for generating line spectral data of
means for storing the line spectrum data; means for retrieving the line spectrum data from the storing means; means for determining a sum and difference polynomial corresponding to the retrieved line spectrum data; means for transforming a difference polynomial to produce a variable representing said audio signal; a decoder arranged to decode said transformation and generating digital data representing said audio signal; and a decoder corresponding to said digital data. means for generating and outputting analog signals. DESCRIPTION OF THE PREFERRED EMBODIMENTS A laptop personal computer 1 consists of a microphone 2 connected via an analog-to-digital converter ADC to a digital signal processing device DSP. If appropriate, the data output by the DSP is written to the mass storage device 4, in this example a battery-equipped RAM cartridge, or to the main RAM 15 of the computer 1 under control of the main CPU 3. The analog-to-digital converter samples the audio received by the microphone and produces a data stream at a rate of 64 kilobits per second. The generated data stream is divided into frames of 25 millisecond duration in this example. For each frame, a number of variables is calculated. These variables are the number of prediction coefficients that define a discrete time-varying linear filter with the audio in the frame as the excitation frequency and a transform function that shapes the behavior of the audio track after the excitation produced by the audio code. Expressed as If a frame consists of P samples, then ak is the general predictor coefficient and Sn is the general sample value, then this predictor error En is:
is the difference between the audio sample Sn and its predicted value given by Equation 1. Therefore, En is given by Equation 2. The value of the predictor coefficient is determined by the unbiased variance of the prediction error <
En 2 > selected to minimize AV. Bee
As described in considerable detail in the above-cited paper by S. Attar et al., applying this constraint to a sampled data set yields a set of simultaneous first-order legal equations that, when solved, yield the predictor coefficients. is required. To further compress the data, the predictors are not stored directly, but are used to form polynomials that are decomposed into sum and difference polynomials. The roots of this sum and difference polynomial constitute a set of line spectra, and this set is
A collection of line spectra output by a DSP for storage by a computer and then used in a complementary synthesis process. An algorithm for finding roots and a complementary algorithm for deriving predictor coefficients from a stored collection of line spectra are described in detail at the end of the specification. Their roots are found by evaluating the polynomial at many frequencies. Rather than using continuous frequencies, this function is evaluated at predetermined intervals in a non-uniform space. In this example, logarithmically spaced frequencies are used for quantization. These frequencies listed in Table 1 are chosen to set a number of quantization levels in this region to which the ear is most sensitive. Next, as shown in FIG. 2, the root position is set between the frequencies f1 and f2 whose signs change. Of the two frequencies, the one that is closer to polynomial P(f) is selected to represent the root. For clarity, FIG. 2 shows a more simplified function as representing a polynomial. As a result of the above processing, each frame is represented by a series of first, second, . . . nth roots, and each root is
It is composed of 1 to 42 indicators corresponding to the quantization level of each frequency. It becomes clear that even more significant increases in compression ratios can be obtained by comparing successive frames and if the new frame differs by less than a predetermined degree from encoding that frame with instructions to repeat the previous frame. It was done. Comparisons are made between corresponding roots of many frames. That is, the first root of one frame is compared with the first root of another frame, the second root of one frame is compared with the root of another frame, and so on. The difference in quantization units of the corresponding roots is determined. It has been found that a frame can be replaced by a "repeat" instruction if there is no significant deterioration in the quality of playback and the roots thus compared do not differ by more than three. If the roots are allowed to differ by 5, audible degradation will occur. If the standard is 3,
Approximately 50% of the frames are repeated. Tighter constraints are used to further improve the quality of playback. For example, if a frame is repeated only if the root does not differ by more than 1 from the corresponding root of the previous frame, the percentage of frames encoded as repeats will be reduced, but the quality of playback will be correspondingly There is improvement. If the higher frequency roots are allowed to vary more than the lower frequency roots, in the preferred embodiment the criterion for the repeating frame is that the first four roots should differ by no more than one quantization unit. , and the remaining roots should differ by no more than 2. In total, each frame output by the DSP is:
A first variable indicating whether the frame is voiced or unvoiced, and a second variable that sets the pitch period of the stimulus tone, together with a set of line spectra computed from the predictor coefficients a1...ap. It consists of variables. The pitch period is determined using any one of a number of common pitch determination methods. This method is described in the Journal of the Acoustical Society of America, August 1969, vol. 46,442-448
Examples include the method described in the paper "Parallel Processing Method for Calculating the Pitch Period of Speech in the Time Domain" by B. Gold et al. Once a set of line spectra is stored, the audio can be retrieved at any time by calling the stored set out of storage and applying a complementary algorithm to calculate the predictor coefficients from the set of line spectra. will be played. Suitable algorithms are listed in Annex 2. The generated data is transferred to a digital-to-analog converter, an audio amplifier, and a speaker (not shown). [0023] The audio data stored in the storage device is edited by the user. A window 7 is provided on the computer display 6 and displays a selected portion of the variable audio data. Using the cursor 8 under the control of an input device such as a mouse or digital pad, the user can manipulate the displayed data to perform functions such as cutting, pasting, copying, or erasing. I can do it. Input from the input device causes the stored variables to change correspondingly. [0024] Whenever the cursor indicating an edit point moves relative to the display, the frames for the edit point under the cursor are played back in succession and at a speed determined by the speed and direction of the cursor movement. Ru. Therefore, the user can select edit points by ear without having to proceed by trial and error, selecting edit points, playing the selected part with a separate next action, etc. I can do it. Since the data is variable with pitch encoded independently of other variables, changes in speed do not change the pitch of the reproduced audio. Therefore,
The cursor can be moved at various speeds without loss of intelligence. In other modes of editing, the cursor is used to indicate anchor points. Audio data from the advancing anchor point is played back. As soon as the cursor indicating the anchor point is moved, data playback begins again at the new anchor point. The manner in which the display is derived from the stored data will vary depending on the requirements of any given area of use and the constraints of the particular display device. In this example, each point plotted on the display device represents the average amplification of four consecutive frames. Appendix 1 /* Extracts of code to do L
BP root finding from pred
ctor coefficients, and
to convert back from root
s to predictor coefficien
ts. written by Lionel Wol
ovitz August 1988. */ [0027] double Eval(x, poly)
/* Evsluate oolynomiel (5 ad
ds, 4 subs, 4 muls) */ double × *poly; { double tempO, templ, temp
2; templ=2.0* × +poly[1]
; temp2=2.0* × *temp1-1.
0 + poly[2]; temp0=2.0* × *
temp2-temp1+poly[3]; temp2
=2.0* × *temp0-temp2+poly
[4];return(× *temp2-temp0
+poly[5]); } FindRoots(q, p, x, table)
/* Find roots of sum and di
ference polynomials. *
/ double *q; /* sum p
oly */ double *p; /
* diff poly */ doub
le *x; /* roots (in orde
r i. e. sun and diff inte
rleaved */ double *t
able; /* quantification
table */ { double *r, *t, temp,
prev;0029;0031;Appendix 2
FredFromRoots(x,a) /
* Compute predictor coeffi
cients from roots of sum
snd diff polys, */ double *x, *a; { double q[10], p[10]
; double temp1, temp2,
temp3;0032;0033;0034; /*calculate predict
or coefficients from sum
and difference poly
nominals */ q[5]=2.0*
q[5]+q[4]; q[4]+=q[
3]; q[3]+=q[2];
q[2]+=q[1]; q[1]
+=1.0; p[4]-=p[3]
; p[3]-=p[2];
p[2]−=p[1];

[Brief explanation of the drawing]

[Figure 1] Configuration diagram of a computer used in the present invention

[Figure 2] Graph showing the roots of a polynomial

[Figure 3] Encoder block diagram

[Figure 4] Block diagram of decoder

FIG. 5 is a diagram showing a display device used for editing audio signals.

Claims (17)

[Claims] 1. Sampling of an audio signal, encoding of sampled audio data by linear predictive coding, generation of a variable representing the audio data thereby, conversion of the variable to obtain a sum and difference polynomial, and the polynomial A method of encoding speech for storage in a computer, comprising: determining the root of , thereby generating a series of line spectral data, and storing said line spectral data in a computer. 2. The stored line spectral data is quantized in the frequency domain with non-uniform quantization, and the roots of the polynomial are determined only by evaluating the polynomial at the quantization frequency. A method according to claim 1, characterized in that: 3. The sampled audio data is divided into frames having a predetermined time, consecutive frames are compared, and if the new frame differs from the previous frame by less than a predetermined degree, the new frame differs from the previous frame. 3. A method according to claim 1 or 2, characterized in that it is encoded by an instruction to repeat the frame. 4. The frames are compared by determining a difference in quantization units between corresponding roots in each successive line spectral data, and the frames are repeated if the difference does not exceed a predetermined limit. 4. The method according to claim 3, characterized in that: 5. A lower predetermined limit is set for a lower frequency term, and a different predetermined limit is set for a different portion of each successive line spectral data. the method of. 6. Encoding and storing an audio signal by a method according to any one of the preceding claims, displaying a representation of the audio signal as a graphic, and responding to an operation of graphically displaying the signal performed by a user. A method of editing a memory, comprising responsively editing said stored variable audio signal. 7. The graphical representation of the signal further comprises moving a cursor to select a point within the audio signal for editing and simultaneously reproducing the portion of the signal indicated by the cursor. 8. The method according to claim 7, characterized in that: 8. The step of graphically displaying the audio signal comprises plotting a variable representing a signal averaged over a plurality of frames as a function of time. the method of. 9. Means for sampling an audio signal and thereby generating digital data representing the audio signal.
2,9) and an encoder (10,11) connected to said means of sampling and arranged to encode said digital data by linear predictive coding, thereby producing variables representative of said audio signal; means (12) for transforming said variables and calculating corresponding sum and difference polynomials; and means (13) for determining roots of said sum and difference polynomials thereby generating a series of line spectral data; , storage means (14) for storing the line spectrum data. 10. Means (19) for comparing successive frames of audio data having a predetermined duration, said means for comparing determining that the current frame differs from the previous frame by less than a predetermined degree, 10. The apparatus of claim 9 further comprising means responsive to said comparing means for generating and storing instructions to repeat frames. 11. The means for comparing (19) comprises means for reading a series of line spectral data corresponding to each frame and determining a difference between corresponding terms in the series of line spectral data; and means for comparing the difference between the frames with a predetermined limit (δ), characterized in that if none of said differences exceeds a predetermined difference, said frames are determined to differ by less than said predetermined difference. Claim 10
The device described in. 12. Said means (19) for determining and comparing differences.
) is set to the first and second different predetermined limits (δ
4. Device according to claim 3, characterized in that the magnitude of the predetermined limit of the low frequency term is smaller than the magnitude of the predetermined limit of the high frequency term. 13. Said means (13) for determining roots of said sum and difference polynomial evaluate said polynomial only at non-uniform quantization frequencies, thereby determining the roots of said sum and difference polynomial. 13. Claims 9 to 12 characterized in that the means (14) for storing the line spectra store quantized data at the non-uniform quantization frequencies in the frequency domain. Apparatus according to any one of the preceding clauses. 14. Means (6) for graphically displaying the representation of the audio signal; and editing means for editing the stored quantized audio signal in response to the graphical display operation performed by the user. 14. Device according to any one of claims 9 to 13, characterized in that the device comprises: 15. A computer having encoding and decoding means, connected to means (2, 9) for sampling an audio signal and thereby generating digital data representing said audio signal, and means for sampling. an encoder (10, 11) arranged to encode the digital data by linear predictive encoding, thereby generating variables representing an audio signal; means (12) for determining the roots of said sum-difference polynomial and thereby generating a series of linear spectral data; and means (14) for storing said line spectral data. , means (16) for retrieving the line spectrum data from the storage means, means (16) for determining a sum and difference polynomial corresponding to the retrieved line spectrum data, and converting the sum and difference polynomial. , means (17) for generating a variable representing the audio signal;
a decoder (18) arranged to decode said variable, thereby generating digital data representing said audio signal; and means (20) for generating and outputting an analog signal corresponding to said digital data. A computer having encoding and decoding means consisting of: 16. The computer of claim 15, wherein the computer is a handheld or laptop computer. 17. A handheld or laptop computer, characterized in that it comprises a device according to any one of claims 9 to 14.