JPH0715640B2 - Sound analyzer synthesizer - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION (1) Technical Field of the Invention The present invention comprises an analyzing device, a main spectrum extracting device, and a synthesizing device for analyzing, extracting, and synthesizing sounds, and is particularly fast for anharmonic sounds. The present invention relates to an improvement of a synthesizer in an acoustic analyzer that can analyze the above.

(2) Conventional technology and problems Generally, natural sound is not analyzed so much, but a speech waveform is considered as a spectral envelope (frequency characteristic of vocal tract) and a sound source, and a spectrum as an information component is considered. The envelope is analyzed and extracted by a band filter group, and the sound source as an energy component is separated into a panel series and random noise. These two pieces of information, namely spectrum information and sound source information, are recorded or transmitted. On the synthesis side, the sound source is approximated by a pulse sequence and random noise from the sound source information, each frequency component is amplitude-modulated by the spectrum information, and the result is added through the band filter group corresponding to the analysis to reproduce the voice waveform. Such vocoder methods, analog methods such as terminal analog and vocal tract analog methods, and digital methods such as RARCOR have already been proposed or put into practical use.

However, these are all limited to harmonic sounds such as voices, that is, sounds whose harmonic relationship is an integral multiple, and have been developed as an analysis method therefor. Here, it is considered to be effective when the above-mentioned method is applied to the analysis of a highly harmonic sound such as a trumpet or thrombone. However, it is not suitable for analysis of anharmonic sounds represented by pianos and chimes, and it is very difficult. Therefore, there is a demand for a method that can analyze all natural sounds, especially nonharmonic sounds. In this case, it is important to reduce the amount of information to the same degree as that of a conventional harmonic sound, and high-speed processing is desired.

On the other hand, the present applicant has proposed a sound analysis device, which will be described in detail later, by another proposal. In summary, the main part consists of an analysis device, a main spectrum extraction device, and a synthesis device.In the analysis device, the digital sample value of the acoustic signal is cut out in a predetermined time window, and the spectrum is calculated in time series while shifting it. Then, the main spectrum extracting device obtains a spectrum envelope from the calculated frequency spectrum, and thereby obtains the frequency value, the amplitude value, and the phase value of the main spectrum component representing the most approximate N or less sine wave components in time series. After extraction, the synthesizer sets the frequency, phase and amplitude by N digital sine wave generators based on these main spectral component values to synthesize an acoustic waveform. According to the proposed invention, it is possible to process anharmonic sounds at high speed with a small amount of information as in the case of harmonic sounds. However, as a result of setting the number of main spectrum component values to N, the synthesizer proposed here synthesizes at intervals of, for example, 10 msec, and when the timbre change is large, the fidelity is poor and noise is generated. Things will happen.

(3) Object of the Invention An object of the present invention is to provide a sound analysis apparatus capable of analyzing, extracting, and synthesizing a whole natural sound, in particular, anharmonic sound at high speed. It is an object of the present invention to provide a synthesizer having high fidelity against changes and having less noise.

(4) Configuration of the Invention In order to achieve the above object, the synthesizer of the acoustic analysis device of the present invention cuts a digital sample value of a musical tone signal which changes with time in a predetermined time window and shifts the digital sample value to address one block. An analyzer for calculating the frequency spectrum of the frequency spectrum of the above in a time series at a predetermined time interval by the Fourier transform method, and "the frequency, phase and amplitude data representing N or less sine wave components sufficiently smaller than the number of the frequency spectrum. A main spectrum extracting device for extracting the main spectrum component value consisting of the frequency spectrum from the envelope in the order of peak levels, and extracting the main spectrum component values according to the time series, and N digital sine capable of individually controlling frequency, phase, and amplitude. It has a wave generator and follows the time series based on each value of the main spectrum component values extracted by the main spectrum extraction device. A synthesizer for synthesizing a musical sound signal that changes with time, and an acoustic analyzer for analyzing and synthesizing harmonic or inharmonic musical tones, wherein the synthesizer comprises N frequencies, phases and amplitudes at a certain time. 1
A main data memory that stores a plurality of sets of main spectrum component values in time series, and a set of main spectrum component values at time T, and stores the main spectra in accordance with the synthesis speed of the synthesizer. A first memory for reading each value of the component value, and a set of main spectral component values at T-1 before the unit time is stored, and each value of the main spectral component value is read in accordance with the combining speed of the combining device. Address control means for sequentially reading the second memory and the main data memory at a speed higher than the speed used for combining the N main spectrum component values at the address times T and T-1 and transferring them to the first and second memories. And an interpolation counter for outputting divided times from time T-1 to time T, and a main spectrum component value read out from the first and second memories in accordance with the combination speed, the interpolation counter outputting the times. Each value time T
-1 to time T for linearly interpolating means, and by inputting the main spectrum component value data interpolated by the interpolating means to the sine wave generator, harmonic or anharmonic with less noise It is characterized by synthesizing various musical sounds that change with time.

(5) Embodiment of the Invention FIG. 1 is a schematic explanatory view of the whole of another proposed acoustic analysis apparatus which is the basis of the present invention. The acoustic signal is input to the analyzer 100, where the frequency spectrum component of the input signal is calculated. The input signal is digitally converted and then analyzed with one block as a sample section of a constant time window. Subsequently, the samples are shifted by several samples and analyzed one after another. This allows
A time series frequency spectrum is obtained. The result of this analysis is represented by a spectral envelope with many peaks (formants). This peak is caused by the fact that the waveform period is a non-integer multiple with respect to the analysis section and the influence of the window function used in the Fourier transform. For this reason,
The spectral component corresponding to the peak of the mountain is extracted, and the data is compressed into an effective number of spectra.

The main waveform terms used in this specification will be collectively described here with reference to FIGS. FIG. 5A shows a so-called frequency spectrum in which the vertical axis represents the frequency and the vertical axis represents the digital sample value of the acoustic signal.

The term "spectrum" is synonymous with the individual frequency spectrum and refers to each line spectrum. On the other hand, the term "frequency spectrum" refers to the state in which this spectrum is aggregated, and the frequency, amplitude, and
It contains each phase information. Next, the "spectral envelope" refers to the envelope of the frequency spectrum as shown in FIG. Further, the "main spectrum" shown below is used as a new term in the present invention and is shown in FIG.
As shown in FIG. 5, it means a line spectrum corresponding to sine wave components of arbitrary frequencies within N extracted from the spectrum envelope, and shows one line segment of frequency, phase, and amplitude data for synthesis. .

The main spectrum extraction device 200 extracts a number N of main spectra, which is considerably smaller than the number of data of the data representing the spectrum envelope output from the analysis device 100. N
Each main spectrum consists of frequency, amplitude, and phase data, and is transferred to the synthesizer 300. The synthesizer 300 has a frequency
It has N sine wave oscillators whose amplitude and phase can be controlled arbitrarily, and the above-mentioned N extracted main spectrum data are assigned to the N oscillators respectively, and the timbre changes at the same time intervals as in the analysis. The waveforms are synthesized moment by moment, and then the sound is reproduced by the sound system 400.

As described above, in the acoustic analysis device, in the process from analysis to synthesis, a number N of important and considerably small number of main spectra are limitedly extracted from the frequency spectrum of the Fourier-transformed acoustic signal, and this is synthesized and reproduced. It is done by doing.

For example, if one analysis interval is 1024 samples, 512 frequency spectra at the maximum will be calculated.

N, for example, 32 main spectra are extracted from the spectrum envelope (envelope curve of the entire frequency spectrum) indicated by X _k . At this time, the value of the main spectrum may be a value calculated by a method of inferring from the interpolated spectrum envelope or its shape. Subsequently, the main spectrum including the extracted frequency, phase, and amplitude is assigned to 32 independent sine wave oscillators, respectively. A waveform x (t) represented by (α _q : amplitude, ω _q : frequency, θ _q : phase) is synthesized. In addition, a set of 32 α _q , ω _q , θ _q is a predetermined time interval, for example, 10 msec.
Each time, it changes according to the analysis result.

In this way, not only the synthesis of the harmonic overtones of the 32nd harmonic, but also the synthesis of the nonharmonic tones and the use of only valid data, for example, since the square wave does not include even harmonics, With 32 composites, up to 64 overtones can be played, and high quality composites can be performed.

FIG. 3 is a detailed explanatory diagram of an embodiment of the analyzer 100.

The acoustic input signal is analog-digital converted by the ADC 101 and then temporarily stored in the waveform memory 102. The waveform memory 102 is a waveform memory of 256 K words if it can store the sampling frequency of 51.2 KHz for 5 seconds. The stored waveform is, for example, 1024 samples of Fast Fourier Transform as one block of the time window.
The frequency spectrum is calculated by inputting it to the FFT calculation circuit 103. By sequentially shifting this block by several words and performing the same calculation, several spectral envelopes that change with time are calculated. This spectrum data is transferred to the main spectrum extraction device 200.

FIG. 4 is a detailed explanatory diagram of an embodiment of the main spectrum extracting apparatus 200. The spectrum data calculated by the analyzer 100 transfers a set of spectrum envelopes to the spectrum envelope memory 201 at each time.

Next, the peak value of the peak of the spectrum envelope in the spectrum envelope memory 201 is detected by the n-th peak level detection circuit 202, and N peak levels, for example 32 peak levels, are detected in order from the largest peak value. As a result, the amplitude value, the frequency value, and the phase value at the detected 32 points are extracted by the amplitude extraction circuit 203, the frequency extraction circuit 204, and the phase extraction circuit 205, respectively.

Here, the 32 main spectral components will be selected such that their waveforms are also best represented. Next extracted N
The main spectrum data are assigned to N, for example, 32 sinusoidal oscillators in the synthesizer 300.

FIG. 5 is a waveform diagram showing a method of extracting eight main spectra from the spectrum envelope. The point showing the highest level of the mountain is higher than the smooth spectral envelope shown by many points (for example, 512 points).
Are shown in that order. Here, when the extraction frequency range up to f _c, 7 th spectrum is unnecessary, 9 th f ₉ are extracted instead.

By limiting the extraction frequency band in this manner, it becomes possible to extract a spectrum of a small level and the usability is improved. In this way, the phase and the amplitude are simultaneously extracted based on the frequencies detected in order of level. The missed spectrum is of a much lower level,
It is not so important to the overall sound characteristics. Therefore, sufficient reproducibility can be obtained even if synthesis is performed with a small number of spectra.

FIG. 6 is a detailed explanatory diagram of an embodiment of the synthesizer 300. All the main spectrum data (amplitude, phase, frequency) extracted at each time by the main spectrum extraction device 200 is temporarily stored in the extracted data memory 301. This is because the analysis and extraction for each time is not performed in real time, and the time interval cannot be met in time so that the next process can be performed. In addition, it is necessary to temporarily store all the extracted data in order to reproduce it many times.

For this reason, the memory capacity is, for example, 10 seconds for audio signals of 5 seconds.
When analysis is performed at msec intervals and main spectrum data of 32 units of frequency, phase, and amplitude are extracted, (5000/10) × 32 ×
3 = 48000 words of memory. Next, the address control circuit 302 has a frequency memory 305, a phase memory 304, and an amplitude memory at each time of 32 main spectrum data consisting of frequency, phase and amplitude at a predetermined time, at a speed irrelevant to the speed used for synthesis. Each piece of data is transferred to 303 and each piece of data is temporarily stored in each memory so that the sound can be resynthesized at the same speed as in the analysis. Next, the frequency memory 305 stores N pieces of frequency data ω _q (q = 1, 2, ... 32) for determining the angular velocity (hereinafter, 32 pieces are taken as an example), and is updated at predetermined time intervals. . This data is always accumulated by the frequency accumulator 306 at a predetermined clock (eg sampling clock in the analyzer) and in 32 time divisions to calculate the phase angle ω _q t at time t. Next, this output ω _q t is input to the phase adder 307, and the phase data θ _q output from the phase memory 304, which is the other input.
(Q = 1, 2, ... 32) is added to calculate (ω _q t + θ _q ). Next, the sine wave value SIN (ω _q t + θ _q ) is read from the sine wave table 308 that digitally stores one cycle of the sine wave by the output of the phase adder 307, and is output from the amplitude memory 303 to this output. Amplitude data α _q (q = 1,2, ...
32) is multiplied by the multiplier 309, and a combined spectrum of α _q sin (ω _q t + θ _q ) is output. Next, the 32 time-division synthesized spectrum waveform data output from the multiplier 309 is accumulated by the spectrum accumulator 310, After the digital composite waveform is calculated, and digital-analog conversion is performed by the DAC311, the sound system
Sound is output from 400.

As described above, the synthesizing device 300 is a device capable of synthesizing and reproducing 32 main spectrum components in a time division manner, and further capable of setting an arbitrary frequency, phase and amplitude for each spectrum.

Further, since the extracted data memory 301 and the address control circuit 302 perform data transfer at a speed that is independent of the speed of synthesis, it is possible to easily produce a polyphonic sound by having a plurality of devices 303 to 311.

The above is a description of the configuration and operation of the separately proposed acoustic analysis device. As a result, it is possible to analyze a relatively small number of N natural sounds including anharmonic sounds High-speed processing is possible by using each main spectrum component value.

The synthesizer used here has a timbre change of 10 ms as described above.
If it is set for each ec, for example, the memory capacity for 5 seconds
It requires 4800 words of memory. When this timbre change is made faster, the timbre fidelity is increased and the noise due to the change is also reduced. However, as a result, a large memory capacity is required. Therefore, as a method of smoothing tone color changes and reducing noise, it is considered to perform processing such as subdividing the main spectrum data change of 10 msec and interpolating between them with only the interpolation circuit at the synthesis stage. is there.

That is, in the present invention, the above-described processing is realized. By using both the main spectrum, that is, the data of the main spectrum, that is, the frequency, the phase, and the amplitude data that change at two times of the main spectrum, and interpolating between the two times. , Which has high fidelity and reduces noise.

FIG. 7 shows a synthesizing device 30 including an interpolation circuit, which is an essential part of the present invention.
It is a structure explanatory view of the example of 0 '. Extracted data memory 301
Stores M sets of main spectrum data (one set consists of N pieces of frequency, phase, and amplitude data). For example, as a result of analyzing 0 to 5.12 seconds every 10 msec, that is, 512 sets of data are stored. The address control circuit 302 converts the two types of main spectrum data at time T and time T-1 into the amplitude memory 303, the phase memory 304,
Transfers to the frequency memory 305 while changing in 10 msec. The main spectrum data S _t at time T and the main spectrum data S _{r-1 at} time T _-1 are stored in each memory. Next, the two-time data in each memory are given to the amplitude interpolation circuit 313, the phase interpolation circuit 314, and the frequency interpolation circuit 315, respectively. On the other hand, the time interval of 10 msec is further divided into m pieces, and the time interval of Δt is generated. For example, it is divided into 64 pieces and Δt = 0.156 mse.
The output K of the interpolation counter 312 that counts the time for performing the interpolation at the time interval of c is the above-mentioned interpolation circuits 313, 314, 315.
Are typing in. Here, each interpolation circuit has main spectrum data S = S _t-1 + (S _t −S _t-1 ) · K or = K · S _t + (1
-K) · S _t (K is an operational circuit that satisfies the output value) of the interpolation counter 312.

This interpolated main spectrum data S is as described in FIG. Then, a digital synthesized waveform is calculated and output as sound. 8th
The figure is a detailed explanatory diagram of the amplitude interpolation circuit 313 as a representative of the interpolation circuit. The phase interpolation circuit 314 and the frequency interpolation circuit 315
Is also composed of exactly the same circuit.

The address control circuit 302 generates an address for transferring the main spectrum data at time T (T) and an address counter 302B for generating the main spectrum data at time T-1 (T-1). ) An address counter 302C and a selector 302A that selects both addresses are used to control the address of the extraction data memory 301,
The main spectrum data at two times is read every 0 msec. Next, of the main spectrum data at two times, the amplitude data is the amplitude memory 303A that stores the amplitude data A at time T and the time T.
It is transferred to the amplitude memory 303 including the amplitude memory 303B for storing the amplitude data B of -1.

Next, the amplitude data A and the amplitude data B, and the interpolation counter 31
The value C obtained by further finely counting the two times output from 2 is input to the amplitude interpolation circuit 313. In the amplitude interpolation circuit 313, the subtractor 313A calculates AB, the multiplier 313B calculates (AB) × C, and then the adder.
B + (A−B) × C is calculated at 313C. The amplitude interpolation circuit 313 is an accumulator-multiplier (for example, TDC1010J manufactured by TRW).
It is also possible to execute A × C + B × (1-C) so as to calculate according to

As described above, the above-mentioned interpolating circuit for interpolating between two values while leaving the N main spectrum component values as it is, a sufficiently finely tuned composite output with little noise can be obtained.

(6) Effects of the Invention As described above, according to the present invention, in the acoustic analysis device including the analysis device according to the proposal, the main spectrum extraction device, and the synthesis device, the time of the synthesis device is the time in the proposed example. By providing means for further dividing and interpolating between T and time T-1 and inputting the main spectrum data interpolated by the interpolating means to the sine wave generator, each of the frequency, phase, and amplitude can be obtained. It is interpolated to synthesize an acoustic waveform. As a result, it is possible to synthesize a fine acoustic waveform compared to the case of the coarse main spectrum component value of the proposed example, and to obtain a fine acoustic waveform of the number of divisions that is interpolated without increasing the amount of information so much and improve the fidelity. And it is possible to reduce noise.

[Brief description of drawings]

FIG. 1 is a schematic explanatory view of a proposed example which is the basis of the present invention, and FIG.
The figure is an explanation of terms related to waveforms, Fig. 3, Fig. 4, Fig. 6
FIG. 4 is a detailed explanatory view of the main part of FIG. 1, and FIG.
FIG. 7 and FIG. 8 are operation waveform diagrams related to the figure, and are detailed explanatory views of the main part of the present invention, in which 100 is an analyzer and 101 is an AD.
C and 102 are waveform memory, 103 is FFT operation circuit, 200 is main spectrum extraction device, 201 is spectrum envelope memory, 202 is nth peak level detection circuit, 203 is amplitude extraction circuit, 204 is frequency extraction circuit, and 205 is phase. Extraction circuit, 300,30
0'is a synthesizer, 301 is an extraction data memory, 302 is an address control circuit, 303 is an amplitude memory, 304 is a phase memory, 305
Is a frequency memory, 306 is a frequency accumulator, 307 is a phase adder, 308 is a sine wave table, 309 is a multiplier, 310 is a spectrum accumulator, 311 is a DAC, 312 is an interpolation counter, 313 is an amplitude interpolation circuit, 314 is a phase interpolation circuit, 315 is a frequency interpolation circuit,
400 indicates a sound system.

Continuation of front page (56) References JP-A-56-51795 (JP, A) JP-A-57-182798 (JP, A) JP-A-58-147798 (JP, A) Electronic Technology Vol. 22-13 (1980-12) P. 22-25 Proceedings of ASJ (S54.10) 3-2-3P. 557-558 Proceedings of the Acoustical Society of Japan (S55.5) 3-4-9P. 639-636 Proceedings of the Acoustical Society of Japan (S55.10) 1-1-1-1. 359-360 Hino “Spectral Analysis” (1977-10-1) Asakura Shoten P. 42-43 Markel and Gray, Suzuki Translation, "Linear Prediction of Speech" (S55-3-25) p. 198-232

Claims (1)

[Claims] 1. A digital sample value of a musical tone signal which changes with time is cut out in a predetermined time window, and the frequency spectrum addressed to one block is shifted in a time series at a predetermined time interval by a Fourier transform method. And a main spectrum component value consisting of frequency, phase, and amplitude data representing N or less sine wave components, which is sufficiently smaller than the number of the frequency spectra, and extracted from the envelope of the frequency spectrum in the order of peak level. A main spectrum extracting device for extracting in accordance with the time series, and N digital sine wave generators capable of individually controlling frequency, phase, and amplitude, and main spectrum component values extracted by the main spectrum extracting device. A synthesizer for synthesizing a musical tone signal that changes with time in accordance with the time series based on each value of Analysis harmony musical tones, the acoustic analyzer for synthesis, the synthesis device comprises at a certain time N frequency, phase, than the amplitude 1
A main data memory that stores a plurality of sets of main spectrum component values in time series, and a set of main spectrum component values at time T, and stores the main spectra in accordance with the synthesis speed of the synthesizer. A first memory for reading each value of the component value, and a set of main spectral component values at T-1 before the unit time is stored, and each value of the main spectral component value is read in accordance with the combining speed of the combining device. Address control for sequentially addressing the second memory and the main data memory, and sequentially reading the N main spectrum component values at time T and time T-1 at a speed higher than the speed used for combining and transferring to the first and second memories Means, an interpolation counter for outputting divided times from time T-1 to time T, and a main spectrum component read out from the first and second memories in accordance with the combining speed Time T each value of
An interpolating means for linearly interpolating from -1 to time T, and by inputting the main spectrum component value data interpolated by the interpolating means to the sine wave generator, harmonic or anharmonic with less noise. A synthesizer for an acoustic analysis device, which synthesizes various musical sounds that change over time.