JP2003084799A - Frequency analysis method and sound signal encoding method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a frequency analysis method and a sound signal encoding method which improve the time resolution without degrading the frequency resolution neither increasing the operation load. SOLUTION: With respect to a method which sets unit sections to a time series signal and analyzes each unit section to separate a signal component corresponding to a prescribed standard frequency, the unit sections are so set that they may overlap by a prescribed length, and a section signal itself in an objective unit section dt is not analyzed but the section signal corrected by the final residual signal in the just preceding unit section dt-τ is analyzed by the generalized harmonic analysis method in the case of analysis of the objective unit section dt as the analysis object.

Description

Detailed Description of the Invention

[0001]

The present invention relates to broadcast media (radio, television), communication media (CS video / audio distribution, Internet music distribution, communication karaoke), package media (CD, MD, cassette, video, LD, CD). −
Production of various audio contents provided in ROM, game cassettes, solid-state memory media for portable music players, etc., as well as music performance recording signals, musical score publishing, MIDI data for communication karaoke distribution, and automatic performance data for electronic musical instruments with performance guide function. , Automatic music transcription technology for automatically creating ringing melody data for mobile phones, PHS, pagers, etc.

[0002]

2. Description of the Related Art A time series signal represented by an acoustic signal is
A plurality of periodic signals are included as its constituent elements. Therefore, a method of analyzing what kind of periodic signal is included in a given time series signal has been known for a long time. For example, Fourier analysis is widely used as a method for analyzing frequency components included in a given time series signal.

By using such a time-series signal analysis method, it is possible to encode an acoustic signal. With the spread of computers, it has become easy to sample the analog sound signal that is the original sound at a predetermined sampling frequency, quantize the signal strength at each sampling, and capture it as digital data. If a method such as Fourier analysis is applied to the data and the frequency components included in the original sound signal are extracted, the original sound signal can be encoded by the code indicating each frequency component.

MIDI (Musical Instrume) was born from the idea of encoding musical instrument sounds by electronic musical instruments.
The nt Digital Interface) standard has also been actively used with the spread of personal computers. Code data according to this MIDI standard (hereinafter referred to as MID
Basically, the I data) is data that describes the operation of the musical instrument playing, such as which keyboard key of the musical instrument was played and with what strength. The MIDI data itself contains the actual sound. Waveform not included. Therefore, when reproducing the actual sound, the MI that stores the waveform of the instrument sound is stored.
Although a DI sound source is required separately, its high coding efficiency has been attracting attention, and the MIDI coding and decoding technology is currently used for musical instrument performance, musical instrument practice, composition, etc. using a personal computer. It is widely adopted in software that does.

Therefore, a time-series signal typified by an acoustic signal is analyzed by a predetermined method to extract a periodic signal which is a constituent element thereof, and the extracted periodic signal is M
Proposals have been made to encode using IDI data. For example, JP-A-10-247099, JP-A-11-73199, and JP-A-11-73200.
JP, JP-A-11-95753, JP, 2000
-99009, JP 2000-99092 A, JP 2000-99093 A, JP 2000-
261322, JP 2001-5450 A,
Japanese Unexamined Patent Application Publication No. 2001-148633 proposes various methods capable of analyzing a frequency as a constituent element of an arbitrary time series signal and creating MIDI data from the analysis result.

[0006]

According to the invention described in the above publication, the frequency analysis accuracy for time series signals and the audio signal encoding accuracy are greatly improved, and the case where the zero cross method for directly measuring the signal waveform is used. Higher accuracy can be obtained. However, in the above invention using the method of generalized harmonic analysis, a unit section that is an analysis unit is set,
Since it overlaps with the adjacent unit section, it has a problem in time resolution as compared with the zero-cross method. In the method using generalized harmonic analysis, as a method of improving the time resolution, there are a method of shortening the length of a unit section which is a unit of analysis, a method of shortening the feed length of the unit section, and a method of using a window function. However, there are limits.

When the method of shortening the length of the unit section is used, the time resolution is improved, but there is a problem that the frequency resolution, which is important as coding accuracy, is lowered. In addition, the frequency component whose period is longer than the unit section cannot be detected, and the detectable frequency range becomes narrow. When the method of shortening the feed length of the unit section is used, the frequency resolution does not change, but the overlap between unit sections becomes large, so
The improvement in time resolution is small and the amount of calculation is large. When the method using the window function is used, the time resolution can be slightly improved by devising the window shape, but the frequency resolution may be lowered due to the window shape, and a remarkable improvement in time resolution cannot be expected.

In view of the above points, the present invention makes it possible to improve the frequency resolution without lowering the frequency resolution and further increasing the time resolution without increasing the calculation load, and the audio signal encoding method. The challenge is to provide.

[0009]

In order to solve the above problems, the present invention provides a frequency analysis method for separating a signal component corresponding to a predetermined standard frequency from a time series signal. A step of setting unit sections of length so that each unit section overlaps with each other by a predetermined length, and a unit immediately before the target unit section to be analyzed overlaps with the target unit section by a predetermined length. A section signal correction step of creating a modified section signal by replacing a signal in an overlapping section with a section with a final residual signal obtained in the immediately preceding unit section, and using a generalized harmonic analysis for the modified section signal According to the generalized harmony, there is a spectrum calculation step for calculating a set of frequencies and corresponding intensities in the target unit section, and the target unit section is a new immediately preceding unit section. Using the analysis to subtract the plurality of contained signals from the corrected section signal as the final residual signal, the section signal correction step and the spectrum calculation step are performed for all set units. It is characterized in that a set of frequencies and intensities is calculated over the entire section by performing the processing for each section.

According to the present invention, in the analysis of the unit section to be analyzed, regarding the signal of the portion overlapping with the immediately preceding unit section, the final residual signal obtained by subtracting the contained signal component from the section signal of the immediately preceding unit section. Since the section signal is corrected by replacing the section signal with, the analysis is performed on the corrected section signal. Therefore, it is possible to perform the analysis while suppressing the influence of the signal component of the overlapping section.

[0011]

BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of the present invention will be described below in detail with reference to the drawings.

(1. Basic Principle) First, the basic principle of the frequency analysis method and the acoustic signal coding method according to the present invention will be described. Since this basic principle is disclosed in each of the above-mentioned publications, only its outline will be briefly described here.

As shown in FIG. 1A, it is assumed that an analog acoustic signal is given as a time series signal. In the example of FIG. 1, the horizontal axis represents time t, and the vertical axis represents amplitude (intensity) to show this acoustic signal. Here, first, a process of taking in the analog acoustic signal as digital acoustic data is performed. This uses the conventional general PCM method,
The analog acoustic signal may be sampled at a predetermined sampling frequency, and the amplitude may be converted into digital data by using a predetermined number of quantization bits. here,
For convenience of explanation, the waveform of the acoustic data digitized by the PCM method is also shown as the same waveform as the analog acoustic signal of FIG.

Then, a plurality of unit sections are set on the time axis of the acoustic signal to be analyzed. In the example shown in FIG. 1A, six times t1 to t6 are equally spaced on the time axis t.
Is defined, and five unit sections d1 to d5 whose start point and end point are the respective time points are set. In the example of FIG. 1, all unit sections having the same section length are set without overlapping on the time axis, but the section setting is performed so that adjacent unit sections partially overlap on the time axis. It doesn't matter. Particularly, in the specific processing of the present invention described later, the unit sections are set to overlap each other.

When the unit section is set in this way, a representative frequency is selected for each acoustic signal (hereinafter referred to as section signal) for each unit section. Each section signal usually contains various frequency components,
For example, a frequency component having a large intensity ratio of the components may be selected as the representative frequency. Here, the representative frequency is generally a so-called fundamental frequency, but a harmonic frequency such as a formant frequency of voice or a peak frequency of a noise sound source may be treated as a representative frequency. Although only one representative frequency may be selected, more accurate encoding becomes possible if a plurality of representative frequencies are selected depending on the acoustic signal. FIG. 1B shows an example in which three representative frequencies are selected for each unit section and one representative frequency is encoded as one representative code (in the figure, it is shown as a note for convenience). Has been done. Here, three tracks T1 for accommodating a representative code (note) are provided.
T2 and T3 are provided so that the three representative codes selected for each unit section are accommodated in different tracks.

For example, the representative codes n (d1,1), n (d1,2), n (d1, selected for the unit section d1.
3) are housed in the tracks T1, T2, T3, respectively. Here, each code n (d1,1), n (d1,
2) and n (d1,3) are codes indicating note numbers in the MIDI code. The note number in the MIDI code takes 128 values from 0 to 127, and each indicates one key on the keyboard of the piano. Specifically, for example, the representative frequency is 440 Hz
Is selected, the frequency is note number n = 6
Since it corresponds to 9 (corresponding to "Ra sound (A3 sound)" at the center of the keyboard of the piano), n = 69 is selected as the representative code. However, FIG. 1B is a conceptual diagram showing the representative code obtained by the above-described method in the form of a musical note, and in fact, each musical note is also provided with data relating to its strength. For example, the track T1 includes data indicating pitches of note numbers n (d1,1), n (d2,1), ..., E (d1,1), e (d
2, 1) ... The data indicating the intensity will be stored. The data indicating this intensity is determined by the degree to which the component of each representative frequency is included in the original section signal. Specifically, the data indicating the intensity is determined based on the correlation value of the section signal of the periodic function having each representative frequency. In addition, FIG.
In the conceptual diagram shown in, the position of each note on the time axis is shown by the horizontal position of the note. Actually, however, data indicating the position on this time axis as a numerical value is shown exactly. Will be added to each note.

The MIDI format does not necessarily have to be adopted as the format for encoding the audio signal, but since the MIDI format is the most popular as this type of encoding format, the MIDI format code data is practically used. Is preferably used. In the MIDI format, “note on” data or “note off” data exists with “delta time” data interposed. The "Note On" data is
The "note-off" data is data for instructing the start of playing a specific sound by designating a specific note number N and velocity V. The "note-off" data is a data of a specific note designated by a specific note number N and velocity V. This is data for instructing the end of performance. The "delta time" data is data indicating a predetermined time interval. Velocity V is a parameter indicating, for example, the speed at which the piano keyboard is pushed down (velocity at note-on) and the speed at which the finger is released from the keyboard (velocity at note-off), and operation to start playing a specific sound. Alternatively, it indicates the strength of the performance ending operation.

In the above method, the i-th unit section di
About J note numbers n (d
i, 1), n (di, 2), ..., N (di, J) are obtained for each of these intensities e (di, 1), e
(Di, 2), ..., E (di, J) are obtained. Therefore, the code data in the MIDI format can be created by the following method. First, as the note number N described in the “note on” data or the “note off” data, the obtained note number n (d
i, 1), n (di, 2), ..., N (di, J) may be used as they are. On the other hand, as the velocity V described in the “note-on” data or the “note-off” data, the obtained intensities e (di, 1), e (d
i, 2), ..., E (di, J) may be standardized by a predetermined method. The “delta time” data may be set according to the length of each unit section.

(2. Specific Method for Obtaining Correlation with Periodic Function) In the method based on the above-mentioned basic principle, one or a plurality of representative frequencies are selected for the section signal and have the representative frequency. The section signal is represented by the periodic signal. Here, the selected representative frequency is literally a frequency representing the signal component in the unit section. As a specific method of selecting the representative frequency, there are a method using a short-time Fourier transform and a method using a generalized harmonic analysis method, as described later. In either method, the basic idea is the same, and a plurality of periodic functions with different frequencies are prepared in advance, and a periodic function having a high correlation with the section signal in the unit section is selected from among the plurality of periodic functions. Is found and the frequency of this highly correlated periodic function is selected as the representative frequency. That is, when the representative frequency is selected, the calculation for obtaining the correlation between the plurality of periodic functions prepared in advance and the section signal in the unit section is performed. Therefore, here, a specific method for obtaining the correlation with the periodic function will be described.

It is assumed that a trigonometric function as shown in FIG. 2 is prepared as a plurality of periodic functions. These trigonometric functions are composed of a pair of a sine function and a cosine function having the same frequency, and 128 standard frequencies f (0) to
For each of f (127), a pair of sine and cosine functions will be defined. Here, a pair of functions including a sine function and a cosine function having the same frequency will be defined as a periodic function for the frequency. That is, the periodic function for a specific frequency is composed of a pair of sine function and cosine function. Thus, the reason why the periodic function is defined by a pair of sine function and cosine function is to consider that the correlation value is influenced by the phase when the correlation value of the periodic function with respect to the signal is obtained. The variables F and k in each trigonometric function shown in FIG. 2 are variables corresponding to the sampling frequency F and the sample number k for the interval signal X. For example, a sine wave for the frequency f (0) is represented by sin (2πf (0) k / F), and given an arbitrary sample number k, the same time position as the kth sample forming the interval signal is given. The amplitude value of the periodic function at is obtained.

Here, 128 standard frequencies f
An example in which (0) to f (127) are defined by equations shown in FIG. 3 will be shown. That is, the nth (0 ≦ n ≦ 1
The standard frequency f (n) of 27) is shown below [Formula 1].
Will be defined in.

[Formula 1] f (n) = 440 × 2 ^{γ (n)} γ (n) = (n−69) / 12

Defining the standard frequency by such a formula is convenient when finally performing coding using MIDI data. This is because there are 128 standard frequencies f (0) to f (12) set by such a definition.
7) is to take frequency values forming a geometric series, and M
This is because the frequency corresponds to the note number used in the IDI data. Therefore, 128 shown in FIG.
The standard frequencies f (0) to f (127) are the frequencies set at equal intervals (semitone unit in MIDI) on the frequency axis shown by the logarithmic scale.

(2.1. Method of Short-Time Fourier Transform) Next, a concrete description will be given of how to obtain the correlation of each periodic function with respect to a section signal of an arbitrary section. For example, as shown in FIG. 4, it is assumed that the section signal X is given for a certain unit section d. Here, it is assumed that the unit section d having the section length L is sampled at the sampling frequency F and w sample values are obtained in total, and the sample number is 0, as shown in the figure. 1,
2, 3, ..., K, ..., W-2, w-1 (the w-th sample indicated by a white circle is a sample included at the beginning of the next unit section adjacent to the right. ). In this case, for any sample number k, the amplitude value X (k) is given as digital data. In the short-time Fourier transform, it is usual to multiply X (k) by a window function W (k) such that the center weight is close to 1 and the weights at both ends are close to 0 for each sample. That is, the following correlation calculation is performed by treating X (k) × W (k) as X (k), and a cosine wave-shaped Hamming window is generally used as the shape of the window function. Here, w is also described as a constant in the following description, but it is generally changed according to the value of n, and the maximum F / f (n) within the range not exceeding the section length L is obtained. It is desirable to set the value to an integral multiple.

The principle of obtaining the correlation value with the sine function Rn having the nth standard frequency f (n) for such a section signal X will be described. The correlation value A (n) between the two can be defined by the first arithmetic expression in FIG. here,
X (k) is the amplitude value of the sample number k in the interval signal X, as shown in FIG. 4, and sin (2πf (n) k
/ F) is the amplitude value of the sine function Rn at the same position on the time axis. This first arithmetic expression can be said to be an expression for obtaining the inner product of the amplitude value of the interval signal X and the amplitude vector of the sine function Rn for each dimension of all sample numbers k = 0 to w−1 in the unit interval d. .

Similarly, the second arithmetic expression in FIG. 5 is an expression for obtaining the correlation value between the interval signal X and the cosine function having the nth standard frequency f (n). Is B (n)
Given in. Note that both the first arithmetic expression for obtaining the correlation value A (n) and the second arithmetic expression for obtaining the correlation value B (n) are finally multiplied by 2 / w. Is for normalizing the correlation value, and w is n as described above.
This coefficient is also a variable dependent on n, since it is generally changed depending on.

The effective value of the correlation between the interval signal X and the standard periodic function having the standard frequency f (n) is the correlation value A (n) with the sine function as shown in the third arithmetic expression of FIG. It can be represented by the square root sum of squares value E (n) with the correlation value B (n) with the cosine function. If the frequency of the standard periodic function with a large correlation effective value is selected as the representative frequency, the section signal X can be encoded using this representative frequency.

That is, one or a plurality of standard frequencies whose correlation value E (n) is greater than a predetermined standard may be selected as the representative frequency. The selection condition that “the correlation value E (n) is greater than or equal to a predetermined reference” is, for example, a threshold that is set in advance and the correlation value E (n) exceeds the threshold. Frequency f (n)
May be set as a representative frequency, but an absolute selection condition may be set. For example, relative selection conditions such as selecting up to the Qth in the order of the magnitude of the correlation value E (n). May be set.

(2.2. Generalized Harmonic Analysis Method) Here,
A method of generalized harmonic analysis useful for encoding an acoustic signal according to the present invention will be described. As already described, when the acoustic signal is encoded, some representative frequencies having a high correlation value are selected for the section signals in each unit section. Generalized harmonic analysis is a method that enables selection of representative frequencies with higher accuracy, and its basic principle is as follows.

It is assumed that the signal S (j) exists in the unit section d as shown in FIG. 6 (a). here,
As will be described later, j is a parameter for iterative processing (j = 1 to J). First, for this signal S (j), correlation values for all 128 periodic functions as shown in FIG. 2 are obtained. Then, the frequency of one periodic function for which the maximum correlation value is obtained is selected as the representative frequency,
A periodic function having the representative frequency is extracted as an element function. Then, the inclusion signal G as shown in FIG.
Define (j). The content signal G (j) is added to the extracted element function as its amplitude and the signal S of the element function.
It is a signal obtained by multiplying the correlation value for (j). For example, as shown in FIG. 2 as a periodic function, when a pair of sine function and cosine function are used and the frequency f (n) is selected as the representative frequency, the sine function A (n) having the amplitude A (n) is used. ) Sin (2πf (n) k / F) and a cosine function B (n) cos (2πf) having an amplitude B (n)
(N) k / F) is the content signal G (j)
That is, only one function is shown in FIG. 6B for convenience of illustration. Where A (n), B
Since (n) is the normalized correlation value obtained by the equation of FIG. 5, the contained signal G (j) eventually has the frequency f (n) contained in the signal S (j). It can be called a signal component.

When the content signal G (j) is obtained in this way, the difference signal S (j + 1) is obtained by subtracting the content signal G (j) from the signal S (j). FIG.6 (c) shows
The difference signal S (j + 1) thus obtained is shown. This difference signal S (j + 1) is the original signal S
It can be said that the signal is composed of the remaining signal components obtained by removing the signal component having the frequency f (n) from (j). Therefore, by increasing the parameter j by 1, the difference signal S (j + 1) is treated as a new signal S (j), and the same processing is performed for the parameter j from j = 1 to 1.
If you repeat J times while incrementing by 1 to J, J
Individual representative frequencies can be selected.

The J contained signals G (1) to G (J) output as a result of such a correlation calculation are signals which are constituent elements of the original section signal X, and are the original section signals. When X is coded, information indicating the frequency and amplitude (intensity) of these J contained signals may be used as code data. Although it has been described that J is the number of representative frequencies, it may be the same as the number of standard frequencies f (n), that is, J = 128, and it is customary to do so for the purpose of obtaining the frequency spectrum. Is.

By the above processing, a frequency group, which is a set of intensity values for each frequency, is obtained for each unit section. When a predetermined number of frequency groups are selected in this way, "information indicating the pitch of the sound" corresponding to each frequency of this frequency group, "sound intensity corresponding to the signal strength of each selected frequency""Informationindicating","information indicating the sound production start time" corresponding to the start point of the unit section, and "information indicating sound production end time" corresponding to the start point of the unit section subsequent to the unit section. If a predetermined number of code data including information is created, the section signal X in the unit section is generated.
Can be encoded by a predetermined number of encoded data.
If MIDI data is created as code data, note number is used as "information indicating pitch of tone", velocity is used as "information indicating intensity of tone", and "start time of sound generation" The note-on time may be used as the “information indicating” and the note-off time may be used as the “information indicating the sound production end time”.

(3. Theoretical Basis of the Frequency Analysis Method According to the Present Invention) The frequency analysis method and the acoustic signal encoding method according to the present invention will be described below. First, the theory of the frequency analysis method according to the present invention will be described. First, consider extraction of frequency components of a time-series signal at a certain time t. If a small time δ is used for realistic calculation, the time series signal g (t) at time t can be approximated by the following [Formula 2].

[Formula 2] g (t) ≅Σ _δ [Σ _n {A _n (δ) sin (2πf _n t) + B _n (δ)
cos (2πf _n t)}]

In the above [Formula 2], Σ_δIs δ = −τ
To δ = + τ, and Σ _nIs from n = 0 to n =
The total sum up to 127 is shown. Note that this n is the above note
It corresponds to the number. Furthermore, in the above [Formula 2],
Kick A_n(δ), B_n(δ) can be expressed by the following [Equation 3].
It

[0037] [Equation 3] _{A n (δ) = (2} / T n) Σ t {g (t + δ) sin (2πf n t)} B n (δ) = (2 / T n) Σ t {g ( t + δ) cos (2πf _n t)}

In the above [Formula 3], T _n is the period 1 / f
It is the maximum value that is an integer multiple of _n and does not exceed the unit interval length.
Σ _t represents the total sum from t = 0 to t = T _n −1. Now,
What is desired here is g (t), which can be approximated by the right side of [Equation 2] when δ = 0. this is,
It is obtained by calculating A _n (0) and B _n (0) using [Equation 3]. When the periodic function is multiplied by the window function W (t) that increases the weight behind the unit section, the range of n can be approximated so that only the immediately preceding unit section −τ is considered. That is, the above [Formula 2] can be transformed into the following [Formula 4].

[Formula 4] g (t) ≈Σ _n {A _n (0) W (t) sin (2πf _n t) + B _n (0)
W (t) cos (2πf _n t)} ＋ Σ _n {A _n (-τ) W (t) sin (2πf _n
t) ＋ B _n (-τ) W (t) cos (2πf _n t)}

Since the first term on the right side of the above [Equation 4] is a parameter to be calculated, if the first term on the right side is g ′ (t), then [Equation 4] becomes g ′ (t) = g (t). −Σ _n {A _n (-τ) W (t) sin (2πf _n t) +
B _n (−τ) W (t) cos (2πf _n t)}, and by adding ± g (t−τ) to the left side, it is transformed as shown in [Equation 5] below.

[Formula 5] g ′ (t) = g (t) −g (t−τ) + g (t−τ) −Σ _n {A
_n (-τ) W (t) sin (2πf _n t) ＋ B _n (-τ) W (t) cos (2πf _n
t)}

A unit section whose end time is t is a unit section d.
If will be referred to as _t, the portion of the "g (t) -g (t- τ) " in the right side of the [Equation 5], in the unit interval d _t-tau from a signal g (t) in the unit interval d _t The signal is excluded, and "g (t
−τ) −Σ _n {A _n (−τ) W (t) sin (2πf _n t) + B _n (−τ) W
(t) cos (2πf _n t)} ”is a difference obtained by removing the contained signal component from the original signal in the unit section d _t− τ. This is specifically shown in FIG. 7. After all, the modifications shown in [Equation 2] to [Equation 5] have the unit interval d _{t when the} frequency analysis is performed on the unit interval d _t.
It is shown that the original signal g (t) in FIG. 7 should not be analyzed but g ′ (t) shown at the bottom of FIG. 7 should be analyzed. Therefore, it is possible to perform a highly accurate analysis without being influenced by the previous section by performing g '(t) in which the signal value in the previous section is small. In addition,
From the calculation, A _n (0) and B _n (0) are obtained by the following [Equation 6].

[Equation 6] A _n (0) = (S _n / T _n ) Σ _t {g ′ (t) W (t) sin (2πf _n t)} B _n (0) = (C _n / T _n ) Σ _t {g '(t) W (t) cos (2πf _n t)}

S _n and C _n in the above [Formula 6] are correction coefficients depending on the window function, and take a value of 2.0 if W (t) is a flat rectangle. Specifically, S _n and C _n are represented by the following [Formula 7].

[Formula 7] S _n = 1 / [Σ _t {W (t) sin (2πf _n t)} ² ] C _n = 1 / [Σ _t {W (t) cos (2πf _n t)} ² ]

(3.1. Specific processing of frequency analysis method according to the present invention) Based on the above theory, the frequency analysis method of the present invention will be described below with reference to the flowchart shown in FIG. It demonstrates using. First, a unit section is set for a time-series acoustic signal, and a section signal is extracted (step S1). This is as explained using FIG. 1 in the above (basic principle) section.
A unit section having a predetermined length is set and extraction is performed. However, in the present invention, as shown in FIG. 7, the unit sections are set to overlap each other by a predetermined amount.

Next, the section signal thus extracted is corrected (step S2). This is performed by replacing the overlapping portion of the immediately preceding unit section and the unit section to be analyzed with the final residual signal analyzed in the immediately preceding unit section, as described in the section (3. Theoretical basis) above. However, the process of step S2 is not performed for the first unit section in which the immediately preceding unit section does not exist. For the first unit section, step S3 is performed on the extracted section signal.
Subsequent processing will be performed.

Next, the correlation between the corrected section signal and the harmonic signal prepared in advance is calculated (step S).
3). This harmonic signal is the same as the periodic function described in the above section (Specific method for obtaining correlation with periodic function). As the frequency of the periodic function, 128 standard frequencies corresponding to the note numbers may be set as shown in the example of FIG. 2, but in the present embodiment, intervals of geometric series are provided between the standard frequencies. , 12 fine frequencies are set, and a total of 128 × 13 harmonic signals are prepared. For example, between the adjacent standard frequencies f (n) and f (n + 1), 12 of f (n + 1/13) to f (n + 12/13)
Individual fine frequencies will be set. In step S3, the correlation between the interval signal and 128 × 13 harmonic signals is obtained by the short-time Fourier transform method. As a result, 128 × 13 correlation values corresponding to the respective fine frequencies are obtained. This correlation value is stored in the fine correlation array prepared in advance.

Then, a fine frequency having a peak (hereinafter referred to as a peak frequency) is selected based on the calculated correlation value (step S4). Specifically, the calculated 12
Based on the 8 × 13 correlation values, the fine frequency whose self correlation value is larger than the correlation values of the fine frequencies on both sides adjacent to each other is selected.

Next, the correlation between the harmonic signal corresponding to each selected peak frequency and the corrected section signal is obtained by the generalized harmonic analysis method (step S5). Specifically, as described in the section (Generalized Harmonic Analysis Method) above, first, among the selected peak frequencies, the harmonic signal and the section signal S (corresponding to the one having the largest correlation value) are selected. 1) (=
X) is calculated. Subsequently, a signal obtained by multiplying the harmonic signal at this time by a correlation value calculated as its amplitude is set as a contained signal G (1), and this contained signal G (1) is subtracted from the section signal S (1). Thus, the difference signal S (2) is obtained. Next, of the peak frequencies selected in step S3, the correlation value with the interval signal X is 2
The harmonic signal with the second largest peak frequency and the difference signal S
The difference signal S (3) is obtained by obtaining the correlation with (2) and subtracting the contained signal G (2) from the section signal. In this way, the correlation values of all the selected peak frequencies are obtained by the method of generalized harmonic analysis. Here, when the predetermined number of frequency components are extracted, the remaining difference signal becomes the final residual signal.

Since the selected peak frequency corresponds to the fine frequency, it is necessary to restore the standard frequency corresponding to the note number at the time of encoding. Therefore, the calculated correlation value is set in the main correlation array as the correlation value corresponding to the standard frequency closest to the peak frequency. As a result, a plurality of sets of standard frequencies and their correlation values are obtained. By using this standard frequency as the note number and the correlation value as the velocity, encoded data of the MIDI standard is obtained (step S6). By performing the above processing for all unit sections, it is possible to code the entire acoustic signal.

(3.2. Two-step processing for determining extraction frequency component)
In the above steps S4 and S5, the peak frequency is selected only once, and the correlation value is calculated for each of the selected peak frequencies by the method of generalized harmonic analysis. The accuracy of conversion can be increased. Such a case will be described below. By repeating the process of subtracting the contained signal from the section signal X as in step S5, different section signals X ′ are obtained. The processing of steps S3 to S5 is repeated for this section signal X '.

As the fine frequencies correlated with the interval signal X ', those corresponding to the standard frequency whose correlation value is already stored in the main correlation array are excluded. For example, the standard frequency f
When the correlation value corresponding to (n) is set in the main correlation array, fine frequencies f (n-6 / 13) to f (n + 6)
/ 13) is excluded from the target of the correlation calculation with the section signal X ′. The correlation between the harmonic signal corresponding to the remaining fine frequency and the section signal X ′ is obtained by the short-time Fourier transform method, and the calculated correlation value is stored in the fine correlation array.

Subsequently, in step S4, the peak frequency is selected again based on the calculated correlation value, and in step S5, the correlation between each selected peak frequency and the section signal X'is obtained, and then calculated. The calculated correlation value is set in the main correlation array.

The processes of steps S3 to S5 can be repeated any number of times, but when the encoding process of the acoustic signal is performed, about twice is sufficient. This is because if performed twice, about 64 tones that can be simultaneously pronounced according to the MIDI standard can be extracted.

(3.3. Determination of Extraction Frequency Component in Same Standard Frequency) The determination of the standard frequency and the calculation of the correlation value in the step S5 correspond to the calculated correlation value for the standard frequency in the vicinity of the peak frequency. Decided as one. This is not a problem when only one peak frequency is included in one standard frequency range.
When there are a plurality of peak frequencies included in one standard frequency range, it becomes a problem which one is selected. Therefore, in the latter case, among the peak frequencies included in one standard frequency, only the one having the maximum correlation value by the short-time Fourier transform is selected as the peak frequency, and the section signal X is obtained by the generalized harmonic analysis method. Try to find the correlation of.

(3.4. Interval Update Width Determination Processing) The frequency component extraction processing and the extracted frequency component encoding as described above are performed for each unit interval, but the unit interval settings are fixed as described above. Rather, it can be changed according to the situation. At this time, how to set the length of one unit section is an important problem that greatly affects the analysis result. That is, the shorter the section length is set, the more the time resolution is improved, which is convenient for analyzing the frequency change of a voice signal (vocal signal) and the like. The problem is that the accuracy is reduced. So conventionally,
A standard interval length is set in advance so that the number of samples (for example, 1024 samples) that seems to be most convenient for analysis is included in one unit interval, and based on this fixed interval length It was common to adopt a method of mechanically dividing the time axis.

However, in the method of mechanically setting a fixed-length unit section, a unique section is always set regardless of what kind of signal the time-series signal to be analyzed is. Therefore, discontinuity may occur at the boundary of the unit section. That is, a large difference may occur in the frequency analysis results of the adjacent unit sections. Generally, when the obtained frequency analysis result is used for observing it as a spectrum graph, even if there is a difference in the analysis result at the boundary of the unit section, it does not cause a big problem. However, when the coded data corresponding to the original acoustic signal is created by using the analysis result of the acoustic signal, the discontinuity at the boundary becomes a big problem. When an original audio signal is reproduced based on the data encoded by such a method, a sharp pitch change or a pitch skip occurs at the boundary of the unit section.

In order to avoid such discontinuity at the boundary, a method of partially overlapping the unit intervals on the time axis has been proposed, but in this case also, the overlapping amount is fixed. As a result, redundant processing is performed up to a portion that originally does not need to be duplicated, which causes a problem that unnecessary calculation processing is increased. In the present invention,
When analyzing a time series signal, at the boundary of the unit interval,
Since the unit continuity is set so that the continuity of the sound is maintained and the duplication of the unit spans is minimized, it will be described below.

Particularly in the present invention, the above step S3
It is characterized in that the unit section is set by using the correlation value obtained by the short-time Fourier transform in. Specifically, when the frequency component is extracted in a certain unit section, the value of the fine correlation array first obtained in step S3 is copied to the immediately preceding correlation array used for section determination. Then, the processes of steps S2 to S4 are performed a predetermined number of times to
After determining the frequency component to be extracted in one unit section, the position of the next unit section is set. In the present invention, since the unit section has a fixed length, the unit section is determined when the start position of the unit section is determined. Therefore, in the present invention, the subsequent unit section is determined by determining the feed width from the start position of the previous unit section to the start position of the subsequent unit section.

First, the previously determined unit section,
From this, the degree of similarity of the unit section for determining the position is obtained (step S7). Specifically, for the peak frequency selection, the value of the fine correlation array obtained as a result of the short-time Fourier transform performed in step S2 is used to calculate by the following [Equation 8].

[Equation 8] SM = 100−100 × [Σ _l {V _dm−1 (l) −V _dm (l)} ² ] ^1/2 /
{Σ _l V _dm-1 (l) ² } ^1/2

In the above [Equation 8], V _dm (l) is the correlation value of the fine frequency k with respect to the section signal of the unit section d _m to be determined, and V _dm-1 (l) is the immediately preceding unit. It is a correlation value of the fine frequency k with respect to the section signal in the section d _{m- 1} . However, l does not indicate the frequency itself,
It is expressed by a value obtained by dividing the standard frequency corresponding to the note number n into 13, and takes values of l = -6 to 127 × 13 + 6. The larger the value of the similarity SM, the more similar the two section signals are.

Next, the value of this similarity SM and the threshold value are compared. When the similarity SM is smaller than the threshold value 2 (step S
8), halve the set feed width (step S
9). The provisional unit section is reset at the position where the feed width is halved (step S1), and the correlation between the section signal of the new provisional unit section and the harmonic signal is obtained (step S3). The processes of steps S1 to S3 and steps S7 to S9 are repeated until the similarity SM becomes equal to or greater than the threshold value 2. That is,
Until the threshold value becomes 2 or more, the range in which the subsequent unit section overlaps with the previous unit section is increased.

When the similarity SM exceeds the threshold 2, the similarity SM and the threshold 1 are compared. If the degree of similarity is less than the threshold value 1, the provisional unit section is determined as a unit section, and the process proceeds to peak position selection processing (step S4). When the similarity is equal to or more than the threshold value 1, the feed width is set to double (step S11), the provisional unit section is confirmed as a unit section, and the peak position selecting process is performed (step S4). Here, the doubled feed width is used as the feed width of the next unit section. After confirming the unit section,
The processes of steps S4 and S5 are performed in the same manner to determine the extraction frequency component. In addition, as a matter of course, the two threshold values satisfy the relationship of threshold value 1> threshold value 2.

When the extraction frequency for the unit section is determined, the position moved by the feed width set at this time is set as the start position (step S12), and the temporary unit section is set (step S1). At this time, normally, the preset standard feed width is used, and step S11
If the feed width is set to 2 by, the feed width is set to twice the standard feed width.

Here, the unit section to be set will be described using a concrete example shown in FIG. FIG. 9A is a diagram showing a time-series acoustic signal. As shown in FIG. 9B, the fixed-length unit section d1 is set from the head position of the acoustic signal, and the processes of steps S3 to S5 are performed to determine the extraction frequency. Next unit section d2
In setting S, in step S12, the standard feed width set in advance is sent afterward and provisionally set. In the present embodiment, the standard feed width is 1 / the fixed length L.
2 Therefore, L / 2 from the beginning of the unit section d1
The unit section d2 is tentatively set so that the position shifted by just the position becomes the head of the unit section d2. FIG. 9C shows the state of the unit section d2 temporarily set at this time.

Then, in step S2, the acoustic signal at the position of the temporarily set unit section d2 is extracted as a section signal and corrected by the final residual signal of the immediately preceding unit section, and the correlation with the corrected section signal is correlated with step S3. It is obtained by the method of. Then, in step S7, the degree of similarity shown in [Equation 2] is calculated. If the calculated similarity is less than the threshold value 2, that is, if the two section signals are not very similar, the process proceeds to step S9 and the feed width is halved. Here, since it is set to L / 2, it means that it is set to L / 4. In this case, step S1
Returning to step 1, the unit section d2 is tentatively set again so that the position displaced by L / 4 from the head of the unit section d1 becomes the head of the unit section d2. The state of the unit section d2 temporarily set at this time is shown in FIG. When the unit section d2 is temporarily set again, the processes of steps S2 and S3S7 are repeated again. Further, in step S8, the degree of similarity is threshold value 2
If it is less than, the feed width is L /
8, the process returns to step S1 and the unit section d2 is provisionally set again so that the position displaced by L / 8 from the start of the unit section d1 becomes the start of the unit section d2. The state of the unit section d2 temporarily set at this time is shown in FIG.

That is, as shown in FIGS. 9 (c) to 9 (e), the unit section d2 is maintained until the similarity between the section signals in the preceding unit section d1 and the succeeding unit section d2 becomes the threshold value 2 or more. The number of overlapping sections with the unit section d1 will be increased. However, if it is left as it is, the unit section d1 and the unit section d2 may approach each other endlessly. Therefore, if the minimum distance of the feed width is set in advance and the feed width becomes equal to or less than the minimum distance, the point Then, the unit section temporarily set is confirmed as a formal unit section, and the process proceeds to step S4.

On the other hand, if the similarity between the section signal of the unit section d1 and the section signal of the temporarily set unit section d2 is greater than or equal to the threshold value 2 and less than the threshold value 1, the temporarily set unit section d2 is set. After the confirmation, the processing from step S4 is performed. Further, even when the similarity becomes equal to or more than the threshold value 1, the provisionally set unit section d2 is determined and the processing from step S4 is performed, but in this case, the feed width is set to double. The processing is also performed in parallel. Then, after the extraction frequency component for the unit section d2 is determined, step S1
In 2, the position moved from the head of the unit section by the doubled feed width is set as the head position of the unit section d3.
For example, when the unit section d2 is determined at the position shown in FIG. 9C, the unit section d3 is provisionally set at the position shown in FIG. 9F.

(4. Modified Example of Frequency Analysis) Next, a modified example will be described. FIG. 10 is a flowchart showing a modified example of the frequency analysis method according to the present invention. First, the unit section is set (step S21). Again, in FIG.
The unit sections are set so that they overlap each other as shown in FIG. Here, the unit section to be analyzed is referred to as a target unit section, and the unit section immediately before the target unit section and the unit section immediately after the target unit section are respectively referred to as an immediately preceding unit section and an immediately following unit section.

Then, the correlation value corresponding to the standard frequency is calculated for each of the target unit section, the immediately preceding unit section, and the immediately following unit section by the short-time Fourier transform method (step S22). The calculation of the correlation value corresponding to each standard frequency by the short-time Fourier transform is the same as the process described in the above section (2.1. Method of short-time Fourier transform), and thus detailed description thereof will be omitted. Here, since 128 standard periodic functions as shown in FIG. 2 are used, a total of 128 × 3 sets of 128 standard frequencies and their correlation values are set for each of the target unit section, the immediately preceding unit section, and the immediately following unit section.
You will get one.

Next, the frequency component to be extracted is determined using the method of generalized harmonic analysis (step S23). In the conventional generalized harmonic analysis method, in the target unit section to be analyzed, the section signal is updated by using the contained signal synthesized at the corresponding standard frequency in descending order of the correlation value of the result of the short-time Fourier transform. Along with this, the updated correlation value was used as the strength component of the standard frequency, but in the present invention, in any of the immediately preceding unit section, the target unit section, and the immediately following unit section, the correlation value as a result of the short-time Fourier transform is in descending order,
The section signal is updated using the contained signal that is synthesized at the corresponding standard frequency, and the correlation value is used as the intensity component of the standard frequency only when the unit section of interest is the target unit section. Specifically, among the standard frequencies of the immediately preceding unit section, the target unit section, and the immediately following unit section, attention is paid to the standard frequency having the maximum correlation value as a result of the analysis by the short-time Fourier transform. In other words, attention is paid to the standard frequency having the maximum correlation value from the 128 × 3 correlation values. Then, the correlation value is recalculated in a predetermined unit section, and the content signal synthesized with the correlation value updated with the standard frequency of interest is subtracted from the section signal in the target unit section of the standard frequency of interest to obtain a difference signal. Is obtained and the section signal is updated. At this time, if the unit section of interest is the immediately preceding unit section or the immediately following unit section, the phase of the contained signal is shifted by an amount corresponding to the time difference from the target unit section and subtracted from the section signal within the range of the target unit section. To update the section signal. If the unit section of interest is the target unit section, the updated correlation value is set as the intensity component of the standard frequency of interest. On the other hand, when the unit section of interest is the immediately preceding unit section or the immediately following unit section, the updated correlation value is only used for the processing of updating the section signal as described above.

For example, assuming that the correlation value corresponding to each standard frequency by the short-time Fourier transform of the immediately preceding unit section / target unit section / immediately following unit section is a value as shown in FIG. 11C, each correlation value The result of updating in sequence is shown in FIG. In addition, in FIG. 11C, the standard frequency is
Corresponding note numbers are shown, and the correlation value is shown as a numerical value according to a predetermined standard. Figure 11
Among the correlation values in (d), the values without brackets are adopted for the intensity component, and the bracketed values are used only for updating the interval signal. In the example of FIG. 11C, the correlation value of the immediately preceding unit section corresponding to the note number 98 is “92”.
Since it is the maximum among all the correlation values, the correlation value is recalculated in the immediately preceding unit section. As a result, as shown in FIG. 11D, the value becomes “92” (in the first time, the interval signal remains unchanged, so the correlation value becomes the same as before the update). And
The standard frequency corresponding to the note number 98 and the contained signal component based on the correlation value “92” of the immediately preceding unit section are subtracted from the section signal of the target unit section to obtain a difference signal, and this signal is set as a new section signal. However, the correlation value "92"
Is not an intensity component because it is not the target unit section. Next, when the maximum correlation value is searched from the remaining correlation values, the correlation value "86" in the immediately following unit section corresponding to the note number 98 is obtained, and the result of recalculating the correlation is "60". Therefore, the contained signal component based on the standard frequency corresponding to the note number 98 and the correlation value “60” of the immediately subsequent unit section is subtracted from the section signal of the target unit section to update the section signal. Similarly, since the correlation value “60” is not the target unit section, it does not become an intensity component. Then, when the maximum correlation value is searched from the remaining correlation values, the note number 99
Correlation value "78" in the unit section immediately after that corresponds to, and the result of recalculating the correlation becomes "70". Therefore, the contained signal component based on the standard frequency corresponding to the note number 99 and the correlation value “70” of the immediately subsequent unit section is subtracted from the section signal of the target unit section to update the section signal. Similarly,
The correlation value “70” is not a target component section and thus does not become an intensity component. Furthermore, when the maximum correlation value is searched from the remaining correlation values, the correlation value of the target unit section corresponding to the note number 97 becomes "70", and the recalculated result becomes "60". Therefore, the section signal is updated by subtracting the contained signal component based on the standard frequency corresponding to the note number 97 and the correlation value “60” of the target unit section from the section signal of the target unit section. At this time, since the correlation value “60” is the target unit section, it becomes the intensity component of the standard frequency corresponding to the note number 97. Repeat the same process,
The contained signal components based on 128 × 3 correlation values are deleted from the interval signal in the order of increasing initial correlation values, and the intensity components based on 128 correlation values are
It is decided as shown in the column of the correlation value of the target unit section of (d). According to the result of the short-time Fourier transform shown in FIG. 11C, noteworthy is that the note number having the maximum correlation value is 98, but FIG.
In the final result shown by, the intensity value of the note number 97 becomes maximum, and it is possible to obtain a highly accurate extracted component in which the influence of the signal in the immediately preceding and following unit sections is deleted.

MIDI code data is obtained by converting the extracted frequency components into note numbers, velocities, and delta times in the same manner as the above basic principle (step S24).

Also in the above modification, since each unit section has the correlation value by the short-time Fourier transform,
It is also possible to change the set position of the unit section by performing the processing shown in steps S7 to S12 of FIG. 8 on these.

Although the preferred embodiment of the present invention has been described above, it goes without saying that the frequency analysis method and the acoustic signal encoding method are executed by a computer or the like.
Specifically, a program for executing the steps shown in the flowcharts of FIGS. 8 and 10 in the above procedure is installed in the computer. Then, after time-series signals such as acoustic signals are digitized by the PCM method or the like, they are loaded into a computer, and steps S1 to S12 are performed.
Alternatively, after performing the processing of steps S21 to S24, the extracted frequency component or code data in the MIDI format or the like is output from the computer as digital data. For example, in the case of MIDI data, the output code data is reproduced as sound using a MIDI sequencer and MIDI sound source.

[0078]

As described above, according to the present invention,
As a frequency analysis method for separating a signal component corresponding to a predetermined standard frequency from a time series signal, a unit section of a predetermined length is added to the time series signal so that each unit section overlaps each other by a predetermined length. , In the setting unit and in the target unit section to be analyzed, the signal in the overlapping section of the immediately preceding unit section that overlaps the target unit section by a predetermined length is replaced with the final residual signal obtained in the immediately preceding unit section. And a spectrum calculation step for calculating a set of frequencies and corresponding intensities in the target unit section by using the generalized harmonic analysis for the modified section signal. , This target unit section is set as a new immediately preceding unit section, and finally obtained by subtracting multiple contained signals from the corrected section signal using generalized harmonic analysis. Signal as the final residual signal, the processing of the interval signal modification step and the spectrum calculation step, by performing for all units sections set, because to calculate the set of frequencies and intensity over the entire interval,
It is possible to perform an analysis that suppresses the influence of signal components in the overlapping section.

[Brief description of drawings]

FIG. 1 is a diagram showing a basic principle of a frequency analysis method and an acoustic signal coding method according to the present invention.

FIG. 2 is a diagram showing an example of a periodic function used in the present invention.

FIG. 3 is a diagram showing a relational expression between a frequency of each periodic function shown in FIG. 2 and a MIDI note number n.

FIG. 4 is a diagram showing a method of calculating a correlation between a signal to be analyzed and a periodic signal.

5 is a diagram showing a calculation formula for performing the correlation calculation shown in FIG.

FIG. 6 is a diagram showing a basic method of generalized harmonic analysis.

FIG. 7 is a diagram showing how a section signal is modified in a unit section.

FIG. 8 is a flowchart of a frequency analysis method and an acoustic signal coding method according to the present invention.

FIG. 9 is a diagram showing a procedure for setting a unit section.

FIG. 10 is a flowchart showing an outline of a modified example of the frequency analysis method and the acoustic signal coding method according to the present invention.

FIG. 11 is a diagram for explaining a modified example of the present invention.

[Explanation of symbols] d: Unit section G: content signal L: Unit section length S ・ ・ ・ Differential signal X: Initial section signal

Claims (3)

[Claims] 1. A frequency analysis method for separating a signal component corresponding to a predetermined standard frequency from a time-series signal, wherein the time-series signal has unit sections of a predetermined length, and each unit section has a predetermined section. And the signal in the overlapping part of the target unit section that is the analysis target and the immediately preceding unit section that overlaps the target unit section by a predetermined length, in the immediately preceding unit section. A section signal correction step of creating a modified section signal replaced with the final residual signal obtained, and a frequency and corresponding intensity value in the target unit section by using a generalized harmonic analysis for the modified section signal. And a spectrum calculation step of calculating a set of a., A target unit section as a new immediately preceding unit section, and using generalized harmonic analysis, a plurality of sections are included from the corrected section signal. The signal finally obtained by subtracting the existing signal is used as the final residual signal, and the processing of the section signal correction step and the spectrum calculation step is performed for all the set unit sections, thereby providing the frequency over the entire section. And a frequency analysis method comprising extracting a set of intensity values. 2. A frequency analysis method for separating a signal component corresponding to a predetermined standard frequency from a time-series signal, wherein the time-series signal has unit sections of a predetermined length, and each unit section has a predetermined section. The short-time Fourier transform is applied to the setting step, the target unit section to be analyzed, the immediately preceding unit section that is the immediately preceding unit section, and the immediately following unit section that is the immediately following unit section so that they overlap each other by the length of A correlation calculation step of calculating a correlation value corresponding to each standard frequency by using, of the correlation values in the immediately preceding unit section, the target unit section, the immediately after unit section, the maximum correlation value is selected,
Along with recalculating that value, the content signal obtained by the correlation value recalculated with the standard frequency is subtracted from the section signal of the target unit section, and the difference signal obtained by subtracting is a new section signal, When the selected correlation value is based on the target unit section, the recalculated correlation value is determined as the intensity value corresponding to the standard frequency, and the unit section of the standard frequency corresponding to the selected correlation value is set to the unit section. A corresponding spectrum is excluded from the subsequent selection targets, and a spectrum calculation step in which these processes are repeated until a predetermined number of sets of the standard frequency and intensity value is determined, and, after the spectrum calculation step, the target unit The section is set as a new immediately preceding unit section, and the immediately following unit section is set as a new target unit section, and the correlation calculation step and the spectrum calculation step are performed for all unit sections. A frequency analysis method, characterized in that a set of a plurality of standard frequencies and intensity values is determined for all unit intervals by repeatedly performing the above. 3. The time-series signal is an acoustic signal, and pitch information of a sound corresponding to the standard frequency selected in the spectrum calculating step, sound intensity information corresponding to an intensity value, and each unit. An encoding step of generating coded data by performing a predetermined conversion on four pieces of information consisting of a sounding start time of a sound corresponding to a start point of a section and a sounding end time of a sound corresponding to a start point of a subsequent unit section. The audio signal encoding method according to claim 1 or 2, further comprising: