JP4156269B2 - Frequency analysis method for time series signal and encoding method for acoustic signal - Google Patents

Description

[0001]
[Industrial application fields]
The present invention includes broadcast media (radio, television), communication media (CS video / audio distribution, Internet music distribution, communication karaoke), package media (CD, MD, cassette, video, LD, CD-ROM, game cassette, mobile phone). Production of various audio contents to be provided on solid-state memory media for music players), music performance recording signals to publish music scores, MIDI data for online karaoke distribution, automatic performance data for electronic musical instruments with performance guide functions, mobile phones, PHS, The present invention relates to an automatic music recording technique for automatically generating incoming melody data such as a pager.
[0002]
[Prior art]
A time-series signal represented by an acoustic signal includes a plurality of periodic signals as its constituent elements. For this reason, a method for analyzing what kind of periodic signal is included in a given time-series signal has been known for a long time. For example, Fourier analysis is widely used as a method for analyzing frequency components included in a given time series signal.
[0003]
By using such a time-series signal frequency analysis method, it is possible to encode an acoustic signal. With the spread of computers, it has become easy to sample an analog audio signal as the original sound at a predetermined sampling frequency, quantize the signal intensity at each sampling, and capture it as digital data. If a method such as Fourier analysis is applied to the data and the frequency components included in the original sound signal are extracted, the original sound signal can be encoded by a code indicating each frequency component.
[0004]
In addition, the MIDI (Musical Instrument Digital Interface) standard born from the idea of encoding musical instrument sounds by electronic musical instruments has come to be actively used with the spread of personal computers. The code data according to the MIDI standard (hereinafter referred to as MIDI data) is basically data that describes the operation of the musical instrument performance such as which keyboard key of the instrument is played with what strength. The data itself does not include the actual sound waveform. Therefore, when reproducing the actual sound, a MIDI sound source storing the waveform of the instrument sound is separately required. However, its high encoding efficiency is attracting attention, and encoding and decoding according to the MIDI standard are being attracted attention. This technology is now widely used in software that uses a personal computer to perform musical instrument performance, practice and compose music.
[0005]
Therefore, by analyzing a time-series signal represented by an acoustic signal by a predetermined method, a periodic signal as a constituent element is extracted, and the extracted periodic signal is encoded using MIDI data. Proposals have been made. For example, JP-A-10-247099, JP-A-11-73199, JP-A-11-73200, JP-A-11-95753, JP-A-2000-99009, JP-A-2000-99092, JP-A-2000-99093, JP-A-2000-261322, JP-A-2001-5450, and JP-A-2001-148633 analyze the frequency as a component of an arbitrary time-series signal, Various methods for creating MIDI data from the analysis results have been proposed.
[0006]
In the invention described in the above publication, frequency analysis of time-series signals is performed using a generalized harmonic analysis technique, and the frequency resolution that is a problem in the short-time Fourier transform method is significantly improved. In the MIDI encoding method, it was an indispensable method.
[0007]
[Problems to be solved by the invention]
In the frequency analysis using the generalized harmonic analysis, there is a problem that the calculation load is large because the number of correlation operations with the harmonic function that is a periodic function is many orders of magnitude compared with the short-time Fourier transform method. In generalized harmonic analysis, the process of extracting a signal component of a predetermined frequency from the original signal and performing a correlation operation on the remaining signal is repeated, so that it depends on the order of the extracted signal components. This causes a problem that the analysis result changes.
[0008]
In order to solve such a problem, the present applicant calculates in advance Japanese Patent Application No. 2002-9223 a cross-correlation between harmonic functions that perform correlation calculation with a time-series signal, and uses this cross-correlation value. Thus, a method for correcting the correlation value between the time series signal and the harmonic function was proposed. This makes it possible to achieve a frequency resolution equivalent to that of generalized harmonic analysis with a calculation load equivalent to that of the short-time Fourier transform method.
[0009]
However, since the frequency interval of the harmonic function that performs correlation calculation with the time-series signal is set on a logarithmic scale, the interval increases as the frequency increases. Therefore, in the correction using the cross-correlation value, there is a problem that the correction is unbalanced, the correction is excessive for low frequencies, and some components are missing. In addition, if correction is made sweet in order to make use of the low frequency band, there is a problem that even adjacent frequency components that are unnecessary are left as a whole.
[0010]
In view of the above points, the present invention enables a frequency analysis method and sound of a time-series signal capable of faithfully extracting frequency components in a low frequency region and suppressing generation of unnecessary adjacent frequency components. It is an object of the present invention to provide a signal encoding method.
[0011]
[Means for Solving the Problems]
In order to solve the above problems, in the present invention, as a frequency analysis method for separating a plurality of signal components from a time-series signal, a plurality of frequencies are set, and a plurality of harmonic functions corresponding to the respective frequencies are prepared. A function preparation stage, a cross-correlation table preparation stage for generating a cross-correlation table in which correlations between the prepared harmonic functions are calculated for all combinations, and setting a plurality of unit sections on the time axis of the time series signal , A section signal extraction stage for extracting section signals for each unit section, calculating correlations between the plurality of harmonic functions and the section signals, and generating a signal correlation array that calculates a correlation square value corresponding to each harmonic function The correlation calculation step, and for the element Po (fn) corresponding to the harmonic function having the frequency fn of the signal correlation array, the signal corresponding to the harmonic function having all the frequencies fm other than fn The value R (fm, fn) of said cross-correlation table corresponding to fn and fm the elements Po correlation sequence (fm) multiplied by the value R (fm, fm) of the cross-correlation table corresponding to fm and fm the Temporary correlation correction stage for obtaining a temporary correction correlation array composed of elements P1 (fn) corrected by subtracting each value of Po (fm) R (fm, fn) / R (fm, fm) divided by the signal, Corresponding to elements Po1 (fm) corresponding to harmonic functions having all frequencies fm other than fn, corresponding to elements P1 (fm) of the provisionally corrected correlation array corresponding to the harmonic functions having the frequency fn of the correlation array correspond to fn and fm P1 (fm) R (fm, fn) / R () obtained by multiplying the cross-correlation table value R (fm, fn) and dividing the cross-correlation table value R (fm, fm) corresponding to fm and fm fm, f ) Becomes the value of each performs regular correlation correction step of obtaining a normalized corrected correlation array of corrected element P2 (fn) by subtracting the correlation computed for the interval signal extraction stage at a defined unit sections The step, the provisional correlation correction step and the normal correlation correction step are repeated to obtain a normal correction correlation array corresponding to each unit section.
[0012]
According to the present invention, a cross-correlation table in which correlation functions between harmonic functions are calculated in advance and a correlation square value is recorded is prepared, and when performing frequency analysis of a time-series signal, a short-time Fourier transform process is performed. Since the correlation square value is corrected in two steps using each cross-correlation square value of the cross-correlation table, it is possible to faithfully extract the frequency component in the low frequency part and unnecessary adjacent frequency component Can be suppressed.
[0013]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
(1. Basic principle)
First, the basic principle of the frequency analysis method and the acoustic signal encoding method according to the present invention will be described. Since this basic principle is disclosed in the above-mentioned publications, only the outline will be briefly described here.
[0014]
As shown in FIG. 1A, it is assumed that an analog acoustic signal is given as a time-series signal. In the example of FIG. 1, the acoustic signal is shown with time t on the horizontal axis and amplitude (intensity) on the vertical axis. Here, first, the analog sound signal is processed as digital sound data. This may be performed by using a conventional general PCM method, sampling the analog acoustic signal at a predetermined sampling frequency, and converting the amplitude into digital data using a predetermined number of quantization bits.
[0015]
Subsequently, a plurality of unit sections are set on the time axis of the acoustic signal to be analyzed. In the example shown in FIG. 1A, six times t1 to t6 are defined at equal intervals on the time axis t, and five unit intervals d1 to d5 having these times as the start point and the end point are set. In the example of FIG. 1, all unit sections having the same section length are set without overlapping on the time axis, but the section setting is performed so that adjacent unit sections partially overlap on the time axis. It doesn't matter.
[0016]
When the unit section is set in this way, representative frequencies are selected for the acoustic signals (hereinafter referred to as section signals) for each unit section. Each section signal usually includes various frequency components. For example, a frequency component having a high component intensity ratio may be selected as the representative frequency. Here, the so-called fundamental frequency is generally used as the representative frequency, but a harmonic frequency such as a formant frequency of speech or a peak frequency of a noise source may be treated as a representative frequency. Although only one representative frequency may be selected, more accurate encoding is possible by selecting a plurality of representative frequencies depending on the acoustic signal. FIG. 1B shows an example in which three representative frequencies are selected for each unit section, and one representative frequency is encoded as one representative code (shown as a note for convenience in the drawing). Has been. Here, three tracks T1, T2, and T3 are provided to accommodate representative codes (notes). This is because three representative codes selected for each unit section are assigned to different tracks. It is for accommodating.
[0017]
For example, representative codes n (d1,1), n (d1,2), n (d1,3) selected for the unit section d1 are accommodated in tracks T1, T2, T3, respectively. Here, each code n (d1,1), n (d1,2), n (d1,3) is a code indicating a note number in the MIDI code. The note number in the MIDI code takes 128 values from 0 to 127, each indicating one key of the piano keyboard. Specifically, for example, when 440 Hz is selected as the representative frequency, this frequency corresponds to the note number n = 69 (corresponding to “ra sound (A3 sound)” in the center of the piano keyboard). N = 69 is selected. However, FIG. 1B is a conceptual diagram showing the representative code obtained by the above-described method in the form of a note. In reality, data on intensity is also added to each note. For example, in the track T1, e (d1,1), e (d2,1)..., Together with data indicating the pitches of note numbers n (d1,1), n (d2,1). Data indicating the strength is accommodated. The data indicating the intensity is determined by the degree to which the component of each representative frequency is included in the original section signal. Specifically, the data indicating the intensity is determined based on the correlation value with respect to the section signal of the periodic function having each representative frequency. Further, in the conceptual diagram shown in FIG. 1B, the position of each unit section on the time axis is indicated by the position of the note in the horizontal direction, but in reality, the position on the time axis is shown. Is accurately added as a numerical value to each note.
[0018]
As a format for encoding an acoustic signal, it is not always necessary to adopt the MIDI format. However, since the MIDI format is the most popular as this type of encoding, code data in the MIDI format is practically used. Is preferred. In the MIDI format, “note-on” data or “note-off” data exists while interposing “delta time” data. The “note-on” data is data for designating a specific note number N and velocity V to instruct the start of a specific sound, and the “note-off” data is a specific note number N and velocity V. This is data that designates the end of the performance of a specific sound. The “delta time” data is data indicating a predetermined time interval. Velocity V is a parameter that indicates, for example, the speed at which a piano keyboard is pressed down (velocity at the time of note-on) and the speed at which the finger is released from the keyboard (velocity at the time of note-off). Or it shows the strength of the performance end operation.
[0019]
In the above-described method, J note numbers n (di, 1), n (di, 2),..., N (di, J) are obtained as representative codes for the i-th unit interval di. Intensities e (di, 1), e (di, 2),..., E (di, J) are obtained for each of these. Therefore, MIDI format code data can be created by the following method. First, as the note number N described in the “note on” data or “note off” data, the obtained note numbers n (di, 1), n (di, 2),..., N (di , J) can be used as they are. On the other hand, as the velocity V described in the “note on” data or “note off” data, the obtained intensities e (di, 1), e (di, 2),..., E (di, A value obtained by normalizing J) by a predetermined method may be used. The “delta time” data may be set according to the length of each unit section.
[0020]
(2. Specific method for obtaining correlation with periodic function)
In the method based on the basic principle described above, one or a plurality of representative frequencies are selected for the section signal, and the section signal is represented by a periodic signal having this representative frequency. Here, the representative frequency to be selected is literally a frequency representing the signal component in the unit section. Specific methods for selecting the representative frequency include a method using a short-time Fourier transform and a method using a generalized harmonic analysis method, as will be described later. Both methods have the same basic concept, and a plurality of periodic functions with different frequencies are prepared as harmonic functions in advance, and the correlation with the section signal in the unit section is selected from the plurality of periodic functions. A method of finding a high periodic function and selecting the frequency of the highly correlated periodic function as a representative frequency is adopted. That is, when selecting a representative frequency, an operation for obtaining a correlation between a plurality of periodic functions prepared in advance and a section signal in a unit section is performed. Therefore, here, a specific method for obtaining the correlation with the periodic function will be described.
[0021]
Assume that trigonometric functions as shown in FIG. 2 are prepared as a plurality of periodic functions. These trigonometric functions are composed of a pair of a sine function and a cosine function having the same frequency. For each of 128 standard frequencies f (0) to f (127), a pair of a sine function and a cosine function. Is defined. Here, a pair of functions consisting of a sine function and a cosine function having the same frequency is defined as a periodic function for the frequency. That is, the periodic function for a specific frequency is constituted by a pair of sine function and cosine function. Thus, the periodic function is defined by a pair of sine function and cosine function in order to consider that the correlation value is influenced by the phase when obtaining the correlation value of the periodic function with respect to the signal. The variables F and k in each trigonometric function shown in FIG. 2 are variables corresponding to the sampling frequency F and the sample number k for the section signal X. For example, a sine wave for the frequency f (0) is indicated by sin (2πf (0) k / F), and given an arbitrary sample number k, the same time position as the k-th sample constituting the section signal The amplitude value of the periodic function at is obtained. Here, 128 standard frequencies f (0) to f (127) are defined by [Formula 1] shown below.
[0022]
[Formula 1]
f (n) = 440 × 2 ^{γ (n)}
γ (n) = (n−69) / 12
However, n = 0, 1, 2,..., 127
[0023]
If the standard frequency is defined by such an expression, it is convenient when finally encoding using MIDI data is performed. This is because the 128 standard frequencies f (0) to f (127) set by such a definition take frequency values forming a geometric series, and correspond to the note numbers used in the MIDI data. This is because it becomes a frequency. Therefore, the 128 standard frequencies f (0) to f (127) shown in FIG. 2 are frequencies set at equal intervals (in semitone units in MIDI) on the frequency axis shown on the logarithmic scale.
[0024]
(2.1. Short-time Fourier transform method)
Next, a specific description will be given of how to obtain the correlation of each periodic function with respect to a section signal in an arbitrary section. For example, as shown in FIG. 3, it is assumed that a section signal X is given for a certain unit section d. Here, it is assumed that sampling is performed at the sampling frequency F for the unit interval d having the interval length L, and w sample values are obtained in total, and the sample numbers are 0, 1, 2, 3,..., K,..., W-2, w-1 (the w-th sample indicated by a white circle is a sample included at the head of the next unit section adjacent to the right And). In this case, for an arbitrary sample number k, an amplitude value of X (k) is given as digital data. In the short-time Fourier transform, it is usual to multiply the window function W (k) such that the center weight is close to 1 and the weights at both ends are close to 0 for each sample with respect to X (k). That is, X (k) × W (k) is treated as X (k) and the following correlation calculation is performed. As the shape of the window function, a cosine wave-shaped Hamming window is generally used. Here, w is described as a constant in the following description, but in general, it is changed according to the value of n, and F / f (n) that is maximum within a range not exceeding the section length L. It is desirable to set the value to an integer multiple.
[0025]
The principle of obtaining a correlation value with such a section signal X and the sine function Rn having the nth standard frequency f (n) is shown. The correlation value A (n) between the two can be defined by the following [Equation 2].
[0026]
[Formula 2]
A (n) = (2 / w) Σk _{= 0, w−1} x (k) sin (2πf _n k / F)
B (n) = (2 / w) Σk _{= 0, w−1} x (k) cos (2πf _n k / F)
E (n) = {A (n) ² + B (n) ² } ^1/2
[0027]
In the above [Expression 2], X (k) is the amplitude value of the sample number k in the section signal X as shown in FIG. 3, and sin (2πf _n k / F) is the same position on the time axis. Is the amplitude value of the sine function Rn. Note that f (n) is expressed as f _n in the mathematical expression in order to avoid complicated expressions. The first arithmetic expression of [Expression 2] is an expression for obtaining the inner product of the amplitude value of the section signal X and the amplitude vector of the sine function Rn for each dimension of all sample numbers k = 0 to w−1 in the unit section d. It can be said.
[0028]
Similarly, the second arithmetic expression of [Formula 2] is an expression for obtaining a correlation value between the interval signal X and the cosine function having the nth standard frequency f (n), and the correlation value between the two. Is given by B (n). The first arithmetic expression for obtaining the correlation value A (n) and the second arithmetic expression for obtaining the correlation value B (n) are finally multiplied by 2 / w. Is for normalizing the correlation value. As described above, since w is generally changed depending on n, this coefficient is also a variable depending on n.
[0029]
The effective correlation value between the interval signal X and the standard periodic function having the standard frequency f (n) is the correlation value A (n) with the sine function, as shown in the third arithmetic expression of the above [Equation 2]. Of the square sum of squares with the correlation value B (n) with the cosine function, it can be represented by E (n) which is a positive value. If the frequency of the standard periodic function having a large correlation effective value is selected as the representative frequency, the section signal X can be encoded using this representative frequency.
[0030]
That is, one or a plurality of standard frequencies whose correlation value E (n) is greater than or equal to a predetermined reference may be selected as the representative frequency. Here, the selection condition that “correlation value E (n) is greater than or equal to a predetermined reference” is, for example, a standard in which some threshold value is set and correlation value E (n) exceeds this threshold value. An absolute selection condition that all frequencies f (n) are selected as representative frequencies may be set. For example, up to the Qth in the order of the correlation value E (n) is selected. A relative selection condition may be set.
[0031]
(2.2. Method of generalized harmonic analysis)
Here, a conventional frequency analysis method and a generalized harmonic analysis method that has been used when encoding an acoustic signal will be described. As already described, when encoding an acoustic signal, several representative frequencies having high correlation values are selected for the section signal in each unit section. Generalized harmonic analysis is a technique that allows a selection of a representative frequency with higher accuracy, although the calculation amount is large, and its basic principle is as follows.
[0032]
Assume that there is a signal S (j) for a unit interval d as shown in FIG. Here, j is a parameter for repetitive processing (j = 1 to J), as will be described later. First, correlation values for all 128 periodic functions as shown in FIG. 2 are obtained for this signal S (j). Then, the frequency of one periodic function having the maximum correlation value is selected as a representative frequency, and the periodic function having the representative frequency is extracted as an element function. Subsequently, the inclusion signal G (j) as shown in FIG. 4B is defined. The inclusion signal G (j) is a signal obtained by multiplying the extracted element function by the correlation value of the element function with respect to the signal S (j) of the element function. For example, as shown in FIG. 2, when a frequency f (n) is selected as a representative frequency using a pair of sine function and cosine function as shown in FIG. 2, a sine function A (n) having an amplitude A (n). ) and _{sin (2πf n k / F)} , it comes to a cosine function B having an amplitude B (n) (n) cos (2πf n k / F) signal composed of the sum of the Inclusion signal G (j) (In FIG. 4B, only one function is shown for convenience of illustration). Here, since A (n) and B (n) are normalized correlation values obtained by the above [Equation 2], the inclusion signal G (j) is eventually included in the signal S (j). It can be said that the signal component has a frequency f (n).
[0033]
Thus, when the content signal G (j) is obtained, the difference signal S (j + 1) is obtained by subtracting the content signal G (j) from the signal S (j). FIG. 4C shows the differential signal S (j + 1) obtained in this way. The difference signal S (j + 1) can be said to be a signal composed of the remaining signal components obtained by removing the signal component having the frequency f (n) from the original signal S (j). Therefore, by increasing the parameter j by 1, this difference signal S (j + 1) is handled as a new signal S (j), and the same processing is performed J times while increasing the parameter j by 1 from j = 1 to J. If it is repeatedly executed, J representative frequencies can be selected.
[0034]
The J inclusion signals G (1) to G (J) output as a result of such correlation calculation are signals that are constituent elements of the original section signal X, and the original section signal X is encoded. In this case, information indicating the frequency of these J inclusion signals and information indicating the amplitude (intensity) may be used as the code data. Although J has been described as the number of representative frequencies, it may be the same as the number of standard frequencies f (n), that is, J = 128. In frequency analysis for the purpose of obtaining a frequency spectrum, J It is customary to do so.
[0035]
Through the processing as described above, a frequency group that is a set of intensity values for each frequency is obtained for each unit section. When a predetermined number of frequency groups are selected in this way, “information indicating the pitch” corresponding to each frequency of this frequency group, and “sound strength corresponding to the signal intensity of each selected frequency”. Information indicating, “information indicating sound start time of sound” corresponding to the start point of the unit section, and “information indicating sound end time of sound” corresponding to the start point of the unit section subsequent to the unit section If a predetermined number of code data including information is created, the section signal X in the unit section can be encoded with the predetermined number of code data. If MIDI data is created as code data, a note number is used as “information indicating the pitch of the sound”, velocity is used as the “information indicating the intensity of the sound”, and “sound generation start time is set. The note-on time may be used as the “information indicating” and the note-off time may be used as the “information indicating the end time of sound generation”.
[0036]
(3.1. Frequency analysis method and acoustic signal encoding method according to the present invention)
Hereinafter, a frequency analysis method and an audio signal encoding method according to the present invention will be described. As described above, when performing frequency analysis, the amount of calculation is greatly different between the technique using the short-time Fourier transform and the generalized harmonic analysis. Therefore, in the present invention, a table in which the correlation between the harmonic functions is calculated for all combinations is prepared in advance, and the correlation value obtained by the short-time Fourier transform is corrected using this table.
[0037]
FIG. 5 is a flowchart showing an outline of the frequency analysis method according to the present invention. First, a plurality of standard frequencies are set, and a standard periodic function corresponding to each standard frequency is prepared as a harmonic function (step S1). The standard frequency set at this time can be arbitrarily set according to the characteristics of frequency analysis. However, in order to use it for encoding an acoustic signal, as shown in FIG. 2 and [Equation 1]. It is preferable to set it corresponding to the note number n of the MIDI standard.
[0038]
Subsequently, cross-correlation square values, which are correlation square values between the harmonic functions, are calculated for all combinations, and a cross-correlation table is created (step S2). At this time, the cross-correlation square value R (f _m , f _n ) for the harmonic function of the frequency f (n) of the harmonic function of the frequency f ( _m ) is calculated by the following [Equation 3]. For convenience of expression, f (n) may be described as f _n , and both are equivalent.
[0039]
[Formula 3]
A (f _m , f _n ) = (2 / T (n)) Σt _{= 0, T (n) −1} sin (2πf _m t) sin (2πf _n t)
B (f _m , f _n ) = (2 / T (n)) Σt _{= 0, T (n) −1} sin (2πf _m t) cos (2πf _n t)
R (f _m , f _n ) = {A (f _m , f _n )} ² + {B (f _m , f _n )} ²
[0040]
The cross-correlation square value R (f _m , f _n ) calculated by the third equation of [Equation 3] indicates one element of a two-dimensional cross-correlation table. As shown in FIG. 2, when m and n correspond to the note numbers, the cross-correlation table records the cross-correlation square values of 128 note numbers corresponding to the respective note numbers m. * 128 cross-correlation square values are recorded.
[0041]
When the cross-correlation table is ready, a unit section is set over all sections of the time series signal to be analyzed, and the time series signal of the set unit section is extracted as a section signal (step S3). As shown in FIG. 1A, the unit section may be set so that the end point of the preceding unit section is the same as the start point of the succeeding unit section so that both unit sections do not overlap. However, both unit sections may be set to overlap each other. This can be set according to the characteristics of the time series signal to be analyzed.
[0042]
Subsequently, a correlation calculation with the entire harmonic function is performed on the extracted section signal (step S4). For example, when the standard frequency is set corresponding to the note number as shown in FIG. 2, correlation calculation with 128 harmonic functions is performed. The correlation calculation with the harmonic function in step S4 is performed by a short-time Fourier transform method. That is, the correlation between the harmonic function and the portion of the interval signal that does not exceed the unit interval length by an integral multiple of the period of the harmonic function for which the correlation calculation is performed is calculated. The calculated correlation square value is stored in a signal correlation array prepared for each unit section. In the present invention, since the correlation calculation is performed by the short-time Fourier transform, the correlation calculation with each harmonic function is performed only once for one interval signal. The correlation square value P ₀ (f _n ) between the harmonic function of the standard frequency f (n) in this step S4 and the section signal x (t) is calculated by the following [Equation 4].
[0043]
[Formula 4]
A (f _n ) = (2 / T (n)) Σt _{= 0, T (n) −1} x (t) sin (2πf _n t)
B (f _n ) = (2 / T (n)) Σt _{= 0, T (n) −1} x (t) cos (2πf _n t)
P ₀ (f _n ) = {A (f _n )} ² + {B (f _n )} ²
[0044]
This [Equation 4] is substantially equivalent to the above [Equation 2] only by replacing the correlation calculation sample number w of [Equation 2] with T (n) which clearly shows that it depends on the note number n. It is a formula. However, the correlation value E (n) calculated by [Equation 2] has a relationship of a correlation square value P ₀ (f _n ) = {E (n)} ² .
[0045]
When the signal correlation array is obtained, the correlation square value that is each element in the array is provisionally corrected using the cross correlation table (step S5). Specifically, the provisional correction value P ₁ (f _n) of the standard frequency f (n) and the correlation square value P ₀ of (f _n), the correlation squared value P ₀ of the standard frequency f _(m) (f m ), The cross-correlation square value R (f _m , f _n ) of the standard frequency f (m) with respect to the standard frequency f ( _n ), and the autocorrelation R (f _m , f _m ) of the standard frequency f (m), It is calculated by the following [Equation 5].
[0046]
[Formula 5]
P ₁ (f _n ) = P ₀ (f _n ) −Σ _{m ≠ n} P ₀ (f _m ) R (f _m , f _n ) / R (f _m , f _m )
[0047]
However, in the term of Σ in the above [Equation 5], the sum total of N−1 pieces excluding m = n in the range of m = 0 to m = N−1 is calculated. The temporary correction value P ₁ (f _n ) calculated by the above [Equation 5] is stored at a position corresponding to each frequency f _n of the temporary correction correlation array. However, at this point, some elements in the array may have a negative value. In this case, the value is set to 0 so that all the temporary correction values P ₁ (f _n ) in the temporary correction correlation array are 0 or a positive value.
[0048]
When the temporary correction correlation array is obtained, the temporary correction values, which are the elements in the array, are normally corrected using the cross correlation table (step S6). Specifically, the normal correction value P ₂ (f _n) of the standard frequency f (n) and the correlation square value P ₀ of (f _n), the correlation squared value P ₀ of the standard frequency f _(n) (f n ), Provisional correction value P ₁ (f _m ), cross-correlation square value R (f _m , f _n ) for standard frequency f (m) with respect to standard frequency f ( _n ), autocorrelation R (f f _m , f _m ) is calculated by the following [Equation 6].
[0049]
[Formula 6]
P ₂ (f _n ) = P ₀ (f _n ) −Σ _{m ≠ n} P ₁ (f _m ) R (f _m , f _n ) / R (f _m , f _m )
[0050]
However, in the term of Σ in the above [Formula 6], the sum total of N−1 pieces excluding m = n in the range of m = 0 to m = N−1 is calculated. The normal correction value P ₂ (f _n ) calculated by the above [Equation 6] is stored at a position corresponding to each frequency f _n of the normal correction correlation array. However, at this point, some elements in the array may have a negative value. In that case, the value is set to 0 so that the normal correction values P ₂ (f _n ) in the normal correction correlation array are all 0 or a positive value. The reason why the value of the normal correction correlation array is set to 0 or more in this manner is to delete an unrealistic value because the correlation square value is basically impossible.
[0051]
At this time, a correlation value E (n) which is a positive square root of the normal correction value P ₂ (f _n ) stored in the normal correction correlation array is calculated, and the frequency f (n) and the correlation value E (n) are calculated. Even if the frequency analysis is completed by obtaining a set, the effect of the present invention can be sufficiently obtained. However, in the processing up to this point, the influence of the adjacent frequency component cannot be completely deleted, and there is a problem that the influence of the adjacent frequency component is noticeable particularly in a bass region where the frequency setting interval of the harmonic function is small. is there. That is, the correlation value E (n) corresponding to each frequency f (n) is a value obtained by adding the correlation value of the adjacent frequency to the correlation value of the original frequency f (n).
[0052]
Therefore, correction for deleting adjacent components is performed on the normal correction value P ₂ (f _n ) stored in the normal correction correlation array (step S7). Specifically, first, the adjacent frequency components slightly remaining in the treble region are completely deleted using the following [Equation 7].
[0053]
[Formula 7]
P ₂ (f _n ) <P ₂ (f _{n + 1} ) × α or P ₂ (f _n ) <P ₂ (f _{n + 2} ) × α
Then P ₂ (f _n ) = 0
[0054]
In the above [Expression 7], α is a coefficient satisfying 0 ≦ α ≦ 1, but preferably α = about 0.25. That is, in the above [Equation 7], when attention is paid to a certain frequency f (n), the normal correction value corresponding to the frequency f (n) is an adjacent frequency f higher by one semitone than the frequency f (n). (N + 1) A process of setting the normal correction value of the focused frequency f (n) to 0 when it is equal to or less than a predetermined ratio of the normal correction value of any one of the adjacent frequencies f (n + 2) higher by two semitones, that is, by one full tone. It is carried out.
[0055]
Subsequently, the adjacent frequency components abundantly distributed in the bass region are deleted using the following [Equation 8]. In this case, since the adjacent frequency component is remarkable, it is difficult to distinguish between the physically existing adjacent frequency component and the pseudo adjacent frequency component generated due to the error of the correlation calculation, and [Expression 7] is applied as it is. It is not possible. Therefore, a method is adopted in which a determination is made based on a harmonic frequency component existing in a high-pitched sound region that can be easily identified. That is, when there is no harmonic frequency component corresponding to the adjacent frequency component to be determined (value is 0), processing for setting the adjacent frequency component to 0 is performed. Specifically, it is as the following [Formula 8].
[0056]
[Formula 8]
(1) P ₂ (f _n )> 0 and P ₂ (f _{n + 1} )> 0 and P ₂ (f _{n + 12} ) = 0 and P ₂ (f _{n + 13} )> 0
Then P ₂ (f _n ) = 0
( ₂ ) P ₂ (f _n )> 0 and P ₂ (f _{n + 2} )> 0 and P ₂ (f _{n + 12} ) = 0 and P ₂ (f _{n + 14} )> 0
Then P ₂ (f _n ) = 0
(3) P ₂ (f _n )> 0 and P ₂ (f _{n + 1} )> 0 and P ₂ (f _{n + 19} ) = 0 and P ₂ (f _{n + 20} )> 0
Then P ₂ (f _n ) = 0
(4) P ₂ (f _n )> 0 and P ₂ (f _{n + 2} )> 0 and P ₂ (f _{n + 19} ) = 0 and P ₂ (f _{n + 21} )> 0
Then P ₂ (f _n ) = 0
[0057]
The above [Equation 8] indicates that the normal correction value P ₂ (f _n ) is set to 0 when any one of the conditions ( ₁ ) to (4) is satisfied. Further, in the above-described [Equation _{8], f n + 12, f n} + 19 shows the double frequency, triple frequency _{f n, f n + 13,} f n + 20 is f _{n + 1} 2 times and 3 times the frequency, and f _{n + 14} and f _{n + 21} are the frequency twice the frequency of f _{n + 2} and 3 times the frequency. That is, the condition (1) is that, when focusing on a certain frequency f (n), the normal correction value P ₂ (f _n ) corresponding to the frequency f ( _n ) is a positive value, and the frequency f ( _n The normal correction value P ₂ (f _{n + 1} ) corresponding to the adjacent frequency f (n + 1) that is one semitone higher than n) is also positive, and the frequency f (n + 12), which is twice the frequency f (n). The corresponding normal correction value P ₂ (f _{n + 12} ) is 0, and the normal correction value P ₂ (f _{n + 13} ) corresponding to the adjacent frequency f (n + 13) that is one semitone higher than the frequency f (n + 12) is positive. In this case, the normal correction value of the focused frequency f (n) is set to 0.
[0058]
In a normal acoustic signal, when a certain frequency component actually exists as a component of a time-series signal, when calculating a correlation with the time-series signal, a frequency that is an integral multiple of that frequency also has a certain correlation. Have. (However, this rule may not apply if the target is a time-series electrical signal or the like.) Therefore, if there is no integer multiple frequency component (the correlation value corresponding to the integer multiple frequency is 0), the original There is also a high possibility that it is influenced by adjacent frequency components. Therefore, in order to check whether adjacent frequency components exist, frequency components that are integral multiples of the adjacent frequency components are examined. If there is a frequency component that is an integral multiple of the adjacent frequency component, the adjacent frequency component is present, so the original frequency component is deleted as a result of calculation based on the influence of the adjacent frequency component ( The normal correction value is set to 0).
[0059]
The basic concept is the same for the conditions (2) to (4). For the condition (2), the adjacent frequency and the doubled frequency are referred to. For the condition (3), the adjacent frequency is used. For condition (4), the frequency component corresponding to the frequency f (n) is inherently present with reference to the frequency of two semitones, that is, the frequency one whole tone higher and the frequency three times higher. If it does not exist, the normal correction value P ₂ (f _n ) = 0 is performed. In this way, a _second corrected correlation array in which the normal correction value P ₂ (f _n ) of the normal correlation array is corrected is obtained.
[0060]
When the second corrected correlation array is obtained, a correlation value E (n) that is a positive square root of the second corrected correlation value that is each value of the second corrected correlation array is calculated, and the frequency f (n) and the correlation value are calculated. A set of E (n) is obtained. By performing correlation calculation in step S4 and correlation correction in steps S5 to S7 for all unit sections set in step S3, N frequency components (a set of frequency and correlation value) in all unit sections are obtained. Will be.
[0061]
(3.2. Coding of acoustic signals)
The frequency analysis of the time series signal is performed as described above, and N frequency components are extracted for each unit section. When an acoustic signal is used as a time-series signal and the encoding of the acoustic signal is performed, the standard frequency f (n) is set at a MIDI note number, that is, a pitch interval in units of semitones as shown in FIG. N (= 128) frequency components corresponding to each note number are obtained. Then, as described above, by converting the frequency component frequency into MIDI data having the note number, the correlation value as the velocity, the start point of the unit interval as the note-on time, and the start point of the subsequent unit interval as the note-off time, An acoustic signal is encoded.
[0062]
(3.3. Concept of correlation correction according to the present invention)
Next, the effect of correcting the correlation value according to the present invention will be conceptually described with reference to FIG. In FIG. 6, the horizontal axis corresponds to the note number (frequency), and the vertical axis corresponds to the signal intensity or the correlation intensity. Here, let us consider a case where two sounds exist as original sounds. The original signal spectra of the two sound sources are expressed by two frequencies as shown in FIG. When the frequency analysis of this acoustic signal is performed, if only two frequencies are extracted as shown in FIG. 6A, the most accurate frequency analysis is performed. However, when frequency analysis by short-time Fourier analysis is performed on this acoustic signal, a large number of frequency components are extracted as shown in FIG. This corresponds to the state in which the correlation calculation in step S4 is performed. In this state, when the provisional correction in step S5 is performed, the value is largely corrected particularly in the low frequency region as shown in FIG. Furthermore, when the normal correction in step S6 is performed, a frequency component as shown in FIG. 6D is obtained.
[0063]
(3.4. About frequency setting)
In the above embodiment, the frequency to be extracted is set as [Expression 1] as a standard frequency corresponding to the note number n of the MIDI standard. It is also possible to perform high detection. Further, the frequency may be set at equal intervals in proportion to the frequency instead of logarithmic intervals in proportion to the note number as in [Formula 1]. However, when an acoustic signal is analyzed as a time-series signal and encoded, it is preferable to set an interval corresponding to the note number. The most preferable frequency setting for use in encoding an acoustic signal is set at a finer interval between each note number. For example, twelve frequencies are set between each note number so as to be 1/13 intervals of the note number. When such a setting is performed, 128 × 13 frequencies are set in total, and (128 × 13) squared correlation square values are obtained in the cross-correlation table. The calculation accuracy is significantly improved. When encoding into an acoustic signal, there is no frequency corresponding to such a fine frequency setting, so the frequency corresponding to the closest note number is used as the extraction component.
[0064]
Although the preferred embodiments of the present invention have been described above, the frequency analysis method and the encoding method are naturally executed by an arithmetic processing device such as a computer. Specifically, a program for executing the steps as shown in the flowchart of FIG. Then, after executing the processing of step S1 and step S2 in advance to prepare a cross-correlation table, the time series signal such as an acoustic signal is digitized by the PCM method or the like, and then taken into a computer, and steps S3 to S7 are performed. After the processing, the extracted frequency component or code data such as MIDI format is output from the computer as digital data. For example, in the case of MIDI data, the output code data is reproduced as sound using a MIDI sequencer and a MIDI sound source.
[0065]
【The invention's effect】
As described above, according to the present invention, as described above, a plurality of frequencies are set, a plurality of harmonic functions corresponding to the respective frequencies are prepared, and correlations between prepared harmonic functions are calculated for all combinations. Generate a cross-correlation table, set multiple unit intervals on the time axis of the time series signal, extract the interval signal for each unit interval, calculate the correlation between multiple harmonic functions and interval signals, A signal correlation array in which a correlation square value corresponding to the harmonic function is calculated is generated, and for each element of the signal correlation array, based on a plurality of other elements of the signal correlation array and a corresponding plurality of elements of the cross-correlation table To obtain a temporary correction correlation array, and correct each element of the temporary correction correlation array based on the other elements of the temporary correction correlation array and the corresponding elements of the cross-correlation table. Regular A normal correlation array is obtained by repeatedly calculating a correlation square value, a temporary correction correlation value, and a normal correction correlation value for each defined unit interval so as to obtain a positive correlation sequence. As a result, it is possible to faithfully extract frequency components in the low frequency region and to suppress the occurrence of unnecessary adjacent frequency components.
[Brief description of the drawings]
FIG. 1 is a diagram showing a basic principle of a frequency analysis method and an acoustic signal encoding method according to the present invention.
FIG. 2 is a diagram showing an example of a periodic function used in the present invention.
FIG. 3 is a diagram illustrating a method of calculating a correlation between a signal to be analyzed and a periodic signal.
FIG. 4 is a diagram showing a basic method of generalized harmonic analysis.
FIG. 5 is a flowchart of a frequency analysis method and an acoustic signal encoding method according to the present invention.
FIG. 6 is a diagram conceptually showing the effect of correlation correction according to the present invention.

Claims (4)

A frequency analysis method for separating a plurality of signal components from a time series signal,
A harmonic function preparation stage for setting a plurality of frequencies and preparing a plurality of harmonic functions corresponding to the respective frequencies;
A cross-correlation table preparation step for generating a cross-correlation table in which the correlation between the prepared harmonic functions is calculated for all combinations;
A section signal extraction stage for setting a plurality of unit sections on the time axis of the time series signal and extracting a section signal for each unit section;
A correlation calculation step of calculating a correlation between the plurality of harmonic functions and the interval signal, and generating a signal correlation array in which a value corresponding to each harmonic function is calculated;
Relative to the corresponding elements to the frequency fn becomes harmonic signal correlation sequence P0 (fn), elements of the signal correlation sequence corresponding to all the frequency fm becomes harmonics other than fn P0 (fm) to fn and fm Is multiplied by the value R (fm, fn) of the cross-correlation table corresponding to, and divided by the value R (fm, fm) of the cross-correlation table corresponding to fm and fm, P0 (fm) R (fm, fn) / A temporary correlation correction stage for obtaining a temporary correction correlation array composed of elements P1 (fn) corrected by subtracting each value of R (fm, fm) ;
For the element P0 (fn) corresponding to the harmonic function having the frequency fn of the signal correlation array, the elements P1 (fm) of the temporary correction correlation array corresponding to the harmonic functions having all the frequencies fm other than fn are represented by fn and fm. Is multiplied by the value R (fm, fn) of the cross-correlation table corresponding to, and divided by the value R (fm, fm) of the cross-correlation table corresponding to fm and fm, P1 (fm) R (fm, fn) / A normal correlation correction stage for obtaining a normal correction correlation array composed of elements P2 (fn) corrected by subtracting each value of R (fm, fm) ,
The correlation calculation step, the temporary correlation correction step, and the normal correlation correction step are repeatedly performed on the unit intervals defined in the interval signal extraction step to obtain a normal correction correlation array corresponding to each unit interval. Frequency analysis method for time series signals. When the element P2 (fn) of the normal correction correlation array is replaced with the element PA, the ratio of the element PA to the element PB is compared with the element PB of the normal correction correlation array whose frequency is adjacent to the element PA. A second correlation correction step of obtaining a second corrected correlation array by performing correction to set the value of the element PA to 0 when the value is equal to or less than a predetermined value;
The time series according to claim 1, wherein the second correlation correction step is repeated for the unit interval defined in the interval signal extraction step to obtain a second correction correlation array corresponding to each unit interval. Signal frequency analysis method. When the element P2 (fn) of the normal correction correlation array is replaced with the element PA, the ratio of the element PA to the element PB is compared with the element PB of the normal correction correlation array whose frequency is adjacent to the element PA. The ratio of the element PC of the normal correction correlation array whose frequency is an integer multiple of the element PA to the element PD of the normal correction correlation array whose frequency is an integer multiple of the element PB is greater than or equal to a predetermined value. A second correlation correction step of obtaining a second corrected correlation array by performing correction to set the value of the element PA to 0 when the value is equal to or less than the value;
The time series according to claim 1, wherein the second correlation correction step is repeated for the unit interval defined in the interval signal extraction step to obtain a second correction correlation array corresponding to each unit interval. Signal frequency analysis method.   A frequency analysis method according to any one of claims 1 to 3 is applied to an acoustic signal given as a time series signal, and the normal correction correlation array or the second correlation correction determined in the normal correlation correction stage. The frequency at which the value of the second correction correlation array determined in the stage reaches a predetermined value is selected, the pitch information corresponding to the selected frequency, and the corresponding normal correction correlation array or the second correction. The sound intensity information corresponding to the value of the correlation array, the sound generation start time corresponding to the start point of the unit section, and the sound generation end time corresponding to the start point of the subsequent unit section are converted into four pieces of information. A method for encoding an acoustic signal, comprising: an encoding step of generating encoded data by applying the encoded data.

Images