JP4662407B2 - Frequency analysis method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a frequency analyzing method and an encoding method for a sound signal which can accurately extract a signal component even when the signal component is present nearby the border between standard frequencies and further perform high-precision encoding. SOLUTION: A section signal is extracted (S1) by setting a unit section for the sound signal and short-time Fourier transformation is used for the section signal to compute correlation with harmonic signals corresponding to respective frequencies set at fine intervals (S2). After the correlation computation, a fine frequency as a peak is selected (S3), the correlation of only the harmonic signal corresponding to the selected peak frequency with the section signal is found by a generalized harmony analyzing method, and a pair of the found correlation value and the standard frequency which is closest to the corresponding peak frequency are extracted as a frequency component (S4).

Description

[0001]
[Industrial application fields]
The present invention includes broadcast media (radio, television), communication media (CS video / audio distribution, Internet music distribution, communication karaoke), package media (CD, MD, cassette, video, LD, CD-ROM, game cassette, mobile phone). Production of various audio contents provided by a solid-state memory medium for music players, etc., music publishing from musical performance recording signals, MIDI data for online karaoke distribution, automatic performance data for electronic musical instruments with performance guide functions, mobile phones, PHS, The present invention relates to an automatic music recording technique for automatically generating incoming melody data such as a pager.
[0002]
[Prior art]
A time-series signal represented by an acoustic signal includes a plurality of periodic signals as its constituent elements. For this reason, a method for analyzing what kind of periodic signal is included in a given time-series signal has been known for a long time. For example, Fourier analysis is widely used as a method for analyzing frequency components included in a given time series signal.
[0003]
By using such a time-series signal analysis method, an acoustic signal can be encoded. With the spread of computers, it has become easy to sample an analog audio signal as the original sound at a predetermined sampling frequency, quantize the signal intensity at each sampling, and capture it as digital data. If a method such as Fourier analysis is applied to the data and the frequency components included in the original sound signal are extracted, the original sound signal can be encoded by a code indicating each frequency component.
[0004]
In addition, the MIDI (Musical Instrument Digital Interface) standard born from the idea of encoding musical instrument sounds by electronic musical instruments has come to be actively used with the spread of personal computers. The code data according to the MIDI standard (hereinafter referred to as MIDI data) is basically data that describes the operation of the musical instrument performance such as which keyboard key of the instrument is played with what strength. The data itself does not include the actual sound waveform. Therefore, when reproducing the actual sound, a MIDI sound source storing the waveform of the instrument sound is separately required. However, its high encoding efficiency is attracting attention, and encoding and decoding according to the MIDI standard are being attracted attention. This technology is now widely used in software that uses a personal computer to perform musical instrument performance, practice and compose music.
[0005]
Therefore, by analyzing a time-series signal represented by an acoustic signal by a predetermined method, a periodic signal as a constituent element is extracted, and the extracted periodic signal is encoded using MIDI data. Proposals have been made. For example, JP-A-10-247099, JP-A-11-73199, JP-A-11-73200, JP-A-11-95753, JP-A-2000-99009, JP-A-2000-99092, JP-A-2000-99093, JP-A-2000-261322, JP-A-2001-5450, and JP-A-2001-148633 analyze the frequency as a component of an arbitrary time-series signal, Various methods for creating MIDI data from the analysis results have been proposed.
[0006]
In the invention described in the above publication, when a correlation with an acoustic signal is obtained, a fine frequency is set at an interval finer than a standard frequency specified by a musical scale, and a periodic function corresponding to the fine frequency is used. . In order to be reproducible as an acoustic signal, it is necessary to encode in accordance with a standard frequency. Specifically, a fine frequency having a high correlation value is obtained, and the correlation value is a standard closest to the fine frequency. It is encoded as a frequency value. At this time, if a fine frequency having a high correlation value is present near the boundary between adjacent standard frequencies, it is determined that the signal is affected by other signal components, and processing that is not extracted as an encoding target is performed.
[0007]
[Problems to be solved by the invention]
However, if there is a fine frequency with a large correlation value near the boundary between adjacent standard frequencies as described above, if the fine frequency is not extracted, the signal component actually exists at the boundary portion. Conversely, if this fine frequency is extracted, there is a problem that it will be extracted even if it does not actually exist because of the influence of other signal components. ing.
[0008]
In view of the above points, the present invention can accurately extract a signal component even when the signal component exists in the vicinity of the boundary between standard frequencies. It is an object of the present invention to provide a frequency analysis method and an audio signal encoding method capable of performing the above.
[0009]
[Means for Solving the Problems]
In order to solve the above problems, in the present invention, as a frequency analysis method for separating a signal component corresponding to a predetermined standard frequency from a time-series signal, a plurality of fine frequencies are obtained at intervals smaller than the standard frequency interval. A harmonic signal preparation stage for setting and preparing a plurality of harmonic signal sets corresponding to each fine frequency, a main correlation array for storing a correlation value corresponding to each standard frequency, and a correlation value corresponding to each fine frequency An array preparation stage for preparing a fine correlation array for storing a signal, a section signal extraction stage for extracting a section signal by setting a unit section of a predetermined length in the time series signal, and all the harmonic signals And calculating the correlation value corresponding to each fine frequency, calculating a correlation value corresponding to each fine frequency, and setting the correlation value calculated in the fine correlation array; and the correlation distribution in the fine correlation array. A peak frequency selection stage that extracts a plurality of fine frequencies that locally peak compared to a correlation value at a surrounding fine frequency, and the correlation between the harmonic signal corresponding to the fine frequency that becomes each peak and the section signal is recalculated. Set to the main correlation array corresponding to the nearest standard frequency of the fine frequency, subtract the inclusion signal generated by the product of the recalculated correlation value and the harmonic signal from the interval signal, the difference signal It is characterized by having a spectrum calculation step of determining the value of the main correlation array by sequentially performing the process of updating the section signal by newly setting the section signal to all the extracted peaks.
[0010]
According to the present invention, a fine frequency is set at intervals finer than the standard frequency, a correlation with respect to a section signal of each fine frequency is obtained, and a fine frequency at which the correlation value locally peaks (hereinafter referred to as a peak frequency). Multiple correlations between the selected peak frequencies and the interval signal should be obtained using a generalized harmonic analysis method, and the calculated correlation value should be extracted as the correlation value of the standard frequency closest to the corresponding peak frequency. Since the frequency component is determined, it is possible to accurately extract the frequency component even when the fine frequency located near the boundary between adjacent standard frequencies has a peak.
[0011]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
[0012]
(1. Basic principle)
First, the basic principle of the frequency analysis method and the acoustic signal encoding method according to the present invention will be described. Since this basic principle is disclosed in the above-mentioned publications, only the outline will be briefly described here.
[0013]
As shown in FIG. 1A, it is assumed that an analog acoustic signal is given as a time-series signal. In the example of FIG. 1, the acoustic signal is shown with time t on the horizontal axis and amplitude (intensity) on the vertical axis. Here, first, the analog sound signal is processed as digital sound data. This may be performed by using a conventional general PCM method, sampling the analog acoustic signal at a predetermined sampling frequency, and converting the amplitude into digital data using a predetermined number of quantization bits. Here, for convenience of explanation, the waveform of the acoustic data digitized by the PCM method is also shown by the same waveform as the analog acoustic signal of FIG.
[0014]
Subsequently, a plurality of unit sections are set on the time axis of the acoustic signal to be analyzed. In the example shown in FIG. 1A, six times t1 to t6 are defined at equal intervals on the time axis t, and five unit intervals d1 to d5 having these times as the start point and the end point are set. In the example of FIG. 1, all unit sections having the same section length are set without overlapping on the time axis, but the section setting is performed so that adjacent unit sections partially overlap on the time axis. It doesn't matter.
[0015]
When the unit section is set in this way, representative frequencies are selected for the acoustic signals (hereinafter referred to as section signals) for each unit section. Each section signal usually includes various frequency components. For example, a frequency component having a high component intensity ratio may be selected as the representative frequency. Here, the so-called fundamental frequency is generally used as the representative frequency, but a harmonic frequency such as a formant frequency of speech or a peak frequency of a noise source may be treated as a representative frequency. Although only one representative frequency may be selected, more accurate encoding is possible by selecting a plurality of representative frequencies depending on the acoustic signal. FIG. 1B shows an example in which three representative frequencies are selected for each unit section, and one representative frequency is encoded as one representative code (shown as a note for convenience in the drawing). Has been. Here, three tracks T1, T2 and T3 are provided to accommodate representative codes (notes), but this means that three representative codes selected for each unit section are assigned to different tracks. It is for accommodating.
[0016]
For example, representative codes n (d1,1), n (d1,2), n (d1,3) selected for the unit section d1 are accommodated in tracks T1, T2, T3, respectively. Here, each code n (d1,1), n (d1,2), n (d1,3) is a code indicating a note number in the MIDI code. The note number in the MIDI code takes 128 values from 0 to 127, each indicating one key of the piano keyboard. Specifically, for example, when 440 Hz is selected as the representative frequency, this frequency corresponds to the note number n = 69 (corresponding to “ra sound (A3 sound)” in the center of the piano keyboard). N = 69 is selected. However, FIG. 1B is a conceptual diagram showing the representative code obtained by the above-described method in the form of a note. In reality, data on intensity is also added to each note. For example, the track T1 includes e (d1,1), e (d2,1)... Along with data indicating the pitches of note numbers n (d1,1), n (d2,1). Data indicating the strength is accommodated. The data indicating the intensity is determined by the degree to which the component of each representative frequency is included in the original section signal. Specifically, the data indicating the intensity is determined based on the correlation value with respect to the section signal of the periodic function having each representative frequency. Further, in the conceptual diagram shown in FIG. 1B, the position of each unit section on the time axis is indicated by the position of the note in the horizontal direction, but in reality, the position on the time axis is shown. Is accurately added as a numerical value to each note.
[0017]
As a format for encoding an acoustic signal, it is not always necessary to adopt the MIDI format. However, since the MIDI format is the most popular as this type of encoding, code data in the MIDI format is practically used. Is preferred. In the MIDI format, “note-on” data or “note-off” data exists while interposing “delta time” data. The “note-on” data is data for designating a specific note number N and velocity V to instruct the start of a specific sound, and the “note-off” data is a specific note number N and velocity V. This is data that designates the end of the performance of a specific sound. The “delta time” data is data indicating a predetermined time interval. Velocity V is a parameter that indicates, for example, the speed at which a piano keyboard is pressed down (velocity at the time of note-on) and the speed at which the finger is released from the keyboard (velocity at the time of note-off). Or it shows the strength of the performance end operation.
[0018]
In the above-described method, J note numbers n (di, 1), n (di, 2),..., N (di, J) are obtained as representative codes for the i-th unit interval di. Intensities e (di, 1), e (di, 2),..., E (di, J) are obtained for each of these. Therefore, MIDI format code data can be created by the following method. First, as the note number N described in the “note on” data or “note off” data, the obtained note numbers n (di, 1), n (di, 2),..., N (di , J) can be used as they are. On the other hand, as the velocity V described in the “note on” data or “note off” data, the obtained intensities e (di, 1), e (di, 2),..., E (di, A value obtained by normalizing J) by a predetermined method may be used. The “delta time” data may be set according to the length of each unit section.
[0019]
(2.1. Specific method for obtaining correlation with periodic function)
In the method based on the basic principle described above, one or a plurality of representative frequencies are selected for the section signal, and the section signal is represented by a periodic signal having this representative frequency. Here, the representative frequency to be selected is literally a frequency representing the signal component in the unit section. Specific methods for selecting the representative frequency include a method using a short-time Fourier transform and a method using a generalized harmonic analysis method, as will be described later. Both methods have the same basic concept. Prepare a plurality of periodic functions with different frequencies in advance, and from these periodic functions, a periodic function that has a high correlation with the section signal in the unit section. And a method of selecting the frequency of the highly correlated periodic function as a representative frequency is adopted. That is, when selecting a representative frequency, an operation for obtaining a correlation between a plurality of periodic functions prepared in advance and a section signal in a unit section is performed. Therefore, here, a specific method for obtaining the correlation with the periodic function will be described.
[0020]
Assume that trigonometric functions as shown in FIG. 2 are prepared as a plurality of periodic functions. These trigonometric functions are composed of a pair of a sine function and a cosine function having the same frequency. For each of 128 standard frequencies f (0) to f (127), a pair of a sine function and a cosine function. Is defined. Here, a pair of functions consisting of a sine function and a cosine function having the same frequency is defined as a periodic function for the frequency. That is, the periodic function for a specific frequency is constituted by a pair of sine function and cosine function. Thus, the periodic function is defined by a pair of sine function and cosine function in order to consider that the correlation value is influenced by the phase when obtaining the correlation value of the periodic function with respect to the signal. The variables F and k in each trigonometric function shown in FIG. 2 are variables corresponding to the sampling frequency F and the sample number k for the section signal X. For example, a sine wave with respect to the frequency f (0) is represented by sin (2πf (0) k / F), and given an arbitrary sample number k, the same time position as the k-th sample constituting the section signal The amplitude value of the periodic function at is obtained.
[0021]
Here, an example in which 128 standard frequencies f (0) to f (127) are defined by the equations as shown in FIG. That is, the nth (0 ≦ n ≦ 127) standard frequency f (n) is defined by the following [Formula 1].
[0022]
[Formula 1]
f (n) = 440 × 2^{γ (n)}
γ (n) = (n−69) / 12
[0023]
If the standard frequency is defined by such an expression, it is convenient when finally encoding using MIDI data is performed. This is because the 128 standard frequencies f (0) to f (127) set by such a definition take frequency values forming a geometric series, and correspond to the note numbers used in the MIDI data. This is because it becomes a frequency. Therefore, the 128 standard frequencies f (0) to f (127) shown in FIG. 2 are frequencies set at equal intervals (in semitone units in MIDI) on the frequency axis shown on the logarithmic scale.
[0024]
Next, a specific description will be given of how to obtain the correlation of each periodic function with respect to a section signal in an arbitrary section. For example, as shown in FIG. 4, it is assumed that a section signal X is given for a certain unit section d. Here, it is assumed that sampling is performed at the sampling frequency F for the unit interval d having the interval length L, and w sample values are obtained in total, and the sample numbers are 0, 1, 2, 3,..., K,..., W-2, w-1 (the w-th sample indicated by a white circle is a sample included at the head of the next unit section adjacent to the right. And). In this case, for an arbitrary sample number k, an amplitude value of X (k) is given as digital data. In the short-time Fourier transform, it is usual to multiply the window function W (k) such that the center weight is close to 1 and the weights at both ends are close to 0 for each sample with respect to X (k). That is, X (k) × W (k) is treated as X (k) and the following correlation calculation is performed. As the shape of the window function, a cosine wave-shaped Hamming window is generally used. Here, w is described as a constant in the following description, but in general, it is changed according to the value of n, and F / f (n) that is maximum within a range not exceeding the section length L. It is desirable to set the value to an integer multiple.
[0025]
The principle of obtaining a correlation value with such a section signal X and the sine function Rn having the nth standard frequency f (n) is shown. Both correlation values A (n) can be defined by the first arithmetic expression of FIG. Here, X (k) is the amplitude value of the sample number k in the section signal X, as shown in FIG. 4, and sin (2πf (n) k / F) is the sine at the same position on the time axis. This is the amplitude value of the function Rn. This first arithmetic expression can be said to be an expression for obtaining the inner product of the amplitude value of the section signal X and the amplitude vector of the sine function Rn for the dimensions of all sample numbers k = 0 to w−1 in the unit section d. .
[0026]
Similarly, the second arithmetic expression in FIG. 5 is an expression for obtaining a correlation value between the interval signal X and the cosine function having the nth standard frequency f (n), and the correlation value between the two is B ( n). The first arithmetic expression for obtaining the correlation value A (n) and the second arithmetic expression for obtaining the correlation value B (n) are finally multiplied by 2 / w. Is for normalizing the correlation value. As described above, since w is generally changed depending on n, this coefficient is also a variable depending on n.
[0027]
The effective correlation value between the interval signal X and the standard periodic function having the standard frequency f (n) is the correlation value A (n) with the sine function, the cosine function, as shown in the third arithmetic expression of FIG. Of the square sum of squares E (n) with the correlation value B (n). If the frequency of the standard periodic function having a large correlation effective value is selected as the representative frequency, the section signal X can be encoded using this representative frequency.
[0028]
That is, one or a plurality of standard frequencies whose correlation value E (n) is greater than or equal to a predetermined reference may be selected as the representative frequency. Here, the selection condition that “correlation value E (n) is greater than or equal to a predetermined reference” is, for example, a standard in which some threshold value is set and correlation value E (n) exceeds this threshold value. An absolute selection condition that all frequencies f (n) are selected as representative frequencies may be set. For example, up to the Qth in the order of the correlation value E (n) is selected. A relative selection condition may be set.
[0029]
(2.2. Method of generalized harmonic analysis)
Here, a generalized harmonic analysis technique useful when encoding an acoustic signal according to the present invention will be described. As already described, when encoding an acoustic signal, several representative frequencies having high correlation values are selected for the section signal in each unit section. Generalized harmonic analysis is a technique that enables the selection of representative frequencies with higher accuracy, and the basic principle thereof is as follows.
[0030]
Assume that there is a signal S (j) for the unit interval d as shown in FIG. Here, j is a parameter for repetitive processing (j = 1 to J), as will be described later. First, correlation values for all 128 periodic functions as shown in FIG. 2 are obtained for this signal S (j). Then, the frequency of one periodic function having the maximum correlation value is selected as a representative frequency, and the periodic function having the representative frequency is extracted as an element function. Subsequently, the inclusion signal G (j) as shown in FIG. 6B is defined. The inclusion signal G (j) is a signal obtained by multiplying the extracted element function by the correlation value of the element function with respect to the signal S (j) of the element function. For example, as shown in FIG. 2, when a frequency f (n) is selected as a representative frequency using a pair of sine function and cosine function as shown in FIG. 2, a sine function A (n) having an amplitude A (n). ) Sin (2πf (n) k / F) and a signal composed of the sum of cosine function B (n) cos (2πf (n) k / F) having amplitude B (n) is included signal G (j) (In FIG. 6B, only one function is shown for convenience of illustration). Here, since A (n) and B (n) are normalized correlation values obtained by the equation of FIG. 5, the inclusion signal G (j) is eventually included in the signal S (j). It can be said that the signal component has a certain frequency f (n).
[0031]
Thus, when the content signal G (j) is obtained, the difference signal S (j + 1) is obtained by subtracting the content signal G (j) from the signal S (j). FIG. 6C shows the difference signal S (j + 1) obtained in this way. The difference signal S (j + 1) can be said to be a signal composed of the remaining signal components obtained by removing the signal component having the frequency f (n) from the original signal S (j). Therefore, by increasing the parameter j by 1, this difference signal S (j + 1) is handled as a new signal S (j), and the same processing is performed J times while increasing the parameter j by 1 from j = 1 to J. If it is repeatedly executed, J representative frequencies can be selected.
[0032]
The J inclusion signals G (1) to G (J) output as a result of such correlation calculation are signals that are constituent elements of the original section signal X, and the original section signal X is encoded. In this case, information indicating the frequency of these J inclusion signals and information indicating the amplitude (intensity) may be used as the code data. Although J has been described as the number of representative frequencies, it may be the same as the number of standard frequencies f (n), that is, J = 128. For the purpose of obtaining a frequency spectrum, this is usually done. It is.
[0033]
Through the processing as described above, a frequency group that is a set of intensity values for each frequency is obtained for each unit section. When a predetermined number of frequency groups are selected in this way, “information indicating the pitch” corresponding to each frequency of this frequency group, and “sound strength corresponding to the signal intensity of each selected frequency”. Information indicating, “information indicating sound start time of sound” corresponding to the start point of the unit section, and “information indicating sound end time of sound” corresponding to the start point of the unit section subsequent to the unit section If a predetermined number of code data including information is created, the section signal X in the unit section can be encoded with the predetermined number of code data. If MIDI data is created as code data, a note number is used as “information indicating the pitch of the sound”, velocity is used as the “information indicating the intensity of the sound”, and “sound generation start time is set. The note-on time may be used as the “information indicating” and the note-off time may be used as the “information indicating the end time of sound generation”.
[0034]
(3.1. Frequency analysis method and acoustic signal encoding method according to the present invention)
Hereinafter, a frequency analysis method and an audio signal encoding method according to the present invention will be described. As described above, in the conventional method, a correlation value with respect to a fine frequency is obtained, and the maximum correlation value is used as the correlation value of the standard frequency in each standard frequency range. For example, consider a case where the spectrum of the original acoustic signal is in a state as shown in FIG. In FIG. 7, the horizontal axis represents frequency and the vertical axis represents intensity value or correlation value. In FIG. 7A, the sound included in the range of note number n (the range of standard frequency f (n)) and the sound included in the range of note number n + 1 are emitted. If the correlation about each fine frequency is calculated | required by the method of a short-time Fourier transform, it will come to show in FIG. In FIG. 7B, the fine frequencies are set at intervals obtained by dividing each standard frequency into 13. In the conventional encoding, the maximum correlation value is extracted within the range of each note number. At this time, in the range of each note number, what is present at the boundary is regarded as an influence of other signal components and is not extracted. Therefore, as shown in FIG. 7C, the maximum correlation value is extracted from the note number n, but not extracted from the note number n + 1.
[0035]
On the other hand, in the present invention, the maximum correlation value is not extracted in the range of each note number, but the correlation value that is a peak is extracted. The correlation value that is a peak means that the correlation value of the fine frequency on both sides of itself is smaller than itself. When a correlation value that is a peak is extracted, it is unlikely that the correlation value is affected by other signal components because the peak itself is a peak. Therefore, even if it exists in the boundary part of the range of a note number, it shall extract. Therefore, a signal component as shown in FIG. 7D is extracted from a set of correlation values as shown in FIG.
[0036]
Consider the second example as shown in FIG. If two sounds as shown in FIG. 8A are emitted, a correlation value corresponding to each fine frequency as shown in FIG. 8B is obtained by the technique of short-time Fourier transform. In the conventional method, as shown in FIG. 8 (c), it is extracted from the note number n but not from the note number n + 1. According to the present invention, FIG. As shown, frequency components corresponding to each note number are extracted.
[0037]
Next, the frequency analysis method of the present invention will be described with reference to the flowchart shown in FIG. 9, taking as an example the case of encoding an acoustic signal. First, a unit interval is set for a time-series acoustic signal, and a segment signal is extracted (step S1). As described above with reference to FIG. 1 in the above (Basic Principle) section, a unit section having a predetermined length is set and extracted.
[0038]
Next, the correlation between the extracted section signal and the harmonic signal prepared in advance is calculated (step S2). This harmonic signal is the same as the periodic function described in the above section (Specific method for obtaining correlation with periodic function). However, in the example of FIG. 2, 128 standard frequencies corresponding to the note number are set. However, in the present invention, 12 fine frequencies are set at intervals of a geometric series between the standard frequencies, and 128 in total. * 13 harmonic signals are prepared. For example, twelve fine frequencies from f (n + 1/13) to f (n + 12/13) are set between adjacent standard frequencies f (n) and f (n + 1). In this step S2, the correlation between the section signal and 128 × 13 harmonic signals is obtained by a short-time Fourier transform technique. As a result, correlation values corresponding to 128 × 13 fine frequencies are obtained. This correlation value is stored in a fine correlation array prepared in advance.
[0039]
Subsequently, a fine frequency that becomes a peak (hereinafter referred to as a peak frequency) is selected based on the calculated correlation value (step S3). Specifically, based on the calculated 128 × 13 correlation values, a fine frequency whose own correlation value is larger than the correlation values of the adjacent fine frequencies on both sides is selected.
[0040]
Next, the correlation between the harmonic signal corresponding to each selected peak frequency and the section signal is obtained by a generalized harmonic analysis technique (step S4). Specifically, in the same manner as described in the above section (Generalized Harmonic Analysis Method), first, the harmonic signal corresponding to the one having the maximum correlation value among the selected peak frequencies and the section signal S ( 1) Find the correlation with (= X). Subsequently, a signal obtained by multiplying the harmonic signal at this time by the correlation value calculated as its amplitude is set as a contained signal G (1), and this contained signal G (1) is subtracted from the section signal S (1). Thus, the difference signal S (2) is obtained. Next, among the peak frequencies selected in step S3, the correlation between the harmonic signal having the second highest peak frequency having the correlation value with the section signal X and the difference signal S (2) is obtained, and the inclusion signal G (2 ) Is subtracted from the section signal to obtain a differential signal S (3). In this way, the correlation values of all the selected peak frequencies are obtained by the generalized harmonic analysis method. Since this peak frequency corresponds to the fine frequency, it is necessary to return to the standard frequency corresponding to the note number at the time of encoding. Therefore, the calculated correlation value is set in the main correlation array as a correlation value corresponding to the standard frequency closest to the peak frequency. As a result, a plurality of sets of standard frequencies and their correlation values are obtained. By using the standard frequencies as note numbers and the correlation values as velocities, encoded data of the MIDI standard can be obtained.
[0041]
By performing the above processing for all unit sections, the entire acoustic signal can be encoded. As shown in FIG. 1, it is rare that the end time of the previous unit section and the start time of the subsequent unit section are set to be the same for the unit section. It is often set so that the following unit sections overlap.
[0042]
(3.2. Two-step process for determining extracted frequency components)
In steps S3 and S4, the peak frequency is selected only once, and the correlation value is calculated for each selected peak frequency by the generalized harmonic analysis method. The accuracy can be increased. Such a case will be described below. By repeatedly performing the process of subtracting the inclusion signal from the section signal X as in step S4, a different section signal X 'is obtained. The processes in steps S2 to S4 are repeated for this section signal X '.
[0043]
The fine frequencies that correlate with the section signal X 'are excluded from those that correspond to the standard frequencies whose correlation values are already stored in the main correlation array. For example, when the correlation value corresponding to the standard frequency f (n) is set in the main correlation array, the fine frequencies f (n−6 / 13) to f (n + 6/13) are the interval signal X ′. Are excluded from the target of correlation calculation. The correlation between the harmonic signal corresponding to the remaining fine frequency and the section signal X ′ is obtained by the short-time Fourier transform method, and the calculated correlation value is stored in the fine correlation array.
[0044]
Subsequently, in step S3, the peak frequency is selected again based on the calculated correlation value, and in step S4, the correlation between the selected peak frequency and the section signal X ′ is obtained, and then the calculated correlation is calculated. Set the value to the main correlation array.
[0045]
The processes in steps S2 to S4 can be repeated any number of times. However, when the process of encoding an acoustic signal is performed, about twice is sufficient. This is because if it is performed twice, about 64 sounds that can be sounded simultaneously by the MIDI standard can be extracted.
[0046]
(3.3. Determination of extracted frequency components within the same standard frequency)
In the determination of the standard frequency and the calculation of the correlation value in step S4, the standard frequency in the vicinity of the peak frequency was determined as corresponding to the calculated correlation value. This is not a problem when there is only one peak frequency included in one standard frequency range, but if there are multiple peak frequencies included in one standard frequency range, which one should be selected? Is a problem. Therefore, in the latter case, only the peak frequency having the maximum correlation value by the short-time Fourier transform is selected as the peak frequency among the peak frequencies included in one standard frequency, and the interval signal X and the interval signal X are selected by a generalized harmonic analysis technique. The correlation is obtained.
[0047]
(3.4. Section update width determination process)
The above frequency component extraction processing and encoding of the extracted frequency components are performed for each unit section. However, as described above, the setting of the unit section is not fixed and can be changed according to the situation. is there. At this time, how to set the length of one unit section is an important problem that greatly affects the analysis result. In other words, the shorter the section length, the better the time resolution, so it is convenient for analyzing frequency changes such as voice signals (vocal signals). There is a problem that the accuracy decreases. Therefore, in the past, a standard section length was determined in advance so that the number of samples (for example, 1024 samples) that would be most convenient for analysis was included in one unit section, and this fixed section In general, the time axis is mechanically divided based on the length.
[0048]
However, in the method of mechanically setting a fixed-length unit section, a unique section setting is always performed regardless of what kind of signal the time series signal to be analyzed is. There is a risk of discontinuity at the boundary of the unit section. That is, a large difference may occur in the frequency analysis results for adjacent unit sections. In general, when the obtained frequency analysis result is used for observing as a graph of a spectrum, even if a difference occurs in the analysis result at the boundary of the unit section, it does not cause a big problem. However, when the code data corresponding to the original sound signal is created using the analysis result of the sound signal, the discontinuity at the boundary becomes a big problem. When an original sound signal is to be reproduced based on data encoded by such a method, a sudden change in pitch occurs or a sound skip occurs at the boundary between unit intervals.
[0049]
In order to avoid such discontinuity at the boundary, a method of setting a unit interval by partially overlapping on the time axis has also been proposed, but since the amount to be overlapped is also fixed in this case, it is inherently overlapping. This results in a problem that redundant processing is increased even in places where it is not necessary to perform processing. In the present invention, when analyzing a time-series signal, a device section is devised for setting the unit section so as to keep the continuity of the sound at the boundary of the unit section and minimize the overlap of the unit sections. This will be described below.
[0050]
In particular, the present invention is characterized in that the unit interval is set by using the correlation value obtained by the short-time Fourier transform in step S2. Specifically, when the frequency component is extracted in a certain unit section, the value of the fine correlation array obtained first in step S2 is copied to the immediately preceding correlation array used for section determination. Then, the processing of step S2 to step S4 is performed a predetermined number of times to determine the frequency component to be extracted in one unit section, and then the position of the next unit section is set. In the present invention, since the unit section has a fixed length, the unit section is determined when the start position of the unit section is determined. Therefore, in the present invention, the subsequent unit section is determined by determining the feed width from the start position of the previous unit section to the start position of the subsequent unit section.
[0051]
First, the similarity between the previously determined unit section and the unit section whose position is to be determined is obtained (step S6). Specifically, in order to select a peak frequency, the value of the fine correlation array obtained as a result of the short-time Fourier transform performed in step S2 is used to calculate by the following [Equation 2].
[0052]
[Formula 2]
SM = 100-100 × [Σk {S_dn-1(k) -S_dn(k)}²]^1/2 / {ΣkS_dn-1(k)²}^1/2
[0053]
In the above [Equation 2], S_dn(k) is a correlation value of the fine frequency k with respect to the section signal of the unit section dn to be determined, and S_dn-1(k) is a correlation value of the fine frequency k with respect to the section signal of the immediately preceding unit section dn-1. However, k does not indicate the frequency itself, but is expressed by a value obtained by dividing the standard frequency corresponding to the note number n by 13, and takes a value of k = −6 to 127 × 13 + 6. The greater the similarity SM, the more similar the signals in both sections.
[0054]
Next, the value of the similarity SM is compared with a threshold value. When the similarity SM is smaller than the threshold 2 (step S7), the set feed width is halved (step S8). The temporary unit section is set again at the position where the feed width is halved (step S1), and the correlation between the section signal of the new temporary unit section and the harmonic signal is obtained (step S2). The processes of steps S1 and S2 and steps S6 to S8 are repeated until the similarity SM becomes equal to or greater than the threshold value 2. That is, the range in which the subsequent unit section overlaps the previous unit section is increased until the threshold value is 2 or more.
[0055]
When the similarity SM is equal to or greater than the threshold 2, the similarity SM is compared with the threshold 1. When the similarity is less than the threshold 1, the temporary unit section is determined as the unit section, and the process proceeds to the peak position selection process (step S3). When the similarity is equal to or greater than the threshold value 1, the feed width is set to double (step S10), the temporary unit section is determined as the unit section, and the process proceeds to the peak position selection process (step S3). Here, the feed width changed to double is used as the feed width of the next unit section. After the unit interval is determined, the processes in steps S3 and S4 are performed in the same manner, and the extraction frequency component is determined. As a matter of course, the above two threshold values satisfy the relationship of threshold value 1> threshold value 2.
[0056]
When the extraction frequency for the unit section is determined, a temporary unit section is set (step S1) with the position moved by the feed width set at this time as the start position (step S5). At this time, a standard feed width that is set in advance is normally used. If the feed width is set to double in step S10, the feed width is set to twice the standard feed width.
[0057]
Here, the set unit section will be described using a specific example shown in FIG.
FIG. 10A is a diagram illustrating time-series acoustic signals. As shown in FIG. 10B, a fixed-length unit section d1 is set from the head position of the acoustic signal, and the extraction frequency is determined by performing the processing from step S2 to step S4. In setting the next unit section d2, in step S5, it is sent after a preset standard feed width and temporarily set. In the present embodiment, the standard feed width is ½ of the fixed length d. Therefore, the unit section d2 is temporarily set so that the position shifted by d / 2 from the head of the unit section d1 becomes the head of the unit section d2. The state of the unit interval d2 temporarily set at this time is shown in FIG.
[0058]
In step S1, the acoustic signal at the position of the temporarily set unit section d2 is extracted as a section signal, and the correlation with the section signal is obtained by the method of step S2. In step S6, the similarity shown in [Formula 2] is calculated. If the calculated similarity is less than the threshold value 2, that is, if both section signals are not very similar, the process proceeds to step S8 and the feed width is halved. Here, since it is set to d / 2, it is set to d / 4. In this case, returning to step S1, the unit section d2 is temporarily set so that the position shifted by d / 4 from the head of the unit section d1 becomes the head of the unit section d2. The state of the unit interval d2 temporarily set at this time is shown in FIG. When the unit section d2 is temporarily set again, the processes of step S2 and step S6 are repeated again. Furthermore, if the similarity is less than the threshold value 2 in step S7, the feed width is set to d / 8 in step S8, and the position shifted by d / 8 from the head of the unit section d1 is returned to step S1. The unit section d2 is temporarily reset so as to be at the head of the section d2.
The state of the unit interval d2 temporarily set at this time is shown in FIG.
[0059]
That is, as shown in FIG. 10C to FIG. 10E, the unit interval d2 is the unit interval d1 until the similarity between the interval signals in the preceding unit interval d1 and the subsequent unit interval d2 is equal to or greater than the threshold value 2. Will increase the number of overlapping sections. However, there is a possibility that the unit section d1 and the unit section d2 will approach endlessly, so if the minimum feed width is set in advance and the feed width is less than the minimum distance, The unit section provisionally set in is determined as a formal unit section, and the process proceeds to step S3.
[0060]
On the other hand, when the similarity between the section signal of the unit section d1 and the section signal of the temporarily set unit section d2 is greater than or equal to the threshold 2 and less than the threshold 1, the temporarily set unit section d2 is determined. Then, the processing after step S3 is performed. Further, when the similarity is equal to or greater than the threshold value 1, the temporarily set unit section d2 is determined and the processing after step S3 is performed. In this case, the feed width is set to double. This process is also performed in parallel. Then, after the extraction frequency component for the unit section d2 is determined, in step S5, the position moved by the feed width set twice from the head of the unit section is set as the head position of the unit section d3. For example, when the unit section d2 is determined at the position shown in FIG. 10C, the unit section d3 is temporarily set at the position shown in FIG.
[0061]
Although the preferred embodiments of the present invention have been described above, the frequency analysis method and the encoding method are naturally executed by a computer or the like. Specifically, a program for executing the steps shown in the flowchart of FIG. 9 according to the above procedure is installed in the computer. Then, after time-series signals such as acoustic signals are digitized by the PCM method or the like, they are taken into a computer and processed in steps S1 to S10, and then the extracted frequency component or code data such as MIDI format is converted into digital data. Output from the computer. For example, in the case of MIDI data, the output code data is reproduced as sound using a MIDI sequencer and a MIDI sound source.
[0062]
【The invention's effect】
As described above, according to the present invention, as a frequency analysis method for separating a signal component corresponding to a predetermined standard frequency from a time series signal, a unit section having a predetermined length is set in the time series signal. To extract a correlation signal between all the harmonic signals and the section signal to calculate a correlation value corresponding to each fine frequency, and out of the fine frequencies corresponding to the calculated correlation value The peak frequency selection stage for extracting a plurality of fine frequencies that locally peak compared to the correlation value at the fine frequency, and recalculating the correlation between the harmonic signal corresponding to the fine frequency that becomes each peak and the interval signal, Set as a correlation value corresponding to the nearest standard frequency of the fine frequency, subtract the inclusion signal generated by the product of the recalculated correlation value and the harmonic signal from the interval signal, the difference signal In addition, since the interval signal is updated to the interval signal, the correlation value corresponding to the standard frequency is determined by sequentially performing the process of updating the interval signal on all the extracted peaks. Even when the fine frequency located at the peak reaches a peak, the frequency component can be accurately extracted. Furthermore, by encoding the extracted frequency component into a format such as MIDI, it is possible to create code data that can faithfully reproduce the original sound signal.
[Brief description of the drawings]
FIG. 1 is a diagram showing a basic principle of a frequency analysis method and an acoustic signal encoding method according to the present invention.
FIG. 2 is a diagram showing an example of a periodic function used in the present invention.
3 is a diagram showing a relational expression between the frequency of each periodic function shown in FIG. 2 and a MIDI note number n. FIG.
FIG. 4 is a diagram illustrating a method of calculating a correlation between a signal to be analyzed and a periodic signal.
FIG. 5 is a diagram showing a calculation formula for performing the correlation calculation shown in FIG. 4;
FIG. 6 is a diagram showing a basic method of generalized harmonic analysis.
FIG. 7 is a diagram showing a first example showing a comparison with a conventional method regarding selection of frequency components.
FIG. 8 is a diagram showing a second example showing comparison with the related art regarding selection of frequency components;
FIG. 9 is a flowchart of a frequency analysis method and an acoustic signal encoding method according to the present invention.
FIG. 10 is a diagram illustrating a procedure for setting unit intervals;
[Explanation of symbols]
d1 to d5 ... Unit interval
d: Unit section length
X: Initial section signal

Claims (7)

A frequency analysis method for separating a signal component corresponding to a predetermined standard frequency from a time series signal,
A harmonic signal preparation step of setting a plurality of fine frequencies at intervals finer than the interval of the standard frequency, and preparing a plurality of harmonic signal sets corresponding to each fine frequency;
Holds a main correlation array for storing correlation values corresponding to each standard frequency, a fine correlation array for storing correlation values corresponding to each fine frequency, and a fine correlation array obtained in the immediately preceding unit section A sequence preparation step for preparing a correlation sequence for determining an interval for
A section signal extraction step for setting a unit section of a predetermined length in the time series signal and extracting the section signal;
Calculating a correlation value corresponding to each fine frequency by calculating a correlation between all the harmonic signals and the section signal, and setting a correlation value calculated in the fine correlation array;
A peak frequency selection step for extracting a plurality of fine frequencies that locally peak compared to a correlation value at a surrounding fine frequency from the correlation array in the fine correlation array;
Recalculate the correlation between the harmonic signal corresponding to the fine frequency that becomes each peak and the section signal, set the main correlation array corresponding to the nearest standard frequency of the fine frequency, and the recalculated correlation value The content signal generated by the product of the harmonic signal is subtracted from the section signal, and the difference signal is newly used as the section signal to update the section signal sequentially for all the extracted peaks. A spectrum calculation stage for determining the value of the correlation sequence;
Calculates the similarity between the value of the correlation array for section determination obtained by copying the value of the fine correlation array in the previous unit section and the value of the fine correlation array in the temporarily set current unit section, and based on the similarity A similarity calculation stage for determining the feed width of the unit section,
The frequency analysis method according to claim 1, wherein in the section signal extraction step, a section signal is extracted from a unit section set at a position based on the determined feed width . The frequency analysis method according to claim 1 , wherein when the similarity calculated in the similarity calculation step is lower than a predetermined value, the feed width is set to be small. A frequency analysis method for separating a signal component corresponding to a predetermined standard frequency from a time series signal,
A harmonic signal preparation step of setting a plurality of fine frequencies at intervals finer than the interval of the standard frequency, and preparing a plurality of harmonic signal sets corresponding to each fine frequency;
An array preparation stage for preparing a main correlation array for storing correlation values corresponding to the respective standard frequencies and a fine correlation array for storing correlation values corresponding to the respective fine frequencies;
A section signal extraction step for setting a unit section of a predetermined length in the time series signal and extracting the section signal;
Calculating a correlation value corresponding to each fine frequency by calculating a correlation between all the harmonic signals and the section signal, and setting a correlation value calculated in the fine correlation array;
A peak frequency selection step for extracting a plurality of fine frequencies that locally peak compared to a correlation value at a surrounding fine frequency from the correlation array in the fine correlation array;
Recalculate the correlation between the harmonic signal corresponding to the fine frequency that becomes each peak and the section signal, set the main correlation array corresponding to the nearest standard frequency of the fine frequency, and the recalculated correlation value The content signal generated by the product of the harmonic signal is subtracted from the section signal, and the difference signal is newly used as the section signal to update the section signal sequentially for all the extracted peaks. a spectrum calculation step of determining the value of the correlation sequence, possess,
In the spectrum calculation step, when a plurality of fine frequencies having one standard frequency closest to each other are selected as peaks, only the fine frequency having the maximum correlation value is obtained as a correlation with the section signal. A frequency analysis method characterized by being a thing. The correlation operation step, the peak frequency selection step, and the spectrum calculation step, from claim 1 for one interval signal extracted from a certain unit section, characterized in that so as to perform a plurality of times The frequency analysis method according to claim 3 . The time series signal is an acoustic signal,
Sound pitch information corresponding to the standard frequency selected by the spectrum calculation step, sound intensity information corresponding to the correlation value, sound start time corresponding to the start point of each unit section, and each unit any one of claims 1 to 4, characterized by further comprising an encoding step of generating encoded data by performing a predetermined conversion to the four information consisting pronunciation end time of the sound corresponding to the end point of the section The frequency analysis method described in 1. A program for causing a computer to perform frequency analysis for separating a signal component corresponding to a predetermined standard frequency from a time-series signal, wherein a plurality of fine frequencies are set at intervals smaller than the intervals of the standard frequencies. Harmonic signal preparation stage for preparing a plurality of harmonic signal sets corresponding to each fine frequency, a main correlation array for storing correlation values corresponding to each standard frequency, and a correlation value corresponding to each fine frequency An array preparation step of preparing a correlation array for determining a section for holding the fine correlation array and the fine correlation array obtained in the immediately preceding unit section, setting a unit section of a predetermined length in the time series signal, An interval signal extraction step for extracting an interval signal, calculating a correlation value corresponding to each fine frequency by calculating a correlation between all the harmonic signals and the interval signal; A correlation calculation step for setting a correlation value calculated in a column; a peak frequency selection step for extracting a plurality of fine frequencies that locally peak compared to a correlation value at a surrounding fine frequency from the correlation array in the fine correlation array; Recalculate the correlation between the harmonic signal corresponding to the fine frequency that becomes each peak and the interval signal, set the main correlation array corresponding to the standard frequency nearest to the fine frequency, and the recalculated correlation value The main correlation is performed by sequentially performing the process of updating the interval signal by subtracting the inclusion signal generated by the product with the harmonic signal from the interval signal and newly setting the difference signal as the interval signal for all the extracted peaks. spectrum calculation step of determining the value of the array, the value of the section determining correlation sequence obtained by copying the value of the fine correlation sequences in the immediately preceding unit segment, it the current unit sections is temporarily set For calculating the similarity of the values of the fine correlation arrays and determining the feed width of the unit section based on the similarity, and causing the computer to execute the section signal extraction step. A program for extracting a section signal from a unit section set at a position based on the determined feed width . A program for causing a computer to perform frequency analysis for separating a signal component corresponding to a predetermined standard frequency from a time series signal, wherein a plurality of fine frequencies are set at intervals smaller than the intervals of the standard frequencies. Harmonic signal preparation stage for preparing a plurality of harmonic signal sets corresponding to each fine frequency, a main correlation array for storing correlation values corresponding to each standard frequency, and a correlation value corresponding to each fine frequency An array preparation stage for preparing a fine correlation array, a section signal extraction stage for extracting a section signal by setting a unit section of a predetermined length in the time series signal, and all the harmonic signals and the section signal A correlation calculation step of calculating a correlation value corresponding to each fine frequency and setting the calculated correlation value in the fine correlation array, and a correlation array in the fine correlation array A peak frequency selection stage that extracts a plurality of fine frequencies that locally peak compared to the correlation values at the surrounding fine frequencies, and recalculates the correlation between the harmonic signal corresponding to each of the fine frequencies and the interval signal. Set to the main correlation array corresponding to the nearest standard frequency of the fine frequency, subtract the inclusion signal generated by the product of the recalculated correlation value and the harmonic signal from the interval signal, the difference signal It is for causing a computer to execute a spectrum calculation stage in which a process of updating a section signal by newly setting a section signal is sequentially performed on all extracted peaks to determine a value of a main correlation array, In the spectrum calculation step, when a plurality of fine frequencies having one standard frequency closest to each other are selected as peaks, a fine value having the maximum correlation value among them is selected. A program characterized by wave number only, and is to determine the correlation between the interval signals.

Images