JP2013020166A - Encoding method and device of sound signal - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an encoding method and device of a sound signal, which are capable of encoding a sound signal played on the basis of a musical score, faithfully to the musical score by removing an influence of harmonics from the sound signal as much as possible.SOLUTION: A harmonic variable h is set to 2 to start processing from the second harmonic (S21). If n-12 logh being a note number corresponding to an h-th harmonic is equal to or larger than 0 (S22), a correlation value between an intensity corresponding to a note number n and an intensity corresponding to a note number (n-12 logh) corresponding to one-h-th frequency is calculated (S23). This processing is repeated up to an H-th harmonic to obtain a sum total V of intensities of harmonics from the second harmonic to the H-th harmonic (S22 to S25). The sum total V is multiplied by a correction value γ, and the multiplication result is subtracted from the intensity corresponding to the note number n, whereby H-th harmonic components are removed from the second harmonic (S26).

Description

  The present invention relates to an audio signal encoding technique, and more particularly to an encoding technique suitable for encoding into code data of MIDI format or the like.

  Conventionally, in order to be able to reproduce an acoustic signal using a MIDI sound source, the acoustic signal is converted into code data such as a MIDI code (see Patent Documents 1 to 5). Since a MIDI sound source is reproduced at a limited frequency such as 32 chords, it is necessary to select and encode a limited number of frequencies when encoding. The applicant has also proposed a technique for selecting and encoding a limited number of frequencies from an acoustic signal (see Patent Documents 1 and 5).

JP 2002-41037 A Japanese Patent No. 3776782 Japanese Patent No. 4061070 Japanese Patent No. 4156268 Japanese Patent Application No. 2011-43549

  However, in the invention described in Patent Document 1, signal components in the vicinity of a frequency having a high intensity selected previously are uniformly attenuated so that adjacent frequency components are not selected as much as possible. For this reason, the component that is originally necessary for expressing the consonant in the vocal reproduction is also deleted, and the reproducibility is lowered.

  In addition, the invention described in Patent Document 1 has a problem that overtones that are not played by the performer remain, and a code faithful to a musical score cannot be completely reproduced. The applicant has proposed a method for removing overtones (see Patent Document 2), but accurate overtone removal has not been realized.

Therefore, the present invention provides an audio signal encoding method and apparatus that can remove the influence of overtones as much as possible from an audio signal played based on a music score and can encode it in a state faithful to the music score. The issue is to provide.

  In order to solve the above-described problem, in the first aspect of the present invention, in encoding for encoding an acoustic signal given as a sample sequence of J time series digitized at a predetermined sampling frequency, the sample is used. A unit interval composed of a predetermined number T (T <J) samples is set for a column while overlapping a predetermined number W (W <T) samples in the time axis direction with an adjacent unit interval. For each unit section, by performing frequency conversion on at least N types of frequencies f (n) to be analyzed, the spectrum intensity corresponding to the N types of frequencies is calculated for each unit section. When there is a spectrum intensity corresponding to f (n) / k that is a frequency that is a fraction of an integer k (k ≧ 2) for each of the N types of frequencies f (n), f (n ) / K Correction for attenuating the spectrum intensity corresponding to the frequency f (n) by the amount obtained by multiplying the spectrum intensity by the correction intensity γ is performed in order from the lowest frequency f (n) to create a corrected spectrum intensity, and the frequency f (n) The higher one uses a previously corrected spectral intensity to attenuate the spectral intensity, corrects the time difference between the head time of the unit section and the head time of the immediately following unit section, and the correction of the unit section. A code code of a predetermined format is generated based on the spectrum intensity.

  According to the first aspect of the present invention, frequency conversion is performed on each unit section to calculate spectrum intensities corresponding to N types of frequencies, and for each of the N types of frequencies f (n), an integer k ( When there is a spectrum intensity corresponding to f (n) / k, which is a frequency of k ≧ 2), the frequency f is equal to the spectrum intensity corresponding to f (n) / k multiplied by the correction intensity γ. Since the correction for attenuating the spectrum intensity corresponding to (n) is performed in order from the lowest frequency f (n) and the code code is generated based on the information after the attenuation correction, an acoustic signal whose fundamental tone is limited Overtones can be accurately removed in order from the lower sound component, and as a result, harmonics based on the lower sound component (fundamental tone) corrected from the higher sound component containing many overtones based on multiple fundamental sounds Can be removed with high accuracy, It is possible to perform the reproduction coded to.

  Further, in the second aspect of the present invention, in the first aspect of the present invention, the calculation of the spectral intensity is performed for frequency conversion for at least N types of frequencies f (n) to be analyzed for each unit section. To calculate a first spectral intensity E1 (p, n) corresponding to the N types of frequencies for the unit interval p, and a unit interval p- positioned immediately before the unit interval p. The unit interval p is q (q ≦ p) only when the evaluation value based on the corresponding frequency change with the first spectral intensity E1 (p−1, n) in 1 is larger than a predetermined threshold value. ) To perform frequency conversion with higher accuracy than the frequency conversion in the first spectrum calculation step for at least N types of frequencies f (n) to be selected and analyzed as the first selection unit interval q. The second spectrum intensity E2 (q, n) is calculated as the spectrum intensity by calculating the second spectrum intensity E2 (q, n) corresponding to the N types of frequencies, In the spectrum correction, correction is performed so as to attenuate the second spectrum intensity E2 (q, n) by a predetermined ratio, and a corrected spectrum intensity E ′ (q, n) is created as the corrected spectrum intensity. It is characterized by doing so.

  According to the second aspect of the present invention, a simple first frequency conversion is performed for each set unit section, and the selection is performed when the intensity is greater than a predetermined reference compared to the immediately preceding unit section. Since it was selected as a unit interval, and the second frequency conversion with higher accuracy was performed on the selected unit interval, and a code code was generated based on the obtained analysis result, the entire acoustic signal was fixed at a fixed interval. Thus, only the characteristic part is encoded while analyzing the information over time, so that it is possible to more appropriately analyze the acoustic signal including chords and the frequency change of the audio signal.

  Further, in the third aspect of the present invention, in the second aspect of the present invention, the first spectrum calculation and the second spectrum calculation do not exceed frequencies adjacent to each of the N types of frequencies f (n). A predetermined M types of sub-frequency f (n, m) are set in a range, and the M types of sub-frequency f1 (p, n) and second spectrum strength E2 (q, n) are set as the first spectrum strength E1 (p, n). An intensity value corresponding to a sub-frequency indicating the greatest intensity among the frequencies is calculated.

  According to the third aspect of the present invention, by setting the frequency interval to be analyzed finely, it is possible to perform more accurate frequency analysis and specify an appropriate frequency component.

  In addition, in the fourth aspect of the present invention, when the code code is generated in the second or third aspect of the present invention, the selection unit section with respect to two adjacent selection unit sections q and selection unit section q + 1. When q is the p-th unit interval p, it is defined as P (q) = p and corresponds to the frequencies f (n), f (n−1), and f (n + 1) in the selected unit interval q + 1. Any one of the first spectral intensities E1 (P (q + 1), n), E1 (P (q + 1), n-1), E1 (P (q + 1), n + 1) and a position immediately before the selected unit interval q + 1 A difference from the first spectral intensity E1 (P (q + 1) -1, n) corresponding to the frequency f (n) in the unit interval P (q + 1) -1 to be less than a predetermined threshold Ldif, and First spectral intensity E1 (P (q + 1), n) Any of E1 (P (q + 1), n−1), E1 (P (q + 1), n + 1) and the first spectral intensity E1 (P (q + 1) −1, n) is greater than a predetermined threshold value Lmin. If it is larger, the selection unit interval q and the selection unit interval q + 1 are connected, and the start time of the selection unit interval q defined in the selection unit interval q and the selection unit interval q + 1 It is characterized in that the time difference from the head time of the immediately subsequent selection unit section q + 2 is used.

  According to the fourth aspect of the present invention, when the code code is generated, the difference in intensity between the succeeding selected unit section and the immediately preceding unit section of the two adjacent selected unit sections is less than the predetermined threshold value. When the intensity of the subsequent selected unit section and the intensity of the previous unit section are both greater than a predetermined threshold, the two adjacent selected unit sections are connected so that the time is not selected. Therefore, the closest unit section is taken into consideration when determining the connection, and the sound components can be appropriately connected.

  According to a fifth aspect of the present invention, in the fourth aspect of the present invention, the first spectral intensities E1 (P (q + 1), n), E1 (P (q + 1), n-1), E1 (P ( q + 1), n + 1) corresponding to at least one of the values mmax1, mmax2, mmax3 corresponding to the sub-frequency and the sub-frequency determining the first spectral intensity E1 (P (q + 1) -1, n) The selection unit section q and the selection unit section q + 1 are connected only when the condition that the difference from the value mmax0 is less than a predetermined threshold value Ndif is further satisfied. As a value corresponding to the sub-frequency, for example, a cent which is 1/100 unit of a note number in the MIDI standard (in the embodiment described later, a slightly coarse 1/12 note number) can be adopted. In the MIDI standard, it is possible to express minute musical instrument pitch fluctuations between note numbers by pitch bend in cent units.

  According to the fifth aspect of the present invention, it is possible to perform more accurate frequency analysis by finely setting the frequency interval to be analyzed, and further, as a sound component connection condition, the subsequent selection unit section and its immediately preceding section Since the difference from the sub-frequency of the unit interval is less than the threshold value, the sound components can be connected based on the analysis result with higher accuracy.

  Also, in the sixth aspect of the present invention, in the fifth aspect of the present invention, when the selection unit section q is already connected to another selection unit section, the head of the selection unit section q is connected. The selected unit interval is qo, and the first spectral intensity E1 (P (q + 1), n), E1 (P (q + 1), n-1), E1 (P (q + 1), n + 1) is determined as a sub-frequency. The difference between at least one of the corresponding values mmax1, mmax2, and mmax3 and the value mmaxo corresponding to the sub-frequency that determines the first spectral intensity E1 (P (qo), n) is a predetermined threshold value Nadif. The selection unit interval q and the selection unit interval q + 1 are connected only when the condition of less than is further satisfied.

  According to the sixth aspect of the present invention, as the sound component connection condition, when the preceding selection unit interval is already connected to another selection unit interval, the subsequent selection unit interval and the immediately preceding selection unit Since the difference between the sub-frequency of the first selection unit section to which the section is connected is less than the threshold value, the subsequent selection unit section belonging to the sound component whose sub-frequency changes slowly and has a different frequency is added. It is possible to prevent erroneous connection and to realize more accurate connection of sound components.

  Also, in a seventh aspect of the present invention, in any one of the fourth to sixth aspects of the present invention, the code code expresses that there are more than a predetermined upper limit of sounding intervals at a certain time t. In the case of each sounding interval, an attenuation value is calculated by attenuating the intensity value based on the elapsed time from the start time of the selected unit interval to the time t, and the attenuation intensity value is minimum at the time t. The sounding section of the code code is corrected so as to be shortened so that the sounding section ends at the time t.

  According to the seventh aspect of the present invention, when the code code represents that there are more than a predetermined upper limit number of sounding intervals at a certain time t, for each sounding interval, from the start time of the selected unit interval An attenuation value is calculated by attenuating the intensity value based on the elapsed time up to time t, and the sounding section of the code code is shortened so that the sounding section having the minimum attenuation intensity value at time t is terminated at time t. Therefore, it is possible to simultaneously generate as many sounds as possible at each time within the limitation of the number of simultaneous sounds.

  Further, in an eighth aspect of the present invention, in any one of the second to seventh aspects of the present invention, the first spectrum calculation is performed using N types of element signals to be constituent elements of the section signal of the unit section. , Each corresponding to an integral multiple of the period of the frequency f (n), prepared as T (n) samples closest to the T, and element signals corresponding to the N frequencies f (n); A first spectrum intensity E1 (p, n) is calculated by performing a correlation operation with a section signal composed of T (n) samples of the corresponding unit section p, and the second spectrum. Is calculated by correlating the prepared element signal corresponding to each of the N frequencies f (n) and the section signal composed of T (n) samples of the corresponding selection unit section q. Calculate the frequency f () with the highest correlation value element) corresponding to max) is selected as a harmonic signal, and T (nmax) samples given by the product of the selected harmonic signal and the correlation value obtained for the harmonic signal are used as the inclusion signal, By subtracting the signal from the interval signal, a differential signal composed of T (nmax) samples is obtained by calculation, and T (n) samples updated to reflect the T (nmax) samples are obtained. As the new interval signal, the selection of the harmonic signal and the calculation of the difference signal are performed to repeatedly obtain the new inclusion signal and the difference signal, thereby obtaining N inclusion signals, and the correlation of the obtained inclusion signals. A second spectrum intensity E2 (q, n) corresponding to the N types of frequencies is calculated based on the value.

  According to the eighth aspect of the present invention, the first spectrum calculation for all unit intervals is performed by simple discrete Fourier transform, and the second spectrum calculation for the selected unit interval is performed by high-precision generalized harmonic analysis. As a result, it is possible to efficiently obtain information on the selected unit section with high accuracy while referring to the analysis results of all the unit sections.

  Further, in the ninth aspect of the present invention, in the eighth aspect of the present invention, the calculation of the correlation in the first spectrum calculation corresponds to each frequency f (n) in the unit interval p-1 positioned immediately before. A correlation calculation corresponding to the first W sample in the unit interval p-1 is performed on the previous correlation calculation result, and a correlation value for each frequency is subtracted from the previous correlation calculation result, and T (n in the unit interval p ) Correlation calculation corresponding to each frequency f (n) in the unit interval p by performing correlation calculation corresponding to the last W sample in the sample and adding the correlation value for each frequency to the previous correlation calculation result This is performed by obtaining a result and calculating the first spectral intensity E1 (p, n) based on the correlation calculation result.

  According to the ninth aspect of the present invention, when performing a simple correlation calculation for each unit section in the first spectrum calculation, the correlation calculation result performed for the previous unit section is used, and the previous correlation calculation result is calculated. Since the head part is removed and the correlation calculation is performed on the tail of the unit interval, and the result is added to the previous correlation calculation result, most of the correlation calculation result of the previous unit interval is used. It is possible to speed up the arithmetic processing for all unit sections.

  According to the present invention, it is possible to remove the influence of harmonics as much as possible from an acoustic signal played based on a musical score and to encode it in a state faithful to the musical score.

It is a hardware block diagram of the encoding apparatus of the acoustic signal in one Embodiment of this invention. It is a flowchart which shows the outline | summary of the encoding method of the acoustic signal which concerns on one Embodiment of this invention. It is a figure which shows the concept of the expansion of a time-axis direction, the increase in a frequency, and reduction | decrease of time information. It is a figure which shows the concept of the area setting of this embodiment compared with the prior art. It is a figure which shows the relationship of the logical / physical range of the analysis frequency in this embodiment. It is a figure which shows the relationship between the unit area and analysis range in this embodiment. It is a figure which shows the mode of the analysis process of the unit area in this embodiment. It is a figure which shows the correspondence of the sample row | line in the unit area extracted from the acoustic signal after a time-axis expansion process, and a harmonic signal. It is a figure explaining the time-axis extension of the analysis frame in case unit interval length T is more than 1/2 period of a harmonic signal and less than 3/4 period. It is a figure explaining the time-axis extension of an analysis frame in case unit interval length T is 1/4 period or more and less than 1/2 period of a harmonic signal. It is a flowchart which shows the detail of the harmonic overtone removal in S8 of FIG. It is a figure which shows the relationship between the selection unit area used as the object of a connection judgment, and a unit area. It is the figure which compared the concept of the adjustment of the number of chords in this embodiment with the past. It is the figure which compared the concept of adjustment of the number of chords in this embodiment with the conventional case in the case of compound melody. It is a flowchart which shows the detail of the chord number adjustment process in S11 of FIG. It is a flowchart which shows the detail of the minimum sound deletion from the note on table in S37 of FIG. It is a figure which shows the original sound waveform of the acoustic signal made into encoding object. It is a figure which shows the result of having performed MIDI encoding, without performing an overtone removal with respect to the acoustic signal shown in FIG. It is a figure which shows the state which converted the encoding result by the conventional method shown in FIG. 18 into the score. It is a figure which shows the result of having performed the MIDI encoding with respect to the acoustic signal shown in FIG. 17 by the conventional overtone removal method. It is a figure which shows the result of having performed the MIDI encoding by the conventional chord number reduction method with respect to the acoustic signal shown in FIG. It is a figure which shows the result of having performed MIDI encoding using the harmonic overtone removal by this invention. It is a figure which shows the result of having performed the MIDI encoding using the harmonics removal by this invention, making the maximum chord number one. It is a figure which shows the state which converted the encoding result shown in FIG. 23 into the score.

DESCRIPTION OF EMBODIMENTS Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the drawings.
FIG. 1 is a hardware configuration diagram of an audio signal encoding device according to an embodiment of the present invention. The audio signal encoding apparatus can be realized by a general-purpose computer. As shown in FIG. 1, a CPU 1 (CPU: Central Processing Unit) and a computer main memory RAM 2 (RAM: Random Access Memory) A large-capacity data storage device 3 (for example, a hard disk) for storing data, a program storage device 4 (for example, a hard disk) for storing a program executed by the CPU, and a key input I such as a keyboard and a mouse / F5, a data input / output interface 6 for data communication with an external device (data storage medium), and a display output interface 7 for sending information to a display device (display), each via a bus It is connected.

  The program storage device 4 of the acoustic signal encoding device is mounted with a dedicated program for operating the CPU 1 and causing the computer to function as the acoustic signal encoding device. By executing the dedicated program, the CPU 1 can select the section setting means, the first spectrum calculation means (including element signal preparation means and correlation calculation means), and the second spectrum calculation means (element signal preparation means, harmonic signal selection). Functions as a spectrum correction means and an encoding means. The data storage device 3 stores various data necessary for processing.

  FIG. 2 is a flowchart showing an outline of an audio signal encoding method according to this embodiment. The acoustic signal encoding method according to the present embodiment is performed by the acoustic signal encoding apparatus shown in FIG. 1 executing the steps (stages) shown in FIG.

  First, an audio signal encoding apparatus reads a digital audio signal to be processed from a data input device via a data input / output I / F 6. The digital audio signal is obtained by sampling an analog audio signal at a predetermined sampling frequency and the number of quantization bits. In the present embodiment, the sampling is performed with a sampling frequency of 44.1 kHz and a quantization bit number of 16 bits as an example. I will explain. When sampling is performed at a sampling frequency of 44.1 kHz, the digital acoustic signal is configured as a sample row (sample array: intensity array) having 44100 samples (intensity values) per second.

  When the digital audio signal is read, the audio signal encoding apparatus expands the digital audio signal by a predetermined magnification K (K is an integer) in the time axis direction (S1). Specifically, the number of samples constituting the digital acoustic signal is multiplied by K. Then, every K samples having the same value as the original sample are arranged, and (K-1) sample values between them are linearly interpolated using the values of the original sample located on both sides. give. Assuming that the sample value for each sample j (j = 0... J-1) of the original sound signal is x (j), the computer expands by executing the processing according to the following [Equation 1]. A sample value x ′ (j · K + k) for each sample j · K + k (0 ≦ k ≦ K−1) of the subsequent acoustic signal is calculated. In the following [Equation 1], w is a real value taking a value of 0 ≦ w ≦ 1 given by k / K.

[Formula 1]
x ′ (j · K + k) = (1−w) · x (j) + w · x (j + 1)

  As a result of the processing in S1, J samples constituting the digital audio signal are expanded to J × K. FIG. 3A shows a change in waveform due to the enlargement process in S1. The waveform in FIG. 3A is obtained by connecting sampled values plotted with line segments, but is expressed in a curved line due to the large number of samples. By executing the processing according to the above [Equation 1], the waveform shown on the left side changes to the waveform shown on the right side. In the example of FIG. 3, the case of K = 2 is shown for convenience of explanation. In this way, by performing analysis by expanding the acoustic signal in the time axis direction, the time resolution in the analysis is improved while maintaining the same frequency analysis accuracy as when performing the analysis directly on the original acoustic signal. Thus, it is possible to extract frequency fluctuations with high accuracy.

  The acoustic signal encoding apparatus performs frequency analysis on a predetermined section in subsequent S2 to S5. In the present embodiment, the section to be subjected to frequency analysis is set by selecting a unit section satisfying a predetermined condition as the selected unit section after setting the unit section. The concept of setting the section to be analyzed in the present embodiment will be described in comparison with the prior art represented by Patent Document 3. FIG. 4 is a diagram showing a concept of section setting of the present embodiment compared with the prior art. As will be described later, both the conventional case and the present embodiment are the same in that a predetermined number of samples obtained by sampling are set as unit sections and frequency analysis is performed for each unit section. As shown in FIG. 4A, in the prior art disclosed in Patent Document 3, a zero crossing point is specified, and the setting position of the unit section is made variable using the zero crossing point. In FIG. 4A, three unit sections are set as an example, but the intervals between these start positions (left ends) are not uniform. Then, both discrete Fourier transform and generalized harmonic analysis are executed for each set unit interval to obtain an analysis result.

  On the other hand, as shown in FIG. 4B, in this embodiment, unit intervals are set at fixed intervals, and discrete Fourier transform is performed on each unit interval to obtain an analysis result. Then, the analysis result is compared with the immediately preceding unit section, and when a predetermined condition is satisfied, it is selected as the selected unit section. In the example of FIG. 4B, unit sections 1, 5, and 6 are selected as selection unit sections 1, 2, and 3, respectively. Then, a generalized harmonic analysis is performed on the selected unit section to obtain an analysis result.

  Specifically, first, the audio signal encoding apparatus sets a unit interval on the sample expanded in the time axis direction (S2). The length of the unit interval (number of samples T) is set in relation to the sampling frequency. However, if the sampling frequency is 44.1 kHz, 4096 samples or more are required to faithfully analyze the low frequency region. . Therefore, in this embodiment, the unit interval is set as the number of samples T per unit interval T = 4096.

  Setting of the unit section is performed by sequentially extracting samples from the head of the digital sound signal as disclosed in Patent Documents 1 and 4. The unit interval is set so as not to leak all samples, and is preferably set so that the samples overlap in continuous unit intervals. In the present invention, the head interval (referred to as shift width) of each unit section is set as a fixed value. That is, the number of samples to be overlapped is set to be constant. In the present embodiment, the shift width W = 64 is a fixed value. Accordingly, when T = 4096, the first unit interval is j = 0 to 4095, the second unit interval is j = 64 to 4159, the third unit interval is j = 128 to 4223, and so on. It will be set with duplicate samples. The value x (j) of each sample is expressed as a value x (p, i) (0 ≦ i ≦ T−1) for each unit interval p (p is an integer of 0 or more).

Next, discrete Fourier transform, which is the first frequency analysis, is executed for each set unit section, and the spectrum of each unit section is calculated (S3). As disclosed in Patent Documents 1, 3, and 4, the spectrum of each unit section is calculated with 128 analysis frequencies f (n) = 440 · 2 ⁽ⁿ⁻⁶⁹⁾ corresponding to MIDI note number n. ^{This is} done by extracting 128 components by a discrete Fourier transform based on a ^{/ 12} harmonic signal (harmonic function). “128 types” and “128 pieces” are examples. For example, in the case of the MIDI standard, it corresponds to the range of note number n = 0 to 127, but the standard range for reproducing a grand piano is note number n. The range is from 21 to 108. Therefore, in this case, 88 components are extracted using 88 types of analysis frequencies. Further, as described above, when the frequency analysis is performed by enlarging the acoustic signal by a predetermined magnification K in the time axis direction, if the time axis expansion process described later is omitted, the analysis cannot be performed at the analysis frequency located in the bass portion. Therefore, the types of analysis frequencies are further reduced. For example, when K = 4, the analysis frequency is limited to a range of note numbers n = 24 to 127 (104 types) even in the MIDI standard, and note numbers n = 24 to 108 (85 types) in the standard range of the grand piano. ).

In the present embodiment, as the acoustic signal is expanded K times in the time axis direction, the upper and lower limits of n are respectively moved downward by α. α is an integer defined by α = 12 · log ₂ K (for example, α = 24 when K = 4). Therefore, in Patent Documents 1, 3, and 4, 0 ≦ n ≦ 127, but in this embodiment, −α ≦ n ≦ 127−α. Thereby, the frequency of each harmonic signal is set to 1 / K times. Here, the relationship of the logical / physical range of the analysis frequency in this embodiment is shown in FIG. As shown in FIG. 5, the standard sound range of the grand piano is in a range of n = 21 to 108. Therefore, when performing a normal analysis, it is performed in a range of n = 21 to 108. However, in the present invention, the time axis is expanded to shift the frequency to the bass side and perform analysis processing. For note number n = 21 or less, since one cycle of the corresponding harmonic signal is longer than the unit interval, a long cycle analysis is performed by extending the time axis (described later). As a result, frequency components are obtained for n = −3 to 84, but by performing correction processing finally, range frequency components of n = 21 to 108 are obtained.

When the analysis frequency is set in correspondence with the note number n, the frequency interval between the note numbers becomes wider as the frequency becomes higher. In particular, when n exceeds 60, the analysis accuracy decreases. Therefore, in the present embodiment, as disclosed in Patent Document 4, 128M element signals f (n, m) = 440 · 2 ⁽ⁿ⁻ ) where the note numbers are divided into M differential sounds (sub-frequency). ^{69 + m / M) / 12} , and 128M components are extracted. Unless special encoding such as addition of a pitch bend code is performed in the code code generation process of S10 described later, information on M differential sounds at each note number is unnecessary, so the maximum value of the components of M differential sounds Is represented as a component in the note number, and as a result, 128 components are extracted.

  As a specific processing procedure by the acoustic signal encoding apparatus, first, for each unit section p, an intensity value array E1 (p, n) (−α ≦ n ≦ 127−α) and a sub-number for each note number are subtracted. The frequency array S (p, n) is set and all initial values are set to 0. Subsequently, the processing according to the following [Formula 2] is executed for -α ≦ n ≦ 127-α and 0 ≦ m ≦ M−1 to maximize E1 (p, n, m) (nmax , Mmax). However, if the bass portion is not required due to the relationship with the processing load, the processing load can be reduced by omitting the processing for n <0.

[Formula 2]
A (p, n, m) = (1 / T (n)). Σi _{= 0, T (n) −1} x (p, i) sin (2πf (n, m) (i + pW) / fs)
B (p, n, m) = (1 / T (n)). Σi _{= 0, T (n) −1} x (p, i) cos (2πf (n, m) (i + pW) / fs)
E1 (p, n, m) = {A (p, n, m)} ² + {B (p, n, m)} ²

  In the above [Equation 2], T (n) is the analysis frame length. When one period of the harmonic signal (harmonic function) is equal to or shorter than the unit section length T, the maximum period of the harmonic signal is within a range not exceeding the unit section length T. Set to be an integer multiple of. However, in the present invention, the time axis is expanded to shift the frequency to the bass side and the analysis process is performed, so that one period of the harmonic signal (harmonic function) exceeds the unit interval length T. Specifically, when one period of the harmonic signal is larger than the unit section length T, it is given by T (n) = g × fs / f (n, m), and the value of x (i) at T <T (n) Is set based on the time axis extension process described later. Note that g is an integer value of 1 or more, and fs is a sampling frequency (for example, 44.1 kHz).

  The processing according to the above [Equation 2] is executed for each unit section, and A (p, n, m), B (p, n, m), E1 (p, n, m) can be obtained. It is. Here, the relationship between the unit interval and the analysis range in the present embodiment is shown in FIG. In FIG. 6, the waveform at the upper end schematically shows the original sound signal, and the waveform at the lower end schematically shows the harmonic function. In the example of FIG. 6, only the target unit section that is the target unit section and the immediately preceding unit section that is the immediately preceding unit section are shown, but each correlation calculation range has a rectangular horizontal length. Become. In this embodiment, since the correlation calculation range T is 4096 samples and the shift width W is 64 samples, the overlapping portion is very large. Therefore, in the present embodiment, the efficiency of the analysis process is improved by using the analysis result in the immediately preceding unit section for the overlapping portion.

  FIG. 7 shows a state of the unit interval analysis processing in the present embodiment. As shown in FIG. 7, when obtaining the analysis result in the target unit section, the overlapping part of the immediately preceding unit section is used. Specifically, the head part of the previous unit section that does not overlap with the target unit section is deleted, and only the tail part of the target unit section that does not overlap with the previous unit section is added by performing correlation calculation. Therefore, the correlation calculation is performed only for the head unit interval (p = 0) over the entire unit interval.

  When p ≧ 1, that is, when processing for the second and subsequent unit intervals p, A (p−1, n, m) and B (p−1, n, m) for the immediately preceding unit interval (p−1). m) has already been calculated. In the present embodiment, by using A (p−1, n, m) and B (p−1, n, m), the processing according to the following [Equation 3] is executed, so that the unit interval p is obtained. A (p, n, m) and B (p, n, m) are calculated.

[Formula 3]
A (p, n, m) = A (p-1, n, m) − (1 / T (n)) · Σ _{i = 0, W−1} x (p−1, i) sin (2πf (n , M) (i + (p-1) W) / fs) + (1 / T (n)). Σi _{= T (n) -W, T (n) -1} x (p, i) sin (2πf (N, m) (i + pW) / fs)
B (p, n, m) = B (p-1, n, m) − (1 / T (n)) · Σ _{i = 0, W−1} x (p−1, i) cos (2πf (n , M) (i + (p-1) W) / fs) + (1 / T (n)). Σi _{= T (n) -W, T (n) -1} x (p, i) cos (2πf (N, m) (i + pW) / fs)
E1 (p, n, m) = {A (p, n, m)} ² + {B (p, n, m)} ²

  Subsequently, for each note number n, (p, n, mmax) that maximizes E1 (p, n, m) in the range of 0 ≦ m ≦ M−1 is obtained, and E1 (p, n) = E1. Processing is performed such that (p, n, mmax) and S (p, n) = mmax. Then, the calculated E1 (p, n) and S (p, n) are temporarily stored in the memory. E1 (p, n) and S (p, n) temporarily stored in the memory are used in a single-tone component connection process to be described later.

  Next, a change between the spectral intensity E1 (p, n) calculated in the unit interval p and the spectral intensity E1 (p-1, n) calculated in the immediately preceding interval (p-1) is evaluated (S4). ). Specifically, first, a change evaluation value dE (p−1, p) with the immediately preceding section (p−1) of the unit section p is calculated by executing processing according to the following [Formula 4]. .

[Formula 4]
dE (p−1, p) = (100 / N) · Σ _{n = 0, N−1} {| E1 (p, n) −E1 (p−1, n) | / (E1 (p, n) + E1 (p-1, n))}

  If the obtained change evaluation value dE (p−1, p) is less than the predetermined threshold value, p ← p + 1 is returned to S2, and the next unit interval p is set. The predetermined threshold value can be set as appropriate, but when [Equation 4] is used, since it is normalized by 100, the threshold value is set to 40.

  On the other hand, when the obtained change evaluation value dE (p−1, p) is equal to or greater than a predetermined threshold value, the unit interval p is selected as the selected unit interval q, and the generalized harmony is selected for the selected unit interval q. Analysis is performed (S5). The value of q is 0 for the first selected unit interval, and thereafter, a value obtained by adding 1 each time it is selected is given. Specifically, first, E1 (p, nmax) that maximizes E1 (p, n) set in S3 is obtained. That is, among all n of −α ≦ n ≦ 127−α, the value of n that maximizes E1 (p, n) is obtained as nmax, and E1 (p, n) at that time is determined as E1 (p, n). nmax). This is performed by executing the process of [Formula 2] for all n and selecting the largest of the calculated n E1 (p, n). Further, using the obtained nmax, mmax = S (p, nmax) is set.

  Then, A (p, nmax, mmax) and B (p, nmax, mmax) are calculated by executing processing according to the following [Equation 5] using the obtained nmax and mmax. When executing the processing according to [Formula 5], first, assuming that the unit interval p is the q-th selected unit interval q, P (q) = p is set, and the selected unit interval q , The correlation intensity array E2 (q, n) for the note number is defined, and the initial values are all set to values less than 0 (for example, −1).

[Formula 5]
A (p, nmax, mmax) = (1 / T (nmax)) · Σi _{= 0, T (nmax) −1} x (p, i) · sin (2πf (nmax, mmax) i / fs)
B (p, nmax, mmax) = (1 / T (nmax)) · Σi _{= 0, T (nmax) −1} x (p, i) cos (2πf (nmax, mmax) i / fs)
E2 (q, nmax) = {A (p, nmax, mmax)} ² + {B (p, nmax, mmax)} ²

  Then, by using the calculated A (p, nmax, mmax) and B (p, nmax, mmax), a process in accordance with the following [Equation 6] is performed, whereby a sample (p , I), the value x (p, i) is updated over 0 ≦ i ≦ T (nmax) −1.

[Formula 6]
x (p, i) ← x (p, i) −A (p, nmax, mmax) · sin (2πf (nmax, mmax) i / fs) −B (p, nmax, mmax) · cos (2πf (nmax , Mmax) i / fs)

  The process of [Formula 6] is a process of removing the contained signal from the original acoustic signal. A new E2 (q, nmax) that maximizes E2 (q, n) is obtained for n other than the processed nmax value for the acoustic signal after removing the contained components, and the new nmax is obtained. Is used to execute processing according to [Equation 5] and [Equation 6]. As a result, the contained signal is further removed from the acoustic signal. The audio signal encoding apparatus performs such processing on all 128 n's to obtain E2 (q, n).

  In the present embodiment, in order to reduce the processing load, the value of M is variably set based on the note number, for example, the frequency interval to be analyzed is about 100 Hz. And note number 60 and below are not divided and M = 1. Further, although the accuracy is slightly reduced, S (p, n) is determined by the processing of [Equation 2] for determining the correlation strength array E1 (p, n), and the correlation strength array E2 (q, n) is determined. The processing of [Formula 5] for this purpose may be performed with m = S (p, n) fixed, and the differential sound analysis may be omitted. Further, in the processing of [Equation 5], there is a possibility that a signal component having a different sub-frequency with respect to the same note number may be analyzed multiple times, but a value is already set in E2 (q, n). If it is, it may be excluded from selection candidates of the maximum value of E1 (p, n).

  Here, the setting of the analysis frame in the unit section will be described. The following description is similarly applied to the above-described selection unit section. FIG. 8 is a diagram illustrating a correspondence relationship between a sample sequence in a unit section extracted from an acoustic signal after time axis expansion processing and a harmonic signal. Among these, Fig.8 (a) is a sample row | line | column in the unit area extracted from the acoustic signal after a time-axis expansion process. By connecting the sample values (4096) in each sample, a wave shape as shown in FIG. Among the 128 harmonic signals, a harmonic signal selected from the unit interval T when performing a correlation operation with an analysis harmonic signal of a treble part whose period is equal to or less than the unit interval length T as shown in FIG. When subtracting a certain contained signal, the analysis frame length T (n) is a length obtained by multiplying the period of the harmonic signal by an integer up to a range in which the period does not exceed the unit section length T. (N) Extract them and use them as analysis frames.

  When one period of the harmonic signal is larger than the unit section length T, the analysis frame length T (n) that is the correlation calculation section is set to one period of the harmonic signal, so that the unit section length T is T (n) −T. Extend the time axis by adding the number of samples. In this case, one period of the harmonic signal is set as four divided sections Q1-Q4. Then, as shown in FIG. 8D, when the unit section length T corresponds to 3/4 period of the harmonic signal, the harmonic signal division section Q3 (1/2 period to 3/4 period) A sample obtained by inverting the sample of the unit section corresponding to に in the time axis direction at the time of 3/4 period of the harmonic signal (270 degrees: boundary between the divided sections Q3 and Q4) is added. Assuming that one period of the harmonic signal is a sine wave, inversion in the time axis direction at the time of 270 degrees is 270 degrees in divided sections Q3 and Q4 (3/4 period to 1 period). It uses a feature that is symmetrical with respect to the axis in the time axis direction. In this way, the time axis is extended, and a waveform analysis frame (5461 samples) as shown in FIG. 8C is obtained. In the examples of FIGS. 8C and 8D, the case where the unit section length T corresponds to 3/4 period of the harmonic signal is shown, but the unit section length T is 3/4 period or more of the harmonic signal. In this case, the same processing is performed, and the analysis frame length T (n) is 5461 samples at the maximum. In this case, any sample in the unit section of the portion exceeding the 3/4 period of the harmonic signal is used by overlapping one of the samples in the unit section.

  Also, as shown in FIG. 9B, when the unit section length T corresponds to more than 1/2 cycle and less than 3/4 period of the harmonic signal (when the last sample of the unit section belongs to the divided section Q3). Since the entire divided section Q4 and a part of the divided section Q3 are missing, a unit section sample corresponding to the divided section Q2 (a section from a quarter cycle to a half cycle) is converted to 1 / of the harmonic signal. Samples that are inverted in the time axis and amplitude axis directions at the time of two periods (180 degrees: the boundary between the divided sections Q2 and Q3) are added to the divided sections Q3, and the samples of the added divided sections Q3 are harmonized. An inverted sample is added at the time of 3/4 period (270 degrees) of the signal. Inversion in the time axis and amplitude axis directions at the time of 180 degrees assumes that one period of the harmonic signal is a sine wave, the divided sections Q2 and Q3 are 180 degrees in the time axis and amplitude axis directions. A characteristic that is a symmetrical shape rotated 180 degrees is used. The time axis is extended in this way, and a waveform analysis frame (up to 8192 samples) as shown in FIG. 9A is obtained. In this case, any sample in the unit section of the portion exceeding the half period of the harmonic signal is used by overlapping one of the samples in the unit section.

Also, as shown in FIG. 10 (b), when the unit section length T corresponds to not less than ¼ period and less than ½ period of the harmonic signal (when the last sample of the unit section belongs to the divided section Q2). Since the whole of the divided sections Q3 and Q4 and a part of the divided section Q2 are missing, the unit section samples corresponding to the Q1 section (section having a quarter period from the head) are used as the quarter period of the harmonic signal. The sample inverted in the time axis direction at the time of (90 degrees: the boundary between the divided section Q1 and the divided section Q2) is added to the divided section Q2, and the added sample of the divided section Q2 is ½ of the harmonic signal. Samples inverted in the time axis and amplitude axis directions at the time of the period (180 degrees) are added to the divided section Q3, and the added samples of the divided section Q3 are added to the 3/4 period (270 degrees) of the harmonic signal. Time at time To add a sample obtained by reversing the direction. The reason for inversion in the time axis direction at the time of 90 degrees is that if one period of the harmonic signal is a sine wave, the divided sections Q1 and Q2 are symmetrical in the time axis direction about 90 degrees. Is used. The time axis is extended in this way, and a waveform analysis frame (maximum 16384 samples) as shown in FIG. 10A is obtained. In this case, any sample in the unit section of the portion exceeding the quarter period of the harmonic signal is used by overlapping any sample in the unit section.
Note that the unit interval length T may correspond to less than ¼ period of the harmonic signal (when the last sample of the unit interval belongs to the divided interval Q1), but the time axis is extended based on this. Even if the correlation calculation is performed, the amount of information used as a source is too small to obtain a significant correlation value. Therefore, the unit section length T is analyzed for a frequency corresponding to less than a quarter cycle of the harmonic signal. It shall not be covered.

  When frequency analysis is performed while changing the analysis frame for each selected unit interval q and a spectrum (128 frequency components) is calculated, the audio signal encoding device is calculated in S5 for each selected unit interval. Based on the obtained spectrum, it is composed of frequency information that can specify each frequency corresponding to each of the N types of frequencies, spectrum intensity corresponding to each frequency, and time information that can specify the start and end of the selected unit section. A single tone component is created (S6). Specifically, the time and time length information of each note number n is added to the calculated spectrum, and [start time, time length, main frequency n, sub frequency S (P (q), n), intensity E2 A single tone component composed of (q, n)] is created. The “start time” may be any information that can identify the start time of the selected unit section in the entire digital sound signal, and in this embodiment, the digital sound signal attached to the start sample (i = 0) of the unit section. The sample number (corresponding to absolute sample address: j) in the whole is recorded. By dividing this absolute sample address by the sampling frequency (44100), the time from the head of the acoustic signal is obtained. In this embodiment, the length of time is variably given for each selected unit section. The time length is the difference up to the start time of the selected unit section that has been subjected to the generalized harmonic analysis that immediately follows (the start of the subsequent selected unit section). Time-start time of the selected unit section). When there is no subsequent selection unit section immediately after (the last selection unit section), the shift width W of the unit section is given as the time length.

Subsequently, in order to correct the variation caused by the processing being enlarged in the time axis direction, processing for correcting each single tone component is performed (S7). When a single tone component is created, a process of adding 12 · log ₂ K to all frequency information (corresponding to the note number value) is performed. For example, when K = 4, the overall pitch is raised by 24 semitones (2 octaves). This process is performed to restore the original state by multiplying the frequency by K times because the frequency is 1 / K by multiplying the number of samples by K in S1. By this correction, the code code having the note number whose note number exceeds the standard value upper limit 127 is deleted. Specifically, the code code with the note number before correction of 128-12 · log ₂ K or more is deleted.

  Subsequently, all start times and time lengths are multiplied by 1 / K. As a result, when converted to a MIDI code by an encoding process described later, the performance time of the entire MIDI code and the sounding time of each note event are reduced to 1 / K. This process is performed in order to set the time to 1 / K and return to the original state because the total performance time is K times by multiplying the number of samples by K in S1.

  As a result of the processing in S7, the frequency (pitch) becomes K times and the time information becomes 1 / K. FIG. 3B shows how the sound component changes due to the correction process in S7. In FIG. 3B, the change of the sound component in the case of K = 2 is indicated by a note. As a result of the correction processing in S7, the left "mi" note is changed to the "mi" note that is one octave higher (double the frequency) on the right side. On the other hand, the left quarter note is changed to a half eighth note on the right side.

Once the processing of increasing the frequency of each single tone component by K times and the time axis by 1 / K times is performed, the acoustic signal encoding apparatus then removes the harmonic components (S8). Specifically, E2 (q, n) obtained by the processing up to S7 is corrected to remove overtone components in ascending order of the value of n and updated to E ′ (q, n). For large E2 (q, n), correction is performed to remove overtone components using the updated E ′ (q, n). Here, the reason for correcting E2 (q, n) in ascending order of the value of n is as follows. The overtone component included in E2 (q, n) where n is relatively large may include overtones based on a plurality of E2 (q, n-12 · log ₂ h) where h is an integer of 2 or more. Furthermore, each E2 (q, n-12 · log ₂ h) is also lower than E2 (q, n-12 · log ₂ h-12 · log ₂ h ′, where h ′ is an integer of 2 or more. ) May be included, and since harmonic components are hierarchically included, the harmonic components cannot be specified and cannot be removed with high accuracy. On the other hand, since there is no sound component corresponding to E2 (q, n-12 · log ₂ h) in E2 (q, n) where n is close to the minimum, there is no overtone component, and it can be specified as a fundamental tone, There is no need to correct. Next, since E2 (q, n-12) where n is slightly larger than the minimum has only one sound component corresponding to E2 (q, n-12 · log ₂ h), E2 (q, n-12 · log ₂ h) can be specified as the fundamental tone of the harmonic component, and can be corrected to E ′ (q, n) from which the harmonic component is removed. For E2 (q, n) where n is larger than this, the harmonic component is accurately obtained using E ′ (q, n−12 · log ₂ h) corrected only to the fundamental component from which the harmonic component is removed. It becomes possible to remove.

  FIG. 11 is a flowchart showing details of overtone component correction in S8. First, initial setting is performed (S21). Specifically, the harmonic variable h is set to the initial value “2”, and the variable V for recording the sum of the harmonic components is set to the initial value “0”. By setting the harmonic variable h to the initial value “2”, processing is performed from the second harmonic, which is the lowest harmonic.

Next, “n−12 · log ₂ h” is calculated, and “n−12 · log ₂ h” is compared with “0” (S22). When n-12 · log ₂ h ≧ 0, {E2 (q, n) · E2 (q, n-12 · log ₂ h)} ^1/2 is added to the variable V, and the harmonic component of A process of updating the total sum V is performed (S23). If it is determined in S22 that n-12 · log ₂ h <0, the total V update processing in S23 is not performed. This is because there is no value corresponding to n <0 in E2 (q, n).

  Subsequently, the overtone variable h is incremented (h ← h + 1) (S24). And it compares with H which is the upper limit of a process target (S25). The upper limit H of the harmonic variable h indicates how many harmonics are processed. In the present embodiment, H = 10 is set and processing is performed up to 10th harmonic. By changing the upper limit value H, the number of overtones to be processed can be changed. If the harmonic variable h is equal to or lower than the upper limit value H as a result of the comparison in S25, the process returns to S22 and the next harmonic component is processed.

  On the other hand, if h is greater than H, the sum V of the harmonic components from the second harmonic to the H harmonic is obtained, so the harmonic component is removed from E2 (q, n) (S26). Specifically, a process according to the following [Formula 7] is executed to obtain a correction value E ′ (q, n).

[Formula 7]
E ′ (q, n) ← E2 (q, n) −V · γ
V = Σh _{= 2,10} {E2 (q, n) · E2 (q, n-12 · log ₂ h)} ^1/2

  As shown in [Formula 7] above, the sum V of harmonic components is multiplied by the harmonic correction strength γ and then subtracted. The overtone correction intensity γ is a real value of 0 ≦ γ ≦ 1, and can be set as appropriate. The greater the value of γ, the stronger the correction strength and the more the harmonics are removed. The sum V added in S23 is represented by the second equation of [Equation 7] in S26.

  When the correction value E ′ (q, n) is obtained, the obtained correction value E ′ (q, n) is compared with “0” (S27). As a result of the comparison, if the correction value E ′ (q, n) is 0 or more, it is directly adopted as the correction value E ′ (q, n), and if the correction value E ′ (q, n) is less than 0, The correction value E ′ (q, n) is set to “0” (S28). As described above, when the correction value E ′ (q, n) becomes less than 0, the negative value is physically set as the correction value E ′ (q, n). This is because it is impossible.

  The harmonic overtone processing shown in FIG. 11 is executed for each of 128 n of 0 ≦ n ≦ 127. The result of harmonic component removal in S8 is output as a corrected correlation strength array E ′ (q, n).

  Once the harmonic component removal processing has been performed, the acoustic signal encoding device next performs processing for concatenating (integrating) the single tone components in successive selection unit sections (S9). Specifically, when a single sound component in consecutive selected unit sections satisfies a predetermined connection condition, two single sound components are connected. Here, FIG. 12 shows the relationship between the selected unit section and the unit section that are the objects of connection determination.

  FIG. 12 shows an example in which unit sections 1, 5, and 6 are selected as selected unit sections 1, 2, and 3 among unit sections 1-6, as in FIG. 4B. The present embodiment is characterized in that it is performed not between selected unit sections selected as sound components for connection determination, but between selected unit sections and adjacent unit sections. In the example of FIG. 12, when determining whether or not to connect the selected unit section 1 and the selected unit section 2 (unit section 5), the connection condition is set between the selected unit section 1 and the selected unit section 2 (unit section 5). Rather than determining, the connection condition is determined between the selected unit interval 2 (unit interval 5) and the immediately preceding unit interval 4. As a result, the closest unit section in time is taken into consideration when determining the connection, and the sound components can be appropriately connected.

  As a connection condition, a state having continuity as the same sound can be appropriately set. In the present embodiment, four conditions are determined when determining whether or not to connect the selection unit interval q and the selection unit interval q + 1. The first condition is that a value obtained by subtracting the intensity of a single sound component in the immediately preceding unit section P (q + 1) -1 from the intensity of the single sound component in the selected unit section q + 1 (P (q + 1) as a unit section) is a predetermined threshold. Must be less than the value Ldif. The second condition is that the intensity of both is greater than a predetermined threshold value Lmin. The third condition is that the difference of the frequency considering the sub-frequency is less than a predetermined threshold value Ndif in consideration of a shift of 1 up and down with respect to the note number. The fourth condition is that when the selection unit section q is already connected to another selection unit section, the sub-frequency of the top selection unit section qo and the selection unit section q + 1 connected to the selection unit section q is considered. The frequency difference is less than a predetermined threshold value Nadif. When all the above four conditions are satisfied, it is assumed that there is continuity, and the subsequent single sound component is connected to the front single sound component. Also in the first condition and the second condition, a shift of 1 up and down with respect to the note number is considered. In the present embodiment, whether or not to perform the connection is determined by determining whether or not four conditions are satisfied according to the following [Equation 8] to [Equation 10] in consideration of the transition of the note number.

[Formula 8]
E1 (P (q + 1), n) -E1 (P (q + 1) -1, n) <Ldif
E1 (P (q + 1) -1, n)> Lmin and E1 (P (q + 1), n)> Lmin
| S (P (q + 1) -1, n) -S (P (q + 1), n) | <Ndif
| S (P (qo), n) -S (P (q + 1), n) | <Nadif

  In the above [Equation 8], the expression on the first line indicates that the value obtained by subtracting the intensity of the preceding single sound component from the intensity of the subsequent single sound component is less than a predetermined threshold value Ldif. , Indicating that the intensity of both the preceding single sound component and the subsequent single sound component is larger than the predetermined threshold value Lmin, and the expression in the third row indicates that the frequency difference in note number units considering the sub frequency is a predetermined threshold value Ndif. It shows that it is less than. The expression on the fourth line is a note number that takes into account the sub-frequency of the head selection unit section qo and the subsequent selection unit section q + 1 when the front selection unit section is already connected to the front selection unit section. It shows that the difference in unit frequency is less than a predetermined threshold value Nadif.

[Formula 9]
E1 (P (q + 1), n-1) -E1 (P (q + 1) -1, n) <Ldif
E1 (P (q + 1) -1, n)> Lmin and E1 (P (q + 1), n-1)> Lmin
| S (P (q + 1) -1, n) -S (P (q + 1), n-1) -M | <Ndif
| S (P (qo), n) -S (P (q + 1), n-1) -M | <Nadif

  The above [Equation 8] compares the same note numbers, whereas the above [Equation 9] differs in that the following single-tone component is reduced by one note number. The expression in the third row is obtained by calculating the frequency S (P (q + 1), n−1) for determining the first spectral intensity E1 (P (q + 1), n−1) and the first The difference from the sub-frequency S (P (q + 1) -1, n) that determines the spectrum intensity E1 (P (q + 1) -1, n) is less than a predetermined threshold value Ndif. The expression in the fourth row is obtained by calculating the frequency S (P (q + 1), n−1) for determining the first spectral intensity E1 (P (q + 1), n−1) and the first It is shown that the difference from the sub-frequency S (P (qo), n) that determines the spectrum intensity E1 (P (qo), n) is less than a predetermined threshold value Nadif.

[Formula 10]
E1 (P (q + 1), n + 1) -E1 (P (q + 1) -1, n) <Ldif
E1 (P (q + 1) -1, n)> Lmin and E1 (P (q + 1), n + 1)> Lmin
| S (P (q + 1) -1, n) -S (P (q + 1), n + 1) + M | <Ndif
| S (P (qo), n) -S (P (q + 1), n + 1) + M | <Nadif

  While the above [Equation 8] compares the same note numbers, the above [Equation 10] is different in that the following single sound component is increased by one note number. The expression in the third row is obtained by calculating the frequency S (P (q + 1), n + 1) for determining the first spectral intensity E1 (P (q + 1), n + 1) and the first spectral intensity E1 when the frequency is in note number units. It shows that the difference from the sub-frequency S (P (q + 1) -1, n) for determining (P (q + 1) -1, n) is less than a predetermined threshold value Ndif. The expression in the fourth row is expressed as follows: when the frequency is in note number units, the frequency S (P (q + 1), n + 1) for determining the first spectral intensity E1 (P (q + 1), n + 1) and the first spectral intensity E1 It shows that the difference from the sub-frequency S (P (qo), n) for determining (P (qo), n) is less than a predetermined threshold value Nadif.

  As a specific processing procedure in the present embodiment, first, it is determined whether or not all the conditions shown in [Formula 8] are satisfied. And when satisfy | filling, it progresses to a connection process. If any of the conditions shown in [Formula 8] does not satisfy one of the conditions, it is determined whether or not all the conditions shown in [Formula 9] are satisfied. And when satisfy | filling, it progresses to a connection process. If any of the conditions shown in [Equation 9] does not satisfy one of the conditions, it is determined whether or not all of the conditions shown in [Equation 10] are satisfied. And when satisfy | filling, it progresses to a connection process. If any of the conditions shown in [Equation 10] does not satisfy one of the conditions, it is determined that the connection is not performed.

  The main frequency, sub frequency, and intensity after connection use each value of the single tone component in the front selection unit section, and the time length is given as the sum of both. That is, when the selection unit interval q and the selection unit interval q + 1 are connected, the main frequency, the sub frequency, and the intensity are those of the selection unit interval q. At this time, the intensity uses the value after removing the overtone component. In this embodiment, specific threshold values as connection conditions are as follows: Ldif = 10 [unit: 128 step velocity conversion], Lmin = 1 [unit: 128 step velocity conversion], Ndif = 4/25 [unit: note number] Conversion], Nadif = 8/25 [unit: note number conversion]. Since the concatenation process is performed before conversion to a code code, each threshold value is converted into a note number and velocity. As a result of the connecting process in S9, the single sound components that have not been connected remain. In addition, a connected sound component is obtained by connection, and the connected sound component is [start time, time length, main frequency n, sub frequency S (P (q), n), intensity E, as with the single sound component. '(Q, n)], of which the time length has a value greater than the single tone component. By the connection process, a single sound component and a connected sound component are mixed, and these are hereinafter collectively referred to as a sound component. It should be noted that the connection processing in S9 is generally preferable because it is expressed by a long note, and the amount of codes is reduced, so that a smooth and natural performance can be performed with a MIDI sound source. If a code is not added, subtle temporal changes in sound such as vibrato disappear, and it may sound unnatural with a MIDI sound source. When the connection process in S9 is not performed, all are expressed as short notes.

When the connection process of S9 is completed, the sound component of [start time, time length, main frequency n, sub frequency S (P (q), n), intensity E ′ (q, n)] obtained finally is obtained. And converted into a code code (S10). The format of the code code may be any format as long as it has frequency information, spectrum intensity corresponding to each frequency, and time information that can specify the start and end of a unit section. However, in this embodiment, the data is converted to the MIDI format. In MIDI, sound generation start and sound generation end are generated as separate events, so in this embodiment, one sound component is converted into two MIDI note events. Specifically, a note-on event of note number n is issued at the “start time”, and the velocity value is 128 · {E ′ (q, n) with the maximum value of intensity E ′ (q, n) as Emax. / Emax} is given by ^1/4 . In Standard MIDI File, it is necessary to give the time as a relative time (delta time) with the immediately preceding event, and the time unit can be defined by an arbitrary integer value, for example, converted to 1/1536 [seconds]. Give. Then, the note-off event of the note number n is issued at the end time specified by “start time” + “time length” (the end time given by “time length” in the delta time). At this time, the time length is multiplied by a real number between 0 and 1. This is because a note-off instruction is given early in consideration of the reverberation of the MIDI sound source, although it depends on the tone color of the MIDI sound source to be used. Even if the time length is used as it is, there is no problem in the processing of the MIDI sound source.

  After completing the code code conversion process of S10, next, necessary adjustments are made to the code code (S11). Specifically, two adjustments are performed: adjustment of the number of chords (number of simultaneous sounds) and adjustment of the bit rate. First, the number of chords is adjusted. This is a process of suppressing the number of code codes (that is, the number of chords) that are simultaneously generated so as to be within the number of simultaneous pronunciations (that is, the maximum number of chords) that can be processed by the code code. In the present embodiment, the code is converted into the MIDI code as described above.

  Here, the concept of adjusting the number of chords in the present embodiment will be described. FIG. 13 is a diagram comparing the concept of adjusting the number of chords in this embodiment with the conventional case. In FIG. 13, the vertical axis indicates the note number and velocity, and the horizontal axis indicates time. Each rectangle indicates a sound generation section specified by a note event pair, the horizontal width indicates a sound generation time, the vertical position indicates a note number, and the vertical width indicates velocity.

  Let us consider a case where MIDI data as shown in FIG. In this case, according to the conventional method, a process of simply deleting one of the pronunciation intervals overlapping at the same time is performed. For this reason, after the adjustment process, as shown in FIG. 13B, even portions that do not overlap are deleted. In the adjustment process according to the present embodiment, an overlapping portion is specified in the pronunciation interval. In FIG. 13C, the overlapping portions of the sound generation sections are shown by dark shading. Then, only the overlapping portions of the sounding sections are deleted and the chord number is adjusted to be 1. As a result, after the adjustment process, it is possible to leave a non-overlapping portion as shown in FIG. In the example of FIG. 13D, the process of deleting the rear part of the note having the earliest note-on time is performed. However, in the present embodiment, the velocity is attenuated based on the elapsed time from the note-on time. In order to delete the smaller corrected velocity at the overlapping time, in reality, the pronunciation interval with the later note-on time may be deleted.

  Although the example of FIG. 13 is a case of a single melody, FIG. 14 shows a case of a double melody. When the MIDI data as shown in FIG. 14A is adjusted to the number of chords 2, in the conventional method, a process of simply deleting a note interval having a later note-on time among the sounding intervals overlapping at the same time is performed. For this reason, after the adjustment process, as shown in FIG. 14 (b), only the note-on time remains earlier, which is different from the actual performance. In the adjustment process according to the present embodiment, a portion in which three or more sound generation intervals overlap is specified among the sound generation intervals. In FIG. 14 (c), the overlapping portions of the sound generation sections are shown by dark shading. Then, only the overlapping portions of the sound generation sections are deleted and the chord number is adjusted to be 2. As a result, after the adjustment process, as shown in FIG. 14D, it is possible to leave the state close to the actual performance.

  Actually, the adjustment of the number of chords in S11 is performed by continuously counting the number of note events during the sounding period (note-on state) in the time axis direction, and where there are note events exceeding the maximum number of chords at the same time. Is found, first, a corrected velocity value corrected by the elapsed time from the note-on time with respect to the velocity value at the time of note-on is calculated. Then, the priority is evaluated based on the calculated corrected velocity value, and a correction process for forcibly taking off a note event pair having a low priority so as to be equal to or less than the maximum number of chords is performed. At this time, if either the velocity value or the duration value is lower than the predetermined lower limit value, the deletion process is also performed regardless of the priority. The note-on time of the i-th note event Ev (i) is set to Ev (i). time, and the velocity value is Ev (i). Assuming that the velocity is Velocity, the corrected velocity value Vc (i, t) is calculated by executing processing according to the following [Equation 11].

[Formula 11]
Vc (i, t) = Ev (i). velocity · exp {(t-Ev (i) .time) · τ}

  In the above [Equation 11], τ is an attenuation coefficient, and in this embodiment, τ = −1 / 1536 is set. For this reason, if the unit of time t is 1536 per second, Vc (i, t) attenuates to 1 / 2.7 after 1 second.

  Details of the adjustment of the number of chords in S11 will be described using the flowchart of FIG. First, initial setting is performed (S31). Specifically, an input MIDI event array Ei (i) is prepared, the number of input MIDI events is Ni, and an initial value 0 is set to the input MIDI event pointer i. The input MIDI event array Ei (i) has an attribute of (time, note_number, channel, velocity), and is sorted in ascending order of time and in ascending order of note_number. All the note-on tables T (n) are set to the initial value -1, and the initial value 0 is set to the note-on counter Cn. Also, an output MIDI event array Eo (i) is prepared, and an initial value 0 is set to the number of output MIDI events No. The output MIDI event array Eo (i) also has attributes of (time, note_number, channel, velocity).

  Subsequently, it is determined whether or not the i-th event is note-on (S32). Specifically, it is determined whether or not the intensity value Ei (i) .velocity, which is one of the attributes of the input MIDI event array Ei (i), is greater than zero.

  If the result of determination in S32 is that the i-th event is note-on, registration is made in the note-on table (S33). Specifically, i is set as the value of T (Ei (i) .note_number). Then, the note-on counter Cn is incremented (Cn ← Cn + 1). Furthermore, it registers in the output MIDI event array. Specifically, Eo (No) .time ← Ei (i) .time, Eo (No) .note_number ← Ei (i) .note_number, Eo (No) .channel ← Ei (i) .channel, Eo (No ) .velocity ← Ei (i). By setting velocity, each attribute value of the i-th input MIDI event is given as each attribute value of the i-th output MIDI event. After registering to the output MIDI event array, the output MIDI event number No is incremented (No ← No + 1).

  On the other hand, if the result of determination in S32 is that the i-th event is not note-on, it is determined whether or not the i-th event has already been registered in the note-on table (S34). Specifically, it is determined whether T (Ei (i) .note_number) is 0 or more.

  As a result of the determination in S34, if the i-th event has already been registered in the note-on table, it is deleted from the note-on table (S35). Specifically, −1 is set as the value of T (Ei (i) .note_number). The note-on counter Cn is decremented (Cn ← Cn−1). Furthermore, it registers in the output MIDI event array. Specifically, Eo (No) .time ← Ei (i) .time, Eo (No) .note_number ← Ei (i) .note_number, Eo (No) .channel ← Ei (i) .channel, Eo (No ) .velocity ← Ei (i). By setting velocity, each attribute value of the i-th input MIDI event is given as each attribute value of the i-th output MIDI event. After registering to the output MIDI event array, the output MIDI event number No is incremented (No ← No + 1). As a result of the determination in S34, if the i-th event has not been registered in the note-on table, the process of S35 is not performed.

  If registration is made in the note-on table in S33, it is determined whether or not the note-on counter exceeds the upper limit (S36). Specifically, it is determined whether the note-on counter Cn is greater than the maximum number of chords set as the upper limit. In the case of performing MIDI encoding, for example, the maximum chord number is set to 32.

  If the note-on counter Cn exceeds the upper limit as a result of the determination in S36, a minimum sound deletion process is performed from the note-on table (S37). When the minimum sound deletion process from the note-on table is completed, the input MIDI event pointer is incremented (S38). Then, it is determined whether or not the input MIDI event pointer i is smaller than the input MIDI event number Ni (S39). If it is determined that the input MIDI event pointer i is smaller than the number of input MIDI events Ni, the process returns to S32 to process the next input event. If i = Ni in S39, all input events have been processed, and the chord number adjustment process is terminated. On the other hand, for the input event determined to be note-off in S32, the minimum sound deletion process from the note-on table in S37 is not performed regardless of whether or not the deletion process from the note-on table in S35 is performed. .

  Next, details of the minimum sound deletion process from the note-on table in S37 will be described using the flowchart of FIG. First, initial setting of the note number n, the minimum intensity note number nmin, and the minimum velocity value Vmin is performed (S41). Specifically, note number n ← 0, intensity minimum note number nmin ← −1, and minimum velocity value Vmin ← 128 are set.

  After the initial setting, it is determined whether or not a note-on event corresponding to the note number n is registered in the note-on table (S42). Specifically, it is determined whether T (n) ≧ 0. If registered, the correction strength of the note-on event is calculated (S43). Specifically, the process according to [Formula 11] is executed. Assuming that the input MIDI event array Ei (i) has the attribute of (time, note_number, channel, velocity), the above [Formula 11] is expressed as the following [Formula 11a].

[Formula 11a]
V (Ei (T (n))) = Ei (T (n)). velocity · exp {(Ei (i) .time-Ei (T (n)). time) · τ}

  When the correction strength is calculated, it is determined whether or not the obtained correction strength V (Ei (T (n))) is smaller than the minimum velocity value (S44). As a result of the determination, when the obtained correction intensity V (Ei (T (n))) is smaller than the minimum velocity value, the value of the note number n is set as the intensity minimum note number nmin, and the obtained correction intensity V The value of (Ei (T (n))) is set as the minimum velocity value Vmin (S45).

  In S42, when it is determined that it is not registered in the note-on table, it is not necessary to delete, so the processing of S43 to S45 is not performed. If it is determined in S42 that it is not registered in the note-on table, or if the minimum intensity note number nmin and the minimum velocity value Vmin are updated in S45, the note number n is incremented (n ← n + 1). (S46). Then, it is determined whether or not the note number n is smaller than the upper limit value (128 in this embodiment) (S47). If it is determined that the note number n is smaller than the upper limit value, the process returns to S42 to process the note-on event of the next note number n. In S47, if n = 128 (upper limit value), the processing has been performed for the note-on events of all the note numbers n, so the note number n set to the minimum intensity note number nmin at that time point. Are deleted from the note-on table (S48). Specifically, T (nmin) ← −1 is set. The note-on counter Cn is decremented (Cn ← Cn−1).

  Then, it is determined whether or not the minimum intensity note number nmin is a newly registered event (S49). Specifically, it is determined whether nmin is Ei (i) .note_number. As a result of the determination, if the minimum intensity note number nmin is a newly registered event, the registration to the output MIDI event array is canceled (S50). Specifically, No ← No−1 is set.

  On the other hand, if the result of determination in S49 is that the minimum intensity note number nmin is not a newly registered event, a note-off event is registered in the output MIDI event array (S51). After registering in the output MIDI event array of S50 or registering the note-off event in the output MIDI event array of S51, the minimum sound deletion processing from the note-on table is finished, and the process proceeds to S38 in FIG. Increment the pointer.

  When the processing shown in FIGS. 15 and 16 is executed and the adjustment of the number of chords is completed, the bit rate is adjusted to further consider the bit rate that can be processed with the code code. When converting to MIDI code, the number of note event pairs is counted in the time axis direction, for example, at 1-second intervals, and the average amount of code data is 5 bytes (40 bits). Is 9000 [bps (bits / second)], when a section in which the number of events per second exceeds 9000/40 = 225 is found, it is paired with a note-on or note-off event existing in the section. Searches for note-off or note-on events that become in the neighborhood, evaluates the priority by the product (energy value) of the velocity value and duration value (note-off time-note-on time) of each note event pair, and designates the event Note event pairs with low priority (low energy value) to be less than the number (in this case "225") It performs a process of deleting Tokoro manner. At this time, if either the velocity value or the duration value is lower than the predetermined lower limit value, the deletion process is also performed regardless of the priority.

  Here, the result of applying the harmonic overtone removal and the chord number adjustment, which are the characteristics of the present embodiment, will be described in comparison with the conventional method. FIG. 17 is a diagram showing an original sound waveform of a target acoustic signal. The horizontal axis represents time, and the vertical axis represents the amplitude value. FIG. 18 is a diagram illustrating a result of performing MIDI encoding on the acoustic signal illustrated in FIG. 17 without removing overtones. FIG. 19 shows the result of converting the encoding result obtained by the conventional method shown in FIG. 18 into a musical score. Since the overtones are recorded as musical notes, the original musical score cannot be reproduced.

  FIG. 20 is a diagram showing a result of MIDI encoding performed on the acoustic signal shown in FIG. 17 by the conventional harmonic removal method described in Patent Document 2. In FIG. In the example of FIG. 20, the harmonics are removed, but at the same time, the data that is originally necessary has been removed. FIG. 21 is a diagram showing a result of MIDI encoding performed on the acoustic signal shown in FIG. 17 by the conventional chord number reduction method described in Patent Document 2. In FIG. In the example of FIG. 21, the harmonic overtone is erroneously extracted, the fundamental tone is dropped, and data loss occurs.

  FIG. 22 is a diagram illustrating a result of performing MIDI encoding using harmonic removal according to the present invention. Compared with FIG. 18, FIG. 20, and FIG. 21, it can be seen that overtones are removed and no data is lost. FIG. 23 is a diagram showing a result of performing MIDI encoding using overtone removal according to the present invention, with the maximum number of chords being 1. In FIG. FIG. 24 is obtained by converting the encoding result shown in FIG. 23 into a musical score. Since the overtones are removed and the number of chords is adjusted, the original musical score can be faithfully reproduced.

  The preferred embodiments of the present invention have been described above. However, the present invention is not limited to the above embodiments, and various modifications can be made. For example, in the above embodiment, the analysis between the note numbers is performed using M differential sounds (sub-frequency), but the differential sounds are not used and the analysis is performed using only N types of frequencies corresponding to the note numbers. You may do it. In this case, the analysis accuracy is slightly reduced, but the processing load is reduced because the number of frequencies to be analyzed is reduced. In the case where the differential sound is not used, in the determination of the connection processing of the single sound component in S9, in the [Equation 8] to [Equation 10], none of the expressions in the third and fourth lines are determined.

  In the above embodiment, before setting the unit section, the number of samples in the time series direction is expanded in S1, and after the creation of the single sound component, the frequency of each single sound component is multiplied by K times as the correction of the single sound component in S7. Although the processing for increasing the time axis to 1 / K is performed, it is not always necessary to perform the processing when a particularly high time resolution is not required.

  In the above embodiment, the frequency analysis is divided into the first frequency analysis and the second frequency analysis. As a result of the first frequency analysis, the second frequency analysis is performed on the selected unit section that satisfies a predetermined condition. Although it was made to perform, you may make it perform the well-known frequency analysis which is disclosed by patent documents 1-4 with respect to each unit area.

  In the above embodiment, two adjustment processes are performed in S11. However, it is desirable to execute both of these adjustment processes. However, only one of these adjustment processes may be executed, or neither of them may be executed. Also good.

  The present invention provides music contents such as music score production, mobile phone ring melody production, and karaoke performance data production using a technology for converting an acoustic signal obtained by PCM into a code code such as MIDI code. In the production field, broadcasting media (digital radio and television broadcasting using terrestrial / BS), communication media (CS broadcasting, Internet streaming broadcasting, mobile phone service, mobile music distribution service, etc.), package media (CD, DVD, It can be used in the audio content production industry for BlueRay, memory IC cards, and the like.

1 ... CPU
2 ... RAM
3 ... Data storage device 4 ... Program storage device 5 ... Key input I / F
6 ... Data I / O I / F
7 ... Display output I / F

Claims (22)

An encoding method for encoding an acoustic signal given as a sample sequence of J time series digitized at a predetermined sampling frequency,
A unit section composed of a predetermined number T (T <J) samples with respect to the sample sequence, while overlapping a predetermined number W (W <T) samples in the time axis direction with an adjacent unit section. Section setting stage to be set,
For each unit section, spectrum conversion corresponding to the N types of frequencies is calculated for each unit section by performing frequency conversion for at least N types of frequencies f (n) to be analyzed. Spectrum calculation stage;
For each of the N types of frequencies f (n), when there is a spectrum intensity corresponding to f (n) / k, which is a frequency that is a fraction of an integer k (k ≧ 2), f (n) / Correction for attenuating the spectrum intensity corresponding to the frequency f (n) by the amount obtained by multiplying the spectrum intensity corresponding to k by the correction intensity γ is performed in order from the lowest frequency f (n) to create a corrected spectrum intensity. If f (n) is higher, a correction is performed to attenuate the spectrum intensity using the previously corrected spectrum intensity, and a spectrum correction stage for creating a corrected spectrum intensity;
An encoding step of generating a code code of a predetermined format based on the time difference between the start time of the unit section and the start time of the immediately following unit section, and the corrected spectrum intensity of the unit section;
A method for encoding an acoustic signal, comprising: In claim 1,
The spectrum calculation step includes:
For each unit section, by performing frequency conversion for at least N types of frequencies f (n) to be analyzed, the first spectral intensity corresponding to the N types of frequencies for the unit section p. A first spectrum calculation stage for calculating E1 (p, n);
An evaluation value based on a change for each frequency corresponding to the first spectral intensity E1 (p-1, n) in the unit section p-1 located immediately before the unit section p is greater than a predetermined threshold value. Only when it is larger, the unit interval p is selected as the q (q ≦ p) -th selection unit interval q, and at least N kinds of frequencies f (n) to be analyzed are selected in the first spectrum calculation stage. A second spectrum calculation step of calculating a second spectrum intensity E2 (q, n) corresponding to the N kinds of frequencies by performing a frequency conversion with higher accuracy than the frequency conversion;
Composed of
A second spectral intensity E2 (q, n) is calculated as the spectral intensity;
The spectrum correction step performs a correction for attenuating the second spectrum intensity E2 (q, n) as the spectrum intensity corresponding to the frequency f (n), and the corrected spectrum intensity E ′ (q , N) is generated. A method for encoding an acoustic signal, In claim 2,
In the first spectrum calculation stage and the second spectrum calculation stage, predetermined M types of sub-frequency f (n, m) are set in a range not exceeding adjacent frequencies for each of the N types of frequencies f (n). Then, as the first spectral intensity E1 (p, n) and the second spectral intensity E2 (q, n), intensity values corresponding to the sub-frequency indicating the highest intensity among the M types of sub-frequency are obtained. A method for encoding an acoustic signal, comprising: calculating an acoustic signal. In claim 2 or claim 3,
The encoding step defines P (q) = p when the selection unit interval q is the p-th unit interval p with respect to two adjacent selection unit intervals q and selection unit interval q + 1. , First spectral intensities E1 (P (q + 1), n), E1 (P (q + 1), n corresponding to frequencies f (n), f (n-1), f (n + 1) in the selected unit interval q + 1. n-1), E1 (P (q + 1), n + 1) and the first spectrum corresponding to the frequency f (n) in the unit section P (q + 1) -1 located immediately before the selected unit section q + 1 The difference from the intensity E1 (P (q + 1) -1, n) is less than a predetermined threshold Ldif, and the first spectrum intensity E1 (P (q + 1), n), E1 (P (q + 1), n -1), E1 (P (q + 1), n + 1) and When the first spectral intensity E1 (P (q + 1) -1, n) is greater than a predetermined threshold value Lmin, the selection unit interval q and the selection unit interval q + 1 are connected, and the time difference that is the basis of the code code And a method of encoding an acoustic signal using a time difference between a head time of a selected unit section q defined in the selected unit section q and a head time of a selected unit section q + 2 immediately after the selected unit section q + 1. . In claim 4,
The encoding step includes at least sub-frequency determining the first spectral intensities E1 (P (q + 1), n), E1 (P (q + 1), n−1), E1 (P (q + 1), n + 1). When the condition that the difference in note number units between any one and the sub-frequency that determines the first spectral intensity E1 (P (q + 1) -1, n) is less than a predetermined threshold value Ndif is further satisfied As long as the selection unit interval q and the selection unit interval q + 1 are connected, the method of encoding an acoustic signal is characterized. In claim 5,
When the selection unit interval q is already connected to another selection unit interval, the first selection unit interval to which the selection unit interval q is connected is defined as qo,
The encoding step includes at least sub-frequency determining the first spectral intensities E1 (P (q + 1), n), E1 (P (q + 1), n−1), E1 (P (q + 1), n + 1). Only when the condition that the difference in note number units between any one and the sub-frequency that determines the first spectral intensity E1 (P (qo), n) is less than a predetermined threshold value Nadif is satisfied A method for encoding an acoustic signal, comprising connecting the selection unit interval q and the selection unit interval q + 1. In any one of Claims 4-6,
When the code code expresses that there are more than a predetermined upper limit number of sounding intervals at a certain time t, the encoding step performs the time from the start time of the selected unit interval for each sounding interval. Attenuation intensity value is calculated by attenuating the intensity value based on the elapsed time until t, and the sounding interval of the code code is terminated so that the sounding interval at which the attenuation intensity value is minimum at the time t is terminated at the time t. A method of encoding an acoustic signal, wherein correction is performed so as to shorten the time. In any one of Claims 2-7,
In the first spectrum calculation step, N element signals to be constituent elements of the section signal of the unit section correspond to integer multiples of the period of the frequency f (n), respectively, and T closest to the T (N) an element signal preparation stage to prepare as samples;
By performing a correlation operation between the element signal corresponding to each of the N frequencies f (n) and a section signal composed of T (n) samples of the corresponding unit section p, It is constituted by a correlation calculation stage for calculating the spectral intensity E1 (p, n),
The second spectrum calculation step includes:
The element signal preparation step;
Correlation is performed between the element signal corresponding to each of the N frequencies f (n) and the section signal composed of T (n) samples of the corresponding selection unit section q, and the correlation value is the highest. A harmonic signal selection step of selecting an element signal corresponding to the high frequency f (nmax) as a harmonic signal;
By using T (nmax) samples given by the product of the selected harmonic signal and the correlation value obtained for the harmonic signal as an inclusion signal, and subtracting the inclusion signal from the interval signal, T (nmax) A differential signal calculation step for obtaining a differential signal composed of a number of samples,
Using the T (n) samples updated to reflect the T (nmax) samples as new interval signals, the harmonic signal selection step and the difference signal calculation step are executed to obtain new inclusion signals and difference signals. N content signals are obtained by repeatedly performing the obtained process, and the second spectrum intensity E2 (q, n) corresponding to the N kinds of frequencies is calculated based on the correlation values of the obtained content signals. A method of encoding an acoustic signal characterized by the above. In claim 8,
The correlation calculation step includes:
A correlation calculation corresponding to the first W sample in the unit interval p-1 is performed on the previous correlation calculation result corresponding to each frequency f (n) in the unit interval p-1 located immediately before, and a correlation value for each frequency is obtained. Is subtracted from the previous correlation calculation result, a correlation calculation corresponding to the last W sample in the T (n) samples in the unit interval p is performed, and a correlation value for each frequency is added to the previous correlation calculation result. Thus, a correlation calculation result corresponding to each frequency f (n) in the unit interval p is acquired, and the first spectral intensity E1 (p, n) is calculated based on the correlation calculation result. A method for encoding an acoustic signal. In any one of Claims 1-9,
The encoding step performs encoding using the MIDI format as the code code, uses a note number as the frequency information of the code code, uses velocity as the second spectral intensity, and selects the selected unit interval q Delta time 1 and delta time 2 that are relative times from the immediately preceding MIDI event, respectively, and the defined note number, velocity, delta A method for encoding an audio signal, wherein a MIDI note-on event is created based on time 1 and a MIDI note-off event is created based on note number and delta time 2. An encoding device for encoding an acoustic signal given as a sample sequence of J time series digitized at a predetermined sampling frequency,
A unit section composed of a predetermined number T (T <J) samples with respect to the sample sequence, while overlapping a predetermined number W (W <T) samples in the time axis direction with an adjacent unit section. Section setting means to set;
For each unit section, spectrum conversion corresponding to the N types of frequencies is calculated for each unit section by performing frequency conversion for at least N types of frequencies f (n) to be analyzed. Spectrum calculation means;
For each of the N types of frequencies f (n), when there is a spectrum intensity corresponding to f (n) / k, which is a frequency that is a fraction of an integer k (k ≧ 2), f (n) / Correction for attenuating the spectrum intensity corresponding to the frequency f (n) by the amount obtained by multiplying the spectrum intensity corresponding to k by the correction intensity γ is performed in order from the lower frequency f (n), and the higher frequency f (n). Performs a correction for attenuating the spectrum intensity using the previously corrected spectrum intensity, and a spectrum correction means for creating a corrected spectrum intensity;
Encoding means for generating a code code of a predetermined format based on the time difference between the start time of the unit section and the start time of the immediately following unit section, and the corrected spectrum intensity of the unit section;
A device for encoding an acoustic signal, comprising: In claim 11,
The spectrum calculating means includes
For each unit section, by performing frequency conversion for at least N types of frequencies f (n) to be analyzed, the first spectral intensity corresponding to the N types of frequencies for the unit section p. First spectrum calculating means for calculating E1 (p, n);
An evaluation value based on a change for each frequency corresponding to the first spectral intensity E1 (p-1, n) in the unit section p-1 located immediately before the unit section p is greater than a predetermined threshold value. Only when it is larger, the unit interval p is selected as the q (q ≦ p) -th selection unit interval q, and at least N kinds of frequencies f (n) to be analyzed are selected in the first spectrum calculation stage. Second spectrum calculation means for calculating a second spectrum intensity E2 (q, n) corresponding to the N types of frequencies by performing a frequency conversion with higher accuracy than the frequency conversion;
Composed of
A second spectral intensity E2 (q, n) is calculated as the spectral intensity;
The spectrum correction means performs correction for attenuating the second spectrum intensity E2 (q, n) as the spectrum intensity corresponding to the frequency f (n), and corrects the corrected spectrum intensity E ′ (q , N) is produced. In claim 12,
The first spectrum calculating means and the second spectrum calculating means set predetermined M kinds of sub-frequency f (n, m) within a range not exceeding adjacent frequencies for each of the N kinds of frequencies f (n). Then, as the first spectral intensity E1 (p, n) and the second spectral intensity E2 (q, n), intensity values corresponding to the sub-frequency indicating the highest intensity among the M types of sub-frequency are obtained. An apparatus for encoding an acoustic signal, comprising: calculating an acoustic signal. In claim 12 or claim 13,
The encoding means defines P (q) = p when the selection unit interval q is the p-th unit interval p for two adjacent selection unit intervals q and selection unit interval q + 1. , First spectral intensities E1 (P (q + 1), n), E1 (P (q + 1), n corresponding to frequencies f (n), f (n-1), f (n + 1) in the selected unit interval q + 1. n-1), E1 (P (q + 1), n + 1) and the first spectrum corresponding to the frequency f (n) in the unit section P (q + 1) -1 located immediately before the selected unit section q + 1 The difference from the intensity E1 (P (q + 1) -1, n) is less than a predetermined threshold Ldif, and the first spectrum intensity E1 (P (q + 1), n), E1 (P (q + 1), n -1), E1 (P (q + 1), n + 1) and When the first spectral intensity E1 (P (q + 1) -1, n) is greater than a predetermined threshold value Lmin, the selection unit interval q and the selection unit interval q + 1 are connected, and the time difference that is the basis of the code code And a time difference between the start time of the selected unit section q defined in the selected unit section q and the start time of the selected unit section q + 2 immediately after the selected unit section q + 1 is used. . In claim 14,
The encoding means includes at least a sub-frequency for determining the first spectral intensity E1 (P (q + 1), n), E1 (P (q + 1), n-1), E1 (P (q + 1), n + 1). When the condition that the difference in note number units between any one and the sub-frequency that determines the first spectral intensity E1 (P (q + 1) -1, n) is less than a predetermined threshold value Ndif is further satisfied As long as the selection unit interval q and the selection unit interval q + 1 are connected, the apparatus for encoding an acoustic signal is characterized. In claim 15,
When the selection unit interval q is already connected to another selection unit interval, the first selection unit interval to which the selection unit interval q is connected is defined as qo,
The encoding means includes at least a sub-frequency for determining the first spectral intensity E1 (P (q + 1), n), E1 (P (q + 1), n-1), E1 (P (q + 1), n + 1). Only when the condition that the difference in note number units between any one and the sub-frequency that determines the first spectral intensity E1 (P (qo), n) is less than a predetermined threshold value Nadif is satisfied An apparatus for encoding an acoustic signal, wherein the selection unit interval q and the selection unit interval q + 1 are connected. In any one of Claims 14-16,
In the case where the code code represents that there are more than a predetermined upper limit number of sounding intervals at a certain time t, the encoding means, for each sounding interval, from the start time of the selected unit interval to the time Attenuation intensity value is calculated by attenuating the intensity value based on the elapsed time until t, and the sounding interval of the code code is terminated so that the sounding interval at which the attenuation intensity value is minimum at the time t is terminated at the time t. An apparatus for encoding an acoustic signal, wherein correction is performed so as to shorten. In any one of Claims 12-17,
The first spectrum calculating means corresponds to N element signals to be constituent elements of the section signal of the unit section, each corresponding to an integral multiple of the period of the frequency f (n), and T closest to the T (N) Element signal preparation means for preparing as samples,
By performing a correlation operation between the element signal corresponding to each of the N frequencies f (n) and a section signal composed of T (n) samples of the corresponding unit section p, It is comprised by the correlation calculating means which calculates spectrum intensity | strength E1 (p, n),
The second spectrum calculation means includes:
The element signal preparation means;
Correlation is performed between the element signal corresponding to each of the N frequencies f (n) and the section signal composed of T (n) samples of the corresponding selection unit section q, and the correlation value is the highest. Harmonic signal selection means for selecting an element signal corresponding to a high frequency f (nmax) as a harmonic signal;
By using T (nmax) samples given by the product of the selected harmonic signal and the correlation value obtained for the harmonic signal as an inclusion signal, and subtracting the inclusion signal from the interval signal, T (nmax) Differential signal calculation means for obtaining a differential signal composed of a number of samples,
The T (nmax) samples updated to reflect the T (nmax) samples are used as new section signals, and the processing by the harmonic signal selection means and the difference signal calculation means is executed to obtain new inclusion signals and differences. N content signals are obtained by repeatedly performing the signal obtaining process, and the second spectrum intensity E2 (q, n) corresponding to the N types of frequencies is calculated based on the correlation values of the obtained content signals. An apparatus for encoding an acoustic signal. In claim 18,
The correlation calculation means includes
A correlation calculation corresponding to the first W sample in the unit interval p-1 is performed on the previous correlation calculation result corresponding to each frequency f (n) in the unit interval p-1 located immediately before, and a correlation value for each frequency is obtained. Is subtracted from the previous correlation calculation result, a correlation calculation corresponding to the last W sample in the T (n) samples in the unit interval p is performed, and a correlation value for each frequency is added to the previous correlation calculation result. Thus, a correlation calculation result corresponding to each frequency f (n) in the unit interval p is acquired, and the first spectral intensity E1 (p, n) is calculated based on the correlation calculation result. An apparatus for encoding an acoustic signal. In any one of Claims 11-19,
The encoding means performs encoding using the MIDI format as the code code, uses a note number as the frequency information of the code code, uses velocity as the second spectral intensity, and selects the selected unit interval q Delta time 1 and delta time 2 that are relative times from the immediately preceding MIDI event, respectively, and the defined note number, velocity, delta An apparatus for encoding an audio signal, wherein a MIDI note-on event is created based on time 1 and a MIDI note-off event is created based on note number and delta time 2.   The program for making a computer perform the encoding method of the acoustic signal as described in any one of Claims 1-10.   A program for causing a computer to function as the acoustic signal encoding device according to any one of claims 11 to 20.

Images