JP5552794B2 - Method and apparatus for encoding acoustic signal - Google Patents

Description

  The present invention relates to an audio signal encoding technique, and more particularly to an encoding technique suitable for encoding into code data of MIDI format or the like.

  As a method for converting an acoustic signal into code data such as a MIDI code, the applicant has proposed a technique for performing frequency analysis such as Fourier transform on the acoustic signal (see Patent Document 1). However, since the given acoustic signal is digitally sampled, there is a problem that in the region where the frequency is high, the number of samples per period is reduced in the correlation calculation with the harmonic function, and the analysis accuracy is significantly deteriorated. On the other hand, the applicant increases the frequency interval of the harmonic function to be referenced in a high frequency region by performing frequency analysis at a logarithmic scale frequency similar to the average temperament used in the MIDI code, and is adjacent. A technique for erroneously determining the frequency (MIDI note number) has been proposed (see Patent Document 2). Along with this, we have proposed a method to analyze by dividing the main frequencies (semitones) to be analyzed into fine frequencies, mainly improving the analysis accuracy in the region where the adjacent frequency interval is wide and the frequency is high (patent) Reference 3). Further, the applicant has proposed a technique using generalized harmonic analysis, and has improved the problem of excessively extracting pseudo frequency components in the Fourier transform (see Patent Document 4).

  Although the proposed method has made it possible to perform frequency analysis of stationary signals with high accuracy, the frequency of acoustic signals fluctuates in time series, so ideally the instantaneous frequency at a certain time should be measured. Is desired, but it is not feasible. Therefore, in any of the above proposals, a method is adopted in which a minute section called a frame is set in the vicinity of a certain time and a short-time frequency analysis is performed in that section. At this time, in order to improve the frequency analysis accuracy (particularly the bass sound frequency), the smaller interval (number of samples) is better. On the other hand, in order to improve the time resolution, the shorter interval is better. That is, frequency resolution and time resolution are in a trade-off relationship (the principle of uncertainty). Therefore, the applicant sets the frame length to a fixed value (4096 samples at 44.1 kHz sampling) so that the lowest sound of the MIDI code can be analyzed, and improves the time resolution while sending the frame in variable steps. Proposed (see Patent Document 5).

Japanese Patent No. 3795201 Japanese Patent No. 4037542 Japanese Patent No. 4156268 Japanese Patent No. 4132362 Japanese Patent No. 4061070

However, there is a limit in the above-described conventional technology, and analysis cannot follow the frequency variation when the frequency variation is sharp in an audio signal or the like. In particular, foreign language voices and relatively fast-spoken Japanese voices have a problem that frequency fluctuations cannot be extracted properly, and clear MIDI playback sounds cannot be obtained. In the case of audio signals, the sampling frequency at the time of recording is generally lower than that of musical instrument sounds, but the main factor is that the speed of frequency change is significant compared to musical instrument performance sounds. However, if the frame length is set to be shorter than this, the frequency analysis accuracy deteriorates.

Accordingly, the present invention provides an audio signal encoding method and apparatus capable of improving time resolution in analysis and extracting frequency fluctuations mainly in audio signals with high accuracy while maintaining frequency analysis accuracy equivalent to that of the prior art. It is an issue to provide.

  In order to solve the above-described problem, in the first aspect of the present invention, in encoding an acoustic signal given as an intensity array of J time series digitized at a predetermined sampling period, a time axis is obtained with respect to the intensity array. Magnified in the direction by a predetermined magnification Q (Q is an integer), converted to a J × Q time-series expanded intensity array, and composed of a predetermined number of samples T (T <J) with respect to the expanded intensity array A plurality of unit sections to be encoded are set while overlapping adjacent unit sections in the time axis direction, spectrum intensities corresponding to P types of frequencies are calculated for each unit section, and individual units are calculated. Each section is composed of frequency information that can specify each frequency, spectrum intensity corresponding to each frequency, and time information that can specify the start and end of the unit section, corresponding to the obtained P types of frequencies. P Code code is generated, the frequency information is corrected so that the frequency of the P code codes is multiplied by Q, and the corrected frequency information corresponds to a frequency that exceeds the maximum frequency information before correction. And the time information is corrected so that the time axis becomes 1 / Q times the remaining Ph code codes.

  According to the first aspect of the present invention, after each intensity array of the digitized acoustic signal is expanded by a predetermined magnification in the time axis direction, a predetermined interval is set for each unit section composed of a predetermined number T of intensity arrays. Spectral intensities corresponding to several P types of frequencies are calculated, P code codes including frequency and time are obtained, and the frequency of the P code codes is corrected to Q times and the time is corrected to 1 / Q times. Therefore, it is possible to improve the time resolution in the analysis while maintaining the frequency analysis accuracy equivalent to the conventional one, and extract the frequency fluctuation mainly in the audio signal with high accuracy.

  Further, in the second aspect of the present invention, in encoding for encoding an acoustic signal given as an intensity array of J time series digitized at a predetermined sampling period, time is applied to the intensity array. Ax is enlarged in the axial direction by a predetermined magnification Q (Q is an integer), converted into J × Q time-series enlarged intensity arrays, and a predetermined number of samples T (T <J) with respect to the expanded intensity array. A plurality of unit sections to be encoded are set while overlapping adjacent unit sections in the time axis direction, spectrum intensities corresponding to P types of frequencies are calculated for each unit section, and the spectrum Corrections are made so that each of the P types of frequencies obtained in the calculation stage is multiplied by Q, and the spectrum intensity corresponding to the frequency after the correction exceeds the maximum frequency before correction is deleted, and the remaining Ph types are corrected. The spectrum intensity corresponding to the wave number is corrected, the start and end times of each unit section are corrected to be 1 / Q times, and the Ph types of frequencies corrected in the spectrum correction step for each unit section. Corresponding to the frequency information that can identify each frequency, the spectrum intensity corresponding to each frequency, and the Ph code code composed of time information that can identify the start and end of the unit section It is characterized by that.

  According to the second aspect of the present invention, after each intensity array of the digitized acoustic signal is enlarged by a predetermined magnification in the time axis direction, a predetermined interval is set for each unit section composed of a predetermined number T of intensity arrays. Spectral intensities corresponding to several P types of frequencies are calculated, the frequency of P code codes is multiplied by Q, spectral intensities corresponding to frequencies exceeding the maximum frequency before correction are deleted, and the remaining Ph types of frequencies After correcting the spectrum intensity to correspond to, and correcting the time to 1 / Q times, Ph code codes including the frequency and time are obtained, so analysis is performed while maintaining the same frequency analysis accuracy as before. It is possible to improve the time resolution and extract the frequency fluctuation mainly in the audio signal with high accuracy.

  Further, in the third aspect of the present invention, in the first or second aspect of the present invention, expansion of the intensity array to the time series is performed using linear interpolation for the intensity array without changing the sampling period. The time axis is enlarged by Q times, the frequency of the acoustic signal is lowered to 1 / Q as a whole, and the time axis is extended by Q times.

  Also, in the fourth aspect of the present invention, in any one of the first to third aspects of the present invention, the encoding is performed using MIDI format as the code code, and the frequency information of the code code is used. Using a note number, using velocity as the spectrum intensity, and using delta time 1 and delta time 2 which are relative times from the immediately preceding MIDI event as the time information, these converted note numbers, velocity, delta time 1 A MIDI note-on event is created based on the note number, and a MIDI note-off event is created based on the note number and delta time 2.

Further, in the fifth aspect of the present invention, in the first aspect of the present invention, the encoding is performed using MIDI format as the code code, and a note number is used as the frequency information of the code code. Velocity is used as the spectrum intensity, and delta time 1 and delta time 2, which are relative times from the immediately preceding MIDI event, are used as the time information, and based on these converted note numbers, velocity, and delta time 1, MIDI A note-on event is created, a MIDI note-off event is created based on the note number and delta time 2, and the note number is set to 12 · log ₂ Q (with 2 as the base for Q) adds a value) obtained by multiplying 12 times the logarithm, Notona over 128-12 · log ₂ Q It deletes the code code with bars, characterized in that correction is performed so as to multiply the 1 / Q to the delta time 1 and delta time 2 remaining Ph number of code code.

  Further, in a sixth aspect of the present invention, in any one of the first to fifth aspects of the present invention, the calculation of the spectrum prepares a plurality of element signals to be constituent elements of the section signal of the unit section. The inclusion signal given as the product of the harmonic signal and the correlation value obtained for the harmonic signal is selected as the harmonic signal from the plurality of component signals as the harmonic signal. Is subtracted from the section signal, and the difference signal is used as a new section signal, and the harmonic signal is selected and the difference signal is calculated to obtain a new inclusion signal and a new difference signal. By repeating the process, P inclusion signals are obtained, and the spectrum intensities corresponding to the P kinds of frequencies are calculated based on the obtained amplitude values of the inclusion signals. It is characterized in.

  Further, in the seventh aspect of the present invention, in the first aspect of the present invention, the J time-series intensity array is an encoding target composed of a predetermined number of samples T (T <J). A plurality of second unit intervals are set while overlapping adjacent second unit intervals in the time axis direction, a second spectrum intensity corresponding to P kinds of frequencies is calculated for each second unit interval, P second code codes composed of P frequencies obtained in the calculation of two spectra, second spectrum intensities corresponding to the respective frequencies, and start time and end time of the second unit section are created and created. Pl second code codes having a frequency lower than the frequency range of the corrected Ph code codes are extracted from the P second code codes and included in the unit section corresponding to the second unit section Said corrected Ph Added code code, characterized in that so as to create a Ph number of corrected code code and Pl pieces of Ph + Pl number of composite code code consists second code code.

  According to the seventh aspect of the present invention, in the first aspect of the present invention, the second code code is generated by performing frequency analysis on the original acoustic signal separately from the acoustic signal expanded in time series, Since the second code code is synthesized with the code code obtained from the acoustic signal expanded in time series, the code part of the bass part that is missing in the code code obtained from the acoustic signal expanded in time series is obtained. It is possible to compensate, and it is possible to perform encoding that can be faithfully reproduced even for an acoustic signal having an important component in the bass part.

  Further, in the eighth aspect of the present invention, in the second aspect of the present invention, the J time series intensity array is an encoding target composed of a predetermined number of samples T (T <J). A plurality of second unit intervals are set while overlapping adjacent second unit intervals in the time axis direction, and second spectral intensities corresponding to P types of frequencies are calculated for each second unit interval, and the creation A second spectral intensity corresponding to a Pl type frequency lower than the corrected Ph type frequency range is extracted from the second spectral intensity corresponding to the P type frequency thus determined, and a unit corresponding to the second unit section is extracted. A spectrum intensity corresponding to the corrected Ph types of frequencies included in the section is added, and a spectrum intensity corresponding to the Ph types of corrected frequencies and a second spectrum intensity corresponding to the Pl types of frequencies are included. A composite spectrum intensity corresponding to the combined Ph + Pl type synthesized frequency is created, and the encoding is performed on each of the frequency information capable of specifying each frequency corresponding to the synthesized Ph + Pl type frequency, It is characterized in that Ph + Pl code codes composed of corresponding synthetic spectrum intensities and time information capable of specifying the start and end of the unit section are created.

  According to the eighth aspect of the present invention, in the second aspect of the present invention, in addition to the acoustic signal expanded in time series, frequency analysis is performed on the original acoustic signal to obtain spectrum intensity, and in time series Since a code code is obtained by combining with the spectrum intensity obtained from the expanded acoustic signal, the code code of the bass part that is missing in the code code obtained from the acoustic signal expanded in time series can be supplemented, It is possible to perform encoding that can be faithfully reproduced even for an acoustic signal having an important component in the bass part.

  According to the present invention, while maintaining the same frequency analysis accuracy as in the past, the time resolution in analysis can be improved, and it is possible to extract frequency fluctuations mainly in audio signals with high accuracy.

It is a flowchart which shows the outline | summary of the encoding method of the acoustic signal which concerns on this invention. It is a figure which shows the concept of the expansion of a time-axis direction, the increase in a frequency, and reduction | decrease of time information. It is a flowchart which shows the modification of the encoding method of the acoustic signal which concerns on this invention.

  DESCRIPTION OF EMBODIMENTS Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the drawings. FIG. 1 is a flowchart showing an outline of an audio signal encoding method according to this embodiment. The audio signal encoding method according to the present embodiment is performed by a computer executing a program that records the detailed procedure of each step (each stage) shown in FIG. As a computer, a CPU and memory for performing arithmetic processing, a storage device such as a hard disk for storing programs and data, a data input device for inputting data such as acoustic signals, a keyboard for inputting instructions, a mouse, etc. A general-purpose computer including a device and a display device such as a liquid crystal display that displays necessary information on a screen can be used. Further, the audio signal encoding apparatus according to the present embodiment is realized by a general-purpose computer in which a program recording the detailed procedure of each step (each stage) shown in FIG. 1 is incorporated.

  First, a computer (encoding device) reads a digital audio signal to be processed from a data input device. The digital audio signal is obtained by sampling an analog audio signal at a predetermined sampling frequency and the number of quantization bits. In the present embodiment, the sampling is performed with a sampling frequency of 44.1 kHz and a quantization bit number of 16 bits as an example. I will explain. When sampling is performed at a sampling frequency of 44.1 kHz, the digital acoustic signal is configured as a sample row (sample array: intensity array) having 44100 samples (intensity values) per second.

  After reading the digital sound signal, the computer enlarges the sample constituting the digital sound signal by a predetermined magnification Q (Q is an integer) in the time axis direction (S1). Specifically, the number of samples constituting the digital acoustic signal is multiplied by Q. Then, every Q samples having the same value as the original sample are arranged, and (Q-1) sample values between them are linearly interpolated using the values of the original samples located on both sides. give. Assuming that the sample value for each sample j (j = 0... J-1) of the original sound signal is x (j), the computer expands by executing the processing according to the following [Equation 1]. A sample value x ′ (j · Q + k) for each sample j · Q + k (0 ≦ k ≦ Q−1) of the subsequent acoustic signal is calculated. In the following [Formula 1], w is a real value taking a value of 0 ≦ w ≦ 1 given by k / (Q−1).

[Formula 1]
x '(j.Q + k) = (1-w) .x (j) + w.x (j + 1)

  As a result of the processing in S1, J samples constituting the digital audio signal are expanded to J × Q. FIG. 2A shows a change in waveform due to the enlargement process in S1. The waveform in FIG. 2A is a plot of sample values connected by line segments, but is expressed in a curved line due to the large number of samples. By executing the processing according to the above [Equation 1], the waveform shown on the left side changes to the waveform shown on the right side. In the example of FIG. 2, the case of Q = 2 is shown for convenience of explanation.

  Next, the computer sets a unit interval on the sample expanded in the time axis direction (S2). The length of the unit interval (number of samples T) is set in relation to the sampling frequency. However, if the sampling frequency is 44.1 kHz, 4096 samples or more are required to faithfully analyze the low frequency region. . Therefore, in this embodiment, the unit interval is set as the number of samples T per unit interval T = 4096.

  Setting of the unit section is performed by sequentially extracting samples from the head of the digital sound signal as disclosed in Patent Documents 1 to 5. The unit interval is set so as not to leak all samples, and is preferably set so that the samples overlap in continuous unit intervals. In this case, the head interval (referred to as shift width) of each unit section can be set according to various rules. The simplest is a method in which the shift width is fixed, that is, the number of overlapping samples is set constant. For example, when T = 4096, the first unit interval is j = 0-4095, the second unit interval is j = 2048-6143, the second unit interval is j = 4096-8191, and so on. / 2) Setting is performed with overlapping samples. However, in order to improve the time resolution, there is a demand for reducing the shift width. On the other hand, there is a problem that the calculation time increases as the shift width is reduced. Also, if the shift width is made smaller than necessary, the connection conditions are not satisfied in the connection processing of single-tone components in FIG. 1 and S4 described later, and the connection processing does not function properly. Therefore, in order to set an optimum shift width in accordance with the state of the acoustic signal, in the present embodiment, the zero crossing point in which the frequency change is noticeable by the denseness or autocorrelation analysis of the zero crossing interval as disclosed in Patent Document 5 is used. The sample is selected and the sample located at this zero crossing is set as the head.

  The zero crossing point is a crossing point between the positive and negative acoustic signals and the 0 level of the signal. Here, the zero crossing point indicates the time when the intensity value (amplitude) of the acoustic signal becomes zero. However, the digitized acoustic signal does not necessarily sample the zero crossing point in the analog signal. Therefore, in actuality, in addition to the case where the intensity value is just 0, when the intensity value of the sampling point changes from positive to negative or from negative to positive, one of the sampling points before and after that is regarded as a zero crossing point. Process. In order to detect the zero crossing, it is necessary that the acoustic signal to be analyzed has both positive and negative polarities. Therefore, it is necessary to remove the direct current component for the acoustic signal including the direct current component. Since various known methods can be applied to the removal of the DC component, detailed description thereof is omitted here. Basically, the unit section is set with the sample located at the zero crossing as the head, but the unit section is set with the position other than the zero crossing as the head so that the shift width of the continuous unit sections falls within a certain range. In some cases. Specifically, when the maximum shift width (for example, T / 2) is exceeded, the unit section is set starting from the position having the maximum shift width even at a position other than the zero intersection. On the other hand, if it is below the minimum shift width (for example, T / 8), a unit section is set starting from the position where several zero crossings are skipped so as to exceed the minimum shift width, exceeding the minimum shift width and maximum If there is no corresponding zero-intersection in the range of the shift width, correction is performed so that the unit section is set with the position having the maximum shift width as the head as described above.

Next, frequency analysis is performed for each set unit section, and a spectrum of each unit section is calculated (S3). As disclosed in Patent Documents 1 to 5, the spectrum of each unit section is calculated by 128 analysis frequencies f (n) = 440 · 2 corresponding to MIDI note numbers n (0 ≦ n ≦ 127). ^{This is} performed by extracting 128 components by generalized harmonic analysis based on ^{(n-69) / 12} element signals (element functions). “128 types” and “128” are examples, and generally P components are extracted using P types of analysis frequencies. When the analysis frequency is set in correspondence with the note number n, the frequency interval between the note numbers becomes wider as the frequency becomes higher. In particular, when n exceeds 60, the analysis accuracy decreases. Therefore, in this embodiment, as disclosed in Patent Document 3, 128M element signals f (n, m) = 440 · 2 ^{(n−69 + m /} ) obtained by dividing a note number into M differential sounds. Analyze using ^{M) / 12} to extract 128M components. Unless special encoding such as addition of a pitch bend code is performed in FIG. 1 and S4, which will be described later, information on M differential sounds at each note number is not necessary. As a component in the note number, 128 components are extracted as a result.

  As a specific processing procedure by the computer, first, an intensity value array E (n) (0 ≦ n ≦ 127) and a sub-frequency array S (n) corresponding to the note number are set, and all initial values are set to 0. . Subsequently, a process according to the following [Formula 2] is executed for 0 ≦ n ≦ 127 and 0 ≦ m ≦ M−1 to obtain (nmax, mmax) that maximizes E (n, m). .

[Formula 2]
A (n, m) = (1 / T (n)) · Σi _{= 0, T (n) −1} x (i) sin (2πf (n, m) i / fs)
B (n, m) = (1 / T (n)) · Σi _{= 0, T (n) −1} x (i) cos (2πf (n, m) i / fs)
{E (n, m)} ² = {A (n, m)} ² + {B (n, m)} ²

  In the above [Equation 2], T (n) is the analysis frame length, and is set so as to be the maximum integer multiple of the period of the element signal within a range not exceeding the unit interval T. (N) = k / f (n, m). F (nmax, mmax) using (nmax, mmax) that maximizes E (n, m) is selected as the harmonic signal. When (nmax, mmax) is obtained, the computer executes processing according to the following [Equation 3] using A (nmax, mmax) and B (nmax, mmax) to obtain a sample array x (i). Update all elements (0 ≦ i ≦ T−1).

[Formula 3]
x (i) ← x (i) −A (nmax, mmax) · sin (2πf (nmax, mmax) i / fs) −B (nmax, mmax) · cos (2πf (nmax, mmax) i / fs)

  In the above [Equation 3], the process of subtracting the content signal from x (i) is performed. Further, processing according to the following [Equation 4] is executed to update the array of intensity values E (n) and the sub-frequency array S (n).

[Formula 4]
E (nmax) ← E (nmax) + E (nmax, mmax)
S (nmax) ← mmax

  The computer executes the processes of [Formula 2] to [Formula 4] for all n (0 ≦ n ≦ 127), and determines the values of all E (n) and S (n).

  In the present embodiment, in order to reduce the processing load, the value of M is variably set based on the note number, for example, the frequency interval to be analyzed is about 100 Hz. And note number 60 and below are not divided and M = 1. Although the accuracy is slightly reduced, S (n) is determined by the first [Formula 2] process, and the second and subsequent [Formula 2] processes are performed with m = S (n) fixed. Sound analysis may be omitted. Further, in the processing of [Equation 2], there is a possibility that signal components having different sub-frequency with respect to the same note number may be analyzed a plurality of times, but E (n) and S (n) are already values. May be excluded from selection candidates for the maximum value of E (n, m).

  When a spectrum (128 frequency components) is calculated for each unit section, a code code including frequency information, spectrum intensity corresponding to each frequency, and time information that can specify the start and end of the unit section Create (S4). In creating a code code, first, information of time and time length of each note number n is added to the calculated spectrum, and [start time, time length, main frequency n, sub-frequency S (n), intensity E (n )] Is created. The “start time” may be information that can specify the start time of the unit section in the entire digital sound signal. In this embodiment, the entire digital sound signal attached to the start sample (i = 0) of the unit section. Sample number (corresponding to absolute sample address: j) is recorded. By dividing this absolute sample address by the sampling frequency (44100), the time from the head of the acoustic signal is obtained. In this embodiment, the time length is variably given for each unit section, and is a difference (start time of the subsequent unit section−start time of the unit section) immediately after the start time of the subsequent unit section. Given.

  For each unit section set in S2, 128 single-tone components are created. Further, in S4, processing for connecting the single-tone components in continuous unit sections is performed. Specifically, when a single note component of the same note number in a continuous unit section satisfies a predetermined connection condition, two single note components are connected. As the connection condition, a state having continuity as the same sound can be set as appropriate, but in this embodiment, the difference between the frequencies (main frequency + sub frequency) considering the sub frequency is less than a predetermined threshold value Ndif. When the two intensities are equal to or greater than the predetermined threshold Lmin and the difference between the two intensities is less than the predetermined threshold Ldif, the subsequent single sound component is connected to the preceding single sound component as having continuity. However, the connected main frequency, sub-frequency, and intensity use each value of the larger single tone component, and the time length is given as the sum of both. In the present embodiment, specific threshold values as connection conditions are Ndif = 8/25 [unit: note number conversion], Lmin = 1 [unit: 128 step velocity conversion], Ldif = 10 [unit: 128 step velocity conversion] It is said that. Since the concatenation process is performed before conversion to a code code, each threshold value is converted into a note number and velocity.

As long as the connection condition is satisfied, the single note components of the same note number are repeatedly applied to the single note component of the subsequent unit section, and finally obtained [start time, time length, main frequency n, sub frequency S ( n), a single tone component of intensity E (n)] is converted into a code code. The format of the code code may be any format as long as it has frequency information, spectrum intensity corresponding to each frequency, and time information that can specify the start and end of a unit section. However, in this embodiment, the data is converted to the MIDI format. In MIDI, sound generation start and sound generation end occur as separate events. Therefore, in this embodiment, one single tone component is converted into two MIDI note events. Specifically, a note-on event of note number n is issued at the “start time”, and the velocity value is 128 · {E (n) / Emax} ^1/4, where the maximum value of the intensity E (n) is Emax. Give in. In Standard MIDI File, it is necessary to give the time as a relative time (delta time) with the immediately preceding event, and the time unit can be defined by an arbitrary integer value, for example, converted to 1/1536 [seconds]. Give. Then, a note-off event of note number n is issued at the end time specified by “start time” + “time length”. At this time, the time length is multiplied by a real number between 0 and 1. This is because a note-off instruction is given early in consideration of the reverberation of the MIDI sound source, although it depends on the tone color of the MIDI sound source to be used. Even if the time length is used as it is, there is no problem in the processing of the MIDI sound source.

  When converting to MIDI code, it is necessary to adjust the number of simultaneous pronunciations in order to consider the number of simultaneous pronunciations that can be processed by the MIDI sound source. When the number of simultaneous sounds that can be processed by the MIDI sound source is 32, the number of note events during the sound generation period (note-on state) is continuously counted in the time axis direction, and there are locations where 32 note events exist simultaneously. If found, each pair of note-off events is searched in the neighborhood, and the priority is evaluated by the product (energy value) of the velocity value and duration value (note-off time-note-on time) of each note event pair. Then, a process of locally deleting note event pairs with low priority (low energy value) so as to be equal to or less than the specified number of chords (in this case “32”). “Locally” means that it is limited to a portion where there are more than 32 note events. At this time, if either the velocity value or the duration value is lower than the predetermined lower limit value, the deletion process is also performed regardless of the priority.

  Furthermore, when converting to a MIDI code, it is necessary to adjust the bit rate in order to consider the bit rate that can be processed by the MIDI sound source. In the time axis direction, the number of note-on or note-off events is counted at one-second intervals, the average code length is 5 bytes (40 bits), and the maximum bit rate that can be processed by the MIDI sound source is 9000 [bps (bits). / Sec)], if an interval in which the number of events per second exceeds 9000/40 = 225 is found, the note-off or note paired with the note-on or note-off event existing in that interval, respectively. An on-event is searched in the neighborhood interval, and the priority is evaluated by the product (energy value) of the velocity value and duration value (note-off time-note-on time) of each note event pair, and the specified number of events (in this case, “225”). ”) Locally delete note event pairs with low priority (low energy value) so that Cormorant. At this time, if either the velocity value or the duration value is lower than the predetermined lower limit value, the deletion process is also performed regardless of the priority.

When the code code is created, a process for correcting each code code is performed in order to correct the variation caused by the enlargement in the time axis direction (S5). Specifically, first, a process of adding 12 · log ₂ Q to the note number values of all the note events (note-on event or note-off event) is performed. For example, when Q = 4, the overall pitch is raised by 24 semitones (2 octaves). This process is performed to restore the original state by multiplying the frequency by Q, since the frequency is 1 / Q by multiplying the number of samples by Q in S1. By this correction, the code code having the note number whose note number exceeds the standard value upper limit 127 is deleted. Specifically, the code code with the note number before correction of 128-12 · log ₂ Q or more is deleted.

  Subsequently, the times (note-on time or note-off time) of all the note events are multiplied by 1 / Q. As a result, the performance time of the entire MIDI code and the sounding time of each note event are reduced to 1 / Q. This process is performed in order to set the time to 1 / Q and to return to the original state because the total performance time has become Q times by multiplying the number of samples by Q in S1. When this process is performed, the number of note events per hour increases by a factor of Q, so the bit rate adjustment executed in S4 is executed again.

  As a result of the processing in S5, the frequency (pitch) becomes Q times and the time information becomes 1 / Q. FIG. 2B shows how the MIDI event (MIDI code note event) is changed by the correction process of S5. In FIG. 2B, the change of the MIDI event in the case of Q = 2 is indicated by a note. As a result of the correction process of S5, the left "mi" note is changed to the "mi" note that is one octave higher (double the frequency) on the right side. On the other hand, the left quarter note is changed to a half eighth note on the right side.

Even if the MIDI code obtained in S5 is outputted as a final code as it is, the effect of the present invention can be obtained. However, in the correction of the code code in S5, since the process of adding 12 · log ₂ Q to the note number value is performed, the entire code code has been moved to the high-pitched part, so a part of the high-pitched part is deleted outside the MIDI standard. As a result, the code code of the bass part does not exist. Therefore, a process for compensating for the bass part is performed by executing the processes in S6 to S9 below. Hereinafter, the process of S6-S9 is demonstrated.

  The processing of S6 to S8 is basically the same as the processing of S2 to S4, but the target digital acoustic signal is a digital acoustic signal with an increased number of samples, whereas S2 to S4 are digital acoustic signals. The processing of S6 to S8 is different in that it is the original digital acoustic signal. Therefore, the processes in S6 to S8 are realized by known techniques disclosed in Patent Documents 1 to 5. In the processing in the present embodiment, the processing shown in S6 to S8 is performed on the original digital acoustic signal.

When the code code is obtained by the process of S8, the code code obtained by S5 (the code code obtained from the digital acoustic signal with the increased number of samples: hereinafter referred to as the first code code) and the code code obtained by S8. A process of synthesizing a code code (a code code obtained from the original digital acoustic signal: hereinafter referred to as a second code code) is performed (S9). In the first code code, a note event having a note number of 0 to 12 · log ₂ Q does not exist by adding a positive value of 12 · log ₂ Q to the note number based on the processing of S5. (Conversely, note events having a note number of 128-12 · log ₂ Q or more before S5 processing are deleted because they enter a high-pitched sound region outside the MIDI standard). Therefore, 12 · log ₂ Q is set as a predetermined threshold for the note number, the first code code uses a value not less than the predetermined threshold, and the second code code has a value less than the predetermined threshold. To obtain a composite code group. At this time, for the note event of the second code code, the velocity value may be adopted as it is, but since the length corresponding to the acoustic signal of the unit section is Q times longer than the first code code, the velocity value May cause imbalance. Therefore, the following correction processing may be performed as necessary.

  Specifically, first, in the first code code group, an average value V1 of velocity values of all note events having a note number equal to or greater than a threshold value is calculated. Subsequently, in the second code code group as well, an average value V2 of velocity values of all note events having note numbers equal to or greater than the same threshold value is calculated. Then, for all the note events whose note numbers of the second code code group are less than the threshold value, the velocity values are multiplied by V1 / V2. As a result, the corrected bass intensity of the second code code group is added to the bass part of the first code code group. When adding, the first code code group and the second code code group can be sorted and mixed based on the time information based on the specification of Standard MIDI File, and the first code code group and the second code code can be mixed. It is possible to store groups as independent tracks. By executing the processing of S6 to S9, code codes can be generated for all frequency bands in the MIDI standard, which is particularly useful for music (instrument sounds) extending to the bass part. . When code code synthesis is performed, the first code code group has an average number of note events Q times that of the second code code group, and the total number of note events per time increases. The adjustment of the number of simultaneous sounds and the adjustment of the bit rate are executed again.

(Modification about composition)
Although the method of synthesizing at the stage of the MIDI event data processed up to the code code has been described above, as a modification, the method of synthesizing at the spectrum calculation stage before the creation of the first code code and the second code code is shown in FIG. Will be described. The processing from S1 to S3 is the same as that in FIG. 1, and the spectrum is corrected as S10 instead of S4. This corresponds to S5 in FIG. 1, and the calculated spectrum frequency (both the main frequency and the sub frequency) is multiplied by Q (in the unit of note number, a value of 12 · log ₂ Q is added). A process of deleting the spectrum intensity corresponding to the high frequency protruding outside and correcting the start time of each unit section to 1 / Q times is performed. On the other hand, S6 to S7 in FIG. 3 are the same as in FIG. 1, and the process of synthesizing the spectrum corrected in S10 and the second spectrum is performed without performing S8 in FIG. 1 (S11). As in S5, the note number corresponds to 0 to 12 · log ₂ Q-1 by adding a positive value of 12 · log ₂ Q to the note number in the spectrum corrected based on the processing of S10. There is no spectral intensity to do. (Conversely, the spectrum intensity having a note number of 128-12 · log ₂ Q or more before S10 processing is deleted because it falls in the high-pitched region outside the MIDI standard). Therefore, only a spectrum intensity less than a predetermined threshold 12 · log ₂ Q is adopted from the second spectrum to obtain a combined spectrum. At this time, for the spectrum intensity of the second spectrum, the intensity value is adopted as it is, and the intensity value correction processing as described in the previous section is not normally performed. (The spectrum whose length corresponding to the acoustic signal of the unit section has been corrected in S10 is as short as 1 / Q compared to the second spectrum, so there is a possibility that the intensity value will vary. When the processing is performed, the subsequent processing of connecting the single-tone components in S12 does not work properly due to the discontinuity of the intensity value. Processing is performed to create MIDI event data. When this synthesis method is used, data close to a predetermined threshold value may be concatenated into a single code code among the two types of spectral intensities generated in S10 and S7, and the code efficiency is higher than that in the method of FIG. It is characterized in that a high code code can be generated. In the embodiment of FIG. 1, the number of simultaneous sounds is adjusted twice at S4 and S9, and the bit rate is adjusted three times at S4, S5, and S9. This may be done once at S12.

  The preferred embodiments of the present invention have been described above. However, the present invention is not limited to the above embodiments, and various modifications can be made. For example, in the above embodiment, the processing of S6 to S9 is added and the processing for compensating for the bass portion is performed. However, as described above, the code portion obtained by the processing up to S5 without supplementing the bass portion is also used. It is possible to reproduce the original acoustic signal sufficiently. In particular, when a human voice or music that does not affect the bass part is used as an acoustic signal, a code code obtained by the processing up to S5 can be handled. By adding the processing of S6 to S9, it becomes possible to more faithfully reproduce a sound signal that is meaningful in a bass part such as a musical instrument sound.

  In the embodiment shown in FIG. 1, the number of simultaneous sounds is adjusted twice at S4 and S9, and the bit rate is adjusted three times at S4, S5, and S9. It only has to be done once. Therefore, the case where the bass portion is compensated may be performed once in S9, and the case where the bass portion is not compensated may be performed once in S5. However, since the processing load as a whole may be reduced by executing in the middle, it may be changed as appropriate according to the situation.

  Moreover, in the said embodiment, although the preferable process example was demonstrated about the process of S2-S4, S6-S8, about these processes, in the range which does not deviate from the meaning of this invention, well-known patent documents 1- 5 can be used.

  The present invention uses a technology for converting an acoustic signal obtained by PCM or the like into a code code such as a MIDI code, and uses broadcast media (digital radio / television broadcast such as terrestrial / BS) and communication media (CS broadcast, Internet). -Streaming broadcasting, mobile phone service, portable music distribution service, etc.) and audio media production industry for package media (CD, DVD, BlueRay, memory IC card, etc.).

Claims (17)

An encoding method for encoding an acoustic signal given as an intensity array of J time series digitized at a predetermined sampling period,
A time-series expansion step of expanding the intensity array by a predetermined magnification Q (Q is an integer) in the time axis direction and converting it into J × Q time-series expanded intensity arrays;
A section setting step for setting a plurality of unit sections to be encoded, which are composed of a predetermined number of samples T (T <J) with respect to the expanded intensity array, while overlapping adjacent unit sections in the time axis direction; ,
A spectrum calculation stage for calculating a spectrum intensity corresponding to P kinds of frequencies for each unit section;
For each unit section, the frequency information that can specify each frequency, the spectrum intensity corresponding to each frequency, and the start and end of the unit section are specified corresponding to the P types of frequencies obtained in the spectrum calculation stage. An encoding stage for creating P code codes composed of possible time information;
The frequency information is corrected so that the frequency of the P code codes is Q times, the code code corresponding to the frequency whose frequency information after correction exceeds the maximum frequency information before correction is deleted, and the remaining Ph For each code code
A code code correcting step for correcting the time information so that the time axis is 1 / Q times;
A method for encoding an acoustic signal, comprising: An encoding method for encoding an acoustic signal given as an intensity array of J time series digitized at a predetermined sampling period,
A time-series expansion step of expanding the intensity array by a predetermined magnification Q (Q is an integer) in the time axis direction and converting it into J × Q time-series expanded intensity arrays;
A section setting step for setting a plurality of unit sections to be encoded, which are composed of a predetermined number of samples T (T <J) with respect to the expanded intensity array, while overlapping adjacent unit sections in the time axis direction; ,
A spectrum calculation stage for calculating a spectrum intensity corresponding to P kinds of frequencies for each unit section;
Correction is performed so that each of the P types of frequencies obtained in the spectrum calculation step is Q times, and the spectrum intensity corresponding to the frequency after the correction exceeds the maximum frequency before correction is deleted, and the remaining Ph A spectral correction step for correcting the spectral intensity corresponding to the type of frequency and correcting the start and end times of each unit section to be 1 / Q times;
For each unit section, frequency information that can specify each frequency, spectrum intensity corresponding to each frequency, and start and end of the unit section corresponding to the Ph types of frequencies corrected in the spectrum correction step. An encoding stage for generating Ph code codes composed of identifiable time information;
A method for encoding an acoustic signal, comprising: In claim 1 or claim 2,
In the time series expansion step, the frequency of the acoustic signal is entirely increased by 1 / Q in the time axis direction using linear interpolation with respect to the intensity array without changing the sampling period. A method for encoding an acoustic signal, characterized in that the time axis is lowered to Q and the time axis is extended by Q times. In any one of Claims 1-3,
The encoding step performs encoding using the MIDI format as the code code, uses a note number as the frequency information of the code code, uses velocity as the spectrum intensity, and uses the velocity as the time information and the previous MIDI event as the time information. The relative note time delta time 1 and delta time 2 are used to create a MIDI note-on event based on the converted note number, velocity, and delta time 1, and based on the note number and delta time 2. A method for encoding an audio signal, wherein a MIDI note-off event is created. In claim 1 ,
The encoding step performs encoding using the MIDI format as the code code, uses a note number as the frequency information of the code code, uses velocity as the spectrum intensity, and uses the velocity as the time information and the previous MIDI event as the time information. The relative note time delta time 1 and delta time 2 are used to create a MIDI note-on event based on the converted note number, velocity, and delta time 1, and based on the note number and delta time 2. Create a MIDI note-off event,
The code code correction step adds 12 · log ₂ Q (a value obtained by multiplying Q by a logarithm value with 2 as a base by 12 times) to the note number,
The code code having a note number of 128-12 · log ₂ Q or more is deleted, and correction is performed such that the delta time 1 and delta time 2 of the remaining Ph code codes are multiplied by 1 / Q. A method for encoding an acoustic signal. In any one of Claims 1-5,
The spectrum calculation step includes:
An element signal preparation step of preparing a plurality of element signals to be components of the section signal of the unit section;
A harmonic signal selection step of selecting, as a harmonic signal, an element signal having the highest correlation value with respect to the section signal from the plurality of element signals;
A difference signal calculation step for obtaining a difference signal by subtracting the inclusion signal given by the product of the harmonic signal and the correlation value obtained for the harmonic signal from the interval signal;
Using the difference signal as a new interval signal, the harmonic signal selection step and the difference signal calculation step are executed to obtain a new inclusion signal and a new difference signal, thereby obtaining P inclusion signals. An acoustic signal encoding method, wherein spectrum intensities corresponding to the P kinds of frequencies are calculated based on the obtained amplitude value of the contained signal. In claim 1,
A second unit interval adjacent to a plurality of second unit intervals to be encoded, which is composed of a predetermined number of samples T (T <J) with respect to the J time-series intensity arrays, is set in the time axis direction. A second section setting stage to be set while overlapping,
A second spectrum calculating step for calculating a second spectrum intensity corresponding to P kinds of frequencies for each second unit section;
A second code code is generated that includes P frequencies obtained in the second spectrum calculation step, second spectrum intensities corresponding to the P frequencies, and a start time and an end time of the second unit section. Two encoding stages;
Pl second code codes having a frequency lower than the frequency range of the Ph code codes corrected by the code code correction stage are extracted from the P second code codes created by the second encoding stage. Then, the Ph code codes corrected by the code code correction step included in the unit section corresponding to the second unit section are added, and the Ph corrected code codes and the Pl second code codes are added. A code synthesis stage for creating Ph + Pl synthesized code codes comprising:
A method for encoding an acoustic signal, comprising: In claim 2,
A second unit interval adjacent to a plurality of second unit intervals to be encoded, which is composed of a predetermined number of samples T (T <J) with respect to the J time-series intensity arrays, is set in the time axis direction. A second section setting stage to be set while overlapping,
A second spectrum calculating step for calculating a second spectrum intensity corresponding to P kinds of frequencies for each second unit section;
A second spectrum intensity corresponding to a Pl type frequency lower than a Ph type frequency range corrected by the spectrum correction step is obtained from a second spectrum intensity corresponding to the P type frequency generated by the second spectrum calculation step. Spectral intensities corresponding to the Ph types of corrected frequencies are extracted and added to the spectral intensities corresponding to the Ph types of frequencies corrected in the spectral correction step included in the unit interval corresponding to the second unit interval. And a spectrum synthesis stage for creating a synthesized spectrum intensity corresponding to the Ph + Pl synthesized frequency composed of the second spectrum intensity corresponding to the Pl kind of frequency, and
Have
The encoding step corresponds to the frequency of the Ph + Pl types synthesized in the spectrum synthesis step, frequency information that can specify each frequency, the synthesized spectrum intensity corresponding to each frequency, and the start and end of the unit interval. An acoustic signal encoding method, wherein Ph + Pl code codes composed of identifiable time information are created. An encoding device for encoding an acoustic signal given as an intensity array of J time series digitized at a predetermined sampling period,
Time series expansion means for enlarging the intensity array by a predetermined magnification Q (Q is an integer) in the time axis direction and converting it into a J × Q time series expanded intensity array;
Section setting means for setting a plurality of unit sections to be encoded, which are composed of a predetermined number of samples T (T <J) with respect to the expanded intensity array, while overlapping adjacent unit sections in the time axis direction; ,
Spectrum calculating means for calculating spectrum intensities corresponding to P types of frequencies for each unit section;
For each unit section, the frequency information that can specify each frequency, the spectrum intensity corresponding to each frequency, and the start and end of the unit section are specified corresponding to the P types of frequencies obtained by the spectrum calculation means . Encoding means for creating P code codes composed of possible time information;
The frequency information is corrected so that the frequency of the P code codes is Q times, the code code corresponding to the frequency whose frequency information after correction exceeds the maximum frequency information before correction is deleted, and the remaining Ph Code code correcting means for correcting the time information so that the time axis is 1 / Q times the number of code codes;
A device for encoding an acoustic signal, comprising: An encoding device for encoding an acoustic signal given as an intensity array of J time series digitized at a predetermined sampling period,
Time series expansion means for enlarging the intensity array by a predetermined magnification Q (Q is an integer) in the time axis direction and converting it into a J × Q time series expanded intensity array;
Section setting means for setting a plurality of unit sections to be encoded, which are composed of a predetermined number of samples T (T <J) with respect to the expanded intensity array, while overlapping adjacent unit sections in the time axis direction; ,
Spectrum calculating means for calculating spectrum intensities corresponding to P types of frequencies for each unit section;
Correction is performed so that the frequency of P types obtained by the spectrum calculation means is multiplied by Q, and the spectrum intensity corresponding to the frequency after the correction exceeds the maximum frequency before correction is deleted, and the remaining frequency is deleted. Spectral correction means that corrects the spectral intensity corresponding to the Ph type of frequency and corrects the start and end times of each unit section to be 1 / Q times;
For each unit section, corresponding to the Ph types of frequencies corrected by the spectrum correction means, frequency information that can specify each frequency, spectrum intensity corresponding to each frequency, and start and end of the unit section Encoding means for creating Ph code codes composed of identifiable time information;
A device for encoding an acoustic signal, comprising: In claim 9 or claim 10,
The time series expansion means expands the frequency of the acoustic signal by 1 / Q in the time axis direction using linear interpolation with respect to the intensity array without changing the sampling period. A sound signal encoding apparatus, wherein the time axis is lowered to Q and the time axis is extended by Q times. In any one of Claims 9-11,
The encoding means performs encoding using the MIDI format as the code code, uses a note number as the frequency information of the code code, uses velocity as the spectrum intensity, and uses the velocity as the time information and the previous MIDI event as the time information. The relative note time delta time 1 and delta time 2 are used to create a MIDI note-on event based on the converted note number, velocity, and delta time 1, and based on the note number and delta time 2. An apparatus for encoding an audio signal, wherein a MIDI note-off event is created. In claim 9 ,
The encoding means performs encoding using the MIDI format as the code code, uses a note number as the frequency information of the code code, uses velocity as the spectrum intensity, and uses the velocity as the time information and the previous MIDI event as the time information. The relative note time delta time 1 and delta time 2 are used to create a MIDI note-on event based on the converted note number, velocity, and delta time 1, and based on the note number and delta time 2. Create a MIDI note-off event,
The code code correcting means adds 12 · log ₂ Q (a value obtained by multiplying Q to a logarithm value with 2 as a base by 12) to the note number to obtain 128−12 · log ₂ Q or more. An acoustic signal in which a code code having a note number is deleted and correction is performed such that 1 / Q is multiplied with respect to the delta time 1 and the delta time 2 of the remaining Ph code codes. Encoding device. In any one of Claims 9-13,
The spectrum calculating means includes
Element signal preparation means for preparing a plurality of element signals to be constituent elements of the section signal of the unit section;
Harmonic signal selection means for selecting, as a harmonic signal, an element signal having the highest correlation value with respect to the section signal from the plurality of element signals;
Differential signal calculation means for obtaining a differential signal by subtracting the inclusion signal given by the product of the harmonic signal and the correlation value obtained for the harmonic signal from the interval signal;
Using the difference signal as a new section signal, the harmonic signal selection means and the difference signal calculation means obtain P content signals by repeatedly performing a process of obtaining a new content signal and a new difference signal. An apparatus for encoding an acoustic signal, wherein spectrum intensities corresponding to the P kinds of frequencies are calculated based on an amplitude value of a contained signal. In claim 9,
A second unit interval adjacent to a plurality of second unit intervals to be encoded, which is composed of a predetermined number of samples T (T <J) with respect to the J time-series intensity arrays, is set in the time axis direction. Second section setting means for setting while overlapping,
Second spectrum calculating means for calculating a second spectrum intensity corresponding to P kinds of frequencies for each second unit section;
A second code code for generating P second code codes composed of the P frequencies obtained by the second spectrum calculating means , the second spectrum intensity corresponding to each of the P frequencies, and the start time and end time of the second unit section. Two encoding means;
Pl second code codes having a frequency lower than the frequency range of the Ph code codes corrected by the code code correction unit are extracted from the P second code codes created by the second encoding unit. Then, Ph code codes corrected by the code code correction means included in the unit section corresponding to the second unit section are added, and Ph corrected code codes and Pl second code codes are added. Code synthesizing means for creating Ph + Pl composed code codes configured;
A device for encoding an acoustic signal, comprising: In claim 10 ,
A second unit interval adjacent to a plurality of second unit intervals to be encoded, which is composed of a predetermined number of samples T (T <J) with respect to the J time-series intensity arrays, is set in the time axis direction. Second section setting means for setting while overlapping,
Second spectrum calculating means for calculating a second spectrum intensity corresponding to P kinds of frequencies for each second unit section;
The second spectrum intensity corresponding to the Pl type frequency lower than the Ph type frequency range corrected by the spectrum correction means than the second spectrum intensity corresponding to the P type frequencies created by the second spectrum calculating means. Spectral intensities corresponding to the Ph types of corrected frequencies are extracted and added to the spectral intensities corresponding to the Ph types of frequencies corrected by the spectrum correcting means included in the unit interval corresponding to the second unit interval. And a spectrum combining means for creating a combined spectrum intensity corresponding to the combined frequency of Ph + Pl types composed of second spectrum intensities corresponding to Pl types of frequencies,
Have
The encoding means corresponds to the frequency of the Ph + P1 types synthesized by the spectrum synthesizing means , frequency information capable of specifying each frequency, synthesized spectrum intensity corresponding to each frequency, and start and end of the unit section. An acoustic signal encoding apparatus characterized in that Ph + Pl code codes composed of identifiable time information are created.   A program for causing a computer to function as the acoustic signal encoding device according to any one of claims 9 to 16.

Images