JP2002366146A - Encoding method for sound signal - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an encoding method for a sound signal which can process even a sound signal consisting of a plurality of channels like a stereophonic sound signal with an operation load nearly as large as that of a sound signal consisting of one channel like a monaural sound signal and can suppress an encoded data amount. SOLUTION: A plurality of unit sections are set on the time base (S1a, S1b) for a sound signal consisting of two right and left channels and frequency analyses are taken by the unit sections to compute phoneme data consisting of frequencies, intensity value, section start times corresponding to the start points of the unit sections, and section end times corresponding to the end points of the unit sections (S2a, S2b); and all phoneme data obtained from the sound signals of both the channels are integrated between phoneme data of both the channels of the same time and the same frequency and integrated phoneme data to which an intensity ratio is added according to the intensity values of a plurality of phoneme data of the integration source are generated (S3).

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to broadcast media (radio, television), communication media (CS video / audio distribution, Internet music distribution, communication karaoke), package media (CD, MD, cassette, video, LD, CD). −
Production of various audio contents provided by ROMs, game cassettes, solid-state memory media for portable music players, etc., and dedicated portable music players, mobile phones and PHs
The present invention relates to an audio signal encoding technique suitable for being used for MIDI transmission of music contents including vocals directed to S. pagers, audio materials of literary works such as kabuki, noh, chanting, poetry, or audiovisual materials for language education.

[0002]

2. Description of the Related Art Time-series signals represented by acoustic signals include:
The components include a plurality of periodic signals. For this reason, a method of analyzing what periodic signal is included in a given time-series signal has been known for a long time. For example, Fourier analysis is widely used as a method for analyzing frequency components included in a given time-series signal.

[0003] If such a time-series signal analysis method is used, it is possible to encode an audio signal. With the spread of computers, it has become easier to sample analog audio signals as original sounds at a predetermined sampling frequency, quantize the signal strength at each sampling, and take in as digital data. If a method such as Fourier analysis is applied to the data and frequency components included in the original sound signal are extracted, the original sound signal can be encoded by a code indicating each frequency component.

On the other hand, MIDI (Musical Instrume) was born from the idea of encoding musical instrument sounds by electronic musical instruments.
The Digital Interface (nt Digital Interface) standard has also been actively used with the spread of personal computers. Code data according to the MIDI standard (hereinafter, MID)
I data) is basically data describing an operation of playing a musical instrument, such as which keyboard key of the musical instrument was played and at what strength, and the MIDI data itself contains the actual sound. No waveform is included. Therefore, when reproducing the actual sound, the MI which stores the waveform of the musical instrument sound is used.
Although a DI sound source is required separately, its high coding efficiency has been attracting attention, and the encoding and decoding technology according to the MIDI standard currently uses a personal computer to play musical instruments, practice musical instruments, compose music, and the like. Is widely adopted in software that performs

Therefore, by analyzing a time-series signal represented by an acoustic signal by a predetermined method, a periodic signal as a component of the signal is extracted, and the extracted periodic signal is converted to an M signal.
There have been proposals to encode using IDI data. For example, JP-A-10-247099, JP-A-11-73199, JP-A-11-73200
JP, JP-A-11-95753, JP-A-2000
-99009, JP-A-2000-99092, JP-A-2000-99093, JP-A-2000-99093
No. 261322, JP-A-2001-5450,
Japanese Unexamined Patent Application Publication No. 2001-148633 proposes various methods capable of analyzing frequencies as constituent elements of an arbitrary time-series signal and creating MIDI data from the analysis result.

[0006]

The MIDI encoding scheme proposed in each of the above publications has made it possible to efficiently encode audio signals obtained from performance recordings and the like. However, since the above-mentioned conventional encoding method is a method for processing an audio signal composed of one channel,
Suitable for processing monaural sound signals,
It is not always possible to efficiently encode a two-channel audio signal such as a stereo audio signal. Of course, it is possible to process signals consisting of multiple channels, but since the signals of each channel are processed independently, it is naturally necessary to perform arithmetic processing for the number of channels, resulting in a huge load. Has become.
Also, the amount of code data obtained as a result of the encoding increases according to the number of channels.

[0007] In view of the above, the present invention provides a method for calculating a sound signal having a plurality of channels such as a stereo sound signal which is substantially the same as a sound signal having a single channel such as a monaural sound signal. It is an object of the present invention to provide an audio signal encoding method capable of performing processing with a load and suppressing the amount of encoded data.

[0008]

In order to solve the above-mentioned problems, according to the present invention, a plurality of unit sections are set on a time axis with respect to a sound signal given by a plurality of independent waveform information, A frequency analysis is performed for each section to calculate phoneme data including a frequency, an intensity value, a section start time corresponding to the start point of the unit section, and a section end time corresponding to the end point of the unit section. For all phoneme data obtained by performing the calculation process for all unit sections in each input channel, phoneme data having the same time and the same frequency are integrated between each input channel, and a plurality of integration source Code data that is a set of integrated phoneme data is obtained by creating integrated phoneme data to which an intensity ratio is added based on the intensity value of phoneme data. And it features.

According to the present invention, for an audio signal having a plurality of input channels such as a stereo audio signal, phoneme data is created for each input channel, the phoneme data of each input channel is integrated, and the intensity ratio is adjusted. Because of the addition, audio signals from a plurality of input channels can be treated as code data of one integrated channel, and the data amount and the processing load are reduced. At this time, since the information of the intensity ratio is added,
There is no loss of the intensity value balance of the original input channel.

[0010]

Embodiments of the present invention will be described below in detail with reference to the drawings.

(Basic Principle of Audio Signal Coding Method) First, the basic principle of the audio signal coding method according to the present invention will be described. Since this basic principle is disclosed in the above-mentioned publications, only an outline thereof will be briefly described here.

As shown in FIG. 1A, it is assumed that an analog audio signal is given as a time-series signal. In the example of FIG. 1, the horizontal axis represents time t, and the vertical axis represents amplitude (intensity), and this acoustic signal is shown. Here, first, a process of capturing the analog audio signal as digital audio data is performed. This uses the conventional general PCM method,
The analog audio signal may be sampled at a predetermined sampling frequency and the amplitude may be converted into digital data using a predetermined number of quantization bits. here,
For convenience of explanation, the waveform of the audio data digitized by the PCM method is also shown by the same waveform as the analog audio signal in FIG.

Subsequently, a plurality of unit sections are set on the time axis of the audio signal to be analyzed. In the example shown in FIG. 1A, six times t1 to t6 are equally spaced on the time axis t.
Are defined, and five unit sections d1 to d5 having these times as a start point and an end point are set. In the example of FIG. 1, unit sections having the same section length are all set, but the section length may be changed for each unit section. Alternatively, a section may be set such that adjacent unit sections partially overlap on the time axis.

When the unit sections are set in this way, a representative frequency is selected for each audio signal (hereinafter, referred to as section signal) for each unit section. Each section signal usually contains various frequency components,
For example, a frequency component having a large intensity ratio of the component may be selected as the representative frequency. Here, the representative frequency is generally a so-called fundamental frequency, but a harmonic frequency such as a formant frequency of a voice and a peak frequency of a noise sound source may be treated as the representative frequency. Although only one representative frequency may be selected, depending on the acoustic signal, selecting a plurality of representative frequencies enables more accurate encoding. FIG. 1B shows an example in which three representative frequencies are selected for each unit section, and one representative frequency is encoded as one representative code (for convenience, shown as a musical note in the figure). Have been. Here, three tracks T1, T1,
T2 and T3 are provided in order to accommodate three representative codes selected for each unit section in different tracks.

For example, the representative codes n (d1,1), n (d1,2), n (d1,
3) are accommodated in the tracks T1, T2, T3, respectively. Here, each code n (d1, 1), n (d1,
2), n (d1, 3) are codes indicating note numbers in the MIDI code. The note number in the MIDI code takes 128 values from 0 to 127 and indicates one key of the piano keyboard. Specifically, for example, 440 Hz as a representative frequency
Is selected, this frequency has a note number n = 6
9 (corresponding to the "ra tone (A3 tone)" at the center of the piano keyboard), so that n = 69 is selected as the representative code. However, FIG. 1B is a conceptual diagram showing a representative code obtained by the above-described method in the form of a musical note, and in practice, data relating to the intensity is added to each musical note. For example, in the track T1, e (d1, 1) and e (d) are added together with data indicating pitches of note numbers n (d1, 1), n (d2, 1).
2, 1)... Are stored. The data indicating the strength is determined based on how much the component of each representative frequency is included in the original section signal. Specifically, data indicating the intensity is determined based on the correlation value of the periodic function having each representative frequency with respect to the section signal. FIG. 1 (b)
In the conceptual diagram shown in Fig. 7, the position of each unit section on the time axis is indicated by the position of the note in the horizontal direction, but in actuality, data that accurately indicates the position on the time axis as a numerical value Is added to each note.

It is not always necessary to adopt the MIDI format as a format for encoding an audio signal. However, since the MIDI format is the most widespread as this type of encoding format, the MIDI format code data is practically used. It is preferable to use In the MIDI format, "note on" data or "note off" data exists with "delta time" data interposed. Note-on data is
The "note-off" data is data specifying a specific note number N and velocity V to designate the start of performance of a specific sound, and "note-off" data is data specifying a specific note number N and velocity V. This is data for instructing the end of the performance. The “delta time” data is data indicating a predetermined time interval. Velocity V is a parameter indicating, for example, the speed at which a piano keyboard or the like is depressed (velocity at the time of note-on) and the speed at which the finger is released from the keyboard (velocity at the time of note-off). Or it indicates the strength of the performance end operation.

In the above method, the i-th unit section di
, J note numbers n (d
i, 1), n (di, 2),..., n (di, J) are obtained, and the intensities e (di, 1), e
(Di, 2),..., E (di, J) are obtained. Therefore, MIDI-format code data can be created by the following method. First, as the note number N described in the “note-on” data or “note-off” data, the obtained note number n (d
i, 1), n (di, 2),..., n (di, J) may be used as they are. On the other hand, as the velocity V described in the “note-on” data or the “note-off” data, the obtained intensities e (di, 1) and e (d
i, 2),..., and e (di, J) may be standardized by a predetermined method. The “delta time” data may be set according to the length of each unit section.

(Specific Method for Determining Correlation with Periodic Function) In the method based on the basic principle described above, one or a plurality of representative frequencies are selected for an interval signal, and a periodic signal having this representative frequency is selected. Thus, the section signal is expressed. Here, the selected representative frequency is, literally, a frequency representative of a signal component in the unit section. Specific methods for selecting the representative frequency include a method using a short-time Fourier transform and a method using a generalized harmonic analysis method, as described later. Both methods have the same basic concept. A plurality of periodic functions having different frequencies are prepared in advance, and a periodic function having a high correlation with the section signal in the unit section is selected from the plurality of periodic functions. , And the frequency of the periodic function having a high correlation is selected as a representative frequency. That is, when selecting a representative frequency, an operation for calculating a correlation between a plurality of periodic functions prepared in advance and a section signal in a unit section is performed. Therefore, here, a specific method for obtaining the correlation with the periodic function will be described.

It is assumed that a trigonometric function as shown in FIG. 2 is prepared as a plurality of periodic functions. These trigonometric functions are composed of a pair of a sine function and a cosine function having the same frequency, and have 128 standard frequencies f (0) to
For each of f (127), a pair of a sine function and a cosine function is defined. Here, a pair of functions consisting of a sine function and a cosine function having the same frequency is defined as a periodic function for the frequency. That is, the periodic function for a specific frequency is constituted by a pair of a sine function and a cosine function. The reason why the periodic function is defined by the pair of the sine function and the cosine function is to consider that the correlation value is affected by the phase when calculating the correlation value of the periodic function for the signal. Variables F and k in each trigonometric function shown in FIG. 2 are variables corresponding to sampling frequency F and sample number k for section signal X. For example, a sine wave for a frequency f (0) is represented by sin (2πf (0) k / F), and given an arbitrary sample number k, the same time position as the k-th sample forming the section signal Is obtained.

Here, 128 standard frequencies f
An example in which (0) to f (127) are defined by equations as shown in FIG. 3 will be shown. That is, the n-th (0 ≦ n ≦ 1
The standard frequency f (n) of 27) is represented by the following [Equation 1].
Is defined as

[Equation 1] f (n) = 440 × 2 ^{γ (n)} γ (n) = (n−69) / 12

Defining the standard frequency using such an expression is convenient for finally performing encoding using MIDI data. This is because 128 standard frequencies f (0) to f (12)
7) takes frequency values forming a geometric series, and M
This is because the frequency corresponds to the note number used in the IDI data. Therefore, 128 shown in FIG.
The standard frequencies f (0) to f (127) are frequencies set at equal intervals (in semitone units in MIDI) on a frequency axis represented by a logarithmic scale.

Next, a specific description will be given of a method of obtaining a correlation of each periodic function with respect to an interval signal of an arbitrary interval. For example, as shown in FIG. 4, it is assumed that a section signal X is given for a certain unit section d. Here, it is assumed that sampling is performed at a sampling frequency F for a unit section d having a section length L, and that a total of w sample values have been obtained. 1, 2, 3, ..., k, ..., w-
2, w-1 (the w-th sample shown by a white circle is
The sample is included at the beginning of the next unit section adjacent to the right.) In this case, for any sample number k,
The amplitude value X (k) is given as digital data. In the short-time Fourier transform, X
It is normal to multiply (k) by a window function W (k) such that the weight at the center is close to 1 and the weight at both ends is close to 0 for each sample. That is, X (k) × W
The following correlation calculation is performed by treating (k) as X (k), and a cosine-wave shaped Hamming window is generally used as the shape of the window function. Here, w is described as a constant in the following description. In general, w is changed according to the value of n, and the maximum value of F / f (n) within the range not exceeding the section length L is obtained. It is desirable to set the value to an integral multiple.

The principle of obtaining a correlation value between such a section signal X and a sine function Rn having an n-th standard frequency f (n) will be described. The correlation value A (n) between the two can be defined by the first arithmetic expression in FIG. here,
X (k) is the amplitude value of the sample number k in the section signal X, as shown in FIG. 4, and sin (2πf (n) k
/ F) is the amplitude value of the sine function Rn at the same position on the time axis. This first arithmetic expression can be said to be an expression for calculating the inner product of the amplitude value of the section signal X and the amplitude vector of the sine function Rn for the dimensions of all sample numbers k = 0 to w−1 in the unit section d. .

Similarly, the second operation expression of FIG. 5 is an expression for obtaining a correlation value between the section signal X and a cosine function having the n-th standard frequency f (n). Is B (n)
Given by It should be noted that both the first equation for obtaining the correlation value A (n) and the second equation for obtaining the correlation value B (n) are finally multiplied by 2 / w. Is for normalizing the correlation value, and w is n
This coefficient is also a variable that depends on n, since it is generally changed depending on.

The effective value of the correlation between the section signal X and the standard periodic function having the standard frequency f (n) is, as shown in the third equation of FIG. 5, the value of the correlation A (n) with the sine function. It can be indicated by the root sum square (E (n)) of the correlation value B (n) with the cosine function. If a frequency of the standard periodic function having a large correlation effective value is selected as a representative frequency, the section signal X can be encoded using the representative frequency.

That is, one or more standard frequencies at which the correlation value E (n) is equal to or larger than a predetermined reference may be selected as a representative frequency. Here, the selection condition that “the correlation value E (n) is equal to or larger than a predetermined reference” is set, for example, by setting a certain threshold value and setting a standard value such that the correlation value E (n) exceeds this threshold value. Frequency f (n)
May be set as the representative frequency, but relative selection conditions such as selecting up to the Qth in the order of the magnitude of the correlation value E (n) may be set. May be set.

(Method of Generalized Harmonic Analysis) Here, a method of generalized harmonic analysis useful in encoding an audio signal according to the present invention will be described. As described above, when encoding an audio signal, some representative frequencies having high correlation values are selected for section signals in each unit section. Generalized harmonic analysis is a technique that enables selection of a representative frequency with higher accuracy, and its basic principle is as follows.

It is assumed that a signal S (j) exists in a unit section d as shown in FIG. here,
j is a parameter for the repetition processing (j = 1 to J) as described later. First, correlation values are obtained for this signal S (j) for all 128 periodic functions as shown in FIG. Then, the frequency of one periodic function at which the maximum correlation value is obtained is selected as a representative frequency,
A periodic function having the representative frequency is extracted as an element function. Subsequently, the content signal G as shown in FIG.
(J) is defined. The content signal G (j) is added to the extracted element function as the amplitude of the signal S of the element function.
This is a signal obtained by multiplying the correlation value for (j). For example, as shown in FIG. 2, when a pair of a sine function and a cosine function is used as a periodic function and a frequency f (n) is selected as a representative frequency, a sine function A (n) having an amplitude A (n) is selected. ) Sin (2πf (n) k / F) and cosine function B (n) cos (2πf) having amplitude B (n)
(N) k / F) is the content signal G (j)
(In FIG. 6B, only one function is shown for convenience of illustration). Where A (n), B
Since (n) is a normalized correlation value obtained by the equation in FIG. 5, the content signal G (j) has the frequency f (n) contained in the signal S (j). Signal component.

When the content signal G (j) is obtained in this way, the difference signal S (j + 1) is obtained by subtracting the content signal G (j) from the signal S (j). FIG. 6 (c)
The difference signal S (j + 1) thus obtained is shown. This difference signal S (j + 1) is equal to the original signal S
From (j), it can be said that it is a signal composed of the remaining signal components obtained by removing the signal components having the frequency f (n). Therefore, by increasing the parameter j by 1, the difference signal S (j + 1) is treated as a new signal S (j), and the same processing is performed by setting the parameter j to j = 1 to j = 1.
If J is repeated J times while increasing by 1 to J, J
Representative frequencies can be selected.

The J contained signals G (1) to G (J) output as a result of the correlation calculation are signals that are components of the original section signal X, When encoding X, information indicating the frequency and amplitude (intensity) of these J contained signals may be used as code data. Although J has been described as being the number of representative frequencies, it may be the same as the number of standard frequencies f (n), that is, J = 128, and this is usually performed for the purpose of obtaining a frequency spectrum. It is.

When a predetermined number of frequency groups are selected for each unit section, "information indicating the pitch" corresponding to each frequency of this frequency group and the signal intensity corresponding to each selected frequency are selected. “Information indicating sound intensity”, “Information indicating sound start time of sound” corresponding to the start point of the unit section, “Information indicating sound end time of sound” corresponding to the start point of a unit section following the unit section By generating a predetermined number of code data including four pieces of information, the section signal X in the unit section can be encoded with the predetermined number of code data. If MIDI data is created as code data, a note number is used as "information indicating the pitch", a velocity is used as "information indicating the intensity of the sound", and "Information indicating the sound ending time" using the note-on time as the "information indicating"
May be used as the note-off time.

(Sound Signal Encoding Method According to the Present Invention) To summarize the basic principle of the present invention which is common to the prior art described so far, a unit section is set in an original sound signal, and a plurality of units are set for each unit section. Calculate the signal strength corresponding to the frequency,
One or a plurality of representative frequencies are selected using a periodic function prepared based on the obtained signal strength, and pitch information corresponding to the selected representative frequency and the strength of the selected representative frequency are selected. By generating code data composed of sound intensity information corresponding to, a sounding start time corresponding to the start point of the unit section, and a sounding end time corresponding to the end point of the unit section, encoding of the acoustic signal can be performed. It will be done.

In the sound signal encoding method of the present invention, based on the above-described basic principle, all frequencies corresponding to the prepared periodic functions are used based on the obtained signal intensities, and these frequencies and the intensities of the respective frequencies are used. And the data composed of the section start time corresponding to the start point of the unit section and the section end time corresponding to the end point of the unit section are defined as “phoneme data”, and the phoneme data is further processed to obtain the final This is to obtain encoded data.

Hereinafter, the audio signal encoding method of the present invention will be described with reference to the flowchart shown in FIG. First, an audio signal composed of a plurality of channels is given as an audio signal. Here, it is assumed that a 2-channel stereo sound signal is given as an example. Then, a unit section is set for the sound signals of the left and right channels over the entire section on the time axis (step S1: S
1a and S1b). The method in step S1 is the same as that described with reference to FIG.

Subsequently, the sound signal of each unit section, that is, the section signal, is subjected to frequency analysis to calculate an intensity value corresponding to each frequency, and the four information of the frequency, the intensity value, the start point and the end point of the unit section are calculated. Is calculated (step S2: shown as S2a and S2b in the figure).
Specifically, the correlation strength of the section signal is obtained for 128 kinds of periodic functions as shown in FIG. 2, and four pieces of information of the frequency of the periodic function, the obtained correlation strength, the start point and the end point of the unit section are obtained. Defined as phoneme data. However, in this embodiment,
Instead of selecting a representative frequency as in the case described in the above basic principle, phoneme data corresponding to all prepared periodic functions is obtained. By performing the processing in step S2 for all unit sections, the phoneme data [m, n] (0
.Ltoreq.m.ltoreq.M-1, 0.ltoreq.n.ltoreq.N-1). Here, N is the total number of periodic functions (N = 128 in the above example),
M is the total number of unit sections set in the audio signal. That is, a phoneme data group including M × N phoneme data is obtained.

The process in step S2 is performed on the audio signal of each channel. Therefore, for each channel, a phoneme data group which is a set of phoneme data is obtained. When the phoneme data group is obtained, a process of integrating the phoneme data group obtained from the audio signal of the left channel and the phoneme data group obtained from the audio signal of the right channel is performed (step S3). Specifically, this is performed by integrating phoneme data having the same unit section and the same frequency. At this time, as an intensity value of the obtained integrated phoneme data, an average value of the intensity values of the two original phoneme data to be integrated is given. Further, balance information calculated by the following [Equation 2] is added to the integrated phoneme data.

[Equation 2] Val = E _L / (E _L + E _R )

In the above [Equation 2], E _L indicates the intensity value of the phoneme data of the left channel, and E _R indicates the intensity value of the phoneme data of the right channel. That is, the higher the numerical value of the balance information according to [Equation 2], the higher the intensity value of the signal of the left channel. After all, the integrated phoneme data is composed of the frequency, the frequency intensity, the section start time corresponding to the start point of the unit section, the section end time corresponding to the end point of the unit section, and balance information.
Further, at this time, for each unit section, data indicating a priority mark is given as an attribute to a predetermined number of integrated phoneme data starting from the highest intensity value. The predetermined number is desirably about 16 in consideration of the number of simultaneous sounds of the MIDI sound source. By the integration processing in step S3,
Two phoneme data groups obtained from the left and right channel acoustic signals were integrated into one integrated phoneme data group. Since the data amounts of the original phoneme data group and the integrated phoneme data group are substantially equal, the data amount is reduced by half as a whole.

Although the integrated phoneme data group can be used as the target code data, the following step S
4. By performing the processing in step S5, the data amount can be reduced. When an integrated phoneme data group, which is a set of integrated phoneme data, is obtained, the integrated phoneme data whose intensity value does not reach a predetermined value is deleted from the integrated phoneme data group, and the remaining integrated phoneme data is replaced with an effective intensity. Extracted as valid phoneme data having a value (step S
4). In step S4, the reason why the integrated phoneme data whose intensity value does not reach the predetermined value is deleted is to delete phonemes whose signal level is almost 0 and for which it is determined that no sound actually exists. is there. Therefore, a value that is regarded as a level at which sound does not actually exist is set as the predetermined value.

When an effective phoneme data group, which is a set of effective phoneme data, is obtained in this way, a plurality of effective phoneme data continuous in the time series direction at the same frequency are connected as one connected phoneme data (step S5). . FIG. 8 is a conceptual diagram for explaining connection of valid phoneme data. FIG.
(A) is a figure which shows the mode of the integrated phoneme data group before connection. In FIG. 8A, each rectangle partitioned in a lattice shape indicates integrated phoneme data, and the shaded rectangle is deleted in step S3 because the intensity value does not reach the predetermined value. This is integrated phoneme data, and the other rectangles indicate valid phoneme data. In step S5,
In order to link valid phoneme data having the same frequency (same note number) and continuing in the time t direction, the linking process is performed on the valid phoneme data group shown in FIG.
A connected phoneme data group as shown in (b) is obtained. For example, the valid phoneme data A1, A2, A shown in FIG.
3 are connected to obtain connected phoneme data A as shown in FIG. At this time, the frequency of the newly obtained connected phoneme data A is the effective phoneme data A
1, A2, and A3 are provided with a common frequency, and as the intensity value, the largest one of the effective phoneme data A1, A2, and A3 is given. As the start time, the first effective phoneme data A1 is used. The section start time t1 is given, the end time is the section end time t4 of the last valid phoneme data A3, and the balance information has the maximum intensity value among the valid phoneme data A1, A2, A3. The balance information of the valid phoneme data is given. Both the effective phoneme data and the connected phoneme data include frequency (note number), intensity value, start time, end time, and balance information.
Three valid phoneme data are 1
By being integrated into one connected phoneme data, the data amount is reduced to one third. This means that the MID
In the case of I-coding, it means that it is expressed as one long note, not three short notes.

Further, in step S5, if the priority mark is given to the valid phoneme data having the maximum intensity value among the valid phoneme data from which the connection was made, the integrated connected phoneme data is A priority mark is given. For example, in FIG.
It is assumed that the intensity value of the valid phoneme data A2 among 1, 1, A2, and A3 is the maximum. In this case, if the priority mark is given to the valid phoneme data A2, the priority mark is given to the connected phoneme data A. However, if the priority mark is not given to the valid phoneme data A2, the valid phoneme data A1 or the valid Even if a priority mark is given to the phoneme data A3, no priority mark is given to the connected phoneme data A.

When the connected phoneme data group is obtained as described above, the connected phoneme data to which the priority mark is not added is deleted from the connected phoneme data group to obtain final code data (step S6). ). Since a normal MIDI sound source has a restriction that the number of simultaneous sounds is 16 to 64, phonemes obtained by analysis must be adjusted to this. Conventionally, before performing the connection processing as shown in the above step S5, a predetermined number is extracted from the one having the highest intensity value for each unit section. In the present invention, however, at this time after the connection of the phoneme data, Perform extraction with. At this time, a predetermined number (approximately 16 in this embodiment) of integrated phoneme data is added to each unit section in step S3, and based on the presence or absence of a priority mark reflected in the connected phoneme data in step S5,
Since the connected phoneme data is extracted, the remaining connected phoneme data is less than a predetermined number at the same time. By setting the number of connected phoneme data existing at the same time to a predetermined number or less in this way, code data can be used without waste when a normal MIDI sound source is used. When the processing in steps S4 and S5 is not performed, a set of integrated phoneme data is the target code data. However, the connected phoneme data is also obtained by deleting invalid data from the integrated phoneme data and then connecting only valid data. Therefore, in a broad sense, the term “integrated phoneme data” includes “connected phoneme data” Will also be included.

When MIDI data is created as code data, "frequency" is converted to a note number,
Convert "frequency intensity" to velocity, convert "section start time corresponding to section start point" to note-on time, convert "section end time corresponding to section end point" to note-off time, Convert balance information to pan pot. The pan pot is a control parameter indicating a volume ratio in the MIDI standard. Specifically, the balance information calculated by the above [Equation 2] is multiplied by 100 to obtain 0 to 10
"0" is the strongest left, "50"
Is the center, and “100” is the strongest on the right. When a normal MIDI sound source reads a panpot from the MIDI data, it transmits sound signals having different intensities to the left and right when decoding the MIDI data according to the panpot. Therefore, by using the code data coded according to the present invention, the stereo reproduction can be performed without losing the balance of the left and right even if the data originally consisting of the left and right two channels is integrated into one.

(Application to Sound Source Separation)
The encoding method according to the present invention makes it possible to efficiently encode an audio signal including a plurality of channels, such as a stereo audio signal. Code data for each sound source can be obtained from the signal. That is, it is possible to separate a sound source from a stereo sound signal in which a plurality of sound sources are mixed. Hereinafter, such a method will be specifically described.

To perform sound source separation, balance information included in each connected phoneme data is used. Normally, when music is to be recorded in stereo, the performance of each sound source such as vocals, musical instruments, etc. is recorded, and it is determined whether each sound source is recorded on the left or right during mixing. At this time, there is a predetermined rule as to whether each instrument is recorded on the left or right. For example, vocals are recorded with substantially the same signal strength on the left and right so that they can be heard from the center. With respect to other musical instruments, those recorded with a strong signal intensity on the left and those recorded with a strong signal intensity on the right are determined.
In the present invention, such a rule and the above-described balance information are used.

Specifically, at the time of encoding in step S6, the balance information of each connected phoneme data is read, and the connected phoneme data groups are classified into a plurality of groups according to a preset threshold. For example, by dividing the balance information Val near "0.5" into one group and decoding the connected phoneme data of the group, only the vocals can be reproduced.

When MIDI data is created as this code data, a group of connected phoneme data
Recording is performed on a MIDI standard channel for each group.
If a pan pot is designated for each channel, the channel is always played with the same left-right balance. The MIDI standard panpot can be added in units of MIDI event information (delta time, note number, velocity) corresponding to connected phoneme data.
It can also affect all MIDI event information recorded on one channel. By setting a panpot for each channel, the data amount can be reduced as compared with the case where the panpot is added for each MIDI event information unit.

The preferred embodiment of the present invention has been described above, but it goes without saying that the encoding method is executed by a computer or the like. Specifically, a program for executing the steps shown in the flowchart of FIG. 7 in the above procedure is installed in a computer. And
After digitizing the audio signal composed of a plurality of channels by the PCM method or the like, the digital signal is taken into a computer, and step S
After performing the processing of steps 1 to S5, the computer outputs code data in the MIDI format or the like. The output code data is, for example, MIDI data, MIDI data.
It is reproduced as sound using a sequencer and a MIDI sound source.

[0050]

As described above, according to the present invention,
For a sound signal given by a plurality of independent waveform information, a plurality of unit sections are set on the time axis, a frequency analysis is performed for each set unit section, and a frequency, an intensity value, and a unit section are set. Is calculated by calculating the phoneme data composed of the section start time corresponding to the start point of the section and the section end time corresponding to the end point of the unit section, and performing the processing of the phoneme data calculation for all the unit sections in each input channel. For all phoneme data to be created, phoneme data having the same time and the same frequency are integrated between each input channel, and integrated phoneme data is added with an intensity ratio based on the intensity values of the multiple phoneme data to be integrated. By doing so, code data which is a set of integrated phoneme data is obtained, so that acoustic signals from a plurality of input channels are
This can be treated as integrated one-channel code data, and the data amount and the processing load can be reduced.

Also, when decoding the code data of each output channel, taking into account the rule of the left and right balance of the sound source,
By allocating sound sources in accordance with the intensity ratio, it is possible to use the sound source for sound signals in which sound sources are mixed.

[Brief description of the drawings]

FIG. 1 is a diagram showing a basic principle of an audio signal encoding method according to the present invention.

FIG. 2 is a diagram showing an example of a periodic function used in the present invention.

FIG. 3 is a diagram showing a relational expression between a frequency of each periodic function shown in FIG. 2 and a MIDI note number n.

FIG. 4 is a diagram showing a method of calculating a correlation between a signal to be analyzed and a periodic signal.

FIG. 5 is a view showing a calculation formula for performing the correlation calculation shown in FIG. 4;

FIG. 6 is a diagram showing a basic method of generalized harmonic analysis.

FIG. 7 is a flowchart of an audio signal encoding method according to the present invention.

FIG. 8 is a conceptual diagram for explaining connection of valid phoneme data.

[Explanation of symbols]

 A (n), B (n)... Correlation value d, d1 to d5... Unit section E (n)... Correlation value G (j). Note number S (j), S (j + 1) ... difference signal X, X (k) ... section signal

Claims (5)

[Claims] 1. A plurality of unit sections are set on a time axis with respect to an acoustic signal given by a plurality of independent waveform information, and a frequency analysis is performed for each of the unit sections. A phoneme data calculation step of calculating phoneme data composed of a section start time corresponding to the start point of the unit section and a section end time corresponding to the end point of the unit section, and performing the processing of the phoneme data calculation step in each input channel. For all phoneme data obtained by performing for all unit sections, phoneme data having the same time and the same frequency are integrated between each input channel, and based on the intensity values of a plurality of phoneme data of the integration source. A phoneme data integration step of creating integrated phoneme data to which an intensity ratio is added, wherein code data which is a set of the integrated phoneme data is obtained. Coding method Hibiki signal. 2. A plurality of unit sections are set on a time axis with respect to a sound signal given by a plurality of independent waveform information, and a frequency analysis is performed for each of the unit sections to obtain a frequency, an intensity value, A phoneme data calculation step of calculating phoneme data composed of a section start time corresponding to the start point of the unit section and a section end time corresponding to the end point of the unit section, and performing the processing of the phoneme data calculation step in each input channel. For all phoneme data obtained by performing for all unit sections, while integrating phoneme data having the same time and the same frequency between channels,
A phoneme data integration step of creating integrated phoneme data to which an intensity ratio is added based on the intensity values of a plurality of phoneme data to be integrated, and using the intensity ratio of the integrated phoneme data, a volume between output channels of a sound source device for reproduction. A decoding step of controlling a balance and reproducing an audio signal from a plurality of output channels. 3. The decoding step further comprising: grouping the integrated phoneme data based on the intensity ratio of the integrated phoneme data, and adding an intensity ratio to the grouped integrated phoneme data group. The method according to claim 2, wherein the grouping step reproduces an acoustic signal by assigning a predetermined sound source to the grouped integrated phoneme data group based on the intensity ratio. A method for encoding and decoding an audio signal. 4. The encoded data is MIDI data comprising a note number, a velocity, and a delta time.
The method according to claim 2 or 3, wherein the intensity ratio is panpot information. 5. A computer sets a plurality of unit sections on a time axis for an acoustic signal given by a plurality of independent waveform information, performs frequency analysis for each of the unit sections, and A phoneme data calculation step of calculating phoneme data composed of an intensity value, a section start time corresponding to the start point of the unit section, and a section end time corresponding to the end point of the unit section; For all phoneme data obtained by performing for all unit sections in the channel, phoneme data having the same time and the same frequency are integrated between each input channel, and the intensity values of a plurality of phoneme data of the integration source are integrated. A program for executing a phoneme data integration step of creating integrated phoneme data to which an intensity ratio is added based on the data.