JP4697919B2 - Method for encoding an acoustic signal - Google Patents

Description

[0001]
[Industrial application fields]
The present invention can be applied to support the following work called music transcription in music production. For music transcription, for example, citation of existing music as a material when musical score is not available, cover music production of existing music, music analysis such as melody, harmony progression, tone analysis of hit music, MIDI data in karaoke Format performance data creation, game machine BGM data creation, mobile phone ringtone data creation, performance data creation for keyboard instruments with automatic piano and performance guide function, score publication and block creation.
[0002]
[Prior art]
A time-series signal represented by an acoustic signal includes a plurality of periodic signals as its constituent elements. For this reason, a method for analyzing what kind of periodic signal is included in a given time-series signal has been known for a long time. For example, Fourier analysis is widely used as a method for analyzing frequency components included in a given time series signal.
[0003]
By using such a time-series signal analysis method, an acoustic signal can be encoded. With the spread of computers, it has become easy to sample an analog audio signal as the original sound at a predetermined sampling frequency, quantize the signal intensity at each sampling, and capture it as digital data. If a method such as Fourier analysis is applied to the data and the frequency components included in the original sound signal are extracted, the original sound signal can be encoded by a code indicating each frequency component.
[0004]
On the other hand, the MIDI (Musical Instrument Digital Interface) standard, which was born from the idea of encoding musical instrument sounds by electronic musical instruments, has been actively used with the spread of personal computers. The code data according to the MIDI standard (hereinafter referred to as MIDI data) is basically data that describes the operation of the musical instrument performance such as which keyboard key of the instrument is played with what strength. The data itself does not include the actual sound waveform. Therefore, when reproducing the actual sound, a MIDI sound source storing the waveform of the instrument sound is separately required. However, its high encoding efficiency is attracting attention, and encoding and decoding according to the MIDI standard are being attracted attention. This technology is now widely used in software that uses a personal computer to perform musical instrument performance, practice and compose music.
[0005]
Therefore, by analyzing a time-series signal represented by an acoustic signal by a predetermined method, a periodic signal as a constituent element is extracted, and the extracted periodic signal is encoded using MIDI data. Proposals have been made. For example, JP-A-10-247099, JP-A-11-73199, JP-A-11-73200, JP-A-11-95753, JP-A-2000-99009, JP-A-2000-99092, JP-A-2000-99093, JP-A-2000-261322, JP-A-2001-5450, and JP-A-2001-148633 analyze the frequency as a component of an arbitrary time-series signal, Various methods for creating MIDI data from the analysis results have been proposed.
[0006]
[Problems to be solved by the invention]
The MIDI encoding method proposed in each of the above publications or specifications has enabled efficient encoding of acoustic signals obtained from performance recordings and the like. In the conventional coding method, as disclosed in Japanese Patent Laid-Open No. 11-95753, a phoneme obtained by performing frequency analysis for each unit section (in this specification, a set of frequency and intensity corresponding to the frequency). Is called a phoneme) to a predetermined number. This is because a normal MIDI sound source has a restriction that the number of simultaneous pronunciations is 16 to 64, and the phoneme obtained by analysis must be matched to this. Therefore, for each unit section, sorting is performed to about 16 on the basis of the intensity value.
[0007]
However, when sorting is performed for each unit section in this way, since the role of phonemes in the whole is not considered, the intensity value is small in a certain unit section, such as the rise or end of sound, but it is important Even if it is a part of a simple sound, it will be deleted, and accurate coding cannot be performed.
[0008]
Accordingly, in the specification of Japanese Patent Application No. 2001-8750, the present applicant assigns a priority mark of about 16 to phonemes having high intensity values for each unit section, and then connects and connects the phonemes in consecutive sections. We proposed a method for obtaining phonemes and creating code data based on these connected phonemes.
[0009]
In this method, the number of simultaneous pronunciations can be controlled to some extent at the stage before connection, and as described above, phonemes that constitute part of important sounds are not deleted, but after connection, Since the overlapping number of connected phonemes at the same time increases by an average of about twice, there is a problem that the number of simultaneous sounds cannot be limited to a specified number range.
[0010]
In view of the above points, the present invention provides a code of an acoustic signal that does not lose a part of an important sound and that can contain overlapping phonemes at the same time so that they can be simultaneously pronounced. It is an object to provide a conversion method.
[0011]
[Means for Solving the Problems]
  In order to solve the above problems, in the present invention, a plurality of unit sections are set on a time axis for a given acoustic signal, and correlations between the acoustic signal and the plurality of periodic functions in the set unit section are obtained. By calculating the intensity value corresponding to each periodic function, the frequency of each periodic function, the intensity value corresponding to each periodic function, the section start time corresponding to the start point of the unit section, and the end point of the unit section The unit phoneme data composed of the end time of the section corresponding to is calculated, and the intensity value has reached a predetermined value from all unit phoneme data obtained by performing this unit phoneme data calculation process for all unit sections. The remaining unit phoneme data is deleted as effective phoneme data having an effective intensity value, and the extracted phoneme data has the same frequency and continuous sections. Concatenated into concatenated phoneme data. As the concatenated phoneme data attribute, the intensity value gives the maximum intensity value of the effective phoneme data, the start time gives the section start time of the first effective phoneme data, and the end time ends. The effective phoneme data section end time is given, and all phoneme data after the concatenation process is searched for temporally overlapping phoneme data,Delete phoneme data whose attribute frequency is an integer multiple of the frequency of other phoneme data between the temporally overlapping phoneme dataAndDeleteAn acoustic signal is expressed by a set of later phoneme data. According to the present invention, the frequency analysis is performed on the acoustic signal for each unit interval, the unit phoneme data is calculated, the connection process is performed, and the phoneme data after the connection process is subjected to temporally overlapping phoneme data. Check for duplicate phoneme dataDelete phoneme data whose attribute frequency is an integer multiple of the frequency of other phoneme dataAs a result, it is possible to reduce the number of phonemes that overlap at the same time to less than the number that can be pronounced simultaneously without losing some of the important sounds.
[0012]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
[0013]
(Basic principle of acoustic signal encoding method)
First, the basic principle of the audio signal encoding method according to the present invention will be described. Since this basic principle is disclosed in the above-mentioned publications or specifications, only the outline will be briefly described here.
[0014]
As shown in FIG. 1A, it is assumed that an analog acoustic signal is given as a time-series signal. In the example of FIG. 1, the acoustic signal is shown with time t on the horizontal axis and amplitude (intensity) on the vertical axis. Here, first, the analog sound signal is processed as digital sound data. This may be performed by using a conventional general PCM method, sampling the analog acoustic signal at a predetermined sampling frequency, and converting the amplitude into digital data using a predetermined number of quantization bits. Here, for convenience of explanation, the waveform of the acoustic data digitized by the PCM method is also shown by the same waveform as the analog acoustic signal of FIG.
[0015]
Subsequently, a plurality of unit sections are set on the time axis of the acoustic signal to be analyzed. In the example shown in FIG. 1A, six times t1 to t6 are defined at equal intervals on the time axis t, and five unit intervals d1 to d5 having these times as the start point and the end point are set. In the example of FIG. 1, unit sections having the same section length are set, but the section length may be changed for each unit section. Alternatively, the section setting may be performed such that adjacent unit sections partially overlap on the time axis.
[0016]
When the unit section is set in this way, representative frequencies are selected for the acoustic signals (hereinafter referred to as section signals) for each unit section. Each section signal usually includes various frequency components. For example, a frequency component having a high component intensity ratio may be selected as the representative frequency. Here, the so-called fundamental frequency is generally used as the representative frequency, but a harmonic frequency such as a formant frequency of speech or a peak frequency of a noise source may be treated as a representative frequency. Although only one representative frequency may be selected, more accurate encoding is possible by selecting a plurality of representative frequencies depending on the acoustic signal. FIG. 1B shows an example in which three representative frequencies are selected for each unit section, and one representative frequency is encoded as one representative code (shown as a note for convenience in the drawing). Has been. Here, three tracks T1, T2 and T3 are provided to accommodate representative codes (notes), but this means that three representative codes selected for each unit section are assigned to different tracks. It is for accommodating.
[0017]
For example, representative codes n (d1,1), n (d1,2), n (d1,3) selected for the unit section d1 are accommodated in tracks T1, T2, T3, respectively. Here, each code n (d1,1), n (d1,2), n (d1,3) is a code indicating a note number in the MIDI code. The note number in the MIDI code takes 128 values from 0 to 127, each indicating one key of the piano keyboard. Specifically, for example, when 440 Hz is selected as the representative frequency, this frequency corresponds to the note number n = 69 (corresponding to “ra sound (A3 sound)” in the center of the piano keyboard). N = 69 is selected. However, FIG. 1B is a conceptual diagram showing the representative code obtained by the above-described method in the form of a note. In reality, data on intensity is also added to each note. For example, the track T1 includes e (d1,1), e (d2,1)... Along with data indicating the pitches of note numbers n (d1,1), n (d2,1). Data indicating the strength is accommodated. The data indicating the intensity is determined by the degree to which the component of each representative frequency is included in the original section signal. Specifically, the data indicating the intensity is determined based on the correlation value with respect to the section signal of the periodic function having each representative frequency. Further, in the conceptual diagram shown in FIG. 1B, the position of each unit section on the time axis is indicated by the position of the note in the horizontal direction, but in reality, the position on the time axis is shown. Is accurately added as a numerical value to each note.
[0018]
As a format for encoding an acoustic signal, it is not always necessary to adopt the MIDI format. However, since the MIDI format is the most popular as this type of encoding, code data in the MIDI format is practically used. Is preferred. In the MIDI format, “note-on” data or “note-off” data exists while interposing “delta time” data. The “note-on” data is data for designating a specific note number N and velocity V to instruct the start of a specific sound, and the “note-off” data is a specific note number N and velocity V. This is data that designates the end of the performance of a specific sound. The “delta time” data is data indicating a predetermined time interval. Velocity V is a parameter that indicates, for example, the speed at which a piano keyboard is pressed down (velocity at the time of note-on) and the speed at which the finger is released from the keyboard (velocity at the time of note-off). Or it shows the strength of the performance end operation.
[0019]
In the above-described method, J note numbers n (di, 1), n (di, 2),..., N (di, J) are obtained as representative codes for the i-th unit interval di. Intensities e (di, 1), e (di, 2),..., E (di, J) are obtained for each of these. Therefore, MIDI format code data can be created by the following method. First, as the note number N described in the “note on” data or “note off” data, the obtained note numbers n (di, 1), n (di, 2),..., N (di , J) can be used as they are. On the other hand, as the velocity V described in the “note on” data or “note off” data, the obtained intensities e (di, 1), e (di, 2),..., E (di, A value obtained by normalizing J) by a predetermined method may be used. The “delta time” data may be set according to the length of each unit section.
[0020]
(Specific method for obtaining correlation with periodic function)
In the method based on the basic principle described above, one or a plurality of representative frequencies are selected for the section signal, and the section signal is represented by a periodic signal having this representative frequency. Here, the representative frequency to be selected is literally a frequency representing the signal component in the unit section. Specific methods for selecting the representative frequency include a method using a short-time Fourier transform and a method using a generalized harmonic analysis method, as will be described later. Both methods have the same basic concept. Prepare a plurality of periodic functions with different frequencies in advance, and from these periodic functions, a periodic function that has a high correlation with the section signal in the unit section. And a method of selecting the frequency of the highly correlated periodic function as a representative frequency is adopted. That is, when selecting a representative frequency, an operation for obtaining a correlation between a plurality of periodic functions prepared in advance and a section signal in a unit section is performed. Therefore, here, a specific method for obtaining the correlation with the periodic function will be described.
[0021]
Assume that trigonometric functions as shown in FIG. 2 are prepared as a plurality of periodic functions. These trigonometric functions are composed of a pair of a sine function and a cosine function having the same frequency. For each of 128 standard frequencies f (0) to f (127), a pair of a sine function and a cosine function. Is defined. Here, a pair of functions consisting of a sine function and a cosine function having the same frequency is defined as a periodic function for the frequency. That is, the periodic function for a specific frequency is constituted by a pair of sine function and cosine function. Thus, the periodic function is defined by a pair of sine function and cosine function in order to consider that the correlation value is influenced by the phase when obtaining the correlation value of the periodic function with respect to the signal. The variables F and k in each trigonometric function shown in FIG. 2 are variables corresponding to the sampling frequency F and the sample number k for the section signal X. For example, a sine wave with respect to the frequency f (0) is represented by sin (2πf (0) k / F), and given an arbitrary sample number k, the same time position as the k-th sample constituting the section signal The amplitude value of the periodic function at is obtained.
[0022]
Here, an example in which 128 standard frequencies f (0) to f (127) are defined by the equations as shown in FIG. That is, the nth (0 ≦ n ≦ 127) standard frequency f (n) is defined by the following [Formula 1].
[0023]
[Formula 1]
f (n) = 440 × 2^{γ (n)}
γ (n) = (n−69) / 12
[0024]
If the standard frequency is defined by such an expression, it is convenient when finally encoding using MIDI data is performed. This is because the 128 standard frequencies f (0) to f (127) set by such a definition take frequency values forming a geometric series, and correspond to the note numbers used in the MIDI data. This is because it becomes a frequency. Therefore, the 128 standard frequencies f (0) to f (127) shown in FIG. 2 are frequencies set at equal intervals (in semitone units in MIDI) on the frequency axis shown on the logarithmic scale.
[0025]
Next, a specific description will be given of how to obtain the correlation of each periodic function with respect to a section signal in an arbitrary section. For example, as shown in FIG. 4, it is assumed that a section signal X is given for a certain unit section d. Here, it is assumed that sampling is performed at the sampling frequency F for the unit interval d having the interval length L, and w sample values are obtained in total, and the sample numbers are 0, 1, 2, 3,..., K,..., W-2, w-1 (the w-th sample indicated by a white circle is a sample included at the head of the next unit section adjacent to the right. And). In this case, for an arbitrary sample number k, an amplitude value of X (k) is given as digital data. In the short-time Fourier transform, it is usual to multiply the window function W (k) such that the center weight is close to 1 and the weights at both ends are close to 0 for each sample with respect to X (k). That is, X (k) × W (k) is treated as X (k) and the following correlation calculation is performed. As the shape of the window function, a cosine wave-shaped Hamming window is generally used. Here, w is described as a constant in the following description, but in general, it is changed according to the value of n, and F / f (n) that is maximum within a range not exceeding the section length L. It is desirable to set the value to an integer multiple.
[0026]
The principle of obtaining a correlation value with such a section signal X and the sine function Rn having the nth standard frequency f (n) is shown. Both correlation values A (n) can be defined by the first arithmetic expression of FIG. Here, X (k) is the amplitude value of the sample number k in the section signal X, as shown in FIG. 4, and sin (2πf (n) k / F) is the sine at the same position on the time axis. This is the amplitude value of the function Rn. This first arithmetic expression can be said to be an expression for obtaining the inner product of the amplitude value of the section signal X and the amplitude vector of the sine function Rn for the dimensions of all sample numbers k = 0 to w−1 in the unit section d. .
[0027]
Similarly, the second arithmetic expression in FIG. 5 is an expression for obtaining a correlation value between the interval signal X and the cosine function having the nth standard frequency f (n), and the correlation value between the two is B ( n). The first arithmetic expression for obtaining the correlation value A (n) and the second arithmetic expression for obtaining the correlation value B (n) are finally multiplied by 2 / w. Is for normalizing the correlation value. As described above, since w is generally changed depending on n, this coefficient is also a variable depending on n.
[0028]
The effective correlation value between the interval signal X and the standard periodic function having the standard frequency f (n) is the correlation value A (n) with the sine function, the cosine function, as shown in the third arithmetic expression of FIG. Of the square sum of squares E (n) with the correlation value B (n). If the frequency of the standard periodic function having a large correlation effective value is selected as the representative frequency, the section signal X can be encoded using this representative frequency.
[0029]
That is, one or a plurality of standard frequencies whose correlation value E (n) is greater than or equal to a predetermined reference may be selected as the representative frequency. Here, the selection condition that “correlation value E (n) is greater than or equal to a predetermined reference” is, for example, a standard in which some threshold value is set and correlation value E (n) exceeds this threshold value. An absolute selection condition that all frequencies f (n) are selected as representative frequencies may be set. For example, up to the Qth in the order of the correlation value E (n) is selected. A relative selection condition may be set.
[0030]
(Method of generalized harmonic analysis)
Here, a generalized harmonic analysis technique useful when encoding an acoustic signal according to the present invention will be described. As already described, when encoding an acoustic signal, several representative frequencies having high correlation values are selected for the section signal in each unit section. Generalized harmonic analysis is a technique that enables the selection of representative frequencies with higher accuracy, and the basic principle thereof is as follows.
[0031]
Assume that there is a signal S (j) for the unit interval d as shown in FIG. Here, j is a parameter for repetitive processing (j = 1 to J), as will be described later. First, correlation values for all 128 periodic functions as shown in FIG. 2 are obtained for this signal S (j). Then, the frequency of one periodic function having the maximum correlation value is selected as a representative frequency, and the periodic function having the representative frequency is extracted as an element function. Subsequently, the inclusion signal G (j) as shown in FIG. 6B is defined. The inclusion signal G (j) is a signal obtained by multiplying the extracted element function by the correlation value of the element function with respect to the signal S (j) of the element function. For example, as shown in FIG. 2, when a frequency f (n) is selected as a representative frequency using a pair of sine function and cosine function as shown in FIG. 2, a sine function A (n) having an amplitude A (n). ) Sin (2πf (n) k / F) and a signal composed of the sum of cosine function B (n) cos (2πf (n) k / F) having amplitude B (n) is included signal G (j) (In FIG. 6B, only one function is shown for convenience of illustration). Here, since A (n) and B (n) are normalized correlation values obtained by the equation of FIG. 5, the inclusion signal G (j) is eventually included in the signal S (j). It can be said that the signal component has a certain frequency f (n).
[0032]
Thus, when the content signal G (j) is obtained, the difference signal S (j + 1) is obtained by subtracting the content signal G (j) from the signal S (j). FIG. 6C shows the difference signal S (j + 1) obtained in this way. The difference signal S (j + 1) can be said to be a signal composed of the remaining signal components obtained by removing the signal component having the frequency f (n) from the original signal S (j). Therefore, by increasing the parameter j by 1, this difference signal S (j + 1) is handled as a new signal S (j), and the same processing is performed J times while increasing the parameter j by 1 from j = 1 to J. If it is repeatedly executed, J representative frequencies can be selected.
[0033]
The J inclusion signals G (1) to G (J) output as a result of such correlation calculation are signals that are constituent elements of the original section signal X, and the original section signal X is encoded. In this case, information indicating the frequency of these J inclusion signals and information indicating the amplitude (intensity) may be used as the code data. Although J has been described as the number of representative frequencies, it may be the same as the number of standard frequencies f (n), that is, J = 128. For the purpose of obtaining a frequency spectrum, this is usually done. It is.
[0034]
Thus, when a predetermined number of frequency groups are selected for each unit section, “information indicating the pitch” corresponding to each frequency of this frequency group, and “sound intensity” corresponding to the signal intensity of each selected frequency. "Information indicating strength", "information indicating the start time of sound generation" corresponding to the start point of the unit section, "information indicating the end time of sound generation" corresponding to the start point of the unit section subsequent to the unit section, If a predetermined number of pieces of code data including the four pieces of information are created, the section signal X in the unit section can be encoded with the predetermined number of pieces of code data. If MIDI data is created as code data, a note number is used as “information indicating the pitch of the sound”, velocity is used as the “information indicating the intensity of the sound”, and “sound generation start time is set. The note-on time may be used as the “information indicating” and the note-off time may be used as the “information indicating the end time of sound generation”.
[0035]
(Acoustic signal encoding method according to the present invention)
To summarize the basic principle of the present invention common to the conventional techniques described so far, unit intervals are set in the original sound signal, signal intensities corresponding to a plurality of frequencies are calculated for each unit interval, and the obtained signal intensities are calculated. One or more representative frequencies are selected using a periodic function prepared based on the sound pitch information corresponding to the selected representative frequency and the sound frequency corresponding to the intensity of the selected representative frequency. The sound signal is encoded by creating code data composed of intensity information, a sounding start time corresponding to the start point of the unit section, and a sounding end time corresponding to the end point of the unit section. become.
[0036]
The acoustic signal encoding method of the present invention uses all the frequencies corresponding to the prepared periodic functions based on the obtained signal strength in the above basic principle, and each of these frequencies, the strength of each frequency, and the unit Data consisting of the section start time corresponding to the start point of the section and the section end time corresponding to the end point of the unit section is defined as “phoneme data”, and final encoded data is obtained by further processing this phoneme data. It is something to get.
[0037]
From here, the acoustic signal encoding method of the present invention will be described with reference to the flowchart shown in FIG. First, a unit section is set over all sections on the time axis of the acoustic signal (step S1). The technique in step S1 is as described with reference to FIG. 1A in the basic principle.
[0038]
Subsequently, for each acoustic signal in each unit section, that is, the section signal, a frequency analysis is performed to calculate an intensity value corresponding to each frequency, and a unit comprising four pieces of information of frequency, intensity value, unit section start point, and end point Phoneme data is calculated (step S2). Specifically, the correlation strength of the section signal is obtained for 128 types of periodic functions as shown in FIG. 2, and four pieces of information of the frequency of the periodic function, the calculated correlation strength, the start point and the end point of the unit section are obtained. It is defined as “unit phoneme data”. It is assumed that the unit phoneme data is created especially in the first unit section of the phoneme data. In the present embodiment, unit phoneme data corresponding to all the prepared periodic functions is acquired instead of selecting a representative frequency as in the case described in the basic principle. The unit phoneme data [m, n] (0 ≦ m ≦ M−1, 0 ≦ n ≦ N−1) group is obtained by performing the process of step S2 on all unit sections. Here, N is the total number of periodic functions (N = 128 in the above example), and M is the total number of unit sections set in the acoustic signal. That is, a unit phoneme data group composed of M × N unit phoneme data is obtained.
[0039]
When the unit phoneme data group is obtained, the phoneme data whose intensity value does not reach the predetermined value is deleted from the unit phoneme data group, and the remaining phoneme data is extracted as effective phoneme data having an effective intensity value. (Step S3). In this step S3, the phoneme data whose intensity value does not reach the predetermined value is deleted because the phoneme whose signal level is almost 0 and which is judged to be actually absent is deleted. . Therefore, a value that is regarded as a level where no sound actually exists is set as the predetermined value.
[0040]
When an effective phoneme data group which is a set of effective phoneme data is obtained in this way, a plurality of effective phoneme data continuous in the time-series direction at the same frequency are connected as one connected phoneme data (step S4). FIG. 8 is a conceptual diagram for explaining connection of effective phoneme data. FIG. 8A is a diagram illustrating a state of a phoneme data group before connection. In FIG. 8 (a), each rectangle partitioned in a lattice pattern indicates phoneme data, and the shaded rectangle is a phoneme deleted because the intensity value does not reach the predetermined value in step S3. The other rectangles indicate effective phoneme data. In step S4, in order to concatenate effective phoneme data continuous in the time t direction at the same frequency (same note number), when the concatenation process is executed on the effective phoneme data group shown in FIG. A phoneme data group including a plurality of connected phoneme data and a plurality of effective phoneme data as shown in b) is obtained. For example, the effective phoneme data A1, A2, and A3 shown in FIG. 8A are connected to obtain connected phoneme data A as shown in FIG. 8B. At this time, as the frequency of the newly obtained connected phoneme data A, a common frequency is given to the effective phoneme data A1, A2, and A3, and the intensity value is the intensity value of the effective phoneme data A1, A2, and A3. The largest one is given, the start time t1 of the effective phoneme data A1 is given as the start time, and the end time t4 of the last effective phoneme data A3 is given as the end time. Since both effective phoneme data and connected phoneme data are composed of four pieces of information of frequency (note number), intensity value, start time, and end time, the three effective phoneme data are integrated into one connected phoneme data. The amount of data is reduced to one third. This means that when MIDI encoding is finally performed, it is expressed not as three short notes but as one long note. In addition, when there is no effective phoneme data continuous in the time series direction at the same frequency as in the effective phoneme data B shown in FIG. 8A, as shown in FIG. Although it remains as it is, in the subsequent processing, the concatenated phoneme data and the effective phoneme data having unit lengths that are not concatenated are collectively treated as “phoneme data”.
[0041]
Subsequently, the number of phoneme data duplicated at each time is checked, and adjustment processing is performed when the number of overlapping phone numbers is larger than the number that can be simultaneously generated (step S5). As a specific process, first, a duplication management table is prepared, and phoneme data is registered in the duplication management table in the order of pronunciation start time. Here, an adjustment process for overlapping phoneme data using the overlap management table will be described with reference to FIG. FIG. 9A shows the phoneme data in the phoneme data group arranged in the order of the start time.
[0042]
FIG. 9A shows only the start time and end time of each phoneme data. For example, phoneme A indicates that sounding starts at time “0” and continues until time “3”. For such a phoneme data group, phonemes are registered in the duplication management table in units of time. In the following description, it is assumed that the simultaneous pronunciation number is set to “4”. First, phonemes are registered in the duplication management table in order of pronunciation start time. Since the number of simultaneous pronunciations is set to “4”, as shown in FIG. 9B, up to phoneme D, phonemes are simply registered in the duplication management table.
[0043]
When the phoneme E is registered in the duplication management table, five phonemes are arranged in the duplication management table as shown in FIG. In this case, the phoneme is reduced by one so that the set number of simultaneous pronunciations is “4”. Specifically, one phoneme having a low priority is selected from the five phonemes registered in the duplication management table, and the selected phoneme is changed. In the example of FIG. 9C, phoneme A and phoneme B with the earliest end time are candidates. Since the phoneme A and the phoneme B have the end time “3” at the same time, the phoneme data is changed for the lower intensity value. For example, if the intensity value of the phoneme B is lower, the end time of the phoneme B is set to the same time “2” as the start time of the phoneme E newly registered in the duplication management table as shown in FIG. Change to When the phoneme data to be changed is determined in the duplication management table, the phoneme data in the actual phoneme data group is also changed. In addition, the phoneme B that does not overlap in time with the other four phonemes on the duplication management table due to the change is deleted from the duplication management table.
[0044]
When the phoneme F that is the next phoneme data is registered in the duplication management table, five phonemes are registered in the duplication management table as shown in FIG. Since the phoneme A having the earliest end time can be specified, the end time of the phoneme A is changed to the same time “2” as the start time of the phoneme F newly registered in the duplication management table. At the same time, the phoneme data in the phoneme data group is also changed. At this time, the state of the phoneme data in the phoneme data group is as shown in FIG. The end times of phoneme A and phoneme B are changed, as can be seen by comparing before the overlap adjustment process of FIG. 9A and after the overlap adjustment process of FIG. 9E. In the duplication management table, phoneme A is deleted, the next phoneme is registered in the duplication management table, and the above-described processing is repeated. By performing the above overlapping phoneme adjustment processing on all phoneme data in the phoneme data group, a phoneme data group that does not exceed the set number of simultaneous pronunciations at all times can be obtained. .
[0045]
Here, the overlapping phoneme adjustment processing in step S5 will be described in an organized manner using the flowchart shown in FIG. First, one phoneme data of the phoneme data group is registered in the duplication management table in order of start time (step S11). Subsequently, the phoneme data that ends by the start time of the newly registered phoneme data is deleted from the duplication management table (step S12). Here, it is determined whether or not the number of phoneme data arranged on the duplication management table exceeds the limit value n (step S13). The limit number n is normally set to about 16, which is the number of simultaneously soundable MIDI standards. If the number of phonemes registered on the duplication management table is n or less, the process returns to step S11 and the process is repeated. The example described with reference to FIG. 9 is the case of n = 4, and the processing from step S11 to step S13 is repeated until the number of phonemes on the duplication management table becomes 5, as shown in FIG. 9C. That's right.
[0046]
If it is determined in step S13 that the number of phonemes registered on the duplication management table exceeds n, a phoneme having a low priority is selected from n + 1 phonemes registered on the duplication management table. (Step S14). In the example described with reference to FIG. 9, when the number of phonemes is five as shown in FIG. 9C, the phoneme B is selected as the lowest priority. In the example of FIG. 9, the priority is determined based on the strength value.
[0047]
When the phoneme having the lowest priority is selected, the end time of the selected phoneme is changed, or the selected phoneme itself is deleted (step S15). Basically, priority is given to changing the end time of phonemes. Specifically, the end time of the selected phoneme is changed to be the same as the start time of the phoneme newly registered in the duplication management table. Thereby, there is no time overlap between the selected phoneme and the newly registered phoneme, and the set limit number is not exceeded. In the example described with reference to FIG. 9, the end time of phoneme B is changed to be the same as the start time “2” of phoneme E newly registered in the duplication management table, as shown in FIG. 9C. As shown in FIG. 9D, the end time of the phoneme A is changed to be the same as the start time “2” of the phoneme F newly registered in the duplication management table. The case where the phoneme itself is deleted is a case where the phoneme itself disappears by changing the end time of the phoneme. For example, for a phoneme whose start time is “1” and whose end time is “2”, when the end time is changed, the end time is also “1” and the sound production time is “0”. In such a case, even if the data exists, there is no meaning, so the phoneme itself is deleted. If the process of said step S11-step S15 is performed about all the phonemes, ie, all the phoneme data which exist in a phoneme data group, a process will be complete | finished (step S16).
[0048]
When the process of step S5 is completed according to the procedure shown in the flowchart of FIG. 10, it may be encoded as it is into MIDI-format encoded data. However, in this embodiment, the total number of phoneme data is adjusted to reduce the amount of data. (Step S6). Specifically, when the total number of phoneme data in the phoneme data group exceeds the set number, the total number of phoneme data is kept within a predetermined range by deleting phonemes having low importance. In the present embodiment, a value calculated by (end time−start time) × intensity value of each phoneme data is adopted as the priority. That is, the ones with low values are deleted sequentially. When the total number of phoneme data is adjusted, encoding is performed in the MIDI format (step S7).
[0049]
(Harmonic component removal processing)
With the encoding method according to the present invention as described above, it is possible to reduce the number of phonemes that overlap at the same time without missing a part of important sounds, In this case, it is possible to perform a harmonic component removal process due to the feature of the method. A harmonic is a sound having a frequency that is an integral multiple of the frequency of the basic sound, which is the original sound. If the harmonic component is encoded as it is, the original sound cannot be accurately reproduced. In terms of the MIDI note number, the harmonic component takes values such as +12, +19, +24, +28, +31,.
[0050]
Next, a case where a harmonic component removal process is performed will be described. Specifically, when the overlapping phoneme is processed in step S5, the frequency of one phoneme among the plurality of phonemes registered in the overlap management table is an integral multiple of the frequency of the other phoneme. Find out if there is a relationship. If such a relationship is found, the ratio of the intensity value of the phoneme having the higher frequency to the intensity value of the phoneme having the lower frequency (considered as a basic sound) is calculated. If this ratio is less than or equal to a predetermined value, the phoneme with the higher frequency is determined to be a harmonic and is deleted. If the intensity value ratio is not less than or equal to the predetermined value, the phoneme is not a harmonic component and is likely to be a basic sound, and is not deleted.
[0051]
Although the preferred embodiments of the present invention have been described above, the encoding method is naturally executed by a computer or the like. Specifically, a program for executing the steps shown in the flowcharts of FIGS. 7 and 10 according to the above procedure is installed in the computer. Then, after the acoustic signal is digitized by the PCM method or the like, it is taken into a computer, and after steps S1 to S6 and steps S11 to S16 are performed, code data such as MIDI format is output from the computer. For example, in the case of MIDI data, the output code data is reproduced as sound using a MIDI sequencer and a MIDI sound source. The duplication management table is realized by assigning a predetermined storage area such as a RAM in the computer.
[0052]
【The invention's effect】
  As described above, according to the present invention, for a given acoustic signal, a plurality of unit sections are set on the time axis, and the correlation between the acoustic signal and the plurality of periodic functions in the set unit section is calculated. By calculating the intensity value corresponding to each periodic function, the frequency of each periodic function, the intensity value corresponding to each periodic function, the section start time corresponding to the start point of the unit section, and the unit section The unit phoneme data composed of the section end time corresponding to the end point is calculated, and the intensity value reaches a predetermined value from all the unit phoneme data obtained by performing this unit phoneme data calculation process for all unit sections. The remaining unit phoneme data is extracted as effective phoneme data having an effective intensity value, and the extracted phoneme data has the same frequency and continuous sections. Concatenated into concatenated phoneme data. As the concatenated phoneme data attribute, the intensity value gives the maximum intensity value of the effective phoneme data, the start time gives the section start time of the first effective phoneme data, and the end time ends. The effective phoneme data section end time is given, and all phoneme data after the concatenation process is searched for temporally overlapping phoneme data,Delete phoneme data whose attribute frequency is an integer multiple of the frequency of other phoneme data between the temporally overlapping phoneme dataAndDeleteSince the acoustic signal is expressed by a set of later phoneme data, it is possible to keep a part of important sound missing and keep the number of overlapping phonemes at the same time or less. There is an effect that it becomes possible.
[Brief description of the drawings]
FIG. 1 is a diagram showing a basic principle of an audio signal encoding method according to the present invention.
FIG. 2 is a diagram showing an example of a periodic function used in the present invention.
3 is a diagram showing a relational expression between the frequency of each periodic function shown in FIG. 2 and a MIDI note number n. FIG.
FIG. 4 is a diagram illustrating a method of calculating a correlation between a signal to be analyzed and a periodic signal.
FIG. 5 is a diagram showing a calculation formula for performing the correlation calculation shown in FIG. 4;
FIG. 6 is a diagram showing a basic method of generalized harmonic analysis.
FIG. 7 is a flowchart of an acoustic signal encoding method according to the present invention.
FIG. 8 is a conceptual diagram for explaining connection of effective phoneme data.
FIG. 9 is a diagram showing a state of a phoneme data group and a duplication management table.
10 is a flowchart showing details of step S5 in FIG. 7;
[Explanation of symbols]
A (n), B (n) ... correlation value
d, d1 to d5 ... unit interval
E (n) ... correlation value
G (j) ... Inclusion signal
n, n1 to n6 ... note number
S (j), S (j + 1)... Differential signal
X, X (k) ... section signal

Claims (5)

A section setting stage for setting a plurality of unit sections on the time axis for a given acoustic signal,
By calculating the correlation between the acoustic signal in the unit section and a plurality of periodic functions, the intensity value corresponding to each periodic function is calculated, the frequency of each periodic function, the intensity value corresponding to each periodic function, A unit phoneme data calculation stage for calculating unit phoneme data composed of a section start time corresponding to the start point of the unit section and a section end time corresponding to the end point of the unit section;
Delete all unit phoneme data obtained by performing the processing of the unit phoneme data for all unit intervals from the unit phoneme data whose intensity value does not reach the predetermined value, and use the remaining unit phoneme data as effective intensity. An effective phoneme data extraction stage to extract as effective phoneme data having a value;
For the effective phoneme data extracted in the effective phoneme data extraction step, those having the same frequency and continuous sections are connected to form connected phoneme data, and the intensity value is configured as an attribute of the connected phoneme data. Giving the maximum intensity value of the effective phoneme data, the start time giving the section start time of the first effective phoneme data, the end time giving the section end time of the last effective phoneme data,
For all phoneme data after the concatenation process, search for phoneme data that overlaps in time, and between the phoneme data that overlap in time, the frequency that is an attribute is an integer multiple of the frequency of other phoneme data A phoneme number adjustment stage for deleting phoneme data ,
An encoding step of expressing an acoustic signal by a set of phoneme data after the deletion ;
A method for encoding an acoustic signal, comprising: After the overlapping phoneme number adjustment step, a total phoneme number adjustment step of reducing the total number of phoneme data by deleting less important phoneme data based on the product of the phoneme data pronunciation time and intensity value The method for encoding an acoustic signal according to claim 1 . The overlapping phoneme number adjustment step moves the end time of the phoneme data having the smallest intensity value among the temporally overlapping phoneme data to the start time side, thereby determining the number of temporally overlapping phoneme data. The method for encoding an acoustic signal according to claim 1 or 2 , wherein the adjustment is performed so as to be equal to or less than a predetermined value. The overlapping phoneme number adjustment step sequentially registers the phoneme data in the duplication management table based on the order of the start time which is an attribute of the phoneme data, and among the phoneme data already registered in the duplication management table The phoneme data whose section end time is set before the start time of the phoneme data to be newly registered is deleted from the duplication management table, and the phoneme data registered in the duplication management table exceeds a predetermined number In this case, one of the phoneme data registered in the duplication management table is selected, the selected phoneme data in the phoneme data group is corrected, and the selected phoneme data is stored in the duplication management table. The number of phoneme data overlapping in time is adjusted so as to become a predetermined value or less by deleting more. Encoding method of the audio signal according to any one of claims 3 to 1. A section setting stage for setting a plurality of unit sections on a time axis for a given acoustic signal to a computer, and by obtaining a correlation between the acoustic signal in the unit section and a plurality of periodic functions, The corresponding intensity value is calculated, the frequency of each periodic function, the intensity value corresponding to each periodic function, the section start time corresponding to the start point of the unit section, and the section end time corresponding to the end point of the unit section A unit phoneme data calculation stage for calculating unit phoneme data to be configured, and intensity values have not reached a predetermined value from all unit phoneme data obtained by performing the processing of the unit phoneme data calculation stage for all unit sections And the remaining unit phoneme data is extracted as effective phoneme data having an effective intensity value, and extracted by the effective phoneme data extraction step. With respect to the effective phoneme data, those having the same frequency and continuous sections are connected to form connected phoneme data, and as the attribute of the connected phoneme data, the intensity value is the maximum intensity value of the effective phoneme data constituting the phoneme data. And the start time gives the section start time of the first effective phoneme data, the end time gives the section end time of the last effective phoneme data, a phoneme data connection stage that gives the time for all the phoneme data after the connection process manner overlapping examine phoneme data, among phonemic data overlapping said temporally overlapping phonemes speed adjusting step of deleting phoneme data whose frequency is the attribute is an integral multiple of the frequency of the other phonemes data, the deletion A program for executing an encoding stage for expressing an acoustic signal by a later set of phoneme data.

Images