JP2001117564A - Device and method for processing musical sound - Google Patents

Abstract

PROBLEM TO BE SOLVED: To interpolate plural spectro-shapes. SOLUTION: An analysis data extracting part 11 receiving a sound generation information signal and an interpolation information signal judges the sound generation state of the sound generation information signal, and extracts a formant shape stored for every sound generation state from an analysis data storage part 12 and stores it in an interpolation buffer for every instrument. Following this, an interpolation processing part 13 repeats interpolation processing of three or more instruments by a spectrum interpolation method using anchor points until the interpolated spectro-shapes become all in one.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a musical sound processing apparatus and a musical sound processing method, and more particularly to a musical sound processing apparatus and a musical sound processing method for generating a new musical instrument sound obtained by synthesizing various instrument features.

[0002]

2. Description of the Related Art In a conventional tone processing apparatus, a new tone color is generated by interpolating two tone colors. As for the technique of interpolating two spectrum shapes, a voice quality conversion method that divides two spectrum envelopes into a plurality of frequency bands on the basis of the energy-concentrated frequency and performs spectrum conversion for each band is described. By extracting corresponding time points between parameters set for two original sounds and Japanese Patent Application Laid-Open No. 9-244694, parameters are interpolated according to the degree of desired closeness to the two original sounds, and new parameters are extracted. Interpolated tone color synthesizing method for generating a
0 ".

[0003]

However, since there is no technique for synthesizing three or more timbres, even if it is attempted to interpolate pronunciation information composed of timbres of a guitar, a piano, and a flute, for example, a guitar and a piano, and a piano and a flute are interpolated. Alternatively, only a tone signal interpolated between two timbres, such as a guitar and a flute, could be output. Therefore, an object of the present invention is to provide a tone processing device and a tone processing method that can interpolate three or more spectral shapes and generate more new tone.

[0004]

In order to solve the above-mentioned problems, the invention according to claim 1 stores analysis data obtained by analyzing sampling data of sounds generated in an instrument. Analysis data storage means based on externally input pronunciation information and interpolation information input externally to generate one tone signal corresponding to three or more types of instruments, based on the analysis data storage means. Means for extracting the analysis data corresponding to three or more instruments from the means; and a spectrum analyzer corresponding to the analysis data.
Interpolation processing means for performing an interpolation process based on a shape to generate one interpolation spectrum shape, and tone signal generation means for generating and outputting a tone signal based on the interpolation spectrum shape. And

According to a second aspect of the present invention, in the tone processing apparatus according to the first aspect, the analysis data is a sine wave component and a residual component obtained as a result of frequency analysis of the sampling data on a frame basis. It is characterized by:

According to a third aspect of the present invention, in the tone processing apparatus according to the second aspect, the analysis data storage means stores the sine wave component and the residual error corresponding to a plurality of tone generation states for each of the instruments. It is characterized by storing components.

According to a fourth aspect of the present invention, in the musical sound processing apparatus of the first aspect, the interpolation processing means performs (n-1) times of the interpolation processing when there are n types of the spectrum shapes. It is characterized by performing.

According to a fifth aspect of the present invention, in the musical sound processing apparatus according to the fourth aspect, the interpolation processing means performs a grouping process of dividing the spectrum shape into two types of sets, and performs each grouping process. Repeat the interpolation process,
It is characterized in that one interpolated spectrum shape is generated.

According to a sixth aspect of the present invention, in the tone processing apparatus according to the fifth aspect, when the number of the spectral shapes is an odd number, the interpolation processing means may perform grouping in the grouping process. Spectrum that could not be
The feature is that the shape is processed in the next grouping process.

According to a seventh aspect of the present invention, in the musical sound processing apparatus according to the first aspect, the interpolation processing means performs the interpolation processing using two spectral shapes when performing the interpolation processing using the two spectral shapes. Is characterized by using a spectral shape transition function.

According to an eighth aspect of the present invention, in the tone processing apparatus of the seventh aspect, the transition function is defined in advance as a linear function or a non-linear function.

According to a ninth aspect of the present invention, in the tone processing apparatus according to the seventh aspect, the interpolation processing means divides the two spectral shapes into a plurality of regions on a frequency axis, and belongs to each region. The interpolation processing using a linear function as the transition function is performed over the plurality of regions with respect to a set of real frequencies and magnitudes on the two spectral shapes.

According to a tenth aspect of the present invention, in the tone processing apparatus according to the ninth aspect, the interpolation processing means includes a first frequency which is a frequency of one of the spectral shapes belonging to each of the regions and the first frequency. Frequency interpolation means for calculating an interpolation frequency by interpolating the second frequency, which is the frequency of the other spectral shape corresponding to the above, using the linear function, and the magnitude of one of the spectral shapes belonging to each of the regions. And a magnitude interpolating means for interpolating the first magnitude and a second magnitude corresponding to the other magnitude corresponding to the first magnitude using the linear function.

According to an eleventh aspect of the present invention, in the tone processing apparatus according to the first aspect, the interpolation information includes overall interpolation information for performing an interpolation process on a synthesis ratio between the instruments, and interpolation for a sounding state. It is characterized by including state interpolation information for performing processing, pitch interpolation information for performing interpolation processing for pitch, and amplitude interpolation information for performing interpolation processing for amplitude.

According to a twelfth aspect of the present invention, in the tone processing apparatus of the eleventh aspect, there is provided an interpolation information setting means for setting the interpolation information.

According to a thirteenth aspect of the present invention, in the tone processing apparatus according to the eleventh aspect, a preset means for holding the preset interpolation information is provided.

According to a fourteenth aspect of the present invention, in the tone processing apparatus according to the second aspect, the interpolation processing means interpolates the sine wave component and the residual component stored by the analysis data storage means. Thus, there is provided an analysis data calculation means for calculating a sine wave component and a residual component which are not stored by the analysis data storage means.

According to a fifteenth aspect of the present invention, in the tone processing apparatus according to the first aspect, the tone signal generating means performs an inverse fast Fourier transform process, an overlap process, and an envelope process on the interpolated spectrum shape. It is characterized in that a tone waveform in the time domain is generated by performing lobe addition processing.

According to a sixteenth aspect of the present invention, in the tone processing apparatus of the first aspect, a MIDI tone color selecting means for selecting a tone color of MIDI data, and an interpolation information storage means for storing MIDI interpolation information of the tone color. And interpolation information generation means for generating the interpolation information based on the timbre and the MIDI interpolation information. The analysis data extraction means, based on the interpolation information and pronunciation information input from the outside, The analysis data is extracted from the analysis data storage means.

According to a seventeenth aspect of the present invention, in the tone processing apparatus of the sixteenth aspect, change input means for inputting change interpolation data, and the interpolation corresponding to the change interpolation data input by the change input means. And interpolating information replacing means for replacing information.

According to an eighteenth aspect of the present invention, in the tone processing apparatus according to the first aspect, voice input means for inputting voice, and frequency analysis of input voice input from the voice input means is performed in frame units. Frequency analysis means for extracting a voice sine wave component and a voice residual component, and one interpolated spectrum shape generated by the interpolation processing means and the voice sine wave component and the voice residual extracted by the frequency analysis means Morphing means for performing morphing processing on the sine component with a predetermined morphing degree, wherein the analysis data extracting means performs the analysis data extraction based on pronunciation information contained in the audio sine wave component and interpolation information input from the outside. Extracting the analysis data corresponding to three or more instruments from a storage means. That.

According to a nineteenth aspect of the present invention, in the tone processing apparatus of the eighteenth aspect, there is provided morphing setting means for setting the predetermined morphing degree, and the morphing means is provided with a morphing set by the morphing setting means. The morphing of the interpolated spectrum shape, the audio sine wave component and the audio residual component is performed according to the degree.

According to a twentieth aspect of the present invention, in the musical tone processing device according to the nineteenth aspect, the musical tone signal generating means comprises:
It is characterized in that a musical tone waveform in the time domain is generated by performing an inverse fast Fourier transform process and an overlap process on the interpolated spectrum shape.

According to a twenty-first aspect of the present invention, in the tone processing apparatus according to the first aspect, voice input means for inputting voice and frequency analysis of the input voice input from the voice input means are performed in frame units. Frequency analysis means for extracting a voice sine wave component and a voice residual component, and the analysis data extraction means, based on pronunciation information included in the voice sine wave component and interpolation information input from the outside, The analysis data corresponding to three or more instruments is extracted from the analysis data storage means. The tone signal generation means generates an interpolation tone signal based on the interpolation spectrum shape, Generating an audio tone signal based on the component and the audio residual component, and mixing and outputting the interpolated tone signal and the audio tone signal. It is a symptom.

According to a twenty-second aspect of the present invention, in the tone processing apparatus according to the twenty-first aspect, the interpolation processing means uses a pitch determined from a predetermined ratio corresponding to a pitch of the input voice. A pitch shift unit that shifts the pitch of the sine wave component; and a residual component addition unit that adds a residual component to the sine wave component whose pitch has been shifted by the pitch shift unit. Features.

According to a twenty-third aspect of the present invention, in the tone processing apparatus according to the twenty-first aspect, the tone signal generating means comprises:
It is characterized in that a musical tone waveform in the time domain is generated by performing an inverse fast Fourier transform process and an overlap process on the interpolation spectrum shape or the audio sine wave component and the audio residual component.

According to a twenty-fourth aspect of the present invention, based on externally input tone generation information and externally input interpolation information for generating one tone signal corresponding to three or more types of instrument timbres. An analysis data extracting step of extracting the analysis data corresponding to three or more instruments from the analysis data obtained by analyzing the sampling data of the sounds pronounced in the instrument; An interpolation process for generating an interpolation spectrum shape by performing an interpolation process based on the corresponding spectrum shape; and a tone signal generation process for generating and outputting a tone signal based on the interpolation spectrum shape. It is characterized by that.

According to a twenty-fifth aspect of the present invention, in the tone processing method of the twenty-fourth aspect, the analysis data is a sine wave component and a residual component obtained as a result of frequency analysis of the sampling data on a frame basis. It is characterized by:

According to a twenty-sixth aspect of the present invention, in the tone processing method according to the twenty-fourth aspect, the interpolation processing step includes (n-1) times of the interpolation processing when there are n types of the spectral shapes. It is characterized by performing.

According to a twenty-seventh aspect of the present invention, in the musical sound processing method according to the twenty-sixth aspect, the interpolation processing is performed for each of the two sets of the spectral shapes. The interpolation processing is repeated to generate one interpolation spectrum shape.

According to a twenty-eighth aspect of the present invention, in the tone processing method according to the twenty-seventh aspect, in the interpolation processing step, when the number of the spectral shapes is an odd number, the grouping processing may be performed. It is characterized in that spectrum shapes that could not be processed are processed in the next grouping process.

According to a twenty-ninth aspect of the present invention, in the tone processing method according to the twenty-fourth aspect, the interpolating step includes the step of performing the interpolating process using two spectral shapes between the two spectral shapes. Is characterized by using a spectral shape transition function.

According to a thirtieth aspect of the present invention, in the tone processing method according to the twenty-ninth aspect, the interpolation processing step divides the two spectral shapes into a plurality of regions on a frequency axis, and belongs to each region. The interpolation processing using a linear function as the transition function is performed over the plurality of regions with respect to a set of real frequencies and magnitudes on the two spectral shapes.

According to a thirty-first aspect of the present invention, in the tone processing method according to the thirty-third aspect, the interpolating process includes a first frequency which is a frequency of one of the spectral shapes belonging to each of the regions and the first frequency. And a frequency interpolation process of calculating an interpolation frequency by interpolating a second frequency which is a frequency of the other spectral shape corresponding to the above-described linear function, and a magnitude of one of the spectral shapes belonging to each of the regions. And a magnitude interpolation step of interpolating the first magnitude and the second magnitude, which is the magnitude of the other spectral shape corresponding to the first magnitude, using the linear function.

According to a thirty-second aspect of the present invention, in the tone processing method according to the twenty-fourth aspect, the interpolation information includes overall interpolation information for performing an interpolation process on a synthesis ratio between the instruments, and interpolation for a tone generation state. It is characterized by including state interpolation information for performing processing, pitch interpolation information for performing interpolation processing for pitch, and amplitude interpolation information for performing interpolation processing for amplitude.

According to a thirty-fourth aspect of the present invention, in the tone processing method according to the twenty-fourth aspect, the MIDI tone color selecting step of selecting a tone color of MIDI data, and the interpolation based on the tone color and MIDI interpolation information of the tone color. An interpolation information generating step of generating information, wherein the analysis data extracting step extracts the analysis data based on the interpolation information and pronunciation information inputted from outside.

According to a thirty-fourth aspect of the present invention, in the tone processing method according to the twenty-fourth aspect, a voice input step of inputting voice, and a voice analysis performed by frequency-analyzing the input voice input in the voice input step in frame units A frequency analysis step of extracting a sine wave component and a voice residual component, one interpolated spectrum shape generated by the interpolation process, and the voice sine wave component and the voice residual component extracted by the frequency analysis process. A morphing process of performing a morphing process on the sine wave component at a predetermined morphing degree. The analysis data extracting process comprises the steps of: The method is characterized in that the analysis data corresponding to the instrument is extracted.

According to a thirty-fifth aspect of the present invention, in the tone processing method according to the thirty-fourth aspect, there is provided a morphing setting step of setting the predetermined morphing degree, and the morphing step is a morphing set by the morphing setting step. The morphing of the interpolated spectrum shape, the audio sine wave component and the audio residual component is performed according to the degree.

According to a thirty-sixth aspect of the present invention, in the tone processing method according to the twenty-fourth aspect, a voice input step of inputting a voice and a frequency analysis of the input voice input in the voice input step are performed on a frame-by-frame basis. A frequency analysis step of extracting a sine wave component and a voice residual component, wherein the analysis data extraction step includes three types of processing based on pronunciation information included in the voice sine wave component and interpolation information input from the outside. Extracting the analysis data corresponding to the above instrument, the tone signal generating step generates an interpolated tone signal based on the interpolated spectrum shape, and generates the audio sine wave component and the audio residual component. It is characterized in that a voice tone signal is generated based on this, and the interpolated tone signal and the voice tone signal are mixed and output.

According to a thirty-seventh aspect of the present invention, in the tone processing method according to the thirty-sixth aspect, the interpolating step is performed based on a pitch obtained from a predetermined ratio corresponding to a pitch of the input voice. A pitch shifting step of shifting a pitch of the sine wave component, and a residual component adding step of adding a residual component to the sine wave component whose pitch has been shifted by the pitch shifting step. Features.

[0041]

Next, a preferred embodiment of the present invention will be described with reference to the drawings. [1] First Embodiment [1.1] Overall Configuration of Music Processing Device FIG. 1 shows the overall configuration of a music processing device according to a first embodiment of the present invention. The musical sound processing device 1 analyzes a sampled audio waveform of an instrument by an SMS analysis described later, and an analysis data storage unit 12 that stores analysis data obtained as a result of the analysis in advance, and an externally input analysis data. An analysis data extraction unit 11 that extracts analysis data stored in an analysis data storage unit 12 based on a pronunciation information signal generated by the user and interpolation information signals generated by interpolation information set by a user; An interpolation processing unit 13 that generates a new frequency component by performing interpolation based on the frequency component; a frequency synthesis processing unit 14 that performs an inverse fast Fourier transform (IFFT) and an overlap process on the generated frequency component; An envelope processing unit 1 for performing an envelope adding process on the frequency signal generated by the processing unit 14 When it is configured to include an output unit 16 for outputting the output musical tone signals generated by the envelope processing unit 15.

Here, the pronunciation information in the present embodiment refers to information including the name and duration of a sound, such as "8th note". Therefore, data such as MDINote which specifies an octave and a name of a sound and MIDIList which specifies a type of a note are also included in the pronunciation information.

Next, the above-mentioned interpolation information will be described in detail. The interpolation information in the present embodiment refers to multidimensional interpolation information including various interpolation rates that can be set for each instrument. Multidimensional interpolation information includes
There are an overall multidimensional interpolation rate, a state multidimensional interpolation rate in each sounding state described later, a pitch multidimensional interpolation rate, and an amplitude multidimensional interpolation rate. The overall multidimensional interpolation ratio indicates a synthesis ratio of the instrument with respect to the entire output musical tone signal when the instrument is synthesized. For example, the total multidimensional interpolation rate is changed from “0” to “1” for each instrument. It can be set between. More specifically, for example, when synthesizing the timbres of a guitar, a piano, and a flute, the timbre of the guitar is set to 50% of the entire timbre, the timbre of the piano is set to 30% of the entire timbre, and the timbre of the flute is set to the When the synthesis is performed with 20% of the timbre, the overall multidimensional interpolation rate of the guitar is “0.5”, the overall multidimensional interpolation rate of the piano is “0.3”, and the overall multidimensional interpolation rate of the flute is “0.5”. 0.2 "respectively. Then, the total multidimensional interpolation rates of the guitar, piano, and flute must be set to be "1" when summed.

The state multidimensional interpolation rate is a rising state multidimensional interpolation rate, a steady state multidimensional interpolation rate, which is an interpolation rate corresponding to each sounding state, for example, a rising state of sound, a steady state, and an open state. And the open state multidimensional interpolation rate. Each state multidimensional interpolation rate indicates a synthesis ratio of the instrument with respect to the entire output tone signal when an instrument is synthesized in each sounding state. It can be set between "0" and "1" for each sounding state of the comment. To be more specific, for example, in the state of rising sound when synthesizing the sounds of the guitar, piano and flute,
If the guitar tone is 40% of the whole tone, the piano tone is 50% of the whole tone, and the flute tone is 10% of the whole tone, the guitar's start-up state multidimensional interpolation rate Is set to "0.4", the multi-dimensional interpolation rate of the rising state of the piano is set to "0.5", and the multi-dimensional interpolation rate of the rising state of the flute is set to "0.1". And
The rising state multidimensional interpolation rates for the guitar, piano, and flute must be set so as to be "1" when summed.

Here, in the rising state, a case where the sound suddenly starts with a strong attack such as cut-in, or a case where the sound starts from a weak sound and gradually increases the sound such as fade-in, etc. State can be set. In the open state, it is possible to set a state in which the sound is suddenly turned off, such as a cutout, or a state in which the sound is ended while gradually weakening the sound, such as a fadeout. .

The pitch multidimensional interpolation rate is used to adjust the pitch of the instrument to interpolate the timbre. For example, when the pitch is output at the same pitch as the key-on designated pitch, it is set to “0”, and It can be set arbitrarily between "-1" and "1" for each instrument. More specifically, for example, when synthesizing the tone of a guitar, a piano and a flute, when increasing the pitch of the guitar, setting the pitch of the piano to the same pitch as the key-on designated pitch, and lowering the pitch of the flute, Sets the guitar pitch multidimensional interpolation rate from "0" to
It is set between "1", the pitch multidimensional interpolation rate of the piano is set to "0", and the pitch multidimensional interpolation rate of the flute is set between "0" and "-1".

The amplitude multidimensional interpolation rate is:
The tone color is interpolated by adjusting the amplitude (amplitude) of the instrument. For example, when the same formant as the key-on designated application is used, “0” is set, and “−1” is set for each instrument. Can be arbitrarily set between "1" and "1". Specifically, for example, when synthesizing the tone of the guitar, the piano and the flute, the amplitude of the guitar is increased, the amplitude of the piano is set to the same amplitude as the amplitude specified for the key-on, and the amplitude of the flute is changed. To lower it, set the amplitude multi-dimensional interpolation rate of the guitar from "0" to
It is set between “1”, the amplitude multidimensional interpolation rate of the piano is set to “0”, and the amplitude multidimensional interpolation rate of the flute is set between “0” and “−1”.

Next, a method for setting the above-described multidimensional interpolation information for each instrument will be specifically described. First, for example, multi-dimensional interpolation information (whole, rising state, steady state, open state, pitch, amplitude)
In this case, when the timbres of the instruments such as guitar, piano, and flute are interpolated so that the timbre of the guitar becomes slightly stronger, the multidimensional interpolation information of the guitar is expressed as (0.4, 0 , 0, 0, 0, 0), and the multidimensional interpolation information of the piano is (0.3, 0, 0,
0, 0, 0), and the multidimensional interpolation information of the flute is set to (0.3, 0, 0, 0, 0, 0). In this case, the total multidimensional interpolation rate of each instrument is “1” when summed.

Next, in the above-mentioned example, the tone of the guitar having a strong attack is output more strongly at the start of the sound generation, the tone of the piano is output more strongly in the steady state, and the flute is output in the open state. In the case of performing interpolation so that a tone is output more strongly, the multidimensional interpolation information of the guitar is set to (0.4, 1, 0, 0, 0, 1),
The multidimensional interpolation information of the piano is (0.3, 0, 1, 0, 0,
0) and set the multidimensional interpolation information of the flute to (0.3,
0, 0, 1, 0, 0). In this case, not only the tone color of the guitar but also the tone color of the piano and the flute are synthesized and output to the tone signal output in the rising state. This is because each state multidimensional interpolation rate is applied according to the ratio of the total multidimensional information rate.

[1.2] Configuration of Each Unit of the Music Processing Unit [1.2.1] Analysis Data Storage Unit FIG. 2 shows the file configuration of the analysis data storage unit 12.
As shown in FIG. 2, the analysis data storage unit 12 divides the amplitude into a certain size and the pitch into a certain size for each sounding state of each instrument. The data is stored in the analysis data table unit represented by the pitch division. Then, formant shape data is stored in a region where the amplitude section and the pitch section intersect. In addition, a residual component is also stored in the area as needed. Note that the formant shape and the residual component are obtained by SMS analysis described later.

Here, the analysis data table shown in FIG. 2 is an analysis data table in which an instrument is a saxophone, and a sounding state is a rising state and a formant shape with a strong attack feeling is stored.
In addition, for example, the instrument is a saxophone, the sounding state is a rising state, and an analysis data table that stores a formant shape with a weak attack feeling, the instrument is a guitar, and the sounding state is a steady state The analysis data table storing a formant shape having a strong attack feeling is also stored in the analysis data storage unit 12. In this manner, the analysis data table has a state in which the instrument and the degree of sound generation are different (pronunciation state). ) Stores the von Mart shape and the residual component.

The standard of the size for dividing the pitch division is changed according to the characteristics of the instrument.
For example, for instruments whose pitch change does not have a significant effect on formant fluctuations, try to reduce the number of pitch divisions. For the musical instruments that have an influence, the number of pitch divisions is set finely to increase. Also, the standard of the size for dividing the amplitude division is determined in the same manner as the standard of the size for dividing the pitch division. In the analysis data table shown in FIG. 2, the pitch represented by the vertical axis is 1
The division is set for each [octave], and the amplitude indicated by the horizontal axis is set for every 10 [dB].

Here, the SMS analysis will be described with reference to FIG. In the SMS analysis, first, a frame obtained by multiplying a sampled speech waveform by a window function is extracted, and then a frequency obtained by performing a fast Fourier transform (FFT) on the extracted frame. From the spectrum, a sine wave component and a residual component are extracted. Here, the sine wave component refers to a component of a fundamental frequency (pitch; Pitch) and a frequency (harmonic) that is a multiple of the fundamental frequency. In this embodiment, the fundamental frequency is stored as pitch, the average amplitude of each component is stored as amplitude, and the spectrum envelope is stored in the analysis data storage unit 12 as formant shape. The residual component is a component obtained by removing the sine wave component from the frequency spectrum described above, and in the present embodiment, is stored in the analysis data storage unit 12 as frequency domain data as shown in FIG. . The residual component is a component that is particularly useful when generating a rising sound.

[1.2.2] Interpolation processing unit The interpolation processing unit 13 temporarily stores the formant shape extracted based on the sounding information signal and the interpolation information signal for each instrument for each instrument unit. Then, an interpolation spectrum shape, which is a new frequency component, is generated by interpolating the formant shape of each instrument stored in the interpolation buffer by a spectrum interpolation method described later.

[1.3] Operation of First Embodiment Next, an operation example of the tone processing apparatus 1 will be described in order. First,
From outside the tone processing device 1, for example, MIDInote
When the pronunciation information signal is input, the analysis data extraction unit 11 divides the input pronunciation information signal into the pronunciation information signals in frame units. Specifically, for example, when a pronunciation information signal indicating “eighth note, pitch of the second octave, amplitude of 6 [dB]” is input, the analysis data extraction unit 11 sets the pronunciation information The signal is divided into frames divided every 5 [ms], and for each frame, "eighth note, pitch of the second octave, amplitude of 6 [dB]" and data indicating the sounding state in the frame Is stored.

Next, the analysis data extraction unit 11 determines whether the sounding state of the sounding information signal divided in frame units is a signal indicating a rising state, a signal indicating a steady state, or a signal indicating an open state. Is determined. The analysis data extraction unit 11 then stores the formant and stored for each instrument and for each state.
A shape is retrieved from the analysis data storage unit 12 based on the pitch and amplitude of the pronunciation information signal interpolated by the pitch multidimensional interpolation rate and the amplitude multidimensional interpolation rate included in the interpolation information signal input from the outside, The analysis data extracted by the search is temporarily stored in an interpolation buffer distinguished for each instrument.

Here, the extraction processing by the analysis data extraction unit 11 will be specifically described with reference to FIG. FIG. 9 shows an analysis data table when the instrument is a piano and the sounding state is a rising state. The analysis data extraction unit 11 determines that the pronunciation information signal of the first frame is
For example, the data is “8th note, pitch is the second octave, amplitude is 6 [dB], rising state”, and the interpolation information signal of the piano is (0.3, 0,
1, 0, 0, 0), the pitch multi-dimensional interpolation rate and the amplitude multi-dimensional interpolation rate included in the interpolation information signal are both “0”, so that the pitch included in the pronunciation information signal is “0”. The formant of a piano whose second octave is "" and whose amplitude is "6 [dB]"
Extract the shape 9a. Then, the formant shape 9a is stored in an area corresponding to the first frame of the piano interpolation buffer. Next, the analysis data extraction unit 11 reads the pronunciation information signal of the second frame, and the pronunciation information signal is, for example, “eighth note, pitch of the second octave, amplitude of 15 [dB], rising edge. State information ", and when the piano interpolation information signal is (0.3, 0, 1, 0, 0, 0), the pitch multidimensional interpolation rate and the amplitude multiplicity included in the interpolation information signal are included. Since the dimensional interpolation rates are both “0”, the pitch included in the pronunciation information signal is “second octave”.
, And the formant shape 9b of the piano whose amplitude is “15 [dB]” is extracted. And
The formant shape 9b is stored in an area corresponding to the second frame of the piano interpolation buffer.

The interpolation processing unit 13 performs an interpolation process of three or more instruments included in the interpolation information signal by a spectrum interpolation method using anchor points described below.

First, the purpose of using the spectrum interpolation is roughly classified into the following two. (1) Spectrum of two temporally consecutive frames
The shape is interpolated to determine the spectral shape of the frame that is temporally between the two frames. (2) Interpolate the spectrum shapes of two different sounds to obtain an intermediate sound spectrum shape.

As shown in FIG. 4 (a), two spectrum shapes (hereinafter referred to as a first spectrum shape SS1 and a second spectrum shape SS2 for convenience) to be interpolated are each on the frequency axis. Multiple zones Z on
Divide into 1, Z2, ... Then, the frequency of the boundary dividing each region is set for each spectrum shape as follows. The frequency at the set boundary is called an anchor point. First spectral shape SS1: RB1,1, RB2,1,..., RBN, 1 Second spectral shape SS2: RB1,2, RB2,2,.

FIG. 4B is an explanatory diagram of linear spectrum interpolation. Linear spectral interpolation is defined by an interpolation rate, where the interpolation rate X ranges from 0 to 1. In this case, the interpolation rate X = 0 corresponds to the first spectrum shape SS1 itself, and the interpolation rate X = 1 corresponds to the second spectrum shape SS2 itself.

FIG. 4B shows a case where the interpolation ratio X = 0.35. Further, in FIG. 4B, white circles (○) on the vertical axis indicate respective sets of frequencies and magnitudes constituting the spectrum shape. Therefore,
It is appropriate to consider that the magnitude axis exists in the direction perpendicular to the paper. The first spectrum on the axis with the interpolation rate X = 0
The anchor points corresponding to a certain region Zi of interest of the shape SS1 are RBi, 1 and RBi + 1,1, and the frequency of any one of a set of a specific frequency and a magnitude belonging to the region Zi = fi1 and magnitude = S1 (fi1).

The anchor points corresponding to a certain region Zi of interest of the second spectral shape SS2 on the axis with the interpolation ratio X = 1 are RBi, 2 and RBi + 1,2, and the concrete points belonging to the region Zi It is assumed that the frequency of any one of the set of the typical frequency and the magnitude is fi2, and the magnitude is S2 (fi2). Here, a spectrum transition function ftrans1 (x) and a spectrum transition function ftrans2 (x) are obtained.

For example, if these are represented by the simplest linear function, the following is obtained. ftrans1 (x) = m1.x + b1 ftrans2 (x) = m2.x + b2 where m1 = RBi, 2-RBi, 1 b1 = RBi, 1 m2 = RBi + 1,2-RBi + 1,1 b2 = RBi + 1,2. Next, a set of frequency and magnitude on the interpolated spectrum shape corresponding to the set of frequency and magnitude actually existing on the first spectrum shape SS1 is determined.

First, a set of a frequency and a magnitude actually existing on the first spectrum shape SS1, specifically, a frequency fi1, a frequency on the second spectrum shape corresponding to the magnitude S1 (fi1) = fi1,2, The magnitude = S2 (fi1,2) is calculated as follows.

(Equation 1) Here, W1 = RBi + 1,1-RBi, 1 W2 = RBi + 1,2-RBi, 2 In calculating the magnitude = S2 (fi1,2), the frequency = fi among the set of the frequency and the magnitude actually existing on the second spectrum shape SS2.
(+), (-)
If it is expressed with a suffix of

(Equation 2) Becomes

From the above, assuming that the interpolation ratio = x, the frequencies fi1, x and the magnitudes Sx (fi1, x) on the interpolated spectrum shape corresponding to the combination of the frequency and the magnitude actually existing on the first spectrum shape SS1 are It is obtained by the following equation.

(Equation 3) Sx (fi1, x) = S1 (fi1) + {S2 (fi1,2) -S1
(Fi1)｝ · x Similarly, calculation is performed for all frequency and magnitude pairs on the first spectral shape SS1. Subsequently, the second spectrum shape SS
2. A set of frequencies and magnitudes on the interpolated spectrum shape corresponding to the set of frequencies and magnitudes actually existing on 2 is obtained.

First, a set of frequencies and magnitudes existing on the second spectrum shape SS2, specifically, a frequency fi2, a frequency on the first spectrum shape corresponding to the magnitude S2 (fi2) = fi1,1, The magnitude = S1 (fi1,1) is calculated as follows.

(Equation 4) Here, W1 = RBi + 1,1-RBi, 1 W2 = RBi + 1,2-RBi, 2 In calculating the magnitude = S1 (fi1,1), the frequency = fi among the set of the frequency and the magnitude actually existing on the first spectrum shape SS1.
(+), (-)
If it is expressed with a suffix of

(Equation 5) Becomes

From the above, assuming that the interpolation ratio = x, the frequency fi2, x and the magnitude Sx (fi2, x) on the interpolated spectrum shape corresponding to the combination of the frequency and the magnitude actually existing on the second spectrum shape SS2 are It is obtained by the following equation.

(Equation 6) Sx (fi2, x) = S2 (fi2) + ｛S2 (fi2) -S1 (f
i1,2)｝ · (x−1) Similarly, calculation is performed for all frequency and magnitude pairs on the second spectrum shape SS2.

As described above, the frequency fi1 and the magnitude S1 actually existing on the first spectrum shape SS1 are used.
The second spectral shape S corresponding to the set of 1 (fi1)
The actual frequencies fi1,2 and magnitude S existing on S2
2 (fi1,2) and the actual frequency fi2 and magnitude S2 (fi) on the second spectral shape SS2.
An interpolation spectrum shape is obtained by rearranging all the calculation results of the frequency fi2, x and the magnitude Sx (fi2, x) on the interpolation spectrum shape corresponding to the set 2) in order of frequency.

These operations are performed for all the regions Z1, Z2,... To calculate the interpolation spectrum shape of the previous frequency band. In the above example of spectral interpolation using anchor points, the spectral transition function ftrans1
Although (x) and ftrans2 (x) are linear functions, it is also possible to define as a non-linear function such as a quadratic function or an exponential function or to prepare a change corresponding to the function as a table.

Next, a method for interpolating three or more instruments by spectral interpolation using the above-described anchor points will be specifically described. First, 3
One or more existing instruments are divided into a set of two instruments, and interpolation is performed on the spectral shape of the two instruments included in each set to perform interpolation for each set. Get the spectral shape. Then, the interpolated spectrum shape obtained by the interpolation is further divided into two sets of spectrum shapes that are one set, and interpolation is performed on the two spectrum shapes included in each set, and each set of the spectrum shapes is executed. An interpolation spectrum shape is obtained, and the interpolation process is repeated until one interpolation spectrum shape is finally obtained. When the above-described interpolation processing is performed on n types of instruments, "n-1" times of interpolation will be performed. It should be noted that when performing the above grouping, the remaining spectral shapes that cannot be grouped may be subjected to interpolation processing in the next step without performing interpolation processing in this step.

The above-mentioned interpolation processing will be described more specifically with reference to FIG. 8. For example, when performing spectral interpolation for three instruments of guitar, piano, and flute, first, the first interpolation is performed. As processing, a spectrum shape 8a of a guitar and a spectrum shape 8b of a piano are paired, and spectrum interpolation is performed on the two spectrum shapes to obtain an interpolated spectrum shape 8d. Next, as the second interpolation processing, an interpolation spectrum shape 8d
And a spectrum shape 8c of the flute that has not been interpolated in the first interpolation processing, and performs spectrum interpolation on the two spectrum shapes, and finally forms one interpolated spectrum shape. An interpolated spectrum shape 8d is obtained. in this way,
When interpolation is performed on three instruments,
Interpolation will be performed twice.

When the analysis data extraction unit 11 has extracted the residual component, the interpolation processing unit 13 performs the interpolation processing described below, and then adds the residual component to the interpolation spectrum shape obtained earlier. I do. However, whether or not to perform the interpolation process of the residual component can be set in advance by the user, and the analysis data extraction unit 11 performs the interpolation process of the residual component according to a preset setting value.

Here, the interpolation process of the residual component will be described. First, the interpolation processing unit 13 analyzes the analysis data extraction unit 11 based on the interpolation information signal for each instrument.
Linear interpolation of the residual components extracted by the above is performed to generate an interpolated residual component for each instrument. next,
The interpolation processing unit 13 interpolates an interpolation residual component between a plurality of instruments in the same manner as when the above-described spectral shape is interpolated.

Further, the frequency synthesis processing unit 14 generates a new frequency signal by performing an inverse Fourier transform and an overlap process on the new frequency component output from the interpolation processing unit 13. Here, the overlap process in the present embodiment refers to a process of smoothly correcting both ends of a frequency signal divided in a frame unit so that no distortion occurs when frames are connected. Then, the envelope processing unit 15 generates an output tone signal by adding an envelope to the new frequency signal output by the frequency synthesis processing unit 14, and the output unit 16 outputs the output tone signal. .

[1.4] Effect of First Embodiment According to the above-described embodiment, the interpolation processing unit 13
By combining and interpolating the spectral shapes of a plurality of instruments, it is possible to end up with one interpolated spectral shape. Therefore, it is possible to perform the interpolation processing of the sound generation information generated from a plurality of instruments.

[1.5] Modification of First Embodiment [1.5.1] First Modification In the first embodiment described above, all the formants stored in the analysis data storage section are stored. The shape is created by a musical sound sample sampled from the instrument, but only the characteristic sound of the instrument is sampled, and only the formant shape that is not directly created by the sampled musical sound waveform May be created by interpolating the formant shapes created by the sampled musical sound waveforms.

Further, only the characteristic sound of the instrument is sampled, and for a formant shape which is not directly created depending on the sampled musical sound waveform, when the musical sound is generated, the interpolation processing section The formant shape created by the sampled musical sound waveform may be created by interpolating in real time.

[1.5.2] Second Modification In the above-described first embodiment, interpolation processing and the like are performed based on interpolation information set by a user. Select a tone,
Interpolation information may be generated based on the MIDI data that generated the tone color, and interpolation processing or the like may be performed based on the interpolation information. FIG. 5 is a schematic block diagram of a tone processing apparatus 1 'according to a third modification. The difference between FIG. 5 and the first embodiment of FIG. 1 is that, in the third modified example of FIG. 5, in addition to the components in the first embodiment of FIG. It is provided with a storage unit 42, a change input unit 43, and an interpolation information generation unit 44.

More specifically, the tone processing device 1 '
Is a MIDI which issues a MIDI message corresponding to a tone when a tone generated by preset MIDI data is selected by a user, in addition to the components provided in the tone processing apparatus 1.
The message selection unit 41 and the M of the MIDI data set when the tone generated by the MIDI data is generated.
When changing the MIDI interpolation information storage unit 42 storing the IDI interpolation information, the ratio of each tone generation state (rising state, steady state, open state) of the preset timbre, and the envelope information of the timbre, Based on the MIDI message issued from the MIDI message selecting unit 41, the MIDI input information stored in the MIDI interpolation information storage unit 42 is extracted to generate interpolation information. And an interpolation information generation unit 44 that changes the interpolation information in accordance with the change data input from the change input unit 43.

Here, various associations set between the MIDI message and the interpolation information will be described. First, in the Program Change message of the MIDI message, predetermined interpolation information is associated with a preset tone color, and this correspondence is stored in the MIDI interpolation information storage unit 42. As a result, when a preset tone is selected from the MIDI selecting unit 41, a Program Change message of the selected tone is issued, and the interpolation information generating unit 44
The interpolation information corresponding to the message is extracted from the MIDI interpolation information storage unit 42 to generate interpolation information.

Next, the MIDI message Control Chan
In the ge message, these parameter values are changed with respect to the parameter value that determines the ratio of each of the preset tone colors (rising state, steady state, open state) and the parameter value that determines the envelope information. Is set so as to receive the change data for performing the change. Thus, when the change input unit 43 receives change data for changing the ratio of each sounding state (rising state, steady state, open state) of the preset tone and the envelope information of the preset tone, the preset is performed. When the control change message of the timbre is issued and the interpolation information generating unit 44 receives the message, the interpolation information generating unit 44 can change the ratio of each sounding state of the timbre selected by the MIDI message selecting unit 41 and the envelope information in real time.

Also, the MIDI message Control Chan
In the ge message, for the parameter that stores each interpolation rate included in the interpolation information, the Control Change
Set the Number parameter in association with it. Accordingly, when change data for changing each interpolation rate is input from the change input unit 43, a Control Change message is issued, and each interpolation rate for the timbre selected by the MIDI message selection unit 41 is changed in real time. Can be.

Next, note on / off of the MIDI message
In the message, the amplitude of the generated tone waveform is associated with the Note Velocity parameter, and the pitch of the generated tone waveform is associated with the Note Key parameter and set. As a result, when the amplitude or pitch value of the generated tone waveform is input from the change input unit 43, the No.
A te On / Off message is issued, and the amplitude or pitch of the output tone waveform can be changed in real time.

Also, note on / off of MIDI message
In the message, the amplitude to be interpolated is associated with the Note Velocity parameter, the pitch to be interpolated is set with the Note Key parameter, and the amplitude associated with the Note Velocity parameter is set. The formant shape is extracted by searching the amplitude division stored in the analysis data storage unit 12 based on the
The formant shape may be extracted by searching the pitch division stored in the analysis data storage unit 12 based on the pitch associated with the ey parameter.
This makes it possible to generate a more realistic musical sound for the musical sound of the instrument.

Next, in the Pitch Bend message of the MIDI message, the Pitch Bend parameter
The pitches of the generated musical sound waveforms are set in association with each other. Thus, when the pitch value of the generated tone waveform is input from the change input unit 43, a Pitch Bend message is issued, and the pitch of the output tone signal can be changed in real time.

[2] Second Embodiment Next, a second embodiment of the present invention will be described. Book second
In the embodiment, the musical sound processing device described in the first embodiment is applied to a karaoke device, and is configured as a karaoke device capable of performing a morphing process on a newly generated musical sound for an input vocal sound. This is an example of the case. The second embodiment is different from the first embodiment in that the tone processing apparatus according to the first embodiment interpolates tone information input from the outside based on interpolation information set by a user to generate a tone. However, in the second embodiment, a new tone signal is generated by performing morphing processing on voice information input by a singer and tone information generated by an interpolation processing unit. It is.

[2.1] Schematic Configuration of Music Processing Device FIG. 6 is a block diagram showing a schematic configuration of a music processing device according to the second embodiment. 6, the same components as those in the first embodiment of FIG. 1 are denoted by the same reference numerals, and the detailed description thereof will be omitted. The musical sound processing device 10 includes an analysis data storage unit 12, an analysis data extraction unit 11, an interpolation processing unit 13, and a singing signal input unit 31 that receives a singer's voice and outputs a singing signal.
An SMS analysis unit 32 that performs an SMS analysis of the singing signal and outputs a spectrum shape (corresponding to a voice sine wave component), a residual component (corresponding to a voice residual component), a pitch, an amplitude, and the like, and an interpolation process A morphing process for performing a morphing process on the basis of morphing information including synthesis ratio data input from the outside with the interpolated spectrum shape and the like output by the unit 13 and the spectrum shape and the like of the singing signal output by the SMS analysis unit 32 The processing unit 33 performs an inverse fast Fourier transform (IF) on the new frequency signal generated by the morphing process.
It is configured to include a frequency synthesis processing unit 14 that performs FT) and overlap processing, and an output unit 16.

[2.2] Configuration of Each Unit of the Musical Sound Processing Device [2.2.1] SMS Analysis Unit The SMS analysis unit 32 multiplies the singing signal input by the singing signal input unit 31 by a window function. Then, a sine wave component and a residual component are extracted from a frequency spectrum obtained by performing fast Fourier transform (FFT) on the extracted frame. Then, the SMS analysis unit 32 outputs the sine wave component and the residual component to the morphing processing unit 33, and also uses the data representing the name and length of the sound in the singing signal as the pronunciation information as the analysis data extraction unit 11 Output to

[2.2.2] Morphing Processing Unit The morphing processing unit 33 includes an interpolation spectrum shape output by the interpolation processing unit 13, a singing signal spectrum shape output by the SMS analysis unit 32, and an external input. A morphing process is performed based on a morphing degree for morphing a spectrum shape included in the obtained morphing information, and a new frequency component having a desired spectrum shape, pitch, and amplitude according to the morphing information is generated. Here, the morphing information includes, for example, the morphing degree at the time of morphing the spectral shape or the residual component of the sound to be subjected to the morphing processing, and each value of the pitch or the amplitude as the morphing processing target. Which value of the sound to match is included.

For example, when performing morphing processing on two sounds, the morphing degree at which only one sound is output is set to “0”, and the morphing degree at which only the other sound is output is set to “0”. To “1”
When the morphing degree is changed from “0” to “1”, the two
It is a value that adjusts the degree of morphing of which sound and how much it resembles. Specifically, for example, when performing a morphing process between a guitar tone and a vocal sound, the morphing degree at which only the guitar tone is output is set to “0”, and the morphing degree at which only the vocal sound is output. Is set to “1”, the morphing degree is set to “0.3”, the timbre of the guitar and the vocal sound are morphed at a ratio of “7: 3”,
A tone that is more similar to the tone of a guitar than a vocal voice is output.

[2.3] Operation of Second Embodiment The operation of the second embodiment is the same as that of the first embodiment except for the operation of the main part. Therefore, only the operation of the main part will be described. First, a signal input process is performed by the singing signal input unit 31, and a voice signal sung by the singer is input. Next, S
The MS analysis unit 32 performs an SMS analysis on the audio signal input via the singing signal input unit 31, and extracts the spectrum shape, pitch, amplitude, and the like of the audio signal.
The analysis data extraction unit 11 searches the analysis data storage unit 12 based on interpolation information on three or more instruments set by the user and the pronunciation information output by the SMS analysis unit 32, and analyzes the analysis data. The extracted analysis data is output to the interpolation processing unit 13 while being extracted.

Then, the morphing processing section 33 compares the interpolation spectrum shape and the interpolation residual component output by the interpolation processing section 13 with the spectrum shape and the residual component of the singing signal extracted by the SMS analysis section 32. The morphing process is performed in between. here,
When performing the morphing process, the morphing processing unit 33 performs the morphing process according to the combination ratio data for performing the morphing included in the morphing information input from the outside. Then, the frequency synthesis processing unit 1
4 is an inverse fast Fourier transform (IFFT) for the new frequency component output by the morphing processing unit 33
And an overlap process.

[2.4] Effect of Second Embodiment According to the above-described embodiment, the morphing processing unit 33 generates a new vocal sound signal and a new interpolation information signal generated for three or more instruments. Since the morphing of the vocal sound can be morphed with a new musical tone like never before, a more fresh musical tone signal can be generated.

[2.5] Modification of Second Embodiment In the above-described second embodiment, the morphing processing section 33 outputs the morphed tone signal, but as shown in FIG. , Without the morphing processing section 33, the interpolation spectrum shape and the like output by the interpolation processing section 13 and the spectrum shape and the like of the singing signal extracted by the SMS analysis section 32 to the frequency synthesis processing section 14. The signals may be separately input to perform frequency synthesis processing, and the generated tone signal and audio signal may be mixed at the output unit 16 and then output. In this case, the output unit 16 may include a mixer for mixing the tone signal and the audio signal. A morphing processing unit 33 is provided similarly to the second embodiment, and a tone signal is generated by the frequency synthesis processing unit 14 using the interpolation spectrum / shape output by the interpolation processing unit 13, and the tone signal and the morphing process are performed. After the morphing process by the unit 33, the tone signal generated by the frequency synthesis processing unit 14 is output to the output unit 1.
6 and may be output after mixing.

The interpolation processing unit 13 changes the pitch of the interpolation spectrum shape to a pitch calculated by a predetermined ratio with respect to the pitch of the pronunciation information input from the SMS analysis unit 32. Is also good.
Further, the interpolated spectrum shapes before and after changing the pitch may be separately processed and output. According to this modification, two harmony or three or more harmony by the interpolated tone signal and the voice signal input by the singer can be realized.

[3] Modified Example [3.1] First Modified Example In each of the above-described embodiments, the multidimensional interpolation information is not limited to being set by the user, but is held as a preset tone color. Alternatively, the user may appropriately change the voice by voicing. Further, the change may be made in real time by the controller.

[3.2] Second Modification In each of the embodiments described above, the interpolation information is limited to three or more instruments. It can be applied to information.

[0099]

As described above, according to the present invention, three or more spectral shapes can be interpolated, and more new musical tones can be generated. Further, since the interpolation of the spectrum shape is performed according to the tone generation state of the timbre, a more natural tone can be generated. Further, since a newly generated musical tone and voice can be synthesized, a fresher musical tone can be generated.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a schematic configuration of a musical sound processing device according to a first embodiment.

FIG. 2 is a diagram showing a file structure of an analysis data storage unit in the musical sound processing device.

FIG. 3 is a diagram illustrating an SMS analysis performed in the musical sound processing device.

FIG. 4 is a diagram illustrating a technique of spectrum interpolation performed in the musical sound processing device.

FIG. 5 is a block diagram illustrating a schematic configuration of a tone processing device according to a modification of the first embodiment.

FIG. 6 is a block diagram illustrating a schematic configuration of a tone processing device according to a second embodiment.

FIG. 7 is a block diagram illustrating a schematic configuration of a tone processing device according to a modification of the second embodiment.

FIG. 8 is a diagram illustrating a specific example of spectrum interpolation performed in the musical sound processing device.

FIG. 9 is a diagram illustrating a specific example of an analysis data storage unit in the musical sound processing device.

[Explanation of symbols]

1, 1 ', 10, 10' ... tone processing device, 11 ... analysis data extraction unit, 12 ... analysis data storage unit, 13 ...
Interpolation processing unit, 14 Frequency synthesis processing unit, 15 Envelope processing unit, 16 Output unit, 31 Singing signal input unit, 32 SMS analysis unit, 33 Morphing processing unit, 41 MIDI tone selection section, 42 MIDI
Interpolation information storage unit, 43 ... Change input unit, 44 ... Interpolation information generation unit

 ──────────────────────────────────────────────────続 き Continued on the front page (72) Inventor Xavier Serra Spain Barcelona 08002 Merce 12 (72) Inventor Jordi Bonada Spain Barcelona 08002 Merce 12 F-term (reference) 5D378 AA01 AD62 BB23 GG34 JC05 KK01 KK07 KK07

Claims (37)

[Claims] 1. An analysis data storage means for storing analysis data obtained by analyzing sampling data of a sound generated in an instrument, and sound information and three or more kinds of instruments inputted from outside. Analysis data extraction for extracting the analysis data corresponding to three or more instruments from the analysis data storage means based on interpolation information input from the outside to generate one musical tone signal corresponding to the tone color of Means, interpolation processing means for performing an interpolation process based on a spectrum shape corresponding to the analysis data to generate one interpolation spectrum shape, and a tone for generating and outputting a tone signal based on the interpolation spectrum shape. A tone processing device comprising: signal generation means. 2. The tone processing apparatus according to claim 1, wherein said analysis data is a sine wave component and a residual component obtained as a result of frequency analysis of said sampling data on a frame basis. apparatus. 3. The musical tone processing device according to claim 2, wherein the analysis data storage means stores the sine wave component and the residual component corresponding to a plurality of tone generation states for each of the instruments. A tone processing device characterized by the following. 4. The musical sound processing apparatus according to claim 1, wherein said interpolation processing means performs (n-1) times of said interpolation processing when there are n types of said spectrum shapes. Processing equipment. 5. The musical sound processing apparatus according to claim 4, wherein the interpolation processing means repeatedly performs a grouping process of dividing the spectrum shape into two types of groups and the interpolation process performed for each group. A tone processing apparatus for generating one interpolation spectrum shape. 6. The musical sound processing device according to claim 5, wherein the interpolation processing means, when there are an odd number of the spectrum shapes, removes the spectrum shapes that could not be divided in the grouping process. A tone processing device for performing processing in the next grouping process. 7. The musical sound processing device according to claim 1, wherein the interpolation processing means performs a spectrum shape transition function between the two spectrum shapes when performing the interpolation process using the two spectrum shapes. A tone processing device characterized by being used. 8. The tone processing apparatus according to claim 7, wherein the transition function is defined in advance as a linear function or a non-linear function. 9. The musical sound processing apparatus according to claim 7, wherein said interpolation processing means divides said two spectral shapes into a plurality of regions on a frequency axis, respectively, and calculates said two spectral shapes belonging to each region. A tone processing apparatus for performing the interpolation process using a linear function as the transition function over the plurality of regions for a set of an actual frequency and a magnitude of the actual sound. 10. The tone processing apparatus according to claim 9, wherein said interpolation processing means includes a first frequency which is a frequency of one of the spectrum shapes belonging to each of the regions and a second spectrum corresponding to the first frequency. Frequency interpolation means for calculating an interpolation frequency by interpolating a second frequency which is a frequency of the shape using the linear function; a first magnitude which is a magnitude of one of the spectral shapes belonging to each of the regions; A tone interpolation device for interpolating a second magnitude, which is the magnitude of the other spectral shape corresponding to the magnitude, using the linear function. 11. The musical sound processing apparatus according to claim 1, wherein the interpolation information is overall interpolation information for performing interpolation processing on a synthesis ratio between the instruments, and state interpolation for performing interpolation processing on a sound generation state. A tone processing device comprising: pitch interpolation information for performing interpolation processing on information and pitch; and amplitude interpolation information for performing interpolation processing on amplitude. 12. The tone processing apparatus according to claim 11, further comprising interpolation information setting means for setting said interpolation information. 13. The tone processing apparatus according to claim 11, further comprising preset means for holding said interpolation information set in advance. 14. The musical sound processing apparatus according to claim 2, wherein the interpolation processing means interpolates the sine wave component and the residual component stored by the analysis data storage means to store the analysis data. A musical sound processing apparatus comprising: analysis data calculation means for calculating a sine wave component and a residual component which are not stored by the means. 15. The tone processing apparatus according to claim 1, wherein the tone signal generating means performs an inverse fast Fourier transform process, an overlap process, and an envelope adding process on the interpolated spectrum shape. A tone processing apparatus for generating a tone waveform in a time domain. 16. The tone processing apparatus according to claim 1, wherein: MIDI tone color selection means for selecting a tone color of MIDI data; interpolation information storage means for storing MIDI interpolation information of the tone color; Interpolation information generating means for generating the interpolation information based on the interpolation information, wherein the analysis data extraction means outputs the analysis data from the analysis data storage means based on the interpolation information and pronunciation information inputted from outside. A tone processing device for extracting analysis data. 17. The tone processing apparatus according to claim 16, wherein: a change input means for inputting change interpolation data; and interpolation information replacement for replacing the interpolation information corresponding to the change interpolation data input by the change input means. Means, and a music processing apparatus. 18. The musical sound processing apparatus according to claim 1, wherein: a voice input means for inputting voice; and a voice sine wave component and voice by frequency-analyzing the input voice input from the voice input means on a frame basis. Frequency analysis means for extracting a residual component, and a predetermined morphing degree between one interpolated spectrum shape generated by the interpolation processing means and the audio sine wave component and the audio residual component extracted by the frequency analysis means. Morphing means for performing a morphing process by using the analytic data extracting means, wherein the analysis data extracting means comprises: A musical sound processing device for extracting the analysis data corresponding to an instrument. 19. The tone processing apparatus according to claim 18, further comprising: morphing setting means for setting said predetermined morphing degree, wherein said morphing means uses said morphing degree set by said morphing setting means to form said interpolated spectrum shape. A morphing process between the voice sine wave component and the voice residual component. 20. The tone processing apparatus according to claim 19, wherein said tone signal generating means performs an inverse fast Fourier transform process and an overlap process on said interpolated spectrum shape to thereby generate a tone waveform in a time domain. A tone processing device for generating a tone signal. 21. The tone processing apparatus according to claim 1, wherein a voice input means for inputting voice, and a frequency analysis of the input voice input from the voice input means on a frame basis, a voice sine wave component and voice. Frequency analysis means for extracting a residual component, wherein the analysis data extraction means is configured to receive three or more data from the analysis data storage means based on pronunciation information included in the voice sine wave component and interpolation information input from the outside. Extracting the analysis data corresponding to more than one kind of instrument, the tone signal generating means generates an interpolated tone signal based on the interpolated spectrum shape, the audio sine wave component and the audio residual component, A tone processing device for generating a voice tone signal based on a tone signal, and mixing and outputting the interpolated tone signal and the voice tone signal. . 22. The tone processing device according to claim 21, wherein the interpolation processing means calculates a pitch of the sine wave component by a pitch obtained from a predetermined ratio corresponding to a pitch of the input voice. A tone processing apparatus comprising: a pitch shifter that shifts the pitch; and a residual component adder that adds a residual component to the sine wave component whose pitch has been shifted by the pitch shifter. 23. The tone processing apparatus according to claim 21, wherein the tone signal generating means performs an inverse fast Fourier transform process on the interpolated spectrum shape or the audio sine wave component and the audio residual component. A musical sound processing device for generating a musical sound waveform in a time domain by performing a lap process. 24. An externally input pronunciation information and 3
It is obtained by analyzing sampling data of a sound generated in an instrument based on interpolation information input from outside to generate one musical tone signal corresponding to a tone color of more than one kind of instrument. An analysis data extraction step of extracting the analysis data corresponding to three or more instruments from the analysis data; and performing an interpolation process based on the spectrum shape corresponding to the analysis data to generate one interpolation spectrum shape A tone signal generating step of generating and outputting a tone signal based on the interpolated spectrum shape. 25. The tone processing method according to claim 24, wherein the analysis data is a sine wave component and a residual component obtained as a result of frequency analysis of the sampling data in frame units. Method. 26. The musical sound processing method according to claim 24, wherein the interpolation processing step performs (n-1) times of the interpolation processing when there are n types of the spectrum shapes. Processing method. 27. The musical sound processing method according to claim 26, wherein the interpolation processing step repeatedly performs a grouping process of dividing the spectrum shape into sets of two types and an interpolation process performed for each set. A tone processing method comprising generating one interpolation spectrum shape. 28. The musical sound processing method according to claim 27, wherein, in the interpolation processing step, when there are an odd number of the spectrum shapes, the spectrum shapes that could not be grouped in the grouping processing are removed. A tone processing method characterized by processing in a next grouping process. 29. The musical sound processing method according to claim 24, wherein the interpolation processing step includes, when performing the interpolation processing using two spectrum shapes, calculating a spectrum shape transition function between the two spectrum shapes. A tone processing method characterized by being used. 30. The musical sound processing method according to claim 29, wherein the interpolation processing step divides the two spectral shapes into a plurality of regions on a frequency axis, respectively, and performs the processing on the two spectral shapes belonging to each region. And performing the interpolation process using a linear function as the transition function over the plurality of regions for the set of the actual frequency and magnitude of the tone. 31. The musical sound processing method according to claim 30, wherein the interpolation processing step includes: a first frequency that is a frequency of one of the spectrum shapes belonging to each of the regions; and a second spectrum corresponding to the first frequency. A frequency interpolation process of calculating an interpolation frequency by interpolating a second frequency that is a frequency of the shape using the linear function, a first magnitude that is a magnitude of one of the spectral shapes belonging to each of the regions, and the first magnitude. A magnitude interpolation step of interpolating the second magnitude, which is the magnitude of the other spectral shape corresponding to the magnitude, using the linear function. 32. The musical sound processing method according to claim 24, wherein the interpolation information is overall interpolation information for performing interpolation processing on a synthesis ratio between the instruments, and state interpolation for performing interpolation processing on a sounding state. A tone processing method comprising: pitch interpolation information for performing interpolation processing on information and pitch; and amplitude interpolation information for performing interpolation processing on amplitude. 33. The tone processing method according to claim 24, wherein: a MIDI timbre selection step of selecting a timbre of MIDI data; and interpolation information generation for generating the interpolation information based on the timbre and MIDI interpolation information of the timbre. And a step of extracting the analysis data based on the interpolation information and pronunciation information input from the outside. 34. The tone processing method according to claim 24, wherein: a voice inputting step of inputting a voice; and a frequency analysis of the input voice input in the voice inputting step on a frame-by-frame basis. A frequency analysis step of extracting components, and morphing one interpolated spectrum shape generated by the interpolation processing step and the audio sine wave component and the audio residual component extracted by the frequency analysis step by a predetermined morphing degree. A morphing process for processing, wherein the analysis data extraction process is based on pronunciation information included in the audio sine wave component and interpolation information input from the outside, and the analysis corresponding to three or more types of instruments. A tone processing method characterized by extracting data. 35. The musical sound processing method according to claim 34, further comprising a morphing setting step of setting the predetermined morphing degree, wherein the morphing step is performed by using the morphing degree set by the morphing setting step. And morphing the voice sine wave component and the voice residual component. 36. The tone processing method according to claim 24, wherein: a voice inputting step of inputting a voice; and a frequency analysis of the input voice input in the voice inputting step on a frame-by-frame basis. A frequency analysis step of extracting a component. The analysis data extraction step corresponds to three or more instruments based on pronunciation information included in the audio sine wave component and interpolation information input from the outside. Extracting the analysis data to generate the tone signal, and generating the tone signal based on the interpolated spectrum shape and generating the tone signal based on the sine wave component and the residual voice component. And mixing and outputting the interpolated tone signal and the voice tone signal. 37. The musical sound processing method according to claim 36, wherein said interpolation processing step comprises a step of determining a pitch of said sine wave component by a pitch obtained from a predetermined ratio corresponding to a pitch of said input voice. And a residual component adding step of adding a residual component to the sine wave component whose pitch has been shifted by the pitch shifting process.