JP2000276194A - Waveform compressing method and waveform generating method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To perform waveform compression which is superior in data compression efficiency and the reproducibility of an original waveform. SOLUTION: Input waveform data are processed by FFT or the like to separate harmonic components and inharmonic components are separated from the waveform data according to the residue. A harmonic vector and an inharmonic vector are prepared beforehand in memories M1 and M2 or the like and the harmonic vector as a representative vector of the separated harmonic vector is selected and specified to perform vector quantization; and the inharmonic vector as the representative vector of the separated inharmonic components is selected and specified to perform its vector quantization. Thus, the harmonic and inharmonic components are quantized into vectors separately. The harmonic vector and inharmonic vector indicated with the vector information of a waveform to be reproduced are used to generate a harmonic component waveform and an inharmonic component waveform, which are put together to reproduce and generate the waveform.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a waveform compression method for compressing waveforms of musical sounds, voices, and other various sounds using a vector quantization technique, and more particularly to a method of compressing a large number of waveform data corresponding to various playing styles. The present invention relates to a waveform compression method capable of effectively performing data compression in a case where the data is stored, and further relates to a waveform generation method for generating an arbitrary waveform using such a waveform compression method. The present invention is applicable to not only electronic musical instruments but also apparatuses, apparatuses, or methods in all fields having a function of generating musical sounds, voices, or other arbitrary sounds, such as automatic performance devices, computers, electronic game devices, and other multimedia devices. It can be widely applied. In this specification, the musical sound waveform is not limited to a musical sound waveform, but may be a sound or any other sound waveform.

[0002]

2. Description of the Related Art Recently, it has become an important issue in electronic musical sound synthesis techniques how to synthesize high-quality musical sounds in consideration of differences in playing style. That is, in the case of a natural musical instrument, even with musical tones having the same musical instrument tone and the same pitch, different musical tone characteristics (particularly, timbres) are applied to different performances such as vibrato and slur. It is known that a musical tone having a waveform is generated. When an electronic musical instrument or the like intends to generate a high-quality musical tone that reflects a difference in playing style of a natural musical instrument by using an electronic musical sound synthesis technique, each musical tone (or range) of each musical instrument tone color is used. For each of a plurality of playing styles, waveform data indicating a tone characteristic (particularly, tone waveform) according to each playing style is stored in a memory in advance, and waveform data corresponding to the playing style to be played is read from the memory. Thus, it is conceivable to generate a musical tone waveform showing a specific musical tone characteristic (particularly, a tone color waveform) corresponding to the playing style.

As described above, when different waveform data is stored for each pitch (or range) of each musical instrument timbre in accordance with each of a plurality of playing styles, it is necessary to use a memory that is not normally stored. Since the storage capacity will increase considerably, the important issue is how to perform effective data compression and save the required memory storage capacity. Therefore, among the conventional waveform data compression technologies,
Some of the notable aspects related to the present invention include the following.

First, in Japanese Patent Application Laid-Open No. 61-104400, an input waveform is separated into a periodic component and an aperiodic component by a filtering process, and each component is coded in a different data compression coding format. A sound recording method to be stored is shown. However, there is an idea that specific waveform data is separated into a periodic component and an aperiodic component and then compressed and stored, but there is no idea of performing vector quantization on all of them. In particular, for a non-periodic component, it is necessary to record a residual waveform for the waveform of the periodic component as it is in time series.

Next, Japanese Patent Laid-Open Publication No. Hei 5-88911 discloses a speech coding method using vector quantization. Here, the speech waveform to be compressed is first processed by a filter having the inverse characteristic of the spectrum extracted from the waveform, and then the excitation vector extracted from a predetermined period in the past of the filtered waveform is applied to the filtered waveform. , Vector quantization of the periodic component is performed, and vector quantization of the non-periodic component is performed using the noise vector. In this case, a noise vector prepared as a fixed vector is used. A non-periodic component (noise component) is extracted from the speech waveform itself to be compressed, and a vector of the non-periodic component is extracted based on the extracted aperiodic component. It does not perform quantization.

[0006]

However, in the former waveform data compression technique described above, since the data compression ratio is low, it is necessary to support a plurality of playing styles for each pitch (or range) of each musical instrument timbre. It has been difficult to realize sufficient data compression suitable for the purpose of storing different waveform data. That is, in the former prior art, since there was no idea of vector quantization of the aperiodic component, the data storage of the aperiodic component had to be recorded in the time series of the residual waveform with respect to the waveform of the periodic component. , Did not achieve sufficient data compression.

On the other hand, in the latter waveform data compression technique described above, a noise vector prepared as a fixed vector is used, and a non-periodic component (noise component) is obtained from a speech waveform itself to be compressed. ) Is extracted and the vector quantization of the aperiodic component is not performed based on this, so that the vector quantization accuracy of the noise vector is poor, the aperiodic component cannot be efficiently compressed, and the non-periodic component cannot be compressed. The reproduction accuracy of the periodic component was also poor. Further, in this waveform data compression technique, since the filtering process by the filter having the inverse characteristic is always performed, the shape of the waveform to be reproduced is the same as the shape of the original waveform by the intervention of the strong inverse characteristic filtering process. There is a disadvantage that the shape becomes different and the waveform reproducibility is poor. Therefore,
Although it can be used to reproduce human voice waveforms that do not require precise waveform reproducibility, it is not suitable for reproducing musical instrument sounds that require precise waveform reproducibility. Further, in this waveform data compression technique, compression using a cycle vector and a noise vector is performed. However, for one waveform on which the inverse characteristic filter processing is performed, a combination of the cycle vector and the noise vector is used. , The method of determining the combination is difficult, and precise compression cannot be performed. Furthermore, since the excitation vector extracted from the past waveform is often used as the above-mentioned periodic vector, the follow-up property with respect to the waveform that is rich in changes is low.

The present invention has been made in view of the above points, and an object of the present invention is to provide a waveform compression method which is excellent in data compression efficiency and excellent in reproducibility of an original waveform. Further, another object of the present invention is to provide a waveform generation method for generating an arbitrary waveform using such a waveform compression method.

[0009]

According to the present invention, there is provided a waveform compressing method comprising the steps of: separating input waveform data into a harmonic component and a non-harmonic component; supplying a harmonic vector; Generating the first compressed data, providing the non-harmonic vector, and vector-quantizing the separated non-harmonic component with the non-harmonic vector,
And generating compressed data.

According to the present invention, the input waveform data is separated into a harmonic component and a non-harmonic component, and a process of separately vector-quantizing the separated harmonic component and the non-harmonic component is performed. Thereby, there is an excellent effect that the compression efficiency can be improved by vector-quantizing both the harmonic component and the non-harmonic component.
In addition, since it is not necessary to perform vector quantization in consideration of the combination of the periodic vector and the noise vector as in the related art, the vector selection of the harmonic component and the non-harmonic component is performed independently of each other. This has an excellent effect that processing can be easily performed. In particular, in vector quantization of non-periodic waveform components,
Since the vector quantization process is performed based on the non-harmonic components separated from the original waveform, the reproduction accuracy of the non-periodic waveform components is improved. As described above, according to the present invention, the vector can be easily selected, the compression efficiency in the vector quantization is excellent, and the reproducibility of the original waveform is very excellent.

As an example, the harmonic component is obtained by analyzing the frequency of input waveform data by fast Fourier transform and extracting its fundamental pitch frequency component and its harmonic components.
Obtainable. That is, a waveform of a harmonic component can be generated by performing an inverse Fourier transform using the extracted basic pitch frequency component and harmonic component. By subtracting the generated harmonic component waveform from the input waveform data, a waveform of the non-harmonic component can be obtained as the residual waveform.

Since the harmonic component is basically a repetitive waveform, for example, a waveform of one cycle or a plurality of cycles is used as a representative vector, and the representative vector is looped (or two successive representative vectors are cross-fade looped). By doing so, it is possible to regenerate the harmonic component. Therefore, such a representative vector can be used as the harmonic vector. The vector quantization of the harmonic component separated from the input waveform data selects, as an example, a harmonic vector that is suitable for the harmonic component, that is, a representative vector,
This can be performed by adding information describing how to use the harmonic vector, that is, the representative vector (for example, a section in which the harmonic vector is used, a method of looping the amplitude vector, or an amplitude envelope assigned to the section). .

On the other hand, for non-harmonic components, vector quantization should be performed in an optimal form corresponding to the content of each non-harmonic component in consideration of a balance between promotion of data compression and improvement of waveform reproducibility. preferable. The out-of-harmonic component basically corresponds to a random noise waveform having no repeatability.For example, in consideration of the presence or absence of importance such as its content and where it is located in the sounding period, A non-harmonic component waveform of a required time length may be reproduced by looping a specific representative vector (this is called loop reproduction), or a representative vector having a required time length is not looped. In this case, the required non-harmonic component waveform may be reproduced by using it only once (this is called one-shot reproduction). Alternatively, a required non-harmonic component waveform may be reproduced by switching a plurality of representative vectors in a predetermined order and combining them (this is called sequence reproduction). Alternatively, the switching order may be set at random to emphasize the noise, that is, the randomness of the non-harmonic component (this is referred to as random sequence reproduction). In consideration of such various optimal vector quantization forms, it is preferable to appropriately use a representative vector suitable for various optimal vector quantization forms as the nonharmonic vector. Therefore, when performing vector quantization of out-of-harmonic components separated from input waveform data,
As an example, a non-harmonic vector, that is, a representative vector suitable for the non-harmonic component is selected, and how the non-harmonic vector, that is, the representative vector is used (for example, the distinction between the loop reproduction or the one-shot reproduction, the period, and the This can be done by adding information describing the type of envelope to be provided.

Each representative vector of the harmonic component, ie, the harmonic vector, and each representative vector of the non-harmonic component, ie, the non-harmonic vector, can be shared between various different waveforms. In other words, when two or more different original waveforms to be compressed have an appropriate commonality in a part of the original waveforms, the same representative vector is appropriately applied between the two waveforms at the time of their vector quantization. Can be shared.

The waveform generating method according to the present invention includes a step of receiving first compressed data containing predetermined harmonic vector designating information and second compressed data containing predetermined non-harmonic vector designating information. Supplying, combining the waveform data of the harmonic component based on the first compressed data and the harmonic vector, providing a non-harmonic vector, and providing the second compressed data and the non-harmonic vector. Synthesizing the waveform data of the out-of-harmonic component based on the That is, based on the compressed data of the harmonic component (first compressed data) and the compressed data of the non-harmonic component (second compressed data) respectively compressed as described above, the waveform of the harmonic component and the waveform of the non-harmonic component are respectively obtained. And can be played independently. The original waveform can be reproduced by matching the waveform of the reproduced harmonic component with the waveform of the non-harmonic component.

The present invention can be constructed and implemented not only as a method invention, but also as an apparatus invention.
It can also be implemented. Further, the present invention can be implemented in the form of a computer program, or can be implemented in the form of a recording medium storing such a computer program. Further, the present invention can be embodied in the form of a recording medium storing waveform data having a novel compressed data structure.

[0017]

Embodiments of the present invention will be described below in detail with reference to the accompanying drawings. First, an example of a hardware configuration used for implementing a waveform compression method and a waveform generation method according to an embodiment of the present invention will be schematically described with reference to FIG. The hardware configuration example shown here is configured using a general-purpose computer such as a personal computer. In this case, the waveform compression and waveform generation processing is performed by the computer according to the waveform compression method and the waveform generation method according to the present invention. This is performed by executing a predetermined program (software) that implements the method. Of course, the waveform compression method and the waveform generation method are not limited to the form of computer software.
(Digital signal processor). The present invention can be implemented in the form of a microprogram processed by a (digital signal processor). The present invention is not limited to this type of program, and includes a discrete circuit, an integrated circuit, or a large-scale integrated circuit. It may be implemented in the form of a dedicated hardware device. In addition, the device configuration is not limited to a general-purpose computer such as a personal computer, and may take any product application form such as an electronic musical instrument, a karaoke device, an electronic game device, or other multimedia devices.

In the example of the hardware configuration shown in FIG. 1, a CPU (central processing unit) 10 serving as a main control unit of a computer is provided with a ROM (read only memory) 1.
1, RAM (random access memory) 12, hard disk device 13, removable disk device (for example, C
D-ROM drive or MO drive) 14,
Input operation device 1 such as display 15, keyboard and mouse
6, a waveform interface 17, a timer 18, an interface 19 for MIDI and a communication network, etc.
It is connected via a U bus 20. The waveform interface 17 converts an analog waveform signal (audio signal) input from outside via a microphone or the like into the CPU 10.
In accordance with the instruction from the digital camera, the digital signal is sampled according to a predetermined sampling frequency and converted into a digital signal.
The function of sending the digital waveform data to the PU bus 20 and the digital waveform data generated based on the compressed data by the waveform generation processing executed by the computer via the CPU bus 20 and converting the digital waveform data into an analog waveform signal in accordance with a predetermined sampling frequency, and a speaker system or the like. And the like.

The digital waveform data input via the waveform interface 17 is temporarily written in a predetermined storage area of the RAM 12 or the hard disk drive 13 or the like. A compression process is performed. The compressed data is stored in an appropriate compressed waveform database. In this case, not only the analog waveform signal but also digital waveform data may be input via an appropriate interface 19 or the like, and the input waveform data may be subjected to required data compression. The function of the waveform database may be performed by any type of data storage device. That is, any of the RAM 12, the hard disk device 13, and the removable disk device 14 may function as the compressed waveform database. In general, an appropriate storage area in the hard disk drive 13 which is a large-capacity storage device or a removable disk drive C
What is necessary is just to make a removable recording medium such as a D-ROM or MO function as the compressed waveform database. Or,
A waveform database provided in an external host or server computer is accessed via an interface 19 and a communication line to write compressed data therein, and to store compressed data necessary for reproduction in a hard disk device 13 or a RAM 12. Etc. may be downloaded. At the time of waveform generation, a predetermined waveform generation process is performed using the compressed data stored in the compressed waveform database to generate digital waveform data. The generated digital waveform data may be output as an analog signal via the waveform interface 17 as described above, or may be transferred and output as digital data to the outside via an appropriate interface 19 or the like.
When an external memory such as the hard disk device 13, the removable disk device 14, or an external host or server computer is used as the compressed waveform database, an internal memory is provided to enable quick access to designated waveform data. Transfer buffer (RAM1
2 or the like) may be transferred in whole or in part to the compressed waveform database.

The display unit 15 displays various graphic screens in response to an instruction from the CPU 10 in the course of the waveform compression processing and the waveform generation processing. For example,
Graphical display of the shape of the recorded waveform, graphical display of the waveform editing status during waveform compression processing, and display of control screens for setting and selecting various data such as tone setting, tone editing, system setting, etc. It is used in a variety of ways. Further, necessary control information, character information, and the like are input in the course of the waveform compression processing and the waveform generation processing using the alphanumeric keyboard and the mouse of the input operation device 16. Also, a MIDI keyboard module, a sequencer (automatic performance device) or another computer is connected via the interface 19 to input performance instruction information for reproducing a musical tone and to exchange various data.

A software program for executing the waveform compression processing and / or the waveform generation processing according to the present invention under the control of the CPU 10 may be stored in the ROM 11, the RAM 12, the hard disk device 13, or the like. . Further, this program may be recorded on a removable recording medium such as a CD-ROM or an MO that can be attached to and detached from the removable disk device 14, or from an external host or server computer via a communication line and an interface 19. The program may be received and downloaded to the hard disk device 13 or the RAM 12 or the like. In practicing the present invention, the computer system as shown in FIG. 1 does not necessarily need to have both functions of the waveform compression processing and the waveform generation processing, and can perform only one of them. Such a configuration may be used.

Next, embodiments of various processes executed under the control of the CPU 10 will be described. FIG.
The procedure of the waveform compression process executed under the control of 0 is schematically illustrated by an equivalent block diagram. First, in the “waveform recording with playing style” (process P1), experienced performers actually play musical sounds having the same pitch and the same intensity in various playing styles on various natural musical instruments. Is sampled via the waveform interface 17, and thus a large number of waveform data with renditions are
It is recorded in a predetermined storage area in the AM 12, the hard disk device 13, or the like. In this case, the recorded performance sound is not limited to a single sound, and may be a series of phrases, chords, or the like. Also, a plurality of different playing styles may be included in a single performance sound composed of a single sound or a series of phrases. May be included.

Even if the general playing style names such as "vibrato" and "slur" are the same, the waveform characteristics of the performance sounds are different for each musical instrument type, and the general playing style names are the same. Even if the type of musical instrument is the same, the waveform characteristics of each performance sound differ depending on the degree of the playing technique (for example, the difference in vibrato depth). Therefore, the performance sounds according to various performance styles recorded in the “waveform recording with performance style” (process P1) take into account differences in performance styles from these various viewpoints. Therefore, the “performance sound with playing style” is mainly
It can be distinguished and described using a combination of three elements: a "musical instrument name", a "performance style name", and a "parameter" indicating the degree of the performance style. In this case, the “parameter” indicating the degree of the playing style includes a plurality of parameters.

An example is as follows. Example 1: In "attack" (performance style name) of "violin" (instrument name), its "rise speed, strength, pitch" and the like are parameters. Example 2: For "slur" (playing style name) of "violin" (musical instrument name), the "slur width, slur speed, intensity,
"Pitch" or the like is a parameter. Example 3: For "vibrato" (playing style name) of "violin" (instrument name), the "vibrato depth, vibrato speed, intensity, pitch" and the like are parameters. Example 4: "Picking" (performance style name) of "guitar" (instrument name) has parameters such as "strength and pitch". Example 5: "Hammering on" of "guitar" (instrument name)
In (playing style name), the “strength, pitch” and the like are parameters. Example 6: In "bend" (performance style name) of "guitar" (instrument name), "bend width, bend speed, intensity, pitch" and the like are parameters. Example 7: “Tremolo arm operation” of “Guitar” (instrument name)
In (playing style name), the “operation width, operation speed, intensity, pitch” and the like are parameters. The parameter may include a parameter indicating a time change mode, such as an intensity time change curve or a slur time change curve.

Next, in a "reproduction style analysis" (process P2), each waveform data prepared in the process P1 of the "waveform recording with rendition style" is analyzed, and one or a series of performance sound waveform data is analyzed. A range corresponding to a common performance style is extracted (this is referred to as a performance style section), and the waveform data is extracted for each of one or more performance styles included in the waveform data. The position to be sectioned is determined, and the “performance style name” and the “parameter” for describing the performance style used in each of the divided sections (performance style section) are determined. The rendition style designation data including the data and the “parameter” is generated. Also, musical instrument designation data indicating the “musical instrument name” for the playing style is generated. In this way, the definition of dividing the waveform data of one or a series of performance sounds into one or a plurality of performance style sections according to one or a plurality of performance styles used therein, and describing the type of the performance style for each performance style section For this purpose, three elements of "instrument name", "performance style name", and "parameter" are defined by the instrument designation data and the performance style designation data. These musical instrument designation data and rendition style designation data are used as commentary information or index information of the compressed data of the rendition style section compressed by the processes P3 to P6 for waveform data compression.

For example, the following three methods are conceivable as analysis methods in the "performance analysis" (process P2). Example 1: The user listens to the sound of the waveform data to be analyzed,
The waveform is displayed on the display 15 as necessary, and the position of the division to be divided as the rendition style section is manually designated by the user using the input operation device 16. In addition, the “reproduction style name” and “parameter” for each of the divided performance style sections specified by the user are determined by the user, and the input operation device 1
Specify manually using 6. This corresponds to a manual analysis. Example 2: The CPU 10 executes a predetermined rendition style analysis program to analyze changes in various tone characteristics, such as pitch and amplitude, of the waveform data to be analyzed, and based on the analysis result, determines the position of a break to be divided as a rendition style section. Automatically determined, and
The “performance style name” and “parameter” for each determined performance style section are automatically determined. This corresponds to a fully automatic analysis. Example 3: A combination of the techniques of Examples 1 and 2 above. That is, the performance analysis is performed by combining the manual analysis by the user and the automatic analysis by the CPU. For example, the division of the performance style section and the determination of the “performance name” are manually performed by the user, and the “parameter” in each performance style section is determined by automatic analysis by the CPU. Alternatively, the division position of each performance style section or the “performance style name” or “parameter” determined by the automatic analysis by the CPU is appropriately corrected by a user's manual operation.

Next, "reproduction style section division" (process P3)
In the above, the waveform data of one or a series of performance sounds prepared in the process P1 of the “waveform recording with performance style” is concretely determined according to the performance style section division position designation information determined in the process P2 of the “performance style analysis” for the waveform data. Performs a process of splitting the data. Subsequent processes of the processes P4 to P6 are performed on the waveform data of each divided performance style section.

Next, in the "harmonic / non-harmonic component separation" (process P4), the waveform data of one rendition style segment divided in the process P3 of "reproduction style segmentation" is fetched from a memory such as the RAM 12 and the like. A process for separating data into a harmonic component and a non-harmonic component is performed. As an example of this separation processing, first, waveform data to be separated is input, and frequency analysis (spectrum analysis) is performed on the waveform data by fast Fourier transform (FFT) to extract its fundamental pitch frequency component and harmonic components. Is subjected to inverse Fourier transform to generate a waveform of a harmonic component. By subtracting the generated harmonic component waveform from the input waveform data to be separated, a waveform of the non-harmonic component is obtained as the residual waveform. As an example, in the above-mentioned FFT processing, a window function having a length corresponding to eight periods of the waveform data to be analyzed is used, and the window function is shifted by a frame corresponding to one-eighth period. You should analyze the whole. When such an FFT analysis is employed, the fluctuation of the frequency of the harmonic component can be analyzed as the fluctuation, and the fluctuation can be included in the harmonic component and extracted. Of course, the size of the window function and the shift amount thereof are not limited to the above example, and can be set arbitrarily, and can be set so that the fluctuation of the frequency or level of the frequency component analyzed by the FFT processing can be detected as the fluctuation. It should just be.

Next, "harmonic component analysis" (process P5)
Then, the characteristic of the waveform data of the harmonic component separated and generated in the process P4 of “harmonic / out-of-harmonic component separation” is analyzed, an optimal vector quantization process is performed according to the characteristic, and the compressed data is compressed according to the vector quantization. The compressed data of the harmonic component waveform is generated. Here, the harmonic component can be basically regarded as a repetitive waveform.
Using the waveform of a cycle or a plurality of cycles as a representative vector, looping the representative vector (or
By performing a cross-fade loop on one representative vector), a harmonic component waveform can be reproduced. Therefore,
Such a representative vector is stored as a harmonic vector in the harmonic vector storage unit M1.
As compressed data generated in (process P5), vector information specifying a harmonic vector to be used as a representative vector of the harmonic component among the harmonic vectors stored in the harmonic vector storage unit M1, and the designated harmonic vector, that is, the representative harmonic vector. It is preferable to generate information including information describing how to use the vector (for example, a section in which the vector is used, a period during which the vector is looped, or a form of an envelope provided to the vector). A detailed example of this “harmonic component analysis” (process P5) will be further described later.

On the other hand, “out-of-harmonic component analysis” (process P
6) In the process P of the “harmonic / unharmonic component separation”
The characteristic of the waveform data of the non-harmonic component separated and generated in step 4 is analyzed, an optimal vector quantization process is performed according to the characteristic, and compressed data of the non-harmonic component waveform compressed according to the vector quantization is generated. Here, the non-harmonic components should be subjected to vector quantization in an optimal form that matches the characteristics or contents of each non-harmonic component in consideration of the balance between promoting data compression and improving waveform reproducibility. ing. The out-of-harmonic component basically corresponds to a random noise waveform having no repeatability.For example, in consideration of the presence or absence of importance such as its content and where it is located in the sounding period, A non-harmonic component waveform of a required time length may be reproduced by looping a specific representative vector (this is called loop reproduction), or a representative vector having a required time length is not looped. In this case, the required non-harmonic component waveform may be reproduced by using it only once (this is called one-shot reproduction). Alternatively, a required non-harmonic component waveform may be reproduced by switching a plurality of representative vectors in a predetermined order and combining them (this is called sequence reproduction). Alternatively, the switching order may be set at random to emphasize the noise, that is, the randomness of the non-harmonic component (this is referred to as random sequence reproduction). In consideration of such various optimal vector quantization forms, it is preferable to appropriately use a representative vector suitable for various optimal vector quantization forms as the nonharmonic vector. Therefore, such various representative vectors are stored in the non-harmonic vector storage unit M2 as non-harmonic vectors, and the compressed data generated in the “non-harmonic component analysis” (process P6) are stored in the non-harmonic vector storage unit M2. Vector information that specifies a non-harmonic vector suitable for use as a representative vector of the non-harmonic component stored in the non-harmonic vector, and how to use the non-harmonic vector, that is, the representative vector (for example, use the vector It is preferable to generate an information including information describing a section to be performed, a distinction between the loop reproduction, the one-shot reproduction, and the like, a period of the reproduction, and a form of an envelope to be added thereto. A detailed example of this “out-of-harmonic component analysis” (process P6) will be further described later.

In FIG. 2, the rendition style information storage unit M3 stores the compressed data (first compressed data) generated in the “harmonic component analysis” (process P5) and the “non-harmonic component” for each rendition style section. The analysis data (second compressed data) generated in the “analysis” (process P6) is combined with the musical instrument designation data and the rendition style designation data generated in the “reproduction style analysis” (process P3) for the relevant rendition style section. And stored as performance style information. Therefore, for waveform reproduction, the musical instrument designation data and the performance style designation data for the desired performance style are used as indices, and the harmony component and the non-harmonic component (that is, the first and second components) for the corresponding performance style section are stored in the performance style information storage unit M3. The second) compressed data can be recalled. Then, in accordance with the vector information included in the called compressed data, predetermined harmonic vectors and non-harmonic vectors are read from the harmonic vector storage unit M1 and the non-harmonic vector storage unit M2, respectively. By reproducing each of the non-harmonic component waveforms and adding them together, reproduction of the original waveform or generation of a musical sound waveform having characteristics of the original waveform can be performed.

Next, a detailed example of “harmonic component analysis” (process P5) will be described with reference to FIG. Note that the processing illustrated in FIG. 3 does not indicate all of the “harmonic component analysis” (process P5), but mainly generates a new harmonic vector and stores it in the harmonic vector storage unit M1. 2 shows a portion related to a process of storing and storing.

First, in the "waveform section division" process S1, the harmonic component waveform for one rendition style section separated and generated in the "harmonic / non-harmonic component separation" of the process P4 is divided into a plurality of waveform sections. The method of dividing the waveform section is as follows.
A section composed of one or more continuous periodic waveforms having common or similar characteristics of the waveform shape for each cycle is extracted as one waveform section, and divided at the boundaries. Therefore, the size of each divided waveform section is generally unequal. In the "waveform section division" process S1, the user may manually designate an appropriate division position while visually checking the waveform shape displayed on the display 15, or C
PU10 executes a predetermined waveform section dividing program,
The division position may be automatically determined based on automatically analyzing the frequency characteristics of the waveform for each cycle. Alternatively, this may be performed by a combination of manual operation and automatic processing. Note that this “waveform section division” processing S1
As a result, as information that describes how the input harmonic component waveform is divided into a plurality of waveform sections,
“Section information” is generated. In order to efficiently determine whether the characteristics of the waveform shape in each cycle are common or similar, a waveform is cut out in each cycle and a normalization process is performed to make the pitch and amplitude level common. It is preferable that only the waveform shape of a one-period waveform can be compared.
In this case, since the subsequent vector quantization processing is performed on the waveform data in which the pitch and the amplitude level are standardized, the temporal change state of the original pitch and the amplitude level with respect to the standardized pitch and the amplitude level "Pitch vector" and "Amplitude vector"
Should be generated. Here, in some cases, it is preferable to control and normalize the time axis position of the waveform data. In such a case, a “time vector” is generated as a vector indicating a temporal change state of the time axis position of the waveform data.

Next, in the "grouping" process S2, for the waveform data of each divided waveform section, sections having a similar waveform shape for one cycle or a plurality of cycles are grouped into the same group. I do. The main purpose of this grouping is not only to perform between the waveform sections of its own rendition style section, but rather to perform the grouping with the waveform data of each waveform section of other rendition style sections (including different musical instruments). It is assumed that. That is, in the group waveform storage unit M4, the waveform data of a large number of waveform sections for each rendition style section already grouped is stored in a grouped state. "Grouping" process S2
Then, referring to the storage content of the group waveform storage unit M4, the waveform data of each waveform section divided this time is
It is checked whether or not a waveform having a similar waveform shape for one cycle or a plurality of cycles exists in an existing group in the group waveform storage unit M4. If it exists, the waveform data of the waveform section is included in the group. That is, the waveform data of the waveform section is stored in the group waveform storage unit M
4 is stored in the storage area of the group, and “group information” indicating the group is generated. If there is no waveform similar to the existing group, the waveform data of the waveform section is registered as a new group. That is, the waveform data of the waveform section is stored in the storage area of the new group in the group waveform storage unit M4, and “group information” indicating the new group is generated. In this case, in order to increase the data sharing efficiency, as described above, appropriate waveform normalization processing is performed so that the pitch and volume level of the original waveform are common, and grouping is performed by comparing only the waveform shapes purely. Good. Of course, irrespective of the size of each waveform section, the waveform shapes of one cycle or a plurality of cycles characterizing the waveform section are compared. Thus, even if waveform data of different musical instruments, different playing styles, or different pitches are similar, if the waveform shape of a certain waveform section is similar, the waveform data of the similar waveform section Will be classified into a common group. This is because the harmony vector in the harmony vector storage unit M1 (the same applies to the out-of-harmony vector in the out-of-harmony vector storage unit M2) is shared between various performance styles having different characteristics such as different instrument sounds, different performance styles, and different pitches. Means to get,
This means that the storage scale of the harmonic vector storage unit M1 (or the non-harmonic vector storage unit M2) is reduced. Note that this “grouping” process S2 may be performed manually by the user as described above,
It may be performed by automatic processing by U10,
Manual operation and automatic processing may be performed in combination.

Next, "representative vector generation of each group"
In the process S3, a representative vector of the waveform data is generated by referring to the storage contents of the group waveform storage unit M4 and taking into consideration the characteristics of each waveform data in the same group. For example, if the same one cycle or a plurality of cycle waveforms are repeated in one waveform section, it is selected as a temporary representative vector for the time being, and the average or the average of the temporary representative vector of each waveform data in the same group is determined. An intermediate or most characteristic vector is determined as a representative vector of the group and generated. The generated representative vector (specifically, waveform data having one or a plurality of cycles) is stored in the harmonic vector storage unit M1, and registered as a harmonic vector. Also, "vector information" for a harmonic component is generated, which indicates which harmonic vector stored in the harmonic vector storage unit M1 corresponds to the representative vector of the group. This "vector information"
Is index data indicating which harmony vector (specifically, waveform data having one or a plurality of periods) stored in the harmony vector storage unit M1 is to be used as a representative vector. Data.

The "vector information" is stored in the rendition style information storage unit M3 as compressed data of harmonic components in the waveform section in combination with the "section information" generated from the "waveform section division" process S1. You. Therefore, the compressed data (first compressed data) of the harmonic component for one rendition style section is a combination of “vector information” and “section information” for each of a plurality of waveform sections obtained by dividing the rendition style section. Contains. In this case, when the vector quantization processing is performed on the waveform data in which the pitch and the amplitude level are standardized as described above, the “vector information” and the “section information” are subjected to the “waveform section division” processing S1. The generated “pitch vector” and “amplitude vector” are further combined and stored in the rendition style information storage unit M3. Also,
When the vector quantization processing is performed on the waveform data in which the time axis position of the waveform data is also normalized, the “time vector” generated from the “waveform section division” processing S1 is further combined, and the rendition style information storage unit M3 To memorize.

Next, an example of processing of "harmonic component analysis" (process P5) in a state where a relatively large number of harmonic vectors are stored in the harmonic vector storage unit M1 will be described.
In this case, a harmony vector that can be used as a representative vector of the harmony component waveform is mainly searched from among many harmony vectors already stored in the harmony vector storage unit M1, and the searched harmony vector is searched for the harmony component. It is selected as a representative vector.

For example, first, a waveform is cut out for each cycle from the harmonic component waveform for one rendition style segment separated and generated in the “harmonic / non-harmonic component separation” of the process P4, and the pitch and the amplitude level are shared. Then, if necessary, a process for controlling the time axis position of the waveform data for normalization is performed. In this manner, the normalized waveform data is stored in the harmonic vector storage unit M every cycle or every plural cycles.
1 compared to many harmonic vectors already stored in
Search for similar harmony vectors. The search in this case also
In addition to the harmonic vector corresponding to the same natural instrument and / or the same rendition style as the harmonic component waveform to be analyzed, a harmonic vector corresponding to a different natural instrument and / or a different rendition style is included in the search target. When a similar harmony vector is found in this way, a section composed of continuous one or more periodic waveforms similar to the harmony vector is extracted as one waveform section to generate “section information”, and the searched harmony vector is obtained. Is generated as a representative vector of the “waveform section”, and these are grouped and stored in the rendition style information storage unit M3. In this case, as before,
When the vector quantization process is performed on the waveform data in which the pitch and the amplitude level are standardized, the “pitch vector” and the “amplitude vector” indicating the temporal change state of the original pitch and the amplitude level with respect to the standardized pitch and the amplitude level "
Are further combined with the “vector information” and “section information” and stored in the rendition style information storage unit M3. When vector quantization processing is performed on waveform data in which the time axis position of the waveform data is also normalized, a “time vector” indicating a temporal change state of the time axis position of the waveform data is further combined, and The information is stored in the information storage unit M3.

Here, as a result of the above search, if a similar harmony vector does not exist in the harmony vector storage unit M1, the same processing as the processing S1, S2, S3 in FIG. Is divided and grouped to generate a new representative vector, which is registered in the harmonic vector storage unit M1 as a new harmonic vector. Then, “vector information” indicating the new harmony vector is generated and stored in the rendition style information storage unit M3 together with “section information” and the like.

FIG. 4 is a graph showing an example of how the harmonic component waveform is vector-quantized. (A) schematically shows an example of an original waveform by an amplitude envelope.
(B) shows an example in which the harmonic component waveform separated from the original waveform is divided into a plurality of “waveform sections” by a diagram showing a break of each waveform section. (C) shows appropriate harmony vectors W0, W1, W composed of waveform data of one or more cycles as representative vectors in each “waveform section”.
2,... W7 indicate a selected state and an example of loop reproduction at the time of waveform reproduction. For example, in the waveform section from the time point t0 to the time point t1, the harmonic vector W0 is used as a representative vector, and at the time of reproduction, this is reproduced in a loop from the time point t0 to the time point t1. As the loop reproduction, an example is shown in which a cross-fade loop reproduction is performed with a representative vector of the next section (in this case, the waveform at the head of the harmony vector W1). From time t1 to t
For waveform sections up to 2, a relatively large harmony vector W1 (comprising a multi-period waveform) is used as a representative vector, which is reproduced once during a predetermined period from the time t1 to the time t2 during reproduction. It is reproduced and loop-reproduced as appropriate in the remaining period (for example, cross-fade loop reproduction is performed between the last waveform of the harmonic vector W1 and the harmonic vector W2 in the next section). Also, at time t2
For the waveform section from to t3, the harmonic vector W2
Is used as the representative vector, and during playback this is
Loop reproduction is performed between time points t2 and t3 (cross-fade loop reproduction is performed between the harmonic vector W2 and the harmonic vector W3). Hereinafter, the same applies to the last time point t.
In the waveform section 7 and thereafter, the harmony vector W8 is used as a representative vector, and at the time of reproduction, this is reproduced in a simple loop. 4D, 4E, and 4F show examples of “time vector”, “amplitude vector”, and “pitch vector” extracted from the harmonic component waveform of FIG. 4A, respectively.

Next, "out-of-harmonic component analysis" (process P
A detailed example of 6) will be described. The processing procedure of the “out-of-harmonic component analysis” (process P6) is basically the same as that of FIG.
, A procedure substantially similar to the processing procedure of “harmonic component analysis” (process P5) shown in FIG. However, since the analysis target is the non-harmonic component waveform for one rendition style section separated and generated by the “harmonic / non-harmonic component separation” in the process P4, the “waveform section division” processing (S1) In the "grouping" process (S2) and the "representative vector generation of each group" process (S3), the concept of one cycle or multiple cycles of the non-harmonic component waveform itself to be analyzed is not applied. Of course. Further, instead of the harmonic vector storage unit M1, a non-harmonic vector storage unit M2 is used,
When a new representative vector of the non-harmonic component waveform is newly generated in the “representative vector generation of each group” process (S3), this is stored in the non-harmonic vector storage unit M2 as a new non-harmonic vector and registered. Needless to say.

Also in "out-of-harmonic component analysis", "waveform section division" processing similar to "waveform section division" processing S1 in FIG. 3 is performed, and vector quantization processing is performed on each divided waveform section. . In that case, the method of dividing the non-harmonic component waveform into a plurality of waveform sections is generally divided into a plurality of waveform sections for each part whose characteristics are common or similar or inferred as such. Although this may be used as a basic method, in this embodiment, it is proposed to perform one or both of the following two methods in combination.

The first method utilizes the periodicity of the harmonic component in the same rendition style section, and converts the non-harmonic component waveform into a plurality of waveform sections in a period that is an integral multiple or a fractional multiple of the cycle of the harmonic component. Is to be divided into Of course, in this case as well, the divided waveform sections do not need to be at equal intervals. One waveform section is two cycles in length of the harmonic component, and another waveform section is 10 cycles in length of the harmonic component. And so on, as appropriate. As described above, the method of performing the section division of the non-harmonic component waveform depending on the periodicity of the harmonic component can perform the division of the section of the non-harmonic component waveform without much human labor. It has the advantage of being easy. In addition, in a case where the characteristics of the non-harmonic component waveform depend to some extent on the periodicity of the harmonic component, appropriate waveform section division can be easily performed. Although it can be said that the division of the non-harmonic component waveform is performed depending on the periodicity of the harmonic component, the waveform division of the non-harmonic component is naturally different from the corresponding waveform division of the harmonic component. Of course, it goes without saying that the harmonic component and the non-harmonic component are each independently divided into sections.

The second method is to specifically search for portions where the characteristics of the nonharmonic component waveforms are common or similar, and
That portion is divided into a plurality of waveform sections. This method is suitable for extracting a relatively important part of the non-harmonic component waveform and selecting and generating a corresponding representative vector, that is, a non-harmonic vector. Conversely, by extracting less important parts of the non-harmonic component waveform and omitting those less important parts as appropriate, the representative vector, that is, the non-harmonic vector is selected.
It is also suitable for generating and increasing data compression efficiency. Further, as will be described later, even in a case where data compression is promoted by performing a loop reproduction process on a non-harmonic vector, a section suitable for a loop can be appropriately selected. In either case of the first method and the second method, it may be performed by a user's manual operation, may be performed by automatic processing by the CPU 10, or may be a combination of the manual operation and the automatic processing. May go.

In this embodiment, depending on which of the first method and the second method is used to divide the waveform section of the non-harmonic component waveform, the subsequent process of selecting and generating the representative vector for the waveform section is performed. The content is somewhat different. This will be described in more detail with reference to a flowchart. FIG. 5 shows the “out-of-harmonic component analysis” (process P6) in the case of performing waveform section division of the out-of-harmonic component waveform according to the first method and then performing data compression (vector quantization) of the out-of-harmonic component. Here is a detailed example. FIG. 6 shows “out-of-harmonic component analysis” (process P6) in the case where the waveform section of the out-of-harmonic component waveform is divided according to the second method, and then data compression (vector quantization) of the out-of-harmonic component is performed. Here is a detailed example.

First, the embodiment shown in FIG. 5 will be described. In the flow shown in FIG. 5A, first, a nonharmonic component waveform to be compressed is prepared (step S10). Next, the non-harmonic component waveform is divided into a plurality of waveform sections according to the cycle length of the harmonic component corresponding to the non-harmonic component waveform, that is, in synchronization with the periodicity of the harmonic component waveform (step S11). . Here, as described above, the nonharmonic component waveform is divided into a plurality of waveform sections in a period that is an integral multiple or a fractional multiple of the cycle of the harmonic component, that is, in synchronization with the periodicity of the waveform of the harmonic component. . Each of the divided waveform sections does not need to be at equal intervals; one waveform section is half the period of the harmonic component, another is eight cycles of the harmonic component, and so on. As appropriate.

Next, the position of each waveform section divided in the previous step is adjusted appropriately (step S12). For example, in the previous step S11, the harmonic component is divided at every period,
In this step S12, the user checks the result of the division in the previous step S11 on the display 15 and determines the length of a certain waveform section by the number of cycles of the harmonic component. In addition, correction is performed such that the division position is slightly shifted so that the phase of the non-harmonic waveform at the division point becomes a well-divided phase (for example, a zero-cross phase). The adjustment or correction processing in this step is not limited to manual operation by the user, and may be performed by automatic processing according to a predetermined correction program, or may be performed by a combination of manual operation and automatic processing. .

Next, the characteristics of the non-harmonic waveforms in each of the divided waveform sections are analyzed to perform grouping (step S13). Similar to the "grouping" process S2 in FIG. 3, this grouping is also performed between the waveform data of each non-harmonic waveform section of another rendition style section (including different musical instruments), and the representative vector, that is, the harmonic vector. The sharing of external vectors shall be promoted. FIG. 5B shows a more detailed flowchart example of the characteristic analysis and grouping processing of the non-harmonic waveform executed in step S13.

In FIG. 5B, "peak value analysis"
In (Step S30), it is analyzed where the peak level of the non-harmonic waveform (that is, the noise-like waveform) is located in the divided waveform section. Next, "location analysis"
In (Step S31), the place where the peak level of the non-harmonic waveform in the divided waveform section exists is determined by which part (for example, the first half part) of one cycle or a plurality of predetermined cycles of the harmonic component corresponding to the waveform section. , The middle part, the latter part, etc.). In the next “zero-crossing frequency analysis” (step S32), the number of times that the amplitude of the non-harmonic waveform crosses zero in the divided waveform section is analyzed. The intensity of the noise change can be analyzed based on the number of zero crossings. In the next step S33, the step S
Based on the results analyzed in S30 to S32, the relevant waveform section of the non-harmonic waveform is classified into a required group. Since the waveform to be grouped is a non-harmonic waveform having no periodicity by itself, a special analysis as in steps S30 to S32 is performed, and the analysis result is used as a condition or reference for grouping. , So that quantitative evaluation for grouping can be performed. Conditions or evaluation criteria for grouping this kind of non-harmonic waveform are not limited to those shown in steps S30 to S32, and include, for example, the degree of the DC offset level in the waveform section and the waveform in the waveform section. Other appropriate conditions or evaluation criteria, such as the number of sign inversions of the differential value (also an index indicating the degree of noise-like change), may be used. An arbitrary number of conditions or evaluation criteria may be used in combination from the three conditions or evaluation criteria. The processing of each step in FIG.
It is good to carry out interactively according to a manual operation of the user. Of course, it may be fully automatic processing according to a predetermined program, or may be performed by a combination of manual operation and automatic processing.

Returning to FIG. 5A, in step S14,
For all out-of-harmonic component waveforms to be subjected to data compression processing in this processing routine, it is checked whether or not the processing of steps S11 to S13, that is, the processing of waveform section division and grouping has been completed. If the processing has not been completed, the process returns to step S11, and the above-described S
Steps S11 to S13 are performed. When the waveform section division and the grouping process for all non-harmonic component waveforms are completed in this way, the process goes to step S15, where the characteristic analysis of the waveform of each group is performed. For example, an amplitude envelope characteristic of each waveform in the group is analyzed. In the next step S16, a representative vector of the group is determined using the analysis result in the previous step S15. For example, a waveform having an average amplitude envelope characteristic of the amplitude envelope characteristics of each waveform in the group may be determined as the representative vector. Alternatively, an average of the frequency characteristics, the number of zero crossings, the level distribution, and the like of each waveform in the group may be determined as the representative vector. Or amplitude envelope characteristics,
A representative vector may be selected or determined on the condition that one or more of the frequency characteristics, the number of zero crossings, the level distribution, and other appropriate characteristics are average or suitable for being representative. . Then, the determined representative vector of the group is stored in the non-harmonic vector storage unit M2 as a non-harmonic vector. Thus, the representative vector of each group is stored and registered as a non-harmonic vector in the non-harmonic vector storage unit M2. The non-harmonic vector stored in the non-harmonic vector storage unit M2 uses a normalized amplitude level, and extracts the amplitude envelope of the non-harmonic component waveform of each waveform section as an “amplitude vector”. And Alternatively, the non-harmonic vector stored in the non-harmonic vector storage unit M2 need not standardize the amplitude level. In that case, the low-level data is stored as it is at the low level, but in the low-level part, the memory storage location can be efficiently used by performing limited storage bit allocation. If the amplitude level of the non-harmonic vector stored in the non-harmonic vector storage unit M2 is not normalized, the difference between the amplitude envelope of the non-harmonic component waveform in each waveform section and the amplitude envelope of the non-harmonic vector is defined as “amplitude vector”. Good to extract.

In the next step S17, the rendition style information of each non-harmonic component ("section information" of each waveform section, "vector information" designating a non-harmonic vector as a representative vector, "amplitude vector", etc. are included. ) Is stored in the rendition style information storage unit M3. In this case, information such as the number of periods of the harmonic component corresponding to the non-harmonic vector, that is, the representative vector, is stored as “section information”. Then, even for harmonic components having different pitches, if the waveform shapes of the non-harmonic components have similarity or commonality, a common non-harmonic vector can be used. Of course, the non-harmonic vector stored in the non-harmonic vector storage unit M2 can be shared in various cases in the same way as the harmonic vector or more. For example, a common non-harmonic vector can be used in different waveform sections in the same rendition style section (the same non-harmonic waveforms having different time axis positions), and different instrument sounds or different rendition style sections can be used. In out-of-harmonic waveforms, a common out-of-harmonic vector can also be used.

Next, the embodiment shown in FIG. 6 will be described. First, a non-harmonic component waveform to be compressed is prepared (step S
20). Next, in step S21, the features of each part of the non-harmonic component waveform to be processed are observed and analyzed, and the non-harmonic component waveform is divided into a plurality of waveform sections by dividing the waveform into portions corresponding to the respective partial features. To divide. In this case, as a partial characteristic of the non-harmonic component waveform, analysis is performed in consideration of the position and performance of the waveform, the characteristics of the waveform shape, and the like, and the waveform section to be divided is determined. For example, as the position, "Attack" (rising part of sound), "Body" (steady part of sound), "Joint" (part connected to other sounds), "Release" (decay part of sound), etc. are considered. Is done. As the playing style, in relation to each position, for example, "attack" takes into account a "fluctuating" playing style, "noise" playing style, and the "body", a "vibrato" playing style, a "tremolo" playing style Considering the playing style etc., "joint"
The "slur" playing style, the "glissand" playing style, the "bend" playing style, etc. are taken into consideration. The "release" takes into account the "mute" playing style, "sustain" playing style, etc. The characteristics of the waveform shape include an amplitude envelope shape (such as an effective value and a peak value), the number of zero crossings or a period between zero crosses, the number of peaks or a period between peaks, a location where the peak value exists, and a plurality of peaks. Various factors, such as level distribution, should be considered as appropriate.

Next, in step S22, waveforms having similar characteristics are grouped based on the characteristics of the non-harmonic waveforms in each of the divided waveform sections. The similarity of the features of the out-of-harmonic waveform may be determined based on an appropriate standard, such as visually observing the waveform displayed on the display device, or generating a non-harmonic waveform for the waveform section and determining the similarity by hearing. Similar to the above, this grouping is also performed between the waveform data of each waveform section of the non-harmonic waveforms for other rendition style sections (including different musical instruments), thereby promoting the sharing of the representative vector, that is, the non-harmonic vector. Shall be.

Next, in step S23, it is checked whether or not the processing of steps S21 to S22, that is, the division of the waveform sections and the grouping thereof have been completed for all the non-harmonic component waveforms to be subjected to the data compression processing in the current processing routine. I do. If not, the process returns to step S21, and the above-mentioned steps S21 to S22 are performed for the next non-harmonic component waveform.
Is performed. In this way, when the waveform section division and the grouping process are completed for all non-harmonic component waveforms,
The process proceeds to step S24 to analyze the characteristics of the waveform of each group. For example, an amplitude envelope characteristic of each waveform in the group is analyzed, and various other characteristics are considered.

In the next step S25, the previous step S2
The representative vector of the group is determined using the analysis result in step 4. The determined representative vector of the group is stored as a non-harmonic vector in the non-harmonic vector storage unit M2. Thus, the representative vector of each group is stored and registered as a non-harmonic vector in the non-harmonic vector storage unit M2. Note that, as described above, the non-harmonic vector stored in the non-harmonic vector storage unit M2 may or may not use a standardized amplitude level. Here, in this embodiment, as a technique for compressing the non-harmonic component waveform, it is considered that various different techniques such as those described below are appropriately adopted. Shall be determined in accordance with the standards in accordance with it.

Compression Example 1: Compression is performed so as to cover the waveform section by reproducing once a representative vector having a predetermined period length. This is a compression mode corresponding to the “one-shot reproduction”. For example, a representative vector for a waveform section of a portion where the waveform shape of the non-harmonic component such as an attack portion or a joint portion also changes abruptly may be determined so as to be suitable for the first compression example. Compression example 2: Compression is performed so as to cover the waveform section by repeatedly reproducing (loop reproducing) a representative vector having a relatively short period length. This is a compression mode corresponding to the “loop reproduction”. By combining a plurality of representative vectors for loops in a predetermined order, the representative vectors can be used for the “sequence reproduction”. Also, by setting this sequential combination at random, it can be used for the "random sequence reproduction". For example, a representative vector for a waveform section of a monotonously decreasing portion or a portion where the same state continues may be determined to be suitable for the second compression example. Compression Example 3: Some non-harmonic component waveforms of less important parts may be omitted. It is not necessary to select a representative vector for such omitted waveform sections.

Next, in step S26, for each waveform section of each non-harmonic component waveform, during reproduction, which non-harmonic vector should be used as a representative vector and how the non-harmonic vector should be pasted To decide. This is related to which type of the above compression example is used. Non-harmonic component reproduction mode information (or “pasting information”) indicating which of the following various reproduction modes should be used for pasting.
It is described as

One-shot reproduction: compression example 1 as described above
Is used, and the non-harmonic component waveform of the waveform section is reproduced by pasting (reproducing) the representative vector once. Sequence reproduction: As described above, a plurality of representative vectors according to the compression example 2 are used, and these representative vectors are sequentially pasted (reproduced) in a predetermined order to reproduce the non-harmonic component waveform in the waveform section. Random sequence reproduction: As described above, a plurality of representative vectors according to the compression example 2 are used, and these representative vectors are sequentially pasted (reproduced) in a random order to reproduce the non-harmonic component waveform of the waveform section. Loop reproduction: As described above, one representative vector according to the compression example 2 is used, and this is repeatedly pasted (reproduced).
Thus, the non-harmonic component waveform in the waveform section is reproduced. This may be combined with the sequence reproduction or the random sequence reproduction. Alternative loop reproduction: One representative vector according to the compression example 2 is used, and this is repeatedly attached (reproduced) while alternately changing the normal / reverse direction, thereby reproducing the non-harmonic component waveform in the waveform section. This may be combined with the sequence reproduction or the random sequence reproduction. Delete reproduction: As described above, according to the third compression example, the non-harmonic component waveform in the waveform section is deleted without using the representative vector. Therefore, in that portion, only the harmonic component waveform is obtained. Note that the entirety of one waveform section may not be in the delete playback mode, but only a part thereof may be in the delete playback mode. In this case, a representative vector is appropriately determined for the remaining portion.

In the next step S27, the rendition style information of each non-harmonic component (“section information” of each waveform section, “vector information” specifying a non-harmonic vector as a representative vector, the above “pasting information” and “ Amplitude vector etc.) is stored in the rendition style information storage unit M3. Note that the various out-of-harmonic component reproduction modes described above are not applied only when the “out-harmonic component analysis” processing according to the second method is performed, but the “out-harmonic component analysis” according to the first method is performed. The processing is also appropriately applied when the processing is performed.

FIG. 7 is a graph showing an example of how the non-harmonic component waveform is vector-quantized. (A) shows FIG.
As in (a), an example of the original waveform is schematically shown by an amplitude envelope. (B) shows an example of the non-harmonic component waveform separated from the original waveform. (C)
The non-harmonic component waveform of (b) is divided into a plurality of "waveform sections" A1, B
1, C1, D1, C2, A2, C3
This is shown by a diagram showing the break of each waveform section. Here, the waveform sections A1 and A2 are sections in which waveform sections are set according to the second method and the one-shot reproduction is applied. The waveform section B1 is a section in which the representative vector is determined in synchronization with the cycle of the harmonic component according to the first method. In this section, one-shot reproduction or loop reproduction may be employed. The waveform sections C1, C2, and C3 are sections corresponding to the loop reproduction. The waveform section D1 is a section corresponding to the delete reproduction.

Further, specific examples of vector quantization of the non-harmonic component waveform are shown in FIGS. 8 to 10, FIG.
In FIG. 3, (a) in the upper part shows an example of the original waveform of the non-harmonic component waveform, and (b) in the lower part shows an example of the waveform obtained by vector-quantizing the reproduced waveform and reproducing it. I have. First, FIG. 8 divides the original waveform into three sections of “attack”, “body”, and “release”,
The “attack” section and the “release” section determine a corresponding representative vector as a section corresponding to the one-shot playback, and the “body” section is a compression in which a non-harmonic component waveform is cut as a section corresponding to the delete playback. An example is shown. If you actually listen to the playback sound based on this, the nonharmonic component waveform will have an “attack” section and a “release” section,
It can be understood that even when the steady tone color stable portion (the “body” section) is omitted, there is not much discomfort.

FIG. 9 shows that the non-harmonic component waveform is divided into four sections F1.
.., F4,..., And F4. Even if the non-harmonic component is used, the periodicity of the component is somewhat increased by looping the representative vector. However, since the level is low, mixing with the reproduced waveform of the harmony component does not give a sense of incongruity. FIG.
Divides the non-harmonic component waveform into three sections F1, F2, and F3, uses the original waveform as it is as the representative vector in the first section F1 where the variation is rich, and uses the representative section of a predetermined size in the next sections F2 and F3. A compression example is shown in which vectors are determined, loop reproduction is performed for each of the vectors, and the amplitude envelope is varied. Thus, by adding an appropriate amplitude envelope to the loop reproduction waveform, a feeling similar to the original waveform can be obtained.

FIG. 11 shows an example of level control in loop reproduction. (A) shows an example of simple loop reproduction, in which level control is not performed. (B) shows an example in which the periodicity is weakened by attenuating the peak level of the representative vector to be looped (an example of loop reproduction with peak correction). (C)
An example in which a change in the amplitude envelope is further added to the reproduction example of (b) to further reduce the periodicity is shown. Thus, it is effective to combine the level control with the loop reproduction. FIG. 12A shows a normal loop playback example,
(B) shows an example of alternative loop reproduction. As can be understood from the figure, the periodicity of the alternative loop reproduction is longer than the periodicity of the normal loop reproduction, so that the size of the representative vector may be further reduced. Therefore, it contributes to further data compression.

FIG. 13 shows that the non-harmonic component waveform is divided into eight sections F
1 to F8, an attack section F1, sections F3, F5 and F7 including vibrato, and a release section F8 are represented by a representative vector (non-harmonic vector) f extracted from the original waveform.
Waveform reproduction is performed using 1, f3, f5, f7, and f8, and for other sections F2, F4, and F6, nonharmonic vectors f2 ', f4', and f6 'extracted from other original waveforms.
Is used as a representative vector to perform waveform reproduction. In each section, an appropriate reproduction mode such as one-shot reproduction or loop reproduction is adopted. For example, as for the body section F2, the non-harmonic vector f2 ′ extracted from another original waveform is used as a representative vector as a representative vector, and is reproduced in a loop, and an amplitude envelope is appropriately added to the original waveform (see FIG. 13A). ). Other sections F4, F6
Also for non-harmonic vector f extracted from other original waveforms
By appropriately assigning an amplitude envelope to 4 ′ and f6 ′ and looping as necessary, the original waveform (FIG. 13)
An out-of-harmonic component waveform similar to (a)) is reproduced and generated. In this way, a common nonharmonic vector can be used for a plurality of different original waveforms. In that case, by controlling various processing methods at the time of reproduction (including data describing an appropriate reproduction method such as amplitude vector and pasting information in the compressed data, etc.), it is optimal for each original waveform. It is possible to reproduce and generate a non-harmonic component waveform.

Next, a description will be given of an embodiment in which the tone generation processing is performed using the compressed data of the harmonic component and the non-harmonic component created as described above. FIG. 14 shows the preparation of a score with a playing style so that the compressed data of the harmonic and non-harmonic components for a large number of playing style waveforms created as described above can be fully utilized in playing any music piece. 1 schematically shows an example of a processing process. First, musical score information (for example, performance data composed of a MIDI form) about a desired musical piece for which a musical score with a playing style is to be prepared is prepared (step S40). Next, for each performance part of the prepared score information, it is analyzed what performance technique should be used in the course of the flow of each music piece (step S41). This rendition style analysis may be performed by a human reading a musical score and making a judgment based on the musical knowledge. Of course, the rendition style analysis may be performed by the CPU 10 using a predetermined score analysis program, or may be performed by a combination of manual operation and automatic processing.

Next, for each performance part of the prepared score information, for each performance data, the necessary performance corresponding to each analyzed performance style in the time-series flow of the performance data. At the point corresponding to the time point, the above step S4
Performance style data indicating each performance style analyzed in 1 is added (step S42). For example, performance style data indicating a required vibrato performance style is added to a location corresponding to the performance time of a note or phrase to which vibrato is to be provided. Alternatively, performance data indicating a required slur performance is added to a location corresponding to a note or phrase to be played by the slur. The place where the required rendition style data is added is a place corresponding to one sound (for example, the same place as the note-on event) or a place corresponding to the middle of one sound (for example, a predetermined time after the note-on event of the sound, and , A performance event is inserted at a location corresponding to an appropriate point in time before the note-off event of the sound) or a location corresponding to one phrase composed of a plurality of notes (for example, an on-event of a predetermined performance style at the beginning of the phrase) , And an off event of the playing style is inserted at the end of the phrase). The contents of the rendition style data to be added include "reproduction style name" data indicating the names of various other rendition styles such as vibrato, slur, staccato, glide, pitch bend, and "parameter" data indicating the degree of the rendition style. And
Further, it includes “range” data indicating a period or a range in which the rendition style is given. The “range” data is data that specifically indicates the rendition style is assigned to only one corresponding tone, or the rendition style is assigned to two sounds before and after the corresponding tone. When the rendition style is assigned in a long range, the range may be specified by the rendition style on event and the off event as described above. As described above, the performance data to which the rendition style data is added is stored in an appropriate memory as the rendition style information.

FIG. 15 is a functional block diagram schematically showing an example of a process of reproducing a musical tone, that is, generating a waveform, based on the musical score information with performance style prepared as described above. This tone reproduction, that is, waveform generation processing,
This is performed by the CPU 10 executing a predetermined software program using the hardware device of FIG.
In the "reproduction control" process S50, performance data is sequentially read out according to a predetermined tempo clock, and a musical tone waveform having a pitch, tone, and the like specified by the performance data is generated, as is known in normal automatic performance sequence control. .
Here, the difference from normal automatic performance sequence control is
The performance data to be played is composed of the “musical score information with performance style” created as described above, and a unique performance style waveform reproduction / generation process corresponding to the performance style data contained therein is performed. It is a point that is performed.

The "reproduction control" process S50 will be further described. First, the performance data (for example, MIDI data) of each performance part and the performance data attached thereto are stored in the storage section M5 of "music score information with performance". Reproduction and reading are performed sequentially according to the timing. `` Score with playing technique ''
When the rendition style data is read for a certain performance part, processing for generating a rendition style waveform corresponding to the rendition style data is performed. First, the "instrument name" (tone color) specified for the performance part and the "performance style name" included in the performance data.
In accordance with the data of the “parameter” and the “parameter”, the “reproduction style information” for the predetermined rendition style waveform specified by the information from the rendition style information storage unit M3, that is, the compressed data (first and ) Is read out. As described above, the compressed data (first compressed data) for the harmonic component includes “vector information” indicating a harmonic vector, “section information”, “pitch vector”, “amplitude vector”, and “time vector”. ,
And “information for controlling the cross-fade loop reproduction mode”. The compressed data (second compressed data) of the non-harmonic component includes “vector information” indicating a non-harmonic vector, “section information”, “pasting information” indicating a reproduction mode thereof, and “amplitude vector”. , "Time vector" and the like.

In the "harmonic reading" process S51, the harmonic vector is read from the harmonic vector storage unit M1 based on the "vector information" of the compressed data (first compressed data) of the harmonic component. In the “harmonic processing” process S52, the harmonic vector storage unit M1
A required harmony component waveform is reproduced / generated based on the harmony vector read out from the PID and other information of the compressed data (section information, pitch vector, amplitude vector, time vector, information for controlling the cross-fade loop reproduction mode, etc.). . That is, the waveform of the harmony vector is appropriately looped (repeated) according to the section information, and the pitch is temporally controlled according to the pitch vector (of course, the basic pitch is set and controlled according to the note data included in the MIDI performance data). And the amplitude envelope is set and controlled according to the amplitude vector, the time axis position of the waveform data is extended or compressed according to the time vector, and the cross-fade loop reproduction mode is controlled according to the information. , The amplitude envelope control for cross-fade is performed with the fade-in or fade-out characteristics.

In the "out-of-harmonic reading" process S53, the out-of-harmonic vector storage unit M based on the "vector information" of the compressed data (second compressed data) of the out-of-harmonic component.
Read out-of-harmonic vectors from 2. In the “out-of-harmonic processing” process S54, the required harmony is determined based on the non-harmonic vector read from the non-harmonic vector storage unit M2 and other information (section information, paste information, amplitude vector, time vector, etc.) of the compressed data. Regenerate and generate external component waveforms. In other words, according to the instruction of the pasting information and the section information, the waveform of the non-harmonic vector is looped appropriately or only once according to the predetermined non-harmonic component reproduction mode such as the one-shot reproduction or the loop reproduction or the sequence reproduction. Paste the required section. Further, the amplitude envelope is set and controlled according to the amplitude vector, and the time axis position of the non-harmonic component waveform data is expanded or compressed according to the time vector.

In the case where the compressed data of the non-harmonic component to be reproduced is data according to the above-described first method (data obtained by performing interval division synchronized with the periodicity of the corresponding harmonic component and performing vector quantization), When or after reading a non-harmonic vector from the non-harmonic vector storage unit M2 based on the "vector information", the time length of the non-harmonic vector is appropriately expanded or contracted in accordance with the cycle of the harmonic component to be reproduced simultaneously at this time. Shall be controlled. This is because, considering the compression method of the compressed data (the first method), it is better to expand and contract the time length of the nonharmonic vector in synchronization with the expansion and contraction of the cycle of the harmonic component to be reproduced. This is because the reproducibility of the waveform is improved. The time length expansion / contraction control of the non-harmonic vector may be a simple way of changing the read rate of the non-harmonic vector storage unit M2 in conjunction with the change of the read cycle (read rate) of the corresponding harmonic vector. Alternatively, the time axis expansion / contraction control may be employed.
Of course, if the compressed data of the non-harmonic component to be reproduced is data according to the second method (vector quantization performed by performing interval division irrespective of the periodicity of the harmonic component), basically such a case will be described. No special consideration is required, but it may be done. Various playback modes such as “one-shot playback”, “loop playback”, “alternative loop playback”, “sequence playback”, “random sequence playback”, “delete playback”, etc. Even if the compressed data of the component is data according to the first method (data obtained by performing interval division synchronized with the periodicity of the corresponding harmonic component and performing vector quantization), it is of course applicable to the present invention. It is.

In the "mixing" process S55, the harmonic component waveform reproduced and generated as described above is mixed with the non-harmonic component waveform, and tone waveform data obtained by mixing both components is generated. Thus, the musical tone waveform data is reproduced and provided (step S56). The reproduced musical sound waveform data may be supplied as digital data to another utilization device, or may be converted into an analog signal and spatially generated through an appropriate sound system. In addition, the signal of the harmonic component waveform and the signal of the non-harmonic component waveform may be separately generated, and the two may be mixed in space. As described above, it is possible to reproduce and generate a musical sound waveform having characteristics quite faithful to the original waveform while employing the compressed data structure. In addition, each processing S52, S
By variably controlling the waveform processing at 54, it is possible to perform a very useful waveform generation process, which is a high-quality waveform faithful to the original waveform, but also very controllable.

[0073]

As described above, according to the present invention, the input waveform data is separated into a harmonic component and a non-harmonic component, and a process of separately vector-quantizing the separated harmonic component and the non-harmonic component is performed. As a result, there is an excellent effect that the compression efficiency can be improved by vector-quantizing both the harmonic component and the non-harmonic component. In addition, since it is not necessary to perform vector quantization in consideration of the combination of the periodic vector and the noise vector as in the related art, the vector selection of the harmonic component and the non-harmonic component is performed independently of each other. This has an excellent effect that processing can be easily performed. In particular, in the vector quantization of the out-of-period component, the vector quantization process is performed based on the out-of-harmonic component separated from the original waveform. Play. As described above, according to the present invention, a waveform compression technique and a waveform generation technique that are easy to select a vector, have excellent compression efficiency in vector quantization, and are very excellent in reproducibility of an original waveform are provided. Can be provided.

[Brief description of the drawings]

FIG. 1 is a block diagram schematically showing an example of a hardware configuration used to execute a waveform compression method and a waveform generation method according to an embodiment of the present invention.

FIG. 2 is an equivalent block diagram schematically showing a procedure of a waveform compression process executed under the control of a CPU in FIG. 1;

FIG. 3 is an equivalent block diagram showing a detailed example of a “harmonic component analysis” process in FIG. 2;

FIG. 4 is a graph showing an example of how a harmonic component waveform is vector-quantized.

FIG. 5 is a flowchart illustrating an example of “out-of-harmonic component analysis” processing in FIG. 2, in which a non-harmonic component waveform is divided into a plurality of waveform sections using the periodicity of a harmonic component, and analysis is performed. FIG.

FIG. 6 is a flowchart illustrating another example of the “out-of-harmonic component analysis” process in FIG. 2, which divides the out-of-harmonic component waveform into a plurality of waveform sections based on the characteristics of the out-of-harmonic component waveform and performs analysis. FIG.

FIG. 7 is a graph showing an example of how a non-harmonic component waveform is vector-quantized.

FIG. 8 is a graph showing a specific example of vector quantization of a non-harmonic component waveform.

FIG. 9 is a graph showing another specific example of vector quantization of a non-harmonic component waveform.

FIG. 10 is a graph showing still another specific example of vector quantization of a non-harmonic component waveform.

FIG. 11 is a graph showing another specific example of vector quantization of a non-harmonic component waveform.

FIG. 12 is a graph showing still another specific example of vector quantization of a non-harmonic component waveform.

FIG. 13 is a graph showing another specific example of vector quantization of a non-harmonic component waveform.

FIG. 14 is an equivalent block diagram schematically illustrating an example of a process of preparing a musical score with a playing style.

FIG. 15 is an equivalent block diagram schematically showing a procedure of tone reproduction, that is, a waveform generation process executed under the control of the CPU in FIG. 1;

[Explanation of symbols]

Reference Signs List 10 CPU (Central Processing Unit) 11 ROM (Read Only Memory) 12 RAM (Random Access Memory) 13 Hard Disk Device 14, Removable Disk Device (for example, CD-RO)
15 Display 16 Input operation device such as keyboard and mouse 17 Waveform interface 18 Timer 19 Interface for MIDI and communication network 20 CPU bus

──────────────────────────────────────────────────続 き Continued on the front page (51) Int.Cl. ⁷ Identification symbol FI Theme coat ゛ (Reference) G10L 9 / 18H

Claims (2)

[Claims] 1. a step of separating input waveform data into a harmonic component and a non-harmonic component; a step of supplying a harmonic vector; and vector-quantizing the separated harmonic component by the harmonic vector to perform first compression. A waveform compression method comprising: generating data; supplying a non-harmonic vector; and vector-quantizing the separated non-harmonic component with the non-harmonic vector to generate second compressed data. Receiving a first compressed data including a predetermined harmonic vector instruction information and a second compressed data including a predetermined non-harmonic vector instruction information; supplying a harmonic vector; Synthesizing the waveform data of the harmonic component based on the first compressed data and the harmonic vector; providing the non-harmonic vector; and the waveform of the non-harmonic component based on the second compressed data and the non-harmonic vector. A method of generating waveforms, the method comprising combining data.