JP3788096B2 - Waveform compression method and waveform generation method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To effectively and efficiently compress inharmonic components. SOLUTION: Waveform data to be compressed are separated by a proper method into harmonic components and inharmonic components, which are supplied separately (S10). To perform vector quantization for at least the inharmonic components, the periodicity of the harmonic components corresponding to the inharmonic components is detected and according to the detected periodicity (e.g. an integral multiple or submultiple of cycles), the inharmonic components are divided into sections (S13). Inharmonic vectors as representative vectors of the inharmonic components are selected and specified respectively by the divided sections, thereby performing the vector quantization (S16). Thus, the inharmonic component waveform itself is analyzed and an original vector quantizing process can easily be performed.

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a waveform compression method for compressing a waveform of a musical tone, a voice or other various sounds using a vector quantization technique, and also relates to a waveform generation method for generating an arbitrary waveform using such a waveform compression method. In particular, the present invention relates to an improved non-harmonic component compression method. The present invention is not limited to electronic musical instruments, but in general performance devices, computers, electronic game devices, other multimedia devices, etc., in a wide range of devices, devices, or methods having a function of generating musical sounds or sounds or other arbitrary sounds. It can be applied in a wide range. In this specification, the term “musical sound waveform” is not limited to a musical sound waveform, but is used in a sense that may include a sound waveform or any other sound waveform.
[0002]
[Prior art]
Recently, in electronic musical tone synthesis technology, it has become an important issue how to synthesize high-quality musical tones in consideration of differences in performance. That is, in the case of natural musical instruments, even if a musical tone has the same musical instrument tone and the same pitch, different musical tone characteristics (especially the timbre) are different for each performance with different performances, such as vibrato and slur. It is known that a musical tone indicating a waveform is generated. When trying to generate high-quality musical sounds that reflect the differences in playing styles of such natural musical instruments using electronic musical tone synthesis technology in electronic musical instruments, etc., for each pitch (or range) of each musical instrument tone Corresponding to a plurality of performance methods, waveform data indicating musical tone characteristics (especially timbre waveforms) according to each performance method are stored in advance in the memory, and waveform data corresponding to the performance method to be played is read from the memory. Thus, it is conceivable to generate a musical sound waveform that shows a specific musical sound characteristic (particularly a timbre waveform) corresponding to the playing style.
[0003]
As described above, when storing different waveform data corresponding to a plurality of performance methods for each pitch (or range) of each instrument tone color, the memory storage capacity required for normal storage is large. Since it will increase considerably, how to perform effective data compression and save necessary memory storage capacity becomes an important issue. Accordingly, among the conventional waveform data compression techniques, there are the following ones that should be noted in connection with the present invention.
[0004]
First, in Japanese Patent Application Laid-Open No. 61-104400, an input waveform is separated into a periodic component and an aperiodic component by filtering, and each component is encoded in a different data compression encoding format and stored in a waveform memory. The sound recording method is shown. However, although there is an idea that specific waveform data is separated into a periodic component and a non-periodic component and is compressed and stored, there is no idea to perform vector quantization for all of them. In particular, for non-periodic components, the residual waveform with respect to the waveform of the periodic component can only be recorded in time series.
[0005]
Japanese Patent Laid-Open No. 5-88911 discloses a speech coding method using vector quantization. In this case, the speech waveform to be compressed is first processed by a filter having an inverse characteristic of the spectrum extracted from the waveform, and then the excitation vector extracted from the past predetermined period of the waveform for the filtered waveform. Is used to perform vector quantization of the periodic component and vector quantization of the non-periodic component using the noise vector. In this case, a noise vector prepared as a fixed vector is used, and a non-periodic component (noise component) is extracted from the speech waveform itself to be compressed and based on this, a non-periodic component vector is used. It is not designed to perform quantization.
[0006]
[Problems to be solved by the invention]
However, in the former waveform data compression technique described above, since the data compression rate was low, different waveform data corresponding to a plurality of performance methods is stored for each pitch (or range) of each instrument tone color. It has been difficult to achieve sufficient data compression suitable for such purposes. That is, in the former prior art, since there was no idea of vector quantization of the aperiodic component, the data storage of the aperiodic component had to record the residual waveform with respect to the waveform of the periodic component as it was in time series. It did not realize sufficient data compression.
[0007]
On the other hand, in the latter waveform data compression technique described above, a noise vector prepared as a fixed vector is used, and an aperiodic component (noise component) is extracted from the speech waveform itself to be compressed. Based on this, the vector quantization of the aperiodic component is not performed, so the vector quantization accuracy of the noise vector is poor, and the aperiodic component cannot be efficiently compressed. The reproduction accuracy was also poor. Furthermore, in this waveform data compression technique, the filtering process using the inverse characteristic filter is always performed, so that the shape of the reproduced waveform is the same as that of the original waveform by the intervention of this powerful inverse characteristic filter process. There was a drawback that the waveform was different and the waveform reproducibility was poor. Therefore, even if it can be used for reproducing a human voice waveform that does not require a high precision of waveform reproducibility, it is not suitable for reproducing a waveform of a musical instrument sound that requires a precise waveform reproducibility. In this waveform data compression technique, compression using a periodic vector and a noise vector is performed. However, as a combination of a periodic vector and a noise vector, one waveform subjected to the inverse characteristic filtering process is used. Therefore, it is difficult to determine the combination, and precise compression cannot be performed. Furthermore, since the excitation vector extracted from the past waveform is often used as the periodic vector, the followability with respect to a waveform rich in change is low.
[0008]
The present invention has been made in view of the above points, and is intended to provide a waveform compression method that is excellent in data compression efficiency and excellent in reproducibility of an original waveform. It is an object of the present invention to provide a waveform compression method in consideration of enabling effective and efficient compression. Furthermore, the present invention intends to provide a waveform generation method for generating an arbitrary waveform, particularly an arbitrary out-of-harmonic component waveform, using such a waveform compression method.
[0009]
[Means for Solving the Problems]
The waveform compression method according to the present invention includes: The process of supplying a plurality of non-harmonic vectors as stored non-harmonic component waveform data, and compression Non-harmonic component of waveform data And corresponding harmonic components And supplying the harmony component period And detecting the harmonic component detected period Based on Of waveform data to be compressed The process of dividing out-of-harmonic components into multiple intervals When, For each divided section In addition, Non-harmonic ingredients A non-harmonic vector that can be used as a plurality of non-harmonic vectors, respectively. Vector quantization, Each including instruction information indicating the selected out-of-harmonic vector And a process of generating compressed data for out-of-harmonic components.
[0010]
Waveform data to be compressed is separated into a harmonic component and a non-harmonic component by an appropriate method and supplied separately. According to the present invention, at least an out-of-harmonic component is processed for vector quantization. In order to vector quantize the non-harmonic component waveform, it is desirable to divide the target non-harmonic component waveform into a plurality of sections and perform optimal vector quantization for each section. As a section division technique for this purpose, in the present invention, the harmonic component corresponding to the out-of-harmonic component is determined. period Detected and detected period The non-harmonic component is divided into a plurality of sections based on the above. That is, the stored non-harmonic component waveform data is supplied with a plurality of non-harmonic vectors, and the non-harmonic vectors that can be used as the non-harmonic component for each section divided based on the period of the harmonic component are the plurality of harmonics. Each out-of-harmonic component is vector-quantized by selecting from outside vectors, and compressed data is generated for each out-of-harmonic component including instruction information indicating the selected out-of-harmonic vector. For example, the non-harmonic component waveform is divided into a plurality of sections in a period that is an integral multiple or a fraction multiple of the harmonic component period. Of course, the sections of the non-harmonic component waveform to be divided do not need to be equally spaced, a certain section has a length corresponding to ½ period of the harmonic component, and another waveform section has a length corresponding to eight periods of the harmonic component. It may be different as appropriate.
[0011]
Thus, the harmonic components period The method of dividing the non-harmonic component waveform depending on the non-harmonic component waveform can automatically perform the division of the non-harmonic component waveform without much labor. . In addition, the characteristics of the non-harmonic component waveform are period In such a case, an appropriate waveform section division can be easily performed. In addition, in vector quantization of the non-periodic waveform component, the vector quantization process is performed based on the inherent non-harmonic component that forms the original waveform together with the harmonic component, so that the reproduction accuracy of the non-periodic waveform component is improved.
[0012]
Incidentally, when vector quantization of a periodic component (harmonic component) waveform is performed, a representative vector (representative waveform segment) is generally selected in a section having a similar waveform shape. Therefore, even in the case of vector quantization of non-periodic components (non-harmonic components), it is very important to determine in which range the waveforms are similar and perform appropriate section division. However, since the non-periodic component (non-harmonic component) itself is a noisy waveform, it is cumbersome to determine the waveform similarity range by visual observation or the like, and it is also automated (automatically waveform) It is also difficult to determine a similar range and divide the section. Therefore, conventionally, the waveform itself of the non-periodic component (non-harmonic component) is analyzed and the vector quantization process is not performed independently. In this regard, according to the present invention, the waveform itself of the non-harmonic component (non-periodic component), which has not been easily performed in the past, can be analyzed and the vector quantization process can be easily performed independently. Of harmonic components period Since it is easy to determine automatically, it becomes easy to introduce | transduce into the division | segmentation process of a non-harmonic component using this. For example, a temporary divided section is set for each predetermined period of the harmonic component that is automatically determined, and then an appropriate divided section is set and confirmed by the user's manual operation (location synchronized with the period of the harmonic component) In this case, it is preferable to select to divide appropriately as well as to make adjustments by appropriately shifting the divided parts before and after that.
[0013]
In addition, when a non-periodic component (non-harmonic component) separated according to frequency analysis is displayed graphically and observed, the frequency analysis cannot be separated as a periodic component, but is similar to the period of the periodic component. It was found that there was a part that appeared to appear as a waveform change of a non-periodic component in the period. Therefore, the harmonic component period Dividing an out-of-harmonic component waveform into a plurality of sections using appropriate points synchronized with the period as a delimiter is appropriate for each non-harmonic component for each similar range during vector quantization of the out-of-harmonic component. It is quite useful for performing the process of dividing into two. Thus, the present invention makes it possible to easily perform compression processing of out-of-harmonic components, and is extremely useful for realizing vector quantization processing of out-of-harmonic components. It contributes to increasing the waveform compression efficiency, and further exhibits various excellent effects such as excellent reproducibility of the original waveform, particularly excellent reproducibility of out-of-harmonic components.
[0014]
In vector quantization of out-of-harmonic components, it is possible to perform vector quantization in an optimal form that matches the content of each out-of-harmonic component, taking into account the balance between promoting data compression and improving waveform reproducibility. preferable. The out-of-harmonic component basically corresponds to a random noise waveform that does not have repeatability.For example, in consideration of the importance and presence of the content and the position in the pronunciation period, It may be possible to reproduce a non-harmonic component waveform having a required time length by looping a specific representative vector (this is called loop reproduction), or a representative vector having a required time length is not looped. The required non-harmonic component waveform may be reproduced by using it only once (this is called one-shot reproduction). Alternatively, a required non-harmonic component waveform may be reproduced by switching and combining a plurality of representative vectors in a predetermined order (this is called sequence reproduction). Alternatively, the switching order may be set at random to emphasize the noise characteristics of the out-of-harmonic component, that is, randomness (this is called random sequence reproduction). Considering these various types of optimum vector quantization forms, it is preferable to appropriately use representative vectors suitable for various optimum vector quantization forms as out-of-harmonic vectors. Therefore, in vector quantization of an out-of-harmonic component, as an example, an out-of-harmonic vector or representative vector suitable for the out-of-harmonic component is selected, and how this out-of-harmonic vector or representative vector is used (for example, the loop reproduction described above). Or information such as a one-shot reproduction or the like, a period thereof, a form of an envelope attached thereto, and the like) can be added.
Each representative vector of out-of-harmonic components, that is, an out-of-harmonic vector, can be shared between various different waveforms. In other words, when there is appropriate commonality between two or more different original waveforms to be compressed, a part of the original waveforms has an appropriate commonality, and in the vector quantization, the same representative vector is appropriately set between the two waveforms. Can be shared.
[0015]
The waveform generation method according to the present invention includes a process of receiving compressed data each including instruction information indicating an out-of-harmonic vector to be used as an out-of-harmonic component for a plurality of sections each having a specific time length. Supplying waveform data of harmonic components to be generated, supplying a plurality of non-harmonic vectors, which are stored non-harmonic component waveform data, When compressing non-harmonic components Divided in synchronization with the period of the harmonic component waveform data , The selection according to the process of selecting the non-harmonic vector indicated by the instruction information in the compressed data for each section from the plurality of non-harmonic vectors, and the period of the waveform data of the harmonic component to be generated A process for controlling the expansion / contraction of the time length of the non-harmonic vector, and a process for synthesizing the waveform data of the non-harmonic component based on the non-harmonic vector subjected to the expansion / contraction control. Thereby, the non-harmonic vector indicated by the instruction information is selected from a plurality of non-harmonic vectors based on the compressed data for each section, and the waveform of the non-harmonic component is independently reproduced based on the non-harmonic vector.・ Can be generated. At this time, the time length expansion / contraction of the selected non-harmonic vector is controlled according to the period of the waveform data of the harmonic component to be generated at the same time, and the waveform data of the non-harmonic component based on the stretch-controlled out-of-harmonic vector Is synthesized. This is advantageous when vector quantization is performed by performing section division synchronized with the period of the corresponding harmonic component as described above when compressing the non-harmonic component. In that case, since the time length of the non-harmonic vector is related to the period of the harmonic component, if the time length of the non-harmonic vector is appropriately expanded and contracted according to the period of the harmonic component to be reproduced at the same time, It becomes easy to reproduce the temporal correspondence between the harmonic component waveform and the non-harmonic component waveform in the original waveform, and the reproducibility of the non-harmonic component waveform is improved. In this way, the waveform of the non-harmonic component can be uniquely reproduced and generated from the compressed data for the non-harmonic component. The original waveform can be reproduced by mixing the reproduced non-harmonic component waveform with the harmonic component waveform.
[0016]
The present invention can be configured and implemented not only as a method invention but also as a device invention. In addition, the present invention can be implemented in the form of a computer program or in the form of a recording medium storing such a computer program. Further, the present invention can be implemented in the form of a recording medium storing waveform data having a new compressed data structure.
[0017]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.
First, referring to FIG. 1, an outline of a hardware configuration example used to implement a waveform compression method and a waveform generation method according to an embodiment of the present invention will be described. The hardware configuration example shown here is configured using a general-purpose computer such as a personal computer, for example, in which the waveform compression and waveform generation processing is performed by the computer using the waveform compression method and waveform generation according to the present invention. This is implemented by executing a predetermined program (software) that realizes the method. Of course, the waveform compression method and the waveform generation method are not limited to the form of computer software, but can also be implemented in the form of a microprogram processed by a DSP (digital signal processor). However, the present invention may be implemented in the form of a dedicated hardware device configured to include a discrete circuit, an integrated circuit, a large-scale integrated circuit, or the like. Further, the device configuration is not limited to a general-purpose computer such as a personal computer, and may take any product application form such as an electronic musical instrument, a karaoke device, an electronic game device, or other multimedia devices.
[0018]
In the hardware configuration example shown in FIG. 1, a CPU (central processing unit) 10 as a main control unit of a computer has a ROM (read only memory) 11, a RAM (random access memory) 12, a hard disk device 13, a removable device. A disk device (for example, a CD-ROM drive or an MO drive) 14, a display 15, an input operation device 16 such as a keyboard and a mouse, a waveform interface 17, a timer 18, an interface 19 such as a MIDI and a communication network, and the like are included in a CPU bus 20. Connected through. The waveform interface 17 samples an analog waveform signal (audio signal) input from the outside via a microphone or the like according to an instruction from the CPU 10 according to a predetermined sampling frequency and converts it into a digital signal, which is converted into a CPU bus. The digital waveform data generated based on the compressed data by the function of sending to the computer 20 and the waveform generation processing executed by this computer is received via the CPU bus 20, converted into an analog waveform signal according to a predetermined sampling frequency, and output to a speaker system or the like It has a function to do.
[0019]
The digital waveform data input via the waveform interface 17 is temporarily written in a predetermined storage area in the RAM 12 or the hard disk device 13, and the predetermined waveform compression processing according to the present invention is performed on the waveform data in accordance with instructions from the CPU. Applied. The compressed data is stored in an appropriate compressed waveform database. In this case, not only the analog waveform signal but also digital waveform data may be input via an appropriate interface 19 or the like, and the input waveform data may be subjected to required data compression. The function of the waveform database may be handled by any type of data storage device. That is, any of the RAM 12, the hard disk device 13, and the removable disk device 14 may function as a compressed waveform database. In general, an appropriate storage area in the hard disk device 13 which is a large-capacity storage device or a removable recording medium such as a CD-ROM or MO that can be attached to and detached from the removable disk device 14 functions as a compressed waveform database. Good. Alternatively, a waveform database provided in an external host or server computer is accessed via the interface 19 and a communication line, and compressed data is written therein, or compressed data necessary for reproduction is stored in the hard disk device 13. Or you may make it download to RAM12 grade | etc.,. At the time of waveform generation, predetermined waveform generation processing is performed using compressed data stored in the compressed waveform database to generate digital waveform data. The generated digital waveform data may be output as analog via the waveform interface 17 as described above, or may be transferred and output as digital data via an appropriate interface 19 or the like. In the case where an external memory such as the hard disk device 13, the removable disk device 14, or an external host or server computer is used as the compressed waveform database, an internal memory is used to enable quick access to the designated waveform data. It is preferable to transfer all or part of the compressed waveform database to the transfer buffer (configured by the RAM 12 or the like).
[0020]
The display unit 15 displays various graphic screens in response to instructions from the CPU 10 during the waveform compression process and the waveform generation process. For example, a graphic display of the shape of the recorded waveform, a graphic display of the waveform editing status during waveform compression processing, and a control screen for setting and selecting various data such as timbre settings, timbre editing, and system settings It is used in various ways such as displaying. Further, necessary control information, character information, and the like are input in the course of the waveform compression process and the waveform generation process using the alphanumeric keyboard and mouse of the input operation device 16. Also, a MIDI keyboard module, a sequencer (automatic performance device) or another computer is connected via the interface 19 to input performance instruction information for playing back musical sounds and to exchange various data.
[0021]
A software program for executing the waveform compression processing and / or waveform generation processing according to the present invention under the control of the CPU 10 may be stored in any of the ROM 11, the RAM 12, the hard disk device 13, and the like. The program may be recorded on a removable recording medium such as a CD-ROM or MO that can be attached to and detached from the removable disk device 14, or from an external host or server computer via a communication line and interface 19. The program may be received and downloaded to the hard disk device 13 or the RAM 12 or the like. In implementing the present invention, the computer system as shown in FIG. 1 does not necessarily have both functions of the waveform compression process and the waveform generation process, and only one of them can be implemented. Such a configuration may be adopted.
[0022]
Next, examples of various processes executed under the control of the CPU 10 will be described. FIG. 2 schematically shows a waveform compression processing procedure executed under the control of the CPU 10 by an equivalent block diagram.
First, in “Waveform recording with rendition” (Process P1), various natural musical instruments are played with various performances by the skilled performer, with the same pitch and intensity, and each performance sound. Are sampled via the waveform interface 17, thus recording a large number of waveform data with performance styles in a predetermined storage area in the RAM 12 or the hard disk device 13. In this case, the recorded performance sound is not limited to a single sound, but may be a series of phrases or chords, or a plurality of different performance methods in a single performance sound consisting of a single sound or a series of phrases. May be included.
[0023]
Even if the general performance method name such as “Vibrato” or “Slur” is the same, the waveform characteristics of the performance sound are different for each instrument type, and the general performance method name is the same and the instrument type is the same. Even if they are the same, the waveform characteristics of each performance sound differ depending on the degree of performance (for example, the difference in vibrato depth). Therefore, the performance sound according to various performance methods recorded in “waveform recording with performance method” (process P1) takes into account the differences in performance methods according to these various viewpoints. Therefore, the “performance sound with performance style” is mainly distinguished and described using a combination of three elements of “musical instrument name”, “performance style name”, and “parameter” indicating the level of the performance style. be able to. In that case, the “parameter” indicating the degree of performance includes a plurality of parameters.
[0024]
An example is as follows.
Example 1: In “attack” (performance style name) of “violin” (instrument name), “rising speed, strength, pitch” and the like are parameters.
Example 2: In “sla” (performance name) of “violin” (instrument name), its “slur width, slur speed, strength, pitch” and the like are parameters.
Example 3: In “vibrato” (performance name) of “violin” (instrument name), the “vibrato depth, vibrato speed, strength, pitch” and the like are parameters.
Example 4: In “picking” (performance method name) of “guitar” (instrument name), the “strength, pitch” and the like are parameters.
Example 5: In “hammering on” (performance method name) of “guitar” (instrument name), the “strength, pitch” and the like are parameters.
Example 6: In “bend” (playing style name) of “guitar” (instrument name), “bend width, bend speed, strength, pitch” and the like are parameters.
Example 7: “Operation width, operation speed, strength, pitch” and the like are parameters in “tremolo arm operation” (performance method name) of “guitar” (instrument name).
Further, the parameter may include a parameter indicating a time change mode, such as an intensity time change curve or a slur time change curve.
[0025]
Next, in the “performance method analysis” (process P2), each waveform data prepared in the process P1 of the “recording with attached performance” is analyzed, and a common performance method is used in the waveform data of one or a series of performance sounds. The range corresponding to the part (which will be referred to as a rendition style section) is extracted, and the waveform data is divided into sections for each or each of the rendition styles included in the waveform data. The position is determined, and the “performance style name” and the “parameter” for describing the performance style used in each divided section (performance style section) are determined, respectively. Performance designation data consisting of “parameter” is generated. Also, musical instrument designation data indicating the “musical instrument name” for the performance style is generated. In this way, the definition of dividing the waveform data of one or a series of performance sounds into one or more performance style sections according to one or more performance styles used therein and the type of performance style for each performance style section are described. The three elements of “instrument name”, “performance style name”, and “parameter” are defined by instrument designation data and performance style designation data. These instrument designation data and rendition style designation data are used as commentary information or index information of compressed data for the rendition style section compressed by the processes P3 to P6 for waveform data compression.
[0026]
For example, the following three methods are conceivable as analysis methods in this “playing style analysis” (process P2).
Example 1: The user listens to the sound of the waveform data to be analyzed, and displays the waveform on the display unit 15 as necessary, and determines the position of the division to be divided as a rendition section according to the user's judgment. Use to specify manually. In addition, the “performance method name” and “parameter” for each performance method section specified for division are determined by the user's judgment, and are manually specified using the input operation device 16. This corresponds to manual analysis.
Example 2: The CPU 10 executes a predetermined rendition style analysis program, analyzes changes in various musical tone characteristics such as pitch and amplitude of the waveform data to be analyzed, and determines the position of the break to be divided as a rendition style section based on the analysis result It is automatically determined, and the “performance style name” and “parameter” for each determined performance style section are automatically determined. This corresponds to a fully automatic analysis.
Example 3: A combination of the techniques of Examples 1 and 2 above. That is, rendition style analysis is performed by combining manual analysis by the user and automatic analysis by the CPU. For example, division of performance style sections and determination of “performance style names” are performed manually by the user, and “parameters” in each performance style section are determined by automatic analysis by the CPU. Alternatively, the division position of each performance style section or “performance style name” or “parameter” determined by automatic analysis by the CPU is appropriately corrected by a user's manual operation.
[0027]
Next, in “Performance section division” (process P3), the waveform data of one or a series of performance sounds prepared in the “recording with attached performance” process P1 is processed for the waveform data. A specific division process is performed in accordance with the rendition style section division position designation information determined in P2. The subsequent processes P4 to P6 are performed on the waveform data for each divided rendition style section.
[0028]
Next, in the “harmonic / non-harmonic component separation” (process P4), the waveform data of one rendition style section divided by the process P3 of the “replay style section division” is extracted from the memory such as the RAM 12, and the waveform data is harmonized. The process which isolate | separates into a component and a nonharmonic component is performed. An example of this separation process is as follows. First, waveform data to be separated is input, and frequency analysis (spectrum analysis) is performed by fast Fourier transform (FFT) to extract the fundamental pitch frequency component and the harmonic component. The harmonic component waveform is generated by performing inverse Fourier transform on this. By subtracting the harmonic component waveform generated in this way from the input waveform data to be separated, an out-of-harmonic component waveform is obtained as the residual waveform. As an example, in the above FFT processing, a window function having a length corresponding to 8 cycles of waveform data to be analyzed is used, and the waveform data is shifted while shifting the window function by a frame corresponding to 1/8 cycle. It is good to analyze the whole. When such FFT analysis is adopted, it is possible to analyze the fluctuation of the frequency of the harmonic component as a fluctuation, and extract it by including it in the harmonic component. Of course, the size of the window function and the shift amount thereof are not limited to the above example, and can be arbitrarily set, and the setting can detect the fluctuation of the frequency and level of the frequency component analyzed by the FFT processing as the fluctuation. It only has to be.
[0029]
Next, in the “harmonic component analysis” (process P5), the characteristics of the waveform data of the harmonic components separated and generated in the process P4 of the “harmonic / non-harmonic component separation” are analyzed, and the optimal vector quantum is analyzed according to the characteristics. To generate compressed data of the harmonic component waveform compressed according to vector quantization. Here, since the harmonic component can be basically regarded as a repetitive waveform, for example, using a waveform of one period or a plurality of periods as a representative vector, the representative vector is looped (or two representative vectors preceding and following are represented). The harmonic component waveform can be reproduced by performing a crossfade loop. Therefore, such a representative vector is stored in the harmonic vector storage unit M1 as a harmonic vector, and the compressed vector data stored in the harmonic vector storage unit M1 is generated as compressed data generated by the “harmonic component analysis” (process P5). Among them, vector information that specifies a harmonic vector to be used as a representative vector of the harmonic component, and how to use the specified harmonic vector, that is, a representative vector (for example, a section in which the vector is used, a period of looping, or the like) It is preferable to generate information including information describing the form of the envelope to be given). A detailed example of this “harmonic component analysis” (process P5) will be described later.
[0030]
On the other hand, in the “non-harmonic component analysis” (process P6), the characteristics of the waveform data of the non-harmonic components separated and generated in the process P4 of the “harmonic / non-harmonic component separation” are analyzed, and the optimum vector is analyzed according to this feature. Quantization processing is performed, and compressed data of the non-harmonic component waveform compressed according to vector quantization is generated. Here, the non-harmonic component is subjected to vector quantization in an optimum form suitable for the characteristics or contents of each non-harmonic component in consideration of the balance between acceleration of data compression and improvement of waveform reproducibility. ing. The out-of-harmonic component basically corresponds to a random noise waveform that does not have repeatability.For example, in consideration of the importance and presence of the content and the position in the pronunciation period, It may be possible to reproduce a non-harmonic component waveform having a required time length by looping a specific representative vector (this is called loop reproduction), or a representative vector having a required time length is not looped. The required non-harmonic component waveform may be reproduced by using it only once (this is called one-shot reproduction). Alternatively, a required non-harmonic component waveform may be reproduced by switching and combining a plurality of representative vectors in a predetermined order (this is called sequence reproduction). Alternatively, the switching order may be set at random to emphasize the noise characteristics of the out-of-harmonic component, that is, randomness (this is called random sequence reproduction). Considering these various types of optimum vector quantization forms, it is preferable to appropriately use representative vectors suitable for various optimum vector quantization forms as out-of-harmonic vectors. Therefore, such various representative vectors are stored in the non-harmonic vector storage unit M2 as non-harmonic vectors, and the non-harmonic vector storage unit M2 is used as the compressed data generated in this “non-harmonic component analysis” (process P6). Vector information specifying an out-of-harmonic vector suitable for use as a representative vector of the out-of-harmonic component, and how to use the out-of-harmonic vector, that is, the representative vector (for example, using the vector) It is preferable to generate information including information that describes a section to be played, a distinction between the above-described loop reproduction and one-shot reproduction, a period thereof, a form of an envelope attached thereto, and the like. A detailed example of this “non-harmonic component analysis” (process P6) will be further described later.
[0031]
In FIG. 2, the rendition style information storage unit M3 includes the compressed data (first compressed data) generated in the “harmonic component analysis” (process P5) and the “non-harmonic component analysis” ( The compressed data (second compressed data) generated in the process P6) is combined with the musical instrument designation data and the rendition style designation data generated in the “representative style analysis” (process P3) for the rendition style section. It is stored as information. Therefore, for waveform reproduction, the musical instrument designation data and the performance style designation data for the desired performance style are used as indexes, and the harmonic component and the out-of-harmonic component for the corresponding performance style section (that is, the first and the non-harmonic components) from the performance style information storage unit M3. The second) compressed data can be invoked. Then, in accordance with the vector information included in the called compressed data, predetermined harmonic vectors and non-harmonic vectors are read from the harmonic vector storage unit M1 and the non-harmonic vector storage unit M2, respectively, and based on these, the harmonic component waveform and By reproducing each of the non-harmonic component waveforms and adding them together, it is possible to reproduce the original waveform or generate a musical sound waveform having the characteristics of the original waveform.
[0032]
Next, a detailed example of “harmonic component analysis” (process P5) will be described with reference to FIG. Note that the processing shown in FIG. 3 does not show all of the “harmonic component analysis” (process P5), but mainly generates a new harmonic vector and stores it in the harmonic vector storage unit M1. The part related to the process of accumulating and storing is shown.
[0033]
First, in the “waveform segment division” process S1, the harmonic component waveforms for one rendition style segment separated and generated in the “harmonic / non-harmonic component separation” of the process P4 are divided into a plurality of waveform segments. As a method of dividing the waveform section, a section composed of continuous one or a plurality of period waveforms having the same or similar characteristics of the waveform shape for each period is extracted as one waveform section and divided by the section. Therefore, the size of each waveform section to be divided is generally unequal. The “waveform segment division” process S1 may be such that the user designates an appropriate division position by manual operation while observing the waveform shape displayed on the display 15, or the CPU 10 determines a predetermined waveform. The division position may be automatically determined based on executing an interval division program and automatically analyzing the frequency characteristics of the waveform for each period. Alternatively, this may be performed by combining manual operation and automatic processing. As a result of the “waveform segmentation” process S1, “section information” is generated as information describing how the input harmonic component waveform is divided into a plurality of waveform segments. In order to efficiently determine whether the characteristics of the waveform shape for each cycle are common or similar, a normalization process is performed to cut out the waveform for each cycle and share the pitch and amplitude level. It is preferable to be able to compare only the waveform shapes of one period waveform. In that case, since the subsequent vector quantization processing is performed on the waveform data in which the pitch and amplitude level are standardized, the temporal change state of the original pitch and amplitude level with respect to the standardized pitch and amplitude level It is preferable to generate a “pitch vector” and an “amplitude vector” as vectors indicating Here, it may be preferable to normalize by controlling the time axis position of the waveform data. In this case, a “time vector” is generated as a vector indicating a temporal change state of the time axis position of the waveform data.
[0034]
Next, in the “grouping” process S <b> 2, the divided waveform data of each waveform section is grouped so that sections having similar waveform shapes for one period or a plurality of periods are made into the same group. This grouping is not only performed between the waveform sections of the performance style section of its own, but rather between the waveform data of each waveform section of other performance style sections (including different musical instruments). It is what. That is, in the group waveform storage unit M4, waveform data of a large number of waveform sections for each rendition style section already grouped is stored in a grouped state. In the “grouping” process S2, with reference to the stored contents of the group waveform storage unit M4, the waveform data for each period or a plurality of periods are similar in the waveform data of each waveform section divided this time. It is checked whether or not an existing group exists in the group waveform storage unit M4. If it exists, the waveform data of the waveform section is included in the group. That is, the waveform data of the waveform section is stored in the storage area of the group in the group waveform storage unit M4, and “group information” indicating the group is generated. If there is no waveform similar to the existing group, the waveform data of the waveform section is registered as a new group. That is, the waveform data of the waveform section is stored in the storage area of the new group in the group waveform storage unit M4, and “group information” indicating the new group is generated. In this case, in order to increase data sharing efficiency, appropriate waveform normalization processing is performed so that the pitch and volume level of the original waveform are the same as described above, and grouping is performed by purely comparing only the waveform shapes. Good. Of course, regardless of the size of each waveform section, the waveform shapes of one period or a plurality of periods that characterize the waveform section are compared. Thus, even if waveform data of different musical instruments, different performances, or different pitches are used, if the waveform shape of a certain waveform section is similar, the waveform data of the similar waveform section Will be classified into a common group. This is because the harmonic vector in the harmonic vector storage unit M1 (the same applies to the non-harmonic vector in the non-harmonic vector storage unit M2) is shared among various performance style waveforms having different characteristics such as different instrument sounds or different performance styles or different pitches. This means that it is useful for reducing the storage scale of the harmonic vector storage unit M1 (or the non-harmonic vector storage unit M2). The “grouping” process S2 may also be performed by a user's manual operation as described above, or may be performed by an automatic process by the CPU 10, or a combination of manual operation and automatic process. You may go.
[0035]
Next, in the “representative vector generation of each group” process S3, the stored contents of the group waveform storage unit M4 are referred to, and the representative vectors of those waveform data are generated in consideration of the characteristics of each waveform data in the same group. To do. For example, if one waveform or a plurality of periodic waveforms similar in one waveform section are repeated, for the time being, it is selected as a temporary representative vector, and the average of the temporary representative vectors of the respective waveform data within the same group or The intermediate or most characteristic vector is determined as the representative vector of the group and is generated. The generated representative vector (specifically, waveform data including one or a plurality of cycles) is stored in the harmonic vector storage unit M1 and registered as a harmonic vector. In addition, “vector information” for the harmonic component that indicates which harmonic vector stored in the harmonic vector storage unit M1 corresponds to the representative vector of the group is generated. Since this “vector information” is index data that indicates which harmonic vector (specifically, waveform data consisting of one or a plurality of cycles) stored in the harmonic vector storage unit M1 is used as a representative vector. It is a small amount of data consisting of a plurality of bits.
[0036]
This “vector information” is stored in the rendition style information storage unit M3 as compressed data of harmonic components in the waveform section in combination with the “section information” generated from the “waveform section division” process S1. Accordingly, the compressed data of the harmonic component related to one rendition style section (first compressed data) is a combination of “vector information” and “section information” for each of a plurality of waveform sections obtained by dividing the rendition style section. Is included. In this case, when the vector quantization processing is performed on the waveform data in which the pitch and the amplitude level are normalized as described above, the “waveform segment division” processing S1 is performed on the “vector information” and the “section information”. The generated “pitch vector” and “amplitude vector” are further combined and stored in the rendition style information storage unit M3. Further, when the vector quantization processing is performed on the waveform data in which the time axis position of the waveform data is also normalized, the “time vector” generated from the “waveform segmentation” processing S1 is further combined to store the rendition style information. Store in part M3.
[0037]
Next, a processing example of “harmonic component analysis” (process P5) in a state where a relatively large number of harmonic vectors are accumulated in the harmonic vector storage unit M1 will be described. In this case, mainly, a harmonic vector that can be used as a representative vector of the harmonic component waveform is searched from many harmonic vectors already stored in the harmonic vector storage unit M1, and the searched harmonic vector is searched for the harmonic component. It consists of selecting as a representative vector.
[0038]
For example, first, a waveform is cut out for each period from the harmonic component waveform for one rendition style section separated and generated by the “harmonic / non-harmonic component separation” of the process P4, and the pitch and the amplitude level are made common. The normalization process is performed, and further, the time axis position of the waveform data is controlled and standardized as necessary. In this way, the normalized waveform data is compared with many harmonic vectors already stored in the harmonic vector storage unit M1 for each cycle or for each plurality of cycles, and similar harmonic vectors are searched. The search in this case also includes not only harmonic vectors corresponding to the same natural instrument and / or the same rendition style as the harmonic component waveform to be analyzed but also harmonic vectors corresponding to different natural instruments and / or different rendition styles. Do. When a similar harmonic vector is found in this way, a section consisting of continuous one or a plurality of periodic waveforms similar to the harmonic vector is extracted as one waveform section to generate “section information” and the searched harmonic vector Is generated as a representative vector of the “waveform section” and stored as a set in the rendition style information storage unit M3. In this case, as described above, when the vector quantization processing is performed on the waveform data in which the pitch and the amplitude level are normalized, the temporal change state of the original pitch and amplitude level with respect to the normalized pitch and amplitude level is shown. The “pitch vector” and the “amplitude vector” are further combined with the “vector information” and the “section information” and stored in the rendition style information storage unit M3. In addition, when vector quantization processing is performed on waveform data that also standardizes the time axis position of the waveform data, a “time vector” that indicates a temporal change state of the time axis position of the waveform data is further combined to perform the performance method. It memorize | stores in the information storage part M3.
[0039]
Here, as a result of the search, if similar harmonic vectors do not exist in the harmonic vector storage unit M1, the same processing as the processing S1, S2, and S3 in FIG. Grouping is performed, a new representative vector is generated, and this is registered in the harmonic vector storage unit M1 as a new harmonic vector. Then, “vector information” indicating this new harmonic vector is generated and stored in the rendition style information storage unit M3 together with “section information” and the like.
[0040]
FIG. 4 is a graph showing an example of how the harmonic component waveform is vector quantized. (A) schematically shows an example of an original waveform by an amplitude envelope. (B) shows an example in which the harmonic component waveform separated from the original waveform is divided into a plurality of “waveform sections” by a diagram showing the division of each waveform section. (C) shows a state in which appropriate harmonic vectors W0, W1, W2,... W7 made up of waveform data of one or a plurality of periods are selected as representative vectors in each “waveform section”, and at the time of waveform reproduction. An example of loop playback is shown. For example, for the waveform section from time t0 to t1, the harmonic vector W0 is used as a representative vector, and during reproduction, this is loop-reproduced between time t0 and t1. As loop reproduction, an example is shown in which crossfade loop reproduction is performed with the representative vector of the next section (in this case, the waveform at the head of the harmonic vector W1). For the waveform section from the time point t1 to the time point t2, the harmonic vector W1 having a relatively large size (consisting of a multi-period waveform) is used as a representative vector, and at the time of reproduction, this is a predetermined period starting from the time point t1 to the time point t2. Is reproduced once and loop reproduction is appropriately performed in the remaining period (for example, crossfade loop reproduction is performed between the waveform at the end of the harmonic vector W1 and the harmonic vector W2 in the next section). In addition, for the waveform section from time t2 to t3, the harmonic vector W2 is used as a representative vector, and during reproduction, this is loop-reproduced between the time t2 and t3 (the harmonic vector W2 and the harmonic vector W3 Crossfade loop playback between). Hereinafter, the same applies, and in the waveform section after the last time t7, the harmonic vector W8 is used as a representative vector, and during reproduction, this is reproduced in a simple loop. (D), (e), and (f) of FIG. 4 show examples of “time vector”, “amplitude vector”, and “pitch vector” extracted from the harmonic component waveform of (a), respectively.
[0041]
Next, a detailed example of “non-harmonic component analysis” (process P6) will be described. The processing procedure of the “non-harmonic component analysis” (process P6) may be basically the same as the procedure of the “harmonic component analysis” (process P5) shown in FIG. However, since the object of analysis is the non-harmonic component waveform for one rendition style section separated and generated by the “harmony / non-harmonic component separation” of the process P4, the “waveform segment division” process (S1) In addition, in the “grouping” process (S2) and the “representative vector generation for each group” process (S3), a concept such as one period or a plurality of periods of the non-harmonic component waveform itself to be analyzed is not applied. Of course. In addition, when the non-harmonic vector storage unit M2 is used instead of the harmonic vector storage unit M1 and the representative vector of the non-harmonic component waveform is newly generated in the “representative vector generation of each group” process (S3), this is new. It goes without saying that the non-harmonic vector is stored and registered in the non-harmonic vector storage unit M2.
[0042]
Also in the “non-harmonic component analysis”, the “waveform segment division” process similar to the “waveform segment division” process S1 of FIG. 3 is performed, and the vector quantization process is performed on each divided waveform segment. In that case, the method of dividing the non-harmonic component waveform into a plurality of waveform sections is generally divided into a plurality of waveform sections for each part whose characteristics are common or similar or inferred as such. However, in this embodiment, it is proposed that one or both of the following two methods be performed in combination.
[0043]
The first technique uses the periodicity of the harmonic component for the same rendition style section, and divides the non-harmonic component waveform into a plurality of waveform sections in a period that is an integral multiple or a fraction multiple of the harmonic component period. Is. Of course, also in this case, the divided waveform sections do not have to be equally spaced, one waveform section has a length corresponding to two cycles of the harmonic component, and another waveform section has a length corresponding to 10 cycles of the harmonic component. It may be different as appropriate. As described above, the method of performing the segmentation of the non-harmonic component waveform depending on the periodicity of the harmonic component can perform the segmentation of the non-harmonic component waveform without much labor, so that the analysis work can be performed. There is an advantage that it is easy. In addition, when the characteristics of the out-of-harmonic component waveform depend to some extent on the periodicity of the harmonic component, appropriate waveform section division can be easily performed. Although the section division of the non-harmonic component waveform is performed depending on the periodicity of the harmonic component, the waveform section division form of the non-harmonic component is naturally different from the waveform section division form of the corresponding harmonic component. It goes without saying that the harmonic component and the non-harmonic component are each divided into sections independently.
[0044]
The second method is to specifically search each part where the characteristics of the non-harmonic component waveform are common or similar, and to divide the part into a plurality of waveform sections. This method is suitable for a case where a relatively important part in the out-of-harmonic component waveform is extracted and a representative vector corresponding to the extracted portion, that is, an out-of-harmonic vector is selected and generated. Conversely, when extracting less important parts of out-of-harmonic component waveforms and omitting those less important parts as appropriate to select and generate representative vectors, that is, out-of-harmonic vectors, to increase data compression efficiency Also suitable for. In addition, as will be described later, even when data compression is promoted by performing loop reproduction processing of non-harmonic vectors, it is suitable because a section suitable for the loop can be appropriately selected. In both cases of the first method and the second method, it may be performed by a user's manual operation, may be performed by automatic processing by the CPU 10, or a combination of manual operation and automatic processing. You may go.
[0045]
In this embodiment, depending on whether the waveform section division of the non-harmonic component waveform is performed according to the first method or the second method, the contents of the selection and generation processing of the representative vector for the waveform section thereafter are somewhat. It has become different. This point will be described in more detail with reference to a flowchart.
FIG. 5 shows the “out-of-harmonic component analysis” (process P6) in the case where the waveform section division of the out-of-harmonic component waveform is performed according to the first method and the data compression (vector quantization) processing of the out-of-harmonic component is performed. A detailed example is shown. FIG. 6 shows “out-of-harmonic component analysis” (process P6) in the case of performing waveform segmentation of the out-of-harmonic component waveform according to the second method and then performing data compression (vector quantization) processing of the out-of-harmonic component. ) Shows a detailed example.
[0046]
First, the embodiment of FIG. 5 will be described.
In the flow shown in FIG. 5A, first, an out-of-harmonic component waveform to be compressed is prepared (step S10).
Next, according to the period length of the harmonic component corresponding to the non-harmonic component waveform, that is, in synchronization with the periodicity of the harmonic component waveform, the non-harmonic component waveform is divided into a plurality of waveform sections (step S11). . Here, as described above, the non-harmonic component waveform is divided into a plurality of waveform sections in a period that is an integral multiple or a fraction multiple of the period of the harmonic component, that is, in synchronization with the periodicity of the waveform of the harmonic component. . Each divided waveform section does not need to be equally spaced, one waveform section has a length corresponding to ½ period of the harmonic component, another waveform section has a length corresponding to eight periods of the harmonic component, and so on. As appropriate.
[0047]
Next, the position of each waveform section divided in the previous step is adjusted as appropriate (step S12). For example, in the previous step S11, the harmonic components are divided by a uniform reference such as every period, and in this step S12, the user confirms the division result in the previous step S11 with the display unit 15. On the other hand, the length of a certain waveform section is adjusted to increase or decrease by the number of cycles of the harmonic component, or the division position is slightly changed so that the phase of the non-harmonic waveform at the part to be divided becomes a well-separated phase (for example, zero cross phase). The correction which shifts is done. The adjustment or correction processing in this step is not limited to manual operation by the user, but may be performed by automatic processing according to a predetermined correction program, or may be performed by a combination of manual operation and automatic processing. .
[0048]
Next, the characteristics of the non-harmonic waveform in each divided waveform section are analyzed, and grouping is performed (step S13). Similar to the “grouping” process S2 of FIG. 3, this grouping is also performed between the waveform data of each waveform section of the non-harmonic waveform for other performance style sections (including different musical instruments), and the representative vector, that is, the harmonic The sharing of outside vectors shall be promoted. FIG. 5B shows a more detailed flowchart example of the non-harmonic waveform characteristic analysis and grouping processing executed in step S13.
[0049]
5B, in the “peak value analysis” (step S30), it is analyzed where the peak level of the non-harmonic waveform (that is, the noise waveform) is in the divided waveform section. Next, in the “presence location analysis” (step S31), the location where the peak level of the non-harmonic waveform exists in the divided waveform section is the period of the harmonic component corresponding to the waveform section or a predetermined multiple period section. Which part (for example, the first half part, the middle part, or the second half part) corresponds to the analysis. In the next “zero cross frequency analysis” (step S32), the number of times that the amplitude of the non-harmonic waveform is zero crossed in the divided waveform section is analyzed. The intensity of noise-like change can be analyzed by the number of zero crossings. In the next step S33, the waveform section of the non-harmonic waveform is classified into a required group based on the results analyzed in steps S30 to S32. Since the waveform to be grouped is a non-harmonic waveform having no periodicity by itself, a special analysis such as steps S30 to S32 is performed, and the analysis result is used as a condition or reference for grouping. It is devised so that a quantitative evaluation for grouping can be performed. Conditions or evaluation criteria for grouping this type of non-harmonic waveform are not limited to those shown in steps S30 to S32. For example, the level of the DC offset level in the waveform section, the waveform in the waveform section, and the like. Other appropriate conditions or evaluation criteria such as the number of sign inversions of the differential value (which also serves as an index indicating the intensity of noise-like change) may be used, and these conditions or evaluation criteria and the above-described steps S30 to S32 are used. Any number of conditions or evaluation criteria may be used in combination among the three conditions or evaluation criteria. The processing of each step in FIG. 6 may be performed in an interactive manner in accordance with a user's manual operation. Of course, it may be fully automatic processing according to a predetermined program, or may be performed by a combination of manual operation and automatic processing.
[0050]
Returning to FIG. 5 (a), in step S14, whether the processing in steps S11 to S13, that is, the waveform segmentation and its grouping processing, have been completed for all non-harmonic component waveforms to be subjected to data compression processing in the current processing routine. Check. If not completed yet, the process returns to step S11, and the processes of S11 to S13 are performed for the next non-harmonic component waveform. Thus, when the waveform section division and the grouping process are completed for all the non-harmonic component waveforms, the process goes to step S15 to analyze the waveform characteristics of each group. For example, the amplitude envelope characteristic of each waveform in the group is analyzed. In the next step S16, the representative vector of the group is determined using the analysis result in the previous step S15. For example, the waveform having the average amplitude envelope characteristic of the amplitude envelope characteristic of each waveform in the group may be determined as the representative vector. Alternatively, the average frequency characteristic of each waveform in the group, the number of zero crossings, the level distribution, or the like may be determined as a representative vector. Alternatively, a representative vector is selected or determined on the condition that it is average or suitable for being representative of any one or more of amplitude envelope characteristics, frequency characteristics, number of zero crossings, level distribution, and other appropriate characteristics. You may make it do. Then, the determined representative vector of the group is stored as a non-harmonic vector in the non-harmonic vector storage unit M2. Thus, the representative vectors of each group are stored and registered in the non-harmonic vector storage unit M2 as non-harmonic vectors. The out-of-harmonic vector stored in the out-of-harmonic vector storage unit M2 uses a standardized amplitude level, and extracts the amplitude envelope of the out-of-harmonic component waveform of each waveform section as an “amplitude vector”. And Alternatively, the out-of-harmonic vector stored in the out-of-harmonic vector storage unit M2 does not have to normalize the amplitude level. In this case, the low level data is stored at the low level, but in the low level part, the memory storage location can be efficiently utilized by performing limited storage bit allocation. When the amplitude level of the non-harmonic vector stored in the non-harmonic vector storage unit M2 is not normalized, the difference between the amplitude envelope of the non-harmonic component waveform and the amplitude envelope of the non-harmonic vector in each waveform section is defined as an “amplitude vector”. It is good to extract.
[0051]
In the next step S17, rendition style information of each non-harmonic component (including "section information" of each waveform section, "vector information" designating a non-harmonic vector as a representative vector, "amplitude vector", etc.) It memorize | stores in the information storage part M3. In this case, information such as the number of periods of harmonic components corresponding to the non-harmonic vector, that is, the representative vector, is also stored as “section information”. In this case, a common non-harmonic vector can be used for harmonic components having different pitches if the waveform shape itself of the non-harmonic component has similarity or commonality. Of course, the out-of-harmonic vector stored in the out-of-harmonic vector storage unit M2 can be shared in various cases in the same manner as or more than the case of the harmonic vector. For example, a common non-harmonic vector can be used in different waveform sections in the same rendition section (the same non-harmonic waveform with different time axis positions), and different instrument sounds or different rendition section sections can be used. A common non-harmonic vector can also be used in the non-harmonic waveform.
[0052]
Next, the embodiment of FIG. 6 will be described.
First, an out-of-harmonic component waveform to be compressed is prepared (step S20).
Next, in step S21, the characteristics of each part of the non-harmonic component waveform to be processed are observed and analyzed, and divided at locations that match the partial characteristics, and the non-harmonic component waveform is divided into a plurality of waveform sections. To divide. In this case, as a partial feature of the non-harmonic component waveform, analysis is performed in consideration of the position of the waveform, rendition style, characteristics of the waveform shape, etc., and the waveform section to be divided is determined. For example, “attack” (rising part), “body” (steady part of sound), “joint” (part connected to other sounds), “release” (sound attenuation part), etc. are considered as positions. Is done. In terms of relevance to each position, for example, “attack” in “attack”, “with noise” performance, etc. are considered, and “body” in “vibrato” performance, “tremolo” In the “joint”, the “slur” playing method, the “grisand” playing method, the “bend” playing method, etc. are considered, and in the “release”, the “mute” playing method, the “sustain” playing method, etc. are considered. The waveform shape characteristics include amplitude envelope shape (its effective value and peak value, etc.), the number of zero crossings or the period between zero crossings, the number of peaks or the period between peaks, the location of the peak value, and multiple peaks. Various factors such as level distribution may be considered as appropriate.
[0053]
Next, in step S22, waveforms having similar characteristics are grouped based on the characteristics of the non-harmonic waveform in each divided waveform section. The similarity of the characteristics of the out-of-harmonic waveform may be performed on an appropriate basis, such as visual observation of the waveform displayed on the display, or determining the similarity based on audibility by generating a non-harmonic waveform for the waveform section. Similar to the above, this grouping is also performed between the waveform data of each waveform section of the non-harmonic waveform for other performance style sections (including different musical instruments), and the common use of the representative vector, that is, the non-harmonic vector is promoted. Shall.
[0054]
Next, in step S23, it is checked whether or not the processing in steps S21 to S22, that is, the waveform segmentation and its grouping processing, have been completed for all non-harmonic component waveforms to be subjected to data compression processing in the current processing routine. If not completed yet, the process returns to step S21, and the processes of S21 to S22 are performed on the next non-harmonic component waveform. Thus, when the waveform section division and the grouping process are completed for all the non-harmonic component waveforms, the process goes to step S24 to analyze the waveform characteristics of each group. For example, the amplitude envelope characteristic of each waveform in the group is analyzed, and other various characteristics are considered.
[0055]
In the next step S25, the representative vector of the group is determined using the analysis result in the previous step S24. The determined representative vector of the group is stored as a non-harmonic vector in the non-harmonic vector storage unit M2. Thus, the representative vectors of each group are stored and registered in the non-harmonic vector storage unit M2 as non-harmonic vectors. As described above, the out-of-harmonic vector stored in the out-of-harmonic vector storage unit M2 may be a normalized amplitude level or may not be normalized. Here, in this embodiment, it is considered that various different methods as described below are appropriately adopted as a method for compressing the non-harmonic component waveform, and the representative vector considers which compression method is used. It shall be decided according to the standard according to it.
[0056]
Compression example 1: A representative vector having a length of a predetermined period is reproduced once so as to cover the waveform section. This is a compression format corresponding to the “one-shot playback”. For example, a representative vector for a waveform section of a part where the waveform shape of an out-of-harmonic component such as an attack part or a joint part also changes abruptly may be determined so as to meet the compression example 1.
Compression example 2: Compression is performed so as to cover the waveform section by repeatedly reproducing (loop reproduction) a representative vector having a relatively short period length. This is a compression format corresponding to the “loop reproduction”. Note that a plurality of loop representative vectors can be used in the “sequence playback” by combining a plurality of representative vectors in a predetermined order. Further, by setting this sequential combination at random, it can be used for the “random sequence reproduction”. For example, the representative vector for the waveform section of the monotonously decreasing portion or the continuous waveform portion of the same state may be determined so as to meet this compression example 2.
Compression example 3: A part of the non-harmonic component waveform of a less important part may be omitted. For such omitted waveform sections, there is no need to select a representative vector.
[0057]
Next, in step S26, for each waveform section of each non-harmonic component waveform, at the time of reproduction, which non-harmonic vector is used as each representative vector and how the non-harmonic vector should be pasted is determined. To do. This is also related to which type of compression example is used. The pasting method is described as non-harmonic component reproduction mode information (or “pasting information”) indicating which of the following various reproduction modes should be adopted.
[0058]
One-shot reproduction: As described above, one representative vector according to the compression example 1 is used, and this representative vector is pasted (reproduced) to reproduce the non-harmonic component waveform in the waveform section.
Sequence playback: As described above, a plurality of representative vectors according to the compression example 2 are used, and these representative vectors are sequentially pasted (reproduced) in a predetermined order to reproduce an out-of-harmonic component waveform in the waveform section.
Random sequence reproduction: As described above, a plurality of representative vectors according to the compression example 2 are used, and these representative vectors are sequentially pasted (reproduced) in a random order to reproduce the non-harmonic component waveform in the waveform section.
Loop reproduction: As described above, one representative vector according to the compression example 2 is used, and this is repeatedly pasted (reproduced) to reproduce the non-harmonic component waveform of the waveform section. This may be combined with the above sequence playback or random sequence playback.
Alternative loop reproduction: One representative vector according to the compression example 2 is used, and this is repeatedly pasted (reproduced) while changing the forward / reverse direction alternately to reproduce the non-harmonic component waveform of the waveform section. This may be combined with the above sequence playback or random sequence playback.
Delete playback: According to the compression example 3 as described above, the representative vector is not used and the non-harmonic component waveform in the waveform section is deleted. Therefore, only the harmonic component waveform is present in that portion. Note that not all of one waveform section may be in the delete playback mode, but only a part thereof may be in the delete playback mode. In that case, representative vectors are appropriately determined for the remaining portions.
[0059]
In the next step S27, rendition style information of each non-harmonic component ("section information" of each waveform section, "vector information" designating a non-harmonic vector as a representative vector, the above-mentioned "pasting information", and "amplitude vector" Are stored in the rendition style information storage unit M3.
The various out-of-harmonic component regeneration modes described above are not applied only when the “out-of-harmonic component analysis” process according to the second method is performed, but the “out-of-harmonic component analysis” according to the first method. This is also applied as appropriate when the process is performed.
[0060]
FIG. 7 is a graph showing an example of a state in which the non-harmonic component waveform is vector quantized. FIG. 4A schematically shows an example of the original waveform with an amplitude envelope as in FIG. (B) shows an example of an out-of-harmonic component waveform separated from the original waveform. (C) shows an example in which the non-harmonic component waveform of (b) is divided into a plurality of “waveform sections” A1, B1, C1, D1, C2, A2, and C3 by a diagram showing the division of each waveform section. ing. Here, the waveform sections A1 and A2 are sections in which the one-shot reproduction is applied by setting the waveform section according to the second method. The waveform section B1 is a section in which the representative vector is determined in synchronization with the period of the harmonic component according to the first method. In this section, one-shot playback may be employed, or loop playback may be employed. Waveform sections C1, C2, and C3 are sections corresponding to the loop reproduction. The waveform section D1 is a section corresponding to the delete reproduction.
[0061]
Further, specific examples of vector quantization of the non-harmonic component waveform are shown in FIGS.
8 to 10 and 13, (a) in the upper stage shows an example of the original waveform of the non-harmonic component waveform, and (b) in the lower stage is obtained by performing vector quantization according to the above-described various methods and reproducing it. The example of the waveform obtained is shown.
First, FIG. 8 shows that the original waveform is divided into “attack”, “body”, and “release” sections, and the “attack” section and the “release” section are representative of the sections corresponding to the one-shot playback. A vector is determined, and the “body” section shows a compression example in which the non-harmonic component waveform is cut as a section corresponding to the delete reproduction. When actually listening to the playback sound based on this, if the non-harmonic component waveform has an “attack” section and a “release” section, even if the steady timbre stable part (“body” section) is omitted, it is very uncomfortable. I can understand that there is no.
[0062]
FIG. 9 shows a compression example in which the non-harmonic component waveform is divided into four sections F1 to F4, a representative vector having a predetermined size is determined in each section, and these are loop reproduced. Even if it is an out-of-harmonic component, the periodicity appears somewhat by looping the representative vector. However, since the level is low, mixing with the reproduced waveform of the harmonic component makes it uncomfortable.
In FIG. 10, the non-harmonic component waveform is divided into three sections F1, F2, and F3, the original waveform is used as a representative vector as it is in the first section F1 rich in change, and a predetermined size is used in the next sections F2 and F3. An example of compression in which representative vectors are determined, these are reproduced in a loop, and the amplitude envelope is changed is shown. By adding an appropriate amplitude envelope to the loop reproduction waveform in this way, a feeling similar to the original waveform can be obtained.
[0063]
FIG. 11 shows an example of level control in loop reproduction. (A) shows a simple loop reproduction example, and level control is not performed. (B) shows an example in which the peak level of the representative vector to be looped is attenuated to weaken the periodicity (loop correction example with peak correction). (C) shows an example in which a change in amplitude envelope is further added to the reproduction example of (b) to further weaken the periodicity. Thus, it is effective to combine level control with loop reproduction.
FIG. 12A shows an example of normal loop reproduction, and FIG. 12B shows an example of alternative loop reproduction. As can be understood from the figure, the periodicity of the alternative loop reproduction is longer than the periodicity of the normal loop reproduction, so that the size of the representative vector may be further reduced. Therefore, it contributes to further data compression.
[0064]
13 divides the non-harmonic component waveform into eight sections F1 to F8. The attack section F1, the sections F3, F5 and F7 including vibrato, and the release section F8 are representative vectors (non-harmonic vectors) extracted from the original waveform. ) Waveform reproduction is performed using f1, f3, f5, f7, and f8. For the other sections F2, F4, and F6, the non-harmonic vectors f2 ′, f4 ′, and f6 ′ extracted from the other original waveforms are The compression example which performs waveform reproduction | regeneration using it as a representative vector is shown. In each section, an appropriate playback mode such as one-shot playback or loop playback is adopted. For example, with respect to the body section F2, the non-harmonic vector f2 ′ extracted from another original waveform is loop-reproduced as a representative vector, and an amplitude envelope is appropriately added to thereby obtain the original waveform (see FIG. 13A). ). For the other sections F4 and F6, the amplitude waveform is appropriately given to the non-harmonic vectors f4 ′ and f6 ′ extracted from the other original waveforms, and a loop or the like is performed as necessary, so that the original waveforms (FIG. )) Is reproduced and generated. Thus, a common non-harmonic vector can be used for a plurality of different original waveforms. In that case, by controlling various processing methods during playback (including data describing the appropriate playback method such as amplitude vector and pasting information in the compressed data, etc.), it is optimal for each original waveform It is possible to reproduce and generate a non-harmonic component waveform.
[0065]
Next, a description will be given of an embodiment in which musical tone generation processing is performed using compressed data of harmonic components and non-harmonic components created as described above.
FIG. 14 shows a musical score with renditions in order to make full use of the compressed data of the harmonic and non-harmonic components of a large number of rendition style waveforms created as described above when playing an arbitrary musical piece. This schematically shows an example of a processing process. First, musical score information (for example, performance data comprising a MIDI form) is prepared for a desired musical piece for which a musical score with performance style is to be created (step S40). Next, for each performance part of the prepared musical score information, it is analyzed what kind of performance method should be used in the course of each song (step S41). This rendition style analysis may be performed by a person reading a score and making a judgment based on the musical knowledge. Of course, the rendition style analysis may be executed by the CPU 10 according to a predetermined score analysis program, or may be performed by a combination of manual operation and automatic processing.
[0066]
Next, for each performance data of the prepared musical score information, for each performance data, it corresponds to a necessary performance point corresponding to each analyzed performance method in the time-series flow of the performance data. At the place to be played, performance style data indicating each performance style analyzed in step S41 is added (step S42). For example, performance data instructing a required vibrato performance method is added to a location corresponding to the performance point of a note or phrase to which vibrato is applied. Alternatively, rendition style data that indicates a required slur performance style is added to a point corresponding to the time point of performance of a note or phrase to be played with a slur. The location where the required performance data is added is a location corresponding to one note (for example, the same location as a note-on event), or a location corresponding to the middle of one note (for example, a predetermined time after the note-on event of the note). , A rendition event is inserted at a position corresponding to an appropriate point in time before the note-off event of the sound, or a position corresponding to one phrase consisting of a plurality of notes (for example, an on event of a predetermined rendition at the beginning of the phrase) , And an off event of the rendition style is inserted at the end of the phrase). In addition, the content of performance style data to be added includes “performance style name” data indicating the name of various performance styles such as vibrato, slur, staccato, glide, pitch bend, etc., and “parameter” data indicating the level of the performance style. In addition, “range” data indicating a period or range in which the performance style is given is included. The “range” data is data that specifically indicates the rendition style assignment range, such as giving the rendition style to only the corresponding one sound, giving the rendition style to the two preceding and following sounds. When a rendition style is given over a long range, the range may be specified by a rendition style on event and an off event as described above. As described above, the performance data to which performance style data is added is stored in a suitable memory as musical score information with performance style (step S43).
[0067]
FIG. 15 is a functional block diagram schematically showing an example of a process for performing musical tone reproduction, that is, waveform generation, based on musical score information with performance style prepared as described above. This musical tone reproduction, that is, waveform generation processing, is performed by executing a predetermined software program by the CPU 10 using the hardware device of FIG. In the “reproduction control” process S50, as is known in normal automatic performance sequence control, performance data is sequentially read in accordance with a predetermined tempo clock, and a musical sound waveform having a pitch and a tone color indicated by the performance data is generated. . Here, what is different from normal automatic performance sequence control is that the performance data to be played is composed of “musical score information with performance style” created as described above, and is included therein. A characteristic rendition style waveform reproduction / generation process according to rendition style data is performed.
[0068]
The “reproduction control” process S50 will be further described. First, performance data (for example, MIDI data) of each performance part and performance data added thereto are stored in accordance with the performance timing from the “music score information with performance method” storage unit M5. Sequentially read and read. When performance data is read for a certain performance part of the “music score with performance”, a process of generating a performance waveform corresponding to the performance data is performed. First, according to the “musical instrument name” (timbre) specified for the performance part and the “performance method name” and “parameter” data included in the performance data, the performance information storage unit M3 uses these pieces of information. “Performance information” for a specified rendition style waveform, that is, compressed data (first and second compressed data) for the harmonic component and the non-harmonic component is read. As described above, the compressed data (first compressed data) on the harmonic component includes “vector information” indicating a harmonic vector, “section information”, “pitch vector”, “amplitude vector”, and “time vector”. , “Information for controlling the cross-fade loop reproduction mode” and the like. The compressed data (second compressed data) for the non-harmonic component includes “vector information” indicating the non-harmonic vector, “section information”, “pasting information” indicating the reproduction mode, and “amplitude vector”. ”,“ Time vector ”and the like.
[0069]
In the “harmonic read” process S51, the harmonic vector is read from the harmonic vector storage unit M1 based on the “vector information” in the compressed data (first compressed data) regarding the harmonic component.
In the “harmonic processing” process S52, the harmonic vector read from the harmonic vector storage unit M1 and other information of the compressed data (section information, pitch vector, amplitude vector, time vector, information for controlling the crossfade loop reproduction mode, etc.) Based on the above, the required harmonic component waveform is reproduced and generated. That is, the harmonic vector waveform is appropriately looped (repeated) according to the section information, and the pitch is temporally controlled according to the pitch vector (of course, the basic pitch is set and controlled according to the note data included in the MIDI performance data) The amplitude envelope is set and controlled according to the amplitude vector, and the time axis position of the waveform data is expanded or compressed according to the time vector, and the crossfade loop reproduction mode is controlled Amplitude envelope control for crossfade is performed with a fade-in or fade-out characteristic.
[0070]
In the “out-of-harmonic reading” process S53, out-of-harmonic vectors are read out from the out-of-harmonic vector storage unit M2 based on “vector information” in the compressed data (second compressed data) for the out-of-harmonic component.
In the “out-of-harmonic processing” process S54, the required harmonization is based on the out-of-harmonic vector read from the out-of-harmonic vector storage unit M2 and other information (section information, pasting information, amplitude vector, time vector, etc.) of the compressed data. Play and generate external component waveforms. That is, according to the instructions of the pasting information and section information, according to the predetermined non-harmonic component reproduction mode such as the one-shot reproduction, loop reproduction, sequence reproduction, etc., the waveform of the non-harmonic vector is appropriately looped or only once. Paste to the required section. The amplitude envelope is set and controlled according to the amplitude vector, and the time axis position of the non-harmonic component waveform data is expanded or compressed according to the time vector.
[0071]
In addition, when the compressed data of the non-harmonic component to be reproduced is in accordance with the first method (the vector quantization is performed by performing segment division synchronized with the periodicity of the corresponding harmonic component), “vector information” The time length of the non-harmonic vector is appropriately expanded or contracted according to the period of the harmonic component to be reproduced at the same time this time or after reading the non-harmonic vector from the non-harmonic vector storage unit M2 And In consideration of the compression method of the compressed data (the first method), it is more effective that the time length of the non-harmonic vector is expanded and contracted in synchronization with the expansion and contraction of the period of the harmonic component to be reproduced. This is because the reproducibility of the waveform is improved. The non-harmonic vector time length expansion / contraction control may be as simple as changing the readout rate of the non-harmonic vector storage unit M2 in conjunction with the change in the readout cycle (reading rate) of the corresponding harmonic vector. The time axis expansion / contraction control may be employed. Of course, if the compressed data of the out-of-harmonic component to be reproduced is one according to the second method (one obtained by performing segmentation regardless of the periodicity of the harmonic component and vector quantization), basically this is the case. No special consideration is required, but it may be done. In addition, the various playback modes such as “One-shot playback”, “Loop playback”, “Alternative loop playback”, “Sequence playback”, “Random sequence playback”, and “Delete playback” described above are not compatible. It goes without saying that the present invention can also be applied to the case where the compressed data of the component is the one according to the first method (the vector divided by performing the interval division synchronized with the periodicity of the corresponding harmonic component). It is.
[0072]
In the “mixing” process S55, the harmonic waveform and the non-harmonic component waveform reproduced and generated as described above are mixed, and musical tone waveform data obtained by mixing both components is generated. In this way, the musical sound waveform data is reproduced and provided (step S56). The reproduced musical sound waveform data may be supplied to other utilization devices as digital data, or may be converted to analog and spatially sounded through an appropriate sound system. Note that the harmonic component waveform signal and the non-harmonic component waveform signal may be generated separately and mixed in space.
As described above, it is possible to reproduce and generate a musical sound waveform having characteristics that are fairly faithful to the original waveform while adopting a compressed data structure. In addition, by variably controlling the waveform processing in each of the processing steps S52 and S54, a highly useful waveform generation process that is highly controllable while being a high-quality waveform faithful to the original waveform is performed. can do.
[0073]
【The invention's effect】
As described above, according to the present invention, when performing vector quantization processing of an out-of-harmonic component, the out-of-harmonic component waveform is divided into a plurality of sections depending on the periodicity of the corresponding harmonic component, and a vector is generated for each section. Since quantization is performed, it is possible to easily perform segment division of non-harmonic component waveforms without much human effort, so that waveform analysis work for segment division can be performed easily. Excellent effect. In addition, in vector quantization of the non-periodic waveform component, the vector quantization process is performed based on the inherent non-harmonic component that forms the original waveform together with the harmonic component, so that the reproduction accuracy of the non-periodic waveform component is improved. Thus, the present invention makes it possible to easily perform compression processing of out-of-harmonic components, and is extremely useful for realizing vector quantization processing of out-of-harmonic components. It contributes to increasing the waveform compression efficiency, and further exhibits various excellent effects such as excellent reproducibility of the original waveform, particularly excellent reproducibility of out-of-harmonic components.
[Brief description of the drawings]
FIG. 1 is a block diagram showing an outline of a hardware configuration example used to implement a waveform compression method and a waveform generation method according to an embodiment of the present invention.
FIG. 2 is an equivalent block diagram schematically showing a procedure of waveform compression processing executed under the control of a CPU in FIG.
3 is an equivalent block diagram showing a detailed example of “harmonic component analysis” processing in FIG. 2; FIG.
FIG. 4 is a graph showing an example of how a harmonic component waveform is vector quantized.
FIG. 5 is a flowchart showing an example of the “non-harmonic component analysis” process in FIG. 2, wherein the non-harmonic component waveform is divided into a plurality of waveform sections using the periodicity of the harmonic component for analysis. The example which made was shown.
6 is a flowchart showing another example of the “non-harmonic component analysis” process in FIG. 2, and divides the non-harmonic component waveform into a plurality of waveform sections based on the characteristics of the non-harmonic component waveform and performs analysis. The example which made it do.
FIG. 7 is a graph showing an example of a state where vector quantization of an out-of-harmonic component waveform is performed.
FIG. 8 is a graph showing a specific example of vector quantization of an out-of-harmonic component waveform.
FIG. 9 is a graph showing another specific example of vector quantization of an out-of-harmonic component waveform.
FIG. 10 is a graph showing still another specific example of vector quantization of an out-of-harmonic component waveform.
FIG. 11 is a graph showing another specific example of vector quantization of an out-of-harmonic component waveform.
FIG. 12 is a graph showing still another specific example of vector quantization of an out-of-harmonic component waveform.
FIG. 13 is a graph showing another specific example of vector quantization of an out-of-harmonic component waveform.
FIG. 14 is an equivalent block diagram schematically showing an example of a process for preparing a musical score with performance style.
FIG. 15 is an equivalent block diagram schematically showing the procedure of musical tone reproduction, that is, waveform generation processing executed under the control of the CPU in FIG. 1;
[Explanation of symbols]
10 CPU (Central Processing Unit)
11 ROM (Read Only Memory)
12 RAM (Random Access Memory)
13 Hard disk devices
14. Removable disk device (eg CD-ROM drive or MO drive)
15 Display
16 Input operation devices such as keyboard and mouse
17 Waveform interface
18 Timer
19 Interfaces such as MIDI and communication networks
20 CPU bus

Claims (2)

Supplying a plurality of non-harmonic vectors that are stored non-harmonic component waveform data;
Supplying a non-harmonic component of waveform data to be compressed and a corresponding harmonic component ;
Detecting the period of the harmonic component;
Dividing the non-harmonic component of the waveform data to be compressed into a plurality of sections based on the detected period of the harmonic component ;
For each of the divided sections , an out-of- harmonic vector that can be used as an out- of- harmonic component is selected from the plurality of out-of-harmonic vectors, each of the out-of-harmonic components is vector quantized, and the selected out-of-harmonic vector is A waveform compression method comprising: generating compressed data for each non-harmonic component including instruction information to be indicated . A process of receiving compressed data each including instruction information indicating an out-of-harmonic vector to be used as an out-of-harmonic component for a plurality of sections each having a specific time length;
Supplying waveform data of harmonic components to be generated simultaneously;
Supplying a plurality of non-harmonic vectors that are stored non-harmonic component waveform data;
The non- harmonic vectors indicated by the instruction information in the compressed data for each section , divided in synchronization with the period of the waveform data of the harmonic components during compression of the non- harmonic components, are respectively selected from the plurality of non-harmonic vectors. The process of choosing,
Controlling the expansion and contraction of the time length of the selected non-harmonic vector according to the period of the waveform data of the harmonic component to be generated;
A waveform generating method comprising: synthesizing waveform data of out-of-harmonic components based on the out-of-harmonic vector subjected to expansion and contraction control.

Images