JP3674527B2 - Waveform generation method and apparatus - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a method and an apparatus for generating a waveform of a musical sound or a voice or any other sound based on reading of waveform data from a waveform memory or the like, and in particular, various performance methods unique to a natural instrument performed by a player or The present invention relates to a device that generates a waveform that faithfully represents a timbre change caused by articulation. The present invention is not limited to electronic musical instruments, but also in devices, apparatuses or methods in all fields having a function of generating musical sounds, voices, or other arbitrary sounds, such as automatic performance apparatuses, computers, electronic game apparatuses and other multimedia devices. It can be applied in a wide range. In this specification, the term “musical sound waveform” is not limited to a musical sound waveform, but is used in a sense that may include a sound waveform or any other sound waveform.
[0002]
[Prior art]
In the waveform memory, waveform data (that is, waveform sample data) encoded by an arbitrary encoding method such as PCM (pulse code modulation), DPCM (differential PCM), or ADPCM (adaptive differential PCM) is stored. The so-called “waveform memory readout” technology that forms a musical sound waveform by reading out in accordance with a desired music pitch is already known, and various types of “waveform memory readout method” technologies are known. ing. Most of the conventionally known “waveform memory reading method” techniques are for generating a single sound waveform from the beginning to the end of sound generation. As an example, there is a method of storing waveform data of all waveforms of one sound from the start to the end of sound generation. As another example, there is a method of storing waveform data of all waveforms for an attack portion having a complicated change, and storing a predetermined loop waveform for a sustain portion having little change. In this specification, “loop waveform” is used to mean a waveform that is repeatedly read (loop read).
[0003]
[Problems to be solved by the invention]
By the way, in the conventional “waveform memory reading method” technique of storing waveform data of all waveforms of one sound from the start to the end of sound generation, or storing waveform data of all waveforms in a part of the waveform such as an attack part. Therefore, a large number of various waveform data corresponding to various performance methods (or articulations) must be stored, and a large storage capacity is required to store the large number of waveform data.
In addition, the above-described method for storing waveform data of all waveforms can faithfully express timbre changes by various performance methods (or articulations) unique to natural instruments. Cannot be reproduced, so the controllability is poor and the editability is poor. For example, it has been very difficult to perform characteristic control such as time axis control according to performance data on waveform data corresponding to a desired performance (or articulation).
[0004]
The present invention has been made in view of the above points, and a waveform generation method capable of easily and easily generating high-quality waveform data corresponding to various performance methods (or articulations) with a high degree of controllability and In particular, it is intended to provide a device, and in particular, by making it possible to control the generation start timing of a rendition style waveform with respect to the timing of a performance event such as a sound generation start event or a mute start event, a more realistic rendition style waveform It is intended to make it possible to synthesize a performance style waveform rich in controllability.
[0005]
[Means for Solving the Problems]
The waveform generation method according to the present invention includes, in a rendition style module describing a rendition style waveform, position information indicating a position to be timed with a predetermined performance event in the middle of the time axis of the rendition style waveform. Designating a rendition style module for a waveform together with a performance event; determining a control timing of a predetermined musical sound characteristic in the rendition style module according to the position information in the designated rendition style module and a designated performance event; based on the rendition style waveform of the rendition style module in which the control timing is determined, and a step of synthesizing a tone corresponding to the rendition style module.
[0006]
According to the present invention, in the rendition style module describing a rendition style waveform, position information indicating a position to be timed with a predetermined performance event is included in the middle of the time axis of the rendition style waveform. The timing of the musical sound characteristic control to be performed can be set in the middle of the time axis of the performance style waveform, and a more realistic performance style waveform can be synthesized. For example, the performance waveform generation start timing can be controlled to appropriately precede the timing of a performance event such as a sound generation start (note-on) event or a mute start (note-off) event.
[0007]
For example, the position information of the note-on timing indicated by reference numeral 2 in FIG. 3 corresponds to the position information included in the rendition style module. In FIG. 3, reference numeral 1 indicates a sample length (waveform section length) of a waveform corresponding to a rendition style module having a rendition style characteristic of an attack portion. The position of the note-on timing indicated by reference numeral 2 indicates which of the rendition style modules. The time position can also be set variably. Note-on timing position 2 is associated with the sound generation start timing indicated by the performance event (note-on event). Therefore, generation of a rendition style waveform is started prior to the sound generation start timing. For example, depending on the playing technique such as a bend attack, the waveform component rise start time may precede the note-on timing. Also, with the violin, the strings are rubbed with the bow before the sound actually comes out. Therefore, according to the present invention, it is suitable for accurately simulating the start of the performance waveform before the start of sound generation or for freely controlling it. Including the position information of the position of the performance event in the middle of the time axis of the performance style waveform is applicable not only to the note-on event but also to the note-off event as shown in FIG.
[0008]
The embodiment described below will be briefly described. A rendition style module is specified by rendition style identification information. The performance style identification information is represented by ID information indicating the performance style characteristics of the performance sound. For example, the performance style identification information corresponds to a plurality of performance modes (that is, “performance style” or “articulation”), and a plurality of performance styles (performance styles). It specifies which of the performance modes (playing styles) is applicable. For example, rendition style identification information may be given corresponding to partial sections of sound, such as attack, body, or release, depending on the rendition characteristics of the performance sound. It may be given corresponding to the section (joint part) of the song, it may be given corresponding to the special performance part of the sound like vibrato, and it is given corresponding to multiple notes like a phrase. Sometimes. Depending on the performance to be reproduced, required performance style identification information (hereinafter also referred to as performance style ID) is given. According to the present invention, when the rendition style identification information is received, a plurality of vector data for generating a waveform indicating the rendition style characteristics is generated according to the rendition style identification information. For example, the vector data corresponds to different types of basic elements for generating the waveform. Such basic elements include, for example, a waveform shape (waveform shape for setting a tone color or timbre), a temporal change in pitch, or a temporal change in amplitude. It is called a vector and an amplitude vector. Further, a time vector indicating the progress of the waveform time axis may be included. With this time vector, time axes such as a waveform vector, a pitch vector, and an amplitude vector can be controlled.
[0009]
Then, by arranging these vector data on the time axis, a waveform or envelope corresponding to each element of the waveform can be constructed along the reproduction time axis of the performance sound. In this way, a waveform of a performance sound is generated based on each vector data arranged on the time axis. For example, by giving the waveform vector a pitch according to the pitch vector and its time change characteristic, and by giving an amplitude according to the amplitude vector and its time change characteristic, the rendition characteristics according to the rendition style identification information The performance sound waveform shown can be generated.
[0010]
The present invention can be constructed and implemented not only as a method invention but also as an apparatus invention. Further, the present invention can be implemented in the form of a program of a processor such as a computer or a DSP, or can be implemented in the form of a storage medium storing such a program.
[0011]
DETAILED DESCRIPTION OF THE INVENTION
Embodiments of the present invention will be described below in detail with reference to the accompanying drawings.
[0012]
FIG. 1 is a block diagram showing a hardware configuration example of a waveform generating apparatus according to the present invention. The hardware configuration example shown here is configured by using a computer, and in the waveform generation process, the computer executes a predetermined program (software) for realizing the waveform generation process according to the present invention. Is implemented. Of course, this waveform generation processing is not limited to the form of computer software, but can be implemented in the form of a microprogram processed by a DSP (digital signal processor), and is not limited to this form of program. You may implement in the form of the dedicated hardware apparatus comprised including the discrete circuit or the integrated circuit or the large-scale integrated circuit. The waveform generation device may take any product application form such as an electronic musical instrument, a karaoke device, an electronic game device, another multimedia device, or a personal computer.
[0013]
In the hardware configuration example shown in FIG. 1, a read only memory (ROM) 102, a random access memory (through a bus line BL (data or address bus, etc.), a random access memory (a RAM) 103, panel switch 104, panel display 105, drive 106, waveform capture unit 107, waveform output unit 108, hard disk 109, and communication interface 111 are connected to each other. The CPU 101 executes processing such as “waveform database creation” and “musical tone synthesis (software sound source) based on the produced database”, which will be described later, based on a predetermined program. These programs are supplied from a network via the communication interface 111 or an external storage medium 106 A such as a CD or MO attached to the drive 106 and stored in the hard disk 109. Then, it is loaded from the hard disk 109 to the RAM 103 at the time of execution. Alternatively, the program may be recorded in the ROM 102. The ROM 102 stores various programs executed by or referred to by the CPU 101, various data, and the like. The ROM 103 is used as a working memory that temporarily stores various information related to performance and various data generated when the CPU 101 executes the program, or as a memory that stores a program currently being executed and related data. A predetermined address area of the RAM 103 is assigned to each function and used as a register, flag, table, memory, or the like. The panel switch 104 is configured to include various operators for performing an instruction to sample a musical tone, editing sampled waveform data, and inputting various information. For example, a numeric keypad for inputting numeric data, a keyboard for inputting character data, or a panel switch. In addition to these, various operators for selecting, setting, and controlling the pitch, timbre, effect, and the like may be included. The panel display 105 is a display such as a liquid crystal display panel (LCD) or a CRT that displays various information input by the panel switch 104, sampled waveform data, and the like.
[0014]
The waveform capturing unit 107 includes an A / D converter, converts (samples) an analog musical tone signal input from an external waveform (for example, input from a microphone) into digital data, and stores the digital waveform in the RAM 103 or the hard disk 109. Data is taken in as original waveform data (waveform data that is the material of the waveform data to be generated). In the “waveform database creation” process executed by the CPU 101, a “waveform database” according to the present invention is created based on the captured original waveform data. In the “musical tone synthesis based on database” process executed by the CPU 101, waveform data of an arbitrary musical tone signal corresponding to performance information is generated using the “waveform database”. Of course, a plurality of musical sound signals can be generated simultaneously. The waveform data of the generated musical sound signal is given to the waveform output unit 108 via the bus line BL, and stored in a buffer as appropriate. The waveform output unit 108 outputs the waveform data stored in the buffer in accordance with a predetermined output sampling frequency, D / A converts this, and sends it to the sound system 108A. Thus, the musical tone signal output from the waveform output unit 108 is sounded via the sound system 108A. The hard disk 109 stores a plurality of types of performance-related data such as waveform data and data for synthesizing waveforms according to performance styles (data such as performance style tables and codebooks described later), timbre data composed of various timbre parameters, and the like. It stores data, and stores data related to control of various programs executed by the CPU 101.
[0015]
The drive 106 has a plurality of types relating to performance such as waveform data and data for synthesizing waveforms in accordance with performance styles (various data such as performance style tables and codebooks to be described later), timbre data composed of various timbre parameters, etc. And a removable disk (external storage medium 106A) for storing data relating to the control of various programs executed by the CPU 101 and the like. The external storage medium 106A driven by the drive 106 is an abbreviation of a compact disk (CD-ROM / CD-RAM), a magneto-optical disk (MO), or a DVD (Digital Versatile Disk) in addition to a floppy disk (FD). ) And other removable storage media in various forms. The external storage medium 106 </ b> A storing the control program may be set in the drive 106, and the contents (control program) may be directly loaded into the RAM 103 without being dropped on the hard disk 109. Note that the method of providing the control program using the external storage medium 106A or via the network is advantageous because the control program can be easily added or upgraded.
[0016]
The communication interface 111 is connected to a communication network (not shown) such as a LAN, the Internet, or a telephone line, and is connected to a server computer or the like (not shown) via the communication network. The control program, various data, performance information, and the like are taken into the waveform generation device side. That is, when the control program and various data are not stored in the ROM 102 and the hard disk 109, it is used for downloading the control program and various data from the server computer. The waveform generation device serving as a client transmits a command requesting download of a control program and various data to the server computer via the communication interface 111. Upon receiving this command, the server computer stores the requested control program and data in the hard disk 109 via the communication interface 111, thereby completing the download. Further, of course, a MIDI interface may be included to receive MIDI performance information. Needless to say, a performance keyboard may be connected to the bus line BL and performance information may be supplied by real-time performance. Of course, the performance information may be supplied by using the external storage medium 106A that stores the performance information of a desired music piece.
[0017]
FIG. 2 is a flowchart showing an example of “waveform database creation processing” executed in the waveform generation apparatus described above. This process is a process for creating vector data using a waveform of a performance sound played by various performance methods (or articulations) as a material in order to cope with various performance methods (or articulations).
In step S1, a database for storing a performance style table and a code book, which will be described later, is prepared. As a medium serving as this database, for example, a hard disk 109 is used. Then, waveform data according to various performance modes of various natural musical instruments is collected (step S2). That is, various actual performance sounds of various natural instruments are captured from an external waveform input (for example, a microphone or the like) via the waveform capture unit 107, and waveform data (original waveform data) of these performance sounds is stored in the hard disk 109. Store in a predetermined area. The waveform data of the performance sound to be captured at this time may be waveform data of the entire performance, or only a certain phrase, one sound, or only a part of waveform data of a characteristic performance such as an attack part or a release part. Also good. Next, the waveform data of the performance sound according to various performance modes unique to the natural musical instrument obtained in this way are divided into characteristic portions, and are tuned and named (step S3). That is, the captured original waveform data is separated for each part of the waveform (for example, the attack part waveform, the body part waveform, the release part waveform, the joint part waveform, etc.) that represents the change in the waveform shape (1). Then, it is determined what pitch each of the separated waveform data of one period or plural periods is (2) tuning, and a file name is assigned to each separated waveform data (3) File naming. However, when waveform data of a part of performance such as an attack portion or a release portion is taken in, such waveform separation (1) separation can be omitted.
Next, component separation by frequency analysis is performed (step S4). That is, part of the waveform data separated and generated in step S3 is subjected to FFT (Fast Fourier Transform) analysis and separated into a plurality of components (in this embodiment, separated into harmonic components and non-harmonic components), and each component ( Performs feature extraction, that is, feature separation for each element of waveform, pitch, and amplitude from harmonic components, non-harmonic components, etc. (However, when separating into harmonic components and non-harmonic components, the non-harmonic components do not have a pitch.) Therefore, it is not necessary to perform pitch separation for non-harmonic components). For example, a “waveform” (Timbre) element is obtained by extracting features only from a waveform shape with normalized pitch and amplitude. The “Pitch” element is a pitch variation characteristic with respect to the reference pitch. The “Amplitude” element is an amplitude envelope characteristic extracted.
[0018]
In step S5, vector data is created. In other words, multiple sample values are continuously distributed as needed for each element of the waveform (Timbre), pitch (Pitch), and amplitude (Amplitude) of each separated component (harmonic component, non-harmonic component, etc.) Are extracted and given different vector IDs (identification information) to the sample value sequences, and stored in the code book together with the data of the time positions of the sample values (hereinafter, such sample data is referred to as vector data). Called). In this embodiment, vector data of the harmonic component waveform (Timbre) element, pitch (Pitch) element vector data, amplitude (Amplitude) element vector data, non-harmonic component waveform (Timbre) element vector data, amplitude (Amplitude) element vector data is created. Thus, the vector data for each of these component elements is data that can change as the time axis progresses. Next, the rendition style module data (details will be described later) are created, and the rendition style module is stored in the rendition style table. The rendition style module and vector data created in this way are written in the rendition style table and code book in the database (step S6), and data accumulation in the database is attempted. As described above, the vector data is data obtained by separating the waveform representing the shape of the imported original waveform for each element, not the original waveform data taken as it is. This is the unit of data. In this way, the code book stores a part of the waveform data representing the extracted waveform shape change in a compressed form. On the other hand, in the rendition style table, the rendition style module data (that is, various data necessary to return the vector data stored in the compressed form to the waveform data of the original waveform shape, and the vector stored in the code book) ID data for designating data) is stored (details will be described later).
[0019]
In the above feature separation (see step S4), feature extraction is performed using time as an element in addition to amplitude, pitch, and waveform elements (hereinafter, the extracted time element vector data is referred to as “time vector data”). Call). For this time element, the time length of the original waveform data in the time interval of a part of the waveform data that is separated and generated is used as it is. Therefore, if the original time length (variable value) of the time interval is indicated by the ratio “1,” it is not necessary to analyze and measure this time length at the time of the “waveform database creation process”. In this case, since the data on the time element (that is, “time vector data”) has the same value “1” in any time interval, it may not be stored in the codebook. Of course, the present invention is not limited to this, and it is possible to implement a modification in which the actual time length is analyzed / measured and stored in the code book as “time vector data”.
[0020]
Then, it is determined whether or not the database has been sufficiently created (step S7). That is, whether or not sufficient performance data and vector data of various performance modules have been obtained by sufficiently collecting original waveform data of performance sounds in various performance modes of various natural instruments obtained from external waveform input. judge. This determination is not limited to automatic determination, but may be performed in accordance with an instruction to determine whether or not to continue processing based on a switch input operation by the user. If it is determined that the collection of the original waveform data and the generation of vector data based thereon have been sufficiently performed (YES in step S7), the processing is terminated. Subsequently, when collecting original waveform data and creating vector data based thereon (NO in step S7), the process returns to step S2, and the above-described processes (steps S2 to S7) are repeatedly executed. The determination of “whether or not the vector data has been sufficiently generated” (step S7) may be performed by “trying to generate a musical sound by actually using the generated vector data”. That is, in step S7, once it is determined that “vector data has been sufficiently created” (YES in step S7) and the flow shown in FIG. 2 is exited, “play the musical sound using the created vector data. Then, since it was not satisfactory, the process of step S2 and the subsequent steps may be performed again to add vector data. That is, the process of “creating vector data and adding it to the database” is performed as needed.
In the above-described “waveform database creation process”, rendition style modules may be arbitrarily added or deleted, or rendition style module data may be edited.
[0021]
Here, the data of the rendition style module will be specifically described.
The rendition style modules are stored in a rendition style table formed as a database on the hard disk 109, and one rendition style module can be specified by a combination of "replay style ID" and "replay style parameters". “Performance ID” includes instrument information and module part names. For example, “playing style ID” is defined as follows. For example, if one “playing style ID” is expressed by a 32-bit (0th to 31st bit) string, 6 bits of them are used to express the musical instrument information. For example, if the 6-bit string is “000000”, the instrument information indicates AltoSax (alto sax), and if “001000”, the instrument information indicates Violin (violin). For the instrument information, the upper 3 bit string of the 6-bit string may be used for the major classification of the instrument type, and the lower 3 bit string may be used for the minor classification of the instrument type. Further, the module part name is expressed using another 6 bits of the 32-bit string. For example, if the 6-bit string is “000000”, NormalAttack, if “00000” is BendAttack, if “000001” is GraceNoteAttack, if “001000” is NormalShortBody, if “001001” is VibBody, “001010”. The module part name indicates NormalLongBody if present, NormalRelease if “010000”, NormalJoint if “011000”, and GraceNoteJoint if “011001”. Of course, it is needless to say that the present invention is not limited to the above-described configuration.
[0022]
As described above, each rendition style module is specified by a combination of the above “play style ID” and “replay style parameter”. In other words, a predetermined rendition style module is specified according to the “performance style ID”, and the contents are variably set according to the “performance style parameter”. This “performance style parameter” is a parameter that characterizes or controls the waveform data corresponding to the performance style module, and there is a predetermined type of “performance style parameter” for each performance style module. For example, in the case of the AltoSax [NormalAttack] module, performance parameters such as the absolute pitch immediately after the attack and the volume immediately after the attack may be given, and in the case of the AltoSax [BendUpAttack] module, the absolute sound at the end of the BendUpAttack Gives performance parameters such as high, initial value of Bend depth at BendUpAttack, time from BendUpAttack start (note on timing) to end, volume immediately after Attack, or time expansion / contraction of default curve during BendUpAttack May be. In the case of the AltoSax [NormalShortBody] module, various performance parameters such as the absolute pitch of the module, the end time-start time of NormalShortBody, the dynamics at the start of NormalShortBody, and the dynamics at the end of NormalShortBody may be given. Note that the rendition style module does not necessarily have data (element data to be described later) corresponding to all values that can be taken by the “replay style parameters”. In some cases, data corresponding to only a part of the values of “playing style parameters” is stored. That is, for example, in the case of the AltoSax [NormalAttack] module, data corresponding to only a part of data may be stored instead of all values of the absolute pitch immediately after the attack and the volume immediately after the attack.
In this way, by allowing the rendition style module to be designated by “replay style ID” and “replay style parameters”, for example, if AltoSax [NormalAttack], a plurality of data (element data to be described later) indicating the normal attack part of the alto sax It is possible to specify data according to the desired rendition style parameter from the inside, and if Violin [BendAttack], according to the desired rendition style parameter from a plurality of data (element data to be described later) indicating the bent attack portion of the violin Data can be specified.
[0023]
In the rendition style table, for each performance style module, data necessary to generate a waveform corresponding to the performance style module, for example, vector data (waveform element, pitch element (pitch envelope), amplitude element (amplitude) for each component element Envelope) etc.) vector ID and representative point value string (data indicating a representative sample point for correction in a plurality of sample strings) or vector data for each component element (waveform element, pitch element) Information such as the start time position and end time position of (pitch envelope) and amplitude element (amplitude envelope) is stored. That is, various data necessary for reproducing a waveform having a normal shape from a waveform stored in the database in a compressed form called vector data is stored (hereinafter, such data is also referred to as “element data”). Call). An example of specific data stored corresponding to one rendition style module in the rendition style table will be described as follows for the case of the AltoSax [NormalAttack] module.
Data 1: Sample length of rendition style module.
Data 2: Note-on timing position.
Data 3: Vector ID and representative point value sequence of amplitude components of harmonic components.
Data 4: Vector ID and representative point value string of the pitch component of harmonic components.
Data 5: Vector ID of the waveform (Timbre) element of the harmonic component.
Data 6: Vector ID and representative point value string of amplitude elements of non-harmonic components.
Data 7: Vector ID of waveform (Timbre) element of non-harmonic component.
Data 8: Start position of the lump of harmonic component waveform (Timbre) element.
Data 9: End position of the lump portion of the harmonic component waveform (Timbre) element (start position of the loop portion of the harmonic component waveform (Timbre) element).
Data 10: Start position of a lump of a waveform (Timbre) element of an out-of-harmonic component.
Data 11: End position of the lump portion of the waveform (Timbre) element of the non-harmonic component (start position of the loop portion of the waveform (Timbre) element of the non-harmonic component).
Data 12: End position of loop portion of waveform (Timbre) element of non-harmonic component.
[0024]
The data 1 to 12 will be described with reference to FIG.
FIG. 3 is a diagram schematically showing an example of each component and element constituting the actual waveform section corresponding to the rendition style module. From the top, the harmonic component amplitude (Amplitude) element and harmonic component pitch ( An example of a Pitch) element, a harmonic component waveform (Timbre) element, an out-of-harmonic component amplitude (Amplitude) element, and an out-of-harmonic component waveform (Timbre) element are shown. The numbers shown in the figure are attached so as to correspond to the numbers of the respective data.
1 is the sample length (waveform section length) of the waveform corresponding to the rendition style module. For example, it corresponds to the entire time length of the original waveform data that is the basis of the rendition style module. 2 is a note-on timing position, which can be variably set at any time position of the rendition style module. In principle, sounding of the performance sound according to the waveform starts from the position of the note-on timing, but depending on the playing method such as bend attack, the rise start time of the waveform component may precede the note-on timing. is there. Reference numeral 3 denotes a vector ID and a representative point value sequence for indicating vector data of amplitude elements of harmonic components stored in the codebook (in the figure, two points indicated by black squares represent representative points) ). Reference numeral 4 denotes a vector ID and a representative point value string for indicating vector data of a pitch element of the harmonic component. Reference numeral 6 denotes a vector ID and a representative point value string for indicating vector data of an amplitude element of an out-of-harmonic component. The representative point value sequence data is data for changing and controlling vector data (consisting of a plurality of sample sequences) indicated by the vector ID, and indicates (specifies) some representative sample points. By changing or correcting the time position (horizontal axis) and level axis (vertical axis) for the identified representative sample point, other remaining sample points are also changed in conjunction with each other, thereby changing the vector shape. . For example, it is data indicating a smaller number of distributed samples than the number of samples. However, the present invention is not limited to this, and the representative point value sequence data may be data at an intermediate position between samples, or a predetermined value. May be data over a range (sequential multiple samples). Further, not the sample value itself but data such as a difference and a multiplier may be used. By moving the representative point along the horizontal axis and / or the vertical axis (time axis), the shape of each vector data can be changed. That is, the shape of the envelope waveform can be changed. Reference numeral 5 denotes a vector ID for indicating vector data of a harmonic component waveform (Timbre) element. Reference numeral 7 denotes a vector ID for indicating vector data of a waveform (Timbre) element of an out-of-harmonic component. Reference numeral 8 denotes the start position of the waveform block of the harmonic component waveform (Timbre) element. 9 is the end position of the waveform block of the harmonic component waveform (Timbre) element (or the start position of the loop portion of the waveform of the harmonic component waveform (Timbre) element). That is, a triangle starting from 8 indicates a non-loop waveform portion in which characteristic waveform shapes are continuously stored, and a subsequent rectangle starting from 9 indicates a loop waveform portion that can be repeatedly read. The non-loop waveform is a high-quality waveform having characteristics such as performance style (or articulation). The loop waveform is a unit waveform of a relatively monotonous sound portion composed of a waveform for one period or a plurality of suitable periods. Reference numeral 10 denotes the start position of the waveform block of the waveform (Timbre) element of the non-harmonic component. 11 is the end position of the waveform block of the waveform (Timbre) element of the non-harmonic component (or the start position of the loop portion of the waveform of the waveform (Timbre) element of the non-harmonic component). Reference numeral 12 denotes the end position of the loop portion of the waveform of the non-harmonic component waveform (Timbre) element. The data 3 to data 7 are identification information data for indicating the vector data stored in the codebook for each component element, and the data 2 and data 8 to data 12 are the original (before separation) from the vector data. (B) Time information data for assembling the waveform. In this way, the rendition style module data is composed of data for indicating vector data and time information data. By using the rendition style module data stored in such a rendition style table, the waveform can be freely assembled using the waveform material (vector data) stored in the codebook. That is, the rendition style module is data representing the behavior of the waveform generated according to the rendition style (or articulation). The type and number of performance style module data may be different for each performance style module. In addition to the data described above, other information may be provided. For example, it may have data for expanding / compressing the time axis of the waveform.
[0025]
In the above example, in order to make the explanation easy to understand, one rendition style module has all the elements of the harmonic component (waveform, pitch, amplitude) and each element of the non-harmonic component (waveform, amplitude). However, the present invention is not limited to this, and the rendition style module may consist of one of the harmonic components (waveform, pitch, amplitude) or one of the nonharmonic components (waveform, amplitude). Of course. For example, if the rendition style module is a harmonic component waveform (Timbre) element, harmonic component pitch (Pitch) element, harmonic component amplitude (Amplitude) element, non-harmonic component waveform (Timbre) element, non-harmonic component amplitude (Amplitude) It may consist of any one element. In this case, it is preferable that the rendition style modules can be freely combined and used for each component.
[0026]
In this way, instead of having the waveform data of the performance sound in various performance modes of various natural instruments as a whole waveform data, some of the waveforms necessary for changing the waveform shape (for example, attack part waveform, body part waveform) , Release part waveform, joint part waveform, etc.), and waveform data is stored in the hard disk 109 in a compressed form using a hierarchical compression method such as component, element, representative point, and so on. The storage capacity of the hard disk 109 necessary for storing data can be reduced.
[0027]
In the waveform generation apparatus shown in FIG. 1, the synthesis of the waveform is performed by the computer executing a predetermined program (software) that realizes the waveform synthesis process according to the present invention. FIG. 4A shows an example of a flowchart of a predetermined program (“musical tone synthesis process based on database”) that realizes the waveform synthesis process. Further, the present invention is not limited to this type of program, and the waveform synthesis processing may be performed in the form of a dedicated hardware device. FIG. 4B is a block diagram showing an embodiment in which the same waveform synthesizing process as in FIG. 4A is configured in the form of a dedicated hardware device. The description will be given mainly according to FIG. 4B, and the corresponding steps are shown in parentheses in FIG. 4A.
The music data playback unit 101A performs playback processing of music data with performance style symbols (step S11). First, the music data reproducing unit 101A receives music data with performance style symbols (performance information). Normal music scores are accompanied by musical symbols such as dynamic symbols (crescendo, decrescendo, etc.), tempo symbols (Allegro, ritardando, etc.), slur symbols, tenuto symbols, accent symbols, etc. . Thus, these symbols are converted into data as “performance style symbols”, and MIDI music data including the “performance style symbols” is “music data with performance style symbols”. The “playing style symbol” is composed of a chart ID and a chart parameter. The chart ID is an ID indicating a music symbol described in the score, and the chart parameter is a parameter indicating the degree of content of the music symbol indicated by the chart ID. For example, when the chart ID indicates “vibrato”, the speed and depth of the vibrato are given as chart parameters, and when the chart ID indicates “crescendo”, the volume at the start of the crescendo, the end of the crescendo A volume, a length of time during which the volume changes, and the like are given as chart parameters.
[0028]
The score interpretation unit (player) 101B performs score interpretation processing (step S12). Specifically, the MIDI data included in the song data and the above-mentioned “performance style symbol” (chart ID and chart parameter) are converted into performance style designation information (performance style ID and performance style parameter), and a performance style synthesis unit (articulator) along with time information. ) Output to 101C. In general, even with the same musical symbols, the interpretation of the symbols differs depending on the performer, and the performance may be performed by a different performance method (ie, performance method or articulation) for each performer. Alternatively, the performance may be performed by a different performance method for each performer depending on how the notes are arranged. Therefore, the score interpretation unit 101B is an expert system that has knowledge of interpreting such symbols on a score (music symbols, how to arrange musical notes, etc.). An example of a standard for interpreting symbols on a score in the score interpretation unit 101B is as follows. For example, vibrato can only be applied if it is 8 or more notes. Staccato naturally increases dynamics. The attenuation rate of the note is determined by the tenuto degree. Legato does not decay in one sound. The speed of the eighth note vibrato is almost determined by the note value. The dynamics vary depending on the pitch. Furthermore, dynamics change due to pitch rise or fall within one phrase, attenuation dynamics are db linear, note length change according to tenuto, staccato, etc., bend up according to attack bend up symbol There are various interpretation criteria such as width and curve. The score interpretation unit 101B converts the score into sound by performing interpretation on the score according to such a standard. Further, the score interpretation unit 101B performs the above-described score interpretation process in accordance with the player designation from the user, that is, according to the designation of who is performing (the playing style) by the user. The score interpretation unit 101B interprets the score by changing the score interpretation method according to the player designation. For example, different score interpretation methods corresponding to the plurality of players are stored in the database, and the score interpretation unit 101B interprets the score by selectively changing the score interpretation method according to the player designation from the user.
[0029]
Note that the song data (performance information) may be configured to include data indicating the score interpretation result in advance. Needless to say, when such music data including data obtained as a result of previously interpreting the score is input, it is not necessary to perform the above-described processing. Further, the score interpretation processing in the score interpretation unit 101B (step S12) may be performed fully automatically, or may be performed by appropriately interposing a human input operation by the user.
[0030]
The rendition style synthesis unit (articulator) 101C refers to the rendition style table based on the rendition style designation (performance style ID + performance style parameter) converted by the score interpretation unit (player) 101B, and a packet corresponding to the rendition style designation (performance style ID + performance style parameter). Vector parameters relating to the stream (or vector stream) and rendition style parameters corresponding to the stream are generated and supplied to the waveform synthesis unit 101D (step S13). The data supplied to the waveform synthesizer 101D as a packet stream is packet time information, vector ID, representative point value sequence, etc. with respect to the pitch (Pitch) element and amplitude (Amplitude) element, and with respect to the waveform (Timbre) element. Vector ID, time information, etc. (details will be described later).
Next, the waveform synthesis unit 101D extracts vector data from the codebook according to the packet stream, deforms the vector data according to the vector parameter, and synthesizes a waveform based on the deformed vector data (step S14). Then, a waveform generation process for other parts is performed (step S15). Here, the other part is a performance part to which a normal musical sound waveform synthesis process is applied that does not perform a performance style synthesis process among a plurality of performance parts. For example, these other parts generate musical sounds using a normal waveform memory tone generator method. This “other-part waveform generation process” may be performed by a dedicated hardware sound source (an external sound source unit or a sound source card that can be attached to a computer). In order to simplify the explanation, in this embodiment, it is assumed that the tone generation according to the performance style (or articulation) is performed for only one part. Of course, it is also possible to reproduce the performance in multiple parts.
[0031]
FIG. 5 is a block diagram for explaining the flow of rendition style synthesis processing in the rendition style synthesis unit 101C described above. Although FIG. 5 shows that the rendition style module and the code book are stored separately, actually both are stored in the database of the hard disk 109.
The rendition style synthesis unit 101C creates various packet streams to be supplied to the waveform synthesis unit 101D based on the rendition style designation (rendition style ID + rendition style parameters) and time information data from the score interpretation unit 101B. The rendition style module used for each tone color in the rendition style synthesis unit 101C is not fixed, and the user newly adds a rendition style module to the rendition style module in use or a part of the rendition style modules in use. You can stop using it. In the rendition style synthesis unit 101C, a process for creating correction information for correcting a deviation between the selected element data and the rendition style parameter value, and a connection for smoothly connecting the waveform characteristics of the previous and next rendition style modules Processing such as smoothing is also performed (details will be described later).
Note that data is normally provided from the score interpretation unit 101B to the performance style synthesis unit 101C, but the present invention is not limited to this, and as described above, performance-designated song data or human beings already interpreted by the score interpretation unit 101B It is also possible to prepare music data with performance style designation to which performance style IDs and performance style parameters are given by interpreting the musical score, and supplying the reproduced data to the performance style synthesis unit 101C.
[0032]
FIG. 6 is a flowchart showing in detail an embodiment of the rendition style synthesis process.
The rendition style synthesis unit 101C selects a rendition style module from the rendition style table according to the rendition style ID and the rendition style parameters (step S21). That is, one performance style module is selected according to the performance style ID (instrument information + module part name) and performance style parameters transmitted from the score interpretation unit 101B. At this time, before interpreting the score, the score interpretation unit 101B checks the database in advance to confirm what module parts exist in the rendition style table corresponding to the tone color indicated by the musical instrument information. Designation method ID is specified in the range of parts. When a non-existing part is designated, a performance style ID having similar characteristics may be selected instead. Next, a plurality of element data are selected according to the designated performance style ID and performance style parameters (step S22). That is, a rendition style module is identified by referring to the rendition style table by the designated rendition style ID and rendition style parameters, and a plurality of element data corresponding to the rendition style parameters are selected from the module. At this time, if there is no element data that completely matches the rendition style parameter in the rendition style module, element data corresponding to the rendition style parameter close to that value is selected.
[0033]
Next, the time of each position in the element data is calculated according to the time information (step S23). That is, each element data is arranged at an absolute time position based on the time information. Specifically, based on the time information, the corresponding absolute time is calculated from the element data indicating each relative time position. Thus, the timing of each element data is determined (see FIG. 3). And the value of each element data is correct | amended according to a rendition style parameter (step S24). In other words, the deviation between the selected element data and the rendition style parameter value is corrected. For example, the volume (performance parameter) immediately after the attack of the AltoSax [NormalAttack] module transmitted from the score interpretation unit 101B is “95”, and the volume immediately after the attack of the AltoSax [NormalAttack] module in the performance table is “100”. , The rendition style synthesis unit 101C selects element data of the AltoSax [NormalAttack] module whose volume immediately after the attack is “100”. However, since the volume immediately after the attack remains “100” as it is, the volume immediately after the attack is corrected to “95” by correcting the representative point of the selected element data. In this way, correction is performed so that the value of the selected element data is close to the value of the transmitted performance parameter. Further, correction according to the set value of the micro tuning (tuning of the instrument), correction of the volume according to the volume change characteristic of the instrument, and the like are also performed. These corrections are performed by changing the representative point value of each element data, and the representative point value may change greatly. That is, data necessary and sufficient for correction is a representative point, and various corrections are performed by controlling the representative point.
In step S23, the time position indicated by the time information may be corrected using correction information such as the performance parameter. For example, when the time position obtained based on the performance data does not match the time position indicated by the time information, the time information indicating the time position close to the time position obtained based on the performance data is selected and acquired there. By correcting the time position information in accordance with the performance data, the time position information intended by the performance data can be obtained. When the performance data includes a variable control factor such as touch or velocity, the time position information can be variably controlled according to the performance data by correcting the time position information according to the variable control factor. it can. The correction information includes information for performing such time position correction.
[0034]
Further, link processing is performed to adjust each element data and smooth the connection portion between adjacent performance style modules (step S25). That is, by connecting the representative points of the connecting portions in the front and rear performance style modules close to each other, the waveform characteristics of the front and rear performance style modules are made smooth. Such connection or link processing is performed for each element such as the waveform (Timbre), amplitude (Amplitude), and pitch (Pitch) of the harmonic component, or for each element of the waveform (Timbre) and amplitude (Amplitude) of the non-harmonic component. Each is done separately.
At this time, the adjustment is performed in the range from the “link start point” of the previous performance style module to the “link end point” of the subsequent performance style module. That is, the representative points within the range of “link start point” to “link end point” are adjusted based on “rate from walking”. This “rate from step” is a parameter for controlling how far the connection is made from the previous performance module and the subsequent performance module, and is determined according to the combination of the previous and next performance modules as will be described later. . Also, if the waveform connection is not successful when the front and back performance style modules are connected, the connection is smoothed by thinning out the vector IDs of the waveform characteristics in either of the front and rear performance style modules. In order to realize this decimation, a “performance method module combination table”, a “decimation execution parameter range table” to be referred to from now on, and a “thinning time table” to be referred to from now on are prepared.
In addition, the waveform characteristics can be smoothly connected by the link processing in the score interpretation unit 101B as described below. For example, discontinuous portions of performance style parameters (dynamics values, pitch parameter values, etc.) are smoothly connected regardless of the performance style module. Alternatively, when moving from vibrato to release, the vibrato is reduced early, thereby connecting smoothly.
[0035]
Here, the above-described link processing will be described in detail. That is, the adjustment of each element data for smoothing the connection part of the front and rear rendition style modules (see step S25) will be briefly described. First, the link processing when the rendition style module corresponds to an amplitude element or a pitch element will be described with reference to FIG.
If there is a step at the connection point between the two points due to the discontinuity of the representative point value at the connection between the previous performance module and the subsequent performance module, first the dynamics connection point (in the case of Amplitude) or the pitch connection point (Pitch) In the case of (1), an index “Rate from step” is determined as an index indicating whether the target value is closer to the value on the side of the rendition style module. In this embodiment, it is assumed that the “rate from walking” is given by a table as shown in the figure. For example, when the vector ID of the previous rendition style module is “3” and the vector ID of the subsequent rendition style module is “7”, the “rate from step” is determined as “30” from the table. The envelope shape is gradually deformed toward the target value from the “link start point” of the previous rendition style module to the “end point of the rendition style module” according to the “rate from the step” thus determined. Further, the envelope shape is gradually deformed toward the target value from the “link end point” of the later performance style module to the “performance style module start point”. For example, when the “rate from the step” is determined to be “30”, the target value for the previous rendition style module is “30”, and the previous rendition style module performs the “30”% step toward the subsequent rendition style module ( In the present embodiment, the last representative point in the previous rendition style module is a downward “30”% step). On the other hand, the later rendition style module performs “70” (100-30)% steps toward the previous rendition style module side (in this embodiment, the first representative point in the later rendition style module is higher than “70”% steps). To do). In addition, the plurality of representative points of the performance module before and after the link start point to the link end point perform the respective steps up and down along with the above steps. In this way, it is performed at a plurality of representative points of the rendition style module that goes back and forth rather than walking. The link start point and link end point may be determined as appropriate, but if the link start point and link end point are set to the same point as the desired representative point, the link start point and link end point as shown in the figure It is desirable because there is no bending of the envelope shape. Of course, even if the link start point and link end point are not set to the same point as the desired representative point, it goes without saying that the step may be performed so that the envelope shape is not bent.
[0036]
The determination of “rate from step” is not limited to the above-described example. For example, it may be determined based on performance style parameters specified before and after the connection point. Or you may determine based on the performance data before becoming performance style ID or a performance style parameter. Or you may determine based on the combination of those data. In the above example, there is only one representative point to be walked based on the “step rate”, and the other representative points are walked by an appropriate amount along with the walk. It is also possible to separately determine the “rate from walking” and to move the representative points according to the “rate from walking” accordingly.
[0037]
Next, the link processing when the rendition style module is a waveform (Timbre) element will be described. FIG. 8A to FIG. 8D are conceptual diagrams for explaining link processing when the rendition style module is a waveform (Timbre) element. FIG. 8A is a conceptual diagram for explaining waveform thinning when an attack portion waveform and a body portion waveform are connected, and FIG. 8B is a diagram explaining waveform thinning when a body portion waveform and a release portion waveform are connected. It is a conceptual diagram for doing. In FIG. 8A, it is assumed that the body portion waveform is composed of five loop waveforms L1 to L5, and each loop reproduction is performed within a predetermined time range. Similarly, it is assumed that the body part waveform of FIG. 8B includes six loop waveforms L1 ′ to L6 ′.
There are various methods for adjusting the element data related to the waveform (that is, the waveform linking process). For example, the connection between the performance module of the attack part or joint part and the performance module of the body part (or the body part We propose a method for smooth connection by partial thinning of the waveform in the rendition style module and the rendition style module of the release part or joint part). It is well known to perform cross-fade synthesis when connecting waveforms. However, when the time t from the connection point to the start position of the first loop waveform L1 is short as in the example of FIG. 8A, it is necessary to perform a rapid crossfade synthesis within the short time t. When such cross-fade waveform synthesis, that is, when the time between connected waveforms is very close, if cross-fade waveform synthesis is performed between the waveforms, large noise is generated accordingly. A waveform is generated, which is not preferable. Therefore, a part of the waveform is thinned out (deleted) so as to widen the time interval between the connected waveform and the waveform, thereby preventing the sudden crossfade waveform synthesis. In this case, the waveform in the attack part, the release part or the joint part is one lump, and the waveform cannot be thinned out. In this case, the loop waveform on the body part side is thinned out. In FIG. 8A and FIG. 8B, the loop waveforms L1 and L6 ′ indicated by the solid black rectangles are thinned out. For example, in FIG. 8A, the second loop waveform L2 having a relatively long time difference from the connection point and the tail waveform of the attack part waveform are cross-fade synthesized, and the first loop waveform L1 is not used. Similarly, in FIG. 8B, cross-fade synthesis is performed between the loop waveform L5 ′ and the release portion waveform, and the waveform L6 ′ is not used.
In addition, a joint part is a waveform area which connects between sound (or between a sound part and a sound part) by arbitrary performance methods.
[0038]
Also, the connection between the attack style playing module and the release section or joint style playing module is smoothed by waveform thinning. 8C and 8D are conceptual diagrams for explaining waveform thinning when an attack part waveform and a release part waveform are connected.
In this case, there may be a case where a rendition style module such as an attack part or a release part can be thinned out and a case where it cannot. An example of an attack module that can perform waveform thinning is a bend attack section (having several loop waveforms in the latter half). Also, waveform thinning can be performed in the case of a release portion having several loop waveforms in the first half. In this way, waveform reciprocation is performed on the performance style module on the side where waveform thinning is possible. For example, when connecting the bent attack part and the release part, the loop waveform on the bent attack part side is thinned out as shown in FIG. 8C (in FIG. 8C, the loop indicated by the black solid rectangle on the bent attack part side). Decimate one waveform). When the normal attack portion and the release portion having the loop waveform are connected, the loop waveform on the release portion side is thinned as shown in FIG. 8D (in FIG. 8D, the release portion side is shown by a black-filled rectangle) Decimate one loop waveform).
The loop waveform to be thinned out is not limited to the loop waveform closest to the connection portion between the rendition style module and the rendition style module (the loop waveform positioned at the beginning or end), but is thinned out from a plurality of loop waveforms according to a predetermined priority order. You may make it identify the loop waveform made into object.
[0039]
In this way, in the combination of a certain rendition style module, the waveform is thinned out when the connection is not successful within a certain rendition style parameter range. In order to realize this, for example, a “rendition style module combination table” is referred to from now on. A “thinning execution parameter range table” and a “thinning time table” to be referred to from now on are prepared. The “performance style module combination table” is a table for determining a predetermined parameter according to a combination of performance style modules before and after connection. The “thinning execution parameter range table” is a table for determining a time range for performing the thinning for each parameter. The “thinning time table” is a table for determining the thinning time. When the time difference (time t shown in FIGS. 8A to 8D) between the connection time point and the first (or last) loop waveform L1 (or L6 ′) is shorter than the reference thinning time, the loop waveform is thinned out.
[0040]
Further, the waveform connection when the sample length of the performance style module is short and ends before the performance style module following the performance style module is started will be described with reference to FIG. However, in FIG. 9, the waveforms (with four performance modules A.Sax [BendUpAttack], A.Sax [NormalShortBody], A.Sax [VibratoBody], and A.Sax [NormalRelease] are shown in time series from left to right in the figure. A case where a packet stream of (Timbre) element is formed will be described. The sample length (section length) of each rendition style module is represented by the length indicated by “length”. In FIG. 9, “Note On” and “Note Off” described in the top row are MIDI data event timings. Further, A.Sax [BendUpAttack] and the like described in the middle are the timings of performance style IDs, and notes, dynamics, depth, and the like indicate the timings of performance style parameters, respectively.
The A.Sax [BendUpAttack] module starts at time t0. The time t1 is the note-on timing in the module, and matches the instructed note-on timing. That is, in the module, information indicating the time t1 is included as position information indicating the time position corresponding to the note-on. At the time of actual performance, a note-on event is given as a performance event for instructing the note-on timing of the rendition style module. At the note-on timing instructed by the note-on event, the time ( Together with t1) in FIG. 9, the waveform generation of the rendition style module is performed. In the figure, “pre_length” indicates a difference between the waveform generation start time t0 of the module and the note-on timing t1. In other words, the generation of a performance style waveform is started ahead of “pre_length”, and the actual sound generation starts at time t1. The content of the packet stream of the module is controlled based on performance parameters such as note, dynamics, and depth. The A.Sax [NormalShortBody] module starts at time t2 immediately after the attack module. Time t3 is the timing at which the vibrato performance is started in the middle of the connecting portion. This timing is determined based on, for example, the start timing of the vibrato symbol assigned to the song data.
Time t5 is a note-off timing in the A.Sax [NormalRelease] module, and is matched with the instructed note-off timing. The start time t4 of the A.Sax [NormalRelease] module is specified accordingly. That is, in the module, information indicating the time t5 is included as position information indicating the time position corresponding to the note-off. At the time of actual performance, a note-on event is given as a performance event for instructing note-off of this rendition style module. At the note-off timing instructed by this note-off event, the time (see FIG. 9 together with t5), the rendition style module waveform is generated. In the figure, “pre_length” indicates the difference between the waveform generation start time t4 of the module and the note-off timing t5. That is, generation of the performance style waveform of the release part is started ahead by “pre_length”, and the actual mute start starts at time t5. Thus, note-on is performed at time t1 and note-off is performed at time t5. Therefore, the time actually generated according to the waveform generated from the packet stream is the time from time t1 to time t5. In addition, after time t5, it becomes an attenuation sound generation period for muting.
In such a packet stream, the time length from time t2 to time t4 and the sum of the sample lengths length of the A.Sax [NormalRelease] module and A.Sax [VibratoBody] module during that time often do not match. Need to be dealt with appropriately. In such a case, by repeating the same rendition style module, the sum of the sample lengths length is adjusted to the time length, the sample length of the rendition style module is changed to match the time length, or the both are used in combination. Adjust the length of time. In this way, the waveform connection is adjusted between the modules. In the above example, the A.Sax [NormalShortBody] module is repeated to connect the waveform to the subsequent A.Sax [VibratoBody] module. Similarly, by repeating the A.Sax [VibratoBody] module, the waveform connection with the subsequent A.Sax [NormalRelease] module is performed.
[0041]
As described above, when the rendition style module is repeatedly connected a plurality of times, the time length of the repetitive style performance module is varied. In this example, the variable control of the time length is performed by moving the representative point in the A.Sax [NormalShortBody] module or the A.Sax [VibratoBody] module. That is, it is realized by an appropriate method such as changing the time of crossfade connection between a plurality of loop waveforms constituting the A.Sax [NormalShortBody] module or the A.Sax [VibratoBody] module. In the case of a loop waveform, the time length of the entire loop reproduction waveform can be variably controlled by changing the number of loops or the loop duration time relatively easily. On the other hand, in the case of a non-loop waveform, it is not so easy to variably control the existence length on the time axis. Therefore, as described above, in a series of sound waveforms composed of a non-loop waveform and a loop waveform, the overall sound generation time length is variably controlled by performing variable control of the time axis of the waveform data in the loop reading section. It is extremely preferable to facilitate time expansion and contraction control. For this purpose, the “Time Stretch & Compress” control (“TSC control” for short) previously proposed by the present applicant in Japanese Patent Laid-Open No. 10-307586 may be used. In particular, the above-described “TSC control” can be preferably applied to vary the length of the time axis in a non-loop waveform corresponding to a special performance style.
[0042]
An example of the packet stream created in this way is conceptually shown in FIG. FIG. 10 shows the respective packet streams in the harmonic component amplitude (Amplitude) element, pitch (Pitch) element, waveform (Timbre) element, non-harmonic component amplitude (Amplitude) element, and waveform (Timbre) element in order from the top. Also, in the harmonic component amplitude element, the pitch element, and the non-harmonic component amplitude element, the black squares are representative points, and the curve connecting them is the packet in the packet stream. The shape of the vector shown by the contained vector ID is shown. In the waveform (Timbre) element of the harmonic component and the waveform (Timbre) element of the non-harmonic component, the white rectangle L indicates a loop waveform, and the other rectangles NL indicate non-loop waveforms. Of the non-loop waveforms, those with diagonal lines indicate particularly characteristic non-loop waveforms. Furthermore, in this embodiment, the waveform components (Timbre) of the harmonic component and the non-harmonic component in the NormalAttack module are each composed of two vectors, and the amplitude component, the pitch element and the non-harmonic component of the harmonic component. Each of the Amplitude elements is composed of one vector. In the present embodiment, both the harmonic component and the non-harmonic component are such that the amplitude (Amplitude) element and the pitch (Pitch) element do not have a vector in a portion where the waveform (Timbre) element is a non-loop waveform. . However, even when the waveform (Timbre) element is a non-loop waveform, the generated waveform may be controlled by having a vector of an amplitude (Amplitude) element and a pitch (Pitch) element. In the VibratoBody module, the harmonic component waveform (Timbre) element is composed of five vectors, the harmonic component amplitude (Amplitude) element and pitch (Pitch) element, and the non-harmonic component waveform (Timbre) element and amplitude (Amplitude). Each element is composed of one vector. Note that the VibratoBody module is repeated three times, but each vector has a different shape for each iteration. This is because different performance parameters are specified for each repetition. Different element data is selected according to different rendition style parameters, or different level control and time axis control are performed according to different rendition style parameters. In the NormalJoint module, the harmonic component and non-harmonic component waveform (Timbre) elements are each composed of three vectors, the harmonic component amplitude (Amplitude) element, the pitch (Pitch) element, and the non-harmonic component amplitude (Amplitude) element. Are each composed of two vectors. The description of the NormalBody module is omitted.
As described above, the rendition style synthesis unit 101C generates a packet stream for each component (harmonic component and non-harmonic component). These packet streams are composed of a plurality of packets, and each packet includes a vector ID and packet time information. In addition, in the case of an amplitude element, a pitch element, and an amplitude element of an out-of-harmonic component, a definite value of each representative point is included. Of course, the present invention is not limited to this, and other information may be included in addition to the vector ID and the packet time information. A packet stream is configured for each component in accordance with the contents of each such packet. Thus, the packet stream includes a plurality of packets and time information (start time) of each packet.
Needless to say, the number of packet streams may differ depending on the type of musical instrument.
[0043]
The waveform synthesis unit 101D synthesizes a waveform based on the packet stream (a plurality of packet strings including vector ID, time information, correction information, etc.) for each component supplied from the rendition style synthesis unit 101C. FIG. 11 is a conceptual diagram showing the overall configuration in order to explain the operation in the waveform synthesizer 101D. 12 to 15 are block diagrams for explaining each operation in the waveform synthesis unit 101D in detail. FIG. 12 is a block diagram simply showing the overall flow of waveform synthesis. FIG. 13 is a block diagram for explaining the vector loader. FIG. 14 is a block diagram for explaining a vector operator. FIG. 15 is a block diagram for explaining the vector decoder.
The packet stream for each component element created by the rendition style synthesis unit (articulator) 101C is sequentially input to predetermined packet queue buffers 21 to 25 provided corresponding to each component element in the waveform synthesis unit 101D ( That is, it is input in packet units). The input packets are accumulated in the packet queue buffers 21 to 25 and sequentially sent to the vector loader 20 in a predetermined order. The vector loader 20 reads (loads) original vector data corresponding to the vector ID from the code book 26 with reference to the vector ID in the packet. The read vector data is sent to predetermined vector decoders 31 to 35 provided corresponding to each component element, and the vector decoders 31 to 35 generate waveforms for each component element. Further, the vector decoders 31 to 35 generate waveforms for each component (harmonic component and out-of-harmonic component) while synchronizing the generated waveform for each component element between the vector decoders 31 to 35. The waveform for each component generated in this way is sent to the mixer 38. In the rendition style synthesis unit (articulator) 101C, packets are input to the packet queue buffers 21 to 25, stream management (management of generation and deletion of individual vector data or connection between vector data) and playback control (desired) The waveform synthesizer 101D is subjected to various controls such as execution of waveform generation or control of reproduction / stop of the desired waveform generated).
[0044]
As described above, the vector loader 20 sequentially receives the packets constituting the packet stream accumulated in the packet queue buffer 21, and the vector decoder 20 reads the vector from the code book 26 based on the vector ID in each packet. Vector data corresponding to the ID is read, and the read vector data is sent to the vector decoder 31 (see FIG. 12). At this time, correction information (for example, correction information related to representative points) may be attached to each read packet. In such a case, the vector loader 20 modifies the read original vector data in accordance with the correction information, and a packet having information of the deformed vector data (referred to as vector information data in order to distinguish it from the original vector data) ( This is called a vector packet in order to distinguish it from the packet input from the rendition style synthesis unit 101C), and is output to the vector decoders 31-35. As described above, the vector loader 20 reads the original vector data from the code book 26 based on the vector ID of the packet input from the rendition style synthesis unit (articulator) 101C, and corrects the vector data with correction information as necessary. After that, the vector packet is transferred to the vector decoders 31 to 35 (see FIG. 13). The correction information related to the representative points of the vector data as described above may include various correction information such as correction information for shifting time information based on random numbers.
[0045]
As shown in FIG. 14, the vector decoders 31 to 35 generate and discard vector operators for processing input vector packets, connect / synchronize management between vector operators, time management, and input from other vector ID streams. Various types of operator operation management such as parameter conversion settings for each vector operator are performed. The vector operators 36 and 37 perform reading of vector information data, vector information data reading position control (Speed input), gain control (Gain input), and the like. Various parameters set in the vector operators 36 and 37 are managed by vector decoders 31-35. Vector decoders 31 to 35 are provided so as to correspond to each component element, and the vector decoders 31 to 35 read out vector information data in the vector packet and generate a desired waveform in time series. For example, as shown in FIG. 15, the vector decoder 31 generates an envelope waveform of the harmonic component Amplitude element, the vector decoder 32 generates the envelope waveform of the harmonic component pitch element, and the vector decoder 33 Generates the waveform of the harmonic component waveform (Timbre) element, the vector decoder 34 generates the envelope waveform of the amplitude component (Amplitude) of the non-harmonic component, and the vector decoder 35 generates the envelope of the waveform component (Timbre) of the non-harmonic component. Generate a waveform. The vector decoder 33 generates a harmonic component waveform to which the envelope waveform of the harmonic component Amplitude element generated by the vector decoders 31 and 32 and the envelope waveform of the harmonic component pitch element are added, and the mixer 38. Output to. That is, for the vector decoder 33, the envelope waveform of the harmonic component pitch element is used as vector information data as a vector operator for performing gain control (Gain input) on the envelope waveform of the harmonic component Amplitude element. Is input as a vector operator for performing the reading position control (Speed input). Further, the vector decoder 35 generates an out-of-harmonic component waveform to which the envelope waveform of the amplitude component of the out-of-harmonic component generated by the vector decoder 34 is added, and outputs the generated waveform to the mixer 38. That is, the envelope waveform of the amplitude component of the out-of-harmonic component is input to the vector decoder 35 as a control command for performing gain control (Gain input).
[0046]
Furthermore, when generating waveforms in time and in the components and elements, waveform generation is performed while synchronizing the waveforms between the vector decoders 31-35. For example, when a vector packet of a waveform (Timbre) element and a vector packet of an amplitude (Amplitude) element are input, the amplitude (Amplitude) is synchronized with the waveform generation time based on the vector packet of the waveform (Timbre) element as a reference. ) Generate an amplitude waveform based on the vector packet of elements. The amplitude of the waveform generated based on the vector packet of the waveform (Timbre) element is controlled by the amplitude waveform. When a vector packet of a waveform (Timbre) element and a vector packet of a pitch (Pitch) element are input, the time of waveform generation based on the vector packet of the waveform (Timbre) element is used as a reference, and the pitch (Pitch) element A pitch waveform based on a vector packet is synthesized. The pitch of the waveform generated based on the vector packet of the waveform (Timbre) element is controlled by the pitch waveform. When inputting the harmonic component waveform (Timbre) element vector packet and the non-harmonic component waveform (Timbre) element vector packet, based on the harmonic component synthesis time based on the harmonic component waveform (Timbre) element vector packet In synchronization therewith, the out-of-harmonic component based on the vector packet of the waveform (Timbre) element of the out-of-harmonic component is synthesized. A desired musical sound waveform is generated by mixing the synthesized harmonic and non-harmonic waveforms.
In this embodiment, the harmonic component and the non-harmonic component can be selected to be synchronized or asynchronous, and based on the above-described vector packet of the harmonic component waveform (Timbre) element only when synchronization is selected. The waveform of the non-harmonic component generated based on the vector packet of the waveform (Timbre) element of the non-harmonic component may be synthesized in synchronism with the waveform synthesis time of the harmonic component generated .
[0047]
As described above, the packet stream is composed of a plurality of packet sequences. In the case of a vector bucket packet stream, each packet includes vector data. That is, a packet stream is a sequence of vector data arranged in the time direction. The vector structure of the amplitude (Amplitude) element, the vector data of the pitch (Pitch) element, the vector data of the waveform (Timbre) element has the data structure and meaning. Although different, it is basically the same when viewed from the vector operators 36 and 37.
FIG. 16 is a conceptual diagram conceptually showing an embodiment of the data structure of vector data. For example, when the unit of the reading position of the vector data is [SEC] and the reading speed is constant, one sample on the vector data coincides with one sample of the output waveform. The unit of the reading speed is 1/1200 [cent] (= 2 to the power of n). When the index n is 0, it is constant speed, and when it is 1.0, it is double (for example, 1 octave in the case of a waveform (Timbre) element) -1.0 is 0.5 times (for example, in the case of a waveform (Timbre) element, it is lowered by one octave) (see the upper diagram of FIG. 16). The code book 26 stores actual vector data. For example, vector data of an amplitude element or vector data of a pitch element includes an array of VECTORPOINT structures and representative point data. The VECTORPOINT structure array contains the sample positions and values of each point in order. For example, the vector data value of the Amplitude element is in [db] units, and the vector data value of the Pitch element Is 1/1200 [cent] unit when MIDI note number 0 is 0.0. The representative point data is a DWORD type array, and stores the index number of the array of VECTORPOINT structures that are representative points (see the lower diagram of FIG. 16). Of course, it goes without saying that the present invention is not limited to the example described above.
[0048]
When the waveform generation apparatus as described above is used for an electronic musical instrument, the electronic musical instrument is not limited to a keyboard instrument, and may be any type of form such as a stringed instrument, a wind instrument, or a percussion instrument. In this case, the music data playback unit 101A, the score interpretation unit 101B, the rendition style synthesis unit 101C, the waveform synthesis unit 101D, etc. are not limited to those built in one electronic musical instrument main body, but each is configured separately, and MIDI Needless to say, the present invention can be similarly applied to a configuration in which each component is connected using communication means such as an interface or various networks. In addition, a configuration of a personal computer and application software may be used. In this case, the processing program may be supplied from a storage medium such as a magnetic disk, an optical disk, or a semiconductor memory, or may be supplied via a network. Furthermore, the present invention may be applied to an automatic performance device such as an automatic performance piano.
[0049]
【The invention's effect】
According to the present invention, in the rendition style module describing a rendition style waveform, position information indicating the position to be timed with a predetermined performance event is included in the time axis of the rendition style waveform, so that the predetermined performance event indicates The timing of the musical sound characteristic control to be performed can be set in the middle of the time axis of the performance style waveform, and a more realistic performance style waveform can be synthesized. For example, the performance waveform generation start timing can be appropriately controlled to precede the timing of a performance event such as a sound generation start (note-on) event or a mute start (note-off) event. For example, depending on the playing technique such as a bend attack, the rising start time of the waveform component may precede the note-on timing, and the violin begins to rub the string with the bow before the sound actually comes out. Therefore, according to the present invention, it is suitable for accurately simulating the start of the performance style waveform before the start of the sound generation or for freely controlling it. Further, when applied to a note-off event, it is suitable for accurately simulating the start of a performance style waveform before the start of mute or for freely controlling it.
[Brief description of the drawings]
FIG. 1 is a block diagram showing a hardware configuration example of a waveform generation apparatus according to the present invention.
FIG. 2 is a flowchart showing an example of “waveform database creation processing” executed in the waveform generation device;
FIG. 3 is a diagram schematically showing an example of each component and element constituting an actual waveform section corresponding to a rendition style module.
FIG. 4A is a flowchart showing an example of “musical tone synthesis processing based on a database”;
FIG. 4B is a block diagram showing an embodiment in which the same waveform synthesis processing as that in FIG. 4A is configured in the form of dedicated hardware.
FIG. 5 is a block diagram for explaining the flow of rendition style composition processing in the rendition style composition unit described above.
FIG. 6 is a flowchart showing in detail an embodiment of a rendition style synthesis process performed by a rendition style synthesis unit.
FIG. 7 is a conceptual diagram for explaining link processing when a rendition style module corresponds to an amplitude element or a pitch element.
FIG. 8A is a conceptual diagram for explaining waveform thinning when an attack waveform and a body waveform are connected.
FIG. 8B is a conceptual diagram for explaining waveform thinning when a body waveform and a release waveform are connected.
FIG. 8C is a conceptual diagram for explaining waveform thinning when a bent attack waveform and a release waveform are connected.
FIG. 8D is a conceptual diagram for explaining waveform thinning when a normal attack waveform and a release waveform having a loop portion are connected.
FIG. 9 is a conceptual diagram for explaining waveform connection in a case where a performance style module before the performance style module that follows is ended.
FIG. 10 is a conceptual diagram for explaining a packet stream generated by a rendition style synthesis unit.
FIG. 11 is a conceptual diagram showing an example of the overall configuration for explaining the operation in the waveform synthesis unit.
FIG. 12 is a block diagram simply showing the overall flow of waveform synthesis.
FIG. 13 is a block diagram for explaining a vector loader.
FIG. 14 is a block diagram for explaining a vector operator.
FIG. 15 is a block diagram for explaining a vector recorder.
FIG. 16 is a conceptual diagram conceptually showing an example of the data structure of vector data.
[Explanation of symbols]
101 ... CPU, 102 ... Read only memory (ROM), 103 ... Random access memory (RAM), 104 ... Panel switch, 105 ... Panel display, 106 ... Drive, 106A ... External storage medium, 107 ... Waveform capture unit, DESCRIPTION OF SYMBOLS 108 ... Waveform output part, 108A ... Sound system, 109 ... Hard disk, 111 ... Communication interface, BL ... Bus line, 101A ... Song data reproduction part, 101B ... Music score interpretation part, 101C ... Rendition style synthesis part, 101D ... Waveform synthesis part, 20 ... Vector loader, 31 (32 to 35) ... Vector decoder, 36 (37) ... Vector operator, 38 ... Mixer

Claims (5)

In the rendition style module describing a rendition style waveform, position information indicating the position to be timed with a predetermined performance event is included in the time axis of the rendition style waveform, and the rendition style module for the rendition style waveform to be generated is included with the performance event. A step to specify;
Determining a control timing of a predetermined musical sound characteristic in the rendition style module according to the position information and a specified performance event in the specified rendition style module;
And a step of synthesizing a musical sound corresponding to the performance style module based on the performance style waveform of the performance style module for which the control timing is determined.   The waveform generation method according to claim 1, wherein the predetermined performance event is an event for instructing sound generation start timing.   The waveform generation method according to claim 1, wherein the predetermined performance event is an event that instructs a mute start timing. The computer-readable storage medium which memorize | stored the program for making a computer perform the said each step in the waveform generation method in any one of Claim 1 thru | or 3 . In the rendition style module describing a rendition style waveform, position information indicating the position to be timed with a predetermined performance event is included in the time axis of the rendition style waveform, and the rendition style module for the rendition style waveform to be generated is included with the performance event. Means to specify;
Means for determining a control timing of a predetermined musical sound characteristic in the rendition style module according to the position information and a specified performance event in the specified rendition style module;
A waveform generating apparatus comprising: means for synthesizing a musical sound corresponding to the performance style module based on the performance style waveform of the performance style module for which the control timing is determined.

Images