JP2002023741A - Synthesizer for acoustic signal - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a synthesizer for acoustic signals capable of reproducing the acoustic signals of a range wider than before even while utilizing standard code data based on a prescribed standard such as an MIDI standard. SOLUTION: A general purpose standard data form (figure 3 (a)) and an extension data form (figure 3 (b)) capable of handling highly accurate data are prepared. In the extension data form, the number of channels for recording each of single sound for constituting chords is increased, a note number can be more finely recorded by providing a fine pitch and further, a lot of sound velocities can be recorded. Thus, highly accurate sounds which can not be reproduced by the conventional standard data form can be reproduced.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to broadcast media (radio, television), communication media (CS video / audio distribution, Internet music distribution, communication karaoke), package media (CD, MD, cassette, video, LD, CD). −
Production of various audio contents provided by ROMs, game cassettes, solid-state memory media for portable music players, etc., and dedicated portable music players, mobile phones and PHs
The present invention relates to an audio signal encoding technique suitable for use in MIDI transmission of music contents including vocals for S. pagers, audio materials of literary works such as kabuki, noh, chanting and poetry, or language teaching audio teaching materials.

[0002]

2. Description of the Related Art Time-series signals represented by acoustic signals include:
The components include a plurality of periodic signals. For this reason, a method of analyzing what periodic signal is included in a given time-series signal has been known for a long time. For example, Fourier analysis is widely used as a method for analyzing frequency components included in a given time-series signal.

[0003] If such a time-series signal analysis method is used, it is possible to encode an audio signal. With the spread of computers, it has become easier to sample analog audio signals as original sounds at a predetermined sampling frequency, quantize the signal strength at each sampling, and take in as digital data. If a method such as Fourier analysis is applied to the data and frequency components included in the original sound signal are extracted, the original sound signal can be encoded by a code indicating each frequency component.

On the other hand, MIDI (Musical Instrume) was born from the idea of encoding musical instrument sounds by electronic musical instruments.
The Digital Interface (nt Digital Interface) standard has also been actively used with the spread of personal computers. Code data according to the MIDI standard (hereinafter, MID)
I data) is basically data describing an operation of playing a musical instrument, such as which keyboard key of the musical instrument was played and at what strength, and the MIDI data itself contains the actual sound. No waveform is included. Therefore, when an actual sound is reproduced, a MIDI sound source storing a waveform (distortion waveform pattern) of the musical instrument sound is required separately. Encoding and decoding techniques are now widely adopted in software for performing musical instruments, practicing musical instruments, composing music, and the like using a personal computer.

Therefore, by analyzing a time-series signal represented by an acoustic signal by a predetermined method, a periodic signal as a component of the signal is extracted, and the extracted periodic signal is converted to an M signal.
There have been proposals to encode using IDI data. For example, JP-A-10-247099, JP-A-11-73199, JP-A-11-73200
JP, JP-A-11-95753, JP-A-2000
-99009, JP-A-2000-99093, Japanese Patent Application No. 11-58431, Japanese Patent Application No. 11-1.
In the specification of 77875 and the specification of Japanese Patent Application No. 11-329297, various methods are proposed that can analyze the frequency as a component element of an arbitrary time-series signal and create MIDI data from the analysis result. Have been.

[0006]

Since the current MIDI sound source device is basically designed to reproduce music (instrument sound), natural sound, voice, and the like can be obtained by the encoding method proposed so far. Even if various audio signals can be faithfully converted to MIDI codes, there is a limit to the reproduction quality of voices and natural sounds particularly rich in noise. However, even in the case of music, there are many cases where instrumental sounds such as singing voices and voices are mixed, and singing voice parts are not represented by MIDI, but are usually handled separately as waveform information.
In the case of music expressed in MIDI, tempo, pitch,
Since there is a merit that the timbre can be freely changed, a special device is required to realize a similar function for a singing voice part played in synchronization with the timbre. For example, in the field of communication karaoke, music information is transmitted in the MIDI format, and a back chorus is transmitted in the form of waveform information. However, the reproduction quality is still remarkably degraded due to the vocal signal processing, and it does not hinder the back chorus, but is difficult to use in the main vocal. Is necessary.

[0007] However, transmission by MIDI code and MIDI
As described above, the form of reproducing with the sound source has such attractiveness that the tempo, the pitch, and the tone can be freely changed. However, the MIDI encoding format includes
There are three problems as follows. First, standard MIDI
The code has a restriction that the frequency is a semitone unit and the intensity is 128 steps, and it is difficult to faithfully reproduce voice and natural sound. Second, the MIDI sound source of the GM standard is limited to 16 channels at maximum and the number of simultaneous sounds is 32 chords / 64 chords, and is not sufficient to express the entire acoustic signal including orchestral music. Thirdly, various noises are important expression means for voice, natural sound, biological signal, etc.
Although a general MIDI sound source in the VE table format can emit these sounds in a single shot, it is not suitable for synthesis, processing, and expression.

Accordingly, the present invention has been made in view of the above points,
It is an object of the present invention to provide a sound signal synthesizing apparatus capable of reproducing a wider range of sound signals than before using standard code data conforming to a predetermined standard such as the MIDI standard.

[0009]

In order to solve the above-mentioned problems, the present invention provides an audio signal synthesizing apparatus comprising a sound generation start time, a sound generation end time, sound frequency information, sound intensity information, and timbre identification information. The phoneme data is composed of a plurality, for an acoustic code that can take a plurality of data formats, fetch the acoustic code from the outside, recognize which data format,
A phoneme data acquisition unit for extracting each phoneme data according to each data format; and a basic distortion waveform having a waveform shape based on the timbre identification information and having a length based on a time from the sounding start time to the sounding end time. Waveform component generation means for preparing data, time axis expansion / contraction means for applying magnification to the basic distortion waveform data in the time axis direction based on the frequency information of the sound, and intensity information of the sound. An amplitude axis expansion / contraction means for applying magnification to the basic distortion waveform data in the amplitude axis direction, and two types for holding the basic distortion waveform data subjected to magnification in the time axis direction and the amplitude axis direction. An exclusive mode in which one type is used for writing and the other type is used for reading is defined with respect to two types of buffer memories of the waveform storage means including the buffer memory of Memory mode switching means for switching between these modes at a time interval of: and a buffer memory for writing corresponding to the sound generation start time and the sound generation end time when the basic distortion waveform data is written to the write buffer memory. Output waveform synthesizing means for performing a process of writing to the write buffer memory while adding the already written waveform data within the address range of And a composite signal reproducing means for outputting the signal to the outside.
According to the present invention, it is possible to input a plurality of data formats as an audio signal, recognize which data format the input audio signal is, extract each phoneme data according to each data format, and perform basic processing from each phoneme data. Obtaining the distorted waveform data, transforming the basic distorted waveform data to obtain waveform data, and synthesizing it with other obtained waveform data for output , It is possible to handle a more precise special data format,
While utilizing the merits of general-purpose data, it is possible to faithfully reproduce all acoustic signals without being restricted by the general-purpose data.

[0010]

Embodiments of the present invention will be described below in detail with reference to the drawings. First, the basic concept of the present invention will be described using a MIDI coding format as an example. FIG. 1 is a conceptual diagram of an extended MIDI encoding / decoding system. In the method shown in FIG.
A major feature is that the encoding system is divided into DI encoding and extended MIDI encoding, and the decoding system is divided into a standard-compliant MIDI sound source and an extended MIDI sound source. When music (instrument sound) is input as a source acoustic signal, a standard-compliant MI
DI encoding is performed, but when natural sound, body sound, or noise is input, the standard compliant M
IDI encoding or extended MIDI encoding is performed,
When voice is input, extended MIDI encoding is performed.

Code data obtained by encoding is stored, edited, and transmitted in accordance with respective standards, and reaches a decoding system. At the time of decoding, the standard-compliant MIDI data is decoded using both the standard-compliant MIDI sound source and the extended MIDI sound source. On the other hand, extended MIDI
Data is decoded using only the extended MIDI sound source. Performing extended MIDI encoding in this way,
By decoding using the extended MIDI sound source,
In particular, reproduction of natural sounds, body sounds, noises, and voices is faithfully performed. The present invention particularly relates to this decoding part.

Next, a specific configuration of the audio signal synthesizing apparatus corresponding to the decoding system will be described with reference to FIG. FIG. 2 is a functional block diagram showing the configuration of the audio signal synthesizing device according to the present invention. In FIG. 2, a phoneme data acquisition unit 1 has a function of inputting an acoustic code such as MIDI data and acquiring phoneme data. Since the acoustic code is a set of phoneme data arranged in time series, each phoneme data is obtained by sequentially extracting the acoustic code from the acoustic code. The waveform component generation means 2 extracts a waveform corresponding to timbre identification information in the phoneme data from a waveform storage means (not shown) such as a MIDI sound source, and extracts the extracted waveform until a sound generation start time and a sound generation end time. It has a function of creating a plurality of basic distortion waveform data connected repeatedly so as to be time. The time axis stretching means 3 has a function of changing the frequency of the basic distortion waveform data based on the frequency information of the sound. Amplitude axis expansion / contraction means 4
Has a function of changing the amplitude of the basic distortion waveform data based on the sound intensity information. The output waveform synthesizing means 5 has a function of performing writing while adding the already written waveform data to the address range of the writing buffer memory corresponding to the sound generation start time and the sound generation end time. The waveform storage unit 6 has two buffer memories in which writing and reading are switched by the memory mode switching unit 7, and has a function of storing a combined output waveform. The composite signal reproducing means 8 has a function of reading out an output waveform from a buffer memory functioning as a readout and reproducing the output waveform as an acoustic signal.

Here, the acoustic code input to the acoustic signal synthesizer will be described. MIDI as acoustic code
FIG. 3A shows a data format when the standard is applied.
It is not always necessary to employ the MIDI format as the acoustic code, but since the MIDI format is the most widespread as this type of encoding format, it is preferable to use MIDI format code data for practical use. In the MIDI format, “Note ON” data or “Note OFF” data is
It exists with intervening "delta time" data.
"Note ON" data is data for designating the start of performance of a specific sound by designating a specific note number N and velocity V, and "Note OFF" data is for specifying a specific note number N and velocity V. This is data that designates the end of performance of a specific sound by designating it. The “delta time” data is data indicating a predetermined time interval. Velocity V is a parameter indicating, for example, the speed at which the keyboard of a piano is depressed (velocity at the time of note ON) and the speed at which the finger is released from the keyboard (velocity at the time of note OFF). Or it indicates the strength of the performance end operation. In the case of the MIDI standard, each of these data is composed of a pair of note ON / note OFF as shown in FIG. 3 (a), and data obtained by adding timbre identification information to the data becomes phoneme data. This set of phoneme data is an acoustic code. FIG.
In (a), when the note is ON and when the note is OFF, the delta time indicates the sounding start time and the sounding end time of the phoneme data, respectively, the note number indicates the sound frequency information, and the velocity indicates the sound intensity information. The channel number is equivalent to timbre identification information, and is used to specify the timbre of each tone (single or chord) of a sound to be simultaneously pronounced. In the standard MIDI standard, a total of 16 channels from 0 to 15 can be used. However, in order to distinguish it from the extended format described below,
In the present embodiment, the number of channels is limited to a total of 15 channels from 0 to 14.

In the present invention, data in the extended MIDI format whose specifications are extended together with data in the standard MIDI format as shown in FIG. FIG. 3B shows the data format of the extended MIDI. Compared with the standard MIDI format, the extended MIDI format first differs in the setting of the channel number. The channel number is fixed at 15, and the means for decoding MIDI data can determine whether the channel number is 15 or something other than the extended MIDI format or the standard MIDI format. In the extended MIDI format, an extended channel number is used to specify a channel in which information of each single note constituting a chord is recorded, and 128 channels can be used for 15 channels in the standard MIDI format. I have. Although the note number is the same, the extended MIDI format has a fine pitch item, and the minimum unit of the note number is a semitone, whereas the fine pitch can be set in 1/100 semitone units. ing.
By having the upper byte and the lower byte, the velocity can be set with a precision 128 times higher than that of the standard MIDI.

A case where the MIDI data shown in FIG. 3 is used as an input audio code and processed by the audio signal synthesizer shown in FIG. 2 will be described. When the acoustic code is input, the phoneme data acquisition means 1 checks the channel number of the acoustic code, and if the channel number is 0 to 14, the standard MID
If the channel number is 15 as the I format, the extension M
Recognize as IDI format. Subsequently, each phoneme data is extracted from the acoustic code according to the recognized data format.

Next, the waveform component generating means 2 generates basic distortion waveform data based on the sounding start time, sounding end time, and timbre identification information among the extracted phoneme data. Specifically, first, based on the timbre identification information included in the phoneme data, the timbre definition table included in the waveform component generation unit 2 is referred to and converted into a corresponding timbre code. Further, based on the timbre code, the spectrum table included in the waveform component generation means 2 is referred to, and the corresponding basic distortion waveform data is obtained. The acquisition of the basic distortion waveform data using the spectrum table will be described with reference to FIG. FIG.
In the graph, the upper left two graphs are spectrum data corresponding to the input timbre code, and the horizontal axis represents frequency and the vertical axis represents intensity. Each spectrum data indicates that timbre 1 and timbre 2 each include five frequencies including the fundamental frequency. In each spectrum data, the frequency indicated by a thick line is the fundamental frequency. Subsequently, sine wave synthesis is performed using the spectrum data. This is done by synthesizing a sine wave having each frequency and an amplitude corresponding to the intensity. As a result, basic distortion waveform data as shown in the upper right of the figure is obtained. Since this basic distortion waveform data has only information of a very short time (several tens of milliseconds), a process of arranging and synthesizing data in a plurality of time directions so as to correspond to the length from the sound generation start time to the sound generation end time is performed. . As a result, basic distortion waveform data corresponding to the sounding time of the phoneme data is obtained.

Next, the time axis expanding / contracting means 3 expands / contracts the obtained basic distortion waveform data in the time axis direction based on the frequency information of the sound. Specifically, the frequency of the basic distortion waveform data is changed so that the frequency becomes the same as the frequency information of the sound included in the phoneme data. Subsequently, the amplitude axis expansion / contraction means 4 performs expansion / contraction processing in the amplitude axis direction on the distortion waveform data obtained by expanding / contracting the basic distortion waveform data in the time axis direction based on the sound intensity information. . Specifically, the amplitude value is scaled so that the maximum amplitude of the distortion waveform data becomes the same as the sound intensity information of the phoneme data.
Further, at the sound generation start time, the rise control is performed so that the amplitude is gradually increased as shown in the output composite waveform in the lower part of FIG. The time from the tone generation start time to the maximum amplitude can be changed by setting. Similarly, with respect to the sound generation end time, the fall control is performed so that the amplitude does not suddenly become zero but gradually decreases. However,
In the fall control, the amplitude is the maximum amplitude until the sound generation end time, and the amplitude is gradually reduced from the sound generation end time. Here, the processing is performed in the order of expansion and contraction of the time axis and expansion and contraction of the amplitude axis. However, the processing may be performed in the reverse order or simultaneously.

Subsequently, the output waveform synthesizing means 5 performs a process of writing the obtained distortion waveform data into the write buffer memory of the waveform storage means 6. At this time, the distortion waveform data recorded in the write buffer memory is first read, the distortion waveform data to be newly written is added to the data, and the combined distortion waveform data is recorded in the write buffer memory. I do. If Figure 4
As shown in the output composite waveform in the lower part, if the sounding end time of the previous distortion waveform data and the sounding start time of the subsequent distortion waveform data are close to each other, the former waveform and the latter waveform partially overlap, thereby providing a smoother waveform. Switching of sounds (switching of pitch, sound intensity, and tone) is performed.

The waveform storage means 6 has two buffer memories A and B. The writing and reading functions of these buffer memories are alternately switched by the memory mode switching means 7 at predetermined time intervals. The distortion waveform data recorded in the read buffer memory is sent to the synthesized signal reproducing means 8 at predetermined time intervals, and reproduced as an acoustic signal. The switching time interval T is set to a value equal to or greater than the maximum value of the sounding time (sounding end time-sounding start time) of the phoneme data. The write buffer memory and the read buffer memory have a capacity capable of storing data twice as long as the switching time interval T. However, the effective area of the data is T
At time, the write buffer memory is switched to the read buffer memory, and the data read by the composite signal reproducing means 8 is only the area for the first half T time. However, for writing, it is assumed that a writing operation to the area for the second half T time of the buffer memory is permitted. As described above, since the data reading process is performed before the writing process to the writing buffer memory, the data that has been subjected to the audio signal synthesizing process needs to be erased. Therefore, this initialization process is performed as follows. In both cases, the contents are all erased in the initial state, and the read buffer memory is switched to the write buffer memory, and at the same time, the data for the second half T of the new read buffer memory (holding the contents of the previous write buffer memory) is newly added. The data area for the first half T of the new write buffer memory is overwritten, and the area for the second half T of the new write buffer memory is erased. Then, even when the time is specified so that the phoneme data sounds over the switching time, the data is not lost. This is because, in such a case, the data protrudes and is written into the area corresponding to the latter half T time of the write buffer memory, and in the next cycle, the protruding data is used for waveform synthesis.

In the above embodiment, the waveform component generating means 2 has a spectrum table, extracts predetermined spectrum data by referring to the spectrum table by tone color code, and synthesizes a sine wave using the spectrum data. Has been described, the waveform component generating means 2 has a basic waveform definition table, and a method of obtaining the basic distortion waveform data by referring to the basic waveform definition table by a tone color code is applied. You can also. Specifically, general-purpose WA
Waveform data is prepared in VE format, and a table in which the waveform data is associated with a tone color code is prepared as a basic waveform definition table.

[0021]

As described above, according to the present invention,
The sound signal synthesizing device sets the sound generation start time, sound generation end time,
Sound frequency information, sound intensity information, phoneme data consisting of timbre identification information is composed of a plurality, for the acoustic code that can take a plurality of data formats, the acoustic code is fetched from the outside, which data format A phoneme data acquisition means for recognizing and extracting each phoneme data according to each data format, and a waveform shape based on the timbre identification information, having a length based on a time from the sounding start time to the sounding end time. Waveform component generation means for preparing basic distortion waveform data, time axis expansion / contraction means for applying magnification to the basic distortion waveform data in the time axis direction based on the frequency information of the sound, Amplitude axis expansion / contraction means for applying magnification to the basic distortion waveform data in the amplitude axis direction based on intensity information, and a basic distortion wave subjected to magnification in the time axis direction and the amplitude axis direction Defining a waveform storage means composed of two types of buffer memories for holding data, and an exclusive mode in which one of the two types of buffer memories of the waveform storage means is for writing and the other is for reading, Memory mode switching means for switching between these modes at predetermined time intervals; and a writing buffer corresponding to the sound generation start time and the sound generation end time when writing the basic distortion waveform data to the writing buffer memory. Output waveform synthesizing means for performing a process of writing to the write buffer memory while adding the already written waveform data to the address range of the memory;
And a synthetic signal reproducing means for outputting the contents of the read buffer memory as a sound signal to the outside in a time-series manner, so that a general-purpose data format can be used and special data with higher accuracy can be used. This makes it possible to handle formats, and makes it possible to faithfully reproduce all audio signals while taking advantage of general-purpose data while being free from restrictions of general-purpose data.

[Brief description of the drawings]

FIG. 1 is a conceptual diagram of an extended MIDI encoding / decoding system.

FIG. 2 is a functional block diagram illustrating a configuration of an acoustic signal synthesis device.

FIG. 3 is a diagram illustrating an example of a data format of an acoustic code that can be handled by the acoustic signal synthesis device.

FIG. 4 is a diagram showing a state of creating basic distortion waveform data and synthesizing an output waveform when a spectrum table is used.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 ... Phoneme data acquisition means 2 ... Waveform component generation means 3 ... Time axis expansion / contraction means 4 ... Amplitude axis expansion / contraction means 5 ... Output waveform synthesis means 6 ... Waveform storage means 7 ... .Memory mode switching means 8... Synthesized signal reproducing means

Claims (10)

[Claims] 1. An acoustic code which is composed of a plurality of phoneme data including a sound generation start time, a sound generation end time, sound frequency information, sound intensity information, and timbre identification information, and which can take a plurality of data formats. The acoustic code is taken in from the outside, which data format is recognized, according to each data format, phoneme data acquisition means for extracting each phoneme data, and a waveform shape based on the timbre identification information, from the sounding start time Waveform component generation means for preparing basic distortion waveform data having a length based on the time to the sound ending time; scaling the basic distortion waveform data in the time axis direction based on the frequency information of the sound A time axis expanding / contracting means for applying a magnification to the basic distortion waveform data on the basis of the sound intensity information in an amplitude axis direction; A waveform storage means comprising two types of buffer memories for holding basic distortion waveform data subjected to scaling in the amplitude axis direction, and writing one of the two types of buffer memories to the waveform storage means; A memory mode switching means for defining an exclusive mode in which the other is used for reading and switching these modes at predetermined time intervals; and when the basic distortion waveform data is written to the writing buffer memory, Output waveform synthesizing means for performing processing such as writing to the write buffer memory while adding the already written waveform data to the address range of the write buffer memory corresponding to the start time and the tone generation end time; Reproducing a synthesized signal for outputting the contents of the read buffer memory as a sound signal to the outside in time series Means for synthesizing an acoustic signal. 2. The apparatus according to claim 1, wherein said waveform component generation means performs attenuation such that rising and falling become smooth with respect to the amplitude near both ends of said basic distortion waveform. Sound signal synthesizer. 3. The apparatus according to claim 2, wherein said waveform component generating means includes a tone color definition table, and converts the tone component into a predetermined tone color code by referring to the tone definition table based on the tone color identification information. The audio signal synthesizing device according to claim 1. 4. The waveform component generating means includes a basic waveform definition table, and refers to the basic waveform definition table based on the timbre code to obtain the basic distortion waveform data. The audio signal synthesizing device according to claim 3. 5. The waveform component generating means includes a spectrum table, extracts predetermined spectrum data by referring to the spectrum table based on the timbre code, and synthesizes a sine wave based on the spectrum data. The sound signal synthesizing apparatus according to claim 3, wherein the basic distortion waveform data is obtained by the following. 6. A control information for updating the contents of each of the tone color definition table, the basic waveform definition table, and the spectrum table and table update data as a part of the acoustic code. The audio signal synthesizing device according to any one of claims 3 to 5, wherein: 7. The sound frequency information, the sound intensity information,
2. The apparatus according to claim 1, wherein all or any of the tone color identification information can be set in a plurality of setting modes having different precisions, and the setting mode is switched based on the tone color identification information. A sound signal synthesizing device according to claim 1. 8. The sound frequency information is described by a note number defined by the MIDI standard, the sound intensity information is described by 128 velocities defined by the MIDI standard, and the timbre identification information is described by: The sound generation start time and the sound generation end time are described by 16 kinds of channels defined by the MIDI standard, the sound generation end time is described by the SMF standard delta time, and the phoneme data is described by the MIDI standard note-on event and note-off event. 2. The sound signal synthesizing device according to claim 1, wherein the sound signal is synthesized. 9. The system according to claim 1, wherein said note number, said velocity, and said channel have two types of setting modes, a standard format and an extended format, and said setting mode is switched according to the value of said channel. An apparatus for synthesizing an acoustic signal according to claim 8. 10. Each buffer memory of said waveform storage means has a capacity twice as large as an address read by said synthesized signal reproducing means, and said output waveform synthesizing means writes in all areas of said write buffer memory. By performing such processing, data in an area where the read buffer memory is not read is transferred to the write buffer memory at an address corresponding to the area where the read buffer memory is read. The audio signal synthesizing device according to claim 1.