JP4296767B2 - Breath sound synthesis method, breath sound synthesis apparatus and program - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a breath sound synthesizing method, a breath sound synthesizing apparatus, and a program suitable for use in audio equipment.
[0002]
[Prior art]
It is a well known technique to correct voice data such as singing sound or sequence data on a personal computer. Therefore, it is considered possible to manually add breath (breathing) sound waveform data or event data to the audio data or the sequence data at an appropriate position.
On the other hand, Patent Document 1 discloses a singing voice synthesizer that automatically synthesizes a singing sound based on performance data including melody data and lyric data for musical instrument performance. It is disclosed that the performance data is appropriately provided with a breath change designation (exhalation flag) for the lyric data, and when this designation is made, a note-off operation is performed on the sound source channel that is sounding.
[0003]
[Patent Document 1]
Japanese Patent No. 3239706 Publication
[Problems to be solved by the invention]
However, in the technique disclosed in Patent Document 1, breath connection designation is performed only by inserting an exhalation flag. The singing sound that is being sounded at the timing of the exhalation flag is note-off, but there is no disclosure regarding a specific instruction for generating a breathing sound or specification of characteristics. In addition, even in the technique of manually adding a breath sound to audio data, there is a problem that an unnatural singing sound can be heard if the same breath sound is reproduced in the same way regardless of the length or weakness of the phrase. On the other hand, in order to synthesize a natural singing sound, manually specifying the timing, length, and strength of the breath each time is extremely complicated.
The present invention has been made in view of the above-described circumstances, and an object thereof is to provide a breath sound synthesis method, a breath sound synthesis device, and a program that allow a user to produce a song without being conscious of breath. And
[0005]
[Means for Solving the Problems]
In order to solve the above problems, the present invention is characterized by having the following configuration. The parentheses are examples.
2. The breath sound synthesizing method according to claim 1, wherein the first and second sound generation periods, which are successively generated in each of the continuous sound generation periods (phrases), are detected from the audio data or the sequence data. And a breath to insert a breath sound or a breath event for generating the breath sound at a timing after the end of the first sound generation period and before the start of the second sound generation period in the audio data or the sequence data. The waveform of the breath sound has a larger amount of breath as the continuous waveform intensity (phrase intensity) related to the second sound generation period is higher, and the longer the continuous waveform length (phrase length) is, a waveform which is set such that the breath volume is increased, breath length is the length of the breath sound, the second sound production period is started after the first sound period ends In the period of up to, which is set within a range in which the breath sound except a predetermined distance from the time to end to said second sound period starts (T _BP2), the strength of the breath sound blow strength is, the higher the breath volume is large large and is intended for the breath length is calculated to be smaller the longer the start timing of the breath sound, and the breath length, said predetermined interval ( And T _BP2 ).
Furthermore, in the configuration according to claim 2, in the breath sound synthesizing method according to claim 1, according to the continuous waveform strength (phrase strength) and the continuous waveform length (phrase length) according to the second pronunciation period. The method further comprises a breath sound insertion determination process for determining whether to insert the breath sound or the breath event according to the determined value of the breath amount.
The breath sound synthesizing apparatus according to claim 3 is characterized in that the breath sound synthesizing method according to claim 1 or 2 is executed.
The program according to claim 4 causes the processing device (50) to execute the breath sound synthesis method according to claim 1 or 2.
[0006]
DETAILED DESCRIPTION OF THE INVENTION
1. Configuration of Embodiment A configuration of a breath sound synthesizer according to an embodiment of the present invention will be described with reference to FIG.
In the figure, reference numeral 10 denotes a liquid crystal display panel, which displays data file designation and displays a real-time reproduction waveform for the data file. An I / O control unit 20 communicates audio data and MIDI data (sequence data) through a MIDI interface and a USB interface. A pointing device for performing various settings is included. Reference numeral 30 denotes an FDD control unit, which exchanges STANDARD_MIDI_FORMAT files and audio files via a flexible disk. Reference numeral 40 denotes an HDD control unit, which stores temporary files and parameters. Reference numeral 50 denotes a CPU which controls each unit. Reference numeral 60 denotes a ROM which stores a program for driving the CPU 50. A RAM 70 is used as a work memory. Reference numeral 90 denotes a bus line, which connects each part. The breath sound synthesizing apparatus 100 is configured as described above.
[0007]
2. The motion breath sound synthesizing apparatus 100 according to the embodiment supports both voice data that is real-time amplitude data and sequence data that is a set of event data. Hereinafter, processing for each data will be described separately.
(1) Operation with Voice Data The operation of the voice data of the breath sound synthesizer 100 will be described with reference to the flowchart of FIG. The routine of FIG. 3 is started by performing a predetermined operation.
In step SP101, the entire audio data file including information on the tune, tempo, and tune data is read into the hard disk. Reading is performed via a USB interface or a flexible disk. Here, FIG. 2A shows a part of a real-time reproduction waveform for the read audio data. A period of a continuous waveform that is continuously generated at a breath is called a phrase, and there are a plurality of phrases in time series. Also, there is a blank period between the previous phrase and the subsequent phrase, and a target breath sound is inserted in the blank period. Note that a phrase that is pronounced at a breath is expressed as “slur” in the staff. Then, after the reading is completed, the process proceeds to step SP103.
[0008]
In step SP103, a phrase detection process is performed. That is, the read audio data file is converted into amplitude data, and it is determined that a period of a continuous waveform, that is, a period in which a signal level equal to or higher than a predetermined threshold is continuous is a phrase. This process is performed for the entire read audio file. And a process progresses to step SP105 and a phrase analysis is performed.
[0009]
In step SP105, the data of the nth phrase and the data of the (n + 1) th phrase are read into the work memory, and the phrase interval is detected. Here, as shown in FIG. 2A, the phrase interval (PHRASE_INTERVAL) refers to the interval between the previous phrase and the subsequent phrase. In particular, in the first process, n = 1 is set, and the phrase interval between the first phrase and the second phrase is detected. Then, the process proceeds to step SP107. In step SP107, the subsequent phrase length (PHRASE_LENGTH) and phrase strength (PHRASE_STRENGTH) are detected. Here, the phrase strength is defined by the average value of the volume levels. Then, the process proceeds to step SP109.
[0010]
In step SP109, a breath amount calculation process is performed. The breath amount (BREATH_POWER) is given by a function of the phrase length (PHRASE_LENGTH) and the phrase strength (PHRASE_STRENGTH) of the subsequent phrase, and specifically, is given by the product of the phrase length and the phrase strength. In other words, the longer the phrase behind or the stronger the phrase, the greater the amount of breath required. Then, the process proceeds to step SP111.
[0011]
In step SP111, it is determined whether or not to insert a breath sound. If the breath amount (BREATH_POWER) is less than the predetermined threshold value, it is not necessary to add a breath sound and it is determined “NO”, and the process proceeds to step SP117. In step SP117, the phrase is updated. That is, the value of n is increased and the data for the next phrase is read into the work memory. Then, the process returns to step SP105. On the other hand, if the amount of breath is equal to or greater than a predetermined threshold, it is necessary to add a breath sound, it is determined “YES”, and the process proceeds to step SP113.
[0012]
In step SP113, processing for calculating the breath strength, the breath length, and the breath timing is performed. Here, the breath length (BREATH_LENGTH) is a function of the phrase interval (PHRASE_INTERVAL) and the time T _BP2 (see FIG. 2B) from the breath end time to the start time of the subsequent phrase. Here, the value of T _BP2 is set to a predetermined value in advance. That is, the time between the end time of the breath and the start time of the subsequent phrase is set at a constant interval, and the breath length is set as large as possible within the range of the phrase interval. However, the maximum value of the breath length is limited to a predetermined value. In particular, when the previous phrase does not exist and the phrase interval is regarded as infinite, the predetermined value is set as the breath length. Note that a time T _BP1 between the end time of the previous phrase and the start time of the breath may be set.
[0013]
The brace strength (BREATH_STRENGTH) is a function of the brace amount (BREATH_POWER) and the brace length (BREATH_LENGTH). The greater the brace amount, the greater the brace strength, and the longer the brace length, the smaller the brace strength (Fig. 2). (See (c)). The breath intensity is equal to the average amplitude of the breath waveform. Here, the breath waveform has a maximum value at the center of the waveform and is symmetrically lowered as it approaches the breath start time and the breath end time (solid line in FIG. 2B).
[0014]
Furthermore, the breath timing (the breath reproduction start time) is a function of the breath length (BREATH_LENGTH) and the time T _BP2 (see FIG. 2B). For example, a time obtained by subtracting the predetermined value and the value of _TBP2 from the start time of the subsequent phrase is set as the breath timing. Then, the process proceeds to step SP114.
[0015]
In step SP114, correction processing based on the tune and tempo is performed. For example, if the music is calm, it is preferable to reduce the brace strength and increase the brace length. On the other hand, if the song has a fast tempo, it is preferable to shorten the breath length and increase the breath strength. Then, the process proceeds to step SP115.
[0016]
In step SP115, it is determined whether or not processing has been completed up to the last phrase. If the process is not completed, “NO” is determined, and the process proceeds to step SP117. In step SP117, the phrase is updated, and the process returns to step SP105. On the other hand, if the processing is completed up to the last phrase, “YES” is determined, and this routine ends.
[0017]
Then, voice data synthesis processing is performed based on the calculated breath intensity, breath length, and breath timing. That is, a breath sound is added to the audio data using the information on the breath intensity and the breath length.
[0018]
(2) Operation for Sequence Data The operation for sequence data (lyric data and melody data) inserted in a MIDI system exclusive message or the like will be described below with reference to FIG.
In step SP101, sequence data is read via a MIDI interface, USB interface, or flexible disk. Here, the lyric data is composed of a plurality of audio sequence data corresponding to each phoneme of the lyric. Further, the audio sequence data is composed of index data for specifying formant data.
[0019]
A specific example of such sequence data is disclosed in Patent Document 1 described above, and the sequence data includes information specifying the note-on timing of each phoneme. However, in Patent Document 1, after note-on occurs for one phoneme, it is a principle that the sounding state of the one phoneme continues until note-on occurs for the next phoneme. Information for directly specifying the note-off timing is not included. On the other hand, in the present embodiment, in order to specify the note-off timing for each phoneme, duration data for specifying the pronunciation time of each phoneme is included. Therefore, the note-off timing of each phoneme is equal to the timing obtained by adding duration data to the corresponding note-on timing.
[0020]
Next, when the process proceeds to step SP103, a “phrase” is detected based on the determined note on / off timing. That is, it is determined that “sections in which the time from the note-off timing of an arbitrary phoneme to the note-on timing of the next phoneme is equal to or less than a predetermined time” constitutes “phrases”. Next, when the process proceeds to step SP105, an interval between the plurality of phrases is detected as a “phrase interval”. The subsequent processing is the same as the processing for “voice data”, and the breath strength, the breath length, the breath timing, and the like of the inserted breath sound are determined based on the interval between phrases, the phrase length, the phrase strength, and the like. When the processing of this routine ends, breath event data for generating these breath sounds is added to the sequence data. If the sound source has a large number of breath sounds, tone color selection information (program change) and note-on information determined by the breath intensity and the breath length may be inserted into the sound file.
[0021]
With the above operation, a natural breath sound is automatically inserted according to the phrase length, phrase strength, and phrase interval, so the user can create audio data and sequence data without being aware of the breath. I can do it.
[0022]
3. Modifications The present invention is not limited to the above-described embodiments. For example, the following various modifications are possible and all fall within the scope of the present invention.
(1) In the above embodiment, the phrase strength is defined by the average strength of the level, but may be defined by the maximum strength and the mean square strength.
(2) In the above embodiment, the brace length is set to a large value within the range of the phrase interval and the brace strength is varied. However, the brace strength may be fixed to a predetermined value and the brace length may be varied. However, when the calculated value of the breath length is longer than the phrase interval, the breath strength is exceptionally set to a value larger than the predetermined value.
(3) In the above embodiment, the breath waveform is set so that it becomes the maximum value at the center of the waveform and decreases symmetrically as it approaches the breath start time and the breath end time (see the solid line in FIG. 2 (b)). A wavy breath waveform (see FIG. 2C) may also be used. When the breath length is shortened, the waveform may be such that the maximum breath value is reached quickly and gradually decreases (see the broken line in FIG. 2B).
(4) In the above embodiment, the entire audio data file is read and breath sound data is inserted as necessary. However, the audio data is read in real time, and the data related to the breath sound is read when the subsequent phrase is read. An inserted audio data file may be generated.
(5) In the above embodiment, the function of executing the breath synthesis method is realized by the program stored in the ROM 60, but the same function can also be realized by an application program running on a personal computer, for example. An audio file obtained by executing this application program or breath sound synthesis method may be stored in a storage medium such as a CD-ROM or a flexible disk and distributed, or distributed through an electric communication line.
[0023]
【The invention's effect】
As described above, according to the present invention,
A breath sound or a breath event is inserted between the first sound generation period and the second sound generation period.
[Brief description of the drawings]
FIG. 1 is a configuration diagram of a breath sound synthesizer according to an embodiment of the present invention.
FIG. 2 is a waveform diagram for real-time reproduction of an audio file and an explanatory diagram thereof.
FIG. 3 is a diagram showing a flowchart.
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 10 ... Liquid crystal display panel, 20 ... I / O control part, 30 ... FDD control part, 40 ... HDD control part, 50 ... CPU, 60 ... ROM, 70 ... RAM, 90 ... Bus line, 100 ... Breath sound synthesizer

Claims (4)

A detection process for detecting first and second sound generation periods, each of which is a continuous sound generation period, sequentially generated from the audio data or the sequence data;
A breath sound insertion process for inserting a breath sound or a breath event for generating the breath sound at a timing after the end of the first sound generation period and before the start of the second sound generation period in the audio data or the sequence data. And
The waveform of the breath sound is a waveform set so that the amount of breath increases as the continuous waveform intensity related to the second sound generation period increases, and the amount of breath increases as the continuous waveform length increases .
The breath length, which is the length of the breath sound, is calculated from the time when the breath sound ends in the period from the end of the first sound generation period to the start of the second sound generation period. It is set within a range excluding a predetermined interval until the period starts,
Blow strength is the intensity of the breath sound, the higher the breath volume is large large and is intended for the breath length is calculated to be smaller as long,
The breath sound start timing is calculated based on the breath length and the predetermined interval.   A breath sound insertion determination process for determining whether to insert the breath sound or the breath event according to the value of the breath amount determined according to the continuous waveform intensity and the continuous waveform length according to the second sound generation period. The breath sound synthesizing method according to claim 1, further comprising:   A breath sound synthesizing apparatus that executes the breath sound synthesizing method according to claim 1.   A program for causing a processing device to execute the breath synthesis method according to claim 1.

Images