JP3963141B2 - SINGLE SYNTHESIS DEVICE, SINGE SYNTHESIS PROGRAM, AND COMPUTER-READABLE RECORDING MEDIUM CONTAINING SINGE SYNTHESIS PROGRAM - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a song synthetic method, a song synthesizer, and a program with high naturalness. <P>SOLUTION: Performance data including phonological information, pitch information, etc. are inputted and the performance data are converted into data in a form (song synthetic score) comprehensive by a song synthetic engine based on a song synthetic data base. Compensation is carried out to phonological transition time length data read from a phonological transition data base in a song synthetic data base based on song tone velocity information, song tone intensity information or the pitch information. Thus, the naturalness of song voice to be outputted is enhanced. <P>COPYRIGHT: (C)2004,JPO

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a song synthesizer, a method, and a program for synthesizing a song based on performance data input in real time.
[0002]
[Prior art]
In a conventional singing voice synthesizing apparatus, data obtained from an actual human singing voice is stored as a database as template data, for example, and data that matches the contents of input performance data (notes, lyrics, facial expressions, etc.) is stored. Read from database. Based on the performance data and the template data, data called a singing synthesis score is created.
[0003]
The singing synthesis score is a singing voice data stored in time series for each parameter such as phoneme, pitch (pitch), phoneme transition (attack, release, etc.). For example, phoneme data is stored in a phoneme track, and pitch data is stored in a pitch track. By adding various kinds of sound source control information to the singing synthesis score, the singing voice close to a real person's singing voice is synthesized.
[0004]
In the singing synthesizer, for example, when singing “sa · i · ta (sai)”, not singing each phoneme separately but singing it smoothly between the preceding phoneme and the following phoneme. Insert a phoneme transition part to connect to. In the above example, the phoneme transition portion “ai” is inserted between the phoneme “sa” and the phoneme “i”, and the phoneme transition portion “it” is inserted between the phoneme “i” and the phoneme “ta”. It is. The quality of the singing voice synthesized from the singing synthesizer has been improved year after year by various improvements, but the quality equivalent to that of an actual human singing has not yet been obtained.
[0005]
[Problems to be solved by the invention]
The inventors of the present invention have considered that one of the causes is the use of phonological transition time length data (data defining the temporal length of the phonological transition portion) output from the database.
That is, as a result of analysis by the inventors, it was found that in the case of an actual human song, the time length of the phoneme transition portion (phoneme transition time length) differs depending on the situation of the preceding and following phonemes. For example, if the pitch of the phonemes before and after is the same, but the pitch is different, the actual phonetic singing has a different phonological transition time length (the phonological transition time length is lower in the case of low sounds than in the case of high sounds. Lengthened).
[0006]
However, in the conventional singing voice synthesizing apparatus, the phoneme transition time length is uniformly determined regardless of the situation of the preceding and following phonemes. In view of this point, the present invention provides a singing synthesis method, a singing synthesis device, a singing synthesis method, and a program capable of easily synthesizing a natural singing voice by giving a change to the phoneme transition time length. Objective.
[0007]
[Means for Solving the Problems]
In order to achieve the above object, the singing synthesis method according to the first invention of the present application stores various singing data in a database, and reads out the singing data that matches the contents of the input performance data from the database. In the singing voice synthesis method for generating a score for singing voice synthesis, the performance data includes at least phonological information representing a phoneme, pitch information representing a pitch, and performance sound speed information. Phonological transition database storing phonological transition time length data associated with at least one of phonological information, pitch information, or performance sound speed information When the performance sound speed is near a predetermined standard value, the phonological transition time length expansion / contraction rate decreases monotonously with an increase in the performance sound speed, while the performance sound speed is sufficiently larger or smaller than the predetermined standard value. A phoneme transition time length expansion rate database that stores the relationship between the performance sound speed and the phoneme transition time length expansion rate so that the phoneme transition time length expansion rate is constant regardless of the performance sound speed And the step of inputting the performance data, and comparing the performance data with the phonological transition database, and reading the phonological transition time length data corresponding to the performance data from the database. A reading step; in front The phoneme transition time length data read in the recording phoneme transition time length reading step is By multiplying by the phoneme transition time length expansion / contraction rate obtained by referring to the phoneme transition time length expansion / contraction rate database based on the performance sound speed information And a phonological transition time length data correction step for correction.
[0008]
According to the first aspect, when performance data is input, the performance data and the phoneme transition database are compared, and phoneme transition time length data corresponding to the input performance data is read from the database. The read phoneme transition time length data is corrected by the performance sound speed information in the performance data. Thereby, since the phoneme transition time changes according to the performance sound speed state, the naturalness of the synthesized singing voice is enhanced.
[0009]
To achieve the above object, the singing composition method according to the second invention of the present application stores various singing data in a database, and reads out the singing data that matches the contents of the input performance data from the database. In the singing synthesis method for generating a score for singing synthesis by the above, the performance data includes at least phonological information representing a phoneme, pitch information representing a pitch, and performance tone strength information, and the database includes: A phoneme transition database storing phoneme transition time length data associated with at least one of the phoneme information and the pitch information When the performance sound intensity is near a predetermined standard value, the phoneme transition time length expansion / contraction rate decreases monotonously with an increase in the performance sound intensity, while the performance sound intensity is sufficiently higher than the predetermined standard value. A phoneme transition time length expansion / contraction database that stores the relationship between performance sound strength and phoneme transition time length expansion / contraction rate so that the phoneme transition time length expansion / contraction rate is constant regardless of the performance sound strength when it is large or small When And the step of inputting the performance data, and comparing the performance data with the phonological transition database, and reading the phonological transition time length data corresponding to the performance data from the database. A reading step; in front The phoneme transition time length data read in the recording phoneme transition time length reading step is By multiplying by the phoneme transition time length expansion / contraction rate obtained by referring to the phoneme transition time length expansion / contraction rate database based on the performance sound intensity information And a phonological transition time length data correction step for correction.
[0010]
According to the second aspect, when performance data is input, the performance data and the phoneme transition database are compared, and phoneme transition time length data corresponding to the input performance data is read from the database. The read phoneme transition time length data is corrected by the performance sound intensity information in the performance data. Thereby, since the phoneme transition time changes according to the performance sound intensity state, the naturalness of the synthesized singing voice is enhanced.
[0011]
In order to achieve the above object, the singing composition method according to the third invention of the present application stores various singing data in a database, and reads out the singing data that matches the contents of the input performance data from the database. In the singing composition method for generating a score for singing composition by the above, the performance data includes at least phonemic information representing a phoneme and pitch information representing a pitch, and the database includes the phoneme information, the pitch Phoneme transition database storing phoneme transition time length data corresponding to at least one of the information When the pitch is included in the first section near the predetermined standard value, the phoneme transition time expansion / contraction rate is constant regardless of the change in the pitch, while the pitch is larger or smaller than the first section. , The phoneme transition time length decreases monotonously as the pitch increases, and the pitch is included in the third interval that is larger or smaller than the second interval. A phoneme transition time length expansion rate database that stores the relationship between the pitch and the phoneme transition time length expansion rate so that the phoneme transition time length expansion rate is constant regardless of the pitch change And the step of inputting the performance data, and comparing the performance data with the phonological transition database, and reading the phonological transition time length data corresponding to the performance data from the database. A reading step; in front The phoneme transition time length data read in the recording phoneme transition time length reading step is By multiplying by the phoneme transition time length expansion / contraction rate obtained by referring to the phoneme transition time length expansion / contraction rate database based on the pitch information. And a phonological transition time length data correction step for correction.
[0012]
According to the third aspect, when performance data is input, the performance data and the phoneme transition database are compared, and phoneme transition time length data corresponding to the input performance data is read from the database. The read phoneme transition time length data is corrected by pitch information in the performance data. Thereby, since the phoneme transition time changes according to the state of the pitch, the naturalness of the synthesized singing voice is enhanced.
[0013]
In the first to third inventions, the performance data includes state transition data indicating transitions of the phonological state, and the phonological transition time length data reading step includes different phonological transition times depending on the state transition data. It can also be configured to read long data. In the first to third aspects of the invention, the phoneme transition time length data reading step may be configured to read different phoneme transition time length data depending on the phoneme type. The phonological transition time length correction step may include a step of adding a fluctuation component given by a random number to the corrected phonological transition time length. In addition, these methods may be configured to be executed by a computer using a computer program, or the computer program may be recorded on a recording medium.
[0014]
In order to achieve the above object, the singing voice synthesizing apparatus according to the fourth invention of the present application stores various singing data in a database, and reads out the singing data that matches the contents of the input performance data from the database. In the singing voice synthesizing apparatus for generating a score for singing voice synthesis, the performance data includes at least phoneme information representing a phoneme, pitch information representing a pitch, and performance sound speed information. Phoneme transition database storing phoneme transition time length data corresponding to at least one of the phoneme information and the pitch information When the performance sound speed is near a predetermined standard value, the phonological transition time length expansion / contraction rate decreases monotonously with an increase in the performance sound speed, while the performance sound speed is sufficiently larger or smaller than the predetermined standard value. A phoneme transition time length expansion rate database that stores the relationship between the performance sound speed and the phoneme transition time length expansion rate so that the phoneme transition time length expansion rate is constant regardless of the performance sound speed A phonological transition time for reading out the phonological transition time length data corresponding to the performance data by comparing the performance data and the phonological transition database with a performance data input unit for inputting the performance data A long data reading unit; in front The phoneme transition time length data read in the recording phoneme transition time length reading step is By multiplying by the phoneme transition time length expansion / contraction rate obtained by referring to the phoneme transition time length expansion / contraction rate database based on the performance sound speed information A phonological transition time length data correction unit for correction is provided.
[0015]
In order to achieve the above object, the singing voice synthesizing apparatus according to the fifth invention of the present application stores various singing data in a database, and reads out the singing data that matches the contents of the input performance data from the database. In the singing voice synthesizing apparatus that generates a score for singing voice synthesis, the performance data includes at least phonemic information representing a phoneme, pitch information representing a pitch, and performance sound strength information, and the database includes: A phoneme transition database storing phoneme transition time length data associated with at least one of the phoneme information and the pitch information When the performance sound intensity is near a predetermined standard value, the phoneme transition time length expansion / contraction rate decreases monotonously with an increase in the performance sound intensity, while the performance sound intensity is sufficiently higher than the predetermined standard value. A phoneme transition time length expansion / contraction database that stores the relationship between performance sound strength and phoneme transition time length expansion / contraction rate so that the phoneme transition time length expansion / contraction rate is constant regardless of the performance sound strength when it is large or small When A phonological transition time for reading out the phonological transition time length data corresponding to the performance data by comparing the performance data and the phonological transition database with a performance data input unit for inputting the performance data A long data reading unit; in front The phoneme transition time length data read in the recording phoneme transition time length reading step is By multiplying by the phoneme transition time length expansion / contraction rate obtained by referring to the phoneme transition time length expansion / contraction rate database based on the performance sound intensity information A phonological transition time length data correction unit for correction is provided.
[0016]
In order to achieve the above object, the singing voice synthesizing apparatus according to the sixth invention of the present application stores various singing data in a database, and reads out the singing data matching the contents of the input performance data from the database. In the singing voice synthesizing apparatus that generates a score for singing voice synthesis, the performance data includes at least phonemic information representing a phoneme and pitch information representing a pitch, and the database includes the phoneme information, the pitch Phoneme transition database storing phoneme transition time length data corresponding to at least one of the information When the pitch is included in the first section near the predetermined standard value, the phoneme transition time expansion / contraction rate is constant regardless of the change in the pitch, while the pitch is larger or smaller than the first section. , The phoneme transition time length decreases monotonously as the pitch increases, and the pitch is included in the third interval that is larger or smaller than the second interval. A phoneme transition time length expansion rate database that stores the relationship between the pitch and the phoneme transition time length expansion rate so that the phoneme transition time length expansion rate is constant regardless of the pitch change A phonological transition time for reading out the phonological transition time length data corresponding to the performance data by comparing the performance data and the phonological transition database with a performance data input unit for inputting the performance data A long data reading unit; in front The phoneme transition time length data read in the recording phoneme transition time length reading step is By multiplying by the phoneme transition time length expansion / contraction rate obtained by referring to the phoneme transition time length expansion / contraction rate database based on the pitch information. A phonological transition time length data correction unit for correction is provided.
[0017]
In the fourth to sixth inventions, the performance data includes state transition data indicating transition of the phonological state, and the phonological transition time length data read-out unit has different phonological transition time lengths according to the state transition data. It can be configured to read data. Further, the phoneme transition time length data reading unit may be configured to read different phoneme transition time length data depending on the type of phoneme.
The phonological transition time length correcting step may include a step of adding a fluctuation component given by a random number to the corrected phonological transition time length.
[0018]
DETAILED DESCRIPTION OF THE INVENTION
Embodiments of the present invention will be described below. Below, it demonstrates as what synthesize | combines a Japanese song voice. In the case of Japanese, the phonemes that appear are generally either (1) a combination of consonants and vowels, (2) only vowels, or (3) only voiced consonants (nasal sounds, semi-vowels). However, in the case of (3) voiced consonant only, the singing start timing of the voiced consonant is similar to that in the case of (2) vowel only. Therefore, in this embodiment described below, (3) is (2) It is assumed that the same processing as in (2) is applied.
[0019]
[First Embodiment]
Hereinafter, specific embodiments of the present embodiment will be described.
FIG. 1 shows the overall configuration of a singing voice synthesizing apparatus according to an embodiment of the present invention. It comprises a MIDI (Musical Instrument Digital Interface) device 39, a computer system CS connected to the MIDI device 39 via a MIDI interface 30, a sound source circuit 28, a sound system 38, and the like. The computer system CS includes a CPU 12, a ROM 14, a RAM 16, an operator group 34, a display circuit 22, a display 36, an external storage device 24, a timer 26, and the like.
[0020]
The CPU 12 is a part that controls the entire computer system CS. The ROM 14 stores various programs such as musical tone generation and singing synthesis. The CPU 12 appropriately reads these programs from the ROM 14 and executes various processes.
The RAM 16 is a storage unit for providing a work area for various processes of the CPU 12, and functions as a reception buffer for writing performance data read from the MIDI device 39, for example.
[0021]
The detection circuit 20 detects operation information from an operator group 34 such as a panel, and the display circuit 22 enables various displays by controlling the display operation of the display 36. is there.
The external storage device 24 is a storage device externally attached to the computer system CS for driving a recording medium such as a hard disk, floppy disk, CD, DVD, magneto-optical disk drive, etc. In addition to transferring the stored contents to the RAM 16, In the case of a writable recording medium such as a hard disk, it is also possible to receive data transfer from the RAM 16. This external storage device can also be used as a program recording means instead of the ROM 14.
[0022]
The timer 26 generates the tempo clock signal TCL at a cycle corresponding to the tempo indicated by the tempo data TM. The tempo clock signal TCL is supplied to the CPU 12 as an interrupt command signal. The CPU 12 performs singing synthesis by interruption processing based on the tempo clock signal TCL. The tempo specified by the tempo data TM can be changed by operating the operator group 34 or the like. The period of the tempo clock signal TCL is about 1 ms as an example.
[0023]
The tone generator circuit 28 includes a large number of tone generation channels and a large number of song synthesis channels. The singing synthesis channel synthesizes the singing voice by the formant synthesis method. In the singing voice synthesis process described later, a singing voice signal is generated from the singing voice synthesis channel. The musical sound signal and the singing voice signal related to the generation are converted into voice by the sound system 38. Instead of the formant synthesis method, another method such as a waveform processing method may be used.
[0024]
The MIDI interface 30 is an interface for transmitting / receiving MIDI data to / from a MIDI device 39 separate from the singing voice synthesizing apparatus.
[0025]
Next, with reference to FIG. 2, an outline of a procedure for performing the song synthesis process according to the present embodiment in the configuration of the above-described song synthesis apparatus will be described. In FIG. 2, the blocks 40-48 on the left side show the procedure of the singing synthesis process, and the singing synthesis DB 14A shown on the right side is composed of the ROM 14 and the like, and the performance data input by the MIDI device 39 etc. The part which memorize | stores the various template data for performing the data conversion according to the content is shown.
[0026]
The singing synthesis DB 14A includes, for example, a phoneme DB 14a that stores data related to phonemes, a phoneme transition DB 14b that stores information about phoneme transitions from preceding phonemes to subsequent phonemes, a state transition DB 14c that stores data related to state transitions (attack, release, etc.), A vibrato DB 14d for storing vibrato information is provided.
[0027]
The plurality of arrows indicate what data is read from the song synthesis DB 14A in which procedure.
[0028]
The procedure of this singing synthesis process (steps 40-48) will be described in order. First, performance data is input from the MIDI device 39, and the RAM 16 as a reception buffer receives it (step 40). The contents of the performance data will be described later. As shown in FIG. 3, the performance data S1-S3 are transmitted at times t1, t2, and t3 earlier than the actual singing times T1, T2, and T3.
[0029]
In the following step 42, a singing synthesis score is formed. The singing synthesis score is a data array in which received performance data is expressed in time series by tracks for each parameter such as phoneme and pitch. The content of the singing synthesis score will be described later.
In order to form a singing synthesis score, phonological data corresponding to the received performance data, phonological transition time length data, and the like are acquired from the phonological DB 14a and the phonological transition DB 14b in the singing synthetic DB 14A. Here, the phoneme transition time length data is data indicating the length of transition time between a plurality of phonemes.
[0030]
In step 44, singing is performed by the singing composition engine. In this step, first, sound source control information is acquired from the phoneme DB 14a, the phoneme transition DB 14b, the state transition DB 14c, and the vibrato DB 14d. Here, the sound source control information includes information on formant frequencies necessary for synthesizing a desired singing voice in the sound source circuit 28, information on control parameters at the formant level, and the like.
And based on this acquired sound source control information, singing synthesis score, and performance data, singing is performed, and singing voice signals are output in the order of singing.
[0031]
This singing voice signal is DA-converted by the sound source circuit 28 (step 46), and is output as a singing voice in the sound system 38 (step 48). At this time, as shown in FIG. 3, the generation of the consonant “s” of the singing voice SS1 is started at time T11 earlier than the actual singing time T1, and the generation of the vowel “a” of SS1 is started at the actual singing time T1.
In addition, the consonant “t” of the singing voice SS3 is started to be generated at time T31 earlier than the actual singing time T3, and the vowel “a” is started to be generated at the actual singing time T3. Further, the vowel “i” of the singing voice SS2 is started to be generated at the actual singing time T2. In this way, when the singing voice is composed of a combination of consonants and vowels, the consonants are pronounced prior to the actual singing time. This eliminates the feeling that the singing voice is delayed with respect to the accompaniment and generates a natural singing voice.
[0032]
Next, the contents of the information included in the performance data input in step 40 will be described with reference to FIG. The performance data includes performance information necessary for singing one syllable, specifically note information, phonological track information, and the like.
The note information includes note on information indicating the actual singing start time, duration information indicating the actual singing length, and singing pitch information indicating the singing pitch (also referred to as note and pitch information). , Velocity representing the speed of singing (Velocity) and the like are included. The phoneme track information includes information for forming a phoneme track Tp to be described later, for example, phoneme (PhUState) information representing a singing phoneme, consonant modification information representing a singing consonant expansion / contraction rate, and the like. As described above, in the present embodiment, processing is performed by regarding a phoneme consisting only of (3) voiced consonants (nasal sounds, semi-vowels) as a phoneme consisting only of (2) vowels. The phoneme (PhUState) information includes (2) information of PhUState = Vowel in the case of a phoneme consisting only of vowels, and (1) information of PhUState = Consonant_Vowel in the case of a phoneme consisting of a combination of consonants and vowels. Shall be given.
In addition to this, the performance data may include transition track information for forming the transition track Tr, vibrato track information for forming the vibrato track Tb, and the like.
[0033]
Next, information stored in the phoneme DB 14a will be described with reference to FIG.
As shown in FIG. 5, the phoneme DB 14a stores sound source control information Control11, Control12,... Having different values for each combination of phonemes (PhU) and pitches (Pitch). The data matching the current performance data is read out as appropriate. For example, when the phoneme of the current performance data is a and the pitch (pitch) is P11, Control11 is read as sound source control information and used to form a phoneme track Tp described later.
[0034]
Next, information stored in the phoneme transition DB 14b will be described with reference to FIG. In the phoneme transition DB 14b, for example, as shown in FIG. 6, different phoneme transition time lengths (Duration11, 12...) And sound source control information (corresponding to combinations of the preceding phoneme PhU1, the subsequent phoneme PhU2, and the pitch Pitch) Control11, Control12... Are stored. For example, when the preceding phoneme is a, the subsequent phoneme is i, and the pitch Pitch is P11, values of Duration11 and Control11 are read as phoneme transition time lengths, respectively. Note that, in the columns of PhU1 and PhU2 in FIG. 6, M represents the vowel “U”, Aspiration represents the expiratory sound, and Sil represents the silence.
[0035]
Next, the stored contents of the state transition DB 14c will be described with reference to FIG. The state transition DB 14c stores different state transition time lengths and sound source control information corresponding to combinations of transition states, state types, phonemes, and pitches. Transition states include attack, note transition (hereinafter referred to as “NtN”) and release, and state types include “Normal”, “Sexy”, “Sharp”, “Soft”, etc. is there. The state transition time length indicates the length of the duration of the attack, note transition, or release transition state. The contents of the DB 14c are used to form a transition track Tr of a singing synthesis score to be described later.
[0036]
Next, the stored contents of the vibrato DB 14d will be described with reference to FIG.
The vibrato DB 14d stores different sound source control information for each combination of vibrato type, phoneme, and pitch such as “Normal”, “Sexy”, “Enka”, etc., and is used to form a vibrato track Tb of a singing synthesis score, which will be described later. Is done.
[0037]
FIG. 9 shows an example of the singing voice synthesis score formed in step 42 and an example of the singing voice synthesis in step 44.
The singing synthesis score SC is formed in the RAM 16 and includes a phonological track Tp and a pitch track Ti. In addition, as shown in FIG. 9, a transition track Tr having state transition data, a vibrato track Tb having vibrato information, and the like are provided.
[0038]
For example, “... (sa): C3: T1...” “I (i): D3: T2...” “Ta (ta): E3: T3. As shown in FIGS. 9 and 10, the phoneme track Tp holds items such as Sil, Sil_s, s_a, a... A_Sil, Sil as phoneme information indicating the type of phoneme. Sil indicates silence, and Sil_s indicates phoneme transition from silence to consonant s.
[0039]
As shown in FIG. 10, each phoneme information Sil, Sil_s, s_a, a... Is composed of start time information (Begin Time), duration information (Duration), and phoneme information (PhU). However, the item indicating the phoneme transition portion has two types of phoneme information: preceding phoneme information (PhU1) and subsequent phoneme information (PhU2). The pitch change information 90 as shown in FIG. 9 is stored in the pitch track Ti.
[0040]
Next, the detailed processing content of each step 40-48 shown in FIG. 2 is demonstrated.
[Step 40 (Entering performance data)]
First, the detailed procedure of performance data input in step 40 will be described with reference to FIG.
[0041]
In step 40-1, initialization processing is performed. Here, the reception number counter N in the RAM 16 is set to zero.
[0042]
In step 40-2, n = Nth performance data is received and written in the RAM 16 as a reception buffer.
In Step 40-3, it is determined whether or not the n = Nth performance data is a data end (data end). When it is a data end (YES), it moves to step 40-6 and performs termination processing. If it is not the data end (NO), the process proceeds to step 40-4.
[0043]
In step 40-4, a singing synthesis score is formed based on the received n = Nth performance data. In step 40-5, the value of the counter N is incremented by 1 to set it to N + 1, and the process returns to step 40-2 to receive the next performance data. By repeating until the performance data reaches the data end in this way, the singing synthesis score SC for all the performance data is input to the reception buffer of the RAM 16.
[0044]
[Step 42 (song synthesis score formation)]
Next, the details of the procedure for forming the singing synthesis score in step 42 will be described with reference to FIG.
First, in step 42-1, the CPU 12 receives performance data from the reception buffer of the RAM 16. In step 42-2, the necessary performance data for forming the singing synthesis score is extracted from the received performance data. In step 42-3, management data is created. The management data is various data used to process data in accordance with the contents of the preceding performance data when processing the subsequent performance data. For example, as described later, the management data includes a phonological state (PhUState), a phoneme. Information such as (Phoneme), pitch (Pitch), singing speed (Velocity), current note on (Current Note On), current note duration (Current Note Duration), full duration (Full Duration), event state (EventState) Is included. Details of the contents of information and management data will be described below.
[0045]
Next, in step 42-4, a phoneme track Tp is formed based on the performance data, the management data created in the previous step, and the singing synthesis score formed and stored for the preceding performance data. In the following step 42-5, a pitch track Ti is formed based on the performance data, the management data created in the previous step, and the song synthesis score formed and stored for the preceding performance data.
Similarly, in subsequent steps 42-6 and 7, the transition track Tr and the vibrato track Tb are formed based on the performance data, the management data created in the previous step, and the singing synthesis score formed and stored for the preceding performance data. To do.
In step 42-8, score data for subsequent performance data is formed and stored. When this is completed for all performance data, the singing synthesis score is completed.
[0046]
[Step 42-3 (Create Management Data)]
Next, a detailed procedure for creating management data shown in Step 42-3 in FIG. 12 will be described with reference to FIG.
First, in step 42-3.1, performance data is received, and in step 42-3.2, the characteristics of phonemes included in the performance data are analyzed. Specifically, whether the phoneme included in the performance data is (1) consonant + vowel or (2) only vowel (only nasal sound) is analyzed, and as a result, PhUState is stored. In the case of (1), PhUState = Consonant_Vowel, and in the case of (2), PhUState = Vowel.
[0047]
Next, in step 42-3.3, the pitch in the performance data is analyzed, and as a result, the pitch is stored.
In subsequent steps 42-3.4, the singing speed of the performance data is analyzed, and as a result, the velocity is saved.
In the subsequent step 42-3.5, data of the phoneme transition time length expansion / contraction rate Rpt is acquired based on the velocity. The phoneme transition time length expansion / contraction rate Rpt is prepared for multiplying the phoneme transition time length in order to correct the phoneme transition time length. Details will be described later.
[0048]
The expansion / contraction rate Rpt can be related to the singing sound velocity Velocity, for example, as shown in FIG. 14, and a table showing this relationship can be stored in the ROM 14 or the like. That is, when the singing sound speed Velocity is around the standard value Default shown in FIG. 14, Rpt can be monotonously decreased with respect to the increase in the singing sound speed Velocity. When the singing sound speed Velocity is sufficiently larger or smaller than the standard value Default, Rpt can be made constant regardless of the size of the singing sound speed Velocity. The relationship shown in FIG. 14 is an example, and it goes without saying that a table showing another relationship may be adopted depending on the purpose and situation.
[0049]
In the subsequent step 42-3.6, the actual singing time included in the performance data is analyzed, and the obtained actual singing start time Current Note On is stored (see FIG. 15). At this time, as shown in FIG. 15, the current note On is updated by Δt given by a random number or the like, whereby fluctuations can be given to the singing start time.
[0050]
Also, the actual singing head is the current note duration,
The time elapsed by the actual singing length from the actual singing start time is defined as the current note off (see FIG. 15).
[0051]
In the subsequent step 42-3.7, the actual singing time of the current performance data is analyzed based on the management data. First, the reception number Previous Event Number after rearrangement of the previous performance data, the data of the singing composite score formed and stored for the previous performance data, the previous note-off information indicating the actual singing end time of the previous performance data, etc. To get. Based on these pieces of information, the connection status of the preceding performance data and the current performance data is analyzed, and as a result, the Event State is stored.
As in Case 1 shown in FIG. 15A, when silence is not inserted and the preceding performance data and the current performance data are continuous, Event State = Transition is set. On the other hand, when silence is inserted between the preceding performance data and the current performance data as in Case 2 in FIG. 15B, Event State = Attack.
[0052]
Next, a detailed procedure for forming each track constituting the singing synthesis score SC will be described. However, since the formation process of the transition track Tr, the vibrato track Tb, and the pitch track Ti is not related to the present invention, the description thereof will be omitted, and only the formation process of the phoneme track Tp will be described.
[0053]
[Step 42-4 (Formation of Phoneme Track Tp)]
Next, the detailed procedure of the phoneme track Tp forming process shown in step 42-4 of FIG. 12 will be described with reference to the flowchart shown in FIG.
First, in step 42-4.1, performance data, management data, and a singing synthesis score are received. In the following step 42-4.2, phoneme transition time length data is acquired from the phoneme transition DB 14b based on the management data. The detailed procedure for acquiring the phoneme transition time length data will be described in the next section.
[0054]
Next, in step 42-4.3, it is determined whether or not the Event State in the management data is Attack. If the determination is affirmative (YES), the procedure goes to the procedure for calculating the Silence song length in step 42-4.4. The significance of Silence Singer will be described later. On the other hand, if the determination is negative (NO), that is, if EventState = Transition, the process proceeds to the procedure for calculating the preceding Vowel song length in step 42-4.5. The significance of the preceding Vowel chief will be described later.
When the process of step 42-4.4 or 5 is completed, the process proceeds to the process of calculating the Vowel song length of step 42-4.6. Details will be described later.
[0055]
Next, the detailed procedure of step 42-4.2 (acquisition of phoneme transition time length) described above will be described with reference to the flowchart shown in FIG.
First, management data and singing synthesis score data are received in step 4-2.1. In the following step 4-2.2, all transition time length data (phoneme transition time length acquired in steps 4-2.6, 7, and 9 to 12 described later) stored in the RAM 16 are initialized.
[0056]
Next, in step 4-2.3, the phoneme transition time length of V_Sil (from vowel to silence) is acquired from the phoneme transition DB 14b based on the management data. This procedure is generally necessary because Japanese always ends with a vowel. As an example, if the phoneme of the management data is the vowel “a” and the pitch is P1, the phoneme transition DB 14b obtains the phoneme transition time length corresponding to “a_Sil” and “P1”.
[0057]
Next, in step 4-2.4, it is determined whether EventState = Attack based on the management data. If the determination is affirmative (YES), the process proceeds to step 4-2.5. If the determination is negative (NO), the process proceeds to step 4-2.8.
In Step 4-2.5, it is determined whether or not PhUState = Consonant_Vowel based on the management data. If this determination result is affirmative (YES), the process proceeds to step 4-2.6, and if negative (NO), the process proceeds to step 4-2.11. Similarly, in step 4-2.8, it is determined whether PhUState = Consonant_Vowel based on the management data. If this determination result is affirmative (YES), the process proceeds to step 4-2.9, and if negative (NO), the process proceeds to step 4-2.12.
[0058]
In short, in steps 4-2.4, 4-2.5, 4-2.8,
The phoneme related to the received management data is
(A) When the phoneme is in the rising part (attack) and the phoneme is a combination of consonant and vowel
(B) When the phoneme is in the rising part (attack) and the phoneme is only a vowel (or only a voiced consonant)
(C) When the phoneme is in a transition and the phoneme is a combination of consonants and vowels
(D) The phoneme is in the transition part (transition), and the phoneme is a vowel, which is divided into four cases, each of which obtains necessary data.
[0059]
In the case of (a) above, the process proceeds to step 4-2.6. In step 4-2.6, the phoneme transition time length Silence_Consonant from silence to consonant is acquired from the phoneme transition DB 14b based on the management data. A specific method of acquisition is the same as the method in Step 4-2.3. In the following step 4-2.7, the phoneme transition time length Consonant_Vowel from the consonant to the vowel is acquired from the phoneme transition DB 14b based on the management data. A specific method of acquisition is the same as the method in Step 4-2.3. Thereby, acquisition of the phoneme transition length data in the case of (a) is completed.
[0060]
In the case of the above (b), the process proceeds to Step 4-2.11, and the phoneme transition time length Silence_Vowel from silence to vowel is acquired based on the management data. A specific method of acquisition is the same as the method in Step 4-2.3.
In the case of (c) above, the process proceeds to step 4-2.9. In the case of Japanese, since the phoneme immediately before the transition part is generally a vowel, in step 4-2.9, the phoneme transition length pVowel_Consonant from the preceding vowel to the consonant is obtained based on the management data and the singing synthesis score. A specific method of acquisition is the same as the method in Step 4-2.3. Subsequently, in step 4-2.10, the phoneme transition length Consonant_Vowel from the consonant to the vowel is acquired based on the management data and the consonant data, and acquisition of necessary data is completed.
[0061]
In the case of (d) above, the process proceeds to step 4.2-12, and the phoneme transition time length pVowel_Vowel from the preceding vowel to the vowel is acquired based on the management data. A specific method of acquisition is the same as the method in Step 4-2.3.
When the necessary phoneme transition time length data is obtained in this way, these phoneme transition time length data are corrected based on the singing sound speed Velocity included in the management data (step 4-2.13).
The correction of the phonological transition time length data is performed by multiplying the phonological transition time length data by the phonological transition time length expansion / contraction rate Rpt acquired in Step 42-3.5.
[0062]
The effect of this step 4.2-13 will be described with reference to FIGS. FIG. 18 shows the state of the phoneme track when Step 4.2-13 of this embodiment is performed, and FIG. 19 shows the state of the phoneme track (prior art) when this is not performed.
In the case of the present embodiment shown in FIG. 18, (a) is “sa (sa): C3: V1...” “I (i): D3: V2...” “Ta (ta): E3. : (V3 ...), (b) is "sa: C3: V4 ...""I (i): D3: V5 ...""ta: E3: V6... "Is shown. That is, (a) and (b) have the same phoneme (sa, i, ta) and pitch (C3, D3, E3), but the velocity (singing sound speed) is changed (example: V1 changes to V4) ).
[0063]
In the present embodiment, based on the magnitude of velocity (singing sound speed), the time lengths of the phoneme portion and the phoneme transition portion change as shown in the graph shown in FIG. 14, for example.
For example, paying attention to the phoneme “sa” portion, the velocity in the case of (b) is higher than the case of (a) (V4> V1), so the phoneme transition portions “#_s” and “s_a” The time length is shorter in the case of (b) than in the case of (a). Similarly, paying attention to the “ta” portion, the velocity in the case of (b) is lower than the case of (a) (V6 <V3), so the time of the phoneme transition portions “i_t” and “t_a” portions The length is longer in the case of (b) than in the case of (a).
[0064]
On the other hand, in the case of the prior art shown in FIG. 19, the phonological transition time length does not change even when the velocity is changed, as shown in FIGS. Remains.
[0065]
Next, the method of calculating the Silence song length at step 42-4.4 shown in FIG. 16 will be described with reference to FIGS.
FIG. 20 is a flowchart showing a procedure for calculating the Silence song length, and FIG. 21 explains the concept of the Silence song length.
As shown in FIG. 21, Silence singing chief is EventState = Attack,
The length of the part which forms a part of silence part is shown. That is, the silent time is
(1) Silence part of phonological transition time length from preceding vowel to silence
▲ 2 ▼ Silence Singing Director
(3) Silent part of the phoneme transition time length from silence to consonant or vowel
It consists of three sums.
[0066]
Therefore, the Silence song length can be calculated based on the performance data, the management data, the length of the silent time obtained from the song synthesis score, and the above information (1) and (3). As shown in FIG. 21, the size of the Silence singing length varies depending on the type of phonemes of the preceding phoneme and the succeeding phoneme to be connected.
[0067]
Next, the procedure for calculating the Silence song length will be described with reference to FIG. First, in step 4.4-1, performance data, management data, and singing synthesis score data are received. In the following step 4.4-2, it is determined whether or not the phonological state PhUState of the management data is Consonant_Vowel. If the determination result is affirmative (YES), the process proceeds to step 4.4-3. If the determination result is negative (NO), step 4.4-3 is skipped and the process proceeds to step 4.4-4.
[0068]
In step 4.4-3, the consonant singing time is calculated. The consonant singing time means a time during which the pronunciation of the consonant is continued when the subsequent phoneme includes a consonant. This consonant singing time varies depending on the type of subsequent phonemes. This will be described with reference to FIG.
FIG. 21A shows the state of the phoneme track Tp when the preceding vowel (“A”) — silence—consonant “sa” is pronounced, and FIG. 21B shows the preceding vowel (“A”) — silence— The state of the phoneme track Tp when the consonant “pa” is pronounced is shown in FIG. 10C. The state of the phoneme track Tp when the sound is pronounced as the preceding vowel (“a”) − silence—the subsequent vowel “i” is shown. ing. As can be seen from FIG. 21, in the case of (C), there is of course no consonant singing time. Therefore, if the subsequent phoneme is only a vowel, step 4.4-3 is skipped by step 4.4-2.
[0069]
Next, in step 4.4-4, the size of the Silence song length is calculated. The consonant singing time varies depending on the singing consonant expansion / contraction rate (Consonant Modulation) included in the performance data. FIG. 22 is a diagram for explaining this.
FIG. 22A shows a case where the singing consonant expansion / contraction rate is larger than one. In this case, by adding the sum of the consonant length of the phoneme transition Sil_C from silence to consonant and the consonant length of the phoneme transition C_V from consonant to vowel as the consonant song length, Extend the consonant singing time.
[0070]
On the other hand, FIG. 22B shows a case where the singing consonant expansion / contraction rate is smaller than 1. In this case, the consonant singing time is shortened by multiplying both the consonant length of Sil_C and the consonant length of the phoneme transition C_V from the consonant to the vowel by the singing consonant expansion / contraction rate.
[0071]
[Step 42-4-5 (preceding Vowel song length calculation)]
Next, a method for calculating the preceding Vowel song length in step 42-4-5 will be described in detail with reference to FIGS. The preceding Vowel singing length is the duration of the vowel part of the preceding phoneme (hereinafter referred to as the preceding vowel singing time) when the preceding phoneme and the following phoneme are continuous without any silence (EventState = Transition). ) Means the length of time set to stretch.
[0072]
The portion obtained by subtracting the time during which the consonant portion of the subsequent phoneme is sung (hereinafter referred to as consonant singing time) from the time between the preceding phoneme and the subsequent phoneme is the preceding vowel singing time (see FIG. 24). Therefore, the length of the preceding vowel singing time to be set is determined based on the consonant singing time. In the present embodiment, the length of the preceding vowel singing time is determined by expanding and contracting the preceding Vowel singing length as a part of the preceding vowel singing time.
[0073]
For example, FIG. 24A shows the case where “sa” (combination of consonant and vowel) is pronounced as the subsequent phoneme following the preceding phoneme pV “a”, and FIG. 24B shows the preceding phoneme pV “a”. In the case where the subsequent phoneme “pa” (consonant and vowel combination) is pronounced, the subsequent phoneme “i” (only the vowel) is pronounced after the preceding phoneme pV “a”. Shows the case.
In the former two cases, there is a consonant singing time, but that in (A) is longer than that in (B). For this reason, the preceding vowel singing time is also shorter in the case of (A) than in the case of (B). In the case of (C), since the consonant singing time does not exist, the preceding vowel singing time is maximized.
[0074]
The consonant singing time varies depending on the singing consonant expansion / contraction rate (Consonant Modulation) included in the performance data. FIG. 25 is a diagram for explaining this.
FIG. 25A shows a case where the singing consonant expansion / contraction rate is larger than one. In this case, the sum of the consonant length of the phoneme transition pV_C from the preceding phoneme to the subsequent phoneme (in this figure, a combination of consonant and vowel) and the consonant length of the phoneme transition C_V from consonant to vowel is multiplied by the singing consonant expansion / contraction rate. The consonant singing time is extended by adding the above as the Consonant singer.
On the other hand, FIG. 25B shows a case where the singing consonant expansion / contraction rate is smaller than 1. in this case,
The consonant singing time is shortened by multiplying both the consonant length of pV_C and the consonant length of the phoneme transition C_V from the consonant to the vowel by the singing consonant expansion / contraction rate.
[0075]
The process of calculating the Vowel song length in step 42-4.6 will be described with reference to FIGS. Vowel singer is connected after the vowel of the subsequent phoneme if the assumption is true under the assumption that "silence is inserted between the next performance data" and discarded if it is not true It is what is done.
In order to calculate the Vowel song length, first, performance data, management data, and song synthesis score data are received (step 4.6-1), and the Vowel song length is calculated based on these received data (step 4). .6-2). This calculation method will be described with reference to FIG. First, assume that silence (Sil) is inserted between the current performance data (X_V, where X can be any of silence, consonant, and preceding vowel) and the next performance data (not shown). .
[0076]
Under this assumption, the vowel singing time (vowel singing time) between X_V and silence Sil is as follows: (1) X_V vowel V singing length, (2) Vowel singing length, (3) V_Sil It is the sum of vowel V singing time length. After the boundary between V and Sil in V_Sil matches the actual singing end time (Current Note Off), the Vowel singing length is calculated based on the vowel singing time determined thereby.
[0077]
When the next performance data is received, the connection state (EventState) with the current performance data is found, and it becomes clear whether or not the above assumption is true.
If it is true (EventState = Attack), the calculated Vowel song length is not updated but is used as it is. If it is not true (EventState = Transition), the preceding Vowel song length is calculated in the above-described step 4.5-4.
[0078]
[Second Embodiment]
Next, a second embodiment of the present invention will be described with reference to FIG. In the first embodiment, in the phoneme transition database 14b, phoneme transition time length data is stored in association with the phoneme information PhU1, PhU2 and pitch information Pitch, while the phoneme transition time read from the phoneme transition database 14b is stored. The long data is corrected based on the velocity magnitude. On the other hand, in this embodiment, the phoneme transition length is corrected based on the pitch information Pitch. That is, the correction method in step 4-2.13 shown in FIG. 17 is different, and the other parts are the same as those in the first embodiment.
[0079]
In this embodiment, instead of the table shown in FIG. 14, as shown in FIG. 28, a table showing the relationship between the pitch p and the phoneme transition time length expansion / contraction rate Rpt is stored in the ROM 14 or the like. Then, pitch data is extracted from the performance data, and the phoneme transition time length expansion / contraction rate Rpt corresponding to the extracted pitch data is read from the table shown in FIG. 28 to correct the phoneme transition time length.
Thus, as shown in FIGS. 29 (a) and 29 (b), even with the same phoneme (sa, i, ta) and velocity (V1, V2, V3), if the pitch is different, the phoneme transition time length Changes.
In the table shown in FIG. 28, when the pitch is in the vicinity of the standard value Default2, the phoneme transition time length expansion / contraction rate Rpt is constant regardless of the change in the pitch, and the pitch is sufficiently larger than Default2 or When it is small, Rpt monotonously decreases as the pitch increases, but it goes without saying that it can be appropriately changed according to the purpose of singing synthesis.
[0080]
[Third Embodiment]
Next, a third embodiment of the present invention will be described. In the third embodiment, information on performance sound intensity is held in the performance data, and the phoneme transition length is changed by changing the phoneme transition time length expansion / contraction ratio Rpt based on the magnitude of the performance sound intensity. Is corrected. That is, a table showing the relationship between the performance sound intensity and the phoneme transition time length expansion / contraction rate Rpt as shown in FIG. 30 is stored in the ROM 14 or the like, and the expansion / contraction rate Rpt is calculated based on the read performance sound intensity information. Has been decided. Other points are the same as in the first embodiment.
[0081]
[Fourth Embodiment]
Next, a fourth embodiment of the present invention will be described. In the fourth embodiment, as in the first embodiment, the phoneme transition time length expansion / contraction rate Rpt is changed based on the magnitude of velocity. However, as shown in FIG. 31, the expansion rate fluctuation range is set for the expansion rate Rpt, and even if the velocity is the same, the size of the expansion rate Rpt changes randomly. The Thereby, a change is added to a synthetic singing voice and it can be set as a more natural singing voice.
[0082]
In FIG. 31, the magnitude of the fluctuation range is set to zero when the magnitude of the velocity is Default, and the magnitude of the fluctuation range increases as the difference between the velocity and Default increases. However, as shown in FIG. 32, the magnitude of the fluctuation range may be substantially constant regardless of the magnitude of the velocity. Also, as shown in FIG. 33, the fluctuation range may be maximized when the velocity is in the vicinity of Default, and the fluctuation range may be reduced as the distance from Default is increased.
[0083]
In the fourth embodiment, in the situation where the phoneme transition time length expansion / contraction rate Rpt is changed based on the magnitude of velocity, fluctuation is given to the expansion / contraction rate Rpt, but the present invention is limited to this. It is not a thing. For example, in the situation where the phoneme transition time length expansion / contraction rate Rpt is changed based on the pitch as in the second embodiment, the expansion / contraction rate Rpt can be fluctuated, and the third As in the embodiment, in the situation where the phoneme transition time length expansion / contraction rate Rpt is changed according to the singing sound intensity, the expansion / contraction rate Rpt can be fluctuated.
[0084]
(Modification)
The present invention is not limited to the above-described embodiment, and can be implemented in various modifications. For example, the following changes are possible.
[0085]
(1) In the embodiment described above, the singing voice is synthesized according to the singing voice synthesis score after the singing voice synthesis score is completed. May be synthesized. For this purpose, for example, the singing synthesis score may be formed based on the received performance data while receiving performance data preferentially by interruption processing.
[0086]
(2) In the above-described embodiment, the singing synthesis score is composed of the four tracks of the phoneme track Tp, the pitch track Ti, the transition track Tr, and the vibrato track Tb. However, the number of tracks is not limited to this. For example, the pitch information may be stored in the phoneme track Tp to combine the phoneme track Tp and the pitch track Ti, or all the tracks may be integrated into one track.
[0087]
【The invention's effect】
As described above, according to the present invention, a natural singing voice can be easily synthesized by changing the phoneme transition time length.
[Brief description of the drawings]
FIG. 1 shows an overall configuration of a singing voice synthesis system according to a first embodiment of the present invention.
FIG. 2 shows an outline of a song synthesis procedure in the song synthesis system shown in FIG.
3 is a time-series representation of the results of the song synthesis procedure execution shown in FIG.
FIG. 4 shows the contents of information included in performance data.
FIG. 5 shows the contents of information included in the phoneme DB 14a.
FIG. 6 shows the contents of information in the phoneme transition DB 14b.
FIG. 7 shows the contents of information in the state transition DB 14c.
FIG. 8 shows the contents of information in the vibrato DB 14d.
9 shows an example of the configuration of a song synthesis score SC formed by the song synthesis system shown in FIG.
FIG. 10 is a table for explaining items included in a phonological track Tp of a singing synthesis score SC.
FIG. 11 is a flowchart showing a detailed procedure of step 40 in the flowchart shown in FIG. 2;
12 is a flowchart showing a detailed procedure of step 42 in the flowchart shown in FIG. 2;
13 is a flowchart showing a detailed procedure of step 42-3 (management data creation) in the flowchart shown in FIG.
FIG. 14 shows a table stored in the ROM 14 or the like for correcting the phoneme time transition length.
FIG. 15 is a conceptual diagram illustrating a method of analyzing actual singing time included in performance data.
16 is a flowchart showing a detailed procedure of Step 42-4 (formation of phoneme track Tp) in the flowchart shown in FIG.
FIG. 17 is a flowchart showing a detailed procedure of Step 42-4.2 (acquisition of phoneme transition time length) in the flowchart shown in FIG. 16;
FIG. 18 shows a state of a phoneme track Tp when the correction of the phoneme transition time length in Step 4-2.13 is executed.
FIG. 19 shows a state of a phoneme track Tp when the correction of the phoneme transition time length in Step 4-2.13 is not executed (prior art).
20 is a flowchart showing a detailed procedure of Steps 42-4.4 (Silence song length calculation) in the flowchart shown in FIG.
FIG. 21 is an explanatory diagram showing a method for calculating the Silence song length.
FIG. 22 is an explanatory diagram showing a method for determining a consonant singing time.
FIG. 23 is a flowchart showing a detailed procedure of Step 42-4.5 (calculation of preceding Vowel song length) in the flowchart shown in FIG. 16;
FIG. 24 is an explanatory diagram showing a method for calculating a preceding Vowel song length.
FIG. 25 is an explanatory diagram showing a technique of step 4.5-3 (consonal singing time calculation) in the flowchart shown in FIG. 23;
26 is a flowchart showing a detailed procedure of Steps 42-4.6 (Vowel song length calculation) in the flowchart shown in FIG.
FIG. 27 is an explanatory diagram for explaining a method of calculating a Vowel song length.
FIG. 28 shows a table for correcting the phoneme time transition length in the second embodiment of the present invention.
FIG. 29 shows a state of a phoneme track Tp when the correction of the phoneme transition time length is executed in the second embodiment of the present invention.
FIG. 30 shows a table for correcting the phoneme time transition length in the third embodiment of the present invention.
FIG. 31 shows a table for correcting the phoneme time transition length in the fourth embodiment of the present invention.
FIG. 32 shows a table for correcting the phoneme time transition length in the fourth embodiment of the present invention.
FIG. 33 shows a table for correcting the phoneme time transition length in the fourth embodiment of the present invention.
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 10 ... Bus, 12 ... CPU, 14 ... ROM, 14A ... Singing synthesis DB, 16 ... RAM, 20 ... Detection circuit, 22 ... Display circuit, 24 ... External storage device 26 ... Timer 28 ... Sound source circuit 30 ... MIDI interface 34 ... Operator group 36 ... Display device 38 ... Sound system 39 ... MIDI equipment, Tp ... Phoneme track, Ti ... Pitch track, Tr ... Transition track, Tb ... Vibrato track

Claims (16)

In the singing composition method for storing various singing data in a database and generating a score for singing composition by reading the singing data that matches the content of the input performance data from the database,
The performance data includes at least phoneme information representing a phoneme, pitch information representing a pitch, and performance sound speed information,
The database includes a phonological transition database storing phonological transition time length data corresponding to at least one of the phonological information and the pitch information, and a performance sound speed when the performance sound speed is near a predetermined standard value. The phonological transition time length expansion ratio decreases monotonously with respect to the increase, while the phonological transition time length expansion ratio is not related to the performance sound speed when the performance sound speed is sufficiently larger or smaller than a predetermined standard value. A phoneme transition time length expansion / contraction database that stores the relationship between the performance sound speed and the phoneme transition time length expansion / contraction rate to be constant, and
An input step for inputting the performance data;
A phoneme transition time length data reading step of comparing the performance data with the phoneme transition database and reading the phoneme transition time length data corresponding to the performance data from the database;
Multiplying the previous SL phoneme the phoneme transition time length data read in the transition time length read step, phoneme transition time length scaling factor obtained with reference to the phoneme transition time length scaling factor database on the basis of the performance sound speed information A singing synthesizing method comprising: a phonological transition time length data correcting step for correcting the phonological transition time. In the singing composition method for storing various singing data in a database and generating a score for singing composition by reading the singing data that matches the content of the input performance data from the database,
The performance data includes at least phoneme information representing a phoneme, pitch information representing a pitch, and performance sound strength information,
The database includes a phonological transition database storing phonological transition time length data corresponding to at least one of the phonological information and the pitch information, and a performance sound when the performance sound intensity is near a predetermined standard value. While the phoneme transition time length expansion rate decreases monotonically with increasing strength, while the performance tone strength is sufficiently larger or smaller than a predetermined standard value, the phoneme transition time length expansion rate is A phoneme transition time length expansion / contraction rate database that stores the relationship between the performance sound strength and the phoneme transition time length expansion / contraction rate, which is constant regardless of the length ,
An input step for inputting the performance data;
A phoneme transition time length data reading step of comparing the performance data with the phoneme transition database and reading the phoneme transition time length data corresponding to the performance data from the database;
Before Symbol phoneme the phoneme transition time length data read in the transition time length read step, phoneme transition time length scaling factor obtained with reference to the phoneme transition time length scaling factor database on the basis of the performance sound intensity information A singing synthesis method comprising: a phonological transition time length data correcting step for correcting by multiplication . In the singing composition method for storing various singing data in a database and generating a score for singing composition by reading the singing data that matches the content of the input performance data from the database,
The performance data includes at least phoneme information representing a phoneme and pitch information representing a pitch,
The database includes a phoneme transition database in which phoneme transition time length data is stored in association with at least one of the phoneme information and the pitch information, and a pitch is included in a first section near a predetermined standard value. In some cases, the phonological transition time expansion rate is constant regardless of the change in pitch, while the pitch is included in the second interval that is larger or smaller than the first interval, When the phoneme transition time length decreases monotonically and the pitch is included in the third section that is larger or smaller than the second section, the phoneme transition time length expansion / contraction rate is set regardless of the change in the pitch. A phoneme transition time length expansion / contraction rate database that stores the relationship between pitches that are constant and phonological transition time length expansion / contraction rates ,
An input step for inputting the performance data;
A phoneme transition time length data reading step of comparing the performance data with the phoneme transition database and reading the phoneme transition time length data corresponding to the performance data from the database;
Multiplying the previous SL phoneme the phoneme transition time length data read in the transition time length read step, phoneme transition time length scaling factor obtained with reference to the phoneme transition time length scaling factor database on the basis of the pitch data singing voice synthesizing method characterized in that a phoneme transition time length data correction step of correcting by.   The performance data includes state transition data indicating a transition of the phoneme state, and the phoneme transition time length data reading step is configured to read different phoneme transition time length data according to the state transition data. The singing voice synthesis method according to any one of Items 1 to 3.   5. The singing synthesis method according to claim 1, wherein the phonological transition time length data reading step is configured to read out different phonological transition time length data according to the type of the phoneme.   4. The singing synthesis method according to claim 1, wherein the phonological transition time length correcting step includes a step of adding a fluctuation component given by a random number to the phonological transition time length. 5. A song synthesis program for storing various song data in a database and causing a computer to execute a procedure for generating a score for song synthesis by reading the song data that matches the content of the input performance data from the database. In
The performance data includes at least phoneme information representing phonemes, pitch information representing pitches, and performance sound speed information, and the database corresponds to at least one of the phoneme information and the pitch information. Phonological transition database that stores phonological transition time length data, and when the performance sound speed is near a predetermined standard value, the phonological transition time length expansion rate decreases monotonously with increasing performance sound speed, while the performance sound speed Stores the relationship between the performance sound speed and the phonological transition time length expansion / contraction rate so that the phonological transition time length expansion / contraction rate is constant regardless of the performance sound speed when the value is sufficiently larger or smaller than the predetermined standard value. In the state where the phoneme transition time length expansion / contraction rate database is included,
An input step for inputting the performance data;
A phoneme transition time length data reading step of comparing the performance data with the phoneme transition database and reading the phoneme transition time length data corresponding to the performance data from the database;
Multiplying the previous SL phoneme the phoneme transition time length data read in the transition time length read step, phoneme transition time length scaling factor obtained with reference to the phoneme transition time length scaling factor database on the basis of the performance sound speed information A program for synthesizing a song characterized by causing a computer to execute a phonological transition time length data correction step to be corrected. A song synthesis program for storing various song data in a database and causing a computer to execute a procedure for generating a score for song synthesis by reading the song data that matches the content of the input performance data from the database. In
The performance data includes at least phoneme information representing a phoneme, pitch information representing a pitch, and performance sound strength information, and the database includes at least one of the phoneme information and the pitch information. Corresponding phoneme transition database storing phoneme transition time length data, and when the performance sound intensity is near the predetermined standard value, while the phoneme transition time length expansion / contraction rate decreases monotonously with increasing performance sound intensity Performance sound strength and phoneme transition time length expansion / contraction rate so that the phoneme transition time length expansion / contraction rate is constant regardless of the performance sound strength when the performance sound strength is sufficiently larger or smaller than a predetermined standard value In a state that includes a phoneme transition time length expansion / contraction rate database that stores the relationship between
An input step for inputting the performance data;
A phoneme transition time length data reading step of comparing the performance data with the phoneme transition database and reading the phoneme transition time length data corresponding to the performance data from the database;
Before Symbol phoneme the phoneme transition time length data read in the transition time length read step, phoneme transition time length scaling factor obtained with reference to the phoneme transition time length scaling factor database on the basis of the performance sound intensity information A singing composition program characterized by causing a computer to execute a phoneme transition time length data correcting step for correcting by multiplication . A song synthesis program for storing various song data in a database and causing a computer to execute a procedure for generating a score for song synthesis by reading the song data that matches the content of the input performance data from the database. In
The performance data includes at least phoneme information representing a phoneme and pitch information representing a pitch, and the database associates at least one of the phoneme information and the pitch information with phoneme transition time length data. When the pitch is included in the first interval near the predetermined standard value, the phoneme transition time expansion / contraction rate is constant regardless of the change in pitch, while the pitch is the first When included in a second section that is larger or smaller than the section, the phonological transition time length monotonously decreases with an increase in pitch, and the pitch is larger or smaller than the second section. When included in the third section, the phoneme transition time length expansion / contraction rate stores the relationship between the pitch and the phoneme transition time length expansion / contraction rate so that the phoneme transition time length expansion / contraction rate is constant regardless of the change in pitch. In the state that includes the database ,
An input step for inputting the performance data;
A phoneme transition time length data reading step of comparing the performance data with the phoneme transition database and reading the phoneme transition time length data corresponding to the performance data from the database;
Multiplying the previous SL phoneme the phoneme transition time length data read in the transition time length read step, phoneme transition time length scaling factor obtained with reference to the phoneme transition time length scaling factor database on the basis of the pitch data A program for synthesizing a song characterized by causing a computer to execute a phoneme transition time length data correction step to be corrected by this. In a song synthesizer that stores various song data in a database and generates a score for song synthesis by reading the song data that matches the content of the input performance data from the database.
The performance data includes at least phoneme information representing a phoneme, pitch information representing a pitch, and performance sound speed information,
The database includes a phonological transition database storing phonological transition time length data corresponding to at least one of the phonological information and the pitch information, and a performance sound speed when the performance sound speed is near a predetermined standard value. The phonological transition time length expansion ratio decreases monotonously with respect to the increase, while the phonological transition time length expansion ratio is not related to the performance sound speed when the performance sound speed is sufficiently larger or smaller than a predetermined standard value. A phoneme transition time length expansion / contraction database that stores the relationship between the performance sound speed and the phoneme transition time length expansion / contraction rate to be constant, and
A performance data input unit for inputting the performance data;
A phoneme transition time length data reading unit that compares the performance data with the phoneme transition database and reads the phoneme transition time length data corresponding to the performance data from the database;
Multiplying the previous SL phoneme the phoneme transition time length data read in the transition time length read step, phoneme transition time length scaling factor obtained with reference to the phoneme transition time length scaling factor database on the basis of the performance sound speed information A singing synthesizing apparatus comprising: a phonological transition time length data correcting unit that corrects the phonological transition time. In a song synthesizer that stores various song data in a database and generates a score for song synthesis by reading the song data that matches the content of the input performance data from the database.
The performance data includes at least phoneme information representing a phoneme, pitch information representing a pitch, and performance sound strength information,
The database includes a phonological transition database storing phonological transition time length data corresponding to at least one of the phonological information and the pitch information, and a performance sound when the performance sound intensity is near a predetermined standard value. While the phoneme transition time length expansion rate decreases monotonically with increasing strength, while the performance tone strength is sufficiently larger or smaller than a predetermined standard value, the phoneme transition time length expansion rate is A phoneme transition time length expansion / contraction rate database that stores the relationship between the performance sound strength and the phoneme transition time length expansion / contraction rate, which is constant regardless of the length ,
A performance data input unit for inputting the performance data;
A phoneme transition time length data reading unit that compares the performance data with the phoneme transition database and reads the phoneme transition time length data corresponding to the performance data from the database;
Before Symbol phoneme the phoneme transition time length data read in the transition time length read step, phoneme transition time length scaling factor obtained with reference to the phoneme transition time length scaling factor database on the basis of the performance sound intensity information A singing synthesizing apparatus comprising: a phonological transition time length data correcting unit that corrects by multiplication . In a song synthesizer that stores various song data in a database and generates a score for song synthesis by reading the song data that matches the content of the input performance data from the database.
The performance data includes at least phoneme information representing a phoneme and pitch information representing a pitch,
The database includes a phoneme transition database in which phoneme transition time length data is stored in association with at least one of the phoneme information and the pitch information, and a pitch is included in a first section near a predetermined standard value. In some cases, the phonological transition time expansion rate is constant regardless of the change in pitch, while the pitch is included in the second interval that is larger or smaller than the first interval, When the phoneme transition time length decreases monotonically and the pitch is included in the third section that is larger or smaller than the second section, the phoneme transition time length expansion / contraction rate is set regardless of the change in the pitch. A phoneme transition time length expansion / contraction rate database that stores the relationship between pitches that are constant and phonological transition time length expansion / contraction rates ,
A performance data input unit for inputting the performance data;
A phoneme transition time length data reading unit that compares the performance data with the phoneme transition database and reads the phoneme transition time length data corresponding to the performance data from the database;
Multiplying the previous SL phoneme the phoneme transition time length data read in the transition time length read step, phoneme transition time length scaling factor obtained with reference to the phoneme transition time length scaling factor database on the basis of the pitch data A singing voice synthesizing apparatus comprising a phonological transition time length data correcting unit that corrects the phonological transition time. The performance data includes state transition data indicating state transition of the phoneme,
The singing synthesizer according to any one of claims 10 to 12, wherein the phonological transition time length data reading unit is configured to read different phonological transition time length data according to the state transition data.   The singing voice synthesizing apparatus according to any one of claims 10 to 13, wherein the phonological transition time length data reading unit is further configured to read different phonological transition time length data according to the type of the phoneme.   The singing voice synthesizing apparatus according to any one of claims 10 to 13, wherein the phonological transition time length correcting step includes a step of adding a fluctuation component given by a random number to the phonological transition time length.   A computer-readable recording medium on which the song synthesis program according to claim 7 is recorded.

Images