JP5157922B2 - Speech synthesizer and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To synthesize natural voice, even when the number of phonemes which are made into a database is reduced than in conventional methods, in the voice synthesis of a phoneme connection system. <P>SOLUTION: Based on musical piece data that indicate notes for composing a musical piece and a lyrics data for indicating its relics, a singing synthesis score (voice synthesis instruction) for indicating a plurality of phonemes used for synthesizing a singing voice of the musical piece, its generation time, and a pitch of its singing voice, is generated. When the voice phoneme indicated by the singing synthesis score in the voice phonemes which are made into the database is selected, for the voice phonemes indicating the transit section from voiced fricative to silence, the voice phoneme in a transit section from the silence to the unvoiced fricative is selected, and it is used for synthesis of singing voice, by reversing the time on the waveform of the voice phoneme. <P>COPYRIGHT: (C)2010,JPO&amp;INPIT

Description

  The present invention relates to a technology for synthesizing speech, and more particularly, to a speech synthesis technology of a unit connection method for synthesizing speech by connecting a plurality of speech units.

  An example of this type of speech synthesis technique is the unit connection type singing synthesis. In the singing synthesis of the unit connection method, speech unit data that defines waveforms of various speech units that are materials of singing speech such as a single phoneme or a transition part from a phoneme to a phoneme may be stored in a database in advance. It is common. And when synthesizing a singing voice that sings a certain lyric according to a certain melody, the speech element data of the speech element constituting the lyric is read from the database, and pitch conversion is performed to match each pitch to the melody. After giving, it connects and synthesize | combines the data which show the waveform of a song voice (refer patent documents 1-3).

JP 2007-240564 A JP 2006-17946 A JP 2000-3200 A

In order to synthesize natural speech by speech synthesis using the unit connection method, it is necessary to store as many speech units as possible in the database, and the type of phoneme (voiced, unvoiced, vowel drop, etc.), before and after Considering the combination of phonemes, voice quality, emotion, etc., the number of speech segments to be stored in a database becomes enormous. For this reason, if a mobile terminal such as a portable game machine, a PDA (Personal Digital Assistants), or a mobile phone is to perform speech synthesis, the number of speech segments stored in a database becomes a big problem. This is because these portable terminals do not have a large-capacity storage device and there is a limit on the size of data that can be stored.
The present invention has been made in view of the above problems, and provides a technology for synthesizing natural speech even when speech units to be databased are fewer than in conventional speech synthesis in the unit connection method. The purpose is to do.

  In order to solve the above problems, the present invention designates a speech unit database for storing speech unit data including waveform data indicating waveforms of various speech units, and a plurality of speech units used for speech synthesis. A means for selecting and outputting speech unit data from the speech unit database in accordance with a speech synthesis instruction in which information is time-sequentially, comprising: a first transition portion from silent to unvoiced friction sound; and silent speech to silent sound When a second transition part is continuously designated by the speech synthesis instruction, one of the first and second transition parts is the corresponding speech element from the speech unit database. Select and output segment data, and for the other, segment selection means for outputting waveform data included in the speech segment data with time reversal, and output from the segment selection means. To provide a speech synthesis device, characterized by having a segment connecting means for outputting the connection while adjusting the waveform data included in each voice segment data that.

  According to such a speech synthesizer, if the speech unit data corresponding to one of the first and second transition parts is stored in the speech unit database, the first and second transitions are performed. It is possible to synthesize speech represented by concatenation of parts. As for the silent friction sound, unlike the plosive sound, even if the waveform data is generated by the time inversion of the waveform as described above, there is no particular influence. That is, according to this speech synthesizer, natural speech synthesis can be performed in the same manner as in the prior art even if one of the first and second transition portions is excluded from the database. it can. In another aspect of the present invention, a program for causing a computer device to function as each of the above-described units (that is, a segment selection unit and a segment coupling unit) may be provided.

  Here, either the first transition portion (that is, the transition portion from silent to silent friction sound) or the second transition portion (the transition portion from the same silent friction sound to silence) is set as a database storage target, or database creation Whether to exclude from the target may be determined in relation to the target language for speech synthesis. For example, when the target language is English, since the appearance frequency of the first transition portion is higher than that of the second transition portion, the first transition portion is set as a storage target of the database (second transition It is preferable to exclude the portion from the storage target). Conversely, when the language in which the appearance frequency of the second transition portion is high is the target language, it is preferable that the second transition portion is the storage target (the first transition portion is excluded from the storage target). Needless to say.

  In order to solve the above problems, the present invention includes a speech unit database for storing speech unit data including waveform data indicating waveforms of various speech units, and a plurality of speech units used for speech synthesis. A means for selecting and outputting speech segment data from the speech segment database in accordance with a speech synthesis instruction in which information to be specified is time-sequentially, wherein a speech segment in a transition portion from an unvoiced friction sound to another consonant is the speech When specified by the synthesis instruction, the speech segment is replaced by connecting the first transition portion from the silent friction sound to silence and the second transition portion from silence to the other consonant, For the transition part, the speech unit data corresponding to the transition part from silence to the silent frictional sound is selected from the speech unit database, and the waveform data included in the speech unit data is time-reversed. A unit selection unit for selecting and outputting corresponding speech unit data from the unit database for the second transition portion, and each speech unit output from the unit selection unit. There is provided a speech synthesizer characterized by comprising a segment coupling means for coupling and outputting waveform data contained in segment data while adjusting.

  According to such a speech synthesizer, the speech segment of the transition portion from the unvoiced friction sound to the other consonant includes the first transition portion from the unvoiced friction sound to the silence and the second transition from the silence to the other consonant. Therefore, even if the speech unit data corresponding to this speech unit is not stored in the speech unit database, the speech including the speech unit can be synthesized. Furthermore, since the speech unit data of the first transition portion is generated by performing time reversal on the speech unit data of the transition portion from silent to the unvoiced friction sound, the speech unit of the first transition portion is generated. Data need not be stored in the speech segment database. That is, according to the speech synthesizer, compared with the conventional speech synthesis of the unit connection method, the speech that needs to be made into a database for the transition portion from the unvoiced friction sound to another consonant and the first transition portion. It becomes possible to reduce fragments. In another aspect of the present invention, a program for causing a computer device to function as each of the above means may be provided.

Hereinafter, an embodiment of the present invention will be described with reference to the drawings.
FIG. 1 is a block diagram showing a configuration of a singing voice synthesizing apparatus which is an embodiment of a voice synthesizing apparatus according to the present invention. This singing voice synthesizing apparatus is obtained by installing a voice synthesizing program in a portable terminal having a function of outputting voice, such as a mobile phone or a portable game machine, and synthesizes singing voice by the above-described unit connection method. In FIG. 1, a CPU (Central Processing Unit) 1 is a control center that controls each part of the singing voice synthesizing apparatus. A ROM (Read Only Memory) 2 is a read only memory storing a control program for controlling basic operations of the singing voice synthesizing apparatus such as a loader. The display unit 3 is, for example, a liquid crystal display and a driving circuit thereof, and displays an operation state of the apparatus, input data, a message to the user, and the like. The operation unit 4 is a means for allowing a user to input various types of information, and includes a plurality of operators (for example, a start button and cursor keys for a portable game machine, a numeric keypad for a mobile phone, etc.), a touch panel, and the like. It consists of The interface group 5 includes a network interface for performing data communication with other devices via a network, an external recording medium such as a UMD (Universal Media Disc) and a CD-ROM (Compact Disk-Read Only Memory). A driver for transferring data between them. The flash memory 6 is a non-volatile memory capable of writing and reading data, and stores information such as various programs and databases. A RAM (Random Access Memory) 7 is a volatile memory used as a work area by the CPU 1. The CPU 1 reads the program in the flash memory 6 into the RAM 7 and executes it in accordance with a command given via the operation unit 4. The sound system 8 is a means for outputting the sound synthesized in this singing voice synthesizing apparatus. The sound system 8 includes a D / A converter that converts a digital audio signal (for example, sampling data that indicates the waveform of the synthesized speech) indicating the waveform of the synthesized speech into an analog audio signal, an amplifier that amplifies the analog audio signal, and And a speaker for outputting the output signal of the amplifier as sound.

  Examples of information stored in the flash memory 6 include a song editing program 61, song data 62, a speech segment database 63, and a speech synthesis program 64. The song data 62 includes note data representing a series of notes constituting the song, lyric data representing lyrics to be generated in accordance with the notes, and other information such as dynamics information for giving a musical expression to the song. Contains. The song data 62 is created for each song and stored in the flash memory 6.

  The song editing program 61 is a program executed by the CPU 1 in order to edit the song data 62. In a preferred embodiment, the song editing program 61 causes the display unit 3 to display a GUI (Graphical User Interface) composed of an image of a piano keyboard. The user can specify a note by operating the operation unit 4 with respect to an image of a desired key on the keyboard displayed on the display unit 3, and can input lyrics to be pronounced according to the note by operating the operation unit 4. Here, the input of lyrics may be performed by kana input as shown in FIG. 2A, or may be performed by phonetic symbol input as shown in FIG. 2B. In this way, the song editing program 61 receives information about the notes and the lyrics that are pronounced according to the notes via the operation unit 4, and the note data and the lyrics data for each note as song data 62 are stored in the flash memory 6. Store in. Further, the user can add dynamic information or the like to the music data 62 by operating the operation unit 4. Instead of inputting all of the music data 62 by operating the operation unit 4 in this way, a note is generated by connecting a keyboard to the singing voice synthesizing device and detecting a user's operation of the keyboard. The lyrics corresponding to the data may be input by operating the operation unit 4. Alternatively, the song data 62 created by another device may be input to the song synthesizing device via the interface group 5 and stored in the flash memory 6. In this embodiment, the song editing program 61 is stored in the flash memory 6. There is no need to keep it.

  The note data corresponding to one note includes information indicating the note generation time, pitch, and note length. The lyric data is data in which lyrics to be pronounced in accordance with the notes are defined for each note. The song data 62 is obtained by arranging note data and lyrics data corresponding to individual notes in time series in accordance with the generation order from the start of the song. In the song data 62, the note data and the lyrics data are arranged. Are associated with each note.

  The voice synthesis program 64 is a program that causes the CPU 1 to perform a process of synthesizing a voice (singing voice in this embodiment) according to the song data 62. In a preferred embodiment, the speech synthesis program 64 and the song editing program 61 are downloaded via a suitable one in the interface group 5 from a site in the Internet, for example, and installed in the flash memory 6. In another aspect, the speech synthesis program 64 or the like is traded in a state recorded on a computer-readable recording medium such as a CD-ROM or UMD. In this aspect, the voice synthesis program 64 and the like are read from the recording medium via an appropriate one in the interface group 5 and installed in the flash memory 6.

  The speech element database 63 includes various speech elements that are materials of singing voices, such as transition parts from consonants to vowels, transition parts from phonemes to phonemes such as transition parts from vowels to other vowels, and vowel extension sounds. Is a collection of speech segment data indicating. These speech segment data are data created based on speech segments extracted from speech waveforms actually emitted by humans. In the speech segment database 63, a group of speech segment data obtained from the singer speech waveform for each singer, such as a male singer, a female singer, a singer with a clear voice, a singer with a husky voice, and the like. It is prepared. At the time of singing synthesis by the speech synthesis program 64, the user selects a group of speech unit data to be used for singing synthesis from among the various groups of speech unit data as described above by operating the operation unit 4. Can do.

  As described above, it is preferable that the speech unit database 63 stores as much speech unit data as possible. However, in this embodiment, since the speech unit database 63 is stored in the flash memory 6, it is necessary to minimize the number of speech unit data stored in the speech unit database 63. This is because the storage capacity of flash memory is generally smaller than that of a hard disk or the like. Therefore, in the present embodiment, the speech unit data of the transition portion from the unvoiced friction sound to another consonant and the speech unit data of the transition portion from the unvoiced friction sound to the silence are excluded from the storage target of the speech unit database ( Note that the above-described narrowing is realized by transitioning from silent to silent frictional sound. The reason why these are excluded from the storage target is as follows.

  In the conventional unit connection type singing synthesis technique disclosed in Patent Document 1 and the like, those indicating a transition part from an unvoiced friction sound to another consonant and a transition part from an unvoiced friction sound to silence are also stored in the speech unit database. ing. This is because the speech segments of these transition portions are used when synthesizing speech with vowels dropped. FIG. 3 (A) is a diagram showing a segment structure of the pronunciation of “su” in which no vowel drop has occurred, and FIG. 3 (B) and FIG. 3 (C) show the drop of the vowel (u), respectively. It is a figure which shows the fragment | piece structure of the pronunciation of "su". As shown in FIGS. 3 (B) and 3 (C), such a vowel drop is caused by the transition from silent to unvoiced friction sound (in FIG. 3 (B), silence to consonant s) and from the unvoiced friction sound. Some are expressed by a continuous transition part to silence, and some are expressed by a transition part from unvoiced friction sound to another consonant (consonant s to consonant t in FIG. 3C). Such omission of vowels is a phenomenon often seen mainly in English. As is clear from FIG. 3B and FIG. 3C, the speech unit database 63 includes speech unit data of a transition portion from an unvoiced friction sound to another consonant and a transition portion from the unvoiced friction sound to the silence. If it is not stored in the, it is not possible to synthesize voices with missing vowels. As shown in FIG. 3C, the transition part from the unvoiced friction sound to another consonant is replaced with a transition part from the unvoiced friction sound to the silence and the transition part from the silence to the other consonant. It is also possible. However, under the situation where the transition from silent frictional sound to silent is not targeted for database, such a method cannot express vowel drop as shown in FIG. In the present embodiment, the occurrence of the above-described problems is avoided by using the speech element data of the transition from silent to silent frictional sound. This point will be described in detail later.

  Each speech unit data stored in the speech unit database 63 includes waveform data indicating the waveform of the speech unit. In the present embodiment, the waveform data is a sample string obtained by sampling the waveform of a speech unit at a predetermined sampling rate, and each speech unit data includes the type of phoneme constituting the speech unit and the start time of each phoneme. Including segmentation data. In this embodiment, a sampling data string is used as the waveform data. However, a frame obtained by dividing a sample string of a speech segment waveform into frames of a certain time length and performing FFT (Fast Fourier Transform). Of course, data representing each spectrum (amplitude spectrum and phase spectrum) may be used as the waveform data.

  In the present embodiment, the singing voice corresponding to an arbitrary melody is synthesized by applying the pitch conversion to the waveform data included in the speech segment data. For this pitch conversion, for example, a method disclosed in Patent Document 1 may be adopted. In order to perform pitch conversion, information on the pitch of the waveform data that is the object is required. Therefore, in a preferred embodiment, for the convenience of pitch conversion at the time of singing synthesis, the pitch of the speech unit is calculated for each frame, and the unit pitch data indicating the pitch in each frame is a part of the speech unit data. It is stored in the speech unit database 63. In another preferred embodiment, for the convenience of calculation processing at the time of singing synthesis, in addition to the above unit pitch data, an envelope of the amplitude spectrum of the speech unit is obtained for each frame, and the spectral envelope in each frame is obtained. Is stored in the speech segment database 63 as a part of the spectrum envelope data indicating.

Next, the configuration of the speech synthesis program 64 will be described.
FIG. 4 is a diagram for explaining the configuration of the speech synthesis program 64. This speech synthesis program 64 is a program that causes the CPU 1 to perform a so-called segment connection type speech synthesis (single synthesis in this embodiment) process, and as shown in FIG. A selection unit 642, a pitch conversion unit 643, and a unit connection unit 644 are included. In the present embodiment, the CPU 1 synthesizes the singing voice by executing a program equivalent to the voice synthesis instruction generating unit 641 and the like, but each of these programs is shared by a plurality of processors and executed in parallel. It may be configured. Further, a part of each program such as the speech synthesis instruction generating unit 641 may be configured by an electronic circuit.

  The voice synthesis instruction generating unit 641 is a program that generates a voice synthesis instruction 650 from the music data 62 specified by the operation of the operation unit 4. The speech synthesis instruction 650 in the present embodiment is a so-called singing synthesis score, and includes a phoneme data track 651, a pitch data track 652, and other data tracks 653. Each of these data tracks has a common time axis. The phoneme data track 651 includes a plurality of speech units used for synthesizing a singing voice for one song, and positions of these speech units on the time axis (specifically, the start timing of the speech unit). Data duration). The pitch data track 652 is a data track indicating the pitch of the singing voice to be synthesized.

  The voice synthesis instruction generating unit 641 basically generates the pitch data track 652 according to the note data, and when there is an instruction of vibrato, portamento or legato. However, if the pitch data track is the same as the note data, the change in pitch will be stepped, resulting in an unnatural singing sound.In this embodiment, the pitch changes in the pitch switching section. Thus, the pitch indicated by the pitch data track 652 is changed. The other data track 653 is created based on the dynamics information included in the song data 62.

  The segment selection unit 642, the pitch conversion unit 643, and the segment connection unit 644 play a role of generating singing voice data that is waveform data indicating the waveform of the singing voice in accordance with the voice synthesis instruction 650. Here, the process of generating the singing voice data from the voice synthesis instruction 650 may be started after the generation of the voice synthesis instruction 650 for one song is completed, and is slightly delayed from the generation start of the voice synthesis instruction 650. It may be started.

  The unit selection unit 642 selects speech unit data corresponding to the speech unit specified in the phoneme data track 651 of the speech synthesis instruction 650 from the speech unit database 63, and the speech unit data is converted into pitch conversion unit. 643 is a program to be output to 643. The unit selection unit 642 in the present embodiment, when delivering the speech unit data to the pitch conversion unit 643, determines the duration of the waveform data included in the speech unit data in the speech unit specified in the speech synthesis instruction 650. It has a function to match the duration of the piece.

  The pitch conversion means 643 is a program that performs pitch conversion on the waveform data included in the speech element data output from the element selection means 642 so that the waveform data corresponds to the pitch specified in the pitch data track 652. is there. More specifically, in this embodiment, since the waveform data included in the speech segment data is a sample string, the pitch converting unit 643 first performs FFT (fast Fourier transform) of the sample string in units of frames each having a predetermined number of samples. ) To obtain the amplitude spectrum and phase spectrum of the speech unit for each frame. Then, the pitch conversion means 643 expands or compresses the amplitude spectrum in each frame in the frequency axis direction so as to correspond to the pitch specified in the pitch data track 652. At that time, in the vicinity of the frequency equivalent to the fundamental tone and the harmonic overtone, nonlinear expansion or compression is performed so that the outline of the original spectrum is maintained, and an amplitude spectrum after pitch conversion is obtained. In addition, after the pitch conversion, the level of the amplitude spectrum that has undergone nonlinear compression or expansion is adjusted so that the spectrum envelope before the pitch conversion is maintained. Regarding the phase spectrum, the original phase spectrum may be used as the phase spectrum after the pitch conversion as it is, but the phase spectrum after the pitch conversion is corrected according to the compression or expansion in the frequency axis direction of the amplitude spectrum. Is preferred. Note that when data representing the amplitude spectrum and phase spectrum of a speech unit is used as the waveform data, it is only necessary to perform expansion or compression in the frequency axis direction without performing the FFT.

  The segment connecting means 644 adjusts the waveform data that has undergone the processing of the pitch converting means 643 so that the finally obtained singing voice is smoothly connected to a series of speech segments, and after this adjustment, This is a program for converting waveform data into a digital audio signal in the time domain by IFFT (Inverse Fast Fourier Transform) and outputting it to the sound system 8.

  Of the components of the speech synthesis program 64 described above, the speech synthesis instruction generation unit 641 (single synthesis score generation unit in this embodiment), the pitch conversion unit 643, and the segment connection unit 644 are disclosed in Patent Document 1. There is no difference from what is disclosed. On the other hand, the segment selection unit 642 of the present embodiment continuously outputs a first transition portion from silent to silent friction sound and a second transition portion from silent friction sound to silence in the speech synthesis instruction 650. If the speech unit of the transition from an unvoiced friction sound to another consonant is specified by the speech synthesis instruction 650, the processing that clearly shows the features of this embodiment is executed. .

  More specifically, when the first transition portion from silent to silent friction sound and the second transition portion from silent friction sound to silence are continuously specified in the speech synthesis instruction 650, the unit selection is performed. For the second transition portion, the means 642 converts the speech unit data corresponding to the first transition portion preceding the second transition portion (that is, the transition portion from silent to unvoiced frictional sound) into speech units. A process of selecting from the database 63 and outputting the waveform data included in the speech segment data that has been time-reversed (hereinafter referred to as silent connection synthesis process) is executed.

  As described above, the case where the first transition portion from silent to silent friction sound and the second transition portion from silent friction sound to silence are continuously designated by the speech synthesis instruction 650 is shown in FIG. In other words, it is instructed to synthesize a voice in which vowels have been dropped in a manner as shown in (B). In this case, the segment selection means 642 first reads out the speech segment data of the transition portion (the first transition portion) from silent to unvoiced frictional sound from the speech segment database 63 and outputs it. Next, the segment selection means 642 obtains the speech segment data obtained by performing time reversal on the speech segment data as speech segment data of the transition portion from the unvoiced friction sound to the silence (that is, the second transition portion). Output (see FIG. 3D). Various specific embodiments of this time reversal are conceivable. For example, when the waveform data included in the speech element data is a sample string, this is realized by rearranging the sample strings in reverse order. Further, when data representing a spectrum (amplitude spectrum and phase spectrum) for each frame is used as the waveform data, time reversal is realized by performing IFFT on the data in reverse order in frame units. As described above, according to the present embodiment, even if the speech segment data of the transition from silent voice to silence is not stored in the speech segment database 63, the type of vowel shown in FIG. Voice can be synthesized.

  On the other hand, when the speech segment of the transition portion from the unvoiced friction sound to another consonant is specified by the speech synthesis instruction 650, the segment selection unit 642 performs the first transition from the unvoiced friction sound to the silence. The speech unit is replaced by connecting the second transition part from the part and the silence to another consonant. For the first transition part, the speech unit data corresponding to the transition part from the silent to the unvoiced friction sound is voiced. While selecting from the unit database 63 and outputting time-reversed waveform data included in the speech unit data, the corresponding speech unit data is selected from the unit database 63 for the second transition portion. (Hereinafter, consonant connection synthesis processing) is executed.

  When the speech segment of the transition part from the unvoiced friction sound to another consonant is designated by the speech synthesis instruction 650, the synthesis of the speech in which the vowel is dropped is instructed in the manner shown in FIG. It means that. In this case, the segment selection means 642 first reads out the speech segment data of the transition portion from silence to unvoiced frictional sound from the speech segment database 63, and applies the time reversal to the speech segment data of the first transition portion. Output as speech segment data. Next, the segment selection means 642 reads the speech segment data of the second transition portion from the segment selection database 63 and outputs it as it is (see FIG. 3E). As described above, according to the present embodiment, the speech unit data of the transition part from the unvoiced friction sound to another consonant and the speech unit data of the transition part from the unvoiced friction sound to the silence are not stored in the speech unit database 63. Even so, it is possible to synthesize a voice in which a vowel of the type shown in FIG.

  As described above, according to the singing voice synthesizing apparatus according to the present embodiment, compared to the prior art, the voice that is created as a database for the transition portion from the silent friction sound to another consonant and the transition portion from the silent friction sound to the silence. It is possible to synthesize natural singing voice while reducing the number of segments.

Although one embodiment of the present invention has been described above, the present embodiment may of course be modified as follows.
(1) In the above-described embodiment, the silent connection synthesis process and the consonant connection synthesis process are caused to be executed by the segment selection unit 642, so that the transition from the silent friction sound to the silence and the silent friction sound to another consonant. Therefore, it is not necessary to create a database of speech units indicating the transition part, and the number of speech units stored in the speech unit database 63 is reduced. However, only one of the two processes that clearly show the features of the present embodiment may be executed by the segment selection unit 642. For example, in an aspect in which only silent connection synthesis processing is executed, it is not necessary to create a database for the transition from silent friction to silence, but a database for the transition from silent friction to other consonants is created in the same manner as before. It is necessary to keep. However, even in such an embodiment, the speech elements to be stored in the speech unit database are compared with the speech synthesis technology of the conventional unit connection method by the amount that the transition from silent friction sound to silence is not made into a database. The number of pieces can be reduced. Similarly, it is of course possible to cause the element selection means 642 to execute only consonant connection synthesis processing and to create a database in advance for the transition from silent friction sound to silence.

(2) In the above-described embodiment, the speech segment data of the transition portion from silent to unvoiced friction sound is stored in the speech segment database 63, and the speech segment data of the transition portion from unvoiced friction sound to silence is stored from the storage target. Although excluded, the speech unit data of the transition part from silent friction sound to silence may be stored in the speech unit database 63, and the speech unit data of transition part from silent to unvoiced friction sound may be excluded from the storage target. . However, in this aspect, when the first transition part from silent to silent frictional sound and the second transition part from silent silent sound to silent are continuously specified by the speech synthesis instruction 650, It is necessary to cause the segment selection means 642 to execute the silent connection synthesis processing of the aspect. That is, for the first transition portion, speech segment data corresponding to the transition portion from silent friction sound to silence (that is, the second transition portion) is selected from the speech segment database 63, and the speech segment data is selected. The unit selection unit 642 is made to execute a process of outputting the waveform data included in the waveform data subjected to time reversal.

  Here, either the first transition part (that is, the transition part from silent to silent frictional sound) or the second transition part (the transitional part from silent frictional sound to silent) is set as the storage target of the speech unit database 63, or Whether to exclude it from the database should be determined in relation to the target language for speech synthesis. For example, when the target language is English, since the appearance frequency of the first transition portion is higher than that of the second transition portion, the first transition portion is set as a storage target of the database (second transition It is preferable to exclude the portion from the storage target). Conversely, when the language in which the appearance frequency of the second transition portion is high is the target language, it is preferable that the second transition portion is the storage target (the first transition portion is excluded from the storage target). Needless to say.

(3) For the pitch data track 652, movement may be given to the pitch of the note switching portion by a method other than that described in the above embodiment. For example, a configuration in which the user moves the pitch by operating the operation unit 4 may be used.

(4) In the embodiment described above, lyrics can be input by either kana input or phonetic symbol input. For example, when lyrics are input by kana input, there is always no vowel dropout. In the case where the singing is performed and the lyrics are input by inputting the phonetic symbols, the presence or absence of vowel dropout may be controlled according to the input content. Further, in the case of kana input, a control character for instructing omission of vowels may be given to the omission location to instruct that effect. For example, when “%” is used as a control character for instructing omission of vowels, and “voicing” is instructed in the form shown in FIG. 3B, “su%” is input, and FIG. In order to instruct dropping of the vowel in the mode shown, it is only necessary to input “%”.

(5) The segment connecting means 644 converts the waveform data (amplitude spectrum, phase spectrum), which is frequency domain information, into a digital audio signal, which is time domain information, and then performs a smoothing process on the digital audio signal. You may make it do. For example, the final n waveform data of the preceding speech unit and the first n waveform data of the subsequent speech unit are subjected to crossfading for the time domain digital speech signal obtained by IFFT, and the final It may be a digital audio signal.

(6) The voice synthesis instruction may be time series information of the voice synthesis parameter, and is not limited to one piece of music, but may be time series information of one part of the piece of music.

(7) In the embodiment described above, the speech unit database 63 composed of a set of speech unit data of each speech unit that is a material for speech synthesis is stored in the flash memory 6 of the singing synthesis device (speech synthesis device). It was. However, the speech unit database 63 is written and distributed in a computer device recording medium such as an SD memory or UMD, and the speech unit database 63 stored in these recording media is accessed via the interface group 5 to generate speech. You may make CPU1 perform the process which reads the speech unit data corresponding to the speech unit designated by the synthetic | combination instruction | indication 650. FIG. In such an aspect, the size of the storage capacity of the storage device included in the singing voice synthesis device (voice synthesis device) does not become a problem. However, since the storage capacity of a UMD or SD memory is generally smaller than that of a hard disk or the like, it is sufficiently meaningful to reduce the number of speech segments to be databased by the method described in the above embodiment.

(8) In the above-described embodiment, the present invention is applied to synthesis of singing voice (that is, voice sung in accordance with a song), but the present invention is also applied to synthesis of voice other than singing voice such as speech voice. Of course it is good. Even in the case of speech speech synthesis, the speech synthesis instruction generation unit 641 executes processing for generating a speech synthesis instruction in which information for designating a plurality of speech units used for speech synthesis is time-sequentially, and unit selection unit Each speech unit data output from 642 is subjected to pitch conversion by the pitch conversion unit 643, and the unit connection unit 644 performs processing of connecting and outputting the waveform data after the pitch conversion while adjusting the waveform data. Just do it.

(9) In the above-described embodiment, by operating the CPU 1 according to the speech synthesis program 64, the CPU 1 is used as the speech synthesis instruction generation unit 641, the unit selection unit 642, the pitch conversion unit 643, and the unit connection unit 644. Made it work. However, in a mode in which the speech synthesis instruction 650 is given from another device via an appropriate interface group 5, it is not necessary for the CPU 1 to function as the speech synthesis instruction generation unit 641. Further, in a mode in which the pitch is not designated for each speech unit constituting the synthesized speech, it is not necessary for the CPU 1 to function as the pitch conversion unit 643. That is, in order to realize the speech synthesis characteristic of the present invention, the speech synthesis instruction generation unit 641 and the pitch conversion unit 643 are not necessarily required, and the CPU 1 is caused to function as the segment selection unit 642 and the segment connection unit 644. I hope you can.

It is a figure which shows the structural example of the song synthesizing | combining apparatus which is one Embodiment of this invention. It is a figure which shows an example of the input mode of pronunciation content (lyrics). It is a figure for demonstrating an example of the speech element from which the vowel was dropped, and the principle of this embodiment. It is a figure for demonstrating the structure of the speech synthesis program 64 stored in the flash memory 6 of the song synthesizing apparatus.

  DESCRIPTION OF SYMBOLS 1 ... CPU, 2 ... ROM, 3 ... Display part, 4 ... Operation part, 5 ... Interface group, 6 ... Flash memory, 61 ... Music editing program, 62 ... Music data, 63 ... Speech unit database, 64 ... Speech synthesis Program, 641... Speech synthesis instruction generation means, 642... Segment selection means, 643... Pitch conversion means, 644.

Claims (4)

A speech unit database for storing speech unit data including waveform data indicating waveforms of various speech units;
A means for selecting and outputting speech segment data from the speech segment database in accordance with a speech synthesis instruction time-sequentially specifying information for specifying a plurality of speech segments used for speech synthesis, from silent to silent friction sound When the first transition part and the second transition part from the silent frictional sound to the silence are successively specified by the speech synthesis instruction, any one of the first and second transition parts A unit selecting means for selecting and outputting corresponding speech unit data from the speech unit database for one, and outputting time-reversed waveform data included in the speech unit data for the other; ,
A speech synthesizer comprising: a unit linking unit that adjusts and outputs waveform data included in each speech unit data output from the unit selection unit. A speech unit database for storing speech unit data including waveform data indicating waveforms of various speech units;
A means for selecting and outputting speech unit data from the speech unit database in accordance with a speech synthesis instruction in which information specifying a plurality of speech units used for speech synthesis is time-sequentially. When the speech segment of the transition portion to is designated by the speech synthesis instruction, the first transition portion from the silent friction sound to silence and the connection of the second transition portion from silence to the other consonant And the speech unit data corresponding to the transition from silent to unvoiced frictional sound is selected from the speech unit database and included in the speech unit data. A unit selection unit for outputting waveform data subjected to time reversal, and selecting and outputting corresponding speech unit data from the unit database for the second transition portion;
A speech synthesizer comprising: a unit linking unit that adjusts and outputs waveform data included in each speech unit data output from the unit selection unit. Computer equipment,
From a speech unit database that stores speech unit data including waveform data indicating waveforms of various speech units, according to a speech synthesis instruction in which information specifying a plurality of speech units used for speech synthesis is time-series A means for selecting and outputting speech segment data, wherein a first transition portion from silence to silent friction sound and a second transition portion from silent friction sound to silence are successively designated by the speech synthesis instruction. If so, for either one of the first and second transition parts, the corresponding speech unit data is selected and output from the speech unit database, and for the other, the speech unit data is selected. Unit selection means for outputting the waveform data included in the time-reversed,
A program which functions as segment connecting means for connecting and outputting waveform data included in each speech segment data output from the segment selecting means while adjusting. Computer equipment,
From a speech unit database that stores speech unit data including waveform data indicating waveforms of various speech units, according to a speech synthesis instruction in which information specifying a plurality of speech units used for speech synthesis is time-series Means for selecting and outputting speech segment data, and when a speech segment of a transition portion from an unvoiced friction sound to another consonant is specified by the speech synthesis instruction, a second unit from the unvoiced friction sound to the silence is selected. The speech unit is replaced by the connection of the first transition part and the second transition part from silence to the other consonant, and for the first transition part, the speech corresponding to the transition part from silence to the unvoiced friction sound While selecting the segment data from the speech segment database and outputting the waveform data included in the speech segment data with time reversal, the corresponding speech element is output for the second transition portion. A segment selection means for outputting the data selected from the segment database,
A program which functions as segment connecting means for connecting and outputting waveform data included in each speech segment data output from the segment selecting means while adjusting.

Images