JP5962925B2 - Speech synthesis device, music playback device, speech synthesis program, and music playback program - Google Patents

Description

  The present invention relates to a speech synthesizer that performs speech synthesis based on input information, and a music player that performs music playback and speech synthesis. The present invention also relates to a speech synthesis program for executing speech synthesis processing in various information processing apparatuses, and a music playback program for executing music playback processing and speech synthesis processing.

  Conventionally, a speech synthesizer for synthesizing speech by inputting text information or the like is known. In such a speech synthesizer, it is required to realize a natural speaking voice in the same way as a person speaks, and various developments have been made for that purpose. In recent years, speech synthesis is used not only for such spoken voices but also for singing voices, and development for realizing natural singing has been promoted. Furthermore, in Patent Document 1, a learner can reliably and efficiently store words such as English words having accents and words such as Japanese translations corresponding to the words in accordance with the rhythm of music or the like. Forming audio information is disclosed.

  By the way, it is conceivable to use speech synthesis that is sounded in accordance with the rhythm of music or the like disclosed in Patent Document 1 for instruction in various exercises such as fitness as well as learning as in Patent Document 1. Such instructions can be recorded in synchronization with the music in advance, but by using speech synthesis, as in the case of instructors in the past, depending on the exercise situation of the user It is also possible to convey appropriate instructions.

  Currently, various fitness software is provided in game machines and the like. In such software, an instruction according to a user's situation captured by a camera is transmitted by voice. Moreover, in the smart phone, the instruction | indication is also transmitted with an audio | voice according to the user's running condition (speed or a pulse) acquired using various sensors, such as GPS.

JP 2011-13295 A

  Various types of exercise such as fitness or running may be performed while listening to music. In such a case, when outputting the above-described instruction by voice, it is important not to lose the music flow and the tempo of music, as in the case of conveying the contents of the instruction. A form in which the length of speech synthesis is adjusted in accordance with the tempo of music is also conceivable, but such a form may result in unnatural pronunciation. In addition, for speech synthesis intended for learning disclosed in Patent Document 1, it is also required to perform speech synthesis more in line with the rhythm of music.

  The present invention is intended to solve such problems, and a speech synthesizer that enables natural speech synthesis that matches the rhythm of a song, and a song that simultaneously performs speech synthesis and song playback. It is an object of the present invention to provide a playback device and a program therefor.

In order to solve the above-described problems, the present invention adopts the following configuration.
A speech synthesizer capable of performing speech synthesis processing based on text information,
The speech synthesis process
A morphological analysis process for dividing text information into accent phrases and setting an accent position for a mora in each accent phrase;
Arrangement processing that arranges the mora corresponding to the accent position of the accent phrase so as to overlap the strong beat timing of the input time signature information, and shrinks the accent phrase so that adjacent accent phrases do not overlap,
A speech waveform synthesis process for synthesizing a speech waveform based on the accent phrase arranged in the placement process is executed.

Furthermore, in the speech synthesizer according to the present invention,
The arrangement process arranges accent phrases so that adjacent accent phrases have a predetermined time or more.

Furthermore, in the speech synthesizer according to the present invention,
The arrangement process arranges an accent phrase without an accent position by connecting it to an adjacent accent phrase.

Moreover, the music reproducing device according to the present invention includes:
A music playback device capable of performing music playback processing based on music information and speech synthesis processing based on text information,
The speech synthesis process
A morphological analysis process for dividing text information into accent phrases and setting an accent position for a mora in each accent phrase;
Arrangement processing that arranges the mora corresponding to the accent position of the accent phrase so that it overlaps the strong beat timing of the time signature information output along with the music reproduction process, and shrinks the accent phrase so that adjacent accent phrases do not overlap When,
A speech waveform synthesis process for synthesizing a speech waveform based on the accent phrase arranged in the placement process is executed.

A speech synthesis program according to the present invention is
A speech synthesis program for causing an information processing apparatus to perform speech synthesis processing based on text information,
The speech synthesis process
A morphological analysis process that divides the input text into accent phrases and sets an accent position to a mora in each accent phrase;
Arrangement processing that arranges the mora corresponding to the accent position of the accent phrase so as to overlap the strong beat timing of the input time signature information, and shrinks the accent phrase so that adjacent accent phrases do not overlap,
A speech waveform synthesis process for synthesizing a speech waveform based on the accent phrase arranged in the placement process is executed.

According to the speech synthesizer, the music playback device, the speech synthesis program, and the music playback program according to the present invention, the mora corresponding to the accent position of the accent phrase is pronounced so as to overlap the strong beat timing of the time signature information, It is possible to perform speech synthesis that matches the time rhythm of music playback. At that time, by shortening the accent phrase so that the accent phrases do not overlap, it is possible to output sound that is easy to hear. With such a configuration, it is possible to obtain appropriate sound with a sense of rhythm when performing speech synthesis in accordance with the time rhythm, such as fitness instructions, music, and the like.

The block diagram which shows the structure of the information processing apparatus which concerns on embodiment of this invention. The flowchart which shows the speech synthesis process which concerns on embodiment of this invention The flowchart which shows calculation (process A) of the utterance timing which concerns on embodiment of this invention The flowchart which shows the tone adjustment (process B) of the accent phrase which concerns on embodiment of this invention. The figure which showed typically the speech synthesis processing example which concerns on embodiment of this invention The figure which showed typically the speech synthesis processing example which concerns on other embodiment of this invention.

  FIG. 1 is a block diagram showing a configuration of an information processing apparatus 1 according to an embodiment of the present invention. As the information processing apparatus 1, various forms such as a personal computer, a smart phone, a tablet terminal, and a portable information terminal such as various music reproducing apparatuses can be considered. The information processing apparatus 1 according to the present embodiment can perform performance processing based on music information and speech synthesis processing based on text information. In addition to this form, the information processing apparatus 1 according to the present invention may be configured to perform only speech synthesis processing based on information related to strong beat timing input from an external apparatus.

  The information processing apparatus 1 according to the present embodiment includes a control unit 10 and a storage unit 13. The control unit 10 includes a CPU, a ROM, a RAM, and the like, and can perform a performance process 11 and a voice synthesis process 12. The storage unit 13 stores a morpheme analysis dictionary 13a and phoneme information 13b necessary for speech synthesis processing. The control unit 10 can execute a performance process 11 that outputs a performance sound based on input music information and a voice synthesis process 12 that synthesizes a speech waveform based on input text information. The performance sound and the sound waveform output from the control unit 10 are synthesized by the mixing unit 14 and emitted from the speaker 15.

  The music information used for the performance processing 11 and the text information used for the speech synthesis processing 12 may be acquired via an external storage medium or communication such as the Internet, even if stored in the storage unit 13. It may be.

  In the performance processing 11 executed by the control unit 10, time signature information is output to the voice synthesis processing 12 side in synchronization with the performance of the music information. The time signature information includes a strong beat timing. The strong beat timing corresponds to, for example, the start timing of the first beat in a 4-beat composed of 1 to 4 beats. The time signature information including the strong beat timing may be included in advance in the music information, or may be formed based on the volume of the performance sound played.

  The voice synthesis process 12 executed by the control unit 10 executes a morpheme analysis process 12a for converting text information into a voice parameter necessary for the voice synthesis process, and a voice waveform synthesis process 12c for synthesizing a voice waveform based on the voice parameter. To do. In particular, in the present embodiment, adjustment processing 12b for adjusting the speech parameters generated in the morphological analysis processing 12a is performed in order to realize speech synthesis synchronized with the strong beat timing of the time signature information.

Now, the details of the speech synthesis process executed by the information processing apparatus 1 of this embodiment will be described. The speech synthesis process of this embodiment is a process executed based on input text information, and a speech waveform is synthesized and output as a processing result. The input text information is subjected to morphological analysis processing (S10) using the morphological analysis dictionary 13a, and various information (speech parameters) necessary for the speech synthesis waveform is acquired (S11). FIG. 5 shows a state of the speech synthesis process of the present embodiment. FIG. 5A shows an example sentence of text information.

  FIG. 5B shows the result of performing morphological analysis processing on the text information of FIG. In the present embodiment, the input text information is divided into accent phrases A1 to A4 with reference to the morphological analysis dictionary 13a. The accent phrase is composed of mora which is a unit of phoneme. Each mora is assigned a sound length of an output sound waveform. For example, the accent phrase A1 is composed of three mora “U”, “DE”, and “O”, and the sound lengths of B1, B2, and B3 are assigned to each mora, respectively. Therefore, the tone length of the accent phrase A1 is the sum of the tone lengths of each mora B1 + B2 + B3 = C1.

  Further, in the morphological analysis process, an accent position is set for each accent phrase A1 to A3. The accent position is information set for the mora in the accent phrases A to A3, and when performing speech synthesis processing, an accent (adjusts at least one of pitch or volume) is given to the mora. In this way, natural sound is realized in listening. In the example sentence of FIG. 5, all accent phrases A1 to A3 have an accent position, but there are also accent phrases without an accent position.

  Usually, speech synthesis can be performed by reading out and outputting various information (sound source information) for forming a speech synthesis waveform, such as phoneme information, based on information obtained by morpheme analysis processing. In the present embodiment, in order to execute the speech synthesis process in synchronization with the strong beat timing accompanying the performance process, various adjustments are made to the information that has been subjected to the morphological analysis process as shown in FIG. . Specifically, the accent phrase utterance timing and the accent phrase sound length are adjusted.

  In the speech synthesis process of FIG. 2, if the interval between the calculation of the occurrence timing for each accent phrase obtained by the morphological analysis process (S20: process A) and the preceding accent phrase is not more than a certain value, the accent phrase is adjusted (S30: The adjustment process including the process B) can be executed. FIG. 3 shows a flowchart of the process A for calculating the utterance timing of the accent phrase. FIG. 5C shows the arrangement of accent phrases for the morphological analysis result of FIG. 5B. In FIG. 5C, T1 to T5 indicate the strong beat timing defined by the time signature information. Looking at the accent phrase A1 ("UDEO"), the next (in this case, first) strong beat timing A = T1 is set.

  Next, the total value of the sound lengths of the mora before the accent position in the accent phrase is calculated (S22). In the case of the accent phrase A1, since the accent position is in the second mora (“de”), the mora sound length B = the sound length of mora (“de”). By moving the accent phrase pronunciation timing A-B based on the strong beat timing A and the sound length B, the head of the mora corresponding to the accent position in the accent phrase can be synchronized with the strong beat timing. Become. The utterance timing of the next accent phrase A2 (“Okiku Foot”) is determined by the same processing.

  In this embodiment, when arranging the accent phrases A1 to A4 in order, the strong beat timing position not occupied by the previously arranged accent phrases is used as the next strong beat timing A, and the accent phrases are arranged. Yes. Therefore, the accent phrase A2 (“Okiku foot”) is the strong beat timing T2, the accent phrase A3 (“Ashibumi”) is the strong beat timing T4, the accent phrase A4 (“Shimasho”) is the strong beat timing T5, and the next strong beat timing. Used as A.

As described above, in this embodiment, in the accent phrases A1 to A4, the utterance timing (start position) of the mora corresponding to the accent position is made coincident with the strong beat timing. As long as the strong beat timing is located during the generation period of the mora corresponding to the accent position, that is, the mora corresponding to the accent position of the accent phrase is arranged so as to overlap the strong beat timing. It is possible to realize speech synthesis that meets the beat rhythm on an auditory basis.

  As can be seen from the relationship between the accent phrases A2 and A3 in FIG. 5C, it is conceivable that the accent phrases overlap each other when arranged based on the accent positions of the accent phrases as described above. When the voice is uttered in such a state, the sound becomes unnatural in terms of audibility, and in some cases, the meaning of the voice cannot be interpreted. In the present embodiment, in order to eliminate such a state, a process (process B) for shortening the length of the accent phrase in which trouble has occurred is performed.

  FIG. 4 shows a flowchart of the process B for shortening the length of the accent phrase in which the trouble has occurred. In the present embodiment, as described with reference to FIG. 2, when the interval (time length) with the preceding accent phrase is less than a predetermined time length in the flowchart of the speech synthesis process, this process B (S30) is performed for the preceding accent phrase. ) Is executed. Hereinafter, a threshold for determining whether an interval (time length) between accent phrases is less than a predetermined time length is referred to as an interval threshold. In this process, the pitch of each mora in the preceding accent phrase is adjusted with the interval threshold-the interval between the preceding accent phrases as A and the number of mora of the construction accent phrase as B as parameters. By making the length of each mora the length obtained by subtracting A / B from the original length (S34), the entire length of the preceding accent phrase to be adjusted is shortened by the length A, and the leading accent A time length defined by an interval threshold is secured between the phrase and the accent phrase. The interval threshold value can be set as appropriate as long as the time length is 0 or more.

  FIG. 5D shows the result of adjusting the tone length by performing the process B on the accent phrase arranged based on the strong beat timing shown in FIG. In this embodiment, the tone length is adjusted with respect to the accent phrase A2 ("Okuikuteofette") and the accent phrase A3 ("Ashibumi"). By synthesizing the speech waveform based on the speech parameters determined by the arrangement of the accent phrase and the mora (S15 in FIG. 2), it is possible to perform natural speech synthesis that matches the rhythm of the music to be played. It is possible. In the speech waveform synthesis process of the present embodiment, the phoneme information 13b in the storage unit 13 is read based on the speech parameters determined through the morpheme analysis process 12a and the placement process 12b, and a speech waveform is formed.

  By the way, in the morphological analysis process, an accent phrase without an accent position may occur. The example sentence explained in FIG. 5 has a form in which all accent phrases have an accent position. However, in the speech synthesis process of this embodiment, an accent phrase without an accent position is arranged in synchronization with the strong beat timing. I can't. Therefore, when an accent phrase without an accent position is generated, it is connected to a subsequent accent phrase (with an accent position) and synchronized with a strong beat timing based on the accent position of the subsequent accent phrase.

FIG. 6 shows a form (front portion) in which an accent phrase A0 (“Mazuwa”) without an accent position is arranged before the example sentence described in FIG. In this case, the accent phrase A0 is connected to the subsequent accent phrase A1, and the accent position of the accent phrase A1 is arranged in synchronization with the strong beat timing T1, as in the example of FIG. The arrangement of A0 is also possible, and speech synthesis with less sense of incongruity is realized. The arrangement of the accent phrase without an accent position is preferably connected to the subsequent accent phrase for the sake of hearing. For example, when the accent phrase without an accent position is located at the end of the sentence, the subsequent accent phrase is not connected. If it is not possible to connect to an accent phrase, an accent phrase without an accent position may be connected to the immediately preceding accent phrase. As described above, in the present invention, an accent phrase without an accent position is connected to an adjacent accent phrase (immediately preceding or following).

  Note that the present invention is not limited to these embodiments, and embodiments configured by appropriately combining the configurations of the respective embodiments also fall within the scope of the present invention.

DESCRIPTION OF SYMBOLS 1 ... Information processing apparatus 12c ... Speech waveform synthesis process 10 ... Control part 13 ... Memory | storage part 11 ... Performance process 13a ... Morphological analysis dictionary 12 ... Speech synthesis process 13b ... Phoneme information 12a ... Morphological analysis process 14 ... Mixing part 12b ... Adjustment process 15 ... Speaker

Claims (5)

A speech synthesizer capable of performing speech synthesis processing based on text information,
The speech synthesis process
A morphological analysis process for dividing text information into accent phrases and setting an accent position for a mora in each accent phrase;
An adjustment process that arranges the mora corresponding to the accent position of the accent phrase so as to overlap the strong beat timing of the input time signature information, and shrinks the accent phrase so that adjacent accent phrases do not overlap,
A speech synthesizer that performs speech waveform synthesis processing that synthesizes speech waveforms based on the accent phrases arranged in the adjustment processing. The speech synthesizer according to claim 1, wherein the adjustment process arranges accent phrases so that adjacent accent phrases have a predetermined time or more. The speech synthesizer according to claim 1, wherein the adjustment process arranges an accent phrase without an accent position in connection with an adjacent accent phrase. A music playback device capable of performing music playback processing based on music information and speech synthesis processing based on text information,
The speech synthesis process
A morphological analysis process for dividing text information into accent phrases and setting an accent position for a mora in each accent phrase;
An adjustment process that arranges the mora corresponding to the accent position of the accent phrase so that it overlaps the strong beat timing of the time signature information output with the music playback process, and shrinks the accent phrase so that adjacent accent phrases do not overlap When,
A music reproducing device that performs voice waveform synthesis processing for synthesizing a voice waveform based on the accent phrase arranged in the adjustment process. A speech synthesis program for causing an information processing apparatus to perform speech synthesis processing based on text information,
The speech synthesis process
A morphological analysis process that divides the input text into accent phrases and sets an accent position to a mora in each accent phrase;
An adjustment process that arranges the mora corresponding to the accent position of the accent phrase so as to overlap the strong beat timing of the input time signature information, and shrinks the accent phrase so that adjacent accent phrases do not overlap,
A speech synthesis program for executing speech waveform synthesis processing for synthesizing a speech waveform based on the accent phrase arranged in the adjustment processing.

Images