JP5678912B2 - Voice identification device, program - Google Patents

Description

  The present invention relates to an utterance specifying device and a program for specifying an utterance section uttered for each syllable from a speech waveform.

2. Description of the Related Art Conventionally, an utterance specifying technique that automatically specifies an utterance section that is an utterance section for each syllable in a speech waveform is known (see, for example, Patent Document 1).
In the utterance specifying technique described in Patent Document 1, a signal corresponding to vocal speech (hereinafter referred to as vocal signal) is extracted from a music acoustic signal of a song including a singing voice and an accompaniment sound, and a unit section of the vocal signal is extracted. Each of the acoustic models prepared in advance for each phoneme is collated. Then, the phoneme corresponding to the acoustic model with the highest likelihood of matching is specified as the phoneme (syllable) of the content uttered in the unit section.

  Further, in a general utterance specifying technique, the above-described utterance contents are sequentially specified along the time axis of the speech waveform, and the timing at which the syllable specified as the utterance contents is switched is determined by the utterance section for one syllable. The switching timing is specified (for example, refer to Patent Document 2).

JP 2009-186687 A JP 2009-139862 A

By the way, in the technique described in Patent Document 1, the specific accuracy of the utterance content depends on the acoustic model to be collated with the speech waveform.
In general, an acoustic model is constructed so as to be adapted to a large number of persons, but the degree of adaptation varies depending on individuals, so that the degree of adaptation is low depending on the person.

  In this way, when matching the acoustic model to a speech waveform uttered by a person with a low acoustic model suitability, the utterance identification technique reduces the recognition accuracy of the utterance content, and consequently recognizes the utterance interval for each syllable. There is a problem that accuracy decreases.

  Therefore, an object of the present invention is to provide a technique for improving the recognition accuracy of the utterance content for each syllable and the recognition accuracy of the utterance section for each syllable.

The present invention made to achieve the above object relates to an utterance specifying device.
In the utterance specifying device of the present invention, the utterance information acquisition means acquires the utterance content information representing the character string of the content to be uttered, and the reference utterance start timing and the reference utterance end timing of each of the characters constituting the character string. Based on the acquired utterance content information, the model section specifying means specifies a model utterance section that is a section to be uttered for each syllable constituting the character string.

  For each weighting section that is a section corresponding to each syllable and the corresponding syllable so that the weight setting means has a larger value for the syllable corresponding to each of the exemplary utterance sections specified by the exemplary section specifying means. Set the weight.

  On the other hand, in the utterance specifying device of the present invention, the voice data acquisition means acquires voice data representing a voice waveform uttered for the character string represented by the utterance content information, and sets a section for the acquired voice data. The means sets a plurality of unit sections so as to be continuous along the time axis in the audio data.

  Then, the phoneme specifying means collates each acoustic model prepared in advance as a model representing the feature quantity for each phoneme against the feature quantity in one unit section of the speech data set by the section setting means, Each of the utterance likelihoods set by the weight setting means and given the weight in the weighting section corresponding to the unit section is derived. Furthermore, in the phoneme specifying means, a content specifying process for specifying a phoneme corresponding to an acoustic model with the largest utterance likelihood as a phoneme uttered in the unit section (hereinafter referred to as a uttered phoneme), Executed sequentially for each unit interval along the time axis in the audio data.

  The timing at which the utterance syllable is switched along the time axis every time the timing specifying means specifies the utterance syllable that is syllable uttered in the voice data based on the utterance phonemes sequentially specified by the phoneme specifying means. The utterance switching timing is specified.

  Further, in the utterance specifying device of the present invention, the utterance syllable immediately before the specified utterance switching timing is set as the estimation completion syllable, and the syllable formed by the utterance phoneme immediately after the utterance switching timing is set as the estimated middle syllable. When the deviation between the timing at which the model utterance section is switched along the time axis (hereinafter referred to as model switching timing) and the utterance switching timing is equal to or greater than a predetermined allowable value, the weight changing unit performs along the time axis. For the unit sections corresponding to the estimated syllables after the estimated middle syllable, a weight is set so that the phonemes that are represented by character strings and that constitute the syllables corresponding to the unit sections are specified as uttered phonemes by the phoneme specifying means. The weighting section after the weighting section corresponding to the estimated middle syllable defined by the means is changed.

  In such an utterance specifying device, when specifying the utterance content in the audio data, the acoustic model is collated with the feature amount extracted from the audio data for each unit section, and the weight specified based on the utterance content information is given. doing. Moreover, in the utterance specifying device of the present invention, the weighting section after the weighting section corresponding to the syllable along the time axis is changed according to the deviation of the actual utterance section with respect to one model utterance section.

That is, according to the utterance specifying device of the present invention, the weighting interval can be adjusted to match the actual utterance interval for each syllable.
As a result, according to the utterance specifying device of the present invention, it is possible to improve the recognition accuracy of the utterance content for each syllable, and consequently improve the recognition accuracy of the utterance section for each syllable.

  For this reason, according to the utterance specifying device of the present invention, when generating a speech parameter from speech data in a speech section, it is possible to add the content of a highly reliable syllable to the speech parameter. As a result, according to the utterance specifying device of the present invention, it is possible to easily realize a voice desired by a person using the voice synthesis at the time of voice synthesis.

  The exemplary utterance section referred to here is a concept including at least each section that actually utters, and in addition to the section that actually utters, the concept includes a silent section that does not actually speak. May be.

  Furthermore, the model switching timing here is a timing at which the model utterance section is switched, and may be, for example, a reference utterance start timing or a timing defined based on the reference utterance start timing.

  In the utterance specifying device of the present invention, the weight changing means may make the weighting section earlier along the time axis if the utterance switching timing is earlier than the model switching timing.

  According to such an utterance identification device, even if the voice is uttered by a person whose utterance start timing is earlier than the reference utterance start timing, the utterance content for each syllable and the utterance section for each syllable are recognized. Accuracy can be improved.

  In this case, the weight changing means in the present invention sets the start timing of the weighting section corresponding to the next syllable of the estimated syllable along the time axis along the time axis if the utterance switching timing is earlier than the model switching timing. (Claim 3).

  A person who speaks at a timing earlier than the model switching timing is highly likely that the voice switching timing of the next syllable will be earlier than the model switching timing. Therefore, if the start timing of the weighting interval corresponding to the next syllable is advanced as in the present invention, the utterance content for the syllable and the recognition accuracy of the utterance interval for the syllable are likely to be improved.

  In the present invention, the weighting section as a target for changing the weight is not limited to the “weighting section corresponding to the syllable next to the estimated syllable” described in claim 3. One piece of music may include a weighting section corresponding to a syllable in the same phrase as the estimated middle syllable (ie, A melody, B melody, rust, etc.).

  Further, the weight changing means in the present invention estimates if the time length of the speech utterance section from the utterance start timing to the utterance end timing of the estimated completion syllable is shorter than the exemplary utterance section for the syllable corresponding to the estimation completed syllable. The end timing of the weighting section corresponding to the middle syllable may be advanced along the time axis (claim 4).

  According to such an utterance specifying device, even if the utterance time for each syllable is a voice uttered by a person whose time length is shorter than the time length of the exemplary utterance section, the utterance content for each syllable and the utterance for each syllable The recognition accuracy of the section can be improved.

By the way, the weight changing means in the present invention may delay the weighting section along the time axis if the utterance switching timing is later than the model switching timing.
According to such an utterance specifying device, even if the voice is uttered by a person whose utterance start timing of each syllable is later than the reference utterance start timing, the utterance content for each syllable and the utterance interval for each syllable are recognized. Accuracy can be improved.

  In this case, the weight changing means according to the present invention sets the start timing of the weighting section corresponding to the syllable next to the estimated syllable along the time axis along the time axis if the utterance switching timing is later than the model switching timing. (Claim 6).

  A person who speaks at a timing when the utterance switching timing is later than the model switching timing has a high possibility that the utterance switching timing of the next syllable will be later than the model switching timing. Therefore, if the start timing of the weighting interval corresponding to the next syllable is delayed as in the present invention, the utterance content for the syllable and the recognition accuracy of the utterance interval for the syllable are likely to be improved.

  In the present invention, the weighting section as a target for changing the weight is not limited to the “weighting section corresponding to the syllable next to the estimated syllable” described in claim 6. One piece of music may include a weighting section corresponding to a syllable in the same phrase as the estimated middle syllable (ie, A melody, B melody, rust, etc.).

  Furthermore, the weight changing means in the present invention estimates if the time length of the speech utterance section from the utterance start timing to the utterance end timing of the estimated completion syllable is longer than the exemplary utterance section for the syllable corresponding to the estimation completed syllable. The end timing of the weighting section corresponding to the middle syllable may be delayed along the time axis.

  According to such an utterance specifying device, even if a voice uttered by a person whose utterance time for each syllable is longer than the time length of the exemplary utterance section, the utterance content for each syllable and the utterance for each syllable The recognition accuracy of the section can be improved.

Normally, when a person utters a continuous syllable along the time axis, the syllable switching rarely coincides perfectly with the model switching timing.
For this reason, in the utterance specifying device, if a weight is given from the unit interval immediately before or after the model switching timing, the sound specifying device may mistakenly utter a phoneme different from the phoneme actually uttered. is there.

Therefore, in the present invention, the weighting section has a shorter time length than the model utterance section, and may be included in the model utterance section (claim 8).
According to such an utterance specifying device, since no weight is given from the unit interval immediately before or after the model switching timing, it is possible to reduce the possibility of misidentifying that the content is different from the content actually uttered.

The present invention may be a program for causing a computer to function as an utterance specifying device.
In this case, the program of the present invention includes an utterance information acquisition procedure, an exemplary segment identification procedure, a weight setting procedure, a voice data acquisition procedure, a segment setting procedure, a phoneme identification procedure, a timing identification procedure, and a weight change procedure. Must be a program that causes a computer to execute.

  If the program of the present invention is made in this way, for example, it can be recorded on a computer-readable recording medium such as a DVD-ROM, CD-ROM, hard disk, etc. If necessary, it can be used by being acquired and activated by a computer via a communication line. And by making a computer perform each procedure, the computer can be functioned as an utterance specific | specification apparatus described in Claim 1.

1 is a block diagram of a speech synthesis system configured mainly with an information processing apparatus to which the present invention is applied. It is a flowchart which shows the process sequence of an audio | voice parameter registration process. It is the flowchart which showed the process sequence of the utterance area estimation process. It is the figure which showed the relationship between a weight provision area and an exemplary utterance area. It is the flowchart which showed the process sequence of the phoneme specific process. It is the flowchart which showed the process sequence of the provision area change process. It is the figure which showed the process outline | summary of the provision area estimation process. It is the figure which summarized the process outline | summary of the provision area estimation process. It is a figure which shows the process sequence of a speech synthesis process.

Embodiments of the present invention will be described below with reference to the drawings.
<Speech synthesis system>
The speech synthesis system 1 shown in FIG. 1 synthesizes speech (that is, synthesized speech) based on speech parameters PM registered in advance so that speech having contents designated by the user of the speech synthesis system 1 is output. ). The speech parameter PM referred to here is a speech feature amount used for so-called formant synthesis, which will be described later in detail.

  In order to realize this, the voice synthesis system 1 is used for voice input device 10 for inputting voice, voice input through voice input device 10 (hereinafter referred to as voice data SV), and karaoke. And a music server 25 for storing various data (hereinafter referred to as music data MD). Furthermore, the speech synthesis system 1 includes an information processing device 30 that generates a speech parameter PM based on the speech data SV and the music data MD stored in the music server 25, and a speech parameter generated by the information processing device 30. And a data storage server 50 for storing PM. In addition, the speech synthesis system 1 includes a speech output terminal 60 that outputs a synthesized sound synthesized by speech based on the speech parameter PM stored in the data storage server 50.

<Music server>
First, the music server 25 is a device that is mainly configured of a storage device configured to be able to read and write stored contents, and is connected to the voice input device 10 via a communication network.

  The music server 25 stores at least music data MD prepared in advance for each song. The music data MD includes music data DM (music data DM is, for example, MIDI data format data or raw sound format data) and a lyrics data group DL for playing a music.

  On the other hand, the lyric data group DL is data relating to the lyrics of the music, and the lyrics telop data DT representing the characters constituting the lyrics of the music (hereinafter referred to as lyrics constituent characters) and the lyrics that are the output timing of the lyrics constituent characters. And lyrics output data DO in which a timing correspondence relationship for associating the output timing with the performance of the music data DM is provided.

  The timing correspondence relationship is that the timing of starting the output of the lyrics telop data DT is associated with the timing of starting the performance of the song data DM, and the output timing of each lyrics constituent character along the time axis of the song is The music data DM is defined by the elapsed time from the start of the performance. Note that the elapsed time referred to here is, for example, a time indicating the timing for executing color change of displayed lyrics constituent characters, and is defined by the speed of color change. Further, the lyric constituent characters referred to here may be each of the characters constituting the lyric, or may be a phrase or a phrase grouped according to a specific rule along the time axis.

<Voice input device>
Next, the voice input device 10 includes a communication unit 11, an input receiving unit 12, a display unit 13, a voice input unit 14, a voice output unit 15, a sound source module 16, a storage unit 17, and a control unit 20. And. That is, the voice input device 10 is configured as a so-called well-known karaoke device.

  Among these, the communication unit 11 communicates with the outside through the communication network (for example, a public wireless communication network or a network line). The input reception unit 12 is an input device (for example, a key, a switch, a remote control reception unit, or the like) that receives input of information and commands in accordance with external operations.

  The display unit 13 is a display device (for example, a liquid crystal display or a CRT) that displays a song name, a song selection number, and lyrics of music data MD, which will be described later. The voice input unit 14 is a device (so-called microphone) that converts sound into an electrical signal and inputs the signal to the control unit 20. The audio output unit 15 is a device (so-called speaker) that converts an electrical signal from the control unit 20 into sound and outputs the sound. Furthermore, the sound source module 16 is a device (for example, a MIDI sound source) that outputs a sound that simulates a sound from a sound source (that is, an output sound) when the music data DM is in the MIDI format.

The storage unit 17 is a non-volatile storage device (for example, a hard disk device or a flash memory) configured to be able to read and write stored contents.
The control unit 20 is stored in the ROM 21 that stores processing programs and data that need to retain stored contents even when the power is turned off, the RAM 22 that temporarily stores processing programs and data, and the ROM 21 and RAM 22. It is mainly configured by a known computer having at least a CPU 23 that executes each process (various operations) according to the processing program.

  In the ROM 21, a singing sound (via a processing program for executing a well-known karaoke performance process or a singing sound input via the voice input unit 14 during a period when one piece of music is being played by the karaoke performance process) A processing program in which the control unit 20 executes a voice storage process stored in the music server 25 in association with music identification information for identifying the target music is stored as voice data SV.

  Hereinafter, the voice storing process will be described. In the voice input device 10, music data MD corresponding to one piece of music (hereinafter referred to as “target music”) designated via the input receiving unit 12 is acquired from the music server 25 in accordance with the karaoke performance processing, and the music data The target music is played based on the music data DM in the MD, and the lyrics are displayed on the display unit 13 at the timing when the performance of the target music should be sung (spoken) based on the lyrics data group DL in the music data MD. Or change the display color.

  Furthermore, song identification information for identifying the target song (for example, song name of music data MD, song selection number, etc.), speaker identification for identifying a person who has input voice from the voice input unit 14 (hereinafter referred to as a speaker) Information (hereinafter referred to as a speaker ID) and audio data SV are stored in the music server 25 in association with the performance of the music data MD. Note that the voice data SV stored in the music server 25 is also associated with speaker feature information representing the features of the speaker. The speaker feature information includes, for example, the gender and age of the speaker. Including.

  Associating the music identification information with the speaker ID, for example, when the speaker logs in to the voice input device 10 using the speaker ID from the input receiving unit 12, the speaker ID becomes the voice input device 10. The music identification information and the speaker ID are associated with each other by selecting the target music information.

  As described above, the target music selected by the speaker (speaker ID) is played by the voice storage process, and the speaker inputs the lyrics displayed in the display unit 13 in accordance with the progress of the performance. The singing voice when singing (speaking) toward (microphone) is stored in the music server 25 as the voice data SV of the speaker ID for the target music.

If the gender, age, etc. are also entered when inputting the speaker ID, the gender, age, etc. are included in the speaker characteristic information and stored in the music server 25.
Thereafter, the control unit 40 of the information processing device 30 makes an inquiry to the music server 25 by inputting the speaker ID from the input receiving unit 32, and obtains the target music of the speaker ID and the voice data SV on the information processing device 30 side. Download to.

<Information processing device>
Next, the information processing apparatus 30 includes a communication unit 31, an input reception unit 32, a display unit 33, a storage unit 34, and a control unit 40.

  Among these, the communication part 31 communicates with the exterior via a communication network. The input receiving unit 32 is an input device that receives input of information and commands in accordance with external operations. The display unit 33 is a display device that displays an image.

  The storage unit 34 is a non-volatile storage device configured to be able to read and write stored contents. The control unit 40 is configured around a known computer having at least a ROM 41, a RAM 42, and a CPU 43.

  The ROM 41 of the information processing apparatus 30 controls the voice parameter registration process for storing the voice parameter PM generated based on the voice data SV and the music data MD stored in the music server 25 in the data storage server 50. A processing program to be executed by 40 is stored.

The data storage server 50 is a device that is mainly configured of a storage device that is configured to be able to read and write stored contents, and is connected to the information processing device 30 via a communication network.
<Voice parameter registration process>
As shown in FIG. 2, when the voice parameter registration process is started after the voice storage process is completed, the music data MD of the music (that is, the target music) designated via the input receiving unit 32 is stored in the music server. 25 (S110). Subsequently, the lyrics data group DL of the target music is extracted (S120), and one piece of audio data SV corresponding to the target music and corresponding to the speaker ID designated via the input receiving unit 32 is obtained from the music server 25. Obtain (S130).

  Further, in the speech data SV acquired in S130, a section that is estimated to be uttered corresponding to each syllable constituting the lyrics constituent characters (hereinafter referred to as a utterance section) is specified, and a utterance section that represents each of the utterance sections. An utterance section estimation process for generating data is executed (S140).

  Then, the speech parameter PM is derived from the speech waveform (hereinafter referred to as syllable waveform) in each utterance interval based on each utterance interval data generated in the utterance interval estimation process (S140) (S150). The audio parameter PM derived in S150 includes at least a fundamental frequency, a mel frequency cepstrum (MFCC), power, and their time differences. Since these fundamental frequencies, MFCC, power, and methods for deriving their time differences are well known, detailed description thereof is omitted here. For example, in the case of a fundamental frequency, along the time axis of the syllable waveform. What is necessary is just to derive | lead-out using methods, such as an autocorrelation, the autocorrelation of the frequency spectrum of a syllable waveform, or a cepstrum method. In addition, in the case of MFCC, a result obtained by applying a time analysis window to a syllable waveform and performing frequency analysis (for example, FFT) for each time analysis window is further obtained by logarithmizing the size for each frequency. It can be derived by frequency analysis. The power may be derived by integrating the result of squaring the amplitude by applying a time analysis window to the syllable waveform along the time axis.

  Subsequently, voice parameter registration for storing the voice parameter PM derived in S150 is executed (S160). Note that the speech parameter PM stored in the data storage server 50 in S160 is associated with the content (type) of the syllable uttered in each utterance section, the speaker ID, and speaker feature information.

Thereafter, the voice parameter registration process is terminated.
<Speech interval estimation processing>
When the speech segment estimation process is started in S140 of the speech parameter registration process, as shown in FIG. 3, a plurality of speech segment estimation processes are performed so as to be continuous over time with respect to the speech data SV acquired in the previous S130. Frame t _n is set (S210). The frame t _{n mentioned} here is a unit time having a predetermined time length (for example, 10 [ms]), and the subscript n indicates that the frame t is the nth along the time axis. Is a subscript representing

Then, the audio parameter PM is calculated from each frame t _n in the audio data SV (S220). Subsequently, the lyrics telop data DT in the lyrics data group DL acquired in S120 is converted into lyrics phoneme data in which each of the lyrics constituent characters is expressed in phoneme notation (S230). Specifically, in S230, a well-known morpheme analysis is performed on the string of lyrics constituent characters represented by the lyrics telop data DT, and the lyrics telop data DT is segmented for each morpheme (word). Further, in S230, each morpheme is converted into a phoneme notation based on a phoneme dictionary prepared in advance as dictionary data representing the phoneme notation of each word. Thereby, the lyrics telop data DT is converted into lyrics phoneme data.

  Based on the converted lyric phoneme data and the lyric output data DO in the lyric data group DL acquired in the previous S120, utterance is made for each of the constituent syllables CS that are syllables constituting the lyric constituent characters. A section ES to be performed (hereinafter referred to as a model utterance section) ES is identified (S240).

  Specifically, the exemplary utterance section ES is a section defined by a plurality of frames t that are continuous along the time axis, and is composed of two lyrics that are defined by the lyrics output data DO and that are continuous along the time axis. The section between the character output start timings is set as an exemplary utterance section ES of the constituent syllable CS. However, in S240, if the lyrics constituent character is “Kanji”, the section between the output start timings is equally divided by the number of characters when the lyrics constituent character is replaced with “Hiragana”, etc. Each divided section is specified as an exemplary utterance section ES of the syllable CS.

Note that the constituent syllable CS of this embodiment includes a silent state sp (that is, a rest defined in the music data DM) indicating that it corresponds to a silent character. CS _N and its code N are codes indicating that the constituent syllable CS is the Nth constituent syllable CS along the time axis, counted from the beginning of the lyrics.

Subsequently, for the phonemes constituting the constituent syllable CS _N corresponding to each exemplary utterance section ES _N identified in S240, a section for assigning weight β in the phoneme specifying process described later (hereinafter referred to as a weight provision section). ) Set GS _N (S250).

The weighting section GS _N in the present embodiment is a section defined by a plurality of frames t continuous along the time axis, and one weighting section GS is set for each exemplary utterance section ES. As shown in FIG. 4A, each of the weighting intervals GS _N is weighted with a time that is later in time length corresponding to the frame t corresponding to the specified number α along the time axis from the beginning of the model utterance interval ES _N. The start timing of the granting section GS _N is set as the end timing of the weighting section GS _{N as} the start timing of the weighting section GS _{N as} the start timing corresponding to the frame t corresponding to the specified number α along the time axis from the end of the exemplary utterance section ES _N To do. That is, each weight applied section GS _N is a time length shorter than the exemplary utterance section ES _N, is an interval to be encompassed model speech section in ES _N.

Then, a weight β is defined for each weighting section GS _N set in S250 (S260). The weight β defined in S260 is a value defined for each phoneme, and the larger the phoneme constituting the constituent syllable CS _N corresponding to each of the model utterance sections ES _N (for example, “1.5”). ). For example, if the syllable corresponding to the exemplary utterance section ES _N is “ka”, the weight β for the phonemes “k” and “a” is set to a large value, and phonemes other than the phonemes “k” and “a” ( For example, the weight β for “i” and “u”) is set to a small value (for example, “1”). However, the weight β is a positive real number.

Furthermore, the utterance content in each frame t _n of the audio data SV is specified as an utterance phoneme, and a syllable uttered in the audio data SV (hereinafter referred to as an utterance syllable VS) based on the specified utterance phoneme. Every time it is specified, a phoneme specifying process for specifying the timing when the utterance syllable VS is switched along the time axis (hereinafter referred to as utterance switching timing) is executed (S270). Note that the utterance syllable VS includes a silent state sp.

  Subsequently, in S270, a section between two utterance switching timings adjacent along the time axis is specified as a utterance section, and the utterance syllable as the utterance content in the utterance section is associated with the syllable waveform. Syllable data is generated (S280). Further, the pitch of the output sound corresponding to the utterance section is associated with each of the generated syllable data (S290).

Thereafter, the process proceeds to S150 of the voice parameter registration process.
<Phoneme specific processing>
Next, when the phoneme identification process is started in S270 of the speech segment estimation process, the frame t _n is set to an initial value (n = “1”) as shown in FIG. 5 (S310). Subsequently, the likelihood for the frame t ₁ is set to an initial value (S320). That is, in S320, when a person utters, there is often no sound at the time of utterance start, so the utterance phoneme for the frame t _{1 is set} to the silence state sp.

Furthermore, one proceeds frames t _n along the time axis in the voice data SV (n = n + 1) causes (S330). In the frame t _n , an utterance likelihood representing the possibility that each phoneme was uttered is derived (S340).

In S340, specifically, an acoustic model corresponding to one of the phonemes (hereinafter referred to as candidate phonemes) that can be taken in the frame t _{n in} accordance with the phoneme arrangement along the time axis defined in the lyric phoneme data, matching the speech parameter PM in frame t _n. This acoustic model is a model prepared in advance as a speech parameter PM representing the phoneme for each phoneme.

Then, the collation result (that is, the probability of uttering each phoneme, hereinafter referred to as the first likelihood) is set in the previous S260 for the weighting section GS corresponding to the frame t _n , and A weight β in the phoneme corresponding to the acoustic model is assigned. That is, by multiplying the first likelihood by the weight β, the utterance likelihood for a phoneme corresponding to one acoustic model is derived (S340).

Subsequently, it is determined whether or not the derivation of utterance likelihood has been performed for all candidate phonemes (S350). If the result of this determination, it is not to derive the utterance likelihood for all the candidate phoneme (S350: NO), it returns to S340, the speech parameters PM in frame t _n, with matching new acoustic model Deriving the utterance likelihood for the candidate phoneme is repeated.

On the other hand, the result of the determination in S350, if the derived utterance likelihood for all the option phonemes (S350: YES), in the utterance likelihood for the frame t _n, utterance likelihood value is maximum The candidate phoneme of the acoustic model corresponding to is specified as the utterance phoneme (S360).

That is, in the present embodiment, the utterance phoneme is specified after developing hypotheses for all phonemes that can be taken by the frame t _n according to the well-known Viterbi algorithm and Viterbi alignment.

Subsequently, it is determined whether or not the frame (hereinafter referred to as the current frame) t _{n for} which S340 has been executed has reached the final frame t _nmax (S370). A result of the determination in this S370, unless the current frame t _n has reached up to the last frame t _nmax (S370: NO), determines whether the switched syllable at the current frame t _n (S380).

More specifically, in S380 of the present embodiment, when utterance phoneme in the frame t _n-1 is a vowel utterance phonemes in frame t _n is the utterance phoneme in the frame t _n-1 is different and vowels, it determines consonants, when a silent state sp, shall syllable is switched in frame t _n. In this case, the S380, the frame t _n and frame t _n-1 and the boundary (or, frame t _n itself) to be identified as utterance switching timing.

As a result of the determination in S380, if no switching syllable at the current frame t _n (S380: NO), returns to S330, allowed to proceed one frame t _n along the time axis (n = n + 1) After that, S350 to S360 are executed.

On the other hand, as a result of the determination in S380, if the syllable is switched in the current frame t _n (S380: YES), the frame t that defines the weighting section GS _N corresponding to the frame t after the frame t _{n + 1} is determined. A grant section changing process to be changed is executed (S390).

In the following, the utterance syllable VS from the current frame t _n to the frame t constituting the next utterance switching timing along the time axis in the utterance syllable VS is referred to as an estimated middle syllable VS _M and is estimated along the time axis. The utterance syllable VS that precedes the middle syllable VS _M is referred to as an estimated completion syllable VS _M-1 . Note that the utterance syllable VS includes a silent state sp.

Here, the symbol M is a symbol representing that the estimated middle syllable VS _M is the Mth utterance syllable VS along the time axis, and the symbol M-1 is the estimated completion syllable VS _M-1 on the time axis. A symbol indicating that it is the (M−1) -th vocal syllable VS.

Thereafter, the process returns to S330, was allowed to proceed one frame t _n along the time axis (n = n + 1), executes S350～S360.
As a result of the determination in S370, if the current frame t _n has reached the final frame t _nmax (S370: YES), each of the sections between two utterance switching timings adjacent along the time axis is set as the utterance section. The utterance section data is generated (S400).

Thereafter, the process proceeds to S280 of the utterance section estimation process.
<Grant section change processing>
Next, when the added section changing process is started in S390 of the phoneme specifying process, as shown in FIG. 6, first, the current frame t _{n is included} in a plurality of frames t constituting the weight applying section GS _M-1. Is included (S610).

As a result of the determination in S610, if the current frame t _n is not included in the plurality of frames t constituting the weighting section GS _M-1 (S610: NO), the process proceeds to S640 described in detail later.

On the other hand, if the result of determination in S610 is that the current frame t _n is included in a plurality of frames t constituting the weighting section GS _M-1 (S610: YES), the weighting section GS _M-1 is configured. The number of frames between the last frame t to be performed and the current frame t _n is derived as a time difference DP (S620).

Subsequently, the first frame t constituting the weighting section GS _{M + 1} is changed so as to be earlier by the time difference DP along the time axis (S630). That is, in S630, the start end of the weighting section GS _{M + 1} corresponding to the next constituent syllable CS _{M + 1} along the time axis from the estimated middle syllable VS _M is advanced along the time axis.

Incidentally, in S640, it is determined whether or not the current frame t _n is included in the plurality of frames t constituting the weighting section GS _M (S640). As a result of the determination in S640, if the current frame t _n is included in the plurality of frames t constituting the weighting interval GS _M (S640: YES), the first frame t constituting the weighting interval GS _M And the number of frames between the current frame t _n is derived as a time difference DC (S650).

Subsequently, the first frame t constituting the weighting section GS _{M + 1} is changed so as to be delayed by the time difference DC along the time axis (S660). That is, in S660, the start end of the weighting section GS _{M + 1} corresponding to the constituent syllable CS _{M + 1} is delayed along the time axis.

Thereafter, the process proceeds to S670.
In S670, when S630 is completed, or when the result of determination in S640 is that the current frame t _n is not included in the plurality of frames t constituting the weighting section GS _M (S640: NO). Transition.

In S670, the section length EL of the utterance section uttered as the estimated completion syllable VS _M-1 is derived. Specifically, the section length EL may be derived from the number of frames t in which the utterance phonemes constituting the estimation completion syllable VS _M-1 are specified.

Subsequently, the section length PL of the exemplary utterance section ES _M-1 of the constituent syllable CS _M-1 corresponding to the estimated completion syllable VS _M-1 is derived (S680). Specifically, the section length PL may be derived as long as it is the number of frames t constituting the exemplary utterance section ES of the constituent syllable CS _M-1 .

Further, it is determined whether or not the section length EL does not match the section length PL (S690). As a result of the determination in S690, if the section length EL does not match the section length PL (S690: YES), a difference D (D = EL−PL) between the section length EL and the section length PL is derived (S700). .
If the difference D is to be real time, the difference D may be multiplied by the time length of one frame.

Then, the final frame t constituting the weighting section GS _M is changed by the difference D along the time axis (S710).
However, in S710 in the present embodiment, if the difference D is a negative value (that is, EL <PL), the final frame t constituting the weighting interval GS _M is changed along the time axis by the difference D (however, (Absolute value) Change to be faster. If the difference D is a positive value (that is, EL> PL), the final frame t constituting the weighting section GS _M is delayed by the difference D (however, absolute value) along the time axis. Change to In the latter case, the model vocal section ES _M configuration syllable CS _M corresponding to the estimated in syllables VS _M, constituting syllables CS _M corresponding to the next utterance syllable VS M _{+ 1} estimated in the syllable VS _M along the time axis _The frame t at which the ₊₁ model utterance section ES _{M + 1} is switched may be set as an upper limit that can be changed.

Then, this provision section change process is complete | finished, and it transfers to S330 of a phoneme specific process.
Even when the section length EL matches the section length PL as a result of the determination in S690 (S690: NO), the present section change process is terminated, and the process proceeds to S330 of the phoneme identification process.

  Note that “match” in S690 means that, in addition to the section length EL completely matching the section length PL, the section length EL matches the section length PL within a predetermined error range. Including. In this case, when determining in S690, if the absolute value (| EL-PL |) of the difference between the section length EL and the section length PL is smaller than a certain threshold γ, the section length EL coincides with the section length PL (S690). : NO).

That is, in applying section changing process, if utterance end timing of the estimated completion syllables VS _M-1 is earlier than the end timing of the model speech section ES _M-1 corresponding to the estimated completion syllables VS _M-1, FIG. 4 (B ), The start timing of the weighting section GS _{M + 1} is advanced along the time axis. In this case, the start timing of the weighting section GS _M is set to the frame t _n (that is, the current frame t _n ) next to the frame t _{n−1 with} respect to the utterance end timing of the estimation completion syllable VS _M−1 .

Then, when the time length of the utterance section for the estimated completion syllable VS _M-1 is shorter than the time length of the model utterance section ES _M-1 , as shown in FIG. 4C, the end timing of the weighting section GS _M Speed up along the time axis. In FIG. 4, it is assumed that M = N + 1.

In addition, in the added section changing process, when the utterance end timing of the estimated completion syllable VS _M-1 is later than the start timing of the exemplary utterance section ES _M corresponding to the estimated middle syllable VS _M , FIG. As shown in (B), the start timing of the weighting section GS _{M + 1} is delayed along the time axis.

Then, when the time length of the utterance section for the estimated completion syllable VS _M-1 is longer than the time length of the model utterance section ES _M-1 , as shown in FIG. 7C, the end timing of the weighting section GS _M Slow down along the time axis. However, in this case, the latest start timing of the weight applying section GS _M, a configuration syllable CS M _{+ 1} configured syllable CS _M and switching timing corresponding to the utterance syllable VS M _{+ 1.} In FIG. 7, it is assumed that M = N + 1.

Furthermore, the grant interval change process, as shown in FIG. 8, utterance end timing of the estimated completion syllables VS _M-1 is, and if it is granted within disabled section PS _M-1, utterance against estimated completion syllables VS _M-1 If the time length of the section matches the time length of the model utterance section ES _M-1 , the process shifts to the next frame t along the time axis without changing the weighting section GS _M. To do. Note that the subsequent syllable described in FIG. 8 is the utterance syllable VS _{M + 1} .

<Audio output terminal>
Next, the audio output terminal will be described (see FIG. 1).
The voice output terminal 60 includes an information receiving unit 61, a display unit 62, a sound output unit 63, a communication unit 64, a storage unit 65, and a control unit 67. As the audio output terminal 60, for example, a known portable terminal (a mobile phone or a portable information terminal) or a known information processing apparatus (a so-called personal computer) may be assumed.

  Among these, the information reception part 61 receives the information input via the input device (not shown). The display unit 62 displays an image based on a signal from the control unit 67. The sound output unit 63 is a known device that outputs sound, and includes, for example, a PCM sound source and a speaker.

  The communication unit 64 is for the voice output terminal 60 to perform information communication with the outside via a known communication network. The storage unit 65 is a non-volatile storage device configured to be able to read and write stored contents, and stores various processing programs and various data.

The control unit 67 is mainly configured by a known computer having at least a ROM, a RAM, and a CPU.
<Speech synthesis processing>
Next, speech synthesis processing executed by the control unit 67 of the speech output terminal 60 will be described.

This voice synthesis process is started when a start command is input via the information receiving unit 61 of the voice output terminal 60.
As shown in FIG. 9, when the speech synthesis process is started, first, information input via the information receiving unit 61 (hereinafter referred to as input information) is acquired (S910). The input information acquired in S910 includes, for example, output text indicating the content (word) of the sound output as synthesized sound, and output property information indicating the nature of the sound output as synthesized sound. Note that the sound property (that is, output property information) mentioned here includes characteristics of the voice of the speaker such as the gender of the speaker and the age of the speaker.

  Subsequently, the speech parameter PM corresponding to each syllable necessary for outputting the output message acquired in S910 as a synthesized sound and associated with information most similar to the output property information acquired in S910 is Extracted from the data storage server 50 (S920).

  Then, the voice parameter PM acquired in S920 is set so that the synthesized sound is output with the content of the output word acquired in S910 (S930). Subsequently, speech synthesis is performed based on the speech parameter PM set in S930 (S940). For the speech synthesis in S940, a method described in Patent Document 1 or a well-known speech synthesis method using formant synthesis may be used.

Further, the synthesized sound generated in S940 is output from the sound output unit 63 (S950).
Thereafter, the speech synthesis process ends.
[Effect of the embodiment]
As described above, in the adding section changing process of the present embodiment, when the utterance end timing of the estimated completion syllable VS _M-1 is earlier than the ending timing of the exemplary utterance section ES _M-1 , the weighting section GS _{M +} The start timing of ₁ is advanced along the time axis. By doing so, weighting corresponding to the syllables after the utterance syllable VS along the time axis is given to the person whose utterance start timing of the utterance syllable VS always precedes the switching timing of the model utterance section ES. The section GS can be reset to an appropriate time zone.

In addition, in the adding section changing process, the end timing of the weight adding section GS _M is set to the time even when the time length of the utterance section for the estimated completion syllable VS _M-1 is shorter than the time length of the model utterance section ES _M-1. It is fast along the axis. By doing in this way, with respect to a person who utters in a section shorter than the section length of the model utterance section ES, the weighting section GS corresponding to the utterance syllable VS can be reset to an appropriate time zone.

Further, in the adding section changing process, when the utterance end timing of the estimated completion syllable VS _M-1 is later than the starting timing of the exemplary utterance section ES _M , the start timing of the weighting section GS _{M + 1} is set along the time axis. It is late. In this way, weighting corresponding to syllables after the utterance syllable VS along the time axis is given to a person whose utterance start timing of the utterance syllable VS is always delayed with respect to the switching timing of the exemplary utterance section ES. The section GS can be reset to an appropriate time zone.

In addition, when the time length of the utterance section with respect to the estimated completion syllable VS _M-1 is longer than the time length of the model utterance section ES _{M-1 in} the assignment section change process, the end timing of the weight addition section GS _M is set to the time. Slow along the axis. By doing in this way, with respect to a person who utters in a section longer than the section length of the model utterance section ES, the weighting section GS corresponding to the utterance syllable VS can be reset to an appropriate time zone.

  From the above, according to the utterance section specifying process of the above embodiment, it is possible to improve the recognition accuracy of the utterance content for each syllable, and consequently improve the recognition accuracy of the utterance section for each syllable.

  Therefore, according to the speech parameter registration process of the above embodiment, when generating speech parameters in each utterance section, the content of syllables with high reliability can be added to the speech parameters. As a result, in the above embodiment, when the speech synthesis process is executed, it is possible to easily realize a synthesized sound desired by a person using the speech synthesis.

  Moreover, in the utterance section specifying process of the above embodiment, the weight provision section GS is set after the unit section immediately before and immediately after the switching timing of the model utterance section ES is set as the provision prohibition section PS. This is because when a person utters a continuous syllable along the time axis, the syllable switching is rarely completely coincident with the model switching timing, and the weight is given immediately before and after the switching timing of the model utterance section ES. This is because by prohibiting the addition of β, it is possible to reduce misidentification that a phoneme different from the phoneme actually spoken is uttered in the phoneme identification process.

[Other Embodiments]
As mentioned above, although embodiment of this invention was described, this invention is not limited to the said embodiment, In the range which does not deviate from the summary of this invention, it is possible to implement in various aspects.

  In the above embodiment, the information processing device 30 and the data storage server 60 are added to the system of the voice input device (karaoke device) 10 and the music server 25, and the voice input device 10 executes the karaoke performance process to obtain the target music. The sound data SV is generated based on the sound input during the performance period, but the sound data SV in the present invention is not limited to this.

  That is, in the present invention, the voice input device 10 may generate the voice data SV using a well-known after-recording function in a karaoke device or the like. That is, in the case of a voice input device (karaoke device) having an after-recording function, dialogue telop data (that is, lyrics telop) representing characters constituting dialogue (hereinafter referred to as dialogue constituent characters) as data relating to dialogue to be uttered. Data) and dialogue output data that defines the timing for displaying dialogue constituent characters on the display unit 13 (that is, data similar to the lyrics output data). Therefore, when acquiring the voice data SV using the after-recording function, the voice input device 10 displays a dialogue based on the dialogue telop data on the display unit 13 and performs voice input during a period in which the dialogue is displayed on the display unit 13. The voice waveform input via the unit 14 may be stored in the music server 25 as voice data SV.

In this case, the information processing apparatus 30 may use the voice data SV generated using the after-recording function as a processing target for the voice parameter registration process.
Moreover, in the said embodiment, although the karaoke apparatus was assumed as the audio | voice input apparatus 10, the apparatus assumed as the audio | voice input apparatus 10 is not restricted to a karaoke apparatus, For example, a well-known portable terminal (a mobile phone or portable information) Terminal) or a known information processing apparatus (so-called personal computer) may be assumed.

In the speech synthesis system of the above embodiment, the music server 25 and the data storage server 50 are provided. However, these may function as auxiliary storage means, and are stored in the storage unit 17 of the speech input device 10. Furthermore, each means of the information processing device 30 may be incorporated into the voice input device 10 so that the voice input device (karaoke device) is used as a device for specifying utterances used for creating voice synthesis data.
[Correspondence between Embodiment and Claims]
Finally, the relationship between the description of the above embodiment and the description of the scope of claims will be described.

  S120 in the voice parameter registration process of the above embodiment corresponds to the utterance information acquisition means in the description of the claims, and S130 corresponds to the voice data acquisition means in the claims. Further, S240 in the utterance section estimation process corresponds to the model section specifying means in the description of the claims, S250 and S260 correspond to the weight setting means in the description of the claims, and S210 is the claim. This corresponds to the section setting means in describing the range.

  S310 to S360 in the phoneme specifying process of the above embodiment correspond to the phoneme specifying means in the claims, and S380 corresponds to the timing specifying means in the claims. And S390 in the phoneme identification process of the said embodiment, ie, an provision area change process, is equivalent to the weight change means in description of a claim.

  DESCRIPTION OF SYMBOLS 1 ... Speech synthesis system 10 ... Voice input device 11 ... Communication part 12 ... Input reception part 13 ... Display part 14 ... Voice input part 15 ... Voice output part 16 ... Sound source module 17 ... Memory | storage part 20 ... Control part 21 ... ROM 22 ... RAM 23 ... CPU 25 ... music server 30 ... information processing device 31 ... communication unit 32 ... input receiving unit 33 ... display unit 34 ... storage unit 40 ... control unit 41 ... ROM 42 ... RAM 43 ... CPU 50 ... data storage server 60 ... Audio output terminal

Claims (9)

Utterance information acquisition means for acquiring utterance content information representing a character string of contents to be uttered, and a reference utterance start timing and a reference utterance end timing of each of the characters constituting the character string;
Based on the utterance content information acquired by the utterance information acquisition means, exemplary section specifying means for specifying an exemplary utterance section that is a section to be uttered for each syllable constituting the character string;
A weight is set for each of the syllables and for each weighting section that is a section corresponding to the syllable so that the syllable corresponding to each of the model utterance sections specified by the model section specifying means has a larger value. Weight setting means;
Voice data acquisition means for acquiring voice data representing a voice waveform uttered for a character string represented by the utterance content information acquired by the utterance information acquisition means;
Section setting means for setting a plurality of unit sections to be continuous along the time axis in the voice data for the voice data acquired by the voice data acquisition means;
Each of the acoustic models prepared in advance as a model representing the feature value for each phoneme is checked against the feature value in one unit section of the voice data set by the section setting means, and set by the weight setting means. And each of the utterance likelihoods given weights in the weighting interval corresponding to the unit interval is derived, and the phoneme corresponding to the acoustic model with the largest utterance likelihood is uttered in the unit interval. Phoneme specifying means for sequentially executing a content specifying process for specifying the uttered phoneme as a phoneme, for each unit section along the time axis in the voice data;
Each time the utterance syllable that is uttered in the speech data is specified based on the utterance phoneme sequentially specified by the phoneme specifying means, the utterance switching is the timing at which the utterance syllable is switched along the time axis. Timing specifying means for specifying timing;
The utterance syllable immediately before the utterance switching timing specified by the timing specifying means is an estimated completion syllable, the syllable formed by the utterance phoneme immediately after the utterance switching timing is the estimated middle syllable, and the exemplary utterance along the time axis When the deviation between the model switching timing, which is the timing at which the section is switched, and the utterance switching timing is greater than or equal to a predetermined tolerance, the unit section corresponding to the syllable after the estimated middle syllable along the time axis, The phoneme that is represented by the character string and that constitutes the syllable corresponding to the unit section is specified by the weight setting means so as to be specified as the utterance phoneme by the phoneme specifying means, and corresponds to the estimated middle syllable A utterance specifying device comprising: weight changing means for changing a weighting section after the weighting section. The weight changing means includes
The utterance identification device according to claim 1, wherein if the utterance switching timing is earlier than the model switching timing, the weighting section is advanced along a time axis. The weight changing means includes
If the utterance switching timing is earlier than the model switching timing, the start timing of the weighting section corresponding to the syllable next to the estimated syllable along the time axis is advanced along the time axis. The utterance identification device according to claim 2. The weight changing means includes
If the time length of the speech utterance section from the utterance start timing to the utterance end timing of the estimated completion syllable is shorter than the exemplary utterance section for the syllable corresponding to the estimated completed syllable, weighting corresponding to the estimated middle syllable The utterance specifying device according to claim 2, wherein the end timing of the section is advanced along the time axis. The weight changing means includes
The utterance identification device according to any one of claims 1 to 4, wherein if the utterance switching timing is later than the model switching timing, the weighting interval is delayed along the time axis. The weight changing means includes
If the utterance switching timing is later than the model switching timing, the start timing of the weighting interval corresponding to the syllable next to the estimated syllable along the time axis is delayed along the time axis. The utterance specifying device according to claim 5. The weight changing means includes
If the time length of the speech utterance section from the utterance start timing to the utterance end timing of the estimated completion syllable is longer than the exemplary utterance section for the syllable corresponding to the estimation completed syllable, weighting corresponding to the estimated middle syllable The utterance specifying device according to claim 5, wherein the end timing of the section is delayed along the time axis. The weighting interval is
The utterance specifying device according to any one of claims 1 to 7, wherein the utterance specifying device has a shorter time length than the exemplary utterance section and is included in the exemplary utterance section. A utterance information acquisition procedure for acquiring utterance content information representing a character string of contents to be uttered, and a reference utterance start timing and a reference utterance end timing of each of the characters constituting the character string;
Based on the utterance content information acquired by the utterance information acquisition procedure, an exemplary section specifying procedure for specifying an exemplary utterance section that is a section to be uttered for each syllable constituting the character string,
A weight is set for each of the syllables and for each weighting section that is a section corresponding to the syllable so that a syllable corresponding to each of the model utterance sections specified by the model section specifying procedure has a larger value. Weight setting procedure;
An audio data acquisition procedure for acquiring audio data representing an audio waveform uttered for a character string represented by the utterance content information acquired by the utterance information acquisition procedure;
An interval setting procedure for setting a plurality of unit intervals so as to be continuous along the time axis in the audio data for the audio data acquired in the audio data acquisition procedure,
Each of the acoustic models prepared in advance as a model representing the feature quantity for each phoneme is checked against the feature quantity in one unit section of the voice data set in the section setting procedure, and set in the weight setting procedure. And each of the utterance likelihoods given weights in the weighting interval corresponding to the unit interval is derived, and the phoneme corresponding to the acoustic model with the largest utterance likelihood is uttered in the unit interval. A phoneme specifying procedure for sequentially executing the content specifying process for specifying the uttered phoneme as the phoneme, for each unit section along the time axis in the voice data;
Each time the utterance syllable, which is the syllable uttered in the speech data, is specified based on the utterance phoneme sequentially specified in the phoneme specification procedure, the utterance switching is the timing at which the utterance syllable is switched along the time axis. A timing identification procedure for identifying timing;
The utterance syllable immediately before the utterance switching timing specified in the timing specifying procedure is set as an estimated completion syllable, the syllable formed by the utterance phoneme immediately after the utterance switching timing is set as an estimated syllable, and the exemplary utterance along the time axis When the deviation between the model switching timing, which is the timing at which the section is switched, and the utterance switching timing is greater than or equal to a predetermined tolerance, the unit section corresponding to the syllable after the estimated middle syllable along the time axis, The phoneme that is represented by the character string and that constitutes the syllable corresponding to the unit interval is specified as the utterance phoneme in the phoneme specifying procedure, and is defined in the weight setting procedure and corresponds to the estimated middle syllable. A program for causing a computer to execute a weight changing procedure for changing a weighting section after the weighting section. Lamb.

Images