JP5481957B2 - Speech synthesizer - Google Patents

Description

  The present invention relates to a technique for obtaining code data that can be used in electronic musical instruments, musical scores, and the like based on a human voice.

  Conventionally, a method of artificially synthesizing a human voice has been used in various fields. The applicant has proposed a technique for synthesizing speech by digitizing a human voice with PCM and then using a Fourier transform to obtain a code code corresponding to a frequency having a large effective intensity (see Patent Document 1). ).

  In addition, the applicant can play back even a low-performance MIDI sound source installed in toys, etc., and read it with an existing score editing tool and convert it to a staff score so that a legible score can be obtained. Have proposed a technique of converting a code code group for each phoneme into a simple code code group (see Patent Document 2).

JP-A-11-95798 Japanese Patent Application No. 2009-41165

  In the technique described in Patent Document 1, since the code code group encoded in the MIDI data format for each phoneme constituting the human voice is based on the premise that the voice is reproduced by an automatic performance by an electronic musical instrument, The sound generation timing and velocity control are complicated, and it was not possible to play with the MIDI sound source with low performance installed in toys. Further, when the code code group is read by an existing score editing tool and converted into a staff score, it becomes an unreadable music score, and it is difficult to reproduce the code code group by playing a musical instrument by a human.

  In the technique described in Patent Document 2, all phonemes other than vowels are uniformly expressed by MIDI data simplified by double chords, but various instrument sounds are set and reproduced by a MIDI sound source. However, there is a problem of lack of clarity of speech. There are the following three causes of unclear audio. First, there is a problem in that all phonemes other than vowels are simplified with two quadratures, etc. In the actual phonemes, formant fluctuations in the time axis direction are large, and the noise and vowel components of consonants are reduced. It is difficult to omit the elapsed sound to be connected to the human auditory interpolation phenomenon (sound pulse concentration), and there is too little change in double chords. Secondly, there is a problem with the method of unifying all phonemes other than vowels into two quadruples, etc. In the actual Japanese phoneme, the structure of the elapsed sound and the pronunciation time are different for each phoneme. It is impossible to unify the number of chords. Third, there is a problem in that there is a variation in the pitch (pitch) of phonemes registered in the database. If it is simply placed in the time axis direction, an unnatural inflection is added to the synthesized speech, and the speech is heard. Is difficult to do.

Also, when building a phoneme database in MIDI format, a set of waveform voice data that records Japanese voiced Japanese syllabary is prepared and converted to high-definition MIDI data using the technology disclosed in Japanese Patent No. 4132362. The method of instructing the position corresponding to the two phoneme sections on the converted MIDI data on the screen, and the subjectivity of the worker enters, the problem that the shaping data becomes uneven and the load of the worker There was a problem that was large.

  Therefore, the present invention reduces the work load in the case of realizing a speech synthesis function based on simplified code data (such as MIDI data) that can be automatically converted into a musical score to the extent that a human can perform with a musical instrument, It is an object of the present invention to provide a phoneme code correction device, a phoneme code database, and a speech synthesizer that can improve the clarity of speech reproduction quality.

In order to solve the above-described problem, in the present invention, a phoneme code reading unit that reads a phoneme code in which one phoneme is expressed by a plurality of code code groups, and a pronunciation start time for each code code constituting the read phoneme code The code with the highest energy value given by the product of the time difference between the sound and the pronunciation end time and the strength of the code code is extracted in the range where the code code that overlaps in time falls within a predetermined type , and the extracted code code a phonemic code converting means for converting the constructed phoneme codes in de group, for each of the reproduction starting time and the sound end time of sign-code that make up the phonemic code, an integral multiple of a predetermined time unit a phonemic code corrected by the phonemic code correction device having a code time correction means for correcting, as, in correspondence with each phoneme in Japanese kana characters, a predetermined or fewer pitch simultaneously have, Phoneme code identification that identifies a phoneme code as a corrected phoneme code composed of a plurality of chords (two or more vowels and three or more consonants) whose length is defined by an integral multiple of a predetermined time unit The phoneme code database recorded in association with the information and the corrected phoneme code corresponding to the phoneme code identification information described in the given synthesis instruction data are extracted from the phoneme code database, and according to the type of the phoneme There is provided a speech synthesizer comprising: phoneme editing processing means for generating synthesized speech data by setting a pronunciation time and a silence time, and setting a time for specifying the start and end of pronunciation .

According to the onset bright, it reads the phonemic code configured as a code code group, the high code code energy value into a composed phonemic code of a predetermined number extracted code code group, that make up the phonemic code marks Since the sound generation start time and the sound end time of the chord code are corrected to be an integral multiple of a predetermined time unit, it can be automatically converted into a staff notation that can be played by a musical instrument. In the case of realizing the speech synthesis function based on the converted code data, it is possible to reduce the work load and improve the clarity of the speech reproduction quality.

According to the onset bright, chords plurality of phonemic code representing the respective phoneme in Japanese kana characters, which have predetermined or fewer pitch simultaneously, the length of the sound is defined by an integral multiple of a predetermined time unit Since the code code group created by performing the speech synthesis can be reproduced with a low-performance MIDI sound source mounted on a toy or the like, the player can use the existing score editing tool to It becomes possible to convert to a staff score that is easy to read when playing a musical instrument.

According to the onset bright, has a database that records the phonemic code composed of a chord plurality of glutinous predetermined or fewer pitch simultaneously, the length of the sound is defined by an integral multiple of a predetermined time unit Extract phoneme code corresponding to input phoneme code identification information, set pronunciation time and silence time according to phoneme type, and generate synthesized speech data by setting the time to specify the start and end of pronunciation As a result, the code code group created by performing speech synthesis can be played back with a low-performance MIDI sound source mounted on a toy or the like. It can be converted into a reproducible and readable staff score.

  According to the present invention, in the case of realizing a speech synthesis function based on simplified code data that can be automatically converted into a musical score to the extent that a human can perform with a musical instrument, the workload is reduced and the sound reproduction quality is clearly defined. It is possible to improve the sex.

It is a figure which shows the basic concept of the speech synthesis in this invention. It is a figure which shows the basic concept of the speech synthesis in this invention. It is a block diagram which shows one Embodiment of the phoneme code correction apparatus which concerns on this invention. It is a figure which shows the mode of the phoneme code | symbol before correction | amendment displayed on the phoneme code | symbol display means. It is a figure which shows the mode of the phoneme code after correction | amendment displayed on the phoneme code | symbol display means. It is a figure which shows the mode of the change of the phoneme code before and after the process by the phoneme code correction means. It is a figure which shows the example of the phoneme code | symbol of the male voice stored in the correction | amendment phoneme code | symbol storage part. It is a figure which shows the example of the phoneme code | symbol of the male voice stored in the correction | amendment phoneme code | symbol storage part. It is a figure which shows the example of the phoneme code | symbol of the female voice stored in the correction | amendment phoneme code | symbol storage part. It is a figure which shows the example of the phoneme code | symbol of the female voice stored in the correction | amendment phoneme code | symbol storage part. It is a figure which shows the example which made the phonetic code | symbol of the male voice shown in FIG. 7, FIG. It is a figure which shows the example which made the phoneme code | symbol of the female voice shown in FIG. 9, FIG. It is a block diagram which shows one Embodiment of the speech synthesizer which concerns on this invention. It is a figure which shows the mode of the correction | amendment of the pitch by the phoneme edit process means. It is a block diagram which shows one Embodiment of the digital watermark embedding apparatus based on this invention. It is a figure which shows the example which performed the process by the phoneme code correction means 21 with respect to 32 code codes.

(1. Basic concept of the present invention)
DESCRIPTION OF EXEMPLARY EMBODIMENTS Hereinafter, preferred embodiments of the invention will be described in detail with reference to the drawings. First, the basic concept of the present invention will be described. It is known that Japanese vowels can be approximated by chords of four or more overtones including two characteristic speech formant components. In addition to vowel components, consonants need three types of connected chords: chords that express noise such as friction sounds and chords that express vowels to vowels. If it is left to the phenomenon (sound pulse concentration), it can be approximated to two chords of the first noise and vowel. Therefore, the applicant thinks that speech synthesis can be realized by expressing all phonemes based on a single quadruple for Japanese vowels and two quadruples for consonants, and connecting them on the time axis. This is proposed in Patent Document 2.

  However, as described above, due to various factors, there is a problem in that the sound is not clear even when various instrument sounds are set and reproduced by the MIDI sound source. In the present invention, the configuration of phonemes such as vowels and consonants is fundamentally different from that of Patent Document 1. Each phoneme is recorded in a unique manner. Furthermore, these phonemes are used to synthesize in various modes according to the characteristics of phonemes.

  In this embodiment, regardless of vowels or consonants, each phoneme has a variable chord configuration in a range of 2 to 6 chords, each chord is a standard 6-tone, and the length is an integral multiple of 1/16 second. give. A configuration is also possible in which chords having different lengths are stacked in a total twelve-tone range. FIG. 1 is a diagram illustrating an example of a basic phoneme configuration in the present embodiment in the case of four stations. In the example of FIG. 1 (a), it has a quadruple configuration consisting of four chords, the first chord and the second chord are 1/16 seconds of the minimum unit interval, and the third chord is 2/2 of the minimum unit interval. 16 seconds, the fourth chord is 3/16 seconds, which is three times the minimum unit interval. In the example of FIG. 1B, the same chord as in the example of FIG. 1A is used, but the fourth chord is superimposed on the second chord and the third chord.

  In the present embodiment, when speech synthesis is performed, phonemes having the above-described configuration are arranged in time series. At this time, in the present embodiment, the phoneme section which is the length of the phoneme is variable. Here, FIG. 2A and FIG. 2B show a fixed phoneme arrangement in which phoneme sections are arranged in a fixed manner and a variable phoneme arrangement in which phoneme sections are arranged in a variable manner. In the example of FIG. 2, the case where there are three phonemes is shown. In the case of the fixed phoneme arrangement, as shown in FIG. 2A, even if the second phoneme from the left is 3/16 seconds in total, the phoneme section is fixed at 4/16 seconds. On the other hand, in the case of variable phoneme arrangement, as shown in FIG. 2B, when the second phoneme from the left is 3/16 seconds in total, the phoneme section is also changed to 3/16 seconds.

  FIG. 3A shows a normal phoneme synthesis pattern. In the case of a normal phoneme, the phoneme interval is set as a standard with the phoneme interval as it is. As shown in FIG. 3A, when the length of the phoneme “K” is 4/16 seconds, the phoneme interval is 4/16 seconds with the phoneme length unchanged, and the phoneme interval is the standard 4/16 seconds. Followed by a subsequent phoneme. FIG. 3B is an example showing a synthesized pattern of long phonemes. In the case of a long phoneme, the phoneme interval is set to twice the phoneme length, and the phoneme interval is set to twice the standard. As shown in FIG. 3B, when the length of the phoneme “K” is 4/16 seconds, the phoneme section is 8/16 seconds that is twice the length of the phoneme, and the phoneme interval is the standard 8 / 16 seconds.

  FIG. 4 is a diagram for explaining a combined pattern of stuttering and prompting sounds by comparison with normal phonemes. When the “tsu” in between is a normal phoneme, such as “Atta”, as shown in FIG. 4A, the phoneme interval before and after that is set to a standard 4/16 second. Although the phoneme section is not particularly shown, the length of the phoneme remains as described above. In the case of “h” in the middle, such as “Atta”, as shown in FIG. 4B, the phoneme interval before and after that is set to 2/16 seconds which is 1/2 of the standard. The phoneme section is exactly the same as in the case of normal phonemes. Thus, in this embodiment, the phoneme itself is not processed, and the stuttering / promoting sound is created by shortening the phoneme interval. In FIG. 2 to FIG. 4, some preceding phonemes and subsequent phonemes are omitted, but a silent phoneme interval is basically set between all phonemes. 1 to 4 are basic styles, the ratio of the pronunciation time of each chord in each phoneme, the length of the pronunciation time (phoneme interval) of each phoneme, and the length of silent time (phoneme interval) are as follows. It can be changed as appropriate. For example, the speech rate can be changed by changing the interval between adjacent phonemes (phoneme interval).

(2. Preparation)
Next, pre-processing using the prior art will be described. As a preparation, a process of encoding a human voice for each phoneme is performed. This corresponds to the phoneme encoding process disclosed in Patent Document 1. However, in the encoding process shown in Patent Document 1, 4, 8, and 16 code codes are defined in one unit section, but in this embodiment, more code codes are defined. It is different in point. Specifically, in this embodiment, 32 code codes corresponding to the note number are defined and encoded.

  As an encoding process, first, a human voice is digitized in units of phonemes. In the same way as shown in Patent Document 1, each phoneme is actually uttered by a person, and the recorded sound is recorded by a technique such as PCM. At this time, the speaker is a native Japanese man or woman, and it is desirable that the same person utter as much as possible with respect to 71 syllables with the same pitch (pitch) and utterance interval. Since it is difficult for ordinary speakers to arrange the utterances in this way, it is desirable to have an announcer or vocalist who has received specialized voice training speak. Furthermore, it is desirable to collect multiple sets of 71 syllable recordings by multiple speakers, including men or women. Subsequently, the digital data of each phoneme is converted into 128 code code groups. This conversion is generally the same as that shown in Patent Document 1. Therefore, each code code has a pitch, a sound intensity, a sound generation start time, a sound generation end time (in this embodiment, MIDI is used as the code code, so the note number, velocity, note-on time, note-off time) Time). However, as described above, in the present embodiment, the code code group is converted to 128 code codes corresponding to all the note numbers, instead of about 4 to 16 as in Patent Document 1, and 32 of them are further converted. A code code group is selected. As a specific method of converting PCM data into 32 code code groups, a known technique disclosed in Japanese Patent No. 4037542, Japanese Patent No. 4132362, or the like is used. In this way, the code code group for each phoneme is recorded in the phoneme database as a phoneme code. In the present embodiment, the phonemes recorded in the phoneme database are a total of 71 sounds including so-called fifty sounds, repelling sounds, muddy sounds, and semi-voiced sounds.

(3. Correction of phoneme code)
Next, phoneme code correction will be described. FIG. 5 is a block diagram showing an embodiment of a phoneme code correction apparatus according to the present invention. The storage means 10 includes a phoneme code storage unit 11 and a corrected phoneme code storage unit 12, and is realized by an external storage device such as a hard disk connected to a computer. The process control unit 20 controls the entire process of the phoneme code correction apparatus, and includes a phoneme code conversion unit 21 and a code time correction unit 22. The processing control means 20 is a computer main body including a CPU and a memory, and the phoneme code conversion means 21 and the code time correction means 22 are realized by the CPU executing a dedicated program. The phoneme code display means 30 displays the phoneme code read by the process control means 20, and is realized by a display device such as a liquid crystal display.

  Next, the processing operation of the phoneme code correction apparatus shown in FIG. 5 will be described. First, the process control means 20 reads a phoneme code for each phoneme from the phoneme code storage unit 11.

  When the processing control unit 20 reads the phoneme code, the phoneme code conversion unit 21 sets the energy sum value for each pitch (note number in the case of MIDI) for all code codes constituting the read phoneme code. calculate. The total energy value is calculated by the sound intensity at each pitch (velocity in the case of MIDI) × sounding time (in the case of MIDI, duration: note-off time−note-on time). When the energy sum value is calculated for all pitches, the phoneme code correction means 21 has an intensity value equal to or higher than a predetermined value (for example, 64 for a velocity having a value of 0 to 127), and the energy sum value is higher. The pitch is selected so that it does not exceed the specified number of chords (eg, 6) in each sounding time interval.

  The specified number of chords is set in advance and can be set freely. In the present embodiment, it is “6” as described above. Therefore, in the present embodiment, six code codes are extracted from the 32 code codes by the phoneme code conversion means 21. Next, the code time correction means 22 corrects the note-on time and note-off time of the extracted code code so as to be an integral multiple of a predetermined unit time, sets the start time and end time, and sets the velocity. Set to the default value.

  As the start time and end time, Ton is the note-on time of the extracted code code, Toff is the note-off time, and Tq is the minimum unit time. 'And Toff' are set.

[Formula 1]
Td ′ = Floor [(Td + Tq · 2/3) / Tq] · Tq
Ton ′ = Floor [{(Toff + Ton) / 2−Td ′ / 2} / Tq] · Tq
Toff ′ = Ton ′ + Td ′

  In the above [Equation 1], Td is a duration, which is Toff-Ton. Floor [] is a function that rounds down the decimal point of a real value to an integer.

  Since the velocity can take a value from “0” to “127”, the maximum value is set to “127” in the present embodiment.

  FIG. 6 shows how phoneme codes change before and after processing by the phoneme code conversion means 21 and code time correction means 22. In FIG. 6, the horizontal axis corresponds to time, and the vertical axis corresponds to frequency (note number). The rectangle arranged in the graph shows the code code, and the horizontal length shows the temporal length according to the horizontal axis, but the vertical length is different from the vertical axis, not the frequency but the intensity. (Velocity).

  FIG. 6A shows a phoneme code before processing by the phoneme code conversion means 21 and the code time correction means 22. As described above, in this embodiment, a phoneme code is generally composed of 32 code codes at the same time, and the designated number of chords is set to 6, but in FIG. For convenience, the number of code codes is 5 at the same time, and the designated number of chords is set to 2. Further, as can be seen from the horizontal and vertical lengths of the rectangles indicating the respective code codes, the reproduction time (end time-start time) and intensity of each code code are also different. After the processing by the phoneme code correcting means 21, as shown in FIG. 6B, the start time and the end time are corrected to a position that is an integral multiple of a predetermined time interval, and the strength of each code code is unified to a specified value. Therefore, in FIG. 6B, the horizontal and vertical lengths of the rectangles indicating the respective code codes are all the same or an integral multiple of the predetermined length. An example in which processing is actually performed on 32 code codes is shown in FIG.

  The phoneme code conversion means 21 and the code time correction means 22 perform processing on each phoneme code stored in the phoneme code storage unit 11 and store each corrected phoneme code in the corrected phoneme code storage unit 12. Examples of phoneme codes stored in the corrected phoneme code storage unit 12 are shown in FIGS. Of these, FIGS. 7 and 8 are encoded male voices, and FIGS. 9 and 10 are encoded female voices. 7 to 10, “C, C #, D, D #, E, F, F #, G, G #, A, A #, B” are “do, de #, re, re #, "Mi, Fah, Fah #, Seo, Seo #, La, La #, Shi" in English notation, the numbers listed indicate octave numbers, and the MIDI standard is a pair symbol of pitch names and octave numbers. The note number can be specified, and in the present application, the MIDI standard note number 69 is expressed as A3 (internationally, there are many notations indicating A4 as MIDI standard note number 69). In FIGS. 7 to 10, “---” means that the left sound is extended. Therefore, for example, in the case of phoneme A in FIG. 7A, if the minimum unit interval is 1/16 second, “F5”, “D # 5”, “A4”, and “F # 4” are 2/16 from the start of phoneme A. "G4", "G # 2", and "G # 1" are sounded for 1/16 second from the start of phoneme A, and "B2" and "B1" are 1/16 from 1/16 after the start of phoneme A. It means that it is pronounced for a second. Further, (2) phoneme I, (3) phoneme U, (4) phoneme E, and (5) phoneme O in FIG. 7 are 4/16 seconds and 3/16, respectively, assuming that the minimum unit interval is 1/16 second. Seconds, 3/16 seconds, 3/16 seconds. When the code code constituting the phoneme code is defined by the MIDI standard, it can be converted into a staff score by a commercially available score editing tool. FIG. 11 shows an example in which the phoneme codes of male voices in FIGS. 7 and 8 are converted into a staff, and FIG. 12 shows an example in which the phoneme codes of female voices in FIGS.

(4. Speech synthesis)
Next, speech synthesis using the corrected phoneme code will be described. FIG. 13 is a block diagram showing an embodiment of a speech synthesizer according to the present invention. In FIG. 13, the phoneme code database 12a records the corrected phoneme code in association with the phoneme code identification information indicated by the synthesis instruction data. The phoneme codes stored in the phoneme code database 12a are the same as those stored in the corrected phoneme code storage unit 12 after being corrected by the above-described phoneme code correction device. Therefore, it can be said that the phoneme code correction apparatus described above is for creating the phoneme code database 12a. The synthesized speech data storage means 13 stores the synthesized speech data synthesized by the phoneme editing processing means 50, and is realized by a storage device such as a hard disk.

  The phoneme editing processing means 50 performs a process of extracting a corresponding phoneme code from the phoneme code database 12a according to the content of the synthesis instruction data, generating a synthesized speech data by performing a predetermined process, and outputting it to a predetermined output destination. . The generated synthesized voice data is output to one or more of the synthesized voice data storage unit 13, the voice output unit 60, and the printing unit 70 according to the setting. The voice output means 60 is for generating the synthesized voice data received from the phoneme editing processing means 50 as an actual voice, and is realized by a MIDI playback device having a MIDI sound source. The printing means 70 converts the synthesized voice data received from the phoneme editing processing means 50 into a staff score and prints it. The conversion to the staff score is realized by executing known conversion software. This is realized by a known printer or the like. The speech synthesizer shown in FIG. 13 is actually realized by including a dedicated program in a computer that includes an input device and an external storage device and is connected to a MIDI playback device.

  The synthesis instruction data input to the speech synthesizer is obtained by arranging phoneme identification information in a predetermined order. This phoneme identification information can be in any format as long as it can identify a phoneme code. There may be. In this embodiment, text data in which a character code corresponding to a phoneme is recorded is used as phoneme identification information. In this case, the phoneme code in the phoneme database needs to be recorded in association with the character code corresponding to the phoneme.

  Next, the processing operation of the speech synthesizer shown in FIG. 13 will be described. First, synthesis instruction data is input to the speech synthesizer. When the speech synthesizer reads the synthesis instruction data, the phoneme editing processing means 50 synthesizes the synthesis instruction data in order from the first phoneme identification information. Specifically, the phoneme editing processing unit 50 extracts a corresponding phoneme code from the phoneme code database 12a with the phoneme identification information in the synthesis instruction data. If the extracted phoneme code is a normal phoneme, set the note-on time as the note-on time of the preceding phoneme and set the note-on time as the phoneme interval. A MIDI event is created with the note number and velocity set as the time and the values recorded in the phoneme code database 12a as they are.

  When the extracted phoneme code is a long sound, as shown in FIG. 3B, the time obtained by adding the phoneme interval to the note-off time of the preceding phoneme is set as the note-on time, and the length of the phoneme is 2 A MIDI event is created in which the time obtained by adding doubles as phoneme intervals is set as the note-off time, and the note number and velocity are the values recorded in the phoneme database. Regardless of the type of the phoneme subsequent to the long phoneme, the note-on time is set 0.5 seconds after the note-off time of the long phoneme. However, the numerical value after 0.5 seconds is a standard value, and the ratio of the pronunciation time of each chord in each phoneme, the duration of each phoneme (phoneme interval), and the length of silent time (phoneme interval) are changed as appropriate. Is possible.

  When the extracted phoneme code is a stuttering / promoting sound, as shown in FIG. 4, the phoneme interval is ½ of the normal phoneme, so the note-on time is 0.125 seconds after the note-off time of the previous phoneme. Set as time and set note-off time according to phoneme length. Then, the note-on time of the subsequent phoneme is set 0.125 seconds after the note-off time of the phoneme. However, the numerical value after 0.125 seconds is a standard value to the last, and the ratio of the pronunciation time of each chord in each phoneme, and the length of the pronunciation time (phoneme interval) and silence period (phoneme interval) of each phoneme are changed as appropriate. Is possible.

  Regardless of the normal phoneme, long sound, stuttering / promotion sound, the pitch (note number in the case of MIDI) may be the value recorded in the phoneme code database, but in that case, the phoneme code database 12a. If the pitches of the phonemes recorded in are inconsistent, an unnatural inflection will occur when synthesized. Therefore, in this embodiment, for each phoneme other than the first phoneme, the difference between the lowest tone of each phoneme and the lowest tone of the first phoneme is obtained, and the lowest tone of each phoneme other than the first phoneme is the lowest of the first phoneme. The pitch of each phoneme is corrected as a whole so as to be the same as the sound.

  Here, FIG. 14 shows a state of pitch correction by the phoneme editing processing means 50. FIG. 14 shows an example in which each phoneme is composed of four code codes for convenience of explanation. FIG. 14A shows a case where the pitch is not corrected (when the pitch recorded in the database is used as it is), and FIG. 14B shows a case where the pitch is corrected. When the pitch is not corrected for the first phoneme, the subsequent phoneme 1, and the subsequent phoneme 2, the lowest tone of the first phoneme (the pitch of the code code constituting the first phoneme) and the lowest tone and the subsequent phoneme 2 of the subsequent phoneme 1 Is the state as shown in FIG. When the synthesized speech as shown in FIG. 14A is reproduced, an unnatural inflection is added. Therefore, the difference 1 between the lowest sound of the first phoneme and the lowermost sound of the subsequent phoneme 1 and the difference 2 between the lowest sound of the first phoneme and the lowermost sound of the subsequent phoneme 2 are obtained, respectively. For all the four code codes to be performed, the pitch is corrected by the difference 1, and for the subsequent phoneme 2, the pitch is corrected by the difference 2 for all the four code codes to be configured. As a result of the pitch correction, as shown in FIG. 14B, the pitches of the lowest sounds of the first phoneme, the subsequent phoneme 1, and the subsequent phoneme 2 are the same, and unnatural inflection is eliminated during reproduction.

  The note number is optionally raised or lowered by adding a pitch offset parameter separately designated by the user, and pitch conversion can be realized.

  The phoneme editing processing unit 50 performs phoneme synthesis processing in units of phoneme identification information in the read synthesis instruction data, and sequentially outputs synthesized voice data (MIDI data) in units of phonemes that have been processed. I will pass it on. The audio output means 60 reproduces the MIDI data received from the phoneme editing processing means 50 in order. As described above, the speech synthesizer can reproduce speech according to the read synthesis instruction data.

  When outputting as a musical score, the synthesized voice data is converted into the musical score data by the printing means 70 and then printed out. Further, as in the above example, voice synthesis can be performed in real time according to the synthesis instruction data, and voice reproduction or stave output can be performed. In this voice synthesis apparatus, the result of processing by the phoneme editing processing unit 50 is used. Some MIDI data may be stored in the synthesized voice data storage means 13, and the MIDI data may be separately played back by a MIDI playback device. As a method for accumulating MIDI data in a storage device, an SMF (Standard MIDI File) format file can be used to pass it to various commercially available music-related software. Can be used to create a score. In this case, the score is created based on the phoneme code recorded in the SMF format. Then, if the generated score is output from the printing device, it can be used as an easy-to-read score when playing a musical instrument.

  As described above, the phoneme editing processing unit 50 extracts the corresponding phoneme code from the phoneme database 12a using the phoneme identification information in the synthesis instruction data, and when creating a MIDI event, the note number is recorded in the phoneme code database 12a. A pitch offset parameter is optionally added to the note number of each note constituting the chord corresponding to the phoneme code being added, and the pitch conversion can be performed up and down as appropriate. . In this case, the pitch of the entire synthesized speech data is raised or lowered, but if the pitch offset parameter is defined for each phoneme together with the phoneme identification information in the synthesis instruction data, the pitch can be raised or lowered for each phoneme. . That is, singing voice synthesis can be realized if a pitch change (pitch information) between adjacent notes of a melody created in advance is given as a pitch offset parameter defined together with phoneme identification information in the synthesis instruction data.

(5. Application to digital watermarking)
The voice synthesizer according to the present invention can be applied to a technique for embedding specific information such as copyright holder information in the form of a voice message in music data, and an audible “digital watermark”. FIG. 15 is a diagram showing a digital watermark embedding device using the basic configuration of the speech synthesizer according to the present invention. In FIG. 15, the phoneme code database 12a is the same as the phoneme code database 12a shown in FIG. 13, and the corrected phoneme code is recorded in association with the phoneme code identification information indicated by the synthesis instruction data. is there. The embedding processing means 51 embeds a message specified by a message text (synthesis instruction data) in music content that is digital data described in the SMF format or the like. Specifically, the embedding processing unit 51 has the function of the phoneme editing processing unit 50 shown in FIG. 13, and extracts a corresponding phoneme code from the phoneme code database 12a according to the content of the message text (synthesis instruction data). A predetermined process is performed to generate synthesized speech. If the music content to be output is a plurality of tracks and there is a dedicated track for messages, the synthesized speech is embedded in the dedicated track and output to the sound output means 61 as a single MIDI format music data. When there is no dedicated track in the music content, the synthesized speech is stored in the silent portion of the music content and output to the sound output means 61.

  The sound output means 61 is substantially the same as the sound output means 60 shown in FIG. 13, and generates sound data received from the embedding processing means 51 as an actual sound. In the digital watermark embedding apparatus shown in FIG. 15, the message text is embedded in real time and output as sound, but the MIDI data is stored in the storage device in the SMF format as the processing result by the embedding processing means 51, The MIDI data can be separately distributed over a network or the like, and the sound can be output by the MIDI playback device on the receiver side. The digital watermark embedding device shown in FIG. 15 is actually realized by incorporating a dedicated program into a computer that includes an external storage device and is connected to a MIDI playback device.

  A method of operating as an audible digital watermark in which the embedded message text is synthesized with the music content and output as a voice message at the time of sound output, and the channel volume of the MIDI data corresponding to the embedded message text is set to the minimum Or, by changing the velocity value of all MIDI events set to a fixed value of 127 to 0, a method of operating as an inaudible digital watermark in which voice messages other than music contents are not reproduced can be used. For example, when a trial version is distributed to a general user as a sample, an audible digital watermark is embedded and distributed, and when a genuinely purchased product version is distributed, an inaudible digital watermark is embedded and distributed. When the music content provider determines whether or not the music content with the inaudible digital watermark embedded is a genuine purchase product, the above operation is reversed, that is, the channel volume of the MIDI data is changed to the maximum value, By performing preprocessing such as changing the velocity values of all MIDI events set to 0 to 127, the state is embedded in an audible form, and the digital watermark extracting apparatus may be applied hereinafter.

  Next, a digital watermark extraction apparatus that extracts a digital watermark embedded in the above-mentioned audible form in MIDI data will be described. The digital watermark extraction apparatus is realized by a computer including an acoustic signal acquisition device such as a microphone and the phoneme code database 12a and a dedicated program for digital watermark extraction. The dedicated program incorporated in the digital watermark extraction apparatus causes the computer to function as frequency analysis means and phoneme code identification information decoding means. Specific contents of the frequency analysis means and the phoneme code identification information decoding means can be realized by the procedures disclosed in Japanese Patent No. 4037542 and Japanese Patent No. 4132362. The digital watermark extracting apparatus records partly the sound signal sent to the space from the sound output means 61 through a microphone or the like connected to the digital watermark extracting apparatus, and frequency analysis means for the recorded PCM data. Performs frequency analysis and extracts chord data. Specifically, it is converted into 32 time-series code code groups using a known technique disclosed in Japanese Patent No. 4037542 and Japanese Patent No. 4132362. Subsequently, the phoneme code identification information decoding means collates the extracted chord data with the phoneme code database 12a, extracts phoneme codes having similar chord data, and decodes the phoneme code identification information. Specifically, the message text (synthesis instruction data) in FIG. 15 is restored by sequentially collating with the code code group recorded in the phoneme code database 12a and sequentially extracting the matching phoneme codes.

  INDUSTRIAL APPLICABILITY The present invention can be used in a music production / composition support industry that imitates human voice reproduction performed for events and entertainment purposes. In the entertainment field, for musical scale reproduction media such as toys (including robots and stuffed animals) mainly made of electronic musical instruments, toy-type acoustic instruments (miniature pianos for interior decoration), music boxes, and ringtones for mobile phones. It can be used in industries that add speech synthesis functions. Further, it can be used in industries such as copyright protection when distributing MIDI music content by SMF (Standard MIDI File) or the like.

DESCRIPTION OF SYMBOLS 10 ... Memory | storage means 11 ... Phoneme code memory | storage part 12 ... Correction | amendment phoneme code memory | storage part 12a ... Phoneme code database 13 ... Synthetic speech data memory | storage means 20 ... Process control means 21 ... Phoneme code conversion means 22 ... Code time correction means 30 ... Phoneme code display means 40 ... Start / end time instruction means 50 ... Phoneme editing processing means 51 ... Embedding processing means 60 ... Speech output Means 61 ... Sound output means 70 ... Printing means

Claims (10)

A phoneme code reading means for reading a phoneme code expressing one phoneme by a plurality of code code groups, and for each code code constituting the read phoneme code, a time difference between a pronunciation start time and a pronunciation end time and a code code A phoneme that has a high energy value given as a product of strength and is extracted in a range where code codes that overlap in time are less than or equal to a predetermined type and is converted to a phoneme code that is composed of the extracted code code group A phoneme code correction apparatus having code conversion means and code time correction means for correcting each of the sounding start time and sounding end time of the code code constituting the phoneme code so as to be an integral multiple of a predetermined time unit a phonemic code corrected by, in correspondence with each phoneme in Japanese kana characters, have a predetermined or fewer pitch simultaneously, the length of the sound regulations an integral multiple of a predetermined time unit Is the chord plurality (vowels two or more consonants are three or more) the phonemic code after correction made up of, and sound Motofugo database recording in association with the phonemic code identification information for identifying the phonemic code ,
The corrected phoneme code corresponding to the phoneme code identification information described in the given synthesis instruction data is extracted from the phoneme code database, and the pronunciation time and the silence time are set according to the type of the phoneme. A speech synthesizer comprising: phoneme editing processing means for generating synthesized speech data by setting a time for specifying a start and an end. In claim 1 ,
The phoneme editing processing means includes the lowest pitch among the code codes constituting the phoneme code corresponding to the head phoneme of the synthesis instruction data and the code code constituting the phoneme code corresponding to each phoneme other than the head phoneme. The difference from the lowest pitch among the phonemes other than the head is obtained, and the pitch is changed by the difference for all code codes constituting the phoneme codes corresponding to the phonemes other than the head. A featured voice synthesizer. In claim 1 or claim 2 ,
A speech synthesizer further comprising speech output means for outputting the synthesized speech data generated by the phoneme editing processing means as speech. In any one of Claims 1-3 ,
A speech synthesizer, further comprising: a printing unit that converts the synthesized speech data generated by the phoneme editing processing unit into a musical score and prints it. In any one of claims 1 to 4,
The speech synthesis apparatus, wherein the phoneme editing processing means increases the pronunciation time by a predetermined value when the phoneme type is a long sound of Japanese Kana characters. In any one of claims 1 to 5,
The phoneme editing processing means, when the type of the phoneme is a roaring sound of “tsu” or “ya”, “yu”, “yo” of the Japanese kana character “tsu”, the phoneme and the phoneme immediately before the phoneme A speech synthesizer characterized by reducing a silent time with a phoneme, a silent time between the phoneme and a phoneme immediately after the phoneme, and a pronunciation time of the phoneme by a predetermined value. In any one of claims 1 to 6,
When the phoneme editing processing means extracts the phoneme code corresponding to the given phoneme code identification information from the phoneme code database and sets the time to specify the start and end of pronunciation according to the type of the phoneme, the silence A speech synthesizer characterized in that time is multiplied by a set time expansion / contraction rate, and a predetermined modification is applied to the time for specifying the start and end of the pronunciation. In any one of claims 1 to 7,
Set when the phoneme editing processing means extracts the phoneme code corresponding to the given phoneme code identification information from the phoneme code database and sets the time to specify the start and end of pronunciation according to the type of the phoneme. Based on the pitch offset parameter, the pitch offset parameter is added to the pitch of each code code constituting the phoneme code recorded in the phoneme code database to constitute the synthesized speech data A speech synthesizer characterized in that a predetermined modification is applied to the pitches of all code codes. In claim 8 ,
In the synthesis instruction data, the pitch offset parameter is defined together with phoneme code identification information for each phoneme, and the phoneme editing processing means includes:
When the phoneme code corresponding to the given phoneme code identification information is extracted from the phoneme code database and the time for specifying the start and end of pronunciation is set according to the type of the phoneme, the phoneme defined for each phoneme is defined. Based on the high offset parameter, the pitch offset parameter is added to the pitch of each code code constituting the phoneme code recorded in the phoneme code database, and all of the synthesized speech data are configured. A speech synthesizer characterized by modifying a pitch of a code code. Program for causing a computer to function as a speech synthesis apparatus according to any one of claims 1 to 9.

Images