JP2011090218A - Phoneme code-converting device, phoneme code database, and voice synthesizer - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a phoneme code-converting device capable of improving articulation at voice synthesis time on the basis of a coded data by obtaining a syllable code in which frequency variation is accurately extracted by improving time resolution while maintaining frequency analysis precision. <P>SOLUTION: A syllable recording signal which consists of J sample rows is expanded in time sequence Q times, to make J×Q sample rows (a). A unit section which consists of T (T<J) samples for sample rows is set over all sections, and P pieces of codes are created by performing frequency analysis in each unit section. For each code, each pitch is multiplied by Q, and time information is multiplied by 1/Q and the code is corrected (b). A total sum of energy values is obtained by multiplying the code by time difference of sounding start time and end time by pitch for the syllable code which is consisted as a code group, and an integral energy value is obtained by multiplying all energy total values by pitch among a plurality of syllable codes. A high rank pitch in which the value is high is extracted, and intensity, the start time and the end time corresponding to each pitch are set, and a phoneme code is obtained. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a technique for obtaining code data that can be used in an electronic musical instrument, a musical score, and the like based on a human voice.

  Conventionally, a method of artificially synthesizing a human voice has been used in various fields. The applicant has proposed a technique for synthesizing speech by digitizing a human voice with PCM and then obtaining a code code corresponding to a frequency having a large effective intensity using Fourier transform (Patent Documents 1 to 3). 5).

  In addition, the applicant can play back even a low-performance MIDI sound source installed in toys, etc., and read it with an existing score editing tool and convert it to a staff score so that a legible score can be obtained. A technique for converting a code code group for each syllable into a code code group for each phoneme has been proposed (see Patent Document 6).

Japanese Patent No. 3795201 Japanese Patent No. 4037542 Japanese Patent No. 4156268 Japanese Patent No. 4132362 Japanese Patent No. 4061070 Japanese Patent Application No. 2009-143825

  In the invention described in Patent Document 6, for a syllable code configured as a code code group such as MIDI, the product of the time difference between the sound start time and the sound end time and the strength of the code code for each pitch. Calculate the energy sum that is the sum of the given energy values, and calculate the integrated energy value by multiplying all energy sum values for each pitch between multiple syllable codes. Since the pitch is extracted and the phoneme code is obtained, the phoneme code that can express all syllables with a smaller number is obtained by using the syllable code actually obtained by human speech. Became possible.

  However, in the invention described in Patent Document 6, a set of waveform voice data obtained by recording Japanese syllable 71 sounds uttered by the same speaker is converted into high-definition MIDI data, and a plurality of converted syllable MIDI data is multiplied. By combining them, a method of extracting phoneme MIDI data corresponding to 20 Japanese phonemes and creating a database was used. In this case, the quality of the converted high-definition MIDI data affects the quality of the phoneme MIDI data in which the separation performance on the time axis of consonant phonemes and vowel phonemes in the syllable is generated. However, when the previously proposed MIDI encoding method (Patent Documents 1 to 5) is applied to a short speech syllable signal targeted at this time, a consonant phoneme component and a vowel phoneme component are mixed, so that the previously proposed syllable MIDI data is mutually exclusive. It has been found that it is difficult to generate phoneme MIDI data with good clarity by the multiplication. This is because the time resolution in the frequency analysis is insufficient, and it is not possible to follow the rapid frequency fluctuation of the speech, and it has not been possible to separate and encode the consonant phonemes from the syllable components. As a method of improving the time resolution, there is a method of setting the frame length at the time of frequency analysis to be small, but in return, the frequency analysis accuracy is lowered and it is difficult to obtain appropriate encoded data. In addition, components that are common to consonant phonemes and vowel phonemes, such as low-pitched vocal cord fundamental vibration sounds, are included in the limited chords, leading to a decrease in clarity.

  Therefore, in the present invention, while maintaining the same frequency analysis accuracy as before, the time resolution in the analysis is improved, and a syllable code in which frequency fluctuations are extracted with high accuracy can be obtained, and a speech synthesis function is provided based on the code data. It is an object of the present invention to provide a phoneme code conversion apparatus capable of improving the clarity of voice reproduction quality when realized.

  In order to solve the above-mentioned problems, in the present invention, a syllable waveform acquisition unit that acquires one syllable uttered as a syllable waveform signal of M samples at a predetermined sampling frequency, and maintains the sampling frequency of the syllable waveform signal. On the time axis, the syllable waveform magnifying means for magnifying by Q (Q is an integer of 2 or more) and converting it to an enlarged waveform signal of J × Q samples, and performing a predetermined frequency analysis on the enlarged waveform signal, Code code group creating means for creating a code code group represented by a plurality of code codes having start time, sound generation end time, pitch, and intensity data, and pitches of all the code codes constituting the code code group The syllable code generator creates a syllable code consisting of the corrected code code group by correcting the sound generation start time and sound end time of all code codes by 1 / Q times. And, for each code code constituting the syllable code, an energy sum value that is the sum of energy values given by the product of the time difference between the sounding start time and the sounding end time and the strength of the code code is calculated for each pitch. A pitch-by-pitch energy calculation unit, a pitch-by-pitch energy integration unit that calculates an integrated energy value by multiplying all energy sum values for each pitch among a plurality of syllable codes, and the integrated energy value is high A predetermined number of upper pitches are extracted, and parameters of a predetermined strength, a predetermined pronunciation start time, and a predetermined pronunciation end time are set in a code code corresponding to each extracted pitch, and a plurality of codes A phoneme code conversion device having code conversion means for converting into a phoneme code composed of codes is provided.

  According to the present invention, after each intensity array of the digitized acoustic signal is enlarged by a predetermined magnification in the time axis direction, a predetermined number P kinds of intensity are obtained for each unit section composed of a predetermined number T intensity arrays. Since the spectrum intensity corresponding to the frequency is calculated, P code codes including the frequency and time are obtained, and the frequency of the P code codes is corrected by Q times and the time by 1 / Q times. While maintaining the same frequency analysis accuracy as the above, it is possible to improve the time resolution in the analysis, and obtain syllable codes mainly extracted with high accuracy in the frequency variation in the speech signal. Calculates the energy sum, which is the sum of the energy values given by the product of the time difference between the pronunciation start time and the pronunciation end time for each pitch, and the strength of the code code, and for each pitch between multiple syllable codes. The Enel -Since the integrated energy value is calculated by multiplying the total value and the higher pitches with the higher integrated energy value are extracted, it is possible to use the syllable code obtained by actually uttering by humans, It becomes possible to obtain phoneme codes that can express all syllables with a small number.

  In the present invention, a phoneme code representing a phoneme constituting each syllable of a Japanese kana character is a plurality of codes having a pitch equal to or lower than a predetermined type and having a uniform sound intensity and sound length. A phoneme code database is provided which is composed of codes and records each phoneme code in association with phoneme code identification information for identifying each phoneme code.

  According to the phoneme code database of the present invention, a phoneme code composed of a plurality of code codes having pitches equal to or less than a predetermined type and having a uniform sound intensity and sound length is recorded, and these phonemes are recorded. Since all the syllables are expressed by combining the codes singly or in combination, two syllables can be expressed with a smaller number of phoneme codes than the syllable codes.

  Further, according to the present invention, a plurality of code codes having a pitch equal to or lower than a predetermined type, and having a uniform sound intensity and sound length, corresponding to the phonemes constituting each syllable of Japanese Kana characters. Convert phoneme codes to phoneme code identification information based on phoneme code database recorded in association with phoneme code identification information for identifying phoneme codes and syllable information described in given synthesis instruction data A corresponding phoneme code is extracted from the phoneme code database, a time for specifying the start and end of pronunciation is set according to the syllable information, and a predetermined time is specified from the time for specifying the end of pronunciation of the phoneme corresponding to the vowel phoneme Provided is a speech synthesizer having phoneme editing processing means for generating synthesized speech data by setting a time including a silent section as a time for specifying the start of pronunciation of a subsequent syllable.

  The speech synthesizer according to the present invention has a phoneme code database in which phonemes constituting each syllable of Japanese kana characters are recorded as phoneme codes, and is based on syllable information described in given synthesis instruction data. The corresponding phoneme code is extracted from the phoneme code database, the time for specifying the start and end of pronunciation is set according to the syllable information, and a predetermined silent interval from the time for specifying the end of pronunciation of the phoneme corresponding to the vowel phoneme The code code group created by performing speech synthesis is a low-performance MIDI sound source mounted on a toy or the like. In addition to being reproducible, the existing music score editing tool can be converted into a readable staff notation that can be played by a performer with a musical instrument.

  According to the present invention, while maintaining the same frequency analysis accuracy as before, the time resolution in the analysis is improved, and the syllable code obtained by extracting the frequency fluctuation with high accuracy is obtained, and the speech synthesis function is provided based on the code data. In the case of realization, it is possible to improve the clarity of the audio reproduction quality.

It is a figure which shows the basic concept of the syllable and phoneme in this invention. It is a figure which shows the basic concept of the syllable and phoneme in this invention. It is a block diagram which shows one Embodiment of the phoneme code conversion apparatus which concerns on this invention. It is a flowchart which shows the outline | summary of the process which produces a syllable code | cord | chord from a syllable recording signal. It is a figure which shows the concept of the expansion of a time-axis direction, the increase in a frequency, and reduction | decrease of time information. It is a figure which shows the conversion table which recorded the correspondence of the syllable and the phoneme. It is a figure which shows the mode of a change of the code code group at the time of converting from a syllable code to a phoneme code. It is a figure which shows the structural example of a syllable code | symbol and a phoneme code | symbol. It is a figure which shows the example of the phoneme code | symbol of the male voice stored in the phoneme code | symbol memory | storage part. It is a figure which shows the example of the phoneme code | symbol of the male voice stored in the phoneme code | symbol memory | storage part. It is a figure which shows the example of the phoneme code | symbol of the female voice stored in the phoneme code | symbol memory | storage part. It is a figure which shows the example of the phoneme code | symbol of the female voice stored in the phoneme code | symbol memory | storage part. It is a figure which shows the example which made the phonetic code | symbol of the male voice shown in FIG. 9, FIG. It is a figure which shows the example which made the female phoneme code | symbol shown in FIG. 11, FIG. 12 the staff score. It is a block diagram which shows one Embodiment of the speech synthesizer which concerns on this invention.

(1. Basic concepts of syllables and phonemes in the present invention)
DESCRIPTION OF EXEMPLARY EMBODIMENTS Hereinafter, preferred embodiments of the invention will be described in detail with reference to the drawings. First, the basic concept of syllables and phonemes in the present invention will be described. It is known that Japanese vowels can be approximated by chords of four or more overtones including two characteristic speech formant components. In addition to vowel components, consonants need three types of connected chords: chords that express noise such as friction sounds and chords that express vowels to vowels. If it is left to the phenomenon (sound pulse concentration), it can be approximated to two chords of the first noise and vowel. Therefore, the applicant thinks that speech synthesis can be realized by expressing all phonemes based on a single quadruple for Japanese vowels and two quadruples for consonants, and connecting them on the time axis. It was.

  However, due to various factors, there is a problem in that the sound is not clear even when various musical instrument sounds are set and reproduced with a MIDI sound source. In the present invention, the configuration of syllables such as vowels and consonants is fundamentally different from the conventional one. Specifically, 71 syllables are selected by combining one or two phoneme codes from 20 phoneme codes corresponding to Japanese 5 vowels and 15 consonants, instead of using 71 unique syllable codes. I decided to express it.

  First, the difference between vowel syllables and consonant syllables will be described. In this embodiment, the vowel syllable is a single phoneme including only the first phoneme. The consonant syllable is composed of two phonemes of a first phoneme and a second phoneme, and the first phoneme is shorter than the second phoneme. In the present embodiment, the first phoneme of the vowel syllable, the first phoneme of the consonant syllable, and the second phoneme are all 8 chords.

  As the first phoneme of the consonant syllable, any one of 14 phonemes “K, S, T, N, H, M, R, G, Z, D, B, P, Y, W” is selected, and the consonant syllable is selected. As the first phoneme of the vowel syllable of the second phoneme, any one of six phonemes “A, I, U, E, O, n” is selected. The consonant syllable includes a muddy sound (such as “B”) and a semi-voiced sound (such as “Pa”). As for the repellent sound (Hatsuon “N”), the first phoneme is “N” and the second phoneme is “n”. "

  The present invention is characterized in that consonant syllables and vowel syllables having the above-described configuration are synthesized in various modes according to the characteristics of speech. There are various possible ways of classifying the synthesis according to the features of the speech. In this embodiment, however, vowels, consonants (including muddy, semi-turbid, and repellent), long vowels, long consonants, prompt sounds, stuttering These are classified into 6 types.

  1 and 2 are diagrams showing the basic concept of syllables and phonemes in the present embodiment. FIGS. 1A to 1D and FIGS. 2A and 2B correspond to the above classifications. 1 and 2, each rectangle indicates a syllable or phoneme, and a space between the rectangles indicates a silent part. The horizontal direction is the time axis, and the horizontal width of the rectangle corresponds to the syllable pronunciation time. The syllable pronunciation time is set based on the basic vowel and consonant pronunciation time as a unit interval, and the other syllable syllables are also defined on the basis of this unit interval. The specific length of the unit section can be set as appropriate, but is set to 0.25 seconds in this embodiment. In addition, as will be described in detail later, in this embodiment, a syllable is encoded with two patterns, and is composed of a syllable composed of one phoneme and two consecutive phonemes (hereinafter referred to as double phonemes). There are syllables.

  FIG. 1A shows an example of a synthesis pattern in the case of a vowel. In the case of a vowel, the syllable is composed of one phoneme (first phoneme). As shown in FIG. 1 (a), after the first phoneme is sounded for a unit interval, it is silenced for the unit interval, and then another syllable continues. FIG.1 (b) is an example which shows the synthetic | combination pattern in the case of a consonant. In the case of consonants, a syllable is composed of two phonemes, a first phoneme and a second phoneme. As shown in FIG. 1B, the first phoneme is pronounced by 1/4 of the unit interval, and then the second phoneme is pronounced by 3/4 of the unit interval. That is, the second phoneme is started to be pronounced simultaneously with the end of the first phoneme, and the unit interval is from the start of the first phoneme to the end of the second phoneme. After that, silence is made for the unit interval, and then another syllable follows.

  FIG.1 (c) is an example which shows the synthetic | combination pattern in the case of a long vowel. In the case of a long vowel, it is composed of one phoneme, but as shown in FIG. 1 (c), the pronunciation time is extended by a unit interval compared to a normal vowel. That is, the first phoneme is pronounced for two unit intervals. After that, silence is made for the unit interval, and then another syllable follows. FIG. 1D is an example showing a synthesis pattern in the case of a long consonant. In the case of a long consonant, it is composed of two phonemes, but as shown in FIG. 1 (d), the pronunciation time of the second phoneme is extended by the unit interval as compared with the normal consonant. The pronunciation time of the first phoneme is 1/4 of the unit interval, as in the case of normal consonants. That is, the pronunciation of the second phoneme is started simultaneously with the end of the pronunciation of the first phoneme, and the period from the start of the pronunciation of the first phoneme to the end of the pronunciation of the second phoneme is two unit intervals. After that, silence is made for the unit interval, and then another syllable follows.

  FIG. 2A shows an example of a composite pattern in the case of a prompt sound. In the case of the prompt sound, as shown in FIG. 2A, the first phoneme of the consonant that is the subsequent syllable is generated immediately before the subsequent syllable. For example, when synthesizing “tsu” in “tta”, the first phoneme of the subsequent syllable “ta” is arranged immediately before the first phoneme of “ta”. At this time, the sounding of the prompt sound is started when 3/4 of the unit interval has elapsed from the end of the sounding of the preceding phoneme. That is, from the point when 3/4 of the unit interval has elapsed since the end of the pronunciation of the preceding phoneme, the first phoneme of the subsequent syllable is repeatedly generated twice by 1/4 of the unit interval, and then the second phoneme of the subsequent syllable is Only 3/4 of the unit interval is pronounced.

  FIG. 2B is an example showing a synthesis pattern in the case of stuttering. In the case of stuttering, as shown in FIG. 2B, the immediately preceding consonant syllable is only the first phoneme. The first phoneme and the second phoneme of the consonant corresponding to the stuttering are continuously pronounced using the pronunciation end time of the first phoneme of the immediately preceding consonant syllable as the pronunciation start time of the consonant corresponding to the stuttering. For example, when synthesizing “a” in “ca”, the first phoneme of the consonant “ya” corresponding to the stuttering “a” is arranged immediately after the first phoneme of the preceding syllable “ki”. At this time, the sounding time of the first phoneme of the immediately preceding consonant syllable is ¼ of the unit interval, and the sounding time of the consonant corresponding to the stuttering is 1 / phoneme of the unit interval as in the case of normal consonant. 4. The second phoneme is 3/4 of the unit interval.

  In FIGS. 1A to 1D and 2B, the preceding syllable is omitted, and in FIGS. 2A and 2B, the subsequent syllable is omitted. Silence intervals corresponding to unit intervals are set between phonemes. The specific length of this unit section is set to 0.25 seconds as a standard as described above, but can be changed as appropriate. 1 and 2 are basic styles, the ratio of the pronunciation time of each phoneme in each syllable, the length of the pronunciation time and the silence time can be changed as appropriate. For example, the speech speed can be changed by changing the interval between adjacent syllables.

(2. Preparation)
Next, preparations using the prior art will be described. As preparation, digitize human voice in syllable units. This is done by using a technique such as PCM to record each syllable actually spoken by a human. At this time, the speaker is a native Japanese man or woman, and it is desirable that the same person utter as much as possible with respect to 71 syllables with the same pitch (pitch) and utterance interval. Since it is difficult for ordinary speakers to arrange the utterances in this way, it is desirable to have an announcer or vocalist who has received specialized voice training speak. Furthermore, it is desirable to collect multiple sets of 71 syllable recordings by multiple speakers, including men or women. A 71-syllable recorded signal can be obtained in advance. The Japanese 71 syllable is a total of 71 sounds including so-called fifty sounds, repelling sounds, muddy sounds, and semi-voiced sounds.

(3. Conversion to phoneme code)
Next, conversion from a recording signal for each syllable to a phoneme code will be described. FIG. 3 is a block diagram showing an embodiment of a phoneme code conversion apparatus according to the present invention. The storage means 10 includes a syllable recording data storage unit 11, a syllable code storage unit 12, and a phoneme code storage unit 13, and is realized by an external storage device such as a hard disk connected to a computer. The syllable code storage unit 11 stores a recording signal created by the above-described preprocessing in association with syllable identification information for specifying a syllable. The syllable recording data storage unit 11 stores a conversion table indicating the correspondence between syllables and phonemes, and stores encoded syllable codes in association with syllable identification information. The syllable code storage unit 12 stores a conversion table indicating the correspondence between syllables and phonemes, and stores encoded syllable codes in association with syllable identification information. The phoneme code storage unit 13 stores the encoded phoneme code in association with the phoneme code identification information.

  The processing control means 20 controls the processing of the entire phoneme code conversion apparatus, and includes a syllable waveform expansion means 21, a code code group creation means 22, a syllable code creation means 23, a pitch-by-pitch energy calculation means 24, and a pitch-by-pitch. It has energy integration means 25 and code conversion means 26. The processing control means 20 is a computer main body including a CPU and a memory, and includes a syllable waveform enlarging means 21, a code code group creating means 22, a syllable code creating means 23, a pitch specific energy calculating means 24, and a pitch specific energy integrating means 25. The code conversion means 26 is realized by the CPU executing a dedicated program. The code display means 30 displays syllable codes and phoneme codes processed by the process control means 20, and is realized by a display device such as a liquid crystal display.

  Next, the processing operation of the phoneme code conversion apparatus according to this embodiment will be described. FIG. 4 is a flowchart showing an outline of processing until a syllable code is generated in the phoneme code conversion apparatus according to this embodiment.

  First, the processing control means 20 reads a syllable recording signal to be processed from a data input device (not shown). The syllable recording signal is obtained by sampling an analog recording signal with a predetermined sampling frequency and the number of quantization bits. In this embodiment, the sampling is performed with a sampling frequency of 44.1 kHz and a quantization bit number of 16 bits. I will explain below. When sampling is performed at a sampling frequency of 44.1 kHz, the syllable recording signal is configured as a sample string (array of samples) having 44100 samples per second.

  When the syllable recording signal is read, the syllable waveform enlarging means 21 expands the samples constituting the syllable recording signal by a predetermined magnification Q (Q is an integer) in the time axis direction (S1). Specifically, the number of samples constituting the syllable recording signal is multiplied by Q. Then, every Q samples having the same value as the original sample are arranged, and (Q-1) sample values between them are linearly interpolated using the values of the original samples located on both sides. give. If the sample value for each sample j (j = 0... J-1) of the original syllable recording signal is x (j), the computer executes the processing according to the following [Equation 1], A sample value x ′ (j · Q + k) is calculated for each sample j · Q + k (0 ≦ k ≦ Q−1) of the expanded syllable recording signal. In the following [Formula 1], w is a real value taking a value of 0 ≦ w ≦ 1 given by k / (Q−1).

[Formula 1]
x '(j.Q + k) = (1-w) .x (j) + w.x (j + 1)

  As a result of the processing in S1, J samples constituting the digital audio signal are expanded to J × Q. FIG. 5A shows a change in waveform due to the enlargement process in S1. The waveform in FIG. 5A is obtained by connecting plots of sample values with line segments, but is expressed in a curved line due to the large number of samples. By executing the processing according to the above [Equation 1], the waveform shown on the left side changes to the waveform shown on the right side. In the example of FIG. 5, the case of Q = 2 is shown for convenience of explanation.

  Next, the code code group creating means 22 sets a unit section on the sample expanded in the time axis direction (S2). The length of the unit interval (number of samples T) is set in relation to the sampling frequency. However, if the sampling frequency is 44.1 kHz, 4096 samples or more are required to faithfully analyze the low frequency region. . Therefore, in this embodiment, the unit interval is set as the number of samples T per unit interval T = 4096.

  Setting of the unit section is performed by sequentially extracting samples from the head of the digital sound signal as disclosed in Patent Documents 1 to 5. The unit interval is set so as not to leak all samples, and is preferably set so that the samples overlap in continuous unit intervals. In this case, the head interval (referred to as shift width) of each unit section can be set according to various rules. The simplest is a method in which the shift width is fixed, that is, the number of overlapping samples is set constant. For example, when T = 4096, the first unit interval is j = 0-4095, the second unit interval is j = 2048-6143, the second unit interval is j = 4096-8191, and so on. / 2) Setting is performed with overlapping samples. However, in order to improve the time resolution, there is a demand for reducing the shift width. On the other hand, there is a problem that the calculation time increases as the shift width is reduced. Also, if the shift width is made smaller than necessary, the connection condition is not satisfied in the connection processing of single-tone components in FIG. Therefore, in order to set an optimum shift width in accordance with the state of the acoustic signal, in the present embodiment, the zero crossing point in which the frequency change is noticeable by the denseness or autocorrelation analysis of the zero crossing interval as disclosed in Patent Document 5 is used. The sample is selected and the sample located at this zero crossing is set as the head.

  The zero crossing point is a crossing point between the positive and negative acoustic signals and the 0 level of the signal. Here, the zero crossing point indicates the time when the intensity value (amplitude) of the acoustic signal becomes zero. However, the digitized acoustic signal does not necessarily sample the zero crossing point in the analog signal. Therefore, in actuality, in addition to the case where the intensity value is just 0, when the intensity value of the sampling point changes from positive to negative or from negative to positive, one of the sampling points before and after that is regarded as a zero crossing point. Process. In order to detect the zero crossing, it is necessary that the acoustic signal to be analyzed has both positive and negative polarities. Therefore, it is necessary to remove the direct current component for the acoustic signal including the direct current component. Since various known methods can be applied to the removal of the DC component, detailed description thereof is omitted here. Basically, the unit section is set with the sample located at the zero crossing as the head, but the unit section is set with the position other than the zero crossing as the head so that the shift width of the continuous unit sections falls within a certain range. In some cases. Specifically, when the maximum shift width (for example, T / 2) is exceeded, the unit section is set starting from the position having the maximum shift width even at a position other than the zero intersection. On the other hand, when it is below the minimum shift width (for example, T / 8), a unit interval is set starting from the position where some zero crossing points are skipped so as to exceed the minimum shift width, exceeding the minimum shift width and maximum If there is no corresponding zero-intersection in the range of the shift width, correction is performed so that the unit section is set with the position having the maximum shift width as the head as described above.

Subsequently, the code code group creating unit 22 performs frequency analysis for each set unit section, and calculates a spectrum of each unit section (S3). As disclosed in Patent Documents 1 to 5, the spectrum of each unit section is calculated by 128 analysis frequencies f (n) = 440 · 2 corresponding to MIDI note numbers n (0 ≦ n ≦ 127). ^{This is} performed by extracting 128 components by generalized harmonic analysis based on ^{(n-69) / 12} element signals (element functions). When the analysis frequency is set in correspondence with the note number n, the frequency interval between the note numbers becomes wider as the frequency becomes higher. In particular, when n exceeds 60, the analysis accuracy decreases. Therefore, in this embodiment, as disclosed in Patent Document 3, 128M element signals f (n, m) = 440 · 2 ^{(n−69 + m /} ) obtained by dividing a note number into M differential sounds. Analyze using ^{M) / 12} to extract 128M components. Unless special encoding such as addition of a pitch bend code is performed in FIG. 4 or S4 described later, information on M differential sounds at each note number is unnecessary. As a component in the note number, 128 components are extracted as a result.

  As a specific processing procedure by the code code group creating means 22, first, an intensity array E (n) (0 ≦ n ≦ 127) and a sub-frequency array S (n) for the note number are set, and all initial values are set. 0. Subsequently, a process according to the following [Formula 2] is executed for 0 ≦ n ≦ 127 and 0 ≦ m ≦ M−1 to obtain (nmax, mmax) that maximizes E (n, m). .

[Formula 2]
A (n, m) = (1 / T (n)) · Σi _{= 0, T (n) −1} x (i) sin (2πf (n, m) i / fs)
B (n, m) = (1 / T (n)). Σi _{= 0, T (n) −1} x (i) cos (2πf (n, m) i / fs)
{E (n, m)} ² = {A (n, m)} ² + {B (n, m)} ²

  In the above [Equation 2], T (n) is the analysis frame length, and is set so as to be the maximum integer multiple of the period of the element signal within a range not exceeding the unit interval T. (N) = k / f (n, m). F (nmax, mmax) using (nmax, mmax) that maximizes E (n, m) is selected as the harmonic signal. When (nmax, mmax) is obtained, the computer executes processing according to the following [Equation 3] using A (nmax, mmax) and B (nmax, mmax) to obtain a sample array x (i). Update all elements (0 ≦ i ≦ T−1).

[Formula 3]
x (i) ← x (i) −A (nmax, mmax) · sin (2πf (nmax, mmax) i / fs) −B (nmax, mmax) · cos (2πf (nmax, mmax) i / fs)

  In the above [Equation 3], the process of subtracting the content signal from x (i) is performed. Further, processing according to the following [Equation 4] is executed to update the intensity array E (n) and the sub-frequency array S (n).

[Formula 4]
E (nmax) ← E (nmax) + E (nmax, mmax)
S (nmax) ← mmax

  The computer executes the processes of [Formula 2] to [Formula 4] for all n (0 ≦ n ≦ 127), and determines the values of all E (n) and S (n).

  In the present embodiment, in order to reduce the processing load, the value of M is variably set based on the note number, for example, the frequency interval to be analyzed is about 100 Hz. And note number 60 and below are not divided and M = 1. Although the accuracy is slightly reduced, S (n) is determined by the first [Formula 2] process, and the second and subsequent [Formula 2] processes are performed with m = S (n) fixed. Sound analysis may be omitted. Further, in the processing of [Equation 2], there is a possibility that signal components having different sub-frequency with respect to the same note number may be analyzed a plurality of times, but E (n) and S (n) are already values. May be excluded from selection candidates for the maximum value of E (n, m).

  When the spectrum (128 frequency components) is calculated for each unit section, the syllable code creation means 23 can specify the frequency information, the spectrum intensity corresponding to each frequency, and the time information that can specify the start and end of the unit section. (S4). In creating a code code, first, information of time and time length of each note number n is added to the calculated spectrum, and [start time, time length, main frequency n, sub-frequency S (n), intensity E (n )] Is created. The “start time” may be information that can specify the start time of the unit section in the entire digital sound signal. In this embodiment, the entire digital sound signal attached to the start sample (i = 0) of the unit section. Sample number (corresponding to absolute sample address: j) is recorded. By dividing this absolute sample address by the sampling frequency (44100), the time from the head of the acoustic signal is obtained. In this embodiment, the time length is variably given for each unit section, and is a difference (start time of the subsequent unit section−start time of the unit section) immediately after the start time of the subsequent unit section. Given.

  For each unit section set in S2, 128 single-tone components are created. Further, in S4, processing for connecting the single-tone components in continuous unit sections is performed. Specifically, when a single note component of the same note number in a continuous unit section satisfies a predetermined connection condition, two single note components are connected. As the connection condition, a state having continuity as the same sound can be set as appropriate, but in this embodiment, the difference between the frequencies (main frequency + sub frequency) considering the sub frequency is less than a predetermined threshold value Ndif. When the two intensities are equal to or greater than the predetermined threshold Lmin and the difference between the two intensities is less than the predetermined threshold Ldif, the subsequent single sound component is connected to the preceding single sound component as having continuity. However, the connected main frequency, sub-frequency, and intensity use each value of the larger single tone component, and the time length is given as the sum of both. In the present embodiment, specific threshold values as connection conditions are Ndif = 8/25 [unit: note number conversion], Lmin = 1 [unit: 128 step velocity conversion], Ldif = 10 [unit: 128 step velocity conversion] It is said that. Since the concatenation process is performed before conversion to a code code, each threshold value is converted into a note number and velocity.

As long as the connection condition is satisfied, the single note components of the same note number are repeatedly applied to the single note component of the subsequent unit section, and finally obtained [start time, time length, main frequency n, sub frequency S ( n), a single tone component of intensity E (n)] is converted into a code code. The format of the code code may be any format as long as it has frequency information, spectrum intensity corresponding to each frequency, and time information that can specify the start and end of a unit section. However, in this embodiment, the data is converted to the MIDI format. In MIDI, sound generation start and sound generation end occur as separate events. Therefore, in this embodiment, one single tone component is converted into two MIDI note events. Specifically, a note-on event of note number n is issued at the “start time”, and the velocity value is 128 · {E (n) / Emax} ^1/4, where the maximum value of the intensity E (n) is Emax. Give in. In Standard MIDI File, it is necessary to give the time as a relative time (delta time) with the immediately preceding event, and the time unit can be defined by an arbitrary integer value, for example, converted to 1/1536 [seconds]. Give. Then, a note-off event of note number n is issued at the end time specified by “start time” + “time length”. At this time, the time length is multiplied by a real number between 0 and 1. This is because a note-off instruction is given early in consideration of the reverberation of the MIDI sound source, although it depends on the tone color of the MIDI sound source to be used. Even if the time length is used as it is, there is no problem in the processing of the MIDI sound source.

  When converting to MIDI code, it is necessary to adjust the number of simultaneous pronunciations in order to consider the number of simultaneous pronunciations that can be processed by the MIDI sound source. When the number of simultaneous sounds that can be processed by the MIDI sound source is 32, the number of note events during the sound generation period (note-on state) is continuously counted in the time axis direction, and there are locations where 32 note events exist simultaneously. If found, each pair of note-off events is searched in the neighborhood, and the priority is evaluated by the product (energy value) of the velocity value and duration value (note-off time-note-on time) of each note event pair. Then, a process of locally deleting note event pairs with low priority (low energy value) so as to be equal to or less than the specified number of chords (in this case “32”). “Locally” means that it is limited to a portion where there are more than 32 note events. At this time, if either the velocity value or the duration value is lower than the predetermined lower limit value, the deletion process is also performed regardless of the priority.

  Furthermore, when converting to a MIDI code, it is necessary to adjust the bit rate in order to consider the bit rate that can be processed by the MIDI sound source. In the time axis direction, the number of note-on or note-off events is counted at one-second intervals, the average code length is 5 bytes (40 bits), and the maximum bit rate that can be processed by the MIDI sound source is 9000 [bps (bits). / Sec)], if an interval in which the number of events per second exceeds 9000/40 = 225 is found, the note-off or note paired with the note-on or note-off event existing in that interval, respectively. An on-event is searched in the neighborhood interval, and the priority is evaluated by the product (energy value) of the velocity value and duration value (note-off time-note-on time) of each note event pair, and the specified number of events (in this case, “225”). ”) Locally delete note event pairs with low priority (low energy value) so that Cormorant. At this time, if either the velocity value or the duration value is lower than the predetermined lower limit value, the deletion process is also performed regardless of the priority.

When the code code is created, the syllable code creating means 23 performs a process of correcting each code code in order to correct the variation caused by the enlargement in the time axis direction (S5). Specifically, first, a process of adding 12 · log ₂ Q to the note number values of all the note events (note-on event or note-off event) is performed. For example, when Q = 4, the overall pitch is raised by 24 semitones (2 octaves). This process is performed to restore the original state by multiplying the frequency by Q, since the frequency is 1 / Q by multiplying the number of samples by Q in S1. By this correction, the code code having the note number whose note number exceeds the standard value upper limit 127 is deleted. Specifically, the code code with the note number before correction of 128-12 · log ₂ Q or more is deleted.

  Subsequently, the times (note-on time or note-off time) of all the note events are multiplied by 1 / Q. As a result, the performance time of the entire MIDI code and the sounding time of each note event are reduced to 1 / Q. This process is performed in order to set the time to 1 / Q and to return to the original state because the total performance time has become Q times by multiplying the number of samples by Q in S1. When this process is performed, the number of note events per hour increases by a factor of Q, so the bit rate adjustment executed in S4 is executed again.

  As a result of the processing in S5, the frequency (pitch) becomes Q times and the time information becomes 1 / Q. FIG. 5B shows how the MIDI event (MIDI code note event) is changed by the correction process of S5. In FIG. 5B, the change in the MIDI event when Q = 2 is indicated by a musical note. As a result of the correction process of S5, the left "mi" note is changed to the "mi" note that is one octave higher (double the frequency) on the right side. On the other hand, the left quarter note is changed to a half eighth note on the right side. The syllable code obtained in this way is stored in the syllable code storage unit 12 in association with the syllable identification information.

  In the above example, the bit rate is adjusted twice at S4 and S5. However, it is sufficient that these are finally performed at least once. Moreover, in the said embodiment, although the preferable process example was demonstrated about the process of S2-S4, these processes were disclosed by well-known patent documents 1-5 in the range which does not deviate from the meaning of this invention. Technology can be used.

  When the syllable code is obtained, the syllable code is converted into 20 phoneme codes based on the syllable code composed of 71 syllable units. Here, a conversion table showing the correspondence between syllables and phonemes is shown in FIG. In FIG. 6, 71 sounds “a”, “i”... Indicated by katakana are syllables, and “A” and “I” indicated by alphabets are phonemes. Vowel phonemes “A”... “O” are determined by an AND operation of 13 syllables in the horizontal direction, and consonant phonemes “K”... “P” are determined by an AND operation of 5 syllables in the vertical direction. The horizontally long frame shown in FIG. 6 indicates that the vowel phoneme “A” is determined by 13 syllables “a”... “Pa”, and the vertically long frame indicates the consonant phoneme “K”. It is determined that five syllables “K”... “K” are determined. The consonant phoneme “Y” is determined by the three syllables “ya”, “yu”, and “yo”, the consonant phoneme “W” is determined by the two syllables “wa” and “wo”, and the phoneme “n” is It is determined by one syllable “n”.

  The process control means 20 reads a plurality of related syllable codes with reference to the conversion table shown in FIG. For example, in order to obtain the phoneme code “K”, five syllable codes “K”, “K”, “K”, “K”, and “K” are read from the syllable code storage unit 11.

  When the processing control means 20 reads the five syllable codes, the pitch-specific energy calculation means 24 applies the pitch (note number in the case of MIDI) to all the code codes that constitute each syllable code unit. ) Separately, calculate the total energy value. The total energy value is calculated by the sound intensity at each pitch (velocity in the case of MIDI) × sounding time (in the case of MIDI, duration: note-off time−note-on time). Here, “total” of energy sum values means that when two or more code codes exist at the same pitch in one syllable code, the sum is taken for each code code.

  After the energy sum value is calculated for all pitches in each code unit, the pitch-based energy integration means 25 then multiplies the energy sum value for the same pitch among the five syllable codes to obtain the integrated energy. Get the value. Since this is multiplication, the pitch component that does not exist in any one of the five syllable codes is “0”. Therefore, by obtaining the integrated energy value, only the components common to the five syllable codes remain. For example, the pitch component constituting the phoneme code “K” must be a pitch component that is commonly included in the syllable codes “K”, “K”, “K”, “K”, and “K”.

  When the integrated energy value is calculated for each pitch, the code conversion means 26 selects a pitch whose integrated energy value does not exceed the specified number of chords (for example, 8).

  The specified number of chords is set in advance and can be set freely. In the present embodiment, it is “8” as described above. Therefore, in this embodiment, 8 code codes are extracted from the 32 code codes by the code conversion means 23. Further, the code conversion means 26 selects the earliest section start time and the latest section end time from among the plurality of code codes constituting each original syllable code, the section start times and sections of all the selected pitches. Set as the end time, and set the velocity of the selected pitch to the specified value. Since the velocity can take a value from “0” to “127”, the maximum value is set to “127” in the present embodiment.

  FIG. 7 shows how the phoneme codes change before and after the processing by the pitch-by-pitch energy calculation unit 24, the pitch-by-pitch energy integration unit 25, and the code conversion unit 26. In FIG. 7, the horizontal axis corresponds to time, and the vertical axis corresponds to frequency (note number). The rectangle arranged in the graph shows the code code, and the horizontal length shows the temporal length according to the horizontal axis, but the vertical length is different from the vertical axis, not the frequency but the intensity. (Velocity).

  Actually, as shown in FIG. 6, one phoneme is obtained by 13 syllables or 5 syllables (except that phoneme Y is 3 syllables, phoneme W is 2 syllables, In the example of FIG. 7, the phoneme n is one syllable), and the phoneme γ is extracted using two syllable codes α and β for convenience of explanation. FIGS. 7A and 7B show code codes constituting the syllable codes α and β before processing by the pitch-by-pitch energy calculation unit 21, respectively. As described above, in this embodiment, a syllable code is generally composed of 32 code codes at the same time, and the designated number of chords is set to 8. However, in FIG. For convenience, there are six code codes at the same time, and the designated chord number is set to four. Further, as can be seen from the horizontal and vertical lengths of the rectangles indicating the respective code codes, the reproduction time (end time-start time) and intensity of each code code are also different.

  After the processing by the pitch-specific energy calculation means 24 and the pitch-specific energy integration means 25, integrated energy values for all pitches are obtained. As shown in FIGS. 7A and 7B, the integrated energy value indicates the time difference between the sound generation start time and the sound generation end time calculated for each pitch by the pitch-specific energy calculation means 24, and the strength of the code code. Since the energy sum value, which is the sum value at the same pitch of the product of (velocity), is multiplied by the energy sum values at the corresponding pitches between the syllable codes α and β, Although it is difficult to express, as shown in FIG. 7 (c), the integrated energy value (represented by the width in the vertical direction of the drawing in FIG. 7 (c)) corresponds to the pitch. can get. Then, as shown in FIG. 7D, the code converting means 26 selects four pitches having a large integrated energy value. Further, thereafter, the code conversion means 26 sets the strength value of the code code corresponding to each pitch to the specified value as described above. Five syllable codes (up to 32 pitches) of “K”, “Ki”, “K”, “K”, and “K”, and “A”, “I”, “U”, “E”, “O”, and “K”. A configuration example of one phoneme code (maximum 8 pitches) is shown in FIG.

  As shown in FIG. 6, the phoneme code of one vowel is determined by multiplying the energy sum values at each pitch of 13 syllable codes, and the phoneme code of one consonant has some exceptions. Except for the five sums of energy at each pitch, the order of determining these 20 types of phoneme codes is not particularly limited. Then, a certain pitch may be selected between the vowel phoneme code and the consonant phoneme code. That is, a consonant phoneme and a vowel phoneme composed of limited pitches such as 8 sounds may be reproduced continuously in time during speech synthesis, which is not preferable in terms of clarity. Therefore, in the present application, a method is proposed in which the same pitch is not included as much as possible between the codes of consonant phonemes and vowel phonemes. For this purpose, first, the phoneme codes of five types of vowels are determined by multiplying the energy sum values at the pitches of the 13 syllable codes. Subsequently, the syllable codes of five consonants including any of the five vowels “A”, “I”, “U”, “E”, “O” (for example, “K”, “K”, “K”, “K”, “ )), The energy sum values corresponding to all the pitches of the already determined phoneme codes of the vowels included in each syllable code are reduced by a certain ratio . For example, the energy sum value is multiplied by 1/1000, and the energy sum value of other pitches is changed to a relatively small value. The value to be multiplied is not necessarily 1/1000, and may be less than 1. Thereafter, the phoneme codes of five types of vowels are determined by multiplying the modified energy sum values at the pitches of the five syllable codes. By performing such correction processing, the probability that the same pitch is selected redundantly between the codes of consonant phonemes and vowel phonemes is reduced. (However, if the number of pitches with a high energy sum is originally low, they may be selected somewhat redundantly.)

  The pitch-by-pitch energy calculation unit 24, the pitch-by-pitch energy integration unit 25, and the code conversion unit 26 perform processing on each syllable code stored in the syllable code storage unit 12, and store the obtained phoneme codes. Stored in the unit 13. Examples of phoneme codes stored in the phoneme code storage unit 13 are shown in FIGS. Of these, FIGS. 9 and 10 are encoded male voices, and FIGS. 11 and 12 are encoded female voices. 9 to 12, “C, C #, D, D #, E, F, F #, G, G #, A, A #, B” are “do, de #, les, les #, "Mi, Fah, Fah #, Seo, Seo #, La, La #, Shi" in English notation, the numbers listed indicate octave numbers, and the MIDI standard is a pair symbol of pitch names and octave numbers. The note number can be specified, and in the present application, the MIDI standard note number 69 is expressed as A3 (internationally, there are many notations indicating A4 as MIDI standard note number 69). When the code code constituting the phoneme code is defined by the MIDI standard, it can be converted into a staff score by a commercially available score editing tool. FIG. 13 shows an example in which the phoneme codes of male voices in FIGS. 9 and 10 are converted to a staff, and FIG. 14 shows an example in which the phoneme codes of female voices in FIGS.

(4. Speech synthesis)
Next, speech synthesis using the obtained phoneme code will be described. FIG. 15 is a block diagram showing an embodiment of a speech synthesizer according to the present invention. In FIG. 15, the phoneme code database 13a records the obtained phoneme codes in association with the phoneme code identification information. The phoneme codes stored in the phoneme code database 13a are the same as those stored in the phoneme code storage unit 13 after being converted by the phoneme code conversion device described above. Therefore, it can be said that the phoneme code conversion apparatus described above is for creating the phoneme code database 13a. The phoneme code database 13a records a conversion table indicating the correspondence between each phoneme code identification information and syllable identification information for specifying a syllable. This conversion table is the same as that shown in FIG. The synthesized speech data storage unit 14 stores the synthesized speech data synthesized by the phoneme editing processing unit 50, and is realized by a storage device such as a hard disk.

  The phoneme editing processing means 50 performs a process of extracting a corresponding phoneme code from the phoneme code database 13a according to the content of the synthesis instruction data, generating a synthesized speech data by performing a predetermined process, and outputting it to a predetermined output destination. . The generated synthesized voice data is output to one or more of the synthesized voice data storage unit 14, the voice output unit 60, and the printing unit 70 according to the setting. The voice output means 60 is for generating the synthesized voice data received from the phoneme editing processing means 50 as an actual voice, and is realized by a MIDI playback device having a MIDI sound source. The printing means 70 converts the synthesized voice data received from the phoneme editing processing means 50 into a staff score and prints it. The conversion to the staff score is realized by executing known conversion software. This is realized by a known printer or the like. The speech synthesizer shown in FIG. 15 is actually realized by incorporating a dedicated program into a computer that includes an input device and an external storage device and is connected to a MIDI playback device.

  The synthesis instruction data input to the speech synthesizer is obtained by arranging syllable identification information in a predetermined order. This syllable identification information can be in any format as long as it can identify a syllable. May be. In this embodiment, text data in which a character code corresponding to a syllable is recorded is used as syllable identification information. In this case, the conversion table in the phoneme code database 13a needs to record the character code corresponding to the syllable identification information and the phoneme code identification information in association with each other.

  Next, the processing operation of the speech synthesizer shown in FIG. 15 will be described. First, synthesis instruction data is input to the speech synthesizer. When the speech synthesizer reads the synthesis instruction data, the phoneme editing processing means 50 synthesizes the synthesis instruction data in order from the first syllable identification information. Specifically, the phoneme editing processing unit 50 refers to the conversion table in the phoneme code database 13a with the syllable identification information in the synthesis instruction data, acquires the phoneme code identification information, and corresponds to the phoneme code identification information. Extract phoneme codes.

  If the extracted phoneme code is only one vowel phoneme, it is a vowel syllable, so 0.25 seconds after the note-off time of the preceding syllable is set as the note-on time, and 0.25 seconds after that. Is a note off time, and a MIDI event is created in which the note number and velocity are the values recorded in the phoneme code database 12a. However, the note number is optionally raised or lowered based on a pitch offset parameter separately designated by the user so that pitch conversion can be performed.

  If there are two extracted phoneme codes, and they are consonant phonemes and vowel phonemes, they are consonant syllables. Therefore, note the previous consonant phoneme code 0.25 seconds after the note-off time of the previous syllable. The on-time is set, and 1/4 of the unit interval, that is, 0.0625 seconds later is set as the note-off time. For the subsequent vowel phoneme code, the note-off time of the previous consonant phoneme code is set as the note-on time, and 3/4 of the unit interval, that is, 0.1875 seconds later, is set as the note-off time. In the case of consonant syllables, as in the case of vowel syllables, the note number and velocity are the values recorded in the phoneme code database 12a. However, the note number is optionally a pitch offset parameter separately designated by the user. The pitch conversion can be realized by adding up and down as appropriate. In addition, the above time values of 0.25 seconds, 0.0625 seconds, and 0.1875 seconds are merely reference values, and are appropriately expanded / contracted by multiplying a time expansion / contraction parameter instructed by the user, so that speech speed conversion is performed. Can be realized.

  If the syllable identification information indicates a long sound (when the syllable identification information is recorded in a character code, it is a character code corresponding to “-”), the two syllable identifications together with the immediately preceding syllable identification information The information identifies one long syllable. For example, when “a” and “−” are consecutive in syllable identification information, it is determined that the syllable is a long vowel syllable by two syllable identification information “A”. When “K” and “-” are consecutive in the syllable identification information, it is determined that the syllable identification information is a long consonant syllable by the two syllable identification information “car”. In the case of a long sound, there is a slight difference between a long vowel syllable and a long consonant syllable. In the case of a long vowel syllable, the interval from the note-on time to the note-off time is set to 0.5 seconds. In the case of a long consonant syllable, the interval between the note-on time and the note-off time is set to 0.0625 seconds for the previous consonant phoneme code as in the normal case of not a long sound, and the note-on for the subsequent vowel phoneme code The interval from time to note-off time is set to 0.4375 seconds. Therefore, in the case of a long sound, the pronunciation time of the entire syllable is the same 0.5 seconds for both the vowel syllable and the consonant syllable. For consonant syllables, the pronunciation time of the first phoneme is 0.0625 seconds, which is the same as in the normal case where it is not a long sound, but the pronunciation time of the second phoneme is longer than in the normal case where it is not a long sound. The time values of 0.5 seconds, 0.25 seconds, 0.4375 seconds, 0.5 seconds, and 0.0625 are also reference values, and are multiplied by a time expansion / contraction parameter separately designated by the user. By doing so, it is appropriately expanded and contracted, and speech speed conversion can be realized.

  In the case of a prompt sound, the same phoneme as the first phoneme of the immediately following syllable is added immediately before the first phoneme of the immediately following syllable. Since the sound generation time of the first phoneme is 0.0625 seconds, the note-on time of the sound is set 0.1875 seconds after the sound generation end time of the preceding syllable, and the note-off time of the sound and the immediately following syllable The first phonemes have the same note-on time. Note that the time values of 0.0625 seconds and 0.1875 seconds are also reference values, and are appropriately expanded and contracted by multiplying a time expansion / contraction parameter instructed by the user to realize speech speed conversion. Can do.

  In the case of stuttering, it is added immediately after the first phoneme of the immediately preceding consonant. Therefore, the note-off time of the first phoneme of the immediately preceding consonant and the note-on time of the stuttering are set to be the same. Since the composition of the stuttering syllable itself is the same as the consonant, the note-off time of the first phoneme and the note-on time of the second phoneme are 0.0625 seconds after the note-on time of the first phoneme. The note-off time of the second phoneme is 0.1875 seconds after the note-on time. Note that the time values of 0.0625 seconds and 0.1875 seconds are also reference values, and are appropriately expanded and contracted by multiplying a time expansion / contraction parameter instructed by the user to realize speech speed conversion. Can do.

  The phoneme editing processing means 50 performs a phoneme synthesis process in units of syllable identification information in the read synthesis instruction data, and sequentially outputs synthesized voice data (MIDI data) to the voice output means 60 in units of processed syllables. I will pass it on. The audio output means 60 reproduces the MIDI data received from the phoneme editing processing means 50 in order. As described above, the speech synthesizer can reproduce speech according to the read synthesis instruction data.

  When outputting as a musical score, the synthesized voice data is converted into the musical score data by the printing means 70 and then printed out. Further, as in the above example, voice synthesis can be performed in real time according to the synthesis instruction data, and voice reproduction or stave output can be performed. In this voice synthesis apparatus, the result of processing by the phoneme editing processing unit 50 is used. Some MIDI data may be stored in the synthesized voice data storage means 13, and the MIDI data may be separately played back by a MIDI playback device. As a method for accumulating MIDI data in a storage device, an SMF (Standard MIDI File) format file can be used to pass it to various commercially available music-related software. Can be used to create a score. In this case, the score is created based on the phoneme code recorded in the SMF format. Then, if the generated score is output from the printing device, it can be used as an easy-to-read score when playing a musical instrument.

As described above, the phoneme editing processing means 50 extracts the corresponding phoneme code from the phoneme code database 13a using the syllable identification information in the synthesis instruction data, and when creating a MIDI event, the note number is stored in the phoneme code database 13a. An optional pitch offset parameter instructed by the user is optionally added to the note number of each note constituting the recorded phoneme code so that the pitch can be converted. In this case, the pitch of the synthesized voice data as a whole is not raised or lowered, but limited to vowel phonemes. Also, if the pitch offset parameter is defined for each syllable together with the syllable identification information in the synthesis instruction data, the pitch of the vowel phonemes configured for each syllable can be individually increased or decreased. That is, singing voice synthesis can be realized by providing a pitch change (pitch information) between adjacent notes of a melody created in advance as a pitch offset parameter defined together with syllable identification information in the synthesis instruction data.

  INDUSTRIAL APPLICABILITY The present invention can be used in a music production / composition support industry that imitates human voice reproduction performed for events and entertainment purposes. In the entertainment field, for musical scale reproduction media such as toys (including robots and stuffed animals) mainly made of electronic musical instruments, toy-type acoustic instruments (miniature pianos for interior decoration), music boxes, and ringtones for mobile phones. It can be used in industries that add speech synthesis functions. Further, it can be used in industries such as copyright protection when distributing MIDI music content by SMF (Standard MIDI File) or the like.

DESCRIPTION OF SYMBOLS 10 ... Memory | storage means 11 ... Syllable recording data memory | storage part 12 ... Syllable code memory | storage part 13 ... Phoneme code memory | storage part 13a ... Phoneme code database 14 ... Synthetic speech data memory | storage means 20 ... Processing control means 21: Syllable waveform expanding means 22 ... Code code group creating means 23 ... Syllable code creating means 24 ... Pitch specific energy calculating means 25 ... Pitch specific energy integrating means 26 ... Code conversion means 30 ... Code display means 50 ... Phoneme editing processing means 60 ... Voice output means 70 ... Printing means

Claims (16)

Syllable waveform acquisition means for acquiring one syllable uttered as a syllable waveform signal of M samples at a predetermined sampling frequency;
Syllable waveform expanding means for expanding the syllable waveform signal by Q (Q is an integer of 2 or more) on the time axis while maintaining the sampling frequency, and converting the syllable waveform signal into an expanded waveform signal of J × Q samples;
Code code group creating means for performing a predetermined frequency analysis on the enlarged waveform signal and creating a code code group represented by a plurality of code codes including data of sound generation start time, sound generation end time, pitch, and intensity; ,
The pitch of all the code codes constituting the code code group is corrected to Q times, the sounding start time and sounding end time of all the code codes are corrected to 1 / Q times, and the corrected code code group is constituted. A syllable code creating means for creating a syllable code to be played;
For each code code constituting the syllable code, a sound for calculating an energy sum value that is a sum of energy values given by the product of the time difference between the sound generation start time and the sound generation end time and the strength of the code code for each pitch High energy calculation means,
An energy integration unit for each pitch that calculates an integrated energy value by multiplying all energy sum values for each pitch between a plurality of syllable codes,
A predetermined number of higher pitches having a high integrated energy value are extracted, and parameters of a predetermined strength, a predetermined sounding start time, and a predetermined sounding end time are added to the code code corresponding to each extracted pitch. Code conversion means for setting and converting to a phoneme code composed of a plurality of code codes;
A phoneme code conversion apparatus comprising: In claim 1,
The pitch-based energy integration means corresponds to a plurality of consonant syllables such as “k”, “sa”, “ta”, “na”, “ha”, and “ma” corresponding to the Japanese kana character “a”. Multiply the corresponding energy sum value between syllable codes by pitch and calculate the integrated energy value,
The phoneme code conversion device, wherein the code conversion means converts to a phoneme code of a common vowel phoneme such as “A”. In claim 1,
The energy integration means according to pitches are “K”, “K”, “K”, “K” corresponding to five vowel syllables composed of Japanese kana characters “A”, “I”, “U”, “E”, “O”. Multiply the corresponding energy sum values by syllable codes between syllable codes corresponding to multiple consonant syllables such as
The phoneme code conversion device, wherein the code conversion means converts to a phoneme code of a common consonant phoneme such as “K”. In claim 3,
The energy integration means according to pitches are “K”, “K”, “K”, “K” corresponding to five vowel syllables composed of Japanese kana characters “A”, “I”, “U”, “E”, “O”. When multiplying the corresponding energy sum values by pitches between syllable codes corresponding to multiple consonant syllables such as
In advance, phoneme codes corresponding to five vowel syllables consisting of “a”, “i”, “c”, “e”, and “o” are determined, and
The energy sum value of each consonant syllable is reduced by multiplying the energy sum value corresponding to all the pitches constituting the determined phoneme code of each corresponding vowel by a predetermined real value less than 1. A phoneme code conversion apparatus characterized by being configured as described above.   A phoneme code representing a phoneme constituting each syllable of Japanese Kana characters is composed of a plurality of code codes having pitches equal to or less than a predetermined type, and having a uniform sound intensity and sound length, Each phoneme code is recorded in association with phoneme code identification information for identifying each phoneme code, and a conversion table showing the correspondence between the phoneme code identification information and the syllable identification information for specifying the syllable is recorded. Phoneme code database. Corresponding to the phonemes that make up each syllable of Japanese Kana characters, phoneme codes that consist of multiple code codes that have pitches equal to or less than a predetermined type and that have a uniform sound intensity and length. A phoneme code database recorded in association with phoneme code identification information for identifying a phoneme code;
Conversion to phoneme code identification information based on the syllable identification information described in the given synthesis instruction data, extraction of the corresponding phoneme code from the phoneme code database, and start and end of pronunciation according to the syllable identification information The time to specify the end of the pronunciation of the phoneme corresponding to the vowel phoneme, and the time obtained by adding a predetermined silent period to the time to specify the end of the pronunciation of the phoneme corresponding to the vowel phoneme Phoneme editing processing means for generating data;
A speech synthesizer characterized by comprising: In claim 6,
A speech synthesizer further comprising speech output means for outputting the synthesized speech data generated by the phoneme editing processing means as speech. In claim 6,
A speech synthesizer, further comprising: a printing unit that converts the synthesized speech data generated by the phoneme editing processing unit into a musical score and prints it. In any one of Claims 6-8,
The phoneme editing processing means, when the syllable identification information described in the synthesis instruction data is a vowel syllable and a long sound of Japanese Kana characters, the phoneme editing processing means to increase the pronunciation time of the entire syllable by a predetermined value A featured voice synthesizer. In any one of Claims 6-8,
When the syllable identification information described in the synthesis instruction data is a Japanese kana character prompting sound, the phoneme editing processing means is the same as the first phoneme corresponding to the syllable arranged immediately after the prompting sound. A speech synthesizer characterized in that a phoneme is placed immediately before the first phoneme, and a time for specifying the start of pronunciation of each phoneme and a time for specifying the end of pronunciation are set. In any one of Claims 6-8,
The phoneme editing processing means, when the syllable identification information described in the synthesis instruction data is a Japanese kana character “ya” “yu” “yo”, the syllable immediately before the stuttering, Only the first phoneme is set, and immediately after the first phoneme, two phonemes corresponding to one of the syllables “ya”, “yu” and “yo” corresponding to the stuttering are arranged, A speech synthesizer characterized in that a time for specifying the start of pronunciation and a time for specifying the end of pronunciation are set. In any one of Claims 6-11,
The phoneme editing processing means extracts a phoneme code corresponding to the phoneme code identification information converted from the syllable identification information from the phoneme code database, and sets a time for specifying the start and end of pronunciation according to the syllable identification information The speech synthesizer is characterized in that the silent section is multiplied by a set time expansion / contraction rate, and a predetermined modification is applied to the time for specifying the start and end of the pronunciation. . In any one of Claims 6-12,
The phoneme editing processing means extracts a phoneme code corresponding to the phoneme code identification information converted from the syllable identification information from the phoneme code database, and sets a time for specifying the start and end of pronunciation according to the syllable identification information When the phoneme code recorded in the phoneme code database is a vowel on the basis of the set pitch offset parameter, the pitch for each code code constituting the phoneme code is set. A speech synthesizer characterized in that an offset parameter is added, and a predetermined modification is applied to the pitches of code codes corresponding to all vowel phonemes constituting the synthesized speech data. In claim 13,
The synthesis instruction data defines the pitch offset parameter together with the syllable identification information for each syllable, and the phoneme editing processing means corresponds to the phoneme code identification information converted from the given syllable identification information. When a code is extracted from the phoneme code database and the time to specify the start and end of pronunciation is set according to the syllable information, the phoneme code database is stored based on the pitch offset parameter defined for each syllable. When the recorded phoneme code is a vowel, the pitch offset parameter is added to the pitch of each code code constituting the phoneme code, and all vowel phonemes constituting the synthesized speech data are supported. A speech synthesizer characterized in that the pitch of the code code to be modified is modified.   The program for functioning a computer as a phoneme code conversion apparatus in any one of Claims 1-4.   The program for functioning a computer as a speech synthesizer in any one of Claims 6-14.

Images