TW201120871A - Singing voice synthesis method - Google Patents

Abstract

The invention discloses a singing voice synthesis method for synthesizing a synthesized voice of singing voice from a plurality of voice samples. The singing voice synthesis method includes the following steps: retrieving a melody from a music file; according to the melody, retrieving the voice samples from a voice sample database; according to the melody, determining a pitch curve relative to the melody; according to the pitch curve and harmonic parameters of the voice samples, calculating harmonic parameters of the synthesized voice of singing voice; and according to the harmonic parameters of the synthesized voice of singing voice, calculating the synthesized voice of singing voice.

Description

201120871 VI. Description of the Invention: [Technical Field to Which the Invention Is Ascribed] The present invention relates to a method of synthesizing a singing voice. [Prior Art] With the rapid development of computer hardware technology, the relatively capable processing can be more and more complicated. Nowadays, speech synthesis has been widely used in various places, such as station name broadcast system on the bus, telephone voice system or It is on the cash machine. However, the current common speech systems are mostly used to simulate speech, and the synthesis of speech samples is mostly a simple combination, so the synthesized speech sounds like a robot. Although in recent years, the problem of discontinuity of adjacent speech has been smoothed, so that the synthesized speech sounds smoother, but for the singing voice with pitch change, the aforementioned discontinuous problem is more obvious.

The aforementioned smoothing process is not enough to simulate the synthetic bamboo sound of approximate singing voice. Especially when the length of a speech to be synthesized is different from the length of the speech sample, the speech sample is simply elongated or shortened. It sounds more unnatural. SUMMARY OF THE INVENTION The present invention provides a recording-sounding synthesis method for synthesizing a plurality of speech samples into a vocal synthesis sound.糸201120871 The vocal synthesis method comprises the following steps: (a) extracting a score from a slang file; (b) extracting the plurality of voice samples from a phonetic sample database according to the score; (C) according to The score of the song determines a pitch curve related to the score; (d) calculating a harmonic parameter relating to the synthesized sound of the vocal sound according to the pitch curve and harmonic parameters of the plurality of voice samples; and (e) The vocal synthesis sound is calculated based on the harmonic parameters of the vocal synthesis sound.

Step (b) is to add a plurality of lyrics to the score, and then based on the plurality of lyrics, the plurality of voice samples corresponding to the plural words are taken from the voice sample library. Step (4) further includes simplifying the pitch curve according to the curve change of the S-curve curve, so that the adjusted & a question curve can be more in line with the actual singer, and the pitch adjustment of the pitch is generated according to the pitch change of the score. . Ye Weizhong's steps (the first part of the sound part of the surface sound sample is performed by Fu Li: 'Get the voice sample line to adjust the plural number of the 9th record «The sound of the touch of the sound. Record, turn county singing voice The sound two::::= this: adjust the frequency domain wave length of the speech sample. The length of the voice sample is generally the same, because the tone of the frequency domain is the tone of the sound sample. The synthesized sound can still retain the voice sample 201120871, wherein step (e) is to transmit the harmonic parameter of the vocal synthesis sound, and copy the corresponding silent part of the voice sample; and then, the corresponding voiced part of the voice sample is trained. Singing the initial phase of the new parameter of the synthesized sound, and then 'according to the corresponding Wei parameter of the vocal synthesized sound of the voiced part of the voice sample, calculating the singer corresponding to the #音 sample by phase superposition The time-domain amplitude of the synthesized sound; finally, according to the corresponding volume, (4) time-domain listening, so that the vocal synthesis sound can be sung. Therefore, the singer synthesis method of the present invention has the color of the original voice sample. and The pitch curve can be adjusted according to the change of the pitch curve, and the synthesized vocal synthesis sound is obviously improved in sharpness, self-finance and fluency compared with the prior art. The advantages and spirits of the present invention can be The invention will be further described with reference to the detailed description of the invention and the accompanying drawings. FIG. 1 is a flow chart showing a method for synthesizing singing voice according to an embodiment of the present invention. According to the specific embodiment, the vocal combination method comprises two parts: - the synthesis of the human voice; the other is the synthesis of the music sound. After the two are made, the mixing program is performed to make the WAV. Slot (or other music format) for subsequent playback. As shown in Figure 1, the part of the vocal synthesis is first taken from a song score 201120871 - the score is as shown in step S1G2... Documented as the digital interface of the instrument (Musical Instrument Digital

Inteffaee ' MIDI) (4) 'Therefore, the songs that are out of this situation do not contain songs (4)', but the county of the invention is limited to this. In order to make the follow-up work more convenient, the songs taken out can be expressed in tabular form. Do not. In order to facilitate the subsequent voice sample retrieval, you can first fill in the table and fill in the songs as shown in the table and flash,; of course, if the song files already contain lyrics, then the scores of the songs can contain the required lyrics, and No need to fill in separately. Lyrics volume 1.1 Table 1:: Part of the score table, the number of tones is flashing 1. Start time 4.70588 5.294115 Yinnan sound length 2 0.490196 2 0.490196 ------- It is added that the pitch in the score is • Positive frequency The value can be calculated according to the twelve-average law shown in the following equation: its true F~Fc4*2 (pitchyi2 rate value ^ is 1^=(four) the actual frequency value U central C frequency value. ', ', .Hz, pitc1^ From the scores of the songs (the sounds that are sold in the form _m) determine the pitch curve of the score, such as the step according to the pitch, the length of the pitch (that is, the duration of the pitch of 201120871) The resulting pitch curve can be used to adjust the pitch curve of the Wei Shuo based on the fact that the singer changes the pitch of the actual singing voice due to the pitch change (ie, the change in pitch scale). Usually, the current tone and current At this time, the pitch curve of the beginning of the current sound will have a rise and fall, or a fall and then rise. Therefore, the present invention proposes an adjustment rule as described later.

In the sound South curve, if the pitch value of the if-mark is not equal to the next pitch value, the south value of the target is adjusted according to the change f of the lower pitch value relative to the pitch value of the target. In this case, it can be further divided into three adjustment rules according to the value of the pitch = value before the value of the pitch value. If the value of the previous pitch value is lower relative to the pitch value of the target, then the first adjustment rule is adjusted according to the following equation: (not a p) call) (1_+[1+纪; where OSxSl , y is the adjustment multiple, pseudo is the current pitch value, and pi+1 is the next pitch value. If the previous pitch value is the same as the target pitch value, then the second is as follows Adjustment rules to adjust: y swear (") i ten L5*pt

J If the previous pitch value is higher relative to the pitch value of the target, adjust it according to the third adjustment rule as shown in the following equation: 201120871 y 1 A sin(5;sc) 10 * e -2.5πχ

(lx)+ l + &i__(jc) V 1-5* AJ In addition, in the pitch curve, if the value of the target pitch is equal to the value of the next tone, it is adjusted by a predetermined multiple value. The pitch value of the target. Similarly, in this case, three adjustments can be made according to the previous pitch value of the target pitch value. If the previous pitch value is lower than the value, then according to the first adjustment rule of the following handle

Sin(5^x:) 10 * e -2.5 ία If the preceding-sound value is the same as the second pitch, the second adjustment rule is adjusted: '^

1+cos(25c) · 200 ' If the previous pitch value is higher than the pitch value of the target, the third adjustment rule is shown: yin(5^c) 10 _ ^ λ-2.5λγ ^ Q 〇 The above adjustment is based on the mutual sound of adjacent pitches s line 'only pass ^ in a certain tone and below - the time difference between the sounds = the next degree 201120871 degree 'this one A sound will not be affected by its next sound. Therefore, before the above rules are applied, it is necessary to judge whether the pitch value of the target and the next south value are different, and the corresponding time difference is greater than a predetermined length of time. This predetermined length of time can be set to 0.9 seconds, but the present invention is not limited thereto. :3⁄4 The result of the judgment is yes, then the next pitch value is regarded as the same pitch value as the target, and the pitch curve is adjusted; that is, it is adjusted according to the latter three rules. _ After the above adjustment of the pitch curve is completed, the following step S106 may be followed or parallel. As shown in step S106, the plurality of speech samples are retrieved from a speech sample database based on the plurality of lyrics entered in the score, and spectral analysis is performed to obtain harmonic parameters for each of the speech samples. Based on the characteristics of the human voice, a sound can observe the silent part and the sound part in the waveform of the time domain. The silent part is a non-periodic waveform, and it cannot be analyzed for harmonic parameters. &#Full-leaf conversion of the voiced part (10) coffee • TranSf〇rm). Since the voice sample is recorded by human voice, it also contains a silent part and a voiced part. The voiced part will perform Fourier transform to obtain the impurity parameters of the = tone sample; the silk phase directly (four) to the vocal synthesis to the Fourier transform of the speech sample, first the sound part of a speech sample according to its time domain waveform For the same length of the sound box, each low-frame package points 256 sample points, the voice sampling frequency is 22G5GHZ, so the time and county of each sound box is about 12 mail. _ sound box (512 sample points: 201120871 multiplied by a Hamming Window), and then do a fast Fourier transform, after conversion to get multiple sets of spectral parameters, each set of wave parameters contain amplitude (Amplitude), Frequency and phase, in which the amplitude, frequency and phase of the fundamental frequency can be obtained by interpolation using Cubic Hermite Spline, and based on the recording of the voice sample by the recorder, try to be the same. Pitch, length and volume are recorded, so when reverting back to 3 舁 fundamental frequency, 'the search range can be set between 2〇〇~4〇〇Hz, and the larger amplitude is the initial value. The fundamental frequency related parameters After the determination, according to the multiple relationship, the other harmonic search initial value 'still recursively counted with three Hermitian splines' to obtain its spectral parameters. Obtain the harmonic parameters of the speech sample and Based on the pitch curve generated by the score, the harmonic parameters of the synthesized sound of the singing voice can be calculated according to the spectral parameters and the pitch curve, as shown in step sl8. Due to the length of each voice sample Roughly the same, but according to The length of the required speech samples corresponding to the pitch curve is not the same, so the length of the speech sample needs to be adjusted. Generally speaking, the voiced part of the speech sample conforms to the ADSR model (as shown in Figure 2), which includes The (Attaek, A) section, the Decay (D) zone is delayed (Sustain 'S) section and the end (Reiease, r) section; while the Tian people stretch the sound, mainly the extended delay zone The length of time of the segment. Therefore, if the length of time of the speech sample is different from the length of time of the pitch curve corresponding to the speech sample, the length of the 201120871 delay segment of the voiced portion of the speech sample is adjusted. If the adjustment range is too large, the rising section, the falling section and the ending section are also required to be shortened according to the ratio of the length of the synthesized sound to the voice sample. To facilitate the above processing, the voice sample can be processed for the voice sample. The length of time for each segment is expressed in tabular form, as shown in Table 2. The number of sample points is the unit of time length, and the length of each voice sample is defined. When the same, the r segment (end segment) can be calculated from the total length minus the silent portion, the AD segment (up, down, and down segments) and the s segment (delay segment), so it is not listed separately; However, the present invention is not limited thereto. Table 2: Partial speech sample table Voice sample number Silent portion AD segment s segment c018 1543 1419 10272 c019 1482 1389 13431 But what is needed is that the speech sample is expressed in its frequency domain wave. • Envelope, so when adjusting the speech samples, the adjusted wave shape should be kept in the large lake, even if the frequency spectrum of the speech sample corresponds to the frequency-domain collinearity of the speech sample. The same is true so that the adjusted synthesized tone still retains the tone of the speech sample. In order to be able to accurately control the spectral wave complexes, the Cubic Hermite Spline used in the previous spectral analysis should be used. The main purpose of the Cubic Hermite Spline is to calculate the amplitude of the harmonic frequency of the synthesized sound. The frequency points at the left and right ends of the interval in which the harmonic frequency of the synthesized sound is located are found to obtain the amplitude value of the frequency amplitude value in the interval, that is, the amplitude of the synthesized harmonic frequency. 201120871 In addition, if there is a transition sound in the score, that is, a speech sample corresponds to several pitch values, the corresponding speech sample needs to be segmented, that is, if the pitch curve corresponding to the speech sample is included When a plurality of pitch values are present, the first pitch value of the plurality of pitch values corresponds to a rising segment, a falling segment, and a partial delay segment of the voiced portion of the voice sample, and the last tone The high value corresponds to the end segment of the voiced portion of the speech sample and a portion of the delay segment. In the case of a turning tone (ie, there are two tones corresponding to the same lyrics), the first pitch value (four) should raise the segment, the falling segment, and the partial delay segment, the second tone. The high value is the delay section and the end section of the corresponding part. In the case of two transitions (ie, there are three pitches to the lyrics), the first pitch value corresponds to the rising segment, the falling segment, and the partial delay segment, the second tone. The high value is that the portion should be delayed by the segment' and the third pitch value is the delay segment and the end segment of the corresponding portion. In the case of more transitions, the analogy is not repeated here. In addition, regarding the simulation of the vibrato effect, based on the actual singing, the vibrato is often used in the case where the same vocal is longer, so a predetermined length of time can be preset as a need to add a vibrato effect. Judging the basis. In the S-curve of the sound, the length of the corresponding-sound value is greater than the predetermined time, and the 贞仏-string function is in the harmonic parameter of the vocal synthesis sound that listens to the pitch value. This predetermined length of time can be set to leap seconds, but the present invention is not limited thereto. 13 201120871 After step S108, the vocal synthesized sound is calculated according to the spectral wave parameter of the singer synthesized sound as shown in step sn 〇. Since the silent portion of the speech sample is not subjected to Fourier transform, when the vocal synth sound is generated, the unvoiced portion is added by copying. Then 'determining the initial phase of the harmonic parameter of the vocal part of the corresponding voiced part of the voice sample; and then, according to the test parameter of the vocal synthesized sound of the corresponding voice part of the voice sample, ❿ The 7-wave equation and the phase superposition are calculated, and the mid-field value of each ship's current point is calculated and added to find the time-domain amplitude of the vocal synthesized sound corresponding to the speech sample. Finally, adjust the time domain vibration t according to the volume recorded in the score. Therefore, the singer's vocal synthesis sound is silk, and then the other sounds, that is, the vocal synthesis sound, can be combined with the complete vocal synthesis sound. Since the actual singer sings when singing, it is relatively vocal for the lower pitch lyrics, which means that the volume will be lower, so in the aforementioned adjustment sound and volume (ie, adjust the time domain amplitude) When the pitch value corresponding to the time domain amplitude is less than a predetermined pitch value, the time domain is adjusted by a predetermined multiple value, for example, the predetermined pitch value is the bass C, and the predetermined multiple value is 〇.6; However, the invention is not limited thereto. And if the pitch value corresponding to the time domain amplitude is equal to or greater than the predetermined pitch value, then the time domain amplitude is linearly adjusted according to the pitch value. For example, the following formula is used to adjust: νηΊ _ 0.25 12~*Wte + 0,85 ; 14 201120871 where vol is the multiple of the time domain amplitude value to be adjusted, and _ is the pitch value read from the score ( Expressed as an integer value, for example, the central c is 0). * According to the above description, the previously calculated time domain amplitude is adjusted, and the resulting vocal synthesis sound will be more closely related to the actual singer's volume change during actual singing. At this point, the complete vocal synthesis sound has been completed, but the background music needs to be synthesized separately (as shown in steps sm, S114 and S116), and then mixed with the vocal synthesis sound (as shown in step S118). After mixing, it can be stored directly in medical format (but the invention is not limited to this), and then played by other players or playback software (as shown in step sm). 1 In the instrument sound synthesis and mixing, the playback program is a habit. Know, no longer repeat them. In addition, many of the above formulas are examples, and they can be adjusted according to the actual measurement situation. In summary, the 'singing sound synthesis method of the present invention is a f-domain wave of Lili 2, so that after adjusting the pitch, the 曰2 曰 color can still be maintained ′ and the pitch is adjusted according to the change of the pitch curve itself. Qu 2: Hey! It is the same change that the real person made when he sang because of the difference in the tune. In addition, the singer-sounding, vibrato, and the like, the method for synthesizing the song of the present invention has a method of making the resolution, self-bribery, and smoothness of the sound port 9 into the first sound port [if more is established on the voice sample ^ The voices of different timbres (adults, children, boys, girls, etc.) can be more flexible in the selection of voice samples, and can also be synthesized to better match the vocal synthesis of the 201115871 singing. The features and spirit of the present invention are more clearly described by the above-described embodiments of the present invention, and the scope of the present invention is limited by the preferred embodiments disclosed above. On the contrary, the intention is to cover various modifications and equivalents within the scope of the patent scope of the invention. BRIEF DESCRIPTION OF THE DRAWINGS Fig. 1 is a flow chart showing a method of synthesizing a singing voice according to an embodiment of the present invention. Figure 2 is a schematic diagram showing each section of the ADSR model. [Main component symbol description] S102~S120: Step 16

Claims (1)

201120871 Seven patent application scope: 1. The second synthesis method is used to synthesize a plurality of speech samples into a vocal synthesis sound. The vocal composition comprises the following steps: (a) extracting a song from a slang file; (9) The line is based on the pure spectrum 'Decision to the money song spectrum (four) - pitch height H according to ttK 'self-speech sample (four) library spine the plural number of voice samples; (dl roots the ί high series and the complex number (four) tone sample harmonic The wave parameter m 4 calculates the test parameter related to the new vocal synthesis sound; and (^ according to the vocal synthesis sound _ the vocal synthesis sound. 2, 3, such as ^ turn _ 丨 丨 rhyme sing The compositing method, the step (C) is carried out by the following steps: ', adding a plurality of lyrics to the lyrics; and according to the plural lyrics, from a madness to a plural number of _ complex two na a synthetic branch, wherein the step kick (bli adjusts the pitch chorus synthesis method according to the curve of the pitch curve, wherein the step kicks in the pitch curve, if a target pitch value is not equal to the next tone High 17 4, 201120871 The sound of the subject - if the The pitch value is equal to; the biliary, and the multiplier value adjusts the value of the target sound/sound value, and then a predetermined: 3⁄4% vocal synthesis method, wherein: step 6, the phase difference is greater than a predetermined;: the next- When the pitch value corresponds to the mark 2 pitch value, when the length of the sub 1 is used, the singer synthesis method according to the predetermined multiple value is adjusted, and the predetermined time length is 2m, 5 items, which makes the pre-step Implementation ···, knife and voiced part, step (d) coefficient sound: carry out Fu Zeng, 峨 魏 储 储 、 、 、 、 、 、 、 、 、 、 、 、 、 、 、 、 、 、 、 、 、 、 、 、 、 、 、 、 、 、 、 、 、 、 、 、 、 、 、 、 、 、 The number of steps. - The human * spline 'calculates the spectral wave of the speech sample 201120871 When the value is high, then the m-a sound sample of the plurality of pitch values is raised. The day-high value is the corresponding segment and the last-pitch value is delayed. The ending section and the partial delay section.有The sound part of the sample is as in the song (4) mentioned in the seventh paragraph of the patent application, the frequency of the voice sample is 方法^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Patent scope 7 (referred to as the use of three Hermite splines, to ^, where the step spectral parameters. 'Issue of the vocal synthesis of the sound 13, as described in the scope of claim 7 (d2) ) further includes: 曰 If the curve corresponds to a pitch value, the length of time is large, = 疋 time length, 'the read wave parameter of the ride on the _ pitch value is added to the person's record — (4) ^ ^ W Breaking out the voice as described in item 7 of the patent scope = including the judgment: 曰俅 心 ..., 邛 ,, and step (e) by the following steps 11, 12, the synthesis method, wherein the step root f sings a synthesizing sound slave parameter for copying a corresponding silent portion of the voice sample; determining an initial phase of the wave parameter of the vocal synthesized sound corresponding to the voiced portion of the voice sample; the corresponding voice sample of the board data The vocal part of the vocal synthesis sound wave parameter' Calculating the time domain amplitude of the vocal synthesized sound corresponding to the speech sample in a phase superposition manner; and f accommodating the time domain amplitude according to the corresponding volume. The singing voice described in claim 14 The synthesis method, wherein step 201120871 (:) step-by-step comprises the following steps: When the amplitude value of the amplitude is less than 1 pitch value, the value is multiplied to adjust the time domain amplitude, and the right corresponds to the time domain amplitude The pitch value is equal to or greater than the predetermined pitch π μΐί', and the root lion pitch value is used to linearize the time domain amplitude. 16. The vocal synthesis method according to claim 15 of the whistle patent scope has a predetermined pitch The value is the bass C ' and the predetermined multiple value is 〇.6. 17. The vocal synthesis method according to claim 1, wherein the spectral wave parameter of the speech sample or the harmonic parameter of the vocal synthesized sound , contains the frequency, amplitude, and phase of the complex array.