JP6442982B2 - Basic frequency adjusting device, method and program, and speech synthesizer, method and program - Google Patents

Description

  The disclosed technology relates to a fundamental frequency adjustment device, method, and program, and speech synthesis device, method, and program.

  In order to output the voice synthesized based on the text as the voice expected by the user, there is a technique for adjusting the accent intensity of the voice based on the designation by the user. Since the accent is defined by the pitch of the voice, and the pitch of the voice is determined by the fundamental frequency (F0), the accent strength is adjusted by adjusting the value of the fundamental frequency.

  In the related technology for adjusting the fundamental frequency, the fundamental frequency at the center of each vowel is estimated based on the linguistic information of the sentence using a statistical method such as quantification class I. The basic frequency is adjusted by acquiring the slope line of the fundamental frequency from the first vowel of the word to the last vowel of the word, and multiplying the fundamental frequency component exceeding the slope line by a value corresponding to the accent intensity for each vowel. Linearly interpolate fundamental frequencies between fundamental frequencies.

JP 2001-249677 A

Keiichi Tokuda, “Basics of Speech Synthesis by HMM”, IEICE Technical Report, IEICE, 19 October 2000, pp. 43-50 Takao Kobayashi et al., “Trends in Corpus-Based Speech Synthesis Technology [IV] -HMM Speech Synthesis Method”, Journal of the Institute of Electronics, Information and Communication Engineers, 2004, Vol. 87, no. 4, pages 322-327

  In the related art, the fundamental frequency of the tuned speech is a collection of simple line segments, not a complex fundamental frequency that includes micro-prosody, which is a small variation unique to the human voice, which impairs the naturalness of the speech. ing. By using the Hidden Markov Model (HMM) data for emphasis, it is possible to adjust the fundamental frequency of speech while maintaining the micro-process. However, it is difficult to prepare emphasis HMM data.

  In one aspect, the disclosed technology aims to adjust the fundamental frequency of speech while maintaining microprocedures without using emphasis HMM data.

  In the disclosed technique, the fundamental frequency pattern estimation unit estimates the fundamental frequency pattern of speech corresponding to the text using information of the hidden Markov model corresponding to the text. The fundamental frequency changing unit changes the value of the fundamental frequency of the designated portion in the estimated fundamental frequency pattern to a value corresponding to the designated accent intensity. In addition, the re-estimation unit re-estimates the fundamental frequency pattern corresponding to the text and the value of the fundamental frequency of the designated portion changed to the value using the information of the hidden Markov model.

  As one aspect, the disclosed technology has the effect of adjusting the fundamental frequency of speech while maintaining the microprocedure without using the emphasis HMM data.

It is a block diagram which shows an example of the principal part function of the computer which concerns on embodiment. It is a conceptual diagram which shows an example of the hidden Markov model database (HMM DB) which concerns on embodiment. It is a block diagram which shows an example of a structure of the electric system of the computer which concerns on embodiment. It is a flowchart which shows an example of the flow of the fundamental frequency adjustment process which concerns on embodiment. It is a flowchart which shows an example of the flow of the fundamental frequency (F0) pattern estimation process which concerns on embodiment. It is a conceptual diagram which shows an example of the user interface which concerns on embodiment. It is a conceptual diagram which shows an example of the user interface which concerns on embodiment. It is a conceptual diagram which shows an example of the part of the hidden Markov model (HMM) corresponding to the sentence which concerns on embodiment. It is a conceptual diagram which shows an example of the F0 pattern estimated using HMM. It is a conceptual diagram which shows an example of F0 pattern in which F0 was changed partially. It is a conceptual diagram which shows an example of the F0 pattern re-estimated using HMM. It is a flowchart which shows an example of the flow of the F0 pattern partial change process which concerns on embodiment. It is a flowchart which shows an example of the flow of the F0 pattern re-estimation process which concerns on embodiment.

  Hereinafter, an example of an embodiment of the disclosed technology will be described in detail with reference to the drawings. In the following description, a case where a computer that is a general-purpose device is used as an example of the fundamental frequency adjustment device according to the disclosed technology will be described as an example. However, the disclosed technology is not limited thereto. The disclosed technology can be applied to, for example, a dedicated device for fundamental frequency adjustment or a substrate on which a device for fundamental frequency adjustment is mounted.

  As an example, the computer 10 shown in FIG. 1 includes a detection unit 12, a language processing unit 14, a parameter estimation unit 16, an accent intensity-fundamental frequency (F0) conversion unit 18, an F0 designation unit 20, an F0 re-estimation unit 22, and analysis and synthesis. Part 24. As an example, the computer 10 shown in FIG. 1 has a hidden Markov model database (HMM DB) 30.

  The detection unit 12 detects the Japanese notation and the designation of the part to change the accent input by the user to the user interface and the designation of the accent strength of the part to change the accent. The language processing unit 14 processes the detected Japanese notation to acquire language information. The parameter estimation unit 16 includes an F0 pattern estimation unit and a mel cepstrum pattern estimation unit. The parameter estimation unit 16 generates an HMM corresponding to a sentence expressed in Japanese using a hidden Markov model (HMM), which is a speech synthesis processing unit, and uses the HMM corresponding to the sentence to generate an F0 pattern and The mel cepstrum pattern is estimated as an output sequence. The accent strength-F0 conversion unit 18 converts the specified accent strength into the height of F0. The F0 designation unit 20 changes the designated part of the estimated F0 pattern to the converted F0. The accent strength-F0 conversion unit 18 and the F0 designation unit 20 are examples of the fundamental frequency changing unit of the disclosed technology. The F0 re-estimator 22 re-estimates the portion of the F0 pattern that has not been changed, using the HMM used for estimating the F0 pattern. The analysis / synthesis unit 24 synthesizes a speech signal using the re-estimated F0 pattern and the estimated mel cepstrum pattern.

  FIG. 2 shows a conceptual diagram of the HMM DB 30. In the HMM DB 30, a learned context-dependent HMM 32 is stored in advance as a processing unit HMM. The context-dependent HMM 32 is a model that takes into account the phoneme context. The acoustic characteristics of phonemes vary greatly due to the influence of context. In order to cope with such a problem, a context-dependent HMM is used as a speech synthesis processing unit. A phoneme has a plurality of context-dependent HMMs 32 according to the context of the phoneme. The context includes, for example, a preceding phoneme, the phoneme, a subsequent phoneme, a mora position in the accent phrase of the phoneme, a preceding part of speech, the relevant part of speech, and a subsequent part of speech.

  As an example, the computer 10 includes a CPU (Central Processing Unit) 60, a primary storage unit 62, a secondary storage unit 64, an external interface 70, a keyboard 72, a mouse 74, a display 76, and a speaker 78, as shown in FIG. I have. The CPU 60, primary storage unit 62, secondary storage unit 64, external interface 70, keyboard 72, mouse 74, display 76, and speaker 78 are connected to each other via a bus 80.

  The keyboard 72 and the mouse 74 receive user operations and input information to the computer 10. The display 76 and the speaker 78 present information to the user. An external device is connected to the external interface 70 and controls transmission / reception of various information between the external device and the CPU 60.

  The primary storage unit 62 is a volatile memory such as a RAM (Random Access Memory), for example. The secondary storage unit 64 is a non-volatile memory such as an HDD (Hard Disk Drive) or an SSD (Solid State Drive).

  As an example, the secondary storage unit 64 stores a detection subprogram 66A, a language processing subprogram 66B, a parameter estimation subprogram 66C, an accent strength-F0 conversion subprogram 66D, and an F0 designation subprogram 66E. In addition, the secondary storage unit 64 stores, as an example, a fundamental frequency adjustment program 66 including an F0 re-estimation subprogram 66F and an analysis / synthesis subprogram 66G. Further, the secondary storage unit 64 has an HMM DB storage area 68 in which information constituting the HMM DB 30 is stored.

  The CPU 60 reads the detection subprogram 66A, the language processing subprogram 66B, the parameter estimation subprogram 66C, and the accent strength-F0 conversion subprogram 66D from the secondary storage unit 64, and expands them in the primary storage unit 62. Further, the CPU 60 reads the F0 designation subprogram 66E, the F0 re-estimation subprogram 66F, and the analysis / synthesis subprogram 66G from the secondary storage unit 64 and expands them in the primary storage unit 62. The CPU 60 operates as the detection unit 12 illustrated in FIG. 1 by executing the detection subprogram 66A. The CPU 60 operates as the language processing unit 14 illustrated in FIG. 1 by executing the language processing subprogram 66B. The CPU 60 operates as the parameter estimation unit 16 shown in FIG. 1 by executing the parameter estimation subprogram 66C. The CPU 60 operates as the accent strength-F0 conversion unit 18 shown in FIG. 1 by executing the accent strength-F0 conversion subprogram 66D. The CPU 60 operates as the F0 designation unit 20 shown in FIG. 1 by executing the F0 designation subprogram 66E. The CPU 60 operates as the F0 re-estimation unit 22 illustrated in FIG. 1 by executing the F0 re-estimation subprogram 66F. The CPU 60 operates as the analysis / synthesis unit 24 shown in FIG. 1 by executing the analysis / synthesis subprogram 66G. Further, the CPU 60 reads out information stored in the HMM DB storage area 68 and develops it in the primary storage unit 62 as an HMM DB 30 as shown in FIG. As a result, the computer 10 that has executed the fundamental frequency adjustment program 66 functions as a fundamental frequency adjustment device.

  Next, the operation of the computer 10 that is a fundamental frequency adjusting device will be described. When the fundamental frequency adjustment program 66 is started, for example, according to a user instruction, the fundamental frequency adjustment process shown in FIG. 4 is started. In the fundamental frequency adjustment process illustrated in FIG. 4, first, F0 pattern estimation is performed in step 100. FIG. 5 illustrates details of F0 pattern estimation in step 100.

  In step 102, the CPU 60 displays the user interface 91 illustrated in FIG. 6A on the display 76. The user interface 91 includes a text box 92 in which Japanese notation and accent strength designation are input, and a “speech synthesis” button 94 selected when a voice synthesis process is instructed. In addition, the user interface 91 has a “re-input” button 95 selected when re-inputting Japanese notation when re-executing speech synthesis, and “end” selected when ending the fundamental frequency adjustment process. Button 96 is included. The CPU 60 detects Japanese notation and accent strength designation input by the user into the text box 92 of the user interface 91 using the keyboard 72. In the example of FIG. 6A, the Japanese notation is “Today is good weather”, and the accent strength designation of the Japanese notation “Today is” is “strong”. That is, the CPU 60 detects the speech synthesis target sentence, the part that changes the accent in the sentence, and the accent intensity of the part that changes the accent, and the text box 92 displays the detected sentence, the part that changes the accent, And the accent intensity are displayed.

  When the user clicks the “speech synthesis” button 94 with the mouse 74, in step 104, the CPU 60 analyzes the sentence represented in Japanese notation detected in step 102, reads the accent phrase position, and the accent phrase boundary. And language information of sentences such as parts of speech.

  Next, in step 106, the CPU 60 estimates the F0 pattern and the mel cepstrum pattern of the sentence expressed in Japanese using the HMM corresponding to the sentence expressed in Japanese. Specifically, the language information acquired in step 104 is used as a context to convert Japanese notation into a context-dependent label, and a context-dependent HMM 32 corresponding to each label is selected from the HMM DB 30. The selected context-dependent HMMs 32 are sequentially connected to generate an HMM corresponding to the sentence. FIG. 7 illustrates one frame of the HMM corresponding to the sentence. One frame corresponds to, for example, 8 ms of voice.

In FIG. 7, reference numeral 35 indicates an HMM corresponding to a sentence, reference numeral 36 indicates a state, reference numeral 37 indicates a state sequence, reference numeral 38 indicates an average vector, and reference numeral 39 indicates a covariance matrix. A state sequence of the HMM 35 corresponding to a sentence when the time t changes from _{1 to T} and the state transitions to q ₁ ,. Further, μ _q1, ..., μ _qT state _q 1, ..., a mean vector of an output probability distribution corresponding to each of the _{q T, U q1, ...,} U qT state _q 1, ..., corresponding to _{q T} This is a covariance matrix of output probability distribution. The average vector and the covariance matrix, which are information of the HMM 35 corresponding to the sentence, are given to the corresponding states. Further, M obtained by transposing each of the average vectors μ _q1 ,..., Μ _qT and further transposing M is represented by Expression (1), and U is _obtained by transposing a diagonal matrix having the covariance matrix U _{q1 ,.} was an expression (2), the state _q 1, ..., _{o 1} output vector corresponding to each of the _{q T,} ..., and o _T. Further, when each of the output vectors is transposed and the transposed output sequence O is expressed by Equation (3), the probability that the output sequence is O when the state sequence is Q and the parameter of the HMM 35 corresponding to the sentence is λ. O which maximizes Expression (4) shown is obtained. By obtaining O that maximizes Equation (4), it is possible to estimate the F0 pattern. λ is λ = (A, B, π), A is a state transition probability, B is an output probability distribution, and π is an initial state probability. If it is a general formula, (2π) ^3T is (2π) ^{(3T × number of dimensions of parameter)} , but here, since the number of dimensions of the fundamental frequency as a parameter is 1, it is set to 2π ^3T . The probability that the state Q transitions to q ₁ ,..., Q _T and the output vector O = {o ₁ ,..., O _T } is output is given by A × B. In the following equations, bold indicates a vector or matrix, and T at the upper right of the vector or matrix indicates transposition of the matrix.

(1)
(2)
(3)
(4)

However, if the F0 pattern is estimated by obtaining O that maximizes the expression (4), F0 causes a discontinuous change when the state transitions. In order not to cause discontinuous changes, the dynamic feature vectors Δc _t and Δ ² c _t of F0 are considered. When static characteristics of the F0 pattern which does not cause discontinuous change is to be represented by the formula (5), the dynamic feature vector .DELTA.c _t is expressed by Equation (6), the delta ² c _t (7) .
(5)
(6)
(7)

w _t ⁽¹⁾ (τ) and w _t ⁽²⁾ (τ) are weighting coefficients for calculating the dynamic feature amount. Moreover, L ⁽¹⁾ and L ⁽²⁾ is a representation of each in the calculation of .DELTA.c _t and delta ² c _t at time t, the time width to be considered before and after the time t the sampling time τ as a unit .

When the relationship between Expression (5) to Expression (7) is expressed in a matrix form, Expression (8) is obtained. C, which is a static feature quantity, is converted into an output vector O including a dynamic feature quantity using a transformation matrix W. Transformation matrix W is expressed by Equation _{(9), c t, Δc} t, weights _{w t} corresponding to each delta ² c _t is expressed by Equation (10).
(8)
(9)
(10)

By taking the logarithm, equation (4) is transformed into equation (11), and P (O | Q, λ) is maximized with respect to C for state series Q. That is, the optimum C is obtained by solving the equation (12) obtained by partial differentiation with C. Since the logarithmic function is a strict increase function, the value of C that maximizes P and the value of C that maximizes logP that takes the logarithm of P are the same value, and therefore, the logarithm of equation (4) is taken. As shown in Expression (8), Expression (12) becomes Expression (13) by replacing O with WC.
(11)
(12)
(13)

Expression (14) included in Expression (13) is modified as shown in Expression (15).
(14)

(15)

In general, since Expression (16) and Expression (17) hold, the first term of Expression (15) is transformed as shown in Expression (18) using Expression (16), and the first term of Expression (15) is obtained. The two terms are transformed using equation (17) as shown in equation (19). By these deformations, Expression (12) becomes Expression (20), and Expression (21) is established.
(16)
(17)
(18)
(19)
(20)
(21)

  By solving the equation (21), it is possible to obtain the static feature amount C of the F0 pattern. In order to solve Equation (21), for example, Cholesky decomposition or QR decomposition is used.

  For example, when Japanese notation “in Fujitsu” is input, an F0 pattern indicated by a curve illustrated by a thick line 41 in FIG. 8A is estimated. A portion surrounded by a circle 42 indicated by a broken line is a small variation of the F0 pattern called microprosody, which contributes to the naturalness of speech.

  The mel cepstrum pattern can be estimated in the same manner as the F0 pattern.

  Next, in step 200 of FIG. 4, F0 included in the F0 pattern estimated in step 106 is partially changed. FIG. 9 illustrates details of the F0 pattern partial change process of FIG. In step 202, the CPU 60 changes F0 at the center of each mora included in the accent phrase designated by the accent strength designation detected in step 102 based on a predetermined rule.

  For example, when the accent phrase is strengthened by using accent type accent type information (0 type: low, high, 1 type: high, low, etc.), the difference in the F0 value in the accent phrase is large. Set rules to be Specifically, for example, it is predetermined to add several hertz to F0 at the center of the mora with high accent height and subtract several hertz from F0 at the center of the mora with low accent. For example, as shown in FIG. 8B, when the accent strength of “in Fujitsu (Fujitsudewa)” is strongly designated by the symbol “′” attached to the “di” portion, the mora “di”, which is the accent height. For example, 5 Hz is added to the center F0. Further, for example, 5 Hz is subtracted from F0 at the center of another mora having low accent intensity. In FIG. 8B, a thick line 43H is a portion where several hertz is added, and a thick line 43L is a portion where several hertz is subtracted. For example, when the accent phrase information is used to weaken the accent strength of the accent phrase, the rule may be determined so that the difference in height of the F0 value in the accent phrase is reduced.

A F0 changed in step 202, in step 204, CPU 60 overwrites the corresponding element _{c t} of F0 pattern C which is estimated in step 106. In step 206, CPU 60 records the time t of the element _{c t} of C that has not been overwritten in step 204 to the primary storage unit 62.

Next, in step 300 of FIG. 4, F0 of the portion that has not been changed in step 200 is re-estimated. FIG. 10 illustrates details of the F0 pattern re-estimation process in FIG. In step 302, the CPU 60 solves the equation (22) for the equation (23). Here, Equation (24) represents a matrix of only the I-th row of the matrix A. I = {I ₁ , I ₂ ,..., Where I ₁ , I ₂ ,... Were not overwritten in step 204, that is, the matrix C recorded in the primary storage unit 62 in step 206. Corresponds to a row that has not been changed in the matrix C indicated by _t of the element c _t .
(22)
(23)
(24)

In order to explain the process of solving Equation (22) for Equation (23), Equation (22) can be considered as Equation (27) when Equation (25) and Equation (26) are used.
(25)
(26)
(27)

Since Equation (27) is a form in the case of solving simultaneous linear equations for the matrix C, it is possible to rearrange the elements of the matrix without changing the solution by the basic deformation of the matrix to obtain Equation (28). It is. Equation (29) shows the line changed in step 200. The elements of the matrix are rearranged so that the matrix C is divided into Expression (30) and Expression (31).
(28)
(29)
(30)
(31)

When formula (32) is divided into formula (33) and formula (34), formula (28) becomes formula (35). Equation (36) shows a matrix including only the I-th column of the matrix A.
(32)
(33)
(34)
(35)
(36)

By transforming equation (35), equation (37) is obtained, and as shown in equation (38), equation (39) can be solved.
(37)
(38)
(39)

  In step 304, the CPU 60 integrates the F0 pattern estimated in step 302 and the F0 changed in step 202. FIG. 8C illustrates the integrated F0 pattern. The thin broken line 47 indicates the F0 pattern before being re-estimated (see also FIG. 8B), the thick line 44 indicates the re-estimated F0 pattern, and the thin solid lines 43L and 43H indicate F0 partially changed. Moreover, the curve which connected the thick line 44 and the thin solid lines 43L and 43H has shown the integrated F0 pattern. F0 other than the designated F0 is re-estimated using the mean vector and covariance matrix of the HMM in which the dynamic features of F0 are introduced to estimate F0. Thereby, in the integrated F0 pattern, the accent intensity is adjusted, and the micro procedure is held in a portion surrounded by a circle 42 indicated by a broken line.

  In step 402 of FIG. 4, the CPU 60 synthesizes an audio signal using the F0 pattern integrated in step 304 and the mel cepstrum pattern estimated in step 106. For the synthesis of the speech signal, for example, an analysis / synthesis method is used in which the speech generation process is modeled and the speech is synthesized using the feature parameters.

  In step 404, the CPU 60 causes the speaker 78 to output sound using the sound signal synthesized in step 402.

  In step 406, when the CPU 60 detects that the user clicks the end button 96 of the user interface 91 shown in FIG. 6A with the mouse 74, the basic frequency adjustment processing is ended. In step 406, when the CPU 60 detects that the user clicks the re-input button 95 with the mouse 74, the CPU 60 returns to step 100 and continues the fundamental frequency adjustment process. The user can correct the Japanese notation and the accent strength designation in the text box 92 of the user interface 91 using the keyboard 72.

  In addition, although the example which a user inputs Japanese description and accent intensity | strength specification using the keyboard 72 was demonstrated in step 102, the technique of an indication is not limited to this. For example, Japanese notation and accent strength designation may be stored in a file in advance, and the CPU 60 may read Japanese notation and accent strength designation from the file. In addition, other language notation such as English notation may be used instead of Japanese notation.

  Moreover, although the example which produces | generates HMM corresponding to a sentence was demonstrated in step 106, the technique of an indication is not limited to this. The HMM corresponding to the sentence is an example of the HMM corresponding to the text of the disclosed technology. The HMM corresponding to the text may be an HMM corresponding to a phrase or a word, for example.

  Moreover, although the example which introduces the dynamic feature of F0 pattern and estimated F0 pattern in step 106 was demonstrated, the technique of an indication is not limited to this. The F0 pattern may be estimated without considering the dynamic features of the F0 pattern.

Moreover, although the example which changes F0 of the center of the mora which is a representative part was demonstrated in step 202, the technique of an indication is not limited to this. For example, instead of mora, syllables, phonemes, or vowels may be changed. Even when syllables, phonemes, or vowels are to be changed, the same effect as when mora is to be changed can be obtained. Further, the representative portion may be the top or the end of the mora instead of the center of the mora. Even when the head or tail of the mora is the representative part, the same effect as when the center of the mora is the representative part can be obtained. Also, rather than F0 corresponding to a single c _t Mora, it may be subject to change F0 corresponding to a plurality of successive c _t mora. Even when the F0 corresponding to a plurality of successive c _t Mora and change the subject, produces effects similar to those of the case where the object of changing the F0 corresponding to a single c _t Mora.

  In addition, since F0 cannot be defined for an unvoiced sound, F0 at the center of the sexual vowel may be changed in a mora represented by unvoiced consonant + voiced vowel. Or you may make it change F0 only with the mora represented only by voiced sound.

  Further, in step 202, an example has been described in which the accent type information of the accent phrase is used to add several hertz to the center F0 of the high mora and subtract several hertz from the center F0 of the low mora. Is not limited to this. For example, the F0 slope line from the word head vowel to the word final vowel is acquired, and for each vowel, the F0 component at the center of the phoneme that crosses the slope line is multiplied by a predetermined value according to the accent intensity, so that F0 is obtained. You may make it adjust.

  After step 106, as shown in FIG. 6B, intermediate notation corresponding to Japanese notation may be displayed in the text box 93 of the user interface 91. Here, “′” indicates that the accent strength is strong.

  Further, the accent intensity may be estimated not only by the user but also by the CPU 60. For example, it is estimated that the strength of an important accent phrase in a sentence is strong. More specifically, the accent strength of proper nouns may be estimated as strong, and the accent strength other than proper nouns may be estimated as medium. Alternatively, the beginning accent strength of the exhalation paragraph may be estimated to be strong, and the accent strength other than the beginning may be estimated to be medium. Further, it may be estimated that the accent intensity of an accent phrase including auxiliary morphemes such as “Like”, “Not”, “Would” is weak. The accent strength designation estimated by the CPU 60 may be displayed in, for example, a color different from the accent strength designation designated by the user in the text box 92 in FIG. 6A and the text boxes 92 and 93 in FIG. 6B. Moreover, the accent intensity designation estimated by the CPU 60 may be changed by the user.

  If it is determined in step 406 that the fundamental frequency adjustment process is not terminated, the user may correct the intermediate notation and its accent strength on the user interface 91 of FIG. 6B instead of the Japanese notation.

  The computer 10 of the disclosed technology may be a stand-alone computer, and the disclosed technology can be used for e-learning audio, audio for exhibition guidance in museums, museums, and the like. In this case, for example, a character string corresponding to a voice desired to be reproduced by the user adjusting F0 is input to the computer 10 using the keyboard 72. It is also possible to use the disclosed technology for reading out an e-mail. In this case, for example, a character string included in the e-mail is acquired from an e-mail application executed on the computer 10 as a character string to be reproduced as a sound by adjusting F0.

  The computer 10 of the disclosed technology may be a computer that operates as a server, and the disclosed technology can be used for e-learning audio, audio for exhibition guidance in museums, museums, and the like. In this case, for example, a character string corresponding to a voice that the user desires to reproduce by adjusting F0 is input to the client connected to the computer 10 by using the keyboard, and the voice is also reproduced by the client. In addition, the disclosed technology can be used as a voice interaction agent for a smartphone or an in-vehicle terminal. In this case, for example, the user inputs a question to the smartphone or the in-vehicle terminal by voice. The computer 10 connected to the smartphone or the in-vehicle terminal via the network recognizes the input voice and adjusts the voice F0 corresponding to the character string of the answer to the question using the disclosed technology. Send to the in-vehicle terminal. The smartphone or the in-vehicle terminal reproduces the sound.

  According to the disclosed technique, F0 other than the designated F0 is re-estimated using the mean vector and the covariance matrix of the HMM by introducing the dynamic feature of F0. The microprocess is held at the adjusted and adjusted F0. Therefore, the naturalness of the sound whose F0 is adjusted by the disclosed technique is not impaired. Further, according to the disclosed technique, F0 other than the designated F0 is re-estimated using the HMM mean vector and covariance matrix by introducing the dynamic features of F0, so that the accent strength is designated. Continuity with parts other than the accent phrase is not impaired. In addition, according to the disclosed technique, since the HMM learned using the learning data related to the accent strength is not used, it is not necessary to collect the learning data related to the accent strength.

  Regarding the above embodiment, the following additional notes are disclosed.

(Appendix 1)
A fundamental frequency pattern estimator for estimating a fundamental frequency pattern of speech corresponding to the text using information of a hidden Markov model corresponding to the text;
A fundamental frequency changing unit that changes the value of the fundamental frequency of the designated portion in the estimated fundamental frequency pattern to a value according to the designated accent intensity;
Using the information of the hidden Markov model, a re-estimator that re-estimates a fundamental frequency pattern corresponding to the text and having a value of the fundamental frequency of the designated portion changed;
Basic frequency adjustment device including

(Appendix 2)
The fundamental frequency adjustment device according to appendix 1, wherein the information of the hidden Markov model is an average vector and a covariance matrix corresponding to the state of the hidden Markov model.

(Appendix 3)
The re-estimator re-estimates the fundamental frequency pattern using the information of the hidden Markov model, the dynamic characteristics of the estimated fundamental frequency pattern, and the changed fundamental frequency value of the designated portion.
The fundamental frequency adjusting device according to appendix 1 or 2.

(Appendix 4)
The specified portion in the estimated fundamental frequency pattern and the specified accent intensity are:
Specified by the user and
Estimated based on linguistic information obtained from the text,
Specified by at least one of
The fundamental frequency adjusting device according to any one of appendices 1 to 3.

(Appendix 5)
The fundamental frequency adjustment according to any one of appendices 1 to 4, wherein the designated part is a center of a mora, a syllable, a phoneme, or a vowel included in a designated representative part in the estimated fundamental frequency pattern apparatus.

(Appendix 6)
The value of the fundamental frequency of the designated portion is determined based on accent type information of an accent phrase included in the designated portion in the estimated fundamental frequency pattern. Basic frequency adjustment device.

(Appendix 7)
Computer
Using information of the hidden Markov model corresponding to the text, estimating the fundamental frequency pattern of the speech corresponding to the text,
Change the value of the fundamental frequency of the specified part in the estimated fundamental frequency pattern to a value according to the specified accent strength,
Using the information of the hidden Markov model, re-estimate a fundamental frequency pattern corresponding to the text and having a value of the fundamental frequency of the designated portion changed.
Basic frequency adjustment method.

(Appendix 8)
The fundamental frequency adjustment method according to appendix 7, wherein the information of the hidden Markov model is an average vector and a covariance matrix corresponding to the state of the hidden Markov model.

(Appendix 9)
Re-estimate the fundamental frequency pattern using the information of the hidden Markov model, the dynamic characteristics of the estimated fundamental frequency pattern and the changed fundamental frequency value of the specified portion;
The fundamental frequency adjusting method according to appendix 7 or 8.

(Appendix 10)
The specified portion in the estimated fundamental frequency pattern and the specified accent intensity are:
Specified by the user and
Estimated based on linguistic information obtained from the text,
Specified by at least one of
The fundamental frequency adjustment method according to any one of appendices 7 to 9.

(Appendix 11)
The fundamental frequency adjustment method according to any one of appendices 7 to 10, wherein the designated portion is a center of a mora, a syllable, a phoneme, or a vowel included in the designated portion in the estimated fundamental frequency pattern. .

(Appendix 12)
The value of the fundamental frequency of the designated portion is determined based on accent type information of an accent phrase included in the designated portion in the estimated fundamental frequency pattern. The basic frequency adjustment method described.

(Appendix 13)
Using information of the hidden Markov model corresponding to the text, estimating the fundamental frequency pattern of the speech corresponding to the text,
Change the value of the fundamental frequency of the specified part in the estimated fundamental frequency pattern to a value according to the specified accent strength,
Using the information of the hidden Markov model, re-estimate a fundamental frequency pattern corresponding to the text and having a value of the fundamental frequency of the designated portion changed.
A program for causing a computer to execute basic frequency adjustment processing including the above.

(Appendix 14)
The program according to appendix 13, wherein the information of the hidden Markov model is an average vector and a covariance matrix corresponding to the state of the hidden Markov model.

(Appendix 15)
Re-estimate the fundamental frequency pattern using the information of the hidden Markov model, the dynamic characteristics of the estimated fundamental frequency pattern and the changed fundamental frequency value of the specified portion;
The program according to appendix 13 or 14.

(Appendix 16)
The specified portion in the estimated fundamental frequency pattern and the specified accent intensity are:
Specified by the user and
Estimated based on linguistic information obtained from the text,
Specified by at least one of
The program according to any one of appendices 13 to 15.

(Appendix 17)
The program according to any one of appendices 13 to 16, wherein the designated portion is a center of a mora, a syllable, a phoneme, or a vowel included in a designated representative portion in the estimated fundamental frequency pattern.

(Appendix 18)
The supplementary frequency according to any one of appendices 13 to 17, wherein a value of a fundamental frequency of the designated portion is determined based on accent type information of an accent phrase included in the designated portion in the estimated fundamental frequency pattern. program.

(Appendix 19)
A fundamental frequency pattern estimator for estimating a fundamental frequency pattern of speech corresponding to the text using information of a hidden Markov model corresponding to the text;
A fundamental frequency changing unit that changes the value of the fundamental frequency of the representative portion of the designated portion in the estimated fundamental frequency pattern to a value according to the designated accent intensity;
Using the information of the hidden Markov model, a re-estimator that re-estimates a fundamental frequency pattern corresponding to the text and having a value of a fundamental frequency of the representative portion changed;
A speech synthesizer that synthesizes a speech signal based on an integrated fundamental frequency pattern and a mel cepstrum pattern estimated using information of the hidden Markov model;
A speech synthesizer.

(Appendix 20)
Computer
Using information of the hidden Markov model corresponding to the text, estimating the fundamental frequency pattern of the speech corresponding to the text,
Change the value of the fundamental frequency of the representative part of the specified part in the estimated fundamental frequency pattern to a value according to the specified accent strength,
Using the information of the hidden Markov model, re-estimate the fundamental frequency pattern corresponding to the text and the value of the fundamental frequency of the representative portion is changed,
Synthesizing a speech signal based on an integrated fundamental frequency pattern and a mel cepstrum pattern estimated using information of the hidden Markov model;
Speech synthesis method.

(Claim 21)
Using information of the hidden Markov model corresponding to the text, estimating the fundamental frequency pattern of the speech corresponding to the text,
Change the value of the fundamental frequency of the representative part of the specified part in the estimated fundamental frequency pattern to a value according to the specified accent strength,
Using the information of the hidden Markov model, re-estimate the fundamental frequency pattern corresponding to the text and the value of the fundamental frequency of the representative portion is changed,
Synthesizing a speech signal based on an integrated fundamental frequency pattern and a mel cepstrum pattern estimated using information of the hidden Markov model;
For causing a computer to execute speech synthesis processing including the above.

10 Computer 16 Parameter estimation unit 18 Accent strength-F0 conversion unit 20 F0 designation unit 22 F0 re-estimation unit 24 Analysis synthesis unit 30 HMM DB
60 CPU
62 Primary storage unit 64 Secondary storage unit 68 HMM DB storage area

Claims (11)

A fundamental frequency pattern estimator for estimating a fundamental frequency pattern of speech corresponding to the text using information of a hidden Markov model corresponding to the text;
A fundamental frequency changing unit that changes the value of the fundamental frequency of the designated portion in the estimated fundamental frequency pattern to a value according to the designated accent intensity;
Using the information of the hidden Markov model, a re-estimator that re-estimates a fundamental frequency pattern corresponding to the text and having a value of the fundamental frequency of the designated portion changed;
Basic frequency adjustment device including   The fundamental frequency adjustment device according to claim 1, wherein the information of the hidden Markov model is an average vector and a covariance matrix corresponding to a state of the hidden Markov model. The re-estimator re-estimates the fundamental frequency pattern using the information of the hidden Markov model, the dynamic characteristics of the estimated fundamental frequency pattern, and the changed fundamental frequency value of the designated portion.
The fundamental frequency adjusting device according to claim 1 or 2. The specified portion in the estimated fundamental frequency pattern and the specified accent intensity are:
Specified by the user and
Estimated based on linguistic information obtained from the text,
Specified by at least one of
The fundamental frequency adjusting device according to any one of claims 1 to 3.   The specified part is the center of a mora, a syllable, a phoneme, or a vowel included in a specified representative part in the estimated fundamental frequency pattern, according to any one of claims 1 to 4. Basic frequency adjustment device.   The value of the fundamental frequency of the designated part is determined based on accent type information of an accent phrase included in the designated part in the estimated fundamental frequency pattern. The fundamental frequency adjusting device described in 1. Computer
Using information of the hidden Markov model corresponding to the text, estimating the fundamental frequency pattern of the speech corresponding to the text,
Change the value of the fundamental frequency of the specified part in the estimated fundamental frequency pattern to a value according to the specified accent strength,
Using the information of the hidden Markov model, re-estimate a fundamental frequency pattern corresponding to the text and having a value of the fundamental frequency of the designated portion changed.
Basic frequency adjustment method. Using information of the hidden Markov model corresponding to the text, estimating the fundamental frequency pattern of the speech corresponding to the text,
Change the value of the fundamental frequency of the specified part in the estimated fundamental frequency pattern to a value according to the specified accent strength,
Using the information of the hidden Markov model, re-estimate a fundamental frequency pattern corresponding to the text and having a value of the fundamental frequency of the designated portion changed.
A program for causing a computer to execute basic frequency adjustment processing including the above. A fundamental frequency pattern estimator for estimating a fundamental frequency pattern of speech corresponding to the text using information of a hidden Markov model corresponding to the text;
A fundamental frequency changing unit that changes the value of the fundamental frequency of the designated portion in the estimated fundamental frequency pattern to a value according to the designated accent intensity;
Using the information of the hidden Markov model, a re-estimator that re-estimates a fundamental frequency pattern corresponding to the text and having a value of the fundamental frequency of the designated portion changed;
A speech synthesizer that synthesizes a speech signal based on an integrated fundamental frequency pattern and a mel cepstrum pattern estimated using information of the hidden Markov model;
A speech synthesizer. Computer
Using information of the hidden Markov model corresponding to the text, estimating the fundamental frequency pattern of the speech corresponding to the text,
Change the value of the fundamental frequency of the specified part in the estimated fundamental frequency pattern to a value according to the specified accent strength,
Using the information of the hidden Markov model, re-estimate a fundamental frequency pattern corresponding to the text and having a value of the fundamental frequency of the designated portion changed.
Synthesizing a speech signal based on an integrated fundamental frequency pattern and a mel cepstrum pattern estimated using information of the hidden Markov model;
Speech synthesis method. Using information of the hidden Markov model corresponding to the text, estimating the fundamental frequency pattern of the speech corresponding to the text,
Change the value of the fundamental frequency of the specified part in the estimated fundamental frequency pattern to a value according to the specified accent strength,
Using the information of the hidden Markov model, re-estimate a fundamental frequency pattern corresponding to the text and having a value of the fundamental frequency of the designated portion changed.
Synthesizing a speech signal based on an integrated fundamental frequency pattern and a mel cepstrum pattern estimated using information of the hidden Markov model;
For causing a computer to execute speech synthesis processing including the above.