JP2020106643A - Language processing unit, language processing program and language processing method - Google Patents

Abstract

To improve prediction accuracy of an acoustic feature amount since an outlier does not occur.SOLUTION: A speech synthesizer 10 comprises a CPU, and the CPU learns a continuous length DNN and an acoustic feature amount DNN on the basis of voice corpus and generates the constructed continuous length DNN and the acoustic feature amount DNN into a synthesized speech corresponding to a text of a free sentence. A language feature amount based on the text used in learning is normalized by taking a ratio of an associated attribute in terms of language by using only a value in one speech. The language feature amount based on the text synthesized when the synthesized speech is generated is similarly normalized. Thus, an outlier does not occur.SELECTED DRAWING: Figure 1

Description

The present invention relates to a language processing device, a language processing program, and a language processing method, and more particularly to, for example, a language processing device, a language processing program, and a language processing method for generating and outputting synthetic speech according to an input text.

An example of the background art is disclosed in Patent Document 1. Patent Document 1 discloses a speech synthesis device that learns a deep neural network acoustic model from speech data and generates synthetic speech using the learned deep neural network acoustic model. In this speech synthesizer, a deep neural network acoustic model in which a language feature vector expressing context data as a numerical vector and a speaker code concatenated are input, and speech parameters corresponding to the speaker and context data are output. Is learned.

Another example of the background art is disclosed in Patent Document 2. According to Patent Document 2, a time length DNN (deep neural network) and an acoustic feature amount DNN are pre-learned from a speech corpus, and a speech waveform corresponding to text is synthesized using the learned time length DNN and acoustic feature amount DNN. A speech synthesizer capable of performing is disclosed. In this speech synthesizer, the pre-learning unit generates a phoneme language feature amount, a phoneme frame language feature amount, a phoneme time length, and a phoneme frame acoustic feature amount from a speech corpus, and adds a speaker label and an emotion label. To do. Then, the pre-learning unit learns the time length DNN by giving the phoneme language feature amount, the speaker label, the emotion label, and the phoneme time length, and the phoneme frame language feature amount, the speaker label, the emotion label, and the phoneme frame. The acoustic feature amount DNN is learned by giving the acoustic feature amount of

JP, 2017-032839, A JP, 2018-146803, A

H.Zen et al, IEICE Trans.Inf. & Syst., vol.E90-D, no.5, pp.825-834, May 2007 Zhizheng Wu et al,ISCA SSW9,vol PS2-13,pp.218-223,Sep 2016

In Patent Document 1 and Patent Document 2, nothing is disclosed in the normalization of the language feature amount, but in Non-Patent Document 1 and Non-Patent Document 2 referred to in these Patent Documents, all learning data is used. Normalization with calculated means and variances or minimum and maximum values is used. However, in these normalization methods, an outlier occurs in a speech synthesizer in which a text of free text is input, because a value outside learning is included in the language feature amount. Further, the extrapolation capability of the neural network is not sufficient, which causes a problem of unstable prediction. As a measure against this problem, generally, a measure is taken to increase the learning data and widen the range to be covered. However, this measure cannot cover all input patterns. In addition, the cost required to collect a lot of learning data becomes high.

Therefore, a main object of the present invention is to provide a novel language processing device, language processing program, and language processing method.

Another object of the present invention is to provide a language processing device, a language processing program, and a language processing method capable of preventing outliers from occurring.

A first aspect of the present invention is a language processing apparatus for normalizing a language feature vector sequence that is input to a deep neural network of a speech synthesis apparatus that generates synthetic speech, and that normalizes a language feature vector sequence. The language processing apparatus includes a normalization unit that normalizes a first attribute in a language feature vector sequence with a second attribute different from the first attribute.

A second invention is according to the first invention, and the first attribute and the second attribute are linguistically related values.

A third invention is according to the first or second invention, and the normalizing means normalizes by dividing the first attribute by the second attribute.

A fourth invention is according to any of the first to third inventions, and the absolute value of the first attribute is equal to or less than the absolute value of the second attribute.

A fifth invention is a language processing program which is input to a deep neural network of a speech synthesizer for generating synthetic speech and is executed by a language processor for normalizing a language feature vector sequence composed of a plurality of different attributes. Then, the processor of the language processing device is caused to execute a normalization step of normalizing the first attribute in the language feature vector sequence for one utterance with a second attribute different from the first attribute. It is a processing program.

A sixth invention is a language processing method for normalizing a language feature vector sequence which is input to a deep neural network of a voice synthesizer which generates a synthesized voice and which is composed of a plurality of different attributes. It is a language processing method for normalizing a first attribute in a language feature vector sequence with a second attribute different from the first attribute.

According to the present invention, since the first attribute in the language feature vector sequence for one utterance is normalized by the second attribute different from the first attribute, it is possible to prevent an outlier from occurring. You can

The above-mentioned objects, other objects, features and advantages of the present invention will become more apparent from the following detailed description of the embodiments with reference to the drawings.

FIG. 1 is a functional block diagram showing an example of the speech synthesizer of this embodiment. FIG. 2 is a diagram for explaining the pre-learning unit shown in FIG. FIG. 3 is a diagram for explaining the text analysis unit shown in FIG. FIG. 4 is a diagram for explaining the acoustic analysis unit shown in FIG. FIG. 5 is a diagram for explaining the combination processing unit shown in FIG. FIG. 6 is a diagram for explaining the relationship between the phoneme language feature amount, which is the input data and output data of the duration DNN and the acoustic feature amount DNN, the phoneme frame language feature amount, the phoneme duration length, and the phoneme frame acoustic feature amount. It is a figure. FIG. 7 is a diagram showing an example of language feature amount data. FIG. 8 is a flowchart showing the attribute value calculation processing of the CPU incorporated in the speech synthesizer. FIG. 9 is a flowchart showing the first attribute value calculation processing of the CPU incorporated in the speech synthesizer. FIG. 10 is a flow chart showing the second attribute value calculation processing of the CPU incorporated in the speech synthesizer.

FIG. 1 is a functional block diagram of a speech synthesizer 10 of this embodiment. The speech synthesizer 10 is a general-purpose personal computer or workstation, and also functions as a language processing device or a normalization processing device that normalizes a language feature amount described later. Although illustration is omitted, the voice synthesis device 10 includes components such as a CPU, a memory (HDD, ROM, RAM) and a communication device (network connection device).

The speech synthesizer 10 will be described below. The CPU of the speech synthesizer 10 processes learning and generation of synthetic speech according to various programs. In addition, each of the storage units 12 and 16 is a memory (HDD or/and RAM) of the voice synthesizer 10, or a memory built in a computer on a network accessible by the voice synthesizer 10 or an accessible network. Means database.

As shown in FIG. 1, the speech synthesis device 10 includes a storage unit 12, a pre-learning unit 14, a storage unit 16, and a synthesis processing unit 18. The storage unit 12 stores a voice corpus. The voice corpus is information about a voice in which a specific sentence is read by one or more speakers. In this example, the information about voice is text and voice waveforms. However, the voice waveforms of the text and the voice reading the text are stored in the storage unit 12 as a pair (associated with each other).

The pre-learning unit 14 learns the duration DNN by performing a predetermined text analysis on the text of the voice corpus read from the storage unit 12 and a predetermined acoustic analysis on the voice waveform of the text. Information such as the acoustic feature amount for learning the language feature amount and the acoustic feature amount DNN for However, DNN means a deep neural network. The pre-learning unit 14 pre-learns the duration DNN and the acoustic feature amount DNN stored in the storage unit 16 using information such as the language feature amount and the acoustic feature amount.

Since the method of text analysis and the method of acoustic analysis are known, detailed description thereof will be omitted in this embodiment. Further, in this embodiment, the duration DNN and the acoustic feature amount DNN are stored in the same storage unit 16, but they may be stored in different storage units.

In this embodiment, the continuation length DNN and the acoustic feature amount DNN are forward-propagation type networks in which a plurality of nodes are configured by an input layer, a plurality of hidden layers (intermediate layers), and an output layer, respectively.

The duration length DNN is given to each unit of the input layer, hidden layer, and output layer by giving the language feature amount of the phoneme to each unit of the input layer and the duration length of the phoneme to the unit of the output layer during learning. Weights and the like are calculated, and phoneme-based learning is performed. In this embodiment, the phoneme language features for learning include, for example, phoneme labels, mora information, accent phrase information, expiration paragraph information, and utterance information. However, the phoneme duration is represented by the number of phoneme frames forming the phoneme. In this embodiment, the length of one phoneme frame is 5 msec.

When executing a speech synthesis process, which will be described later, a phoneme linguistic feature amount is given to each unit of the input layer having the duration DNN. Then, the unit of the output layer having the duration DNN outputs the phoneme duration corresponding to the language feature amount of the phoneme given to the input layer.

In addition, the acoustic feature amount DNN is given by inputting the language feature amount of the phoneme frame to each unit of the input layer and giving the acoustic feature amount of the phoneme frame to each unit of the output layer at the time of learning. Also, the weight of each unit in the output layer is calculated, and learning is performed for each phoneme frame. In this embodiment, the acoustic feature amount of the phoneme frame includes information such as a spectrum coefficient, a noise characteristic coefficient, a pitch, and a voiced/unvoiced determination.

When executing a speech synthesis process described later, the language feature amount of the phoneme frame is given to each unit of the input layer of the acoustic feature amount DNN. Then, each unit of the output layer of the acoustic feature amount DNN outputs the acoustic feature amount of the phoneme frame corresponding to the language feature amount of the phoneme frame given to the input device.

FIG. 2 is a diagram for explaining the pre-learning unit 14 shown in FIG. As shown in FIG. 2, the pre-learning unit 14 includes a text analysis unit 14a and an acoustic analysis unit 14b. As shown in FIG. 3, the text analysis unit 14a includes a text analysis unit 140, a frame processing unit 142, and a normalization processing unit 144.

The text analysis unit 140 performs text analysis such as morphological analysis on the text read from the speech corpus of the storage unit 12, generates phoneme language feature amounts for each phoneme, and frame-processes the phoneme language feature amounts. The phoneme label included in the language feature amount of the phoneme is output to the acoustic analysis unit 14b while being output to the unit 142 and the normalization processing unit 144.

Here, the linguistic feature amount of a phoneme means information generated by text analysis. For example, the phoneme language feature amount generated by text analysis includes various information such as phoneme labels, accent positions, part-of-speech information, accent phrase information, expiration paragraph information, and total number information for each phoneme. However, the phoneme label is information (phoneme information) for identifying the phonemes forming the text, and includes the preceding and following phonemes in addition to the phoneme.

The frame processing unit 142 receives the phoneme language feature amount for pre-learning from the text analysis unit 140, and also receives the phoneme duration from the acoustic analysis unit 14b. The frame processing means 142 generates the language feature amount of the phoneme frames for the number of phoneme frames indicated by the phoneme duration length based on the phoneme language feature amount and the phoneme duration length for pre-learning. The language feature amount of the generated phoneme frame is output to the normalization processing unit 144.

The normalization processing unit 144 normalizes each of the phoneme language feature amount and the phoneme frame language feature amount, and outputs the normalized phoneme language feature amount to the duration DNN, and also the normalized phoneme frame. The language feature amount of is output to the acoustic feature amount DNN.

The normalization processing in the normalization processing means 144 will be described later in detail.

As shown in FIG. 4, the acoustic analysis unit 14b includes a phoneme segmentation processing unit 150 and an acoustic analysis unit 152. The phoneme segmentation processing unit 150 receives a phoneme label from the text analysis unit 14a, and performs acoustic analysis on the speech waveform read from the speech corpus of the storage unit 12 using predetermined learning data. The phoneme segmentation processing unit 150 identifies the position of the phoneme indicated by the phoneme label in the speech waveform, and obtains the segmentation position of the phoneme. The obtained phoneme delimiter position is output to the acoustic analysis unit 152.

The phoneme segmentation processing unit 150 also obtains the phoneme duration indicated by the phoneme label based on the segmentation positions of the phonemes. As described above, the phoneme duration is represented by the number of phoneme frames forming the phoneme. The obtained phoneme duration is output to each unit of the output layer in the duration DNN of the storage unit 16 and also to the text analysis unit 14a (frame processing unit 142).

The acoustic analysis unit 152 receives the phoneme delimiter positions from the phoneme delimiter processing unit 150, performs an acoustic analysis on the speech waveform read from the speech corpus of the storage unit 12, and analyzes a plurality of phoneme frames forming a phoneme. For each of them, the acoustic feature amount of the phoneme frame is generated. For example, the acoustic feature amount of a phoneme frame includes information such as spectrum coefficient, noise factor, pitch, voice/unvoiced determination, and the like. The acoustic feature amount of the generated phoneme frame is output to each unit of the output layer in the acoustic feature amount DNN of the storage unit 16.

Since a method of obtaining a phoneme delimitation position and a phoneme duration by acoustic analysis and generating an acoustic feature amount of a phoneme frame is known, detailed description thereof will be omitted in this embodiment.

As described above, the text analysis unit 14a outputs the phoneme language feature amount for pre-learning to the input layer of the duration DNN, and the acoustic analysis unit 14b outputs the phoneme duration of the phoneme to the output layer of the duration DNN. Output to. Thereby, the pre-learning of the continuation length DNN is performed. Further, the text analysis unit 14a outputs the language feature amount of the phoneme frame to the input layer of the acoustic feature amount DNN, and the acoustic analysis unit 14b outputs the acoustic feature amount of the phoneme frame to the output layer of the acoustic feature amount DNN. .. As a result, the acoustic feature amount DNN is pre-learned.

FIG. 5 is a diagram showing an example of a specific configuration of the synthesis processing unit 18 shown in FIG. As shown in FIG. 5, the synthesis processing unit 18 includes a text analysis unit 180, a duration generation unit 182, an acoustic feature amount generation unit 184, and a voice waveform synthesis unit 186.

The text analysis unit 180 performs the same processing as the text analysis unit 14a shown in FIG. Specifically, the text analysis unit 180 is input with a text in free text, performs a text analysis on this text, generates a phoneme language feature amount for each phoneme, and normalizes it. The text analysis unit 180 is similar to the phoneme language feature amount for pre-learning generated by the text analysis unit 14a shown in FIG. 2 based on the phoneme language feature amount generated and normalized by the text analysis. Generate phoneme language features. Then, the text analysis unit 180 outputs the generated language feature amount of the phoneme to the duration length generation unit 182 and the acoustic feature amount generation unit 184.

In addition, the text analysis unit 180 includes the duration length of the phoneme corresponding to the language feature amount of the phoneme output from the duration length generation unit 182 and the acoustic feature amount generation unit 184 to the duration length generation unit 182 and the acoustic feature amount generation unit 184. Is input, and the language feature amount of the phoneme frames for the number of phoneme frames indicated by the phoneme duration length is generated based on the phoneme language feature amount and the phoneme duration length. Then, the text analysis unit 180 outputs the language feature amount of the phoneme frame to the duration length generation unit 182 and the acoustic feature amount generation unit 184.

The duration length generation unit 182 receives the phoneme language feature amount from the text analysis unit 180, and uses the duration length DNN of the storage unit 16 to generate a phoneme duration length based on the phoneme language feature amount. Then, the duration length generation unit 182 outputs the duration length of the phoneme to the text analysis unit 180. Further, the acoustic feature amount generation unit 184 receives the language feature amount of the phoneme frame from the text analysis unit 180, and uses the acoustic feature amount DNN of the storage unit 16 based on the language feature amount of the phoneme frame to extract the phoneme frame The acoustic feature amount is generated, and the acoustic feature amount of the phoneme frame is output to the speech waveform synthesis unit 186.

The speech waveform synthesis unit 186 receives the acoustic feature amount of the phoneme frame from the acoustic feature amount generation unit 184, synthesizes the speech waveform based on the acoustic feature amount of the phoneme frame, and outputs the synthesized speech waveform. Specifically, the speech waveform synthesizer 186 generates a vocal cord sound source waveform based on information such as pitch and noise characteristics included in the acoustic feature amount of the phoneme frame. Then, the voice waveform synthesizing unit 186 performs vocal tract filter processing on the vocal cord sound source waveform based on information such as a spectral coefficient included in the acoustic feature amount of the phoneme frame to synthesize a voice waveform. That is, the synthetic voice corresponding to the text is generated and output.

Since a method of synthesizing a speech waveform based on the acoustic feature amount of a phoneme frame is well known, detailed description thereof will be omitted in this embodiment.

FIG. 6 is for explaining the relationship between the phoneme language feature amount, which is the input data and output data of the duration DNN and the acoustic feature amount DNN, the phoneme frame language feature amount, the phoneme duration length, and the phoneme frame acoustic feature amount. FIG.

As shown in FIG. 6, when the text for one utterance is "This is this, which is it?", the exhalation paragraphs are "that is this" and "that is which". Also, in this case, the accent phrases of "that is this" are "that" and "this". Further, in this case, the mora of "that" is "a", "re", and "ga". In this case, the phoneme label of "a" is set to "a", the phoneme label of "re" is set to "r" and "e", and the phoneme label of "ga" is set to "g" and "a". .. In the example shown in FIG. 6, the phoneme durations of the phoneme labels “a”, “r”, “e”, “g” and “a” are “6”, “3”, “5” and “5”, respectively. 5”, “5” and “3”. As described above, the utterance, the exhalation paragraph, the accent phrase, the mora, and the phoneme have a hierarchical structure, and the information having attributes regarding these is also a hierarchical structure. As described above, in this embodiment, the length of one phoneme frame is 5 msec.

As shown in FIG. 6, in the time section of the phoneme label “a”, the language feature amount of each set of phonemes (each of the above information) is generated corresponding to this one phoneme, and the language of the six sets of phoneme frames is generated. The feature amount (each information item) is generated, and the acoustic feature amount (each information item) of the six phoneme frames is generated. Further, in the time section of the phoneme label “r”, the language feature amount of one set of phonemes is generated corresponding to this one phoneme, the language feature amount of three sets of phoneme frames is generated, and three sets of phoneme frames are generated. Acoustic features are generated. In addition, in the time section of the phoneme label “e”, the linguistic feature amount of one set of phonemes is generated, the linguistic feature amount of five sets of phoneme frames is generated, and the five sets of phonemes are generated corresponding to this one phoneme. The acoustic feature amount of the frame is generated.

Thus, in the pre-learning, each unit of the input layer with the duration DNN is given the linguistic feature amount of the phoneme, and the unit of the output layer is given the duration of the phoneme. It is done as a unit. That is, the phoneme language feature amount and the phoneme duration are given to the duration DNN for each phoneme, and pre-learning is performed. Further, in speech synthesis, the duration length DNN is used for each phoneme, and the duration length of the phoneme is generated and output based on the language feature amount of the phoneme.

Further, as described above, in the pre-learning, the language feature amount of the phoneme frame is given to each unit of the input layer of the acoustic feature amount DNN, and the acoustic feature amount of the phoneme frame is given to each unit of the output layer. This pre-learning is performed in units of phoneme frames. That is, the acoustic feature amount DNN is given the language feature amount of the phoneme frame and the acoustic feature amount of the phoneme frame for each phoneme frame, and pre-learning is performed. In speech synthesis, the acoustic feature amount DNN is used for each phoneme frame, and the acoustic feature amount of the phoneme frame is generated and output based on the language feature amount of the phoneme frame.

FIG. 7 is a diagram showing an example of language feature amount data. As described above, the linguistic feature amount includes a phoneme-related attribute, a mora-related attribute, an accent phrase-related attribute, an expiratory paragraph-related attribute, and an utterance-related attribute, and is represented by a d-dimensional vector at time t. As the language feature amount, information on each attribute at time t is represented by a numerical value. Although the detailed description of each attribute is omitted, as an example, the attribute relating to the accent phrase includes “the ascending order position of the mora in the accent phrase”. However, “corresponding” means that the process is a target when the normalization process is performed.

Here, in the above-described normalization processing unit 144, the language feature amount of the phoneme subjected to the text analysis by the text analysis unit 140 and the language feature amount of the phoneme subjected to the text analysis are processed by the frame processing unit 142. The language feature amount of the phoneme frame is input. The input linguistic feature quantity is a vector series and can be expressed by Equation 1. As shown in Formula 2, the normalization processing unit 144 sets the first attribute (first attribute value) to be an attribute different from the first attribute and having an absolute value larger than that of the first attribute. Alternatively, the language feature amount is normalized by dividing it by an equal second attribute (second attribute value), and the normalized language feature amount is output. However, it is assumed that the first attribute and the second attribute are related. Further, the linguistic feature amount L for one utterance is a series of linguistic feature amount vectors having d-dimensional attributes at time t as elements, as shown in FIG. However, in Equation 2, |·| means an absolute value. In this embodiment, the absolute value of the first attribute value is less than or equal to the absolute value of the second attribute value.

Further, the above-mentioned "related things" will be described with reference to FIGS. When the text analysis unit 140 analyzes the text, as shown in FIG. 7, information having attributes relating to utterances, expiration paragraphs, accent phrases, mora, and phonemes as elements is obtained. These pieces of information have a hierarchical structure as shown in FIG. 6, and the information of each hierarchy is mainly composed of the information of the lower hierarchy. For example, the hierarchy of accent phrases has a hierarchy of mora and phonemes at the lower level, and the attributes of accent phrases include the ascending position of the mora in the accent phrase and the total number of mora in the accent phrase. Therefore, in the normalization processing in the normalization processing unit 144 of this embodiment, basically, the attribute related to the position is divided by the attribute related to the total number of the same hierarchy under the relation of the same hierarchy, and the attribute related to the total number is called the total number. It will be divided by the attribute on the total number of different tiers under relevance. The continuation length is the same as the total length. The normalization process is applied to all layers.

The normalization method shown in Formula 2 does not include conditions other than the utterance such as the average and variance or the minimum and maximum values calculated from all learning data as in Non-Patent Document 1 and Non-Patent Document 2. Outliers do not occur because they are calculated under the limited conditions within one utterance. Therefore, it is possible to avoid a decrease in the prediction performance due to an outlier, and it is possible to more stably predict the acoustic feature amount than before.

FIG. 8 is a flow chart showing an example of attribute normalization processing of the CPU of the speech synthesizer 10 shown in FIG. A program for this attribute normalization processing (corresponding to a “language processing program”) is stored in the memory of the speech synthesizer 10 and executed by the CPU. The same applies to other programs and data necessary for the learning process and the voice synthesis process.

Further, in this embodiment, a process for normalizing an attribute of “ascending position of mora in the accent phrase” among a plurality of attributes included in the language feature will be described. Although detailed description is omitted, similar attribute normalization processing is executed for other attributes.

As shown in FIG. 8, when the CPU starts the attribute normalization process, the CPU executes a first attribute value calculation process (see FIG. 9) described below in step S1, and a second attribute value calculation described below in step S3. The process (see FIG. 10) is executed.

In the next step S5, the variable t for time is initialized and the array array m[T] is set. That is, the CPU substitutes 0 for the variable t and sets an array array m[T] capable of storing elements up to the maximum value T of the phoneme frame (here, the ascending order position of the mora in the accent phrase). However, the variable t is a variable for counting the number of frames. This is the same in the first attribute value calculation process (FIG. 9) and the second attribute value calculation process (FIG. 10) described later. The array array m[T] is an array for storing normalized attribute values. The maximum value T is the total number of phoneme frames in one utterance.

In the next step S7, it is determined whether the variable t is smaller than the maximum value T of the phoneme frame. If “NO” in the step S7, that is, if the variable t is equal to or larger than the maximum value T of the phoneme frame, it is determined that “the ascending position of the mora in the accent phrase” is normalized for all the mora, and the attribute The normalization process ends.

On the other hand, if “YES” in the step S7, that is, if the variable t is less than the maximum value T of the phoneme frame, it is determined that there remains a mora for which “the ascending order position of the mora in the accent phrase” is not normalized. In step S9, the element m[t] in which the ascending position is the variable t is calculated (m[t]=x[t]/y[t]), and the variable t is incremented by 1 in step S11. (T=t+1), the process returns to step S7.

FIG. 9 is a flowchart showing the first attribute value calculation processing of the CPU executed in step S1 shown in FIG. 8 and step S51 in FIG. 10 described later. Hereinafter, the first attribute value calculation process will be described, but the same process as the process already described will be briefly described.

As shown in FIG. 9, when starting the first attribute value calculation processing, the CPU initializes the variables t and i in step S21 (t=0, i=1), and the array array x[T ] (X[0],x[1],...,x[T-1]). However, the variable i is a variable for counting the ascending order position of the mora in the accent phrase. The same applies to FIG. 10. The array array x[T] is an array for storing the ascending order position of the mora for each attribute value.

In the next step S23, it is determined whether the variable t is smaller than the maximum value T of the phoneme frame. If “NO” in the step S23, the first attribute value calculation process is ended, and the process returns to the attribute normalization process shown in FIG.

On the other hand, if “YES” in the step S23, the numerical value of the variable i is substituted into the element x[t] in a step S25. That is, the ascending numbers of the mora in the accent phrase are assigned. In the next step S27, it is determined whether or not the mora ends in the variable t. Here, the CPU determines whether the variable t indicates the final frame in the mora. If "YES" in the step S27, that is, if the mora is not the end in the variable t, the process proceeds to a step S31. On the other hand, if “NO” in the step S27, that is, if the mora ends in the variable t, the variable i is incremented by 1 (i=i+1) in a step S29, and the process proceeds to the step S31.

In step S31, it is determined whether the variable t is the end of the accent phrase. Here, the CPU determines whether the variable t indicates the final frame in the accent phrase.

If "NO" in the step S31, and if the accent phrase is not the end in the variable t, the process proceeds to a step S35. On the other hand, if “YES” in the step S31, that is, if the accent phrase ends in the variable t, the variable i is set to an initial value in a step S33, and the variable t is incremented by 1 in a step S35 (t= t+1), and returns to step S23.

FIG. 10 is a flowchart of the second attribute value calculation process of step S3 shown in FIG. As shown in FIG. 10, when the second attribute value calculation process is started, the first attribute value calculation process shown in FIG. 9 is executed in step S51.

In the next step S53, the variables t and i are initialized (t=T, i=x[T-1]), and the array array y[T] is prepared (y[0], y[1]). ,…,Y[T-1]). However, the array array y[T] is an array for storing each element of the second attribute value (here, the total number of moras in the accent phrase) for normalizing each element of the first attribute value. Is. In the second attribute value calculation process, the array array y[T] is assigned values from the last element y[T-1] to the first element y[0].

Succeedingly, in a step S55, it is determined whether or not the variable t is larger than 0. If “NO” in the step S55, the second attribute value calculation process is ended and the process returns to the attribute normalization process shown in FIG.

On the other hand, if "YES" in the step S55, the variable t is decremented by 1 (t=t-1) in a step S57, and it is determined in a step S59 whether or not the accent phrase in the variable t is the end. If "NO" in the step S59, that is, if it is not the end of the accent phrase in the variable t, the process proceeds to a step S63.

On the other hand, if “YES” in the step S59, that is, if the accent phrase in the variable t is the end, in a step S61, the element x[t] is substituted for the variable i (i=x[t]), and further, In step S63, the variable i is assigned to the element y[t] (y[t]=i), and the process returns to step S55.

According to this embodiment, since the linguistic feature quantity is normalized by taking the ratio of the linguistically related attributes using only the value within one utterance, it is possible to prevent the occurrence of outliers. it can. Therefore, the prediction accuracy of the acoustic feature amount is good.

Further, according to this embodiment, since the outlier does not occur, it is not necessary to increase the learning data in order to prevent the outlier from occurring. That is, according to this embodiment, even with a small amount of learning data, the acoustic feature quantity prediction accuracy is good.

In this embodiment, the input value of the DNN is set by the condition that the absolute value of the first attribute value (L _td ) is less than or equal to the absolute value of the second attribute value (L _tδ ). Although the language feature amount (each attribute of the language feature amount) is normalized so as to fall within the range (or range) from 0 to 1, the present invention is not limited to this. For example, on condition that the absolute value of the first attribute value is larger than the absolute value of the second attribute value, a predetermined constant is added to the first attribute value (L _td ) and the second attribute value (L _tδ ). The scale may be changed by multiplying the normalized value to a value exceeding the range of 0 to 1. In this case, since the scale of each attribute value only changes, the same effect as before changing the scale can be obtained. In addition, depending on the range of values after normalization, the condition may be that the absolute value of the second attribute value is larger than the absolute value of the first attribute value (|L _td |<|L _tδ |).

It should be noted that the specific numerical values shown in the above-mentioned embodiments are merely examples and should not be limited, and can be appropriately changed according to the product to be implemented.

10... Speech synthesizer 12, 16... Storage unit

Claims (12)

A language processing device that is input to a deep neural network of a speech synthesis device that generates synthetic speech and normalizes a language feature vector sequence composed of a plurality of different attributes,
A language processing device, comprising: a normalization unit that normalizes a first attribute in the language feature vector sequence for one utterance with a second attribute different from the first attribute. The language processing apparatus according to claim 1, wherein the first attribute and the second attribute are values that are linguistically related. The language processing device according to claim 1, wherein the normalizing unit normalizes the first attribute by dividing the first attribute by the second attribute. The language processing device according to claim 1, wherein an absolute value of the first attribute is equal to or less than an absolute value of the second attribute. A language processing program that is input to a deep neural network of a speech synthesizer that generates synthetic speech, and that is executed by a language processing apparatus that normalizes a language feature vector sequence composed of a plurality of different attributes,
Language processing for causing a processor of the language processing device to execute a normalization step of normalizing a first attribute in the language feature vector sequence for one utterance with a second attribute different from the first attribute program. The language processing program according to claim 3, wherein the first attribute and the second attribute are linguistically related values. 7. The language processing program according to claim 5, wherein the normalizing unit normalizes by dividing the first attribute by the second attribute. 8. The language processing program according to claim 5, wherein the absolute value of the first attribute is equal to or less than the absolute value of the second attribute. A language processing method for normalizing a language feature vector sequence, which is input to a deep neural network of a speech synthesizer for generating synthetic speech, and which comprises a plurality of different attributes,
A language processing method for normalizing a first attribute in the language feature vector sequence for one utterance with a second attribute different from the first attribute. The language processing method according to claim 9, wherein the first attribute and the second attribute are linguistically related values. The language processing method according to claim 9, wherein normalization is performed by dividing the first attribute by the second attribute. The language processing program according to claim 9, wherein the absolute value of the first attribute is equal to or less than the absolute value of the second attribute.

Images