JP6552146B1 - Audio processing apparatus and audio processing method - Google Patents

Abstract

A speech synthesis technique capable of synthesizing smooth and natural speech is provided. An audio processing apparatus includes an acquisition unit configured to acquire a multidimensional first feature quantity related to audio, and a first feature quantity that converts a first feature quantity into a multidimensional second feature quantity every predetermined first period. An acoustic for generating a speech waveform for each second period longer than the first period using one conversion unit and a neural network capable of processing the second feature quantity in time series A second conversion unit for converting into a feature amount. [Selection] Figure 6

Description

  The present invention relates to an audio processing device and an audio processing method.

  As a speech processing apparatus in recent years, a device that synthesizes speech using a neural network is known. In the technique described in Patent Document 1, a speech waveform is synthesized using an acoustic feature amount generated by a neural network.

JP 2018-146803 A

A. van den Oord et al. , “Wavenet: A Generative Model for Raw Audio”, arXiv preprint arXiv: 1609.03499, 2016

  However, in the technique described in Patent Document 1, since the acoustic feature amount is generated independently or sequentially on the time axis, the time structure of speech cannot be expressed sufficiently, and mechanical and unnatural speech is generated. May be generated. Therefore, a speech synthesis technology that can appropriately process the temporal structure of speech and is capable of synthesizing smooth and natural speech has been desired.

  The present invention has been made to solve the above-described problems, and can be realized as the following modes.

(1) According to an aspect of the present invention, an audio processing device is provided. The speech processing apparatus includes an acquisition unit for acquiring a multi-dimensional first feature quantity related to speech, and a first conversion for converting the first feature quantity into a multi-dimensional second feature quantity every predetermined first period. An audio waveform for each second period longer than the first period, using a neural network capable of processing the second feature amount in time series, and the second feature amount. A second conversion unit for converting into an acoustic feature amount. According to the voice processing apparatus of this aspect, since the first feature amount is converted into the acoustic feature amount every long period, when the voice is synthesized using this acoustic feature amount, smooth and natural voice can be synthesized.
(2) In the speech processing apparatus of the above aspect, the second conversion unit may convert the second feature into the acoustic feature using a convolutional neural network as the neural network. According to the speech processing apparatus of this aspect, the second feature value can be converted into the acoustic feature value with high quality by using the existing technology.
(3) In the speech processing device of the above aspect, the second period may be variable length. According to the speech processing apparatus of this aspect, it can be converted into an acoustic feature amount having an arbitrary length.
(4) In the speech processing device according to the above aspect, the second conversion unit may change the length of the second period according to a silent portion in the first feature amount. According to the speech processing apparatus of this aspect, for example, when a singing voice is synthesized, synthesis can be performed for each phrase.
(5) In the speech processing device according to the above aspect, the first conversion unit may convert the first feature value into the second feature value using a feedforward neural network. According to the voice processing apparatus of this aspect, it is possible to convert the first feature amount into the second feature amount at high speed.
(6) In the speech processing apparatus of the above aspect, the second conversion unit inputs a specific parameter included in the first feature amount to the neural network in addition to the second feature amount to the acoustic feature amount. Conversion to may be performed. According to the speech processing device of this aspect, since the specific parameter included in the first feature amount is added as the auxiliary information to the second feature amount, it can be converted into an acoustic feature amount that improves the accuracy of the synthesized speech.
(7) In the voice processing device of the above aspect, the parameter may include pitch information. According to the voice processing apparatus of this aspect, it is possible to convert into the acoustic feature quantity that improves the sound quality of the synthesized voice.
(8) In the speech processing apparatus of the above aspect, the pitch information of the silent portion in the first feature amount may be information interpolated by preceding and following pitch information. According to the speech processing apparatus of this aspect, it can be converted into an acoustic feature quantity that further improves the quality of the synthesized speech.
(9) In the speech processing device according to the above aspect, the first feature amount may include at least one of a language feature amount, a score feature amount, and a voice quality feature amount. According to the speech processing apparatus of this embodiment, for example, the first feature amount can be converted into an acoustic feature amount for performing text speech synthesis, singing voice synthesis, and voice quality conversion.
(10) The speech processing apparatus according to the above aspect may further include a vocoder unit that generates a speech waveform using the acoustic feature amount. According to the speech processing apparatus of this aspect, synthesized speech can be generated using acoustic feature amounts.
(11) The speech processing apparatus according to the above aspect may further include a learning unit configured to learn the relationship between the first feature amount and the acoustic feature amount by supervised machine learning and reflect the result on the neural network. According to the speech processing device of this aspect, the relationship between the first feature value and the acoustic feature value can be learned, and the learning result can be reflected in the second conversion unit. When the first conversion unit performs conversion using a neural network, the learning result can be reflected in the first conversion unit.
(12) In the speech processing apparatus according to the above aspect, the second conversion unit may be configured to display the second feature quantity in a two-dimensional manner by arranging data of each dimension of the second feature quantity by the length of the second period. A speech processing apparatus that converts the acoustic feature value into data using the data. According to this type of speech processing apparatus, it is possible to effectively handle changes in the time direction.
(13) An audio processing apparatus, which is an acquisition unit for acquiring multi-dimensional feature quantities relating to speech, and acoustic features for generating speech waveforms using a convolutional neural network for each predetermined period of the feature quantities. A conversion unit configured to convert into an amount, wherein the conversion unit uses the feature amount as two-dimensional data represented by arranging data of each dimension of the feature amount by the length of the period, the acoustic feature amount Convert to According to the speech processing apparatus of this aspect, it is possible to effectively handle changes in the time direction, and since speech is synthesized using this acoustic feature amount because it is converted into an acoustic feature amount for each long period, it is smooth and natural It can synthesize voice.

  Note that the present invention can be realized in various modes. For example, a speech processing system using the speech processing apparatus of this embodiment, a method executed by the information processing apparatus to realize the functions of the speech synthesis apparatus and the speech synthesis system, a computer program, and a server for distributing the computer program The present invention can be realized in the form of a device, a non-temporary storage medium storing the computer program, and the like.

It is explanatory drawing which shows the outline | summary of the audio | voice processing apparatus in one Embodiment of this invention. It is a figure which shows an example of the various parameters in a 1st feature value. It is a figure which shows an example of the various parameters in an acoustic feature-value. It is explanatory drawing for demonstrating the machine learning by a deep neural network. It is a flowchart showing a speech synthesis process. It is explanatory drawing which represented the speech synthesis process typically. It is the figure which showed the experimental result of the subjective evaluation experiment. It is explanatory drawing of CNN in 2nd Embodiment. It is explanatory drawing which shows an example at the time of interpolating pitch information.

A. First embodiment:
FIG. 1 is an explanatory diagram showing an overview of a speech processing apparatus 100 according to an embodiment of the present invention. The speech processing apparatus 100 includes an acquisition unit 10, a first conversion unit 20, a second conversion unit 30, a vocoder unit 40, a learning unit 50, and an acoustic model 60. The acquisition unit 10, the first conversion unit 20, the second conversion unit 30, the vocoder unit 40, and the learning unit 50 are software-like as one or more CPUs execute programs stored in the memory. To be realized. Note that some or all of these may be implemented in hardware by a circuit.

  The acquisition unit 10 acquires a multidimensional first feature quantity related to speech. Details of the first feature amount will be described later. For example, the acquisition unit 10 may extract the first feature amount from a pre-recorded speech waveform using a well-known speech recognition technique, and the first feature amount may be generated in advance according to the text or musical score to be spoken. One feature amount may be acquired.

  The first conversion unit 20 converts the first feature quantity acquired by the acquisition unit 10 into a multidimensional second feature quantity every predetermined first period. The second feature amount is data that can be easily handled in the conversion to the acoustic feature amount by the second conversion unit 30. In the present embodiment, the first conversion unit 20 converts the first feature amount into a multi-dimensional second feature amount using a feedforward neural network (FFNN). The first conversion unit 20 is not limited to the FFNN, and may use a recurrent neural network (Recurrent Neural Network (RNN)) having a recurrent structure such as a long short-term memory network (LSTM). Alternatively, a Hidden Markov Model (HMM) may be used. Moreover, you may use combining these.

  The second conversion unit 30 uses the neural network capable of processing the second feature amount converted by the first conversion unit 20 in a time series manner, and the second conversion unit 30 is longer than the first period. It converts into the acoustic feature-value for producing | generating an audio | voice waveform for every period. Details of the acoustic feature amount will be described later. The second period may be variable length. In the case where the second period has a variable length, it is preferable to change and set the second period in accordance with the silent portion in the first feature amount. Thereby, an acoustic feature amount can be generated for each phrase. In the present embodiment, the second conversion unit 30 converts the second feature into an acoustic feature using a convolutional neural network (CNN). When the second period is variable or the like, the second conversion unit 30 uses a Fully recurrent network (FRN) or a Fully Convolutional network (FCN). Moreover, the 2nd conversion part 30 may use not only CNN but RNN.

  The vocoder unit 40 generates a speech waveform from the acoustic feature amount converted by the second conversion unit 30. As the vocoder unit 40, for example, a conventional vocoder technique may be used, or a vocoder technique using a neural network such as wavenet (described in Non-Patent Document 1) may be used. The voice processing apparatus 100 may not include the vocoder unit 40. In that case, the voice waveform is generated by an external voice synthesizer.

  The learning unit 50 learns the relationship between the first feature value and the acoustic feature value by supervised machine learning. The learning unit 50 reflects the learning result on a statistical model such as a neural network or a Hidden Markov Model (HMM) of the first conversion unit 20 or a neural network used in the second conversion unit 30. By doing this, the first conversion unit 20 and the second conversion unit 30 can perform conversion reflecting the learning result of the learning unit 50. The speech processing apparatus 100 may not include the learning unit 50. In this case, the first conversion unit 20 and the second conversion unit 30 can perform conversion by reflecting learning results obtained by a learning device or the like that performs external machine learning. When the first conversion unit 20 uses a neural network, more accurate learning can be performed by performing simultaneous learning in connection with the neural network used by the second conversion unit 30. Alternatively, the learning result may be alternately reflected on any one of the neural network used by the first conversion unit 20 and the neural network used by the second conversion unit 30.

  FIG. 2 is a diagram illustrating an example of multi-dimensional parameters included in the first feature amount acquired by the acquisition unit 10 in singing voice synthesis. In the present embodiment, the first feature amount is a score feature amount. The score information includes song information, phrase information, and note information. The note information includes, for example, information such as the length and pitch of the note, and the position of the note in the phrase. The linguistic information includes syllable information and phoneme information. The syllable information includes information such as the number of phonemes and the position of a syllable in a note. The phoneme information includes, for example, information such as the type (for example, vowel, voiced consonant, unvoiced consonant, etc.) and the position of phoneme in a syllable. The continuation length information includes in-phoneme position information and in-state position information. The in-phoneme position information includes, for example, information such as the length and ratio from the start position of the phoneme. The in-state position information includes, for example, information such as length and ratio from the start position of the state.

  FIG. 3 is a diagram illustrating an example of various parameters in the acoustic feature amount output by the second conversion unit 30. Examples of the spectrum parameter include a mel cepstrum and a line spectrum pair (Line Spectrum Pair (LSP)). These are sometimes referred to as spectral information. As the sound source information, the fundamental frequency is generally treated as a logarithmic fundamental frequency, and the related parameter may be a voiced / unvoiced distinction or an aperiodicity index. These are sometimes called sound source information. Since the unvoiced portion does not have a logarithmic fundamental frequency value, instead of including the voiced / unvoiced distinction in the sound source information, the voiced / unvoiced distinction may be performed by a method such as inserting a predetermined constant in the unvoiced portion. . Furthermore, in addition to such sound source information and spectrum information, in the present embodiment, singing expression information is included in the acoustic feature amount.

  The song expression information includes the period and amplitude of the pitch vibrato and its presence and absence, and the period and amplitude of the sound magnitude vibrato and its presence and absence. Instead of including the presence / absence of the pitch vibrato in the singing expression information, the presence / absence of the pitch vibrato may be determined by a method such as inserting a predetermined constant in a portion where there is no pitch vibrato. Similarly, instead of including the distinction of the presence or absence of the loudness of the sound in the singing expression information, the presence or absence of the loudness or the presence of the loudness of the sound may be determined by inserting a predetermined constant in a portion where the loudness of the loudness is not present. A distinction may be made.

  FIG. 4 is an explanatory diagram for explaining the conversion of the first feature amount by the deep neural network. The deep neural network 200 is a network that models a learning mechanism in a human brain nervous system. The deep neural network 200 includes an input layer L1, a plurality of intermediate layers L2, and an output layer L3. The number of intermediate layers L2 can be determined arbitrarily.

  The input layer L1 is a layer to which information is input. The intermediate layer L2 is a layer that calculates feature amounts based on information transmitted from the input layer L1. The output layer L3 is a layer that outputs a result based on information transmitted from the intermediate layer L2. Each layer includes a plurality of nodes.

  The conversion by the deep neural network 200 will be described. In the present embodiment, the first conversion unit 20 converts the first feature value into the second feature value using the deep neural network 200. When a plurality of parameters included in the first feature shown in FIG. 2 are input, the input layer L1 transmits those parameters to the intermediate layer L2. In the intermediate layer L2, various operations are performed stepwise in each layer on the parameters transmitted from the input layer L1. In the output layer L3, the finally calculated parameter is output as the second feature amount shown in FIG.

  FIG. 5 is a flowchart showing a voice synthesis process using the voice processing apparatus 100 according to this embodiment. First, the acquisition unit 10 acquires a first feature amount in step S100. Next, in Step S110, the first conversion unit 20 converts the first feature amount acquired in Step S100 into a second feature amount. Subsequently, in step S120, the second conversion unit 30 converts the second feature amount converted in step S110 into an acoustic feature amount. Finally, in step S130, the vocoder unit 40 generates an audio waveform using the acoustic feature quantity converted in step S120.

  FIG. 6 is an explanatory diagram schematically showing the speech synthesis process shown in FIG. As shown in FIG. 6, in step S110, the first conversion unit 20 converts the first feature amount into the second feature amount using FFNN, and in step S120, the second conversion unit 30 converts the first feature amount using CNN. Two feature quantities are converted to acoustic feature quantities. In the present embodiment, the first period of the first feature value converted by the first conversion unit 20 is, for example, 5 milliseconds. Further, the second period of the second feature value converted by the second conversion unit 30 is, for example, 10 seconds. That is, the second conversion unit 30 bundles 2000 second feature values and performs conversion using the CNN. The second conversion unit 30 converts the second feature amount into an acoustic feature amount by using the second feature amount as two-dimensional data D1 expressed by arranging data of each dimension of the second feature amount by the length of the second period. . In the present embodiment, the two-dimensional data D1 is data in which 2000 second feature values are arranged in time series. That is, the data is represented by [data of each dimension of the second feature value] × [time]. The second feature amount is not limited to the two-dimensional data D1, and may be represented as multidimensional data of three or more dimensions. The concept of the size of input data in CNN is three-dimensional in height, width, and number of channels (number of filters), since it is based on image processing. In this embodiment, the height is 1, the width is the length of the second period, and the number of channels is the number of dimensions of the second feature amount. The CNN includes a portion that convolves the second feature amount by convolution. In addition, a part that reduces the size of the column of the two-dimensional data D1 by convolution, a part that increases to return to the original second period number by a deconvolution (translated convolution) or a transposed convolution (transposed convolution), You may have.

  According to the speech processing apparatus 100 of the present embodiment described above, the acoustic feature is generated every second period longer than the first predetermined period which is a unit of time in which the first feature amount such as the score feature amount is represented. Since the sound is synthesized using this acoustic feature amount in order to convert it into a quantity, a smooth and natural voice can be synthesized. In addition, the second conversion unit 30 uses the second feature amount as two-dimensional data D1 represented by arranging the data of each dimension of the second feature amount by the number of first periods included in the second period as an acoustic feature amount. Therefore, changes in the time direction can be effectively handled. More specifically, for example, as compared with the case where the second feature value is used as one-dimensional data represented by arranging the data of each dimension for the second period, the change in the data of each dimension in the time direction is more Can learn effectively. In addition, since the second conversion unit 30 performs conversion using CNN, it is possible to convert the second feature amount into an acoustic feature amount with high quality using the existing technology.

  Further, in the present embodiment, since the first conversion unit 20 converts the first feature amount into the second feature amount using FFNN, the conversion can be performed at high speed.

Experimental result:
FIG. 7 is a diagram showing an average opinion score (MOS), which is an experimental result of a subjective evaluation experiment on the generated speech waveform. In this experiment, the quality of synthetic speech of the four methods was evaluated by a five-step subjective evaluation experiment of “1: very bad, 2: bad, 3: normal, 4: good, 5: very good”. There were 15 test subjects, and each test subject evaluated 10 phrases for each method from 5 songs as test data. The speech waveform of synthetic speech to be evaluated was generated using the same first feature amount in all four methods.

  Example 1 and Example 2 are acoustic feature values obtained by converting the first feature value by the speech processing apparatus 100 according to the first embodiment described above. More specifically, the second conversion unit 30 uses the CNN to obtain the second feature value. A speech waveform was generated using the acoustic feature value obtained by converting the quantity. In Comparative Example 1 and Comparative Example 2, the sound waveform was generated using the acoustic feature quantity obtained by converting the second feature quantity by the second conversion unit 30 using FFNN. In addition, in Example 1 and Comparative Example 1, a speech waveform is generated from an acoustic feature using an MLSA filter that is a conventional vocoder technique, and in Example 2 and Comparative Example 2, speech is generated from an acoustic feature using wavenet. A waveform was generated. As shown in FIG. 7, the score of Example 1 and Example 2 in which the second conversion unit 30 uses CNN is higher than the score of Comparative Example 1 and Comparative Example 2 in which the second conversion unit 30 uses FFNN. It was. That is, when the second conversion unit 30 performs conversion using CNN according to the above embodiment, it is possible to convert the second feature amount into an acoustic feature amount with higher quality.

B. Second embodiment:
FIG. 8 is an explanatory diagram of the CNN in the second embodiment. The CNN of the second embodiment is different from the first embodiment in that, as indicated by hatching in FIG. 8, in addition to the second feature amount, a specific parameter included in the first feature amount is input to the input layer and used. . Since the configuration of the speech processing apparatus 100 of the second embodiment is the same as the configuration of the speech processing apparatus 100 of the first embodiment, description of the configuration is omitted.

  In the present embodiment, the specific parameter included in the first feature amount is pitch information. “Pitch information” is information on the logarithmic fundamental frequency of the pitch in the score information. In the pitch information, it is preferable that silent parts in the first feature amount be interpolated by the pitch information of the first feature amount before and after on the time axis. Other examples of parameters include MIDI pitch numbers and phoneme information.

  FIG. 9 is an explanatory diagram showing an example when pitch information is interpolated. In the pitch information shown in FIG. 9, the vertical axis indicates the logarithmic fundamental frequency, and the horizontal axis indicates time. In FIG. 9, the pitch information of the first feature value in the n-th note n (n is an integer of 2 or more), which is a silent part, is the pitch information P0 of the first feature value in the note n−1 and the note n + 1. The pitch information P1 is linearly interpolated using the pitch information P2 of the first feature amount. The interpolation of the pitch information is not limited to linear interpolation, and other interpolation methods such as spline interpolation and Lagrange interpolation may be applied.

  According to the speech processing apparatus 100 of the present embodiment described above, since the pitch information which is a parameter included in the first feature amount is added as the auxiliary information to the second feature amount, an acoustic feature that improves the sound quality of the synthesized voice Can be converted into a quantity. Note that pitch information may be input to the intermediate layer instead of the input layer.

C. Other embodiments:
In the said embodiment, the 1st feature-value which the acquisition part 10 acquires is a score feature-value. Instead, the acquisition unit 10 may acquire a language feature amount as the first feature amount. The language feature quantity is a multidimensional parameter in which the score information is omitted from the score feature quantity shown in FIG. 2 and information such as part of speech and accent is added. According to this aspect, it is possible to generate an acoustic feature amount for performing simple text synthesized speech that is not a singing voice. The acquisition unit 10 may acquire a voice quality feature amount as the first feature amount. The voice quality feature amount is an acoustic feature amount extracted from another person's voice. According to this aspect, it is possible to generate an acoustic feature amount for performing voice quality conversion for converting an acoustic feature amount of a certain speaker into an acoustic feature amount of another speaker.

  Further, in the above embodiment, the audio processing device 100 converts the first feature amount into an acoustic feature amount by the conversion by the first conversion unit 20 and the conversion by the second conversion unit 30. Instead, the second conversion unit 30 may directly convert the first feature value into the acoustic feature value. In this case, the second conversion unit 30 performs conversion using the first feature value as two-dimensional data represented by arranging data of each dimension of the first feature value by the length of a predetermined period by CNN. .

  In the above embodiment, the first conversion unit 20 converts the first feature amount into the second feature amount using FFNN. The first conversion unit 20 may perform dropout in the FFNN, which does not transmit information of randomly or arbitrarily selected nodes in the middle layer L2. Thereby, the robustness in FFNN can be improved.

  In the above-described embodiment, the first conversion unit 20 and the second conversion unit 30 have a skip structure in which a route transmitted to the next layer without converting a parameter input to an arbitrary layer is added in the neural network. There may be. Thereby, it can propagate, without impairing the information of arbitrary parameters. For example, the pitch information P0 in the first feature value is skipped in the FFNN of the first conversion unit 20, and the second feature value including the pitch information P0 that has not been converted is converted into the acoustic feature value in the CNN of the second conversion unit 30. May be converted to Further, by adding a skip structure to the neural network of the second conversion unit 30, even if the number of intermediate layers is increased, the information can be propagated without losing information of any input parameters (for example, pitch information of a score). can do.

  Further, in the above embodiment, the learning unit 50 is a dynamic used for considering temporal fluctuation which is a first derivative or a second derivative with respect to the acoustic feature amount and the teacher data generated by the second conversion unit 30. A feature amount may be obtained and a learning result obtained by comparing these may be reflected in the neural network. Thereby, since the relationship of the acoustic feature amount in the time variation for each first period is further considered, it is possible to synthesize a smooth and natural voice. Further, in the above embodiment, the second conversion unit 30 does not generate a dynamic feature amount, but may generate a dynamic feature amount. In this case, the vocoder unit 40 can generate a voice waveform from the static feature amount and the dynamic feature amount included in the acoustic feature amount by performing parameter generation in consideration of these relationships. As a result, the vocoder unit 40 can correct the static feature amount in consideration of the dynamic feature amount generated by the second conversion unit 30, so that smoother, natural speech can be synthesized. In addition, the learning unit 50 can learn the relationship between the static feature amount and the dynamic feature amount in the acoustic feature amount by supervised machine learning, including the dynamic feature amount generated by the second conversion unit 30. .

  The present invention is not limited to the above-described embodiment, and can be realized in various configurations without departing from the scope of the invention. For example, the technical features in the embodiments corresponding to the technical features in the respective forms described in the section of the summary of the invention are for solving the problems described above or for achieving some or all of the effects described above It is possible to replace or combine as appropriate. Further, if the technical feature is not described as essential in the present specification, it can be deleted as appropriate.

DESCRIPTION OF SYMBOLS 10 ... Acquisition part, 20 ... 1st conversion part, 30 ... 2nd conversion part, 40 ... Vocoder part, 50 ... Learning part, 60 ... Acoustic model, 100 ... Speech processing apparatus, 200 ... Deep neural network, D1 ... Two-dimensional Data, L1 ... input layer, L2 ... middle layer, L3 ... output layer

Claims (12)

A voice processing device,
An acquisition unit for acquiring a first multi-dimensional feature amount related to speech;
A first conversion unit that converts the first feature value into a multidimensional second feature value for each predetermined first period;
An acoustic feature for generating a speech waveform for each second period longer than the first period using a convolutional neural network capable of processing the second feature amount in time series. And a second conversion unit for converting into a quantity. The speech processing apparatus according to claim 1 ,
The speech processing apparatus, wherein the second period is variable length. The speech processing apparatus according to claim 2 ,
The second processing unit is an audio processing device that changes a length of the second period according to a silent portion in the first feature amount. The speech processing apparatus according to any one of claims 1 to 3 , wherein
The first conversion unit is a voice processing device that converts the first feature value into the second feature value using a feedforward neural network. The speech processing apparatus according to any one of claims 1 to 4 , wherein
The voice processing device according to claim 1, wherein the second conversion unit inputs pitch information included in the first feature amount into the convolutional neural network in addition to the second feature amount and converts the pitch information into the acoustic feature amount. The speech processing apparatus according to claim 5 ,
The speech processing apparatus, wherein the pitch information of the silent portion in the first feature amount is information interpolated by preceding and following pitch information. The speech processing apparatus according to any one of claims 1 to 6 , wherein
The speech processing apparatus, wherein the first feature amount includes at least one of a language feature amount, a score feature amount, and a voice quality feature amount. The speech processing apparatus according to any one of claims 1 to 7 , further comprising:
An audio processing apparatus comprising a vocoder that generates an audio waveform using the acoustic feature amount. The speech processing apparatus according to any one of claims 1 to 8 , further comprising:
A speech processing apparatus comprising: a learning unit that learns a relationship between the first feature value and the acoustic feature value by supervised machine learning and reflects the result in the convolutional neural network. The speech processing apparatus according to any one of claims 1 to 9 ,
The second conversion unit converts the second feature amount into the acoustic feature amount by using data of each dimension of the second feature amount as two-dimensional data represented by arranging the length of the second period. , Voice processing device. An audio processing method,
An acquisition step of acquiring a first multi-dimensional feature quantity related to speech;
A first conversion step of converting the first feature quantity into a multidimensional second feature quantity for each predetermined first period;
An acoustic feature amount for generating an audio waveform for each second period longer than the first period using a convolutional neural network capable of processing the second feature amount in time series. A voice conversion method comprising: a second conversion step of converting to a second conversion step. A voice processing device,
An acquisition unit for acquiring multi-dimensional feature values related to speech;
A conversion unit that converts the feature amount into an acoustic feature amount for generating a speech waveform using a convolutional neural network for each predetermined period; and
The speech processing device, wherein the conversion unit converts the feature quantity into the acoustic feature quantity using two-dimensional data represented by arranging each dimension data of the feature quantity for the length of the period.

Images