WO2020173134A1 - 一种基于注意力机制的语音合成方法及装置 - Google Patents

一种基于注意力机制的语音合成方法及装置 Download PDF

Info

Publication number
WO2020173134A1
WO2020173134A1 PCT/CN2019/117785 CN2019117785W WO2020173134A1 WO 2020173134 A1 WO2020173134 A1 WO 2020173134A1 CN 2019117785 W CN2019117785 W CN 2019117785W WO 2020173134 A1 WO2020173134 A1 WO 2020173134A1
Authority
WO
WIPO (PCT)
Prior art keywords
audio
text
matrix
target text
target
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2019/117785
Other languages
English (en)
French (fr)
Inventor
房树明
程宁
王健宗
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Ping An Technology Shenzhen Co Ltd
Original Assignee
Ping An Technology Shenzhen Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Ping An Technology Shenzhen Co Ltd filed Critical Ping An Technology Shenzhen Co Ltd
Publication of WO2020173134A1 publication Critical patent/WO2020173134A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01L—MEASURING FORCE, STRESS, TORQUE, WORK, MECHANICAL POWER, MECHANICAL EFFICIENCY, OR FLUID PRESSURE
    • G01L13/00—Devices or apparatus for measuring differences of two or more fluid pressure values
    • G01L13/02—Devices or apparatus for measuring differences of two or more fluid pressure values using elastically-deformable members or pistons as sensing elements
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L13/00—Speech synthesis; Text to speech systems
    • G10L13/02—Methods for producing synthetic speech; Speech synthesisers
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
    • G10L19/008—Multichannel audio signal coding or decoding using interchannel correlation to reduce redundancy, e.g. joint-stereo, intensity-coding or matrixing
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D30/00—Reducing energy consumption in communication networks
    • Y02D30/70—Reducing energy consumption in communication networks in wireless communication networks

Definitions

  • This application relates to the technical field of speech synthesis, and in particular to a speech synthesis method and device based on an attention mechanism.
  • Speech synthesis is a technology that generates artificial speech through mechanical and electronic methods. It is also called Text to Speech (TTS). TTS technology belongs to speech synthesis. It is the text generated by the computer or input from outside. Information is transformed into a technology that can be understood and spoken Chinese fluently.
  • the original algorithm in the existing attention-based speech synthesis technology can make the attention mechanism have the property of forcibly aligning the text and speech signals with obvious position deviations, but the complexity is too high, and usually requires a lot of sample data and training time. In order to achieve the desired effect, how to reduce the difficulty of the algorithm while ensuring the forced alignment of text and voice signals with obvious position deviations is a problem to be solved at present.
  • the present application is proposed in order to provide an attention mechanism-based speech synthesis method and device that overcomes or at least partially solves the above-mentioned problems.
  • the embodiment of the present application provides a speech synthesis method based on an attention mechanism, which may include: determining a text coding matrix and an audio coding matrix according to the target text;
  • the audio decoding matrix is determined by the function L(A), where the function L(A) is the attention mechanism loss function determined according to Ant and W nt , where Ant is used to
  • the text encoding matrix is converted. If the alignment strength of the target text is less than the alignment strength threshold, W nt changes linearly; the alignment strength of the target text is determined by the position of the nth character in the target text and the position of the nth character in the target text.
  • the time point t of the pronunciation of n characters is determined, n is greater than 0 and less than or equal to the number of characters in the target text, and t is greater than 0 and less than or equal to the time point of the total pronunciation of the target text;
  • An embodiment of the present application provides a speech synthesis device based on an attention mechanism, which may include: a first determining unit configured to determine a text coding matrix and an audio coding matrix according to a target text;
  • the second determining unit is used to determine the audio decoding matrix through the function L(A) according to the text encoding matrix and the audio encoding matrix, where the function L(A) is the attention mechanism loss function determined according to Ant and W nt , where , Ant is used to transform the text encoding matrix. If the alignment strength of the target text is less than the alignment strength threshold, W nt changes linearly; the alignment strength of the target text is determined by the position of the nth character in the target text Determined with the time point t of the pronunciation of the nth text in the target text, n is greater than 0 and less than or equal to the number of characters in the target text, and t is greater than 0 and less than or equal to the time point of the total pronunciation of the target text;
  • the third determining unit is used to determine the Mel cepstrum coefficient according to the audio decoding matrix, and determine the target audio according to the Mel cepstrum coefficient.
  • the application embodiment provides a computer-readable storage medium that stores program instructions, and when the program instructions are executed by a processor, the processor executes any of the above-mentioned attention-based speech synthesis method.
  • the embodiment of the application provides a speech synthesis device based on an attention mechanism, including a storage component, a processing component and a communication component, a storage component, and the processing component and the communication component are connected to each other.
  • the storage component is used to store data processing code and communicate.
  • the component is used for information interaction with external devices; the processing component is configured to call program code and execute any speech synthesis method based on the attention mechanism, which will not be repeated here.
  • the embodiment of the application provides a speech synthesis method based on the attention mechanism.
  • the audio decoding matrix is determined by the function L(A) according to the text encoding matrix and the audio encoding matrix of the target text
  • the audio decoding matrix can be further determined according to the audio decoding matrix.
  • Mel cepstrum coefficient and determine the target audio frequency according to Mel cepstrum coefficient.
  • the function L(A) is the attention mechanism loss function determined according to Ant and Wnt, where Ant is used to transform the text encoding matrix.
  • the alignment strength of the target text is less than the alignment strength threshold, Wnt changes linearly ; Further, the alignment strength of the target text is determined by the position of the nth character in the target text and the time point t of the pronunciation of the nth character in the target text. If the alignment strength of the target text is less than the alignment strength threshold, making Wnt change linearly will not only greatly reduce the difficulty of the algorithm in the original attention mechanism, but also can not require a lot of sample data and training time. Ensuring that the text and speech signals with obvious position deviations are forced to align, which helps to achieve the purpose of approximate alignment of the attention mechanism matrix faster, so that the speech synthesis is more organized.
  • FIG. 1 is a schematic diagram of a speech synthesis system architecture based on an attention mechanism provided by an embodiment of the present application
  • FIG. 2 is a schematic diagram of a terminal interface when synthesizing speech provided by an embodiment of the present application
  • 3A is a schematic diagram of the process of a speech synthesis method based on an attention mechanism provided by an embodiment of the present application
  • FIG. 3B is a schematic diagram of the framework of a speech synthesis technology method based on an improved attention mechanism based on an embodiment of the present application;
  • FIG. 4 is a schematic diagram of another method for speech synthesis based on attention mechanism provided by an embodiment of the present application.
  • FIG. 5 is a schematic structural diagram of a speech synthesis device based on an attention mechanism provided by an embodiment of the present application
  • FIG. 6 is a schematic diagram of a physical device structure of a simplified speech synthesis device based on an attention mechanism provided by an embodiment of the present application.
  • server used in this application are used to denote computer-related entities, hardware, firmware, a combination of hardware and software, software, or software in execution.
  • the server may be, but is not limited to, a processor, a data processing platform, a computing device, a computer, two or more computers, etc.
  • Speech synthesis is the input of a text to output a speech corresponding to the text. It is also a technology that generates artificial speech through mechanical and electronic methods.
  • TTS technology also known as text-to-speech technology
  • Attention Mechanism is derived from the study of human vision. In cognitive science, due to the bottleneck of information processing, humans will selectively focus on a part of all information while ignoring other visible information. The above mechanism is usually referred to as the attention mechanism. Different parts of the human retina have different degrees of information processing capabilities, namely acuity, and only the fovea has the strongest acuity. In order to make rational use of the limited visual information processing resources, humans need to select a specific part of the visual area and then focus on it. For example, when people are reading, usually only a few words to be read will be paid attention to and processed. In summary, the attention mechanism mainly has two aspects: decide which part of the input needs to be paid attention to; and allocate limited information processing resources to important parts.
  • Short-term Fourier transform is a variant of Fourier transform, used to determine the sinusoidal frequency and phase of the local part of the signal that changes with time.
  • STFT short-time Fourier transform
  • FIG. 1 is a schematic diagram of an attention mechanism-based speech synthesis system architecture provided by an embodiment of the present application, including: an attention mechanism-based speech synthesis device 101 and a terminal device 102.
  • the speech synthesis apparatus 101 based on the attention mechanism may be a server, where the server may be, but is not limited to, a processor, a data processing platform, a computing device, a computer, two or more computers, etc.
  • the server is a way of acquiring, processing, analyzing, and extracting valuable, massive and diversified data, based on interactive data, and bringing various benefits to third parties. A convenient service equipment.
  • the speech synthesis device 101 based on the attention mechanism can determine the text coding matrix and the audio coding matrix according to the target text; according to the text coding matrix and the audio coding matrix, the audio decoding matrix is determined by the function L(A), where the function L(A) is The attention mechanism loss function determined according to Ant and W nt , where Ant is used to transform the text encoding matrix.
  • W nt changes linearly;
  • the alignment strength is determined by the position of the nth character in the target text and the time point t when the nth character in the target text is pronounced, n is greater than 0 and less than or equal to the number of characters in the target text, and t is greater than 0 and less than or It is equal to the time point of the total pronunciation of the target text;
  • the Mel cepstrum coefficient is determined according to the audio decoding matrix, and the target audio is determined according to the Mel cepstrum coefficient.
  • the terminal device 102 may be a computer network such as a communication terminal, a portable terminal, a mobile device, a user terminal, a mobile terminal, a wireless communication device, a user agent, a user device, a service device, or a user equipment (User Equipment, UE) that is at the periphery of the network.
  • the device is mainly used for data input and processing result output or display, etc. It can also be a software client, application level, etc. installed or running on any of the above-mentioned devices.
  • the client can be a smartphone, computer, or tablet device used by the target user or the current rental user, or a software client, application level, etc. installed or running on a smartphone, computer, or tablet device. Please refer to FIG.
  • the terminal device 102 is a computer, it can be used to send target text to the attention mechanism-based speech synthesis device 101, and receive and play the target audio sent by the attention mechanism-based speech synthesis device 101.
  • the speech synthesis device 101 based on the attention mechanism can simultaneously receive different target texts sent by multiple different terminal devices 102.
  • FIG. 3A is a schematic diagram of a process of a speech synthesis method based on an attention mechanism provided by an embodiment of the present application. It can be applied to the system in FIG. 1 described above.
  • the following describes the speech synthesis device 101 based on the attention mechanism from one side of the speech synthesis device 101 based on the attention mechanism in conjunction with FIG. 3A.
  • the method may include the following steps S301-S303.
  • Step S301 Determine the text coding matrix and the audio coding matrix according to the target text.
  • the target text may be obtained, the target text includes a text of N characters; the offset audio is obtained, and the duration of the offset audio including the target text is The audio of T; then determine the text encoding matrix according to the target text; determine the audio encoding matrix according to the offset audio.
  • the target text can be the sample text "Ping An Technology Co., Ltd.” input by the user, and the word order of the input text is marked.
  • Obtaining the offset audio may be determining the offset audio by matching the audio corresponding to the target text in a speech library according to the target text. For example: the sample text "Ping An Technology Co., Ltd.” can be matched with “Ping”, “An”, “Ke”, “Technology”, “Technology”, “Technology”, “Yes”, “Limit”, Corresponding audio of the ten characters of " ⁇ ” and " ⁇ ” form offset audio.
  • the target text is obtained, and the target text includes text of N characters; the offset audio is obtained, and the offset audio includes the audio of the target text whose duration is T; and the target text is determined according to the target text.
  • Text encoding matrix; the audio encoding matrix is determined according to the offset audio, and the sequence of the four steps is not specifically limited.
  • the target text may be acquired first, the text encoding matrix may be determined according to the target text, then the offset audio may be acquired, and finally the audio encoding matrix may be determined according to the offset audio.
  • Step S302 According to the text encoding matrix and the audio encoding matrix, the audio decoding matrix is determined through the function L(A).
  • Figure 3B is a schematic diagram of the framework of an improved attention mechanism-based speech synthesis technology method provided by an embodiment of the present application, including: text encoding module, audio encoding module, attention matrix module, audio decoding module, and Short-time Fourier spectrum module.
  • the target text and the offset audio are respectively input from the text encoding module and the audio encoding module to obtain the corresponding text decoding matrix and audio decoding matrix
  • the audio decoding matrix and the text decoding matrix are forcibly aligned with the text after attention
  • the decoding matrix is input into the audio decoding module to obtain the target audio corresponding to the target text.
  • Encoder-Decoder is a very general computing framework.
  • the specific model functions used by Encoder and Decoder there is no limitation.
  • Convolutional Neural Network CNN, Recurrent Neural Network RNN, Bidirectional Long and Short-term Memory Recurrent Neural Network BiRNN, gated loop unit GRU, long short-term memory network LSTM, etc. can all be used as model functions of Encoder and Decoder.
  • the function L(A) is an attention mechanism loss function determined according to Ant and W nt , where Ant is used to transform the text encoding matrix. If the alignment strength of the target text is less than the alignment strength threshold, Then W nt changes linearly; the alignment strength of the target text is determined by the position of the nth character in the target text and the time point t when the nth character in the target text is pronounced, n is greater than 0 and less than or equal to the target The number of text characters, t is greater than 0 and less than or equal to the time point of the total pronunciation of the target text.
  • the preset function L(A) can be For example: the function L(A) is applied to the attention mechanism matrix A in the attention mechanism module described in Figure 3B, A ⁇ R N ⁇ T , and its meaning is to evaluate the correspondence between the nth character and the T time, that is The nth character, the 1st and the tth time frame S 1:F,t are related, where Ant can be Means that the attention mechanism module looks at the nth character at time t, it will look at the nth character or the n+1th character or the characters around them at the subsequent time t+1, where d is the length of the text Related preset parameters. That is, L(A) can use the LSTM standard function normalized exponential function (Softmax function) to obtain the weight of the sound feature of the nth character in the training process through the attention mechanism, and then normalize after the summation.
  • Softmax function LSTM standard function normalized exponential function
  • ⁇ is the alignment strength threshold
  • N is the total number of characters in the N characters of the target text
  • n is the number of characters in the nth word in the N characters
  • T is the The time point when the Nth character of the target text is pronounced
  • t is the time point when the nth character is pronounced.
  • W nt is a segmentation function related to the alignment strength of the target text. If the alignment strength of the target text is less than the alignment strength threshold, W nt decreases as the target text increases.
  • Step S303 Determine the Mel cepstrum coefficient according to the audio decoding matrix, and determine the target audio according to the Mel cepstrum coefficient.
  • determining the mel cepstrum coefficient according to the audio decoding matrix, and determining the target audio according to the mel cepstrum coefficient is specifically: determining the mel cepstrum coefficient according to the audio decoding matrix, and performing short-time Fu The inner transform, and then according to the short-time Fourier spectrum, determine the target audio.
  • Mel-Frequency Cepstrum is a linear transformation of the logarithmic energy spectrum based on the non-linear mel scale of sound frequency.
  • Mel-Frequency Cepstral Coefficients are the coefficients that make up the Mel-Frequency Cepstral Coefficients. It is derived from the cepstrum of audio fragments.
  • cepstrum and mel frequency cepstrum The difference between cepstrum and mel frequency cepstrum is that the band division of mel frequency cepstrum is equally spaced on the mel scale, which is more approximate than the linearly spaced frequency band used in normal cepstrum
  • the human auditory system Such a non-linear representation can provide a better representation of the sound signal in multiple fields.
  • the determination of the corresponding Mel cepstrum coefficients can be pre-emphasis, framing, and windowing of the speech of the audio decoding matrix; for each short-term analysis window, the fast discrete Fourier transform can be used.
  • the algorithm (Fast Fourier Transformation, FFT) obtains the corresponding frequency spectrum; the above frequency spectrum is passed through the Mel filter bank to obtain the Mel frequency spectrum (Mel spectrum).
  • the spectrogram is the spectrogram sound describing the speech signal.
  • the spectrogram can represent the target audio. It can be understood that the embodiment of the present invention does not specifically limit the manner of determining the target audio according to the Mel cepstrum coefficient.
  • the Mel cepstrum coefficients are further determined according to the audio decoding matrix, and the Mel cepstrum coefficients are determined according to the Mel cepstrum coefficients. Determine the target audio.
  • the function L(A) is the attention mechanism loss function determined according to Ant and W nt , where Ant is used to transform the text encoding matrix.
  • W nt changes linearly; further, the alignment strength of the target text is determined by the position of the nth character in the target text and the time point t when the nth character in the target text is pronounced. Therefore, when the alignment strength of the target text is less than the alignment strength threshold, the linear change of W nt can not only greatly reduce the difficulty of the algorithm in the original attention mechanism, but also ensure that the text and speech signals with obvious position deviations are forced to align, which is helpful In order to make the attention mechanism matrix achieve the purpose of approximate alignment faster, so that the speech synthesis is more organized.
  • FIG. 4 is a schematic diagram of another method for speech synthesis based on an attention mechanism provided by an embodiment of the present application. It can be applied to the system in FIG. 1 described above, and the following will describe from a single side of the attention mechanism-based speech synthesis device 101 in conjunction with FIG. 4.
  • the method may include the following steps S401 to S403.
  • Step S401 Perform function L(A) model training according to the sample text and sample speech, and determine the alignment intensity threshold ⁇ of the function L(A).
  • the embodiment of the present application can be applied to a speech synthesis scene based on a directed attention mechanism.
  • the positions of text and audio signal segments are roughly related. Therefore, when a person speaks a sentence, the position n of the character and the time point t have an approximate linear relationship, that is, n ⁇ at, where a ⁇ N/T.
  • ⁇ is a linearly adjustable alignment intensity threshold, which is used to indicate the preset alignment intensity between the position of the nth character and the time point t when the nth character is pronounced, and the value range of ⁇ is ⁇ ⁇ (0,1).
  • the threshold ⁇ is closer to zero, it means that the corresponding intensity of the font position and the speech time obtained by it is stronger, and the corresponding intensity of speech and text is higher.
  • Step S402 Determine the text coding matrix and the audio coding matrix according to the target text.
  • Step S403 According to the text encoding matrix and the audio encoding matrix, the audio decoding matrix is determined by the function L(A).
  • Step S404 Determine the Mel cepstrum coefficient according to the audio decoding matrix, and determine the target audio according to the Mel cepstrum coefficient.
  • step S402-step S404 may correspond to the related descriptions of step S301-step S303 in FIG. 3A, and details are not repeated here.
  • the function L(A) is the attention mechanism loss function determined according to Ant and W nt , where Ant is used to transform the text encoding matrix.
  • the alignment strength of the target text is less than the alignment strength threshold, then W nt changes linearly; further, the alignment strength of the target text is determined by the position of the nth character in the target text and the time point t when the nth character in the target text is pronounced.
  • the size of the threshold ⁇ it is determined that when the threshold ⁇ is closer to zero, the alignment strength of the target text is closer to the threshold ⁇ , it is proved that the corresponding strength of the font position and the speech time is stronger, and the corresponding strength of the speech and text is stronger. high. Therefore, the speech synthesis technology through the improved attention mechanism model can ensure the forced alignment of text and speech signals with obvious position deviations, while reducing the difficulty of the algorithm and greatly reducing the time for speech synthesis.
  • the linear change of W nt can not only greatly reduce the difficulty of the algorithm in the original attention mechanism, but also ensure that the text and speech signals with obvious position deviations are forced to align, which is helpful In order to make the attention mechanism matrix achieve the purpose of approximate alignment faster, so that the speech synthesis is more organized.
  • this application can determine the audio decoding matrix through the function L(A) according to the text coding matrix and the audio coding matrix of the target text, and then further determine the Mel cepstrum coefficients according to the audio decoding matrix, and according to the Mel cepstrum system The number determines the target audio.
  • the attention mechanism loss function L(A) changes with the change of the alignment intensity of the target text.
  • the alignment intensity threshold because W nt changes linearly, by adjusting the calculation method of the loss function W nt , the attention mechanism matrix
  • the loss function of has a linear adjustable ⁇ threshold, which makes the loss function linear. At this time, it can not only greatly reduce the difficulty of the algorithm in the original attention mechanism, but also can not require a lot of sample data and training time. , To ensure that the text and voice signals with obvious position deviations are forced to align.
  • the following provides a speech synthesis device based on the attention mechanism related to the embodiment of the application.
  • the speech synthesis device based on the attention mechanism can be a method of rapid acquisition, processing, analysis and Extract valuable data, based on interactive data, to bring various convenient service equipment for third parties.
  • FIG. 5 is a schematic structural diagram of a speech synthesis device based on an attention mechanism provided by an embodiment of the present application. It may include a first determining unit 501, a second determining unit 502, and a third determining unit 503, and may also include a fourth determining unit 504.
  • the first determining unit 501 is configured to determine a text encoding matrix and an audio encoding matrix according to the target text;
  • the second determining unit 502 is configured to determine the audio decoding matrix through the function L(A) according to the text encoding matrix and the audio encoding matrix, where the function L(A) is the attention mechanism loss function determined according to Ant and W nt , Wherein, Ant is used to transform the text encoding matrix.
  • W nt changes linearly;
  • the alignment strength of the target text is determined by the nth character in the target text
  • the position and the time point t of the pronunciation of the nth text in the target text are determined, n is greater than 0 and less than or equal to the number of characters in the target text, and t is greater than 0 and less than or equal to the time point of the total pronunciation of the target text;
  • the third determining unit 503 is configured to determine the Mel cepstrum coefficient according to the audio decoding matrix, and determine the target audio according to the Mel cepstrum coefficient.
  • the first determining unit 501 is specifically configured to: obtain a target text, the target text includes a text of N characters; obtain an offset audio, the offset audio includes the duration of the target text Is the audio of T; the text encoding matrix is determined according to the target text; the audio encoding matrix is determined according to the offset audio.
  • ⁇ is a linearly adjustable alignment intensity threshold, which is used to represent the preset alignment intensity between the position of the nth character and the time point t when the nth character is pronounced, and the value of ⁇ The value range is ⁇ (0,1).
  • the device further includes a fourth determining unit 504, configured to determine the audio decoding matrix according to the sample code matrix and the audio coding matrix by using the function L(A) Text and sample speech are trained on the function L(A) model, and the alignment intensity threshold ⁇ of the function L(A) is determined.
  • a fourth determining unit 504 configured to determine the audio decoding matrix according to the sample code matrix and the audio coding matrix by using the function L(A) Text and sample speech are trained on the function L(A) model, and the alignment intensity threshold ⁇ of the function L(A) is determined.
  • the fourth determining unit 504 is specifically configured to: set ⁇ at preset intervals to automatically change from 0 to 1; for the ⁇ after each change, perform actions based on sample text and sample voice.
  • the function L(A) is trained; it is determined that when the alignment strength of the sample text and the sample speech reaches a first threshold, the shortest time ⁇ for the speech synthesis is the alignment strength threshold ⁇ .
  • ⁇ is the alignment strength threshold
  • N is the total number of characters in the N characters of the target text
  • n is the number of characters in the nth word in the N characters
  • T is the The time point when the Nth character of the target text is pronounced
  • t is the time point when the nth character is pronounced.
  • the third determining unit 503 is specifically configured to determine the Mel cepstrum coefficients according to the audio decoding matrix, and perform short-time Fourier transform on the Mel cepstrum coefficients, and then according to the short-term Time Fourier spectrum to determine the target audio frequency.
  • FIG. 6 is a simplified physical device structure diagram of an attention mechanism-based speech synthesis device provided by an embodiment of the present application, which is easy to understand and easy to illustrate.
  • the device 60 in FIG. 6 may include the following: Or multiple components: storage component 601, processing component 602, and communication component 603.
  • the storage component 601 may include one or more storage units, and each unit may include one or more memories.
  • the storage component can be used to store programs and various data, and can complete the programs or data at high speed and automatically during the operation of the device 60 Access.
  • a physical device with two stable states can be used to store information, and the two stable states are represented as "0" and "1" respectively.
  • the storage component can be used to store target text, target audio, and other related data.
  • the processing component 602 may also be called a processor, a processing unit, a processing board, a processing module, a processing device, and so on.
  • the processing component may be a central processing unit (CPU), a network processor (NP) or a combination of CPU and NP.
  • CPU central processing unit
  • NP network processor
  • the processing component 603 is used to call the data of the storage component 601 to execute the method described in FIGS. 3A to 4 above. Related descriptions are not repeated here.
  • the communication component 603 may also be called a transceiver, or a transceiver, etc., which may include a unit for wireless, wired, or other communication methods.
  • the device for implementing the receiving function in part 603 can be regarded as the receiving unit, and the device for implementing the sending function as the sending unit, that is, the part 603 can receive target text or send target audio.
  • the units described as separate parts may or may not be physically separated, and the parts displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed in multiple locations.
  • Network unit Some or all of the units may be selected according to actual needs to achieve the objectives of the solutions of the embodiments of the present application.
  • the functional components in the various embodiments of the present application may be integrated into one component, or each component may exist alone physically, or two or more components may be integrated into one component.
  • the above-mentioned integrated components can be implemented in the form of hardware or software functional units.
  • the integrated component is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium.
  • an embodiment of the present application provides a computer-readable storage medium, characterized in that the computer-readable storage medium stores a computer program, and the computer program is executed by a processor to implement FIGS. 3A and 4 Corresponding description of the method embodiment shown.
  • the technical solution of this application is essentially or the part that contributes to the existing technology, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium and includes several instructions It is used to enable a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the embodiments of the present application.
  • the aforementioned storage media include: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disk and other media that can store program code .
  • the size of the sequence numbers of the above-mentioned processes does not mean the order of execution, and the execution order of each process should be determined by its function and internal logic, rather than corresponding to the embodiments of the present application.
  • the implementation process constitutes any limitation.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Computational Linguistics (AREA)
  • Health & Medical Sciences (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Human Computer Interaction (AREA)
  • Acoustics & Sound (AREA)
  • Multimedia (AREA)
  • General Physics & Mathematics (AREA)
  • Mathematical Physics (AREA)
  • Signal Processing (AREA)
  • Machine Translation (AREA)

Abstract

一种基于注意力机制的语音合成方法及装置，其中，所述方法包括：根据目标文本确定文本编码矩阵和音频编码矩阵（S301）；根据文本编码矩阵和音频编码矩阵，通过函数L(A)确定音频解码矩阵（S302），其中，函数L(A)为根据A nt以及W nt确定的注意力机制损失函数；根据音频解码矩阵确定梅尔倒谱系数，并根据梅尔倒谱系数确定目标音频（S303）。该方法可以在目标文本的对齐强度小于对齐强度阈值时，使W nt呈线性变化，不仅可以大大降低原注意力机制中的算法难度，而且可以在不需要常需要大量的样本数据和训练时间的情况下，保证将明显位置偏离的文本和语音信号进行强制对齐。

Description

一种基于注意力机制的语音合成方法及装置
本申请要求于2019年02月27日提交中国专利局、申请号为201910149065.5、申请名称为“一种基于注意力机制的语音合成方法及装置”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及语音合成技术领域，尤其涉及一种基于注意力机制的语音合成方法及装置。
背景技术
语音合成是通过机械的、电子的方法产生人造语音的技术，又叫文语转换技术(Text to Speech，TTS)，TTS技术隶属于语音合成，它是将计算机自己产生的、或外部输入的文字信息转变为可以听得懂的、流利的汉语口语输出的技术。现有的基于注意力机制语音合成技术中原来的算法能够使得注意力机制具有将明显位置偏离的文本和语音信号进行强制对齐的性质，但是复杂度太高，通常需要大量的样本数据和训练时间才能达到预期效果，所以如何在保证将明显位置偏离的文本和语音信号进行强制对齐的性质的情况下降低算法的难度，是目前要解决的问题。
发明内容
鉴于上述问题，提出了本申请以便提供一种克服上述问题或者至少部分地解决上述问题的一种基于注意力机制的语音合成方法及装置。
本申请实施例提供了一种基于注意力机制的语音合成方法，可包括：根据目标文本确定文本编码矩阵和音频编码矩阵；
根据文本编码矩阵和音频编码矩阵，通过函数L(A)确定音频解码矩阵，其中，函数L(A)为根据A nt以及W nt确定的注意力机制损失函数，其中，A nt用于对所述文本编码矩阵进行转换，若目标文本的对齐强度小于对齐强度阈值，则W nt呈线性变化；目标文本的对齐强度由所述目标文本中第n个文字所处位置和所述目标文本中第n个文字发音的时间点t确定，n大于0且小于或等于目标文本字符数，t大于0且小于或等于目标文本总发音的时间点；
根据音频解码矩阵确定梅尔倒谱系数，并根据梅尔倒谱系数确定目标音频。
本申请实施例提供了一种基于注意力机制的语音合成装置，可包括：第一确定单元，用于根据目标文本确定文本编码矩阵和音频编码矩阵；
第二确定单元，用于根据文本编码矩阵和音频编码矩阵，通过函数L(A)确定音频解码矩阵，其中，函数L(A)为根据A nt以及W nt确定的注意力机制损失函数，其中，A nt用于对所述文本编码矩阵进行转换，若目标文本的对齐强度小于对齐强度阈值，则W nt呈线性变化；目标文本的对齐强度由所述目标文本中第n个文字所处位置和所述目标文本中第n个文字发音的时间点t确定，n大于0且小于或等于目标文本字符数，t大于0且小于或等于目标文本总发音的时间点；
第三确定单元，用于根据音频解码矩阵确定梅尔倒谱系数，并根据梅尔倒谱系数确定目标音频。
申请实施例提供了一种计算机可读存储介质，该计算机可读存储介质存储有程序指令，该程序指令当被处理器运行时，该处理器执行上述任一一种基于注意力机制的语音合成方法。
本申请实施例提供了一种基于注意力机制的语音合成装置，包括存储组件，处理组件和通信组件，存储组件，处理组件和通信组件相互连接，其中，存储组件用于存储数据处 理代码，通信组件用于与外部设备进行信息交互；处理组件被配置用于调用程序代码，执行任一一种基于注意力机制的语音合成方法，此处不再赘述。
本申请与现有技术相比，具有以下有益效果：
通过本申请实施例提供了一种基于注意力机制的语音合成方法，可以根据目标文本的文本编码矩阵和音频编码矩阵通过函数L(A)确定音频解码矩阵后，进一步的根据音频解码矩阵确定梅尔倒谱系数，并根据梅尔倒谱系数确定目标音频。其中，函数L(A)为根据Ant以及Wnt确定的注意力机制损失函数，其中，Ant用于对所述文本编码矩阵进行转换，若目标文本的对齐强度小于对齐强度阈值，则Wnt呈线性变化；进一步的目标文本的对齐强度由所述目标文本中第n个文字所处位置和所述目标文本中第n个文字发音的时间点t确定。若在目标文本的对齐强度小于对齐强度阈值时，使Wnt呈线性变化，不仅可以大大降低原注意力机制中的算法难度，而且可以在不需要常需要大量的样本数据和训练时间的情况下，保证将明显位置偏离的文本和语音信号进行强制对齐，有助于更快地使得注意力机制矩阵达到近似对齐的目的，从而使得语音合成更加有条理。
本申请附加的方面和优点将在下面的描述中部分给出，这些将从下面的描述中变得明显，或通过本申请的实践了解到。
附图说明
本申请上述的和/或附加的方面和优点从下面结合附图对实施例的描述中将变得明显和容易理解，其中：
图1是本申请实施例提供的一种基于注意力机制的语音合成系统架构的示意图；
图2是本申请实施例提供的一种合成语音时的终端界面示意图；
图3A是本申请实施例提供的一种基于注意力机制的语音合成方法流程的示意图；
图3B是本申请实施例提供的基于一种改进的基于注意力机制的语音合成技术方法框架示意图；
图4是本申请实施例提供的另一种基于注意力机制的语音合成方法流程的示意图；
图5是本申请实施例提供的一种基于注意力机制的语音合成装置的结构示意图；
图6是本申请实施例提供的一种简化的基于注意力机制的语音合成装置的实体装置结构示意图。
具体实施方式
下面将结合本申请实施例中的附图，对本申请实施例进行描述。
本申请的说明书和权利要求书及所述附图中的术语“第一”、“第二”、“第三”和“第四”等是用于区别不同对象，而不是用于描述特定顺序。此外，“包括”和“具有”以及它们任何变形，意图在于覆盖不排他的包含。例如包含了一系列步骤或单元的过程、方法、系统、产品或设备没有限定于已列出的步骤或单元，而是可选地还包括没有列出的步骤或单元，或可选地还包括对于这些过程、方法、产品或设备固有的其它步骤或单元。
在本文中提及“实施例”意味着，结合实施例描述的特定特征、结构或特性可以包含在本申请的至少一个实施例中。在说明书中的各个位置出现该短语并不一定均是指相同的实施例，也不是与其它实施例互斥的独立的或备选的实施例。本领域技术人员显式地和隐式地理解的是，本文所描述的实施例可以与其它实施例相结合。
在本申请中使用的术语“服务器”、“单元”、“系统”等用于表示计算机相关的实体、硬件、固件、硬件和软件的组合、软件、或执行中的软件。例如，服务器可以是但不限于，处理器，数据处理平台，计算设备，计算机，两个或更多个计算机等。
首先，对本申请中的部分用语进行解释说明，以便于本领域技术人员理解。
(1)语音合成，语音合成是将一段文字输入进去输出与文字对应的一段语音，也是通过机械的、电子的方法产生人造语音的技术。TTS技术(又称文语转换技术)隶属于语音合成，它是将计算机自己产生的、或外部输入的文字信息转变为可以听得懂的、流利的汉语口语输出的技术。
(2)注意力机制，注意力机制(Attention Mechanism)源于对人类视觉的研究。在认知科学中，由于信息处理的瓶颈，人类会选择性地关注所有信息的一部分，同时忽略其他可见的信息。上述机制通常被称为注意力机制。人类视网膜不同的部位具有不同程度的信息处理能力，即敏锐度(Acuity)，只有视网膜中央凹部位具有最强的敏锐度。为了合理利用有限的视觉信息处理资源，人类需要选择视觉区域中的特定部分，然后集中关注它。例如，人们在阅读时，通常只有少量要被读取的词会被关注和处理。综上，注意力机制主要有两个方面：决定需要关注输入的哪部分；分配有限的信息处理资源给重要的部分。
(3)短时距傅里叶变换，短时距傅里叶变换是傅里叶变换的一种变形，用于决定随时间变化的信号局部部分的正弦频率和相位。实际上，计算短时傅里叶变换(STFT)的过程是将长时间信号分成数个较短的等长信号，然后再分别计算每个较短段的傅里叶变换。通常拿来描绘频域与时域上的变化，为时频分析中其中一个重要的工具。
其次，对本申请实施例所基于的其中一种基于注意力机制的语音合成系统架构进行描述。请参考附图1，图1是本申请实施例提供的一种基于注意力机制的语音合成系统架构的示意图，包括：基于注意力机制的语音合成装置101和终端装置102。
基于注意力机制的语音合成装置101，可以是服务器，其中服务器可以是但不限于，处理器，数据处理平台，计算设备，计算机，两个或更多个计算机等。例如当基于注意力机制的语音合成装置101为服务器时，服务器是一种通过获取、处理、分析和提取有价值的、海量和多样化数据，以交互数据为基础，为第三方使用带来各种便利的服务设备。基于注意力机制的语音合成装置101可以根据目标文本确定文本编码矩阵和音频编码矩阵；根据文本编码矩阵和音频编码矩阵，通过函数L(A)确定音频解码矩阵，其中，函数L(A)为根据A nt以及W nt确定的注意力机制损失函数，其中，A nt用于对所述文本编码矩阵进行转换，若目标文本的对齐强度小于对齐强度阈值，则W nt呈线性变化；目标文本的对齐强度由所述目标文本中第n个文字所处位置和所述目标文本中第n个文字发音的时间点t确定，n大于0且小于或等于目标文本字符数，t大于0且小于或等于目标文本总发音的时间点；根据音频解码矩阵确定梅尔倒谱系数，并根据梅尔倒谱系数确定目标音频。
终端装置102可以是通信终端、便携式终端、移动设备、用户终端、移动终端、无线通信设备、用户代理、用户装置、服务设备或用户设备(User Equipment，UE)等计算机网络中处于网络最外围的设备，主要用于数据的输入以及处理结果的输出或显示等，也可以是安装于或运行于上述任一一设备上的软件客户端、应用程度等。例如，客户端可以是目标用户或当前租赁用户所使用的智能手机、电脑、平板设备，或安装于或运行于智能手机、电脑、平板设备上的软件客户端、应用程度等。请见附图2，图2是本申请实施例提供的一种合成语音时的终端界面示意图。当终端装置102为电脑时，可以用于向基于注意力机制的语音合成装置101发送目标文本，接收并播放基于注意力机制的语音合成装置101发送的目标音频。
可以理解的是，基于注意力机制的语音合成装置101可以同时接收多个不同终端装置102发送的不同的目标文本。
参考附图3A，图3A是本申请实施例提供的一种基于注意力机制的语音合成方法流程的示意图。可应用于上述图1中的系统，下面将结合图3A以执行主体是基于注意力机制的语音合成装置101为例，从基于注意力机制的语音合成装置101的单侧进行描述。该方 法可以包括以下步骤S301-步骤S303。
步骤S301：根据目标文本确定文本编码矩阵和音频编码矩阵。
具体地，根据目标文本确定文本编码矩阵和音频编码矩阵可以包括通过文本编码器Encoder，根据输入的所述目标文本X＝(x 1x 1,x 2,…x n…,x N-1,x N)x 1x 2x nx N-1x N确定所述文本编码矩阵P＝f(x 1,x 2,…x n…,x N-1,x N)，其中，x n为第n个文本，f(x n)为输入的第n个文本x n通过第一编码变换确定的文本编码；通过音频编码器Encoder，根据输入的所述偏移音频Y＝(y 1,y 2,…y t…,y T-1,y T)确定所述音频编码矩阵Q＝g(y 1,y 2,…y t…,y T-1,y T)，其中y t为第n个文字在输入所述偏移音频的时长为t时的音频，g(y t)为输入所述偏移音频的时长为t时的音频y t通过第二编码变换确定的音频编码。可以理解的：文本编码是对输入目标文本X进行编码，将输入文本通过非线性变换转化为中间语义表示P，此过程即为文本编码。
可选的，根据目标文本确定文本编码矩阵和音频编码矩阵之前可以获取目标文本，所述目标文本包括N个字符的文字；获取偏移音频，所述偏移音频包括所述目标文本的时长为T的音频；再根据所述目标文本确定所述文本编码矩阵；根据所述偏移音频确定所述音频编码矩阵。举例来说：获取目标文本可以为接收用户输入的样本文本“平安科技技术有限公司”，对输入的文本进行字序标注，每个句子中的字体位置表示为向量X＝(x 1,x 2,…x n…,x N-1,x N)，N为源句子长度，其中x n为文本中第n个字字符的文本。获取偏移音频可以为根据所述目标文本，在语音库中匹配与所述目标文本相对应的音频，确定所述偏移音频。例如：样本文本“平安科技技术有限公司”可以在语音库中分别匹配“平”、“安”、“科”、“技”、“技”、“术”、“有”、“限”、“公”、“司”十个文字的对应音频，并组成偏移音频。其中，所述语音库可以包括文本与该文本发音音频的映射关系获取偏移音频还可以为接收输入的时长为T＝5s的偏移音频“平安科技技术有限公司”的偏移音频。其中，可以为每个目标文本中的句子的发音表示为向量Y＝(y 1,y 2,…y t…,y T-1,y T)，T为目标字体的发音的时间长,其中y t为句子中第n个字的发音的时间帧t时的音频。可以理解的是，获取目标文本，所述目标文本包括N个字符的文字；获取偏移音频，所述偏移音频包括所述目标文本的时长为T的音频；根据所述目标文本确定所述文本编码矩阵；根据所述偏移音频确定所述音频编码矩阵，四个步骤的先后顺序并不做具体限定。举例来说：可以首先获取目标文本，根据所述目标文本确定所述文本编码矩阵后，再获取偏移音频，最后根据所述偏移音频确定所述音频编码矩阵。
步骤S302：根据文本编码矩阵和音频编码矩阵，通过函数L(A)确定音频解码矩阵。
具体地，对于音频解码矩阵来说可以利用解码器Decoder获得，即对于解码器Decoder来说，其任务是根据文本编码矩阵获得的文本X的中间语义表示P和音频编码矩阵获得的音频表示Q＝g(y 1,y 2,…y t…,y T-1,y T)来生成t时刻要生成的音频y t，即所述音频解码矩阵。可选的，所述音频解码矩阵S＝(s 1,s 2,…s n…,s N-1,s N)，其中s n＝h(L(A),y 1,y 2,…y t-1)，其中h为第一解码变换函数。请参见图3B，图3B是本申请实施例提供的基于一种改进的基于注意力机制的语音合成技术方法框架示意图，包括：文本编码模块、音频编码模块、注意力矩阵模块、音频解码模块和短时傅里叶频谱模块。其中，目标文本和偏移音频分别从文本编码模块和音频编码模块输入，获得对应的文本解码矩阵和音频解码矩阵，将所述音频解码矩阵与文本解码矩阵经注意力强制对齐后与所述文本解码矩阵输入音频解码模块中，得到与所述目标文本对应的目标音频。可以理解的，Encoder-Decoder是个非常通用的计算框架，至于Encoder和Decoder具体使用什么模型函数即都不做限定，举例来说：卷积神经网络CNN、循环神经网络RNN、双向长短时记忆循环神经网络BiRNN、门控循环单元GRU、长短期记忆网络LSTM等等，都可以做Encoder和Decoder的模型函数。
可选的，函数L(A)为根据A nt以及W nt确定的注意力机制损失函数，其中，A nt用于对 所述文本编码矩阵进行转换，若目标文本的对齐强度小于对齐强度阈值，则W nt呈线性变化；目标文本的对齐强度由所述目标文本中第n个文字所处位置和所述目标文本中第n个文字发音的时间点t确定，n大于0且小于或等于目标文本字符数，t大于0且小于或等于目标文本总发音的时间点。预设函数L(A)可以为
Figure PCTCN2019117785-appb-000001
举例来说：函数L(A)应用于图3B所述注意力机制模块中的注意力机制矩阵A，A∈R N×T，其意义为评估第n次字符和T时间的对应关系，即第n个字符第1和第t个时间帧S 1:F,t是相关的，其中A nt可以为
Figure PCTCN2019117785-appb-000002
意味着注意力机制模块在时间t处查看第n个字符，它将在随后的时间t+1处查看第n个字符或第n+1个字符或它们周围的字符，其中d为与文本长度相关的预设参数。即L(A)可以通过注意力机制，使用LSTM标准函数归一化指数函数(Softmax函数)去获得在训练过程中第n个字符声音特征的权重，然后进行求和后归一化，。
可选的，
Figure PCTCN2019117785-appb-000003
其中，
Figure PCTCN2019117785-appb-000004
为所述目标文本的对齐强度，θ为对齐强度阈值，N是所述目标文本的N个文字的字符总字符数，n是在N个文字中第n个字的字符数，T是所述目标文本的第N个文字发音时所在的时间点，t是第n个文字发音时所在的时间点。W nt与所述目标文本对齐强度有关的分段函数，若目标文本的对齐强度小于对齐强度阈值，则W nt随目标文本的增大而减小。
步骤S303：根据音频解码矩阵确定梅尔倒谱系数，并根据梅尔倒谱系数确定目标音频。
具体地，根据音频解码矩阵确定梅尔倒谱系数，并根据梅尔倒谱系数确定目标音频具体为：根据音频解码矩阵确定梅尔倒谱系数，并将梅尔倒谱系数，进行短时傅里叶变换，再根据短时傅里叶频谱，确定目标音频。在声音处理领域中，梅尔频率倒谱(Mel-Frequency Cepstrum)是基于声音频率的非线性梅尔刻度(mel scale)的对数能量频谱的线性变换。梅尔频率倒谱系数(Mel-Frequency Cepstral Coefficients，MFCCs)就是组成梅尔频率倒谱的系数。它衍生自音讯片段的倒频谱(cepstrum)。倒谱和梅尔频率倒谱的区别在于，梅尔频率倒谱的频带划分是在梅尔刻度上等距划分的，它比用于正常的对数倒频谱中的线性间隔的频带更能近似人类的听觉系统。这样的非线性表示，可以在多个领域中使声音信号有更好的表示。举例来说，所述确定对应的梅尔倒谱系数，可以为先对音频解码矩阵的语音进行预加重、分帧和加窗；对每一个短时分析窗，可以通过离散傅氏变换的快速算法(Fast Fourier Transformation，FFT)得到对应的频谱；将上述频谱通过梅尔滤波器组得到梅尔频谱(Mel频谱)。其中，公式表述可以为：log X[k]＝log(Mel-Spectrum)。因此我们可以在log X[k]上进行倒谱分析：1)取对数：log X[k]＝log H[k]+log E[k]；2)进行逆变换：x[k]＝h[k]+e[k]。在Mel频谱上面获得的倒谱系数h[k]就称为梅尔频率倒谱系数(Mel-Frequency Cepstral Coefficients，MFCCs)。根据梅尔倒谱系数确定目标音频，可以包括将梅尔倒谱系数转换为短时傅里叶频谱，可以得到一个随着时间变化的频谱图，所述频谱图这个就是描述语音信号的spectrogram声谱图，声谱图中可以表示目标音频。可以理解的是，本发明实施例对根据梅尔倒谱系数确定目标音频的方式不做具体限定。
实施本申请实施例，可以根据目标文本的文本编码矩阵和音频编码矩阵通过函数L(A)确定音频解码矩阵后，进一步的根据音频解码矩阵确定梅尔倒谱系数，并根据梅尔倒谱系数确定目标音频。其中，函数L(A)为根据A nt以及W nt确定的注意力机制损失函数，其中，A nt用于对所述文本编码矩阵进行转换，若目标文本的对齐强度小于对齐强度阈值，则W nt 呈线性变化；进一步的目标文本的对齐强度由所述目标文本中第n个文字所处位置和所述目标文本中第n个文字发音的时间点t确定。因此，在目标文本的对齐强度小于对齐强度阈值时，W nt呈线性变化不仅可以大大降低原注意力机制中的算法难度，而且可以保证将明显位置偏离的文本和语音信号进行强制对齐，有助于更快地使得注意力机制矩阵达到近似对齐的目的，从而使得语音合成更加有条理。
参考附图4，图4是本申请实施例提供的另一种基于注意力机制的语音合成方法流程的示意图。可应用于上述图1中的系统，下面将结合图4从基于注意力机制的语音合成装置101的单侧进行描述。该方法可以包括以下步骤S401-步骤S403。
步骤S401：根据样本文本和样本语音做函数L(A)模型训练，确定函数L(A)的对齐强度阈值θ。
具体地，按预设间隔设置θ从0到1进行自动变化；对每次变化后的θ，根据样本文本和样本语音做所述函数L(A)训练；确定在所述样本文本和所述样本语音的对齐强度达到第一阈值的情况下，所述语音合成所费时间最短的θ为对齐强度阈值θ。举例来说，本申请实施例可应用于基于导向性注意力机制的语音合成的场景中，在语音合成建模过程中，文字和音频信号片段的位置有着大致的关联性。因此，当一个人说一句话的时候，文字所处的位置n与所在的时间点t有着近似的线性关系，即n～at，其中，a～N/T。进一步的将这样一个先验知识纳入到语音合成的注意力机制中。还可以先确定θ使字体位置和语音时间的对应强度较强的阈值范围，再按照预设间隔在其阈值范围进行自动变化，对每次变化后的θ，根据样本文本和样本语音做所述函数L(A)训练，所获得的训练后的θ可以更好地使字体位置和语音时间的对应强度越强。可以理解的是，本申请实施例对确定函数L(A)的对齐强度阈值θ的方法，不做具体限定。可选的，θ为线性可调节的对齐强度阈值，用于表示第n个文字所处位置与第n个文字发音的时间点t之间预设的对齐强度，且θ的取值范围为θ∈(0,1)。当当阈值θ越接近零时，说明其获得的字体位置和语音时间的对应强度越强，及语音与文字的对应强度越高。
步骤S402：根据目标文本确定文本编码矩阵和音频编码矩阵。
步骤S403：根据文本编码矩阵和音频编码矩阵，通过函数L(A)确定音频解码矩阵。
步骤S404：根据音频解码矩阵确定梅尔倒谱系数，并根据梅尔倒谱系数确定目标音频。
具体地，上述对步骤S402-步骤S404的相关描述可以对应参照图3A的步骤S301-步骤S303的相关描述，此处不再赘述。
实施本申请实施例，可以先对函数L(A)的模型做样本训练，再根据目标文本的文本编码矩阵和音频编码矩阵通过函数L(A)确定音频解码矩阵后，进一步的根据音频解码矩阵确定梅尔倒谱系数，并根据梅尔倒谱系数确定目标音频。其中，函数L(A)为根据A nt以及W nt确定的注意力机制损失函数，其中，A nt用于对所述文本编码矩阵进行转换，若目标文本的对齐强度小于对齐强度阈值，则W nt呈线性变化；进一步的目标文本的对齐强度由所述目标文本中第n个文字所处位置和所述目标文本中第n个文字发音的时间点t确定。通过训练阈值θ的大小，确定在当阈值θ越接近零，目标文本的对齐强度越接近于阈值θ的时候，证明其字体位置和语音时间的对应强度越强，及语音与文字的对应强度越高。所以通过改进的注意力机制模型的语音合成技术，可以在保证将明显位置偏离的文本和语音信号进行强制对齐的性质的同时，降低算法的难度，大大减少了语音合成的时间。因此，在目标文本的对齐强度小于对齐强度阈值时，W nt呈线性变化不仅可以大大降低原注意力机制中的算法难度，而且可以保证将明显位置偏离的文本和语音信号进行强制对齐，有助于更快地使得注意力机制矩阵达到近似对齐的目的，从而使得语音合成更加有条理。
总的来说，本申请可以根据目标文本的文本编码矩阵和音频编码矩阵通过函数L(A)确定音频解码矩阵后，进一步的根据音频解码矩阵确定梅尔倒谱系数，并根据梅尔倒谱系数 确定目标音频。而注意力机制损失函数L(A)随在目标文本的对齐强度的变化而变化，当对齐强度阈值时，因W nt呈线性变化，通过调整损失函数W nt的计算方式，使得注意力机制矩阵的损失函数，具有线性可调节的θ阈值，使得损失函数变为线性，此时不仅可以大大降低原注意力机制中的算法难度，可以在不需要常需要大量的样本数据和训练时间的情况下，保证将明显位置偏离的文本和语音信号进行强制对齐。
上述详细阐述了本申请实施例的方法，下面提供了与本申请实施例的相关基于注意力机制的语音合成装置，基于注意力机制的语音合成装置可以是一种通过快速获取、处理、分析和提取有价值的数据，以交互数据为基础，为第三方使用带来各种便利的服务设备。请参考附图5，图5是本申请实施例提供的一种基于注意力机制的语音合成装置的结构示意图。可以包括第一确定单元501、第二确定单元502、第三确定单元503，还可以包括第四确定单元504。
第一确定单元501，用于根据目标文本确定文本编码矩阵和音频编码矩阵；
第二确定单元502，用于根据文本编码矩阵和音频编码矩阵，通过函数L(A)确定音频解码矩阵，其中，函数L(A)为根据A nt以及W nt确定的注意力机制损失函数，其中，A nt用于对所述文本编码矩阵进行转换，若目标文本的对齐强度小于对齐强度阈值，则W nt呈线性变化；目标文本的对齐强度由所述目标文本中第n个文字所处位置和所述目标文本中第n个文字发音的时间点t确定，n大于0且小于或等于目标文本字符数，t大于0且小于或等于目标文本总发音的时间点；
第三确定单元503，用于根据音频解码矩阵确定梅尔倒谱系数，并根据梅尔倒谱系数确定目标音频。
在一种可能实现的方式中，第一确定单元501具体用于：获取目标文本，所述目标文本包括N个字符的文字；获取偏移音频，所述偏移音频包括所述目标文本的时长为T的音频；根据所述目标文本确定所述文本编码矩阵；根据所述偏移音频确定所述音频编码矩阵。
在一种可能实现的方式中，所述音频解码矩阵S＝(s 1,s 2,…s n…,s N-1,s N)，其中s n＝h(L(A),y 1,y 2,…y t-1)，其中h为第一解码变换函数；所述第二确定单元502用于根据所述目标文本确定所述文本编码矩阵时，具体用于：通过文本编码器，根据输入的所述目标文本X＝(x 1,x 2,…x n…,x N-1,x N)确定所述文本编码矩阵P＝f(x 1,x 2,…x n…,x N-1,x N)，其中，x n为第n个文本，f(x n)为输入的第n个文本x n通过第一编码变换确定的文本编码。
在一种可能实现的方式中，所述音频解码矩阵S＝(s 1,s 2,…s n…,s N-1,s N)，其中s n＝h(L(A),y 1,y 2,…y t-1)，其中h为第一解码变换函数；所述第二确定单元502用于根据所述偏移音频确定所述音频编码矩阵时，具体用于：通过音频编码器，根据输入的所述偏移音频Y＝(y 1,y 2,…y t…,y T-1,y T)确定所述音频编码矩阵Q＝g(y 1,y 2,…y t…,y T-1,y T)，其中y t为第n个文字在输入所述偏移音频的时长为t时的音频，g(y t)为输入所述偏移音频的时长为t时的音频y t通过第二编码变换确定的音频编码。
在一种可能实现的方式中，θ为线性可调节的对齐强度阈值，用于表示第n个文字所处位置与第n个文字发音的时间点t之间预设的对齐强度，且θ的取值范围为θ∈(0,1)。
在一种可能实现的方式中，所述装置还包括第四确定单元504，用于在根据所述文本编码矩阵和所述音频编码矩阵，通过函数L(A)确定音频解码矩阵之前，根据样本文本和样本语音做所述函数L(A)模型训练，确定所述函数L(A)的对齐强度阈值θ。
在一种可能实现的方式中，所述第四确定单元504具体用于：按预设间隔设置θ从0到1进行自动变化；对每次变化后的θ，根据样本文本和样本语音做所述函数L(A)训练；确定在所述样本文本和所述样本语音的对齐强度达到第一阈值的情况下，所述语音合成所费时间最短的θ为对齐强度阈值θ。
在一种可能实现的方式中，
Figure PCTCN2019117785-appb-000005
其中，
Figure PCTCN2019117785-appb-000006
为所述目标文本的对齐强度，θ为对齐强度阈值，N是所述目标文本的N个文字的字符总字符数，n是在N个文字中第n个字的字符数，T是所述目标文本的第N个文字发音时所在的时间点，t是第n个文字发音时所在的时间点。
在一种可能实现的方式中，所述第三确定单元503，具体用于根据音频解码矩阵确定梅尔倒谱系数，并将梅尔倒谱系数，进行短时傅里叶变换，再根据短时傅里叶频谱，确定目标音频。
请参考附图6，图6是本申请实施例提供的一种简化的基于注意力机制的语音合成装置的实体装置结构示意图，便于理解和图示方便，图6装置60中，可以包括以下一个或多个组件：存储组件601，处理组件602，通信组件603。
存储组件601可以包括一个或多个存储单元，每个单元可以包括一个或多个存储器，存储组件可用于存储程序和各种数据，并能在装置60运行过程中高速、自动地完成程序或数据的存取。可以采用具有两种稳定状态的物理器件来存储信息，所述两种稳定状态分别表示为“0”和“1”。当图6所示的装置60，为图1所述基于注意力机制的语音合成装置时，存储组件可以用来存储目标文本、目标音频以及其他相关数据等。
处理组件602，处理组件也可以称为处理器，处理单元，处理单板，处理模块、处理装置等。处理组件可以是中央处理器(central processing unit，CPU)，网络处理器(network processor，NP)或者CPU和NP的组合。当图6所示的装置60，为图1所述基于注意力机制的语音合成装置时，所述处理组件603用于调用所述存储组件601的数据执行上述图3A至图4所述方法的相关描述，此处不再赘述。
通信组件603，也可以称为收发机，或收发器等，其中可以包括用来进行无线、有线或其他通信方式的单元。可选的，可以将603部分中用于实现接收功能的器件视为接收单元，将用于实现发送功能的器件视为发送单元，即603部分可以接收目标文本或发送目标音频。
需要说明的是，各个操作的具体实现还可以对应参照图3A、图4所示的方法实施例的相应描述，此处不再赘述
在本申请中，所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本申请实施例方案的目的。
另外，在本申请各个实施例中的各功能组件可以集成在一个组件也可以是各个组件单独物理存在，也可以是两个或两个以上组件集成在一个组件中。上述集成的组件既可以采用硬件的形式实现，也可以采用软件功能单元的形式实现。
所述集成的组件如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。基于这样的理解，本申请实施例提供了一种计算机可读存储介质，其特征在于，所述计算机可读存储介质存储有计算机程序，所述计算机程序被处理器执行以实现图3A、图4所示的方法实施例的相应描述。本申请的技术方案本质上或者说对现有技术做出贡献的部分，或者该技术方案的全部或部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)执行各个本申请实施例所述方法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器(Read-Only Memory，ROM)、随机存取存储器(Random Access Memory，RAM)、磁碟或者光盘等各种 可以存储程序代码的介质。
以上所述，仅为本申请的具体实施方式，但本申请的保护范围并不局限于此，任何熟悉本技术领域的技术人员在本申请揭露的技术范围内，可轻易想到各种等效的修改或替换，这些修改或替换都应涵盖在本申请的保护范围之内。因此，本申请的保护范围应以权利要求的保护范围为准。
应理解，在本申请的各种实施例中，上述各过程的序号的大小并不意味着执行顺序的先后，各过程的执行顺序应以其功能和内在逻辑确定，而不应对本申请实施例的实施过程构成任何限定。尽管在此结合各实施例对本申请进行了描述，然而，在实施例所要求保护的本申请过程中，本领域技术人员可理解并实现公开实施例的其他变化。

Claims (20)

  1. 一种基于注意力机制的语音合成方法，其特征在于，包括：
    根据目标文本确定文本编码矩阵和音频编码矩阵；
    根据所述文本编码矩阵和所述音频编码矩阵，通过函数L(A)确定音频解码矩阵，其中，所述函数L(A)为根据A nt以及W nt确定的注意力机制损失函数，其中，A nt用于对所述文本编码矩阵进行转换，若所述目标文本的对齐强度小于对齐强度阈值，则W nt呈线性变化；所述目标文本的对齐强度由所述目标文本中第n个文字所处位置和所述目标文本中第n个文字发音的时间点t确定；
    根据所述音频解码矩阵确定梅尔倒谱系数，并根据所述梅尔倒谱系数确定目标音频。
  2. 根据权利要求1所述方法，其特征在于，所述根据目标文本确定文本编码矩阵和音频编码矩阵包括：
    获取目标文本，所述目标文本包括N个字符的文字；
    获取偏移音频，所述偏移音频包括所述目标文本的时长为T的音频；
    根据所述目标文本确定所述文本编码矩阵；
    根据所述偏移音频确定所述音频编码矩阵。
  3. 根据权利要求2所述方法，其特征在于，所述音频解码矩阵S＝(s 1,s 2,…s n…,s N-1,s N)，其中s n＝h(L(A),y 1,y 1,…y t-1)，其中h为第一解码变换函数；
    所述根据所述目标文本确定所述文本编码矩阵，包括：
    通过文本编码器，根据输入的所述目标文本X＝(x 1,x 2,…x n…,x N-1,x N)确定所述文本编码矩阵P＝f(x 1,x 2,…x n…,x N-1,x N)，其中，x n为第n个文本，f(x n)为输入的第n个文本x n通过第一编码变换确定的文本编码。
  4. 根据权利要求2所述方法，其特征在于，所述音频解码矩阵S＝(s 1,s 2,…s n…,s N-1,s N)，其中s n＝h(L(A),y 1,y 2,…y t-1)，其中h为第一解码变换函数；
    所述根据所述偏移音频确定所述音频编码矩阵，包括：
    通过音频编码器，根据输入的所述偏移音频Y＝(y 1,y 2,…y t…,y T-1,y T)确定所述音频编码矩阵Q＝g(y 1,y 2,…y t…,y T-1,y T)，其中y t为第n个文字在输入所述偏移音频的时长为t时的音频，g(y t)为输入所述偏移音频的时长为t时的音频y t通过第二编码变换确定的音频编码。
  5. 根据权利要求1所述方法，其特征在于，对齐强度阈值θ为线性可调节，用于表示第n个文字所处位置与第n个文字发音的时间点t之间预设的对齐强度，且θ的取值范围为θ∈(0,1)。
  6. 根据权利要求1所述方法，其特征在于，所述根据所述文本编码矩阵和所述音频编码矩阵，通过函数L(A)确定音频解码矩阵之前，还包括：
    根据样本文本和样本语音做所述函数L(A)模型训练，确定所述函数L(A)的对齐强度阈值θ。
  7. 根据权利要求6所述方法，其特征在于，所述根据样本文本和样本语音做所述函数L(A)模型训练，确定所述函数L(A)的对齐强度阈值θ包括：
    按预设间隔设置θ从0到1进行自动变化；
    对每次变化后的θ，根据样本文本和样本语音做所述函数L(A)训练；
    确定在所述样本文本和所述样本语音的对齐强度达到第一阈值的情况下，所述语音合成所费时间最短的θ为对齐强度阈值θ。
  8. 根据权利要求1所述方法，其特征在于，
    所述
    Figure PCTCN2019117785-appb-100001
    其中，
    Figure PCTCN2019117785-appb-100002
    为所述目标文本的对齐强度，θ为对齐强度阈值，N是所述目标文本的N个文字的字符总字符数，n是在N个文字中第n个字的字符数，T是所述目标文本的第N个文字发音时所在的时间点，t是第n个文字发音时所在的时间点。
  9. 根据权利要求1所述方法，其特征在于，所述根据所述音频解码矩阵确定梅尔倒谱系数，并根据所述梅尔倒谱系数确定目标音频，包括：
    根据音频解码矩阵确定梅尔倒谱系数，并将梅尔倒谱系数，进行短时傅里叶变换，再根据短时傅里叶频谱，确定目标音频。
  10. 一种基于注意力机制的语音合成装置，其特征在于，包括：
    第一确定单元，用于根据目标文本确定文本编码矩阵和音频编码矩阵；
    第二确定单元，用于根据所述文本编码矩阵和所述音频编码矩阵，通过函数L(A)确定音频解码矩阵，其中，所述函数L(A)为根据A nt以及W nt确定的注意力机制损失函数，其中，A nt用于对所述文本编码矩阵进行转换，若所述目标文本的对齐强度小于对齐强度阈值，则W nt呈线性变化；所述目标文本的对齐强度由所述目标文本中第n个文字所处位置和所述目标文本中第n个文字发音的时间点t确定；
    第三确定单元，用于根据所述音频解码矩阵确定梅尔倒谱系数，并根据所述梅尔倒谱系数确定目标音频。
  11. 根据权利要求10所述装置，其特征在于，第一确定单元具体用于：
    获取目标文本，所述目标文本包括N个字符的文字；
    获取偏移音频，所述偏移音频包括所述目标文本的时长为T的音频；
    根据所述目标文本确定所述文本编码矩阵；
    根据所述偏移音频确定所述音频编码矩阵。
  12. 根据权利要求11所述装置，其特征在于，所述音频解码矩阵S＝(s 1,s 2,…s n…,s N-1,s N)，其中s n＝h(L(A),y 1,y 2,…y t-1)，其中h为第一解码变换函数；
    所述第二确定单元用于根据所述目标文本确定所述文本编码矩阵时，具体用于：
    通过文本编码器，根据输入的所述目标文本X＝(x 1,x 2,…x n…,x N-1,x N)确定所述文本编码矩阵P＝f(x 1,x 2,…x n…,x N-1,x N)，其中，x n为第n个文本，f(x n)为输入的第n个文本x n通过第一编码变换确定的文本编码。
  13. 根据权利要求11所述装置，其特征在于，所述音频解码矩阵S＝(s 1,s 2,…s n…,s N-1,s N)，其中s n＝h(L(A),y 1,y 2,…y t-1)，其中h为第一解码变换函数；
    所述第二确定单元用于根据所述偏移音频确定所述音频编码矩阵时，具体用于：
    通过音频编码器，根据输入的所述偏移音频Y＝(y 1,y 2,…y t…,y T-1,y T)确定所述音频编码矩阵Q＝g(y 1,y 2,…y t…,y T-1,y T)，其中y t为第n个文字在输入所述偏移音频的时长为t时的音频，g(y t)为输入所述偏移音频的时长为t时的音频y t通过第二编码变换确定的音频编码。
  14. 根据权利要求10所述装置，其特征在于，对齐强度阈值θ为线性可调节，用于表示第n个文字所处位置与第n个文字发音的时间点t之间预设的对齐强度，且θ的取值范围为θ∈(0,1)。
  15. 根据权利要求10所述装置，其特征在于，所述装置还包括第四确定单元，用于根据所述文本编码矩阵和所述音频编码矩阵，通过函数L(A)确定音频解码矩阵之前，根据样本文本和样本语音做所述函数L(A)模型训练，确定所述函数L(A)的对齐强度阈值θ。
  16. 根据权利要求15所述装置，其特征在于，所述第四确定单元具体用于：
    按预设间隔设置θ从0到1进行自动变化；
    对每次变化后的θ，根据样本文本和样本语音做所述函数L(A)训练；
    确定在所述样本文本和所述样本语音的对齐强度达到第一阈值的情况下，所述语音合成所费时间最短的θ为对齐强度阈值θ。
  17. 根据权利要求10所述装置，其特征在于，
    所述
    Figure PCTCN2019117785-appb-100003
    其中，
    Figure PCTCN2019117785-appb-100004
    为所述目标文本的对齐强度，θ为对齐强度阈值，N是所述目标文本的N个文字的字符总字符数，n是在N个文字中第n个字的字符数，T是所述目标文本的第N个文字发音时所在的时间点，t是第n个文字发音时所在的时间点。
  18. 根据权利要求10所述装置，其特征在于，所述第三确定单元，具体用于：
    根据音频解码矩阵确定梅尔倒谱系数，并将梅尔倒谱系数，进行短时傅里叶变换，再根据短时傅里叶频谱，确定目标音频。
  19. 一种基于注意力机制的语音合成装置，其特征在于，包括处理组件、存储组件和通信模组件，处理组件、存储组件和通信组件相互连接，其中，存储组件用于存储数据处理代码，通信组件用于与外部设备进行信息交互；处理组件被配置用于调用程序代码，执行如权利要求1-9任一项所述的方法。
  20. 一种计算机可读存储介质，其特征在于，所述计算机可读存储介质存储有计算机程序，所述计算机程序被处理器执行以实现权利要求1-9任意一项所述的方法。
PCT/CN2019/117785 2019-02-27 2019-11-13 一种基于注意力机制的语音合成方法及装置 Ceased WO2020173134A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201910149065.5A CN109767752B (zh) 2019-02-27 2019-02-27 一种基于注意力机制的语音合成方法及装置
CN201910149065.5 2019-02-27

Publications (1)

Publication Number Publication Date
WO2020173134A1 true WO2020173134A1 (zh) 2020-09-03

Family

ID=66457333

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/117785 Ceased WO2020173134A1 (zh) 2019-02-27 2019-11-13 一种基于注意力机制的语音合成方法及装置

Country Status (2)

Country Link
CN (1) CN109767752B (zh)
WO (1) WO2020173134A1 (zh)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112257407A (zh) * 2020-10-20 2021-01-22 网易（杭州）网络有限公司 音频中的文本对齐方法、装置、电子设备及可读存储介质
CN112397047A (zh) * 2020-12-11 2021-02-23 平安科技（深圳）有限公司 语音合成方法、装置、电子设备及可读存储介质
CN113112987A (zh) * 2021-04-14 2021-07-13 北京地平线信息技术有限公司 语音合成方法、语音合成模型的训练方法及装置
CN113539232A (zh) * 2021-07-10 2021-10-22 东南大学 一种基于慕课语音数据集的语音合成方法
CN115410550A (zh) * 2022-06-02 2022-11-29 柯登峰 一种细粒度韵律可控的情感语音合成方法、系统及存储介质

Families Citing this family (30)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109767752B (zh) * 2019-02-27 2023-05-26 平安科技（深圳）有限公司 一种基于注意力机制的语音合成方法及装置
CN110264991B (zh) * 2019-05-20 2023-12-22 平安科技（深圳）有限公司 语音合成模型的训练方法、语音合成方法、装置、设备及存储介质
CN112133279B (zh) * 2019-06-06 2024-06-21 Tcl科技集团股份有限公司 车载信息播报方法、装置及终端设备
US11183201B2 (en) * 2019-06-10 2021-11-23 John Alexander Angland System and method for transferring a voice from one body of recordings to other recordings
CN110264987A (zh) * 2019-06-18 2019-09-20 王子豪 基于深度学习的和弦进行生成方法
CN111508466A (zh) * 2019-09-12 2020-08-07 马上消费金融股份有限公司 一种文本处理方法、装置、设备及计算机可读存储介质
CN110808027B (zh) * 2019-11-05 2020-12-08 腾讯科技（深圳）有限公司 语音合成方法、装置以及新闻播报方法、系统
CN111133506A (zh) * 2019-12-23 2020-05-08 深圳市优必选科技股份有限公司 语音合成模型的训练方法、装置、计算机设备及存储介质
CN111259188B (zh) * 2020-01-19 2023-07-25 成都潜在人工智能科技有限公司 一种基于seq2seq网络的歌词对齐方法及系统
CN113314096A (zh) * 2020-02-25 2021-08-27 阿里巴巴集团控股有限公司 语音合成方法、装置、设备和存储介质
CN111524503B (zh) * 2020-04-15 2023-01-17 上海明略人工智能(集团)有限公司 音频数据的处理方法、装置、音频识别设备和存储介质
US11580956B2 (en) * 2020-05-07 2023-02-14 Google Llc Emitting word timings with end-to-end models
CN111862934B (zh) * 2020-07-24 2022-09-27 思必驰科技股份有限公司 语音合成模型的改进方法和语音合成方法及装置
CN112466272B (zh) * 2020-10-23 2023-01-17 浙江同花顺智能科技有限公司 一种语音合成模型的评价方法、装置、设备及存储介质
US11798527B2 (en) 2020-08-19 2023-10-24 Zhejiang Tonghu Ashun Intelligent Technology Co., Ltd. Systems and methods for synthesizing speech
CN112837673B (zh) * 2020-12-31 2024-05-10 平安科技（深圳）有限公司 基于人工智能的语音合成方法、装置、计算机设备和介质
CN112908294B (zh) * 2021-01-14 2024-04-05 杭州倒映有声科技有限公司 一种语音合成方法以及语音合成系统
CN113345413B (zh) * 2021-06-01 2023-12-29 平安科技（深圳）有限公司 基于音频特征提取的语音合成方法、装置、设备及介质
CN113299268A (zh) * 2021-07-28 2021-08-24 成都启英泰伦科技有限公司 一种基于流生成模型的语音合成方法
CN113707127B (zh) * 2021-08-30 2023-12-15 中国科学院声学研究所 一种基于线性自注意力的语音合成方法及系统
US11869483B2 (en) * 2021-10-07 2024-01-09 Nvidia Corporation Unsupervised alignment for text to speech synthesis using neural networks
CN114220414B (zh) * 2021-10-29 2025-07-04 广州虎牙科技有限公司 语音合成方法以及相关装置、设备
CN114373480B (zh) * 2021-12-17 2025-08-05 腾讯音乐娱乐科技（深圳）有限公司 语音对齐网络的训练方法、语音对齐方法及电子设备
CN116705005B (zh) * 2022-02-25 2026-03-17 北京字跳网络技术有限公司 语音与文本的对齐方法、装置、电子设备、介质及程序
CN114974202B (zh) * 2022-04-11 2026-02-17 平安科技（深圳）有限公司 基于神经网络的合成语音方法、系统、设备及存储介质
CN115019795B (zh) * 2022-05-31 2025-08-26 四川虹美智能科技有限公司 分布式语音设备参与选举的方法、装置及语音设备
CN115691476B (zh) * 2022-06-06 2023-07-04 腾讯科技（深圳）有限公司 语音识别模型的训练方法、语音识别方法、装置及设备
CN115910032B (zh) * 2022-12-06 2025-06-10 腾讯音乐娱乐科技（深圳）有限公司 音素对齐模型训练方法、计算机设备及计算机存储介质
CN119580698A (zh) * 2023-12-31 2025-03-07 上海稀宇极智科技有限公司 语音识别方法、模型训练方法、模型、装置
CN119314463B (zh) * 2024-10-23 2025-12-02 平安科技（深圳）有限公司 基于门控注意力机制的语音合成方法、装置、设备及介质

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101651788A (zh) * 2008-12-26 2010-02-17 中国科学院声学研究所 一种在线语音文本对齐系统及方法
US20180330713A1 (en) * 2017-05-14 2018-11-15 International Business Machines Corporation Text-to-Speech Synthesis with Dynamically-Created Virtual Voices
CN109036371A (zh) * 2018-07-19 2018-12-18 北京光年无限科技有限公司 用于语音合成的音频数据生成方法及系统
CN109767752A (zh) * 2019-02-27 2019-05-17 平安科技（深圳）有限公司 一种基于注意力机制的语音合成方法及装置

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP4291755B2 (ja) * 2004-08-13 2009-07-08 京セラ株式会社 携帯端末装置及び音声信号の出力方法
JP2008225254A (ja) * 2007-03-14 2008-09-25 Canon Inc 音声合成装置及び方法並びにプログラム
JP6716397B2 (ja) * 2016-08-31 2020-07-01 株式会社東芝 音声処理装置、音声処理方法およびプログラム
CN107943405A (zh) * 2016-10-13 2018-04-20 广州市动景计算机科技有限公司 语音播报装置、方法、浏览器及用户终端

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101651788A (zh) * 2008-12-26 2010-02-17 中国科学院声学研究所 一种在线语音文本对齐系统及方法
US20180330713A1 (en) * 2017-05-14 2018-11-15 International Business Machines Corporation Text-to-Speech Synthesis with Dynamically-Created Virtual Voices
CN109036371A (zh) * 2018-07-19 2018-12-18 北京光年无限科技有限公司 用于语音合成的音频数据生成方法及系统
CN109767752A (zh) * 2019-02-27 2019-05-17 平安科技（深圳）有限公司 一种基于注意力机制的语音合成方法及装置

Cited By (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112257407A (zh) * 2020-10-20 2021-01-22 网易（杭州）网络有限公司 音频中的文本对齐方法、装置、电子设备及可读存储介质
CN112257407B (zh) * 2020-10-20 2024-05-14 网易（杭州）网络有限公司 音频中的文本对齐方法、装置、电子设备及可读存储介质
CN112397047A (zh) * 2020-12-11 2021-02-23 平安科技（深圳）有限公司 语音合成方法、装置、电子设备及可读存储介质
CN113112987A (zh) * 2021-04-14 2021-07-13 北京地平线信息技术有限公司 语音合成方法、语音合成模型的训练方法及装置
CN113112987B (zh) * 2021-04-14 2024-05-03 北京地平线信息技术有限公司 语音合成方法、语音合成模型的训练方法及装置
CN113539232A (zh) * 2021-07-10 2021-10-22 东南大学 一种基于慕课语音数据集的语音合成方法
CN113539232B (zh) * 2021-07-10 2024-05-14 东南大学 一种基于慕课语音数据集的语音合成方法
CN115410550A (zh) * 2022-06-02 2022-11-29 柯登峰 一种细粒度韵律可控的情感语音合成方法、系统及存储介质
CN115410550B (zh) * 2022-06-02 2024-03-26 北京听见科技有限公司 一种细粒度韵律可控的情感语音合成方法、系统及存储介质

Also Published As

Publication number Publication date
CN109767752B (zh) 2023-05-26
CN109767752A (zh) 2019-05-17

Similar Documents

Publication Publication Date Title
WO2020173134A1 (zh) 一种基于注意力机制的语音合成方法及装置
US20230127787A1 (en) Method and apparatus for converting voice timbre, method and apparatus for training model, device and medium
US20220383876A1 (en) Method of converting speech, electronic device, and readable storage medium
CN110111775B (zh) 一种流式语音识别方法、装置、设备及存储介质
CN110246488B (zh) 半优化CycleGAN模型的语音转换方法及装置
CN104732977B (zh) 一种在线口语发音质量评价方法和系统
WO2022126924A1 (zh) 基于域分离的语音转换模型的训练方法及装置
CN113963679B (zh) 一种语音风格迁移方法、装置、电子设备及存储介质
WO2020215666A1 (zh) 语音合成方法、装置、计算机设备及存储介质
WO2020098269A1 (zh) 一种语音合成方法及语音合成装置
CN110600013B (zh) 非平行语料声音转换数据增强模型训练方法及装置
Rammo et al. Detecting the Speaker Language Using CNN Deep LearningAlgorithm
WO2018223727A1 (zh) 识别声纹的方法、装置、设备及介质
CN112185363B (zh) 音频处理方法及装置
US20230368777A1 (en) Method And Apparatus For Processing Audio, Electronic Device And Storage Medium
CN114255740A (zh) 语音识别方法、装置、计算机设备和存储介质
CN114283788B (zh) 发音评测方法、发音评测系统的训练方法、装置及设备
CN112951276A (zh) 一种综合评价语音的方法、装置及电子设备
Mian Qaisar Isolated speech recognition and its transformation in visual signs
Priyadarshani et al. Dynamic time warping based speech recognition for isolated Sinhala words
CN113053409B (zh) 音频测评方法及装置
Dar et al. Bi-directional LSTM-based isolated spoken word recognition for Kashmiri language utilizing Mel-spectrogram feature
WO2023142454A1 (zh) 语音翻译和模型训练方法、装置、电子设备以及存储介质
CN114203153B (zh) 一种跨语种语音合成方法、装置、电子设备及存储介质
Drăguşin et al. Command Recognition System Using Convolutional Neural Networks

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19917374

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19917374

Country of ref document: EP

Kind code of ref document: A1