CN110164413B - Speech synthesis method, apparatus, computer device and storage medium - Google Patents
Speech synthesis method, apparatus, computer device and storage medium Download PDFInfo
- Publication number
- CN110164413B CN110164413B CN201910394665.8A CN201910394665A CN110164413B CN 110164413 B CN110164413 B CN 110164413B CN 201910394665 A CN201910394665 A CN 201910394665A CN 110164413 B CN110164413 B CN 110164413B
- Authority
- CN
- China
- Prior art keywords
- processed
- nth
- word
- text
- words
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
- G10L13/00—Speech synthesis; Text to speech systems
- G10L13/02—Methods for producing synthetic speech; Speech synthesisers
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
- G10L13/00—Speech synthesis; Text to speech systems
- G10L13/08—Text analysis or generation of parameters for speech synthesis out of text, e.g. grapheme to phoneme translation, prosody generation or stress or intonation determination
Abstract
The application provides a speech synthesis method, a speech synthesis device, computer equipment and a storage medium, wherein the method comprises the following steps: the method has the advantages that the text-to-speech conversion result of the word to be processed is generated once, and the sound characteristics of the processed word can be considered, so that the generated text-to-speech conversion result can be smooth, the feeling of pause cannot be generated, namely, the text-to-speech conversion result of the word can be received, and after the sub-text-to-speech conversion result segments in one sentence are combined, the whole effect is not influenced, the speech effect is ensured while the speech synthesis efficiency is improved, the technical problem that in the prior art, one sentence is split into a plurality of text-to-speech conversion results, the pause speech signals are easy to generate, the engagement is very poor, or the technical problem that the time delay is large because the complete speech synthesis result generated by a speech synthesis system can be sent to relevant equipment to be played is solved.
Description
Technical Field
The present application relates to the field of speech processing technologies, and in particular, to a speech synthesis method, apparatus, computer device, and storage medium.
Background
Generally, in a conventional speech synthesis system, speech synthesis can only be performed for a whole sentence, speech synthesis of a word or a phrase cannot be accepted, or a final synthesized speech experience formed by a synthesis result of splicing phrase segments is very poor, a feeling of frustration is easily generated, and the connection between the speech segments is very unnatural. Especially in some real-time scenarios, such as simultaneous interpretation scenarios, it is necessary to generate speech signals in real time according to the translation result of the speaker, and if the speaker waits for a sentence to be spoken or a partial translation result is spliced, the speech synthesis effect is not ideal.
In the related technology, text voice conversion results of a plurality of words are generated once according to needs and are independently called and played; or waiting for the speech synthesis system to generate a complete text sentence and uniformly generating a text-to-speech conversion result, however, if a sentence is split into a plurality of text-to-speech conversion results, a frustrated speech signal is easily generated, and the engagement is very poor; and waiting for the complete sentence generated by the speech synthesis system to have longer time delay, and sending the complete speech synthesis result to the relevant equipment for playing.
Disclosure of Invention
The present application is directed to solving, at least to some extent, one of the technical problems in the related art.
To this end, the present application proposes a speech synthesis method, apparatus, computer device and storage medium, is used for solving the problems that in the prior art, one sentence is split into a plurality of text-to-speech conversion results, a pause speech signal is easy to generate, the engagement is very poor, or the technical problem of larger time delay caused by that the complete speech synthesis result generated by the speech synthesis system can be transmitted to the relevant equipment for playing is solved, by only generating the text-to-speech conversion result of a word to be processed at one time, meanwhile, the voice characteristics of the processed words can be considered, so that the generated text voice conversion result can be smooth, the feeling of pause can not be generated, namely, the text voice conversion result of the words can be received, and after sub-text voice conversion result fragments in a sentence are combined, the overall effect is not influenced, and the voice effect is ensured while the voice synthesis efficiency is improved.
In order to achieve the above object, an embodiment of a first aspect of the present application provides a speech synthesis method, including:
acquiring a text to be processed, and performing word segmentation processing on the text to be processed to generate a plurality of words to be processed;
coding the Nth word to be processed to generate an Nth semantic space vector; wherein N is a positive integer;
acquiring N-1 sound characteristics of a processed word before an Nth word to be processed;
decoding the Nth semantic space vector and the N-1 sound features of the processed words to generate target sound features corresponding to the Nth words to be processed;
generating an Nth voice corresponding to the Nth word to be processed according to the target sound characteristics, and synthesizing voice information corresponding to the text to be processed according to the Nth voices.
The speech synthesis method of the embodiment generates a plurality of words to be processed by acquiring a text to be processed and performing word segmentation processing on the text to be processed, and generates an nth semantic space vector by performing coding processing on an nth word to be processed; wherein N is a positive integer, obtaining N-1 sound characteristics of a processed word before the Nth word to be processed, decoding the N-1 sound characteristics according to the Nth semantic space vector and the N-1 sound characteristics of the processed word to generate a target sound characteristic corresponding to the Nth word to be processed, generating the Nth voice corresponding to the Nth word to be processed according to the target sound characteristic, synthesizing voice information corresponding to a text to be processed according to a plurality of Nth voices, solving the technical problems that in the prior art, a sentence is split into a plurality of text voice conversion results, a voice signal which is easy to generate and is not smooth, or the technical problem that the time delay is large because a complete voice synthesis result generated by a voice synthesis system can be sent to relevant equipment to be played is solved, and only one text voice conversion result of the word to be processed is generated at one time, meanwhile, the voice characteristics of the processed words can be considered, so that the generated text voice conversion result can be smooth, the feeling of pause can not be generated, namely the text voice conversion result of the words can be received, the overall effect is not influenced after sub-text voice conversion result segments in one sentence are combined, and the voice effect is ensured while the voice synthesis efficiency is improved.
In order to achieve the above object, a second aspect of the present application provides a speech synthesis apparatus, including:
the first acquisition module is used for acquiring a text to be processed and performing word segmentation processing on the text to be processed to generate a plurality of words to be processed;
the encoding module is used for encoding the Nth word to be processed to generate the Nth semantic space vector; wherein N is a positive integer;
the second acquisition module is used for acquiring N-1 sound characteristics of the processed words before the Nth word to be processed;
the decoding module is used for decoding and processing the Nth semantic space vector and the N-1 sound features of the processed words to generate target sound features corresponding to the Nth to-be-processed words;
and the processing module is used for generating an Nth voice corresponding to the Nth word to be processed according to the target sound characteristics and synthesizing voice information corresponding to the text to be processed according to the Nth voices.
The speech synthesis device of the embodiment generates a plurality of words to be processed by acquiring a text to be processed and performing word segmentation processing on the text to be processed, and generates an nth semantic space vector by encoding an nth word to be processed; wherein N is a positive integer, obtaining N-1 sound characteristics of a processed word before the Nth word to be processed, decoding the N-1 sound characteristics according to the Nth semantic space vector and the N-1 sound characteristics of the processed word to generate a target sound characteristic corresponding to the Nth word to be processed, generating the Nth voice corresponding to the Nth word to be processed according to the target sound characteristic, synthesizing voice information corresponding to a text to be processed according to a plurality of Nth voices, solving the technical problems that in the prior art, a sentence is split into a plurality of text voice conversion results, a voice signal which is easy to generate and is not smooth, or the technical problem that the time delay is large because a complete voice synthesis result generated by a voice synthesis system can be sent to relevant equipment to be played is solved, and only one text voice conversion result of the word to be processed is generated at one time, meanwhile, the voice characteristics of the processed words can be considered, so that the generated text voice conversion result can be smooth, the feeling of pause can not be generated, namely the text voice conversion result of the words can be received, the overall effect is not influenced after sub-text voice conversion result segments in one sentence are combined, and the voice effect is ensured while the voice synthesis efficiency is improved.
To achieve the above object, a third aspect of the present application provides a computer device, including: a processor and a memory; wherein the processor executes a program corresponding to the executable program code by reading the executable program code stored in the memory, so as to implement the speech synthesis method according to the embodiment of the first aspect.
To achieve the above object, a fourth aspect of the present application provides a non-transitory computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements the speech synthesis method according to the first aspect.
To achieve the above object, a fifth aspect of the present application provides a computer program product, where instructions of the computer program product, when executed by a processor, implement the speech synthesis method according to the first aspect.
Additional aspects and advantages of the present application will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the present application.
Drawings
The foregoing and/or additional aspects and advantages of the present application will become apparent and readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings of which:
fig. 1 is a schematic flowchart of a speech synthesis method according to an embodiment of the present application;
fig. 2 is a schematic flow chart of another speech synthesis method provided in the embodiment of the present application;
FIG. 3 is a diagram illustrating an example of a speech synthesis method according to an embodiment of the present application;
fig. 4 is a schematic structural diagram of a speech synthesis apparatus according to an embodiment of the present application; and
fig. 5 is a schematic structural diagram of a computer device according to an embodiment of the present application.
Detailed Description
Reference will now be made in detail to embodiments of the present application, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to the same or similar elements or elements having the same or similar function throughout. The embodiments described below with reference to the drawings are exemplary and intended to be used for explaining the present application and should not be construed as limiting the present application.
A speech synthesis method, apparatus, computer device, and storage medium of the embodiments of the present application are described below with reference to the accompanying drawings.
Fig. 1 is a flowchart illustrating a speech synthesis method according to an embodiment of the present application.
As shown in fig. 1, the speech synthesis method may include the steps of:
In practical application, there are many scenes that need to generate and play voice information in real time, and in the prior art, if a sentence is split into a plurality of text-to-voice conversion results, a voice signal which is frustrated is easy to generate, and the connectivity is very poor; or the problem of large time delay exists when the complete speech synthesis result is generated and is transmitted to the relevant equipment for playing.
Therefore, the application provides a speech synthesis method, which can generate a text-to-speech conversion result of only one word to be processed at a time and simultaneously can take the sound characteristics of the processed word into account, so that the generated text-to-speech conversion result can be smooth, and the feeling of pause can not be generated, namely, the text-to-speech conversion result of the word can be received and combined into the speech conversion result of a sentence, thereby not affecting the overall effect, and ensuring the speech effect while improving the speech synthesis efficiency.
Specifically, a text to be processed, that is, a text to be subjected to speech synthesis processing is obtained, it can be understood that the text to be processed can be obtained according to different scenes and other factors of different users, for example, in a simultaneous interpretation scene, a translated text result can be used as the text to be processed; and for example, the text input by the user through the relevant equipment as the text to be processed according to the requirement.
Further, after the text to be processed is obtained, word segmentation processing is performed on the text to be processed to generate a plurality of words to be processed, and word segmentation processing can be performed in many ways, for example, the text to be processed is analyzed, and word segmentation processing is performed according to the part of speech and the correlation between words; and performing word segmentation on the text to be processed according to the characteristics of the habit of the user, the intention of the text to be processed and the like.
102, coding the Nth word to be processed to generate an Nth semantic space vector; wherein N is a positive integer.
And 103, acquiring N-1 sound characteristics of the processed word before the Nth word to be processed.
Specifically, in the embodiment of the present application, the speech synthesis model may be implemented by using an end-to-end framework, and is divided into two main parts, namely an encoder and a decoder, where the encoder mainly encodes words to be processed and maps the words to semantic space vectors, and the decoder mainly decodes the semantic space vectors to sound features. The sound features may be mel-frequency spectrum, linear predictive coding, etc., and the mel-frequency spectrum is preferably selected to ensure the effect of synthesizing sound.
Therefore, when the nth word to be processed is processed, the nth word to be processed is encoded to generate a corresponding nth semantic space vector, for example, the 1 st word to be processed is encoded to generate a corresponding 1 st semantic space vector; and for example, the 5 th word to be processed is encoded to generate a corresponding 5 th semantic space vector.
Further, acquiring N-1 sound features of the processed word before the nth word to be processed, wherein it can be understood that the 1 st word to be processed is not processed before the processed word, that is, the sound features of the processed word are not obtained; for example, the 5 th word to be processed, i.e., 4 words that have been processed before. Thus, 4 sound features are acquired, namely 4 mel-frequency spectra when the sound features are the 1 st, 2 nd, 3 rd and fourth sound features, respectively.
It should be noted that, N-1 sound features of the processed word before the nth to-be-processed word may be obtained in many ways, for example, as follows:
in a first example, a preset database is searched for N-1 sound features corresponding to the processed words.
Specifically, after the word is processed to generate the corresponding sound features, the corresponding sound features can be stored in the corresponding database, so that the N-1 sound features corresponding to the processed word can be directly searched in the preset database.
In a second example, N-1 processed words are obtained, and each processed word is encoded and decoded in real time to generate N-1 sound features.
Specifically, N-1 sound features may also be generated by encoding and decoding each processed word in real time.
And 104, decoding the Nth semantic space vector and the N-1 sound features of the processed words to generate target sound features corresponding to the Nth to-be-processed words.
Specifically, for example, the 1 st word to be processed in the above example is not preceded by a processed word, that is, there is no sound feature of the processed word, and the target sound feature corresponding to the 1 st word to be processed is directly generated according to the 1 st semantic space vector; for another example, in the above example, the 5 th semantic space vector and the 1 st sound feature, the 2 nd sound feature and the 3 rd sound feature and the fourth sound feature are used to generate the target sound feature corresponding to the 5 th word to be processed.
It can be understood that, in order to further improve the efficiency and accuracy of speech synthesis, a decoding process may be performed according to the nth semantic space vector and the N-1 sound features of the processed word in many ways to generate a target sound feature corresponding to the nth to-be-processed word, for example, as follows:
in the first example, N-1 sound features are subjected to summation processing, and decoding processing is carried out according to the summation processing result and the Nth semantic space vector to generate target sound features corresponding to the Nth to-be-processed word.
In the second example, the average processing is performed on the N-1 sound features, and the decoding processing is performed according to the average processing result and the Nth semantic space vector to generate the target sound feature corresponding to the Nth word to be processed.
And 105, generating an Nth voice corresponding to the Nth word to be processed according to the target sound characteristics, and synthesizing voice information corresponding to the text to be processed according to the Nth voices.
Specifically, after the target sound feature is obtained, the corresponding nth speech may be synthesized by using GriffinLim or WavenetVocoder and the like according to the target sound feature, and the speech information corresponding to the text to be processed may be synthesized according to the plurality of nth speech. As an example of a scene, a plurality of nth voices are spliced according to a preset sequence to generate a target-segment voice, and the target-segment voice is used as voice information corresponding to a text to be processed. The preset sequence is selected and set according to actual application requirements.
The speech synthesis method of the embodiment generates a plurality of words to be processed by acquiring a text to be processed and performing word segmentation processing on the text to be processed, and generates an nth semantic space vector by performing coding processing on an nth word to be processed; wherein N is a positive integer, obtaining N-1 sound characteristics of a processed word before the Nth word to be processed, decoding the N-1 sound characteristics according to the Nth semantic space vector and the N-1 sound characteristics of the processed word to generate a target sound characteristic corresponding to the Nth word to be processed, generating the Nth voice corresponding to the Nth word to be processed according to the target sound characteristic, synthesizing voice information corresponding to a text to be processed according to a plurality of Nth voices, solving the technical problems that in the prior art, a sentence is split into a plurality of text voice conversion results, a voice signal which is easy to generate and is not smooth, or the technical problem that the time delay is large because a complete voice synthesis result generated by a voice synthesis system can be sent to relevant equipment to be played is solved, and only one text voice conversion result of the word to be processed is generated at one time, meanwhile, the voice characteristics of the processed words can be considered, so that the generated text voice conversion result can be smooth, the feeling of pause can not be generated, namely the text voice conversion result of the words can be received, the overall effect is not influenced after sub-text voice conversion result segments in one sentence are combined, and the voice effect is ensured while the voice synthesis efficiency is improved.
Fig. 2 is a flowchart illustrating another speech synthesis method according to an embodiment of the present application.
As shown in fig. 2, the speech synthesis method may include the steps of:
It should be noted that steps 201 to 202 are the same as steps 101 to 102 in the above embodiment, and detailed description is omitted here, and reference is specifically made to the description of steps 101 to 102.
Specifically, after each word to be processed is encoded to generate a semantic space vector, and when a sound feature is generated by decoding according to the semantic space vector, the sound feature is recorded and stored in a preset database.
Therefore, N-1 sound characteristics corresponding to the processed words can be directly searched in the preset database, and the speech synthesis efficiency is further improved.
And 204, carrying out average processing on the N-1 sound features, and carrying out decoding processing according to an average processing result and the Nth semantic space vector to generate a target sound feature corresponding to the Nth word to be processed.
Specifically, the sound features of the 1+2+ … N-1 word are averaged, the number of frames is reduced, the average is used as an additional input of the decoder, and the average is decoded together with the nth semantic space vector to generate the target sound features corresponding to the nth to-be-processed word.
Specifically, the generated multiple voices are spliced according to a preset sequence to generate a section of target-segment voice, that is, voice information corresponding to the text to be processed, and the voice information can be sent to the relevant device for playing.
In order to make the above process more clear to those skilled in the art, the following description is made by way of example with reference to fig. 3.
Specifically, the whole text-to-speech conversion model can be implemented by adopting an end-to-end framework and is divided into two main parts, namely an encoder and a decoder, wherein the encoder mainly encodes a word sequence to be synthesized and maps the word sequence into a semantic space, and the decoder mainly decodes the semantic space into a mel frequency spectrum.
If the current word to be processed is the 1 st word, generating the Mel frequency spectrum of the 1 st word according to a standard text voice conversion model, and synthesizing sound by using wavenet according to the Mel frequency spectrum of the 1 st word.
If the current to be processed is the 2 nd word, the input of the encoder is 1+2 words, and when decoding, the Mel frequency spectrum of the 1 st word is searched out, and meanwhile, the Mel frequency spectrum generated by the 1 st word is continuously used as the additional input of the decoder to generate the Mel frequency spectrum of the second word.
Therefore, considering the Nth word and the operation similar to the 2 nd word, 1+2+ … N words are input into the encoder, and during decoding, the Mel frequency spectrums of the 1+2+ … N-1 words are forcibly decoded, and meanwhile, the Mel frequency spectrums of the 1+2+ … N-1 words are summed or averaged to reduce the number of frames, and the sum is used as an additional input of the decoder to generate the Mel frequency spectrums of the Nth word.
As shown in fig. 3, taking the content W1 of the first word as input, generating Mel1 of the 1 st word, taking the content W2 of the 2 nd word as input, simultaneously using Mel frequency spectrum Mel1 generated by the first word as input of DensePre-net, taking the content W3 of the 3 rd word as input, simultaneously using Mel frequency spectrum Mel1 generated by the first word and the second word, and Mel2 as input of Dense Pre-net, here, the Mel1 and Mel2 may be summed and averaged to reduce the length of the sequence, and text-to-speech conversion of the subsequent word may be implemented by a similar process.
The speech synthesis method of the embodiment generates a plurality of words to be processed by acquiring a text to be processed and performing word segmentation processing on the text to be processed, and generates an nth semantic space vector by performing coding processing on an nth word to be processed; and step 203, searching N-1 sound features corresponding to the processed words in a preset database, performing average processing on the N-1 sound features, performing decoding processing according to an average processing result and an Nth semantic space vector to generate target sound features corresponding to the Nth word to be processed, generating Nth voice corresponding to the Nth word to be processed according to the target sound features, splicing a plurality of Nth voices according to a preset sequence to generate target section voice, and taking the target section voice as voice information corresponding to the text to be processed. Therefore, the text-to-speech conversion result can be dynamically generated in real time, namely, after a sentence can be split into a plurality of words for synthesis, the generated result still does not influence the final user experience.
In order to implement the above embodiments, the present application also provides a speech synthesis apparatus.
Fig. 4 is a schematic structural diagram of a speech synthesis apparatus according to an embodiment of the present application.
As shown in fig. 4, the speech synthesis apparatus may include: a first acquisition module 410, an encoding module 420, a second acquisition module 430, a decoding module 440, and a processing module 450. Wherein the content of the first and second substances,
the first obtaining module 410 is configured to obtain a text to be processed, and perform word segmentation processing on the text to be processed to generate a plurality of words to be processed.
The encoding module 420 is configured to perform encoding processing on an nth word to be processed to generate an nth semantic space vector; wherein N is a positive integer.
The second obtaining module 430 is configured to obtain N-1 sound features of a processed word before the nth word to be processed.
And the decoding module 440 is configured to perform decoding processing according to the nth semantic space vector and the N-1 sound features of the processed word to generate a target sound feature corresponding to the nth word to be processed.
The processing module 450 is configured to generate an nth voice corresponding to the nth to-be-processed word according to the target sound feature, and synthesize voice information corresponding to the to-be-processed text according to the plurality of nth voices.
In a possible implementation manner of the embodiment of the present application, the second obtaining module 430 is specifically configured to: searching N-1 sound characteristics corresponding to the processed words in a preset database; or acquiring N-1 processed words, and respectively coding and decoding each processed word in real time to generate N-1 sound characteristics.
In a possible implementation manner of the embodiment of the present application, the decoding module 440 is specifically configured to: carrying out average processing on the N-1 sound characteristics; and decoding according to the average processing result and the Nth semantic space vector to generate target sound characteristics corresponding to the Nth word to be processed.
In a possible implementation manner of the embodiment of the present application, the decoding module 440 is further specifically configured to: summing the N-1 sound characteristics; and decoding according to the summation processing result and the Nth semantic space vector to generate target sound characteristics corresponding to the Nth word to be processed.
In a possible implementation manner of the embodiment of the present application, the processing module 450 is specifically configured to: splicing the multiple Nth voices according to a preset sequence to generate a target section of voice; and the target section of voice is used as voice information corresponding to the text to be processed.
It should be noted that the foregoing explanation of the embodiment of the speech synthesis method is also applicable to the speech synthesis apparatus of the embodiment, and the implementation principle thereof is similar, and is not repeated here.
The speech synthesis device of the embodiment of the application generates a plurality of words to be processed by acquiring the text to be processed and performing word segmentation processing on the text to be processed, and generates an Nth semantic space vector by encoding an Nth word to be processed; wherein N is a positive integer, obtaining N-1 sound characteristics of a processed word before the Nth word to be processed, decoding the N-1 sound characteristics according to the Nth semantic space vector and the N-1 sound characteristics of the processed word to generate a target sound characteristic corresponding to the Nth word to be processed, generating the Nth voice corresponding to the Nth word to be processed according to the target sound characteristic, synthesizing voice information corresponding to a text to be processed according to a plurality of Nth voices, solving the technical problems that in the prior art, a sentence is split into a plurality of text voice conversion results, a voice signal which is easy to generate and is not smooth, or the technical problem that the time delay is large because a complete voice synthesis result generated by a voice synthesis system can be sent to relevant equipment to be played is solved, and only one text voice conversion result of the word to be processed is generated at one time, meanwhile, the voice characteristics of the processed words can be considered, so that the generated text voice conversion result can be smooth, the feeling of pause can not be generated, namely the text voice conversion result of the words can be received, the overall effect is not influenced after sub-text voice conversion result segments in one sentence are combined, and the voice effect is ensured while the voice synthesis efficiency is improved.
By in order to implement the above embodiments, the present application also provides a computer device, including: a processor and a memory. Wherein the processor runs a program corresponding to the executable program code by reading the executable program code stored in the memory, for implementing the speech synthesis method as described in the foregoing embodiments.
FIG. 5 is a block diagram of a computer device provided in an embodiment of the present application, illustrating an exemplary computer device 90 suitable for use in implementing embodiments of the present application. The computer device 90 shown in fig. 5 is only an example, and should not bring any limitation to the function and the scope of use of the embodiments of the present application.
As shown in fig. 5, the computer device 90 is in the form of a general purpose computer device. The components of computer device 90 may include, but are not limited to: one or more processors or processing units 906, a system memory 910, and a bus 908 that couples the various system components (including the system memory 910 and the processing unit 906).
The system Memory 910 may include computer system readable media in the form of volatile Memory, such as Random Access Memory (RAM) 911 and/or cache Memory 912. The computer device 90 may further include other removable/non-removable, volatile/nonvolatile computer system storage media. By way of example only, storage system 913 may be used to read from and write to non-removable, nonvolatile magnetic media (not shown in FIG. 5, and commonly referred to as a "hard disk drive"). Although not shown in FIG. 5, a disk drive for reading from and writing to a removable, nonvolatile magnetic disk (e.g., a "floppy disk") and an optical disk drive for reading from or writing to a removable, nonvolatile optical disk (e.g., a Compact disk Read Only Memory (CD-ROM), a Digital versatile disk Read Only Memory (DVD-ROM), or other optical media) may be provided. In these cases, each drive may be connected to bus 908 by one or more data media interfaces. System memory 910 may include at least one program product having a set (e.g., at least one) of program modules that are configured to carry out the functions of embodiments of the application.
A computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.
Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
Computer program code for carrying out operations for aspects of the present application may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C + +, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server.
Program/utility 914 having a set (at least one) of program modules 9140 may be stored, for example, in system memory 910, such program modules 9140 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which or some combination of these examples may comprise an implementation of a network environment. Program modules 9140 generally perform the functions and/or methods of embodiments described herein.
The computer device 90 may also communicate with one or more external devices 10 (e.g., keyboard, pointing device, display 100, etc.), with one or more devices that enable a user to interact with the terminal device 90, and/or with any devices (e.g., network card, modem, etc.) that enable the computer device 90 to communicate with one or more other computing devices. Such communication may occur via input/output (I/O) interfaces 902. Moreover, computer device 90 may also communicate with one or more networks (e.g., a Local Area Network (LAN), a Wide Area Network (WAN), and/or a public Network such as the Internet) via Network adapter 900. As shown in FIG. 5, network adapter 900 communicates with the other modules of computer device 90 via bus 908. It should be appreciated that although not shown in FIG. 5, other hardware and/or software modules may be used in conjunction with computer device 90, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, among others.
The processing unit 906 executes various functional applications and speech synthesis based on the in-vehicle scene by running a program stored in the system memory 910, for example, implementing the speech synthesis method mentioned in the foregoing embodiments.
In order to implement the above embodiments, the present application also proposes a non-transitory computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the speech synthesis method as described in the foregoing embodiments.
In order to implement the above embodiments, the present application also proposes a computer program product, wherein when the instructions of the computer program product are executed by a processor, the speech synthesis method according to the foregoing embodiments is implemented.
In the description herein, reference to the description of the term "one embodiment," "some embodiments," "an example," "a specific example," or "some examples," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the application. In this specification, the schematic representations of the terms used above are not necessarily intended to refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, various embodiments or examples and features of different embodiments or examples described in this specification can be combined and combined by one skilled in the art without contradiction.
Furthermore, the terms "first", "second" and "first" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of the present application, "plurality" means at least two, e.g., two, three, etc., unless specifically limited otherwise.
Any process or method descriptions in flow charts or otherwise described herein may be understood as representing modules, segments, or portions of code which include one or more executable instructions for implementing steps of a custom logic function or process, and alternate implementations are included within the scope of the preferred embodiment of the present application in which functions may be executed out of order from that shown or discussed, including substantially concurrently or in reverse order, depending on the functionality involved, as would be understood by those reasonably skilled in the art of the present application.
The logic and/or steps represented in the flowcharts or otherwise described herein, e.g., an ordered listing of executable instructions that can be considered to implement logical functions, can be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor-containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions. For the purposes of this description, a "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of the computer-readable medium would include the following: an electrical connection (electronic device) having one or more wires, a portable computer diskette (magnetic device), a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). Additionally, the computer-readable medium could even be paper or another suitable medium upon which the program is printed, as the program can be electronically captured, via for instance optical scanning of the paper or other medium, then compiled, interpreted or otherwise processed in a suitable manner if necessary, and then stored in a computer memory.
It should be understood that portions of the present application may be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, the various steps or methods may be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. If implemented in hardware, as in another embodiment, any one or combination of the following techniques, which are known in the art, may be used: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application specific integrated circuit having an appropriate combinational logic gate circuit, a Programmable Gate Array (PGA), a Field Programmable Gate Array (FPGA), or the like.
It will be understood by those skilled in the art that all or part of the steps carried by the method for implementing the above embodiments may be implemented by hardware related to instructions of a program, which may be stored in a computer readable storage medium, and when the program is executed, the program includes one or a combination of the steps of the method embodiments.
In addition, functional units in the embodiments of the present application may be integrated into one processing module, or each unit may exist alone physically, or two or more units are integrated into one module. The integrated module can be realized in a hardware mode, and can also be realized in a software functional module mode. The integrated module, if implemented in the form of a software functional module and sold or used as a stand-alone product, may also be stored in a computer readable storage medium.
The storage medium mentioned above may be a read-only memory, a magnetic or optical disk, etc. Although embodiments of the present application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present application, and that variations, modifications, substitutions and alterations may be made to the above embodiments by those of ordinary skill in the art within the scope of the present application.
Claims (12)
1. A speech synthesis method, comprising the steps of:
acquiring a text to be processed, and performing word segmentation processing on the text to be processed to generate a plurality of words to be processed;
coding the Nth word to be processed to generate an Nth semantic space vector; wherein N is a positive integer;
acquiring N-1 sound characteristics of a processed word before an Nth word to be processed;
decoding the Nth semantic space vector and the N-1 sound features of the processed words to generate target sound features corresponding to the Nth words to be processed;
generating an Nth voice corresponding to the Nth word to be processed according to the target sound characteristics, and synthesizing voice information corresponding to the text to be processed according to a plurality of voices corresponding to the plurality of words to be processed.
2. The method of claim 1, wherein said obtaining N-1 sound features of processed words preceding an nth word to be processed comprises:
searching N-1 sound characteristics corresponding to the processed words in a preset database; or
N-1 processed words are obtained, and each processed word is respectively encoded and decoded in real time to generate N-1 sound characteristics.
3. The method of claim 1, wherein the decoding process according to the nth semantic space vector and the N-1 sound features of the processed word to generate a target sound feature corresponding to the nth word to be processed comprises:
carrying out average processing on the N-1 sound characteristics;
and decoding according to the average processing result and the Nth semantic space vector to generate target sound characteristics corresponding to the Nth word to be processed.
4. The method of claim 1, wherein the decoding process according to the nth semantic space vector and the N-1 sound features of the processed word to generate a target sound feature corresponding to the nth word to be processed comprises:
summing the N-1 sound features;
and decoding according to the summation processing result and the Nth semantic space vector to generate target sound characteristics corresponding to the Nth word to be processed.
5. The method of claim 1, wherein said synthesizing speech information corresponding to the text to be processed from a plurality of the speech corresponding to the plurality of words to be processed comprises:
splicing a plurality of voices corresponding to the plurality of words to be processed according to a preset sequence to generate a target section voice;
and the target section of voice is used as voice information corresponding to the text to be processed.
6. A speech synthesis apparatus, comprising:
the first acquisition module is used for acquiring a text to be processed and performing word segmentation processing on the text to be processed to generate a plurality of words to be processed;
the encoding module is used for encoding the Nth word to be processed to generate the Nth semantic space vector; wherein N is a positive integer;
the second acquisition module is used for acquiring N-1 sound characteristics of the processed words before the Nth word to be processed;
the decoding module is used for decoding and processing the Nth semantic space vector and the N-1 sound features of the processed words to generate target sound features corresponding to the Nth to-be-processed words;
and the processing module is used for generating an Nth voice corresponding to the Nth word to be processed according to the target sound characteristics and synthesizing voice information corresponding to the text to be processed according to a plurality of voices corresponding to the plurality of words to be processed.
7. The apparatus of claim 6, wherein the second obtaining module is specifically configured to:
searching N-1 sound characteristics corresponding to the processed words in a preset database; or
N-1 processed words are obtained, and each processed word is respectively encoded and decoded in real time to generate N-1 sound characteristics.
8. The apparatus of claim 6, wherein the decoding module is specifically configured to:
carrying out average processing on the N-1 sound characteristics;
and decoding according to the average processing result and the Nth semantic space vector to generate target sound characteristics corresponding to the Nth word to be processed.
9. The apparatus of claim 6, wherein the decoding module is further specifically configured to:
summing the N-1 sound features;
and decoding according to the summation processing result and the Nth semantic space vector to generate target sound characteristics corresponding to the Nth word to be processed.
10. The apparatus of claim 6, wherein the processing module is specifically configured to:
splicing a plurality of voices corresponding to the plurality of words to be processed according to a preset sequence to generate a target section voice;
and the target section of voice is used as voice information corresponding to the text to be processed.
11. A computer device comprising a processor and a memory;
wherein the processor executes a program corresponding to the executable program code by reading the executable program code stored in the memory for implementing the speech synthesis method according to any one of claims 1 to 5.
12. A non-transitory computer-readable storage medium on which a computer program is stored, the program, when executed by a processor, implementing the speech synthesis method according to any one of claims 1-5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910394665.8A CN110164413B (en) | 2019-05-13 | 2019-05-13 | Speech synthesis method, apparatus, computer device and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910394665.8A CN110164413B (en) | 2019-05-13 | 2019-05-13 | Speech synthesis method, apparatus, computer device and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110164413A CN110164413A (en) | 2019-08-23 |
| CN110164413B true CN110164413B (en) | 2021-06-04 |
Family
ID=67634315
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910394665.8A Active CN110164413B (en) | 2019-05-13 | 2019-05-13 | Speech synthesis method, apparatus, computer device and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110164413B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110473516B (en) | 2019-09-19 | 2020-11-27 | 百度在线网络技术（北京）有限公司 | Voice synthesis method and device and electronic equipment |
| CN111524500B (en) * | 2020-04-17 | 2023-03-31 | 浙江同花顺智能科技有限公司 | Speech synthesis method, apparatus, device and storage medium |
| CN111951781A (en) * | 2020-08-20 | 2020-11-17 | 天津大学 | Chinese prosody boundary prediction method based on graph-to-sequence |
| CN113595868B (en) * | 2021-06-28 | 2023-05-16 | 深圳云之家网络有限公司 | Voice message processing method and device based on instant messaging and computer equipment |
Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104867491A (en) * | 2015-06-17 | 2015-08-26 | 百度在线网络技术（北京）有限公司 | Training method and device for prosody model used for speech synthesis |
| CN105185372A (en) * | 2015-10-20 | 2015-12-23 | 百度在线网络技术（北京）有限公司 | Training method for multiple personalized acoustic models, and voice synthesis method and voice synthesis device |
| CN105206257A (en) * | 2015-10-14 | 2015-12-30 | 科大讯飞股份有限公司 | Voice conversion method and device |
| CN105355193A (en) * | 2015-10-30 | 2016-02-24 | 百度在线网络技术（北京）有限公司 | Speech synthesis method and device |
| US20160343366A1 (en) * | 2015-05-19 | 2016-11-24 | Google Inc. | Speech synthesis model selection |
| US9508338B1 (en) * | 2013-11-15 | 2016-11-29 | Amazon Technologies, Inc. | Inserting breath sounds into text-to-speech output |
| CN106297766A (en) * | 2015-06-04 | 2017-01-04 | 科大讯飞股份有限公司 | Phoneme synthesizing method and system |
| WO2017094500A1 (en) * | 2015-12-02 | 2017-06-08 | 株式会社電通 | Determination device and voice provision system provided therewith |
| CN107564511A (en) * | 2017-09-25 | 2018-01-09 | 平安科技（深圳）有限公司 | Electronic installation, phoneme synthesizing method and computer-readable recording medium |
| CN108364632A (en) * | 2017-12-22 | 2018-08-03 | 东南大学 | A kind of Chinese text voice synthetic method having emotion |
| CN108492818A (en) * | 2018-03-22 | 2018-09-04 | 百度在线网络技术（北京）有限公司 | Conversion method, device and the computer equipment of Text To Speech |
| CN109036371A (en) * | 2018-07-19 | 2018-12-18 | 北京光年无限科技有限公司 | Audio data generation method and system for speech synthesis |
| CN109697973A (en) * | 2019-01-22 | 2019-04-30 | 清华大学深圳研究生院 | A kind of method, the method and device of model training of prosody hierarchy mark |
-
2019
- 2019-05-13 CN CN201910394665.8A patent/CN110164413B/en active Active
Patent Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9508338B1 (en) * | 2013-11-15 | 2016-11-29 | Amazon Technologies, Inc. | Inserting breath sounds into text-to-speech output |
| US20160343366A1 (en) * | 2015-05-19 | 2016-11-24 | Google Inc. | Speech synthesis model selection |
| CN106297766A (en) * | 2015-06-04 | 2017-01-04 | 科大讯飞股份有限公司 | Phoneme synthesizing method and system |
| CN104867491A (en) * | 2015-06-17 | 2015-08-26 | 百度在线网络技术（北京）有限公司 | Training method and device for prosody model used for speech synthesis |
| CN105206257A (en) * | 2015-10-14 | 2015-12-30 | 科大讯飞股份有限公司 | Voice conversion method and device |
| CN105185372A (en) * | 2015-10-20 | 2015-12-23 | 百度在线网络技术（北京）有限公司 | Training method for multiple personalized acoustic models, and voice synthesis method and voice synthesis device |
| CN105355193A (en) * | 2015-10-30 | 2016-02-24 | 百度在线网络技术（北京）有限公司 | Speech synthesis method and device |
| WO2017094500A1 (en) * | 2015-12-02 | 2017-06-08 | 株式会社電通 | Determination device and voice provision system provided therewith |
| CN107564511A (en) * | 2017-09-25 | 2018-01-09 | 平安科技（深圳）有限公司 | Electronic installation, phoneme synthesizing method and computer-readable recording medium |
| CN108364632A (en) * | 2017-12-22 | 2018-08-03 | 东南大学 | A kind of Chinese text voice synthetic method having emotion |
| CN108492818A (en) * | 2018-03-22 | 2018-09-04 | 百度在线网络技术（北京）有限公司 | Conversion method, device and the computer equipment of Text To Speech |
| CN109036371A (en) * | 2018-07-19 | 2018-12-18 | 北京光年无限科技有限公司 | Audio data generation method and system for speech synthesis |
| CN109697973A (en) * | 2019-01-22 | 2019-04-30 | 清华大学深圳研究生院 | A kind of method, the method and device of model training of prosody hierarchy mark |
Non-Patent Citations (3)
| Title |
|---|
| A comparison of recent waveform generation and acoustic modeling methods for neural-network-based speech synthesis;Wang X, Lorenzo-Trueba J, Takaki S, et al;《2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)》;20180420;全文 * |
| 基于深度学习的小尺度单元拼接语音合成方法研究;周志平;《中国优秀硕士学位论文全文数据库》;20171115;全文 * |
| 语音合成方法和发展综述;张斌,全昌勤,任福继;《小型微型计算机系统》;20160115;第37卷(第01期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110164413A (en) | 2019-08-23 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110164413B (en) | Speech synthesis method, apparatus, computer device and storage medium | |
| WO2020098115A1 (en) | Subtitle adding method, apparatus, electronic device, and computer readable storage medium | |
| CN109754783B (en) | Method and apparatus for determining boundaries of audio sentences | |
| US20240021202A1 (en) | Method and apparatus for recognizing voice, electronic device and medium | |
| WO2016037440A1 (en) | Video voice conversion method and device and server | |
| US9412359B2 (en) | System and method for cloud-based text-to-speech web services | |
| CN108492818B (en) | Text-to-speech conversion method and device and computer equipment | |
| CN110136715B (en) | Speech recognition method and device | |
| CN112365878B (en) | Speech synthesis method, device, equipment and computer readable storage medium | |
| CN113793591A (en) | Speech synthesis method and related device, electronic equipment and storage medium | |
| CN111554281A (en) | Vehicle-mounted man-machine interaction method for automatically identifying languages, vehicle-mounted terminal and storage medium | |
| CN114596870A (en) | Real-time audio processing method and device, computer storage medium and electronic equipment | |
| CN112927677B (en) | Speech synthesis method and device | |
| CN114360485A (en) | Voice processing method, system, device and medium | |
| CN112185340B (en) | Speech synthesis method, speech synthesis device, storage medium and electronic equipment | |
| CN111105781B (en) | Voice processing method, device, electronic equipment and medium | |
| CN113257218A (en) | Speech synthesis method, speech synthesis device, electronic equipment and storage medium | |
| CN115798455B (en) | Speech synthesis method, system, electronic device and storage medium | |
| CN113808576A (en) | Voice conversion method, device and computer system | |
| CN116129859A (en) | Prosody labeling method, acoustic model training method, voice synthesis method and voice synthesis device | |
| CN113948062B (en) | Data conversion method and computer storage medium | |
| CN116229935A (en) | Speech synthesis method, device, electronic equipment and computer readable medium | |
| Shankarappa et al. | A faster approach for direct speech to speech translation | |
| US20110010179A1 (en) | Voice synthesis and processing | |
| JP2019197149A (en) | Pitch emphasis device, method thereof, and program |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |