CN111667810A - Method and device for acquiring polyphone corpus, readable medium and electronic equipment - Google Patents

Method and device for acquiring polyphone corpus, readable medium and electronic equipment Download PDF

Info

Publication number
CN111667810A
CN111667810A CN202010515132.3A CN202010515132A CN111667810A CN 111667810 A CN111667810 A CN 111667810A CN 202010515132 A CN202010515132 A CN 202010515132A CN 111667810 A CN111667810 A CN 111667810A
Authority
CN
China
Prior art keywords
text
target
pinyin sequence
voice synthesis
polyphone
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202010515132.3A
Other languages
Chinese (zh)
Other versions
CN111667810B (en
Inventor
潘俊杰
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Youzhuju Network Technology Co Ltd
Original Assignee
Beijing Youzhuju Network Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Youzhuju Network Technology Co Ltd filed Critical Beijing Youzhuju Network Technology Co Ltd
Priority to CN202010515132.3A priority Critical patent/CN111667810B/en
Publication of CN111667810A publication Critical patent/CN111667810A/en
Application granted granted Critical
Publication of CN111667810B publication Critical patent/CN111667810B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G10MUSICAL INSTRUMENTS; ACOUSTICS
    • G10LSPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
    • G10L13/00Speech synthesis; Text to speech systems
    • G10L13/02Methods for producing synthetic speech; Speech synthesisers
    • GPHYSICS
    • G10MUSICAL INSTRUMENTS; ACOUSTICS
    • G10LSPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
    • G10L13/00Speech synthesis; Text to speech systems
    • G10L13/02Methods for producing synthetic speech; Speech synthesisers
    • G10L13/04Details of speech synthesis systems, e.g. synthesiser structure or memory management
    • GPHYSICS
    • G10MUSICAL INSTRUMENTS; ACOUSTICS
    • G10LSPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
    • G10L13/00Speech synthesis; Text to speech systems
    • G10L13/08Text analysis or generation of parameters for speech synthesis out of text, e.g. grapheme to phoneme translation, prosody generation or stress or intonation determination
    • G10L13/10Prosody rules derived from text; Stress or intonation
    • GPHYSICS
    • G10MUSICAL INSTRUMENTS; ACOUSTICS
    • G10LSPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00Speech recognition
    • G10L15/06Creation of reference templates; Training of speech recognition systems, e.g. adaptation to the characteristics of the speaker's voice
    • G10L15/063Training
    • GPHYSICS
    • G10MUSICAL INSTRUMENTS; ACOUSTICS
    • G10LSPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00Speech recognition
    • G10L15/08Speech classification or search
    • G10L15/16Speech classification or search using artificial neural networks
    • GPHYSICS
    • G10MUSICAL INSTRUMENTS; ACOUSTICS
    • G10LSPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00Speech recognition
    • G10L15/22Procedures used during a speech recognition process, e.g. man-machine dialogue
    • GPHYSICS
    • G10MUSICAL INSTRUMENTS; ACOUSTICS
    • G10LSPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00Speech recognition
    • G10L15/26Speech to text systems
    • GPHYSICS
    • G10MUSICAL INSTRUMENTS; ACOUSTICS
    • G10LSPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
    • G10L25/00Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
    • G10L25/27Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 characterised by the analysis technique
    • G10L25/30Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 characterised by the analysis technique using neural networks

Abstract

The disclosure relates to a method and a device for acquiring polyphone corpus, a readable medium and electronic equipment, which relate to the technical field of voice processing, and the method comprises the following steps: the method comprises the steps of obtaining a target text, wherein the target text comprises at least one polyphone, inputting the target text into a plurality of voice synthesis models respectively to obtain intermediate audio output by each voice synthesis model, wherein each voice synthesis model is different from one another, inputting the intermediate audio output by each voice synthesis model into a voice recognition model to obtain an intermediate pinyin sequence output by each voice recognition model, corresponding to the voice synthesis model, determining a target pinyin sequence according to the intermediate pinyin sequences corresponding to the plurality of voice synthesis models, and taking the target text and the target pinyin sequence as polyphone linguistic data. According to the method and the device, the polyphone corpus can be quickly obtained through the combination of the voice recognition model and the voice synthesis models, manual labeling is not needed, and the efficiency and the accuracy of obtaining the polyphone corpus are improved.

Description

Method and device for acquiring polyphone corpus, readable medium and electronic equipment
Technical Field
The present disclosure relates to the field of speech processing technologies, and in particular, to a method and an apparatus for acquiring polyphone corpus, a readable medium, and an electronic device.
Background
With the continuous development of computer technology and voice processing technology, voice is widely used in daily life and work as an important carrier for people to obtain information. Speech processing typically includes two parts: speech synthesis and speech recognition. The speech synthesis refers to synthesizing a text designated by a user into audio, and the speech recognition refers to recognizing the audio designated by the user as the text. For a scene with an audio content in chinese or a scene with a text content in chinese, since chinese includes a large number of polyphones, pronunciation of an audio output by speech synthesis may be inaccurate or a text output by speech recognition may be inaccurate. Therefore, a large amount of polyphonic corpora needs to be obtained in advance to assist speech synthesis and speech recognition. However, polyphonic corpora are usually obtained by manually labeling a large amount of text, which is inefficient and less accurate.
Disclosure of Invention
This summary is provided to introduce a selection of concepts in a simplified form that are further described below in the detailed description. This summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter.
In a first aspect, the present disclosure provides a method for obtaining polyphonic corpus, where the method includes:
acquiring a target text, wherein the target text comprises at least one polyphone;
respectively inputting the target text into a plurality of voice synthesis models to obtain intermediate audio output by each voice synthesis model, wherein each voice synthesis model is different;
aiming at each voice synthesis model, inputting the intermediate audio output by the voice synthesis model into a voice recognition model to obtain an intermediate pinyin sequence output by the voice recognition model and corresponding to the voice synthesis model;
and determining a target pinyin sequence according to the intermediate pinyin sequence corresponding to each of the plurality of voice synthesis models, and taking the target text and the target pinyin sequence as a polyphone corpus.
In a second aspect, the present disclosure provides an apparatus for obtaining polyphonic corpus, the apparatus comprising:
the text acquisition module is used for acquiring a target text, and the target text comprises at least one polyphone;
the audio acquisition module is used for respectively inputting the target text into a plurality of voice synthesis models so as to acquire intermediate audio output by each voice synthesis model, and each voice synthesis model is different from each other;
the pinyin obtaining module is used for inputting the intermediate audio output by the voice synthesis model into the voice recognition model aiming at each voice synthesis model so as to obtain an intermediate pinyin sequence output by the voice recognition model and corresponding to the voice synthesis model;
and the determining module is used for determining a target pinyin sequence according to the intermediate pinyin sequence corresponding to each of the plurality of voice synthesis models, and taking the target text and the target pinyin sequence as a polyphone corpus.
According to the technical scheme, the method comprises the steps of firstly obtaining a target text comprising at least one polyphone, then respectively inputting the target text into a plurality of different voice synthesis models, outputting an intermediate audio frequency by each voice synthesis model, inputting the intermediate audio frequency output by each voice synthesis model into a voice recognition model so as to obtain an intermediate pinyin sequence output by the voice recognition model and corresponding to the voice synthesis model, finally determining a target pinyin sequence according to the intermediate pinyin sequence corresponding to each voice synthesis model, and taking the target text and the target pinyin sequence as polyphone linguistic data. According to the method and the device, the plurality of intermediate audios corresponding to the target text are obtained through the plurality of voice synthesis models, the plurality of intermediate audios are identified through the voice identification model, the target pinyin sequence corresponding to the target text is determined according to the obtained plurality of intermediate pinyin sequences, manual marking is not needed, a large amount of accurate polyphonic corpora can be rapidly obtained, and the efficiency and the accuracy of obtaining the polyphonic corpora are improved.
Additional features and advantages of the disclosure will be set forth in the detailed description which follows.
Drawings
The above and other features, advantages and aspects of various embodiments of the present disclosure will become more apparent by referring to the following detailed description when taken in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numbers refer to the same or similar elements. It should be understood that the drawings are schematic and that elements and features are not necessarily drawn to scale. In the drawings:
FIG. 1 is a flow diagram illustrating a method for obtaining polyphonic corpora in accordance with an exemplary embodiment;
FIG. 2 is a flow diagram illustrating another method of obtaining polyphonic corpora in accordance with an illustrative embodiment;
FIG. 3 is a flow diagram illustrating another method of obtaining polyphonic corpora in accordance with an illustrative embodiment;
FIG. 4 is a block diagram illustrating an apparatus for obtaining polyphonic corpora according to an exemplary embodiment;
FIG. 5 is a block diagram illustrating another apparatus for obtaining polyphonic corpora according to an example embodiment;
FIG. 6 is a block diagram illustrating another apparatus for obtaining polyphonic corpora according to an example embodiment;
FIG. 7 is a block diagram illustrating an electronic device in accordance with an example embodiment.
Detailed Description
Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While certain embodiments of the present disclosure are shown in the drawings, it is to be understood that the present disclosure may be embodied in various forms and should not be construed as limited to the embodiments set forth herein, but rather are provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the disclosure are for illustration purposes only and are not intended to limit the scope of the disclosure.
It should be understood that the various steps recited in the method embodiments of the present disclosure may be performed in a different order, and/or performed in parallel. Moreover, method embodiments may include additional steps and/or omit performing the illustrated steps. The scope of the present disclosure is not limited in this respect.
The term "include" and variations thereof as used herein are open-ended, i.e., "including but not limited to". The term "based on" is "based, at least in part, on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Relevant definitions for other terms will be given in the following description.
It should be noted that the terms "first", "second", and the like in the present disclosure are only used for distinguishing different devices, modules or units, and are not used for limiting the order or interdependence relationship of the functions performed by the devices, modules or units.
It is noted that references to "a", "an", and "the" modifications in this disclosure are intended to be illustrative rather than limiting, and that those skilled in the art will recognize that "one or more" may be used unless the context clearly dictates otherwise.
The names of messages or information exchanged between devices in the embodiments of the present disclosure are for illustrative purposes only, and are not intended to limit the scope of the messages or information.
Before introducing the method, the apparatus, the readable medium, and the electronic device for obtaining polyphonic corpora provided by the present disclosure, an application scenario related to each embodiment of the present disclosure is first introduced. The application scenario is an acquisition process of polyphonic corpora, which can be understood as a text-pinyin sequence pair, where the pinyin sequence includes the pinyin corresponding to each Chinese character in the text. For example, if the text is "hello o", and the corresponding pinyin sequence is "ni 3 hao2 hao3 a 5" (where the number indicates a tone, 1 indicates a first sound, 2 indicates a second sound, 3 indicates a third sound, 4 indicates a fourth sound, and 5 indicates a light sound), then "hello" -ni 3 hao2 hao3 a5 "may be used as a polyphonic corpus. Aiming at the process of voice synthesis, the specified text can be synthesized into audio through a voice synthesis model, and in the process, if polyphones appear in the specified text, the voice synthesis model can use the existing polyphone linguistic data as reference to judge which pronunciation (namely pinyin) the polyphones in the specified text are specific, so that the corresponding audio is generated. Therefore, in the speech synthesis process, in order to synthesize accurate audio, it is necessary to use polyphonic corpora. Aiming at the process of voice recognition, the appointed audio frequency can be recognized as a pinyin sequence through an acoustic model in a voice recognition model, then the pinyin sequence is recognized as a text through a language model in the voice recognition model, and in the process, the existing multi-voice word corpus is required to be used for training the language model to ensure the accuracy of recognizing the pinyin sequence as the text. Therefore, in the training process of speech recognition, the polyphonic corpus is also used.
Fig. 1 is a flowchart illustrating a method for obtaining polyphonic corpora according to an exemplary embodiment, where as shown in fig. 1, the method includes:
step 101, a target text is obtained, wherein the target text comprises at least one polyphone.
For example, to obtain the polyphone corpus, a target text including at least one polyphone may be obtained first, where the target text may be one or multiple. For example, a large amount of text may be randomly crawled from the internet, or obtained from a designated information source. Then, a large amount of texts are screened to obtain a target text comprising at least one polyphone. In common words, at least 140 polyphones are included, and the number of occurrences of each polyphone is greatly different, for example, three polyphones, "i", "di", "d", account for 80% of the total number of occurrences of the polyphones, while the other polyphones only account for 20% of the occurrences. Therefore, when there are a plurality of target texts, the target texts may be filtered according to a preset rule, so as to equalize the distribution of polyphones included in the target texts. For example, the number of occurrences of each polyphone in the target texts may be counted, and then some target texts including the polyphone with the largest occurrence number are filtered out, and the target texts including the polyphone with the smaller occurrence number are retained. For example, due to three polyphones of "in", and "out", which are usually present more times, the target text including only "in", and "out" can be filtered out. Furthermore, the purpose of balancing the distribution of polyphones in the target text can be achieved by limiting the number of the target texts containing the specified polyphones.
It should be noted that, in this embodiment, a language family to which the target text belongs may be predefined, and for example, a mandarin text including at least one polyphone may be used as the target text. When a plurality of target texts are available, each target text can be guaranteed to belong to the same language family, for example, each target text is a mandarin text.
And 102, respectively inputting the target text into a plurality of speech synthesis models to obtain the intermediate audio output by each speech synthesis model, wherein each speech synthesis model is different from each other.
Step 103, aiming at each voice synthesis model, inputting the intermediate audio output by the voice synthesis model into the voice recognition model to obtain the intermediate pinyin sequence output by the voice recognition model and corresponding to the voice synthesis model.
For example, a plurality of different Speech synthesis models may be preselected, and for example, different TTS (Text To Speech, Chinese: Text To Speech) systems may be obtained from different audio providers as the Speech synthesis models. The speech synthesis model may also be pre-trained, and the Neural Network used for speech synthesis may be, for example, RNN (current Neural Network, chinese), CNN (Convolutional Neural Network, chinese), LSTM (Long Short-Term Memory, chinese), etc., which is not specifically limited by this disclosure. The plurality of speech synthesis models can be obtained by utilizing different training data or selecting neural networks with different structures for training. And respectively inputting the target text into a plurality of speech synthesis models to obtain the intermediate audio output by each speech synthesis model. It can be understood that the target text is synthesized by using each speech synthesis model respectively, and the intermediate audio output by each speech synthesis model is obtained. For example, there are 3 speech synthesis models: TTS1, TTS2 and TTS3 respectively input the target texts into TTS1, TTS2 and TTS3 to obtain intermediate Audio1 output by TTS1, intermediate Audio2 output by TTS2 and intermediate Audio3 output by TTS 3.
And then, aiming at any one of the plurality of voice synthesis models, inputting the intermediate audio output by the voice synthesis model into a preset voice recognition model to obtain an intermediate pinyin sequence output by the voice recognition model and corresponding to the voice synthesis model. Wherein, the middle pinyin sequence at least comprises one pronunciation (including tone). The Speech Recognition model, for example, may be an ASR (Automatic Speech Recognition) system, which is capable of recognizing a pinyin sequence corresponding to an input audio. The speech recognition model may also be a pre-trained neural network for speech recognition. Each speech synthesis model corresponds to one intermediate audio sequence, namely the number of the intermediate audio sequences is the same as that of the speech synthesis models. For example, Audio1 may be input into the ASR system to obtain an intermediate pinyin sequence pinyin1 corresponding to TTS1, Audio2 may be input into the ASR system to obtain an intermediate pinyin sequence pinyin2 corresponding to TTS1, and Audio3 may be input into the ASR system to obtain an intermediate pinyin sequence pinyin3 corresponding to TTS 3.
And step 104, determining a target pinyin sequence according to the intermediate pinyin sequence corresponding to each voice synthesis model in the plurality of voice synthesis models, and taking the target text and the target pinyin sequence as polyphone linguistic data.
For example, the target pinyin sequence corresponding to the target text may be determined according to the intermediate pinyin sequence corresponding to each of the plurality of speech synthesis models. Specifically, the strategy for determining the target pinyin sequence may compare a plurality of intermediate pinyin sequences, and if the plurality of intermediate pinyin sequences are the same, any one of the intermediate pinyin sequences may be used as the target pinyin sequence. If different pinyin sequences exist in the multiple intermediate pinyin sequences, the multiple intermediate pinyin sequences are discarded, or the multiple intermediate pinyin sequences and the target text are sent to a specified service platform, and the service platform corrects the multiple intermediate pinyin sequences to select the target pinyin sequence. In another implementation, multiple intermediate pinyin sequences may be compared, and if different pinyin sequences exist in the multiple intermediate pinyin sequences, the occurrence frequency of each pinyin sequence may be counted, and the pinyin sequence with the highest occurrence frequency is taken as the target pinyin sequence. If different pinyin sequences exist in the multiple intermediate pinyin sequences and the occurrence frequency of each pinyin sequence is the same, discarding the multiple intermediate pinyin sequences, or sending the multiple intermediate pinyin sequences and the target text to a specified service platform, and correcting the multiple intermediate pinyin sequences by the service platform to select the target pinyin sequence.
It will be appreciated that a plurality of intermediate pinyin sequences are input to a voting system, and the voting system selects a target pinyin sequence among the plurality of intermediate pinyin sequences. Finally, the target text and the target pinyin sequence are used as polyphonic corpora, for example, the polyphonic corpora can be stored in a designated database for use in a speech synthesis process or a training process for speech recognition. Therefore, through the combination of the voice recognition model and the voice synthesis models, the target text is synthesized into a plurality of intermediate audios, then the intermediate audios are recognized into a plurality of intermediate pinyin sequences, and finally the target pinyin sequence is determined according to the intermediate pinyin sequences. Based on the Boosting principle, the accuracy of a target pinyin sequence obtained according to the combination of the voice recognition model and the voice synthesis models is higher than that of a pinyin sequence obtained by using one voice synthesis model. Thus, the process of obtaining the polyphone corpus in the embodiment can quickly obtain a large amount of accurate polyphone corpora without manual participation.
In summary, the present disclosure first obtains a target text including at least one polyphone, then inputs the target text into a plurality of different speech synthesis models, outputs an intermediate audio from each speech synthesis model, inputs the intermediate audio output from the speech synthesis model into a speech recognition model for each speech synthesis model to obtain an intermediate pinyin sequence output from the speech recognition model corresponding to the speech synthesis model, and finally determines a target pinyin sequence according to the intermediate pinyin sequence corresponding to each speech synthesis model, and uses the target text and the target pinyin sequence as a polyphone corpus. According to the method and the device, the plurality of intermediate audios corresponding to the target text are obtained through the plurality of voice synthesis models, the plurality of intermediate audios are identified through the voice identification model, the target pinyin sequence corresponding to the target text is determined according to the obtained plurality of intermediate pinyin sequences, manual marking is not needed, a large amount of accurate polyphonic corpora can be rapidly obtained, and the efficiency and the accuracy of obtaining the polyphonic corpora are improved.
Fig. 2 is a flowchart illustrating another method for obtaining polyphonic corpora according to an exemplary embodiment, where, as shown in fig. 2, step 101 may include the following steps:
at step 1011, a plurality of texts are obtained.
Step 1012, for each text, matching the text with a preset polyphone list, and determining the number of polyphones included in the text.
And 1013, determining whether the text is the target text according to the number of polyphones included in the text.
For example, to obtain the target text, a plurality of texts may be obtained, and then the target text may be screened from the plurality of texts. The multiple text acquisition channels may be texts randomly captured from the internet, or texts acquired from a specified information source (e.g., a database, a service platform, etc.). And then, matching each text with a preset polyphone list to determine whether the text comprises polyphones, and determining the number of the polyphones in the text in the case that the text comprises the polyphones. The polyphone list stores commonly used polyphone characters, each character in any text can be matched with the polyphone list, and if the characters are matched, the characters are determined to be polyphone characters.
And then, determining whether the text is the target text or not according to the number of polyphones included in the text. In commonly used polyphones, the number of occurrences of each polyphone is very different, for example, three polyphones, "ones" account for 80% of the total number of occurrences of the polyphones. Therefore, the plurality of texts can be filtered according to the number of polyphones included in the text, so that the distribution of the polyphones included in the plurality of target texts is balanced.
Specifically, the implementation manner of step 1013 may be:
and if the number of the polyphones in the text is greater than or equal to the first number and the number of records corresponding to the target polyphone in the text is less than the second number, taking the text as the target text, wherein the target polyphone is one or more polyphones in the text.
And updating the number of records corresponding to the target polyphones.
Taking the example that the polyphone list includes M polyphones, the number of target texts required for each polyphone is X (i.e. the second number). The initial value of the number of records for each polyphone may be set to 0 prior to performing step 1012. Acquiring N texts (N > > X), respectively acquiring the number of polyphones included in each text, and determining the number of records corresponding to each polyphone included in the text if the number of polyphones included in the text is greater than or equal to a first number (which may be 1, for example). If there is a target polyphone in the text whose number of corresponding records is less than the first number (i.e., X), the text may be used as the target text, and the number of records corresponding to the target polyphone is + 1. If the target polyphones are not present in the text (i.e., the number of records corresponding to each polyphone in the text is greater than or equal to the first number), the text may be discarded. Wherein, the target polyphones can be one or more. And repeating the steps until the number of records corresponding to each polyphone in the polyphone list is equal to X.
For example, the number of target texts required for the polyphone "music" is 100, and 36 target texts containing "music" have been recorded before, that is, the number of records corresponding to "music" is 36. If the new text is "i'm happy", it is determined that "i'm happy" contains "music" by matching with the polyphone list, the number of polyphones is equal to 1, and the number of records 36 corresponding to "music" is less than 100, then "i'm happy" may be taken as the target text, and the number of records corresponding to "music" is updated to 37. And continuing to judge the next text until the number of records corresponding to the music is 100.
Fig. 3 is a flowchart illustrating another method for obtaining polyphonic corpora according to an exemplary embodiment, where as shown in fig. 3, step 103 may include:
and step 1031, forcibly aligning the target text and the intermediate audio output by the speech synthesis model so as to label each frame of audio in the intermediate audio output by the speech synthesis model.
Step 1032, inputting the marked intermediate audio output by the speech synthesis model into the speech recognition model to obtain an intermediate pinyin sequence corresponding to the speech synthesis model.
For example, before the intermediate audio output by a certain speech synthesis model is input into the speech recognition model, the target text and the intermediate audio output by the speech synthesis model may be forcibly aligned, so as to label each frame of audio in the intermediate audio. Since the intermediate audio typically has more than one frame, it is necessary to determine which chinese character in the target text corresponds to each frame of audio in the intermediate audio. Specifically, the forced alignment of each frame of audio in the intermediate audio may be understood as adding a tag to each frame of audio in the intermediate audio, where the content of the tag is a serial number of the corresponding chinese character in the target text. For example, the duration of the intermediate audio is 10s, the intermediate audio includes 3000 frames of audio, and the target text includes 10 chinese characters, then, according to the corresponding relationship between the intermediate audio and the target text, 1-10 frames of audio are labeled as 1 (i.e., the first chinese character in the target text), 11-300 frames of audio are labeled as 2 (i.e., the second chinese character in the target text), 301-450 frames of audio are labeled as 3 (i.e., the third chinese character in the target text), and so on. And then, inputting the marked intermediate audio output by the voice synthesis model into the voice recognition model to obtain an intermediate pinyin sequence corresponding to the voice synthesis model.
Specifically, in an implementation manner, the implementation manner of step 104 may be:
if the middle pinyin sequences corresponding to each voice synthesis model are the same, the middle pinyin sequence corresponding to any voice synthesis model is taken as the target pinyin sequence.
For example, a plurality of intermediate pinyin sequences corresponding to a plurality of speech synthesis models may be compared, and if the plurality of intermediate pinyin sequences are the same, any one of the intermediate pinyin sequences may be used as the target pinyin sequence. If different pinyin sequences exist in the multiple intermediate pinyin sequences, the multiple intermediate pinyin sequences are discarded, or the multiple intermediate pinyin sequences and the target text are sent to a specified service platform, and the service platform corrects the multiple intermediate pinyin sequences to select the target pinyin sequence.
The target text is 'i want to drink cola', the plurality of speech synthesis models are TTS1, TTS2 and TTS3, and the speech recognition model is an ASR system for example. Firstly, TTS1, TTS2 and TTS3 are respectively input to 'I want to drink cola', TTS1 outputs intermediate Audio Audio1, TTS2 outputs intermediate Audio2, and TTS3 outputs intermediate Audio Audio 3. Then, the Audio1 is input into an ASR system to obtain an intermediate pinyin sequence 'wo 3xiang3 he1 ke3le 4' corresponding to TTS1, the Audio2 is input into the ASR system to obtain an intermediate pinyin sequence 'wo 3xiang3 he1 ke3le 4' corresponding to TTS2, and the Audio3 is input into the ASR system to obtain an intermediate pinyin sequence 'wo 3xiang3 ke 1 ke3le 4' corresponding to TTS 3. All three intermediate pinyin sequences are the same, so that the target audio sequence can be 'wo 3xiang3 he1 ke3le 4', and the target audio sequence can be 'i want to drink cola' and 'wo 3xiang3 he1 ke3le 4' can be used as polyphonic corpora.
In another implementation, the implementation of step 104 may be:
and determining the occurrence frequency of each intermediate pinyin sequence in the intermediate pinyin sequences corresponding to each voice synthesis model, and taking the intermediate pinyin sequence with the occurrence frequency meeting the preset condition as a target pinyin sequence.
For example, a plurality of intermediate pinyin sequences may be compared, and if different intermediate pinyin sequences exist in the plurality of intermediate pinyin sequences, the occurrence frequency of each intermediate pinyin sequence may be counted, and the intermediate pinyin sequence whose occurrence frequency satisfies a preset condition may be used as the target pinyin sequence. The preset condition may be, for example, the intermediate pinyin sequence with the largest occurrence number, or the intermediate pinyin sequence with the occurrence number larger than a preset threshold (e.g., 80%) in proportion to the total number of the intermediate pinyin sequences. If different pinyin sequences exist in the multiple intermediate pinyin sequences and the occurrence frequency of each pinyin sequence is the same, the multiple intermediate pinyin sequences can be discarded, or the multiple intermediate pinyin sequences and the target text are sent to a specified service platform, and the service platform corrects the multiple intermediate pinyin sequences to select the target pinyin sequence.
The target text is "slow vehicle running", the plurality of speech synthesis models are TTS1, TTS2, TTS3 and TTS4, and the speech recognition model is an ASR system for example. Firstly, TTS1, TTS2, TTS3 and TTS4 are respectively input to 'vehicle driving slowly', the TTS1 outputs intermediate Audio1, the TTS2 outputs intermediate Audio2, the TTS3 outputs intermediate Audio3, and the TTS4 outputs intermediate Audio 4. Then, Audio1 is input into an ASR system to obtain an intermediate pinyin sequence "che 1liang 4xing 2shi3 hu 3 man 4" corresponding to TTS1, Audio2 is input into the ASR system to obtain an intermediate pinyin sequence "che 1liang 4xing 2shi3 hu 3 man 4" corresponding to TTS2, Audio3 is input into the ASR system to obtain an intermediate pinyin sequence "che 1liang4 hang 2shi3 hu 3 man 4" corresponding to TTS3, and Audio4 is input into the ASR system to obtain an intermediate pinyin sequence "che 1liang 4xing 2shi3 man 3 man 4" corresponding to TTS 4. Of the four intermediate pinyin sequences, the occurrence frequency of 'che 1liang 4xing 2shi3 huan3 man 4' is 3, and the occurrence frequency of 'che 1liang4 hang 2shi3 huan3 man 4' is 1, so that 'che 1liang 4xing 2shi3 man 3 man 4' can be used as a target audio sequence, and 'car driving is slow' and 'che 1liang 4xing 2shi3 man 3 man 4' can be used as polyphonic linguistic data.
In summary, the present disclosure first obtains a target text including at least one polyphone, then inputs the target text into a plurality of different speech synthesis models, outputs an intermediate audio from each speech synthesis model, inputs the intermediate audio output from the speech synthesis model into a speech recognition model for each speech synthesis model to obtain an intermediate pinyin sequence output from the speech recognition model corresponding to the speech synthesis model, and finally determines a target pinyin sequence according to the intermediate pinyin sequence corresponding to each speech synthesis model, and uses the target text and the target pinyin sequence as a polyphone corpus. According to the method and the device, the plurality of intermediate audios corresponding to the target text are obtained through the plurality of voice synthesis models, the plurality of intermediate audios are identified through the voice identification model, the target pinyin sequence corresponding to the target text is determined according to the obtained plurality of intermediate pinyin sequences, manual marking is not needed, a large amount of accurate polyphonic corpora can be rapidly obtained, and the efficiency and the accuracy of obtaining the polyphonic corpora are improved.
Fig. 4 is a block diagram illustrating an apparatus for obtaining polyphonic corpora according to an exemplary embodiment, and as shown in fig. 4, the apparatus 200 includes:
the text obtaining module 201 is configured to obtain a target text, where the target text includes at least one polyphone.
The audio obtaining module 202 is configured to input the target text into a plurality of speech synthesis models respectively to obtain an intermediate audio output by each speech synthesis model, where each speech synthesis model is different from each other.
The pinyin obtaining module 203 is configured to, for each speech synthesis model, input the intermediate audio output by the speech synthesis model into the speech recognition model to obtain an intermediate pinyin sequence output by the speech recognition model and corresponding to the speech synthesis model.
The determining module 204 is configured to determine a target pinyin sequence according to the intermediate pinyin sequence corresponding to each of the plurality of voice synthesis models, and use the target text and the target pinyin sequence as a polyphonic corpus.
Fig. 5 is a block diagram illustrating another apparatus for acquiring polyphonic corpus according to an exemplary embodiment, and as shown in fig. 5, the text acquiring module 201 includes:
the text acquiring sub-module 2011 is configured to acquire a plurality of texts.
The matching sub-module 2012 is configured to match, for each text, the text with a preset polyphone list, and determine the number of polyphones included in the text.
The first determining submodule 2013 is configured to determine whether the text is the target text according to the number of polyphones included in the text.
Specifically, the first determining submodule 2013 may be configured to:
and if the number of the polyphones in the text is greater than or equal to the first number and the number of records corresponding to the target polyphone in the text is less than the second number, taking the text as the target text, wherein the target polyphone is one or more polyphones in the text.
And updating the number of records corresponding to the target polyphones.
Fig. 6 is a block diagram illustrating another apparatus for obtaining polyphonic corpus according to an exemplary embodiment, and as shown in fig. 6, the pinyin obtaining module 203 may include:
the alignment sub-module 2031 is configured to forcedly align the target text and the intermediate audio output by the speech synthesis model, so as to label each frame of audio in the intermediate audio output by the speech synthesis model.
The pinyin obtaining sub-module 2032 is configured to input the labeled intermediate audio output by the speech synthesis model into the speech recognition model to obtain an intermediate pinyin sequence corresponding to the speech synthesis model.
In one implementation scenario, the determining module 204 is configured to: if the middle pinyin sequences corresponding to each voice synthesis model are the same, the middle pinyin sequence corresponding to any voice synthesis model is taken as the target pinyin sequence.
In another implementation scenario, the determining module 204 is configured to: and determining the occurrence frequency of each intermediate pinyin sequence in the intermediate pinyin sequences corresponding to each voice synthesis model, and taking the intermediate pinyin sequence with the occurrence frequency meeting the preset condition as a target pinyin sequence.
With regard to the apparatus in the above-described embodiment, the specific manner in which each module performs the operation has been described in detail in the embodiment related to the method, and will not be elaborated here.
In summary, the present disclosure first obtains a target text including at least one polyphone, then inputs the target text into a plurality of different speech synthesis models, outputs an intermediate audio from each speech synthesis model, inputs the intermediate audio output from the speech synthesis model into a speech recognition model for each speech synthesis model to obtain an intermediate pinyin sequence output from the speech recognition model corresponding to the speech synthesis model, and finally determines a target pinyin sequence according to the intermediate pinyin sequence corresponding to each speech synthesis model, and uses the target text and the target pinyin sequence as a polyphone corpus. According to the method and the device, the plurality of intermediate audios corresponding to the target text are obtained through the plurality of voice synthesis models, the plurality of intermediate audios are identified through the voice identification model, the target pinyin sequence corresponding to the target text is determined according to the obtained plurality of intermediate pinyin sequences, manual marking is not needed, a large amount of accurate polyphonic corpora can be rapidly obtained, and the efficiency and the accuracy of obtaining the polyphonic corpora are improved.
Referring now to FIG. 7, a block diagram of an electronic device 300 suitable for use in implementing embodiments of the present disclosure is shown. The terminal device in the embodiments of the present disclosure may include, but is not limited to, a mobile terminal such as a mobile phone, a notebook computer, a digital broadcast receiver, a PDA (personal digital assistant), a PAD (tablet computer), a PMP (portable multimedia player), a vehicle terminal (e.g., a car navigation terminal), and the like, and a stationary terminal such as a digital TV, a desktop computer, and the like. The electronic device shown in fig. 7 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present disclosure.
As shown in fig. 7, the electronic device 300 may include a processing means (e.g., a central processing unit, a graphics processor, etc.) 301 that may perform various appropriate actions and processes in accordance with a program stored in a Read Only Memory (ROM)302 or a program loaded from a storage means 308 into a Random Access Memory (RAM) 303. In the RAM 303, various programs and data necessary for the operation of the electronic apparatus 300 are also stored. The processing device 301, the ROM 302, and the RAM 303 are connected to each other via a bus 304. An input/output (I/O) interface 305 is also connected to bus 304.
Generally, the following devices may be connected to the I/O interface 305: including input devices 306 such as touch screens, touch pads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc. Including output devices 307 such as Liquid Crystal Displays (LCDs), speakers, vibrators, etc. Storage devices 308 including, for example, magnetic tape, hard disk, etc.; and a communication device 309. The communication means 309 may allow the electronic device 300 to communicate wirelessly or by wire with other devices to exchange data. While fig. 7 illustrates an electronic device 300 having various means, it is to be understood that not all illustrated means are required to be implemented or provided. More or fewer devices may alternatively be implemented or provided.
In particular, according to an embodiment of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program carried on a non-transitory computer readable medium, the computer program containing program code for performing the method illustrated by the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network through the communication means 309, or installed from the storage means 308, or installed from the ROM 302. The computer program, when executed by the processing device 301, performs the above-described functions defined in the methods of the embodiments of the present disclosure.
It should be noted that the computer readable medium in the present disclosure can be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present disclosure, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In contrast, in the present disclosure, a computer readable signal medium may comprise a propagated data signal with computer readable program code embodied therein, either in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: electrical wires, optical cables, RF (radio frequency), etc., or any suitable combination of the foregoing.
In some embodiments, the terminal devices, servers may communicate using any currently known or future developed network protocol, such as HTTP (hypertext transfer protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communications network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), the Internet (e.g., the Internet), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future developed network.
The computer readable medium may be embodied in the electronic device; or may exist separately without being assembled into the electronic device.
The computer readable medium carries one or more programs which, when executed by the electronic device, cause the electronic device to: acquiring a target text, wherein the target text comprises at least one polyphone; respectively inputting the target text into a plurality of voice synthesis models to obtain intermediate audio output by each voice synthesis model, wherein each voice synthesis model is different; aiming at each voice synthesis model, inputting the intermediate audio output by the voice synthesis model into a voice recognition model to obtain an intermediate pinyin sequence output by the voice recognition model and corresponding to the voice synthesis model; and determining a target pinyin sequence according to the intermediate pinyin sequence corresponding to each of the plurality of voice synthesis models, and taking the target text and the target pinyin sequence as a polyphone corpus.
Computer program code for carrying out operations for the present disclosure may be written in any combination of one or more programming languages, including but not limited to an object oriented programming language such as Java, Smalltalk, C + +, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The modules described in the embodiments of the present disclosure may be implemented by software or hardware. The name of the module does not in some cases constitute a limitation of the module itself, and for example, the text acquisition module may also be described as a "module for acquiring a target text".
The functions described herein above may be performed, at least in part, by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Products (ASSPs), systems on a chip (SOCs), Complex Programmable Logic Devices (CPLDs), and the like.
In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
Example 1 provides a method of obtaining polyphonic corpora, including: acquiring a target text, wherein the target text comprises at least one polyphone; respectively inputting the target text into a plurality of voice synthesis models to obtain intermediate audio output by each voice synthesis model, wherein each voice synthesis model is different; aiming at each voice synthesis model, inputting the intermediate audio output by the voice synthesis model into a voice recognition model to obtain an intermediate pinyin sequence output by the voice recognition model and corresponding to the voice synthesis model; and determining a target pinyin sequence according to the intermediate pinyin sequence corresponding to each of the plurality of voice synthesis models, and taking the target text and the target pinyin sequence as a polyphone corpus.
Example 2 provides the method of example 1, the obtaining the target text comprising: acquiring a plurality of texts; aiming at each text, matching the text with a preset polyphone list, and determining the number of polyphones in the text; and determining whether the text is the target text or not according to the number of polyphones included in the text.
Example 3 provides the method of example 2, and the determining whether the text is the target text according to the number of polyphones included in the text includes: if the number of polyphones included in the text is larger than or equal to a first number and the number of records corresponding to the target polyphone included in the text is smaller than a second number, taking the text as the target text, wherein the target polyphone is one or more polyphones in the text; and updating the record quantity corresponding to the target polyphones.
Example 4 provides the method of example 1, where the inputting the intermediate audio output by the speech synthesis model into a speech recognition model to obtain an intermediate pinyin sequence output by the speech recognition model, where the intermediate pinyin sequence corresponds to the speech synthesis model includes: forcibly aligning the target text and the intermediate audio output by the speech synthesis model so as to label each frame of audio in the intermediate audio output by the speech synthesis model; and inputting the marked intermediate audio output by the voice synthesis model into the voice recognition model to obtain the intermediate pinyin sequence corresponding to the voice synthesis model.
Example 5 provides the method of example 1, wherein determining a target pinyin sequence based on the intermediate pinyin sequence corresponding to each of the plurality of speech synthesis models, including: and if the intermediate pinyin sequences corresponding to each voice synthesis model are the same, taking the intermediate pinyin sequence corresponding to any one voice synthesis model as a target pinyin sequence.
Example 6 provides the method of example 1, wherein determining a target pinyin sequence based on the intermediate pinyin sequence corresponding to each of the plurality of speech synthesis models, includes: and determining the occurrence frequency of each intermediate pinyin sequence in the intermediate pinyin sequences corresponding to each voice synthesis model, and taking the intermediate pinyin sequence with the occurrence frequency meeting the preset condition as the target pinyin sequence.
Example 7 provides an apparatus for acquiring polyphonic corpus, according to one or more embodiments of the present disclosure, comprising: the text acquisition module is used for acquiring a target text, and the target text comprises at least one polyphone; the audio acquisition module is used for respectively inputting the target text into a plurality of voice synthesis models so as to acquire intermediate audio output by each voice synthesis model, and each voice synthesis model is different from each other; the pinyin obtaining module is used for inputting the intermediate audio output by the voice synthesis model into the voice recognition model aiming at each voice synthesis model so as to obtain an intermediate pinyin sequence output by the voice recognition model and corresponding to the voice synthesis model; and the determining module is used for determining a target pinyin sequence according to the intermediate pinyin sequence corresponding to each of the plurality of voice synthesis models, and taking the target text and the target pinyin sequence as a polyphone corpus.
Example 8 provides the apparatus of example 7, the text acquisition module comprising: the text acquisition sub-module is used for acquiring a plurality of texts; the matching submodule is used for matching the text with a preset polyphone character list aiming at each text and determining the number of polyphone characters in the text; and the first determining submodule is used for determining whether the text is the target text or not according to the number of polyphones included in the text.
Example 9 provides a computer readable medium having stored thereon a computer program that, when executed by a processing apparatus, implements the steps of the methods of examples 1-6, in accordance with one or more embodiments of the present disclosure.
Example 10 provides, in accordance with one or more embodiments of the present disclosure, an electronic device comprising: a storage device having a computer program stored thereon; processing means for executing the computer program in the storage means to implement the steps of the methods of examples 1-6.
The foregoing description is only exemplary of the preferred embodiments of the disclosure and is illustrative of the principles of the technology employed. It will be appreciated by those skilled in the art that the scope of the disclosure herein is not limited to the particular combination of features described above, but also encompasses other embodiments in which any combination of the features described above or their equivalents does not depart from the spirit of the disclosure. For example, the above features and (but not limited to) the features disclosed in this disclosure having similar functions are replaced with each other to form the technical solution.
Further, while operations are depicted in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Likewise, while several specific implementation details are included in the above discussion, these should not be construed as limitations on the scope of the disclosure. Certain features that are described in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination.
Although the subject matter has been described in language specific to structural features and/or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims. With regard to the apparatus in the above-described embodiment, the specific manner in which each module performs the operation has been described in detail in the embodiment related to the method, and will not be elaborated here.

Claims (10)

1. A method for obtaining polyphone corpus is characterized by comprising the following steps:
acquiring a target text, wherein the target text comprises at least one polyphone;
respectively inputting the target text into a plurality of voice synthesis models to obtain intermediate audio output by each voice synthesis model, wherein each voice synthesis model is different;
aiming at each voice synthesis model, inputting the intermediate audio output by the voice synthesis model into a voice recognition model to obtain an intermediate pinyin sequence output by the voice recognition model and corresponding to the voice synthesis model;
and determining a target pinyin sequence according to the intermediate pinyin sequence corresponding to each of the plurality of voice synthesis models, and taking the target text and the target pinyin sequence as a polyphone corpus.
2. The method of claim 1, wherein obtaining the target text comprises:
acquiring a plurality of texts;
aiming at each text, matching the text with a preset polyphone list, and determining the number of polyphones in the text;
and determining whether the text is the target text or not according to the number of polyphones included in the text.
3. The method of claim 2, wherein determining whether the text is the target text according to the number of polyphones included in the text comprises:
if the number of polyphones included in the text is larger than or equal to a first number and the number of records corresponding to the target polyphone included in the text is smaller than a second number, taking the text as the target text, wherein the target polyphone is one or more polyphones in the text;
and updating the record quantity corresponding to the target polyphones.
4. The method of claim 1, wherein inputting the intermediate audio output by the speech synthesis model into a speech recognition model to obtain an intermediate pinyin sequence output by the speech recognition model corresponding to the speech synthesis model comprises:
forcibly aligning the target text and the intermediate audio output by the speech synthesis model so as to label each frame of audio in the intermediate audio output by the speech synthesis model;
and inputting the marked intermediate audio output by the voice synthesis model into the voice recognition model to obtain an intermediate pinyin sequence corresponding to the voice synthesis model.
5. The method of claim 1, wherein determining a target pinyin sequence based on the intermediate pinyin sequence associated with each of the plurality of speech synthesis models includes:
and if the intermediate pinyin sequences corresponding to each voice synthesis model are the same, taking the intermediate pinyin sequence corresponding to any one voice synthesis model as a target pinyin sequence.
6. The method of claim 1, wherein determining a target pinyin sequence based on the intermediate pinyin sequence associated with each of the plurality of speech synthesis models includes:
and determining the occurrence frequency of each intermediate pinyin sequence in the intermediate pinyin sequences corresponding to each voice synthesis model, and taking the intermediate pinyin sequence with the occurrence frequency meeting the preset condition as the target pinyin sequence.
7. An apparatus for obtaining polyphonic corpus, the apparatus comprising:
the text acquisition module is used for acquiring a target text, and the target text comprises at least one polyphone;
the audio acquisition module is used for respectively inputting the target text into a plurality of voice synthesis models so as to acquire intermediate audio output by each voice synthesis model, and each voice synthesis model is different from each other;
the pinyin obtaining module is used for inputting the intermediate audio output by the voice synthesis model into the voice recognition model aiming at each voice synthesis model so as to obtain an intermediate pinyin sequence output by the voice recognition model and corresponding to the voice synthesis model;
and the determining module is used for determining a target pinyin sequence according to the intermediate pinyin sequence corresponding to each of the plurality of voice synthesis models, and taking the target text and the target pinyin sequence as a polyphone corpus.
8. The apparatus of claim 7, wherein the text acquisition module comprises:
the text acquisition sub-module is used for acquiring a plurality of texts;
the matching submodule is used for matching the text with a preset polyphone character list aiming at each text and determining the number of polyphone characters in the text;
and the first determining submodule is used for determining whether the text is the target text or not according to the number of polyphones included in the text.
9. A computer-readable medium, on which a computer program is stored, characterized in that the program, when being executed by processing means, carries out the steps of the method of any one of claims 1 to 6.
10. An electronic device, comprising:
a storage device having a computer program stored thereon;
processing means for executing the computer program in the storage means to carry out the steps of the method according to any one of claims 1 to 6.
CN202010515132.3A 2020-06-08 2020-06-08 Method and device for acquiring polyphone corpus, readable medium and electronic equipment Active CN111667810B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010515132.3A CN111667810B (en) 2020-06-08 2020-06-08 Method and device for acquiring polyphone corpus, readable medium and electronic equipment

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010515132.3A CN111667810B (en) 2020-06-08 2020-06-08 Method and device for acquiring polyphone corpus, readable medium and electronic equipment

Publications (2)

Publication Number Publication Date
CN111667810A true CN111667810A (en) 2020-09-15
CN111667810B CN111667810B (en) 2021-10-15

Family

ID=72387040

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010515132.3A Active CN111667810B (en) 2020-06-08 2020-06-08 Method and device for acquiring polyphone corpus, readable medium and electronic equipment

Country Status (1)

Country Link
CN (1) CN111667810B (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112309366A (en) * 2020-11-03 2021-02-02 北京有竹居网络技术有限公司 Speech synthesis method, speech synthesis device, storage medium and electronic equipment
CN113408291A (en) * 2021-07-09 2021-09-17 平安国际智慧城市科技股份有限公司 Training method, device and equipment for Chinese entity recognition model and storage medium

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105225657A (en) * 2015-10-22 2016-01-06 百度在线网络技术(北京)有限公司 Polyphone mark template generation method and device
CN105336322A (en) * 2015-09-30 2016-02-17 百度在线网络技术(北京)有限公司 Polyphone model training method, and speech synthesis method and device
CN109977361A (en) * 2019-03-01 2019-07-05 广州多益网络股份有限公司 A kind of Chinese phonetic alphabet mask method, device and storage medium based on similar word
CN110188171A (en) * 2019-05-30 2019-08-30 上海联影医疗科技有限公司 A kind of voice search method, device, electronic equipment and storage medium
CN110534100A (en) * 2019-08-27 2019-12-03 北京海天瑞声科技股份有限公司 A kind of Chinese speech proofreading method and device based on speech recognition
CN110956948A (en) * 2020-01-03 2020-04-03 北京海天瑞声科技股份有限公司 End-to-end speech synthesis method, device and storage medium
CN111144110A (en) * 2019-12-27 2020-05-12 科大讯飞股份有限公司 Pinyin marking method, device, server and storage medium

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105336322A (en) * 2015-09-30 2016-02-17 百度在线网络技术(北京)有限公司 Polyphone model training method, and speech synthesis method and device
CN105225657A (en) * 2015-10-22 2016-01-06 百度在线网络技术(北京)有限公司 Polyphone mark template generation method and device
CN109977361A (en) * 2019-03-01 2019-07-05 广州多益网络股份有限公司 A kind of Chinese phonetic alphabet mask method, device and storage medium based on similar word
CN110188171A (en) * 2019-05-30 2019-08-30 上海联影医疗科技有限公司 A kind of voice search method, device, electronic equipment and storage medium
CN110534100A (en) * 2019-08-27 2019-12-03 北京海天瑞声科技股份有限公司 A kind of Chinese speech proofreading method and device based on speech recognition
CN111144110A (en) * 2019-12-27 2020-05-12 科大讯飞股份有限公司 Pinyin marking method, device, server and storage medium
CN110956948A (en) * 2020-01-03 2020-04-03 北京海天瑞声科技股份有限公司 End-to-end speech synthesis method, device and storage medium

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
于夏薇等: "融合语料库的论文作者姓名中英自动翻译研究", 《探索与研究》 *

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112309366A (en) * 2020-11-03 2021-02-02 北京有竹居网络技术有限公司 Speech synthesis method, speech synthesis device, storage medium and electronic equipment
CN112309366B (en) * 2020-11-03 2022-06-14 北京有竹居网络技术有限公司 Speech synthesis method, speech synthesis device, storage medium and electronic equipment
CN113408291A (en) * 2021-07-09 2021-09-17 平安国际智慧城市科技股份有限公司 Training method, device and equipment for Chinese entity recognition model and storage medium

Also Published As

Publication number Publication date
CN111667810B (en) 2021-10-15

Similar Documents

Publication Publication Date Title
CN111667810B (en) Method and device for acquiring polyphone corpus, readable medium and electronic equipment
CN110971969B (en) Video dubbing method and device, electronic equipment and computer readable storage medium
CN111583903B (en) Speech synthesis method, vocoder training method, device, medium, and electronic device
CN110969012A (en) Text error correction method and device, storage medium and electronic equipment
CN112115706B (en) Text processing method and device, electronic equipment and medium
CN111897976A (en) Virtual image synthesis method and device, electronic equipment and storage medium
CN111767740A (en) Sound effect adding method and device, storage medium and electronic equipment
CN112242143B (en) Voice interaction method and device, terminal equipment and storage medium
CN111986655B (en) Audio content identification method, device, equipment and computer readable medium
CN112669816A (en) Model training method, speech recognition method, device, medium and equipment
CN112883968A (en) Image character recognition method, device, medium and electronic equipment
CN112906381A (en) Recognition method and device of conversation affiliation, readable medium and electronic equipment
CN111916050A (en) Speech synthesis method, speech synthesis device, storage medium and electronic equipment
CN111460214A (en) Classification model training method, audio classification method, device, medium and equipment
CN110675865A (en) Method and apparatus for training hybrid language recognition models
CN111562864B (en) Picture display method, electronic device and computer readable medium
CN112489620B (en) Speech synthesis method, device, readable medium and electronic equipment
CN110675896B (en) Character time alignment method, device and medium for audio and electronic equipment
CN111935541B (en) Video correction method and device, readable medium and electronic equipment
US20210201895A1 (en) Method and apparatus for outputting analysis abnormality information in spoken language understanding
CN114697760A (en) Processing method, processing device, electronic equipment and medium
CN112883967A (en) Image character recognition method, device, medium and electronic equipment
CN111859902A (en) Text processing method, device, equipment and medium
CN112364653A (en) Text analysis method, apparatus, server and medium for speech synthesis
CN110931014A (en) Speech recognition method and device based on regular matching rule

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant