CN111667810A

CN111667810A - Method and device for acquiring polyphone corpus, readable medium and electronic equipment

Info

Publication number: CN111667810A
Application number: CN202010515132.3A
Authority: CN
Inventors: 潘俊杰
Original assignee: Beijing Youzhuju Network Technology Co Ltd
Current assignee: Beijing Youzhuju Network Technology Co Ltd
Priority date: 2020-06-08
Filing date: 2020-06-08
Publication date: 2020-09-15
Anticipated expiration: 2040-06-08
Also published as: CN111667810B

Abstract

The disclosure relates to a method and a device for acquiring polyphone corpus, a readable medium and electronic equipment, which relate to the technical field of voice processing, and the method comprises the following steps: the method comprises the steps of obtaining a target text, wherein the target text comprises at least one polyphone, inputting the target text into a plurality of voice synthesis models respectively to obtain intermediate audio output by each voice synthesis model, wherein each voice synthesis model is different from one another, inputting the intermediate audio output by each voice synthesis model into a voice recognition model to obtain an intermediate pinyin sequence output by each voice recognition model, corresponding to the voice synthesis model, determining a target pinyin sequence according to the intermediate pinyin sequences corresponding to the plurality of voice synthesis models, and taking the target text and the target pinyin sequence as polyphone linguistic data. According to the method and the device, the polyphone corpus can be quickly obtained through the combination of the voice recognition model and the voice synthesis models, manual labeling is not needed, and the efficiency and the accuracy of obtaining the polyphone corpus are improved.

Description

Method and device for acquiring polyphone corpus, readable medium and electronic equipment

Technical Field

The present disclosure relates to the field of speech processing technologies, and in particular, to a method and an apparatus for acquiring polyphone corpus, a readable medium, and an electronic device.

Background

With the continuous development of computer technology and voice processing technology, voice is widely used in daily life and work as an important carrier for people to obtain information. Speech processing typically includes two parts: speech synthesis and speech recognition. The speech synthesis refers to synthesizing a text designated by a user into audio, and the speech recognition refers to recognizing the audio designated by the user as the text. For a scene with an audio content in chinese or a scene with a text content in chinese, since chinese includes a large number of polyphones, pronunciation of an audio output by speech synthesis may be inaccurate or a text output by speech recognition may be inaccurate. Therefore, a large amount of polyphonic corpora needs to be obtained in advance to assist speech synthesis and speech recognition. However, polyphonic corpora are usually obtained by manually labeling a large amount of text, which is inefficient and less accurate.

Disclosure of Invention

This summary is provided to introduce a selection of concepts in a simplified form that are further described below in the detailed description. This summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter.

In a first aspect, the present disclosure provides a method for obtaining polyphonic corpus, where the method includes:

acquiring a target text, wherein the target text comprises at least one polyphone;

respectively inputting the target text into a plurality of voice synthesis models to obtain intermediate audio output by each voice synthesis model, wherein each voice synthesis model is different;

aiming at each voice synthesis model, inputting the intermediate audio output by the voice synthesis model into a voice recognition model to obtain an intermediate pinyin sequence output by the voice recognition model and corresponding to the voice synthesis model;

and determining a target pinyin sequence according to the intermediate pinyin sequence corresponding to each of the plurality of voice synthesis models, and taking the target text and the target pinyin sequence as a polyphone corpus.

In a second aspect, the present disclosure provides an apparatus for obtaining polyphonic corpus, the apparatus comprising:

the text acquisition module is used for acquiring a target text, and the target text comprises at least one polyphone;

the audio acquisition module is used for respectively inputting the target text into a plurality of voice synthesis models so as to acquire intermediate audio output by each voice synthesis model, and each voice synthesis model is different from each other;

the pinyin obtaining module is used for inputting the intermediate audio output by the voice synthesis model into the voice recognition model aiming at each voice synthesis model so as to obtain an intermediate pinyin sequence output by the voice recognition model and corresponding to the voice synthesis model;

and the determining module is used for determining a target pinyin sequence according to the intermediate pinyin sequence corresponding to each of the plurality of voice synthesis models, and taking the target text and the target pinyin sequence as a polyphone corpus.

According to the technical scheme, the method comprises the steps of firstly obtaining a target text comprising at least one polyphone, then respectively inputting the target text into a plurality of different voice synthesis models, outputting an intermediate audio frequency by each voice synthesis model, inputting the intermediate audio frequency output by each voice synthesis model into a voice recognition model so as to obtain an intermediate pinyin sequence output by the voice recognition model and corresponding to the voice synthesis model, finally determining a target pinyin sequence according to the intermediate pinyin sequence corresponding to each voice synthesis model, and taking the target text and the target pinyin sequence as polyphone linguistic data. According to the method and the device, the plurality of intermediate audios corresponding to the target text are obtained through the plurality of voice synthesis models, the plurality of intermediate audios are identified through the voice identification model, the target pinyin sequence corresponding to the target text is determined according to the obtained plurality of intermediate pinyin sequences, manual marking is not needed, a large amount of accurate polyphonic corpora can be rapidly obtained, and the efficiency and the accuracy of obtaining the polyphonic corpora are improved.

Additional features and advantages of the disclosure will be set forth in the detailed description which follows.

Drawings

The above and other features, advantages and aspects of various embodiments of the present disclosure will become more apparent by referring to the following detailed description when taken in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numbers refer to the same or similar elements. It should be understood that the drawings are schematic and that elements and features are not necessarily drawn to scale. In the drawings:

FIG. 1 is a flow diagram illustrating a method for obtaining polyphonic corpora in accordance with an exemplary embodiment;

FIG. 2 is a flow diagram illustrating another method of obtaining polyphonic corpora in accordance with an illustrative embodiment;

FIG. 3 is a flow diagram illustrating another method of obtaining polyphonic corpora in accordance with an illustrative embodiment;

FIG. 4 is a block diagram illustrating an apparatus for obtaining polyphonic corpora according to an exemplary embodiment;

FIG. 5 is a block diagram illustrating another apparatus for obtaining polyphonic corpora according to an example embodiment;

FIG. 6 is a block diagram illustrating another apparatus for obtaining polyphonic corpora according to an example embodiment;

FIG. 7 is a block diagram illustrating an electronic device in accordance with an example embodiment.

Detailed Description

Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While certain embodiments of the present disclosure are shown in the drawings, it is to be understood that the present disclosure may be embodied in various forms and should not be construed as limited to the embodiments set forth herein, but rather are provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the disclosure are for illustration purposes only and are not intended to limit the scope of the disclosure.

It should be understood that the various steps recited in the method embodiments of the present disclosure may be performed in a different order, and/or performed in parallel. Moreover, method embodiments may include additional steps and/or omit performing the illustrated steps. The scope of the present disclosure is not limited in this respect.

The term "include" and variations thereof as used herein are open-ended, i.e., "including but not limited to". The term "based on" is "based, at least in part, on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Relevant definitions for other terms will be given in the following description.

It should be noted that the terms "first", "second", and the like in the present disclosure are only used for distinguishing different devices, modules or units, and are not used for limiting the order or interdependence relationship of the functions performed by the devices, modules or units.

It is noted that references to "a", "an", and "the" modifications in this disclosure are intended to be illustrative rather than limiting, and that those skilled in the art will recognize that "one or more" may be used unless the context clearly dictates otherwise.

The names of messages or information exchanged between devices in the embodiments of the present disclosure are for illustrative purposes only, and are not intended to limit the scope of the messages or information.

Before introducing the method, the apparatus, the readable medium, and the electronic device for obtaining polyphonic corpora provided by the present disclosure, an application scenario related to each embodiment of the present disclosure is first introduced. The application scenario is an acquisition process of polyphonic corpora, which can be understood as a text-pinyin sequence pair, where the pinyin sequence includes the pinyin corresponding to each Chinese character in the text. For example, if the text is "hello o", and the corresponding pinyin sequence is "ni 3 hao2 hao3 a 5" (where the number indicates a tone, 1 indicates a first sound, 2 indicates a second sound, 3 indicates a third sound, 4 indicates a fourth sound, and 5 indicates a light sound), then "hello" -ni 3 hao2 hao3 a5 "may be used as a polyphonic corpus. Aiming at the process of voice synthesis, the specified text can be synthesized into audio through a voice synthesis model, and in the process, if polyphones appear in the specified text, the voice synthesis model can use the existing polyphone linguistic data as reference to judge which pronunciation (namely pinyin) the polyphones in the specified text are specific, so that the corresponding audio is generated. Therefore, in the speech synthesis process, in order to synthesize accurate audio, it is necessary to use polyphonic corpora. Aiming at the process of voice recognition, the appointed audio frequency can be recognized as a pinyin sequence through an acoustic model in a voice recognition model, then the pinyin sequence is recognized as a text through a language model in the voice recognition model, and in the process, the existing multi-voice word corpus is required to be used for training the language model to ensure the accuracy of recognizing the pinyin sequence as the text. Therefore, in the training process of speech recognition, the polyphonic corpus is also used.

Fig. 1 is a flowchart illustrating a method for obtaining polyphonic corpora according to an exemplary embodiment, where as shown in fig. 1, the method includes:

step 101, a target text is obtained, wherein the target text comprises at least one polyphone.

For example, to obtain the polyphone corpus, a target text including at least one polyphone may be obtained first, where the target text may be one or multiple. For example, a large amount of text may be randomly crawled from the internet, or obtained from a designated information source. Then, a large amount of texts are screened to obtain a target text comprising at least one polyphone. In common words, at least 140 polyphones are included, and the number of occurrences of each polyphone is greatly different, for example, three polyphones, "i", "di", "d", account for 80% of the total number of occurrences of the polyphones, while the other polyphones only account for 20% of the occurrences. Therefore, when there are a plurality of target texts, the target texts may be filtered according to a preset rule, so as to equalize the distribution of polyphones included in the target texts. For example, the number of occurrences of each polyphone in the target texts may be counted, and then some target texts including the polyphone with the largest occurrence number are filtered out, and the target texts including the polyphone with the smaller occurrence number are retained. For example, due to three polyphones of "in", and "out", which are usually present more times, the target text including only "in", and "out" can be filtered out. Furthermore, the purpose of balancing the distribution of polyphones in the target text can be achieved by limiting the number of the target texts containing the specified polyphones.

It should be noted that, in this embodiment, a language family to which the target text belongs may be predefined, and for example, a mandarin text including at least one polyphone may be used as the target text. When a plurality of target texts are available, each target text can be guaranteed to belong to the same language family, for example, each target text is a mandarin text.

And 102, respectively inputting the target text into a plurality of speech synthesis models to obtain the intermediate audio output by each speech synthesis model, wherein each speech synthesis model is different from each other.

Step 103, aiming at each voice synthesis model, inputting the intermediate audio output by the voice synthesis model into the voice recognition model to obtain the intermediate pinyin sequence output by the voice recognition model and corresponding to the voice synthesis model.

For example, a plurality of different Speech synthesis models may be preselected, and for example, different TTS (Text To Speech, Chinese: Text To Speech) systems may be obtained from different audio providers as the Speech synthesis models. The speech synthesis model may also be pre-trained, and the Neural Network used for speech synthesis may be, for example, RNN (current Neural Network, chinese), CNN (Convolutional Neural Network, chinese), LSTM (Long Short-Term Memory, chinese), etc., which is not specifically limited by this disclosure. The plurality of speech synthesis models can be obtained by utilizing different training data or selecting neural networks with different structures for training. And respectively inputting the target text into a plurality of speech synthesis models to obtain the intermediate audio output by each speech synthesis model. It can be understood that the target text is synthesized by using each speech synthesis model respectively, and the intermediate audio output by each speech synthesis model is obtained. For example, there are 3 speech synthesis models: TTS1, TTS2 and TTS3 respectively input the target texts into TTS1, TTS2 and TTS3 to obtain intermediate Audio1 output by TTS1, intermediate Audio2 output by TTS2 and intermediate Audio3 output by TTS 3.

And then, aiming at any one of the plurality of voice synthesis models, inputting the intermediate audio output by the voice synthesis model into a preset voice recognition model to obtain an intermediate pinyin sequence output by the voice recognition model and corresponding to the voice synthesis model. Wherein, the middle pinyin sequence at least comprises one pronunciation (including tone). The Speech Recognition model, for example, may be an ASR (Automatic Speech Recognition) system, which is capable of recognizing a pinyin sequence corresponding to an input audio. The speech recognition model may also be a pre-trained neural network for speech recognition. Each speech synthesis model corresponds to one intermediate audio sequence, namely the number of the intermediate audio sequences is the same as that of the speech synthesis models. For example, Audio1 may be input into the ASR system to obtain an intermediate pinyin sequence pinyin1 corresponding to TTS1, Audio2 may be input into the ASR system to obtain an intermediate pinyin sequence pinyin2 corresponding to TTS1, and Audio3 may be input into the ASR system to obtain an intermediate pinyin sequence pinyin3 corresponding to TTS 3.

And step 104, determining a target pinyin sequence according to the intermediate pinyin sequence corresponding to each voice synthesis model in the plurality of voice synthesis models, and taking the target text and the target pinyin sequence as polyphone linguistic data.

For example, the target pinyin sequence corresponding to the target text may be determined according to the intermediate pinyin sequence corresponding to each of the plurality of speech synthesis models. Specifically, the strategy for determining the target pinyin sequence may compare a plurality of intermediate pinyin sequences, and if the plurality of intermediate pinyin sequences are the same, any one of the intermediate pinyin sequences may be used as the target pinyin sequence. If different pinyin sequences exist in the multiple intermediate pinyin sequences, the multiple intermediate pinyin sequences are discarded, or the multiple intermediate pinyin sequences and the target text are sent to a specified service platform, and the service platform corrects the multiple intermediate pinyin sequences to select the target pinyin sequence. In another implementation, multiple intermediate pinyin sequences may be compared, and if different pinyin sequences exist in the multiple intermediate pinyin sequences, the occurrence frequency of each pinyin sequence may be counted, and the pinyin sequence with the highest occurrence frequency is taken as the target pinyin sequence. If different pinyin sequences exist in the multiple intermediate pinyin sequences and the occurrence frequency of each pinyin sequence is the same, discarding the multiple intermediate pinyin sequences, or sending the multiple intermediate pinyin sequences and the target text to a specified service platform, and correcting the multiple intermediate pinyin sequences by the service platform to select the target pinyin sequence.

It will be appreciated that a plurality of intermediate pinyin sequences are input to a voting system, and the voting system selects a target pinyin sequence among the plurality of intermediate pinyin sequences. Finally, the target text and the target pinyin sequence are used as polyphonic corpora, for example, the polyphonic corpora can be stored in a designated database for use in a speech synthesis process or a training process for speech recognition. Therefore, through the combination of the voice recognition model and the voice synthesis models, the target text is synthesized into a plurality of intermediate audios, then the intermediate audios are recognized into a plurality of intermediate pinyin sequences, and finally the target pinyin sequence is determined according to the intermediate pinyin sequences. Based on the Boosting principle, the accuracy of a target pinyin sequence obtained according to the combination of the voice recognition model and the voice synthesis models is higher than that of a pinyin sequence obtained by using one voice synthesis model. Thus, the process of obtaining the polyphone corpus in the embodiment can quickly obtain a large amount of accurate polyphone corpora without manual participation.

In summary, the present disclosure first obtains a target text including at least one polyphone, then inputs the target text into a plurality of different speech synthesis models, outputs an intermediate audio from each speech synthesis model, inputs the intermediate audio output from the speech synthesis model into a speech recognition model for each speech synthesis model to obtain an intermediate pinyin sequence output from the speech recognition model corresponding to the speech synthesis model, and finally determines a target pinyin sequence according to the intermediate pinyin sequence corresponding to each speech synthesis model, and uses the target text and the target pinyin sequence as a polyphone corpus. According to the method and the device, the plurality of intermediate audios corresponding to the target text are obtained through the plurality of voice synthesis models, the plurality of intermediate audios are identified through the voice identification model, the target pinyin sequence corresponding to the target text is determined according to the obtained plurality of intermediate pinyin sequences, manual marking is not needed, a large amount of accurate polyphonic corpora can be rapidly obtained, and the efficiency and the accuracy of obtaining the polyphonic corpora are improved.

Fig. 2 is a flowchart illustrating another method for obtaining polyphonic corpora according to an exemplary embodiment, where, as shown in fig. 2, step 101 may include the following steps:

at step 1011, a plurality of texts are obtained.

Step 1012, for each text, matching the text with a preset polyphone list, and determining the number of polyphones included in the text.

And 1013, determining whether the text is the target text according to the number of polyphones included in the text.

For example, to obtain the target text, a plurality of texts may be obtained, and then the target text may be screened from the plurality of texts. The multiple text acquisition channels may be texts randomly captured from the internet, or texts acquired from a specified information source (e.g., a database, a service platform, etc.). And then, matching each text with a preset polyphone list to determine whether the text comprises polyphones, and determining the number of the polyphones in the text in the case that the text comprises the polyphones. The polyphone list stores commonly used polyphone characters, each character in any text can be matched with the polyphone list, and if the characters are matched, the characters are determined to be polyphone characters.

And then, determining whether the text is the target text or not according to the number of polyphones included in the text. In commonly used polyphones, the number of occurrences of each polyphone is very different, for example, three polyphones, "ones" account for 80% of the total number of occurrences of the polyphones. Therefore, the plurality of texts can be filtered according to the number of polyphones included in the text, so that the distribution of the polyphones included in the plurality of target texts is balanced.

Specifically, the implementation manner of step 1013 may be:

and if the number of the polyphones in the text is greater than or equal to the first number and the number of records corresponding to the target polyphone in the text is less than the second number, taking the text as the target text, wherein the target polyphone is one or more polyphones in the text.

And updating the number of records corresponding to the target polyphones.

Taking the example that the polyphone list includes M polyphones, the number of target texts required for each polyphone is X (i.e. the second number). The initial value of the number of records for each polyphone may be set to 0 prior to performing step 1012. Acquiring N texts (N > > X), respectively acquiring the number of polyphones included in each text, and determining the number of records corresponding to each polyphone included in the text if the number of polyphones included in the text is greater than or equal to a first number (which may be 1, for example). If there is a target polyphone in the text whose number of corresponding records is less than the first number (i.e., X), the text may be used as the target text, and the number of records corresponding to the target polyphone is + 1. If the target polyphones are not present in the text (i.e., the number of records corresponding to each polyphone in the text is greater than or equal to the first number), the text may be discarded. Wherein, the target polyphones can be one or more. And repeating the steps until the number of records corresponding to each polyphone in the polyphone list is equal to X.

For example, the number of target texts required for the polyphone "music" is 100, and 36 target texts containing "music" have been recorded before, that is, the number of records corresponding to "music" is 36. If the new text is "i'm happy", it is determined that "i'm happy" contains "music" by matching with the polyphone list, the number of polyphones is equal to 1, and the number of records 36 corresponding to "music" is less than 100, then "i'm happy" may be taken as the target text, and the number of records corresponding to "music" is updated to 37. And continuing to judge the next text until the number of records corresponding to the music is 100.

Fig. 3 is a flowchart illustrating another method for obtaining polyphonic corpora according to an exemplary embodiment, where as shown in fig. 3, step 103 may include:

and step 1031, forcibly aligning the target text and the intermediate audio output by the speech synthesis model so as to label each frame of audio in the intermediate audio output by the speech synthesis model.

Step 1032, inputting the marked intermediate audio output by the speech synthesis model into the speech recognition model to obtain an intermediate pinyin sequence corresponding to the speech synthesis model.

For example, before the intermediate audio output by a certain speech synthesis model is input into the speech recognition model, the target text and the intermediate audio output by the speech synthesis model may be forcibly aligned, so as to label each frame of audio in the intermediate audio. Since the intermediate audio typically has more than one frame, it is necessary to determine which chinese character in the target text corresponds to each frame of audio in the intermediate audio. Specifically, the forced alignment of each frame of audio in the intermediate audio may be understood as adding a tag to each frame of audio in the intermediate audio, where the content of the tag is a serial number of the corresponding chinese character in the target text. For example, the duration of the intermediate audio is 10s, the intermediate audio includes 3000 frames of audio, and the target text includes 10 chinese characters, then, according to the corresponding relationship between the intermediate audio and the target text, 1-10 frames of audio are labeled as 1 (i.e., the first chinese character in the target text), 11-300 frames of audio are labeled as 2 (i.e., the second chinese character in the target text), 301-450 frames of audio are labeled as 3 (i.e., the third chinese character in the target text), and so on. And then, inputting the marked intermediate audio output by the voice synthesis model into the voice recognition model to obtain an intermediate pinyin sequence corresponding to the voice synthesis model.

Specifically, in an implementation manner, the implementation manner of step 104 may be:

if the middle pinyin sequences corresponding to each voice synthesis model are the same, the middle pinyin sequence corresponding to any voice synthesis model is taken as the target pinyin sequence.

For example, a plurality of intermediate pinyin sequences corresponding to a plurality of speech synthesis models may be compared, and if the plurality of intermediate pinyin sequences are the same, any one of the intermediate pinyin sequences may be used as the target pinyin sequence. If different pinyin sequences exist in the multiple intermediate pinyin sequences, the multiple intermediate pinyin sequences are discarded, or the multiple intermediate pinyin sequences and the target text are sent to a specified service platform, and the service platform corrects the multiple intermediate pinyin sequences to select the target pinyin sequence.

The target text is 'i want to drink cola', the plurality of speech synthesis models are TTS1, TTS2 and TTS3, and the speech recognition model is an ASR system for example. Firstly, TTS1, TTS2 and TTS3 are respectively input to 'I want to drink cola', TTS1 outputs intermediate Audio Audio1, TTS2 outputs intermediate Audio2, and TTS3 outputs intermediate Audio Audio 3. Then, the Audio1 is input into an ASR system to obtain an intermediate pinyin sequence 'wo 3xiang3 he1 ke3le 4' corresponding to TTS1, the Audio2 is input into the ASR system to obtain an intermediate pinyin sequence 'wo 3xiang3 he1 ke3le 4' corresponding to TTS2, and the Audio3 is input into the ASR system to obtain an intermediate pinyin sequence 'wo 3xiang3 ke 1 ke3le 4' corresponding to TTS 3. All three intermediate pinyin sequences are the same, so that the target audio sequence can be 'wo 3xiang3 he1 ke3le 4', and the target audio sequence can be 'i want to drink cola' and 'wo 3xiang3 he1 ke3le 4' can be used as polyphonic corpora.

In another implementation, the implementation of step 104 may be:

and determining the occurrence frequency of each intermediate pinyin sequence in the intermediate pinyin sequences corresponding to each voice synthesis model, and taking the intermediate pinyin sequence with the occurrence frequency meeting the preset condition as a target pinyin sequence.

For example, a plurality of intermediate pinyin sequences may be compared, and if different intermediate pinyin sequences exist in the plurality of intermediate pinyin sequences, the occurrence frequency of each intermediate pinyin sequence may be counted, and the intermediate pinyin sequence whose occurrence frequency satisfies a preset condition may be used as the target pinyin sequence. The preset condition may be, for example, the intermediate pinyin sequence with the largest occurrence number, or the intermediate pinyin sequence with the occurrence number larger than a preset threshold (e.g., 80%) in proportion to the total number of the intermediate pinyin sequences. If different pinyin sequences exist in the multiple intermediate pinyin sequences and the occurrence frequency of each pinyin sequence is the same, the multiple intermediate pinyin sequences can be discarded, or the multiple intermediate pinyin sequences and the target text are sent to a specified service platform, and the service platform corrects the multiple intermediate pinyin sequences to select the target pinyin sequence.

The target text is "slow vehicle running", the plurality of speech synthesis models are TTS1, TTS2, TTS3 and TTS4, and the speech recognition model is an ASR system for example. Firstly, TTS1, TTS2, TTS3 and TTS4 are respectively input to 'vehicle driving slowly', the TTS1 outputs intermediate Audio1, the TTS2 outputs intermediate Audio2, the TTS3 outputs intermediate Audio3, and the TTS4 outputs intermediate Audio 4. Then, Audio1 is input into an ASR system to obtain an intermediate pinyin sequence "che 1liang 4xing 2shi3 hu 3 man 4" corresponding to TTS1, Audio2 is input into the ASR system to obtain an intermediate pinyin sequence "che 1liang 4xing 2shi3 hu 3 man 4" corresponding to TTS2, Audio3 is input into the ASR system to obtain an intermediate pinyin sequence "che 1liang4 hang 2shi3 hu 3 man 4" corresponding to TTS3, and Audio4 is input into the ASR system to obtain an intermediate pinyin sequence "che 1liang 4xing 2shi3 man 3 man 4" corresponding to TTS 4. Of the four intermediate pinyin sequences, the occurrence frequency of 'che 1liang 4xing 2shi3 huan3 man 4' is 3, and the occurrence frequency of 'che 1liang4 hang 2shi3 huan3 man 4' is 1, so that 'che 1liang 4xing 2shi3 man 3 man 4' can be used as a target audio sequence, and 'car driving is slow' and 'che 1liang 4xing 2shi3 man 3 man 4' can be used as polyphonic linguistic data.

Fig. 4 is a block diagram illustrating an apparatus for obtaining polyphonic corpora according to an exemplary embodiment, and as shown in fig. 4, the apparatus 200 includes:

the text obtaining module 201 is configured to obtain a target text, where the target text includes at least one polyphone.

The audio obtaining module 202 is configured to input the target text into a plurality of speech synthesis models respectively to obtain an intermediate audio output by each speech synthesis model, where each speech synthesis model is different from each other.

The pinyin obtaining module 203 is configured to, for each speech synthesis model, input the intermediate audio output by the speech synthesis model into the speech recognition model to obtain an intermediate pinyin sequence output by the speech recognition model and corresponding to the speech synthesis model.

The determining module 204 is configured to determine a target pinyin sequence according to the intermediate pinyin sequence corresponding to each of the plurality of voice synthesis models, and use the target text and the target pinyin sequence as a polyphonic corpus.

Fig. 5 is a block diagram illustrating another apparatus for acquiring polyphonic corpus according to an exemplary embodiment, and as shown in fig. 5, the text acquiring module 201 includes:

the text acquiring sub-module 2011 is configured to acquire a plurality of texts.

The matching sub-module 2012 is configured to match, for each text, the text with a preset polyphone list, and determine the number of polyphones included in the text.

The first determining submodule 2013 is configured to determine whether the text is the target text according to the number of polyphones included in the text.

Specifically, the first determining submodule 2013 may be configured to:

And updating the number of records corresponding to the target polyphones.

Fig. 6 is a block diagram illustrating another apparatus for obtaining polyphonic corpus according to an exemplary embodiment, and as shown in fig. 6, the pinyin obtaining module 203 may include:

the alignment sub-module 2031 is configured to forcedly align the target text and the intermediate audio output by the speech synthesis model, so as to label each frame of audio in the intermediate audio output by the speech synthesis model.

The pinyin obtaining sub-module 2032 is configured to input the labeled intermediate audio output by the speech synthesis model into the speech recognition model to obtain an intermediate pinyin sequence corresponding to the speech synthesis model.

In one implementation scenario, the determining module 204 is configured to: if the middle pinyin sequences corresponding to each voice synthesis model are the same, the middle pinyin sequence corresponding to any voice synthesis model is taken as the target pinyin sequence.

In another implementation scenario, the determining module 204 is configured to: and determining the occurrence frequency of each intermediate pinyin sequence in the intermediate pinyin sequences corresponding to each voice synthesis model, and taking the intermediate pinyin sequence with the occurrence frequency meeting the preset condition as a target pinyin sequence.

With regard to the apparatus in the above-described embodiment, the specific manner in which each module performs the operation has been described in detail in the embodiment related to the method, and will not be elaborated here.

Referring now to FIG. 7, a block diagram of an electronic device 300 suitable for use in implementing embodiments of the present disclosure is shown. The terminal device in the embodiments of the present disclosure may include, but is not limited to, a mobile terminal such as a mobile phone, a notebook computer, a digital broadcast receiver, a PDA (personal digital assistant), a PAD (tablet computer), a PMP (portable multimedia player), a vehicle terminal (e.g., a car navigation terminal), and the like, and a stationary terminal such as a digital TV, a desktop computer, and the like. The electronic device shown in fig. 7 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present disclosure.

As shown in fig. 7, the electronic device 300 may include a processing means (e.g., a central processing unit, a graphics processor, etc.) 301 that may perform various appropriate actions and processes in accordance with a program stored in a Read Only Memory (ROM)302 or a program loaded from a storage means 308 into a Random Access Memory (RAM) 303. In the RAM 303, various programs and data necessary for the operation of the electronic apparatus 300 are also stored. The processing device 301, the ROM 302, and the RAM 303 are connected to each other via a bus 304. An input/output (I/O) interface 305 is also connected to bus 304.

Generally, the following devices may be connected to the I/O interface 305: including input devices 306 such as touch screens, touch pads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc. Including output devices 307 such as Liquid Crystal Displays (LCDs), speakers, vibrators, etc. Storage devices 308 including, for example, magnetic tape, hard disk, etc.; and a communication device 309. The communication means 309 may allow the electronic device 300 to communicate wirelessly or by wire with other devices to exchange data. While fig. 7 illustrates an electronic device 300 having various means, it is to be understood that not all illustrated means are required to be implemented or provided. More or fewer devices may alternatively be implemented or provided.

In particular, according to an embodiment of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program carried on a non-transitory computer readable medium, the computer program containing program code for performing the method illustrated by the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network through the communication means 309, or installed from the storage means 308, or installed from the ROM 302. The computer program, when executed by the processing device 301, performs the above-described functions defined in the methods of the embodiments of the present disclosure.

It should be noted that the computer readable medium in the present disclosure can be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present disclosure, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In contrast, in the present disclosure, a computer readable signal medium may comprise a propagated data signal with computer readable program code embodied therein, either in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: electrical wires, optical cables, RF (radio frequency), etc., or any suitable combination of the foregoing.

In some embodiments, the terminal devices, servers may communicate using any currently known or future developed network protocol, such as HTTP (hypertext transfer protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communications network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), the Internet (e.g., the Internet), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future developed network.

The computer readable medium may be embodied in the electronic device; or may exist separately without being assembled into the electronic device.

The computer readable medium carries one or more programs which, when executed by the electronic device, cause the electronic device to: acquiring a target text, wherein the target text comprises at least one polyphone; respectively inputting the target text into a plurality of voice synthesis models to obtain intermediate audio output by each voice synthesis model, wherein each voice synthesis model is different; aiming at each voice synthesis model, inputting the intermediate audio output by the voice synthesis model into a voice recognition model to obtain an intermediate pinyin sequence output by the voice recognition model and corresponding to the voice synthesis model; and determining a target pinyin sequence according to the intermediate pinyin sequence corresponding to each of the plurality of voice synthesis models, and taking the target text and the target pinyin sequence as a polyphone corpus.

Computer program code for carrying out operations for the present disclosure may be written in any combination of one or more programming languages, including but not limited to an object oriented programming language such as Java, Smalltalk, C + +, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).

The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

The modules described in the embodiments of the present disclosure may be implemented by software or hardware. The name of the module does not in some cases constitute a limitation of the module itself, and for example, the text acquisition module may also be described as a "module for acquiring a target text".

The functions described herein above may be performed, at least in part, by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Products (ASSPs), systems on a chip (SOCs), Complex Programmable Logic Devices (CPLDs), and the like.

In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

Example 1 provides a method of obtaining polyphonic corpora, including: acquiring a target text, wherein the target text comprises at least one polyphone; respectively inputting the target text into a plurality of voice synthesis models to obtain intermediate audio output by each voice synthesis model, wherein each voice synthesis model is different; aiming at each voice synthesis model, inputting the intermediate audio output by the voice synthesis model into a voice recognition model to obtain an intermediate pinyin sequence output by the voice recognition model and corresponding to the voice synthesis model; and determining a target pinyin sequence according to the intermediate pinyin sequence corresponding to each of the plurality of voice synthesis models, and taking the target text and the target pinyin sequence as a polyphone corpus.

Example 2 provides the method of example 1, the obtaining the target text comprising: acquiring a plurality of texts; aiming at each text, matching the text with a preset polyphone list, and determining the number of polyphones in the text; and determining whether the text is the target text or not according to the number of polyphones included in the text.

Example 3 provides the method of example 2, and the determining whether the text is the target text according to the number of polyphones included in the text includes: if the number of polyphones included in the text is larger than or equal to a first number and the number of records corresponding to the target polyphone included in the text is smaller than a second number, taking the text as the target text, wherein the target polyphone is one or more polyphones in the text; and updating the record quantity corresponding to the target polyphones.

Example 4 provides the method of example 1, where the inputting the intermediate audio output by the speech synthesis model into a speech recognition model to obtain an intermediate pinyin sequence output by the speech recognition model, where the intermediate pinyin sequence corresponds to the speech synthesis model includes: forcibly aligning the target text and the intermediate audio output by the speech synthesis model so as to label each frame of audio in the intermediate audio output by the speech synthesis model; and inputting the marked intermediate audio output by the voice synthesis model into the voice recognition model to obtain the intermediate pinyin sequence corresponding to the voice synthesis model.

Example 5 provides the method of example 1, wherein determining a target pinyin sequence based on the intermediate pinyin sequence corresponding to each of the plurality of speech synthesis models, including: and if the intermediate pinyin sequences corresponding to each voice synthesis model are the same, taking the intermediate pinyin sequence corresponding to any one voice synthesis model as a target pinyin sequence.

Example 6 provides the method of example 1, wherein determining a target pinyin sequence based on the intermediate pinyin sequence corresponding to each of the plurality of speech synthesis models, includes: and determining the occurrence frequency of each intermediate pinyin sequence in the intermediate pinyin sequences corresponding to each voice synthesis model, and taking the intermediate pinyin sequence with the occurrence frequency meeting the preset condition as the target pinyin sequence.

Example 7 provides an apparatus for acquiring polyphonic corpus, according to one or more embodiments of the present disclosure, comprising: the text acquisition module is used for acquiring a target text, and the target text comprises at least one polyphone; the audio acquisition module is used for respectively inputting the target text into a plurality of voice synthesis models so as to acquire intermediate audio output by each voice synthesis model, and each voice synthesis model is different from each other; the pinyin obtaining module is used for inputting the intermediate audio output by the voice synthesis model into the voice recognition model aiming at each voice synthesis model so as to obtain an intermediate pinyin sequence output by the voice recognition model and corresponding to the voice synthesis model; and the determining module is used for determining a target pinyin sequence according to the intermediate pinyin sequence corresponding to each of the plurality of voice synthesis models, and taking the target text and the target pinyin sequence as a polyphone corpus.

Example 8 provides the apparatus of example 7, the text acquisition module comprising: the text acquisition sub-module is used for acquiring a plurality of texts; the matching submodule is used for matching the text with a preset polyphone character list aiming at each text and determining the number of polyphone characters in the text; and the first determining submodule is used for determining whether the text is the target text or not according to the number of polyphones included in the text.

Example 9 provides a computer readable medium having stored thereon a computer program that, when executed by a processing apparatus, implements the steps of the methods of examples 1-6, in accordance with one or more embodiments of the present disclosure.

Example 10 provides, in accordance with one or more embodiments of the present disclosure, an electronic device comprising: a storage device having a computer program stored thereon; processing means for executing the computer program in the storage means to implement the steps of the methods of examples 1-6.

The foregoing description is only exemplary of the preferred embodiments of the disclosure and is illustrative of the principles of the technology employed. It will be appreciated by those skilled in the art that the scope of the disclosure herein is not limited to the particular combination of features described above, but also encompasses other embodiments in which any combination of the features described above or their equivalents does not depart from the spirit of the disclosure. For example, the above features and (but not limited to) the features disclosed in this disclosure having similar functions are replaced with each other to form the technical solution.

Further, while operations are depicted in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Likewise, while several specific implementation details are included in the above discussion, these should not be construed as limitations on the scope of the disclosure. Certain features that are described in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination.

Although the subject matter has been described in language specific to structural features and/or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims. With regard to the apparatus in the above-described embodiment, the specific manner in which each module performs the operation has been described in detail in the embodiment related to the method, and will not be elaborated here.

Claims

1. A method for obtaining polyphone corpus is characterized by comprising the following steps:

2. The method of claim 1, wherein obtaining the target text comprises:

acquiring a plurality of texts;

aiming at each text, matching the text with a preset polyphone list, and determining the number of polyphones in the text;

and determining whether the text is the target text or not according to the number of polyphones included in the text.

3. The method of claim 2, wherein determining whether the text is the target text according to the number of polyphones included in the text comprises:

if the number of polyphones included in the text is larger than or equal to a first number and the number of records corresponding to the target polyphone included in the text is smaller than a second number, taking the text as the target text, wherein the target polyphone is one or more polyphones in the text;

and updating the record quantity corresponding to the target polyphones.

4. The method of claim 1, wherein inputting the intermediate audio output by the speech synthesis model into a speech recognition model to obtain an intermediate pinyin sequence output by the speech recognition model corresponding to the speech synthesis model comprises:

forcibly aligning the target text and the intermediate audio output by the speech synthesis model so as to label each frame of audio in the intermediate audio output by the speech synthesis model;

and inputting the marked intermediate audio output by the voice synthesis model into the voice recognition model to obtain an intermediate pinyin sequence corresponding to the voice synthesis model.

5. The method of claim 1, wherein determining a target pinyin sequence based on the intermediate pinyin sequence associated with each of the plurality of speech synthesis models includes:

and if the intermediate pinyin sequences corresponding to each voice synthesis model are the same, taking the intermediate pinyin sequence corresponding to any one voice synthesis model as a target pinyin sequence.

6. The method of claim 1, wherein determining a target pinyin sequence based on the intermediate pinyin sequence associated with each of the plurality of speech synthesis models includes:

and determining the occurrence frequency of each intermediate pinyin sequence in the intermediate pinyin sequences corresponding to each voice synthesis model, and taking the intermediate pinyin sequence with the occurrence frequency meeting the preset condition as the target pinyin sequence.

7. An apparatus for obtaining polyphonic corpus, the apparatus comprising:

8. The apparatus of claim 7, wherein the text acquisition module comprises:

the text acquisition sub-module is used for acquiring a plurality of texts;

the matching submodule is used for matching the text with a preset polyphone character list aiming at each text and determining the number of polyphone characters in the text;

and the first determining submodule is used for determining whether the text is the target text or not according to the number of polyphones included in the text.

9. A computer-readable medium, on which a computer program is stored, characterized in that the program, when being executed by processing means, carries out the steps of the method of any one of claims 1 to 6.

10. An electronic device, comprising:

a storage device having a computer program stored thereon;

processing means for executing the computer program in the storage means to carry out the steps of the method according to any one of claims 1 to 6.