CN112037769A - A training data generation method, apparatus, and computer-readable storage medium - Google Patents

Abstract

The invention discloses a training data generation method, device and computer-readable storage medium, comprising: receiving audio information and corresponding marked text information; generating speech recognition text information and first time stamp information corresponding to the audio information; The content matches the marked text information and the speech recognition text information, and second timestamp information corresponding to the marked text information is generated according to the first timestamp information; and the marked text is obtained according to the second timestamp information sub-text training information in the information and sub-audio training information in the audio information. By obtaining the original audio information and annotated text information, and using the timestamp information of the audio information to obtain multiple sub-audio training information and corresponding sub-text training information from the original audio information and the annotated text information, a large amount of high-quality speech can be obtained. training data, this process is efficient and reduces cost.

Description

A training data generation method, apparatus, and computer-readable storage medium

technical field

The present invention relates to the field of artificial intelligence, and in particular, to a training data generation method, device and computer-readable storage medium.

Background technique

Training a speech recognition system requires a large amount of training data labeled with speech and text. The existing solutions for obtaining training data are to obtain speech, and the annotator will label the text corresponding to the speech through the speech annotation system; or specify a large amount of text, and different speech Humans record speech based on specified text. Through a large number of manual recordings and annotations, the training data for training the speech recognition system can be obtained. Existing solutions for obtaining training data require a lot of manpower and time costs, and it is very difficult to obtain a large amount of high-quality speech training data, resulting in a lack of training sets for training speech recognition systems.

SUMMARY OF THE INVENTION

Embodiments of the present invention provide a training data generating method, apparatus, and computer-readable storage medium, which have the technical effects of efficiently acquiring a large amount of high-quality speech training data and reducing cost.

One aspect of the present invention provides a method for generating training data, the method comprising: receiving audio information and corresponding labeled text information; generating speech recognition text information and first timestamp information corresponding to the audio information; Annotating text information and speech recognition text information, generating second timestamp information corresponding to the annotating text information according to the first timestamp information; Text training information and sub-audio training information in the audio information.

In a possible implementation manner, the content matching the annotated text information and the speech recognition text information includes: using an edit distance algorithm to perform text similarity matching on the annotated text information and the speech recognition text information; The information is used as a reference, and text alignment processing is performed on the words/words in the matched speech and text information.

In a possible implementation manner, the generating second time stamp information corresponding to the marked text information according to the first time stamp information includes: acquiring the speech recognition text information from the first time stamp information For each word/word information in the marked text information, copy the corresponding words/words in the speech recognition text information The start time stamp information and the end time stamp information of the information are used to generate second time stamp information corresponding to the marked text information.

In a possible implementation manner, before the content matches the marked text information and the speech recognition text information, the method includes: obtaining the confidence level corresponding to the word/word information in the speech recognition text information through a speech recognition system; Each confidence level of the word/word information is detected and replaced with the corresponding word/word information in the marked text information.

In a possible implementation manner, the acquiring, according to the second timestamp information, the sub-text training information in the annotated text information and the sub-audio training information in the audio information includes: According to the set character or the specified number of characters, it is divided into a plurality of sub-text training information, and respectively obtain a plurality of start timestamps and end timestamp information corresponding to the sub-text training information from the second timestamp information; The audio information is split into a plurality of sub-audio training information according to the start timestamp and end timestamp information corresponding to the plurality of sub-text training information.

In a possible implementation manner, before generating the speech recognition text information and the first time stamp information corresponding to the audio information, the method further comprises: inputting the marked text information into a language model in the speech recognition system for performing the processing. training, or dynamically increase the probability value of the marked text information when the speech recognition system performs decoding.

Another aspect of the present invention provides an apparatus for generating training data, the apparatus comprising: an information receiving module for receiving audio information and corresponding marked text information; a first information generating module for generating a training data corresponding to the audio information Speech recognition text information and first time stamp information; a second information generation module for content matching the marked text information and speech recognition text information, and generating a first time stamp corresponding to the marked text information according to the first time stamp information; Second timestamp information; a training data generation module, configured to acquire sub-text training information in the marked text information and sub-audio training information in the audio information according to the second timestamp information.

In a possible implementation manner, the second information generation module is specifically configured to: use an edit distance algorithm to perform text similarity matching between the annotated text information and the speech recognition text information; The words/words in the matched voice text information are processed for text alignment.

In a possible implementation manner, the training data generation module is specifically configured to: split the labeled text information into multiple sub-text training information according to a set character or a specified number of characters, and extract the data from the second timestamp information. Respectively obtain the start time stamp and end time stamp information corresponding to the plurality of sub-text training information; according to the start time stamp and end time stamp information corresponding to the plurality of sub-text training information, Split into multiple sub-audio training information.

Another aspect of the present invention provides a computer-readable storage medium, the storage medium comprising a set of computer-executable instructions for performing the training data generation method described in any one of the above when the instructions are executed.

In the embodiment of the present invention, by obtaining the original audio information and the marked text information, the timestamp information of the audio information is used to obtain a plurality of sub-audio training information and corresponding sub-text training information from the original audio information and the marked text information, thereby Obtaining a large amount of high-quality speech training data, this process is efficient and reduces the cost.

Description of drawings

The above and other objects, features and advantages of exemplary embodiments of the present invention will become readily understood by reading the following detailed description with reference to the accompanying drawings. In the accompanying drawings, several embodiments of the present invention are shown by way of example and not limitation, wherein:

In the drawings, the same or corresponding reference numerals denote the same or corresponding parts.

1 is a schematic diagram of an implementation flow of a method for generating training data according to an embodiment of the present invention;

FIG. 2 is a schematic structural composition diagram of an apparatus for generating training data according to an embodiment of the present invention.

Detailed ways

In order to make the objectives, features and advantages of the present invention more obvious and understandable, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described The embodiments are only some of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative efforts shall fall within the protection scope of the present invention.

FIG. 1 is a schematic diagram of an implementation flowchart of a method for generating training data according to an embodiment of the present invention.

As shown in FIG. 1, one aspect of the present invention provides a method for generating training data, the method comprising:

Step 101, receiving audio information and corresponding marked text information;

Step 102, generating speech recognition text information and first time stamp information corresponding to the audio information;

Step 103, the content matches the marked text information and the speech recognition text information, and generates second time stamp information corresponding to the marked text information, wherein the second time stamp information corresponds to the first time stamp information;

Step 104: Acquire sub-text training information in the marked text information and sub-audio training information in the audio information according to the second timestamp information.

In this embodiment, in step 101, the audio information and the corresponding marked text information are preferably long audio and long marked text information, which can be audio books, speech audio, interview records, etc. fetched from the database or from a local database.

In step 102, the speech recognition text information and the first time stamp information can be obtained by inputting the received audio information into an existing speech recognition system or through manual measurement; the first time stamp information includes text corresponding to the speech recognition The start and end timestamp information of each word or word in the information, for example, the marked text information is: "The sky is very hot, and the glaciers in the Antarctic of the earth have collapsed." Suppose the text information of speech recognition is "It is very hot. Revealed" timestamp information may be:

It is very hot: [day, 19.83, 20.49], [very, 20.49, 20.79], [hot, 20.79, 21.00];

Earth South Pole: [Earth, 21.90, 22.05], [South Pole, 22.05, 22.62];

Glacier Reveal: [Glacier, 23.67, 24.00], [ Reveal, 24.00, 24.24].

In step 103, content matching is performed between the marked text information and the speech recognition text information, so that second time stamp information corresponding to the marked text information is generated.

Next, in step 104, the sub-text training information in the marked text information and the sub-audio training information in the audio information are acquired according to the second timestamp information.

Therefore, by obtaining the original audio information and the labeled text information, and using the timestamp information of the audio information to obtain a plurality of sub-audio training information and corresponding sub-text training information from the original audio information and the labeled text information, a large number of high-quality audio sub-text training information can be obtained. High-quality speech training data, the process is efficient and cost-effective.

In an embodiment, the content matches the annotation text information and the speech recognition text information, including:

Use edit distance algorithm to perform text similarity matching between labeled text information and speech recognition text information;

Based on the marked text information, text alignment processing is performed on the words/words in the matched voice text information.

In this embodiment, the edit distance algorithm refers to the minimum number of editing operations required to convert two character strings from one to the other, and the larger the distance between them, the lower the similarity is.

When the similarity is matched, the labeled text information and the speech recognition text information can be split into multiple long sentence information or word-level sentence information according to punctuation marks or word segmentation tools, respectively, and the long sentence information in the labeled text information and the speech recognition text information can be split. The sentence information or word-level sentence information is matched for pairwise similarity, and the two sentences with the highest similarity are selected as matching.

Then perform text alignment processing. In the processing process, you can first use the existing word segmentation tool to perform word segmentation processing on the matching voice text information to obtain multiple word/word information, and then use the marked text information as the benchmark to classify the voice text information. The words/words in the text correspond to the words/words in the marked text information, and the unaligned parts can be filled by adding specific symbols to complete content matching. For the above example, the alignment is expressed as:

Label text: It is very hot, and the glaciers in the south pole of the earth have collapsed;

Identifying text: It's hot __Earth's South Pole__ glaciers are all exposed__ (underscores indicate blanks).

In a possible implementation manner, generating the second time stamp information corresponding to the marked text information according to the first time stamp information includes:

Obtain the start time stamp information and the end time stamp information corresponding to each word/word information in the speech recognition text information from the first time stamp information;

For each word/word information in the marked text information, copy the start time stamp information and the end time stamp information corresponding to the matching word/word information in the speech recognition text information, and generate the second time stamp information corresponding to the marked text information .

In this embodiment, the generation process of the second time stamp information is specifically:

After the content matching is completed, obtain the start time stamp information and end time stamp information corresponding to each word/word information in the speech recognition text information, and copy the obtained start time stamp information and end time stamp information according to the character index For the word/word corresponding to the index position in the marked text information, second timestamp information corresponding to the marked text information is generated.

In a possible implementation manner, before the content matches the annotation text information and the speech recognition text information, the method includes:

Obtain the confidence level corresponding to the word/word information in the speech recognition text information through the speech recognition system;

According to the confidence of each word/word information, the corresponding word/word information in the annotated text information is detected and replaced.

In this embodiment, there may be errors in the marked text information obtained in step 101, for example, "reveal" in "the sky is very hot and the Antarctic glaciers are exposed" is an identification error. Therefore, before performing step 103, while utilizing the speech recognition system to generate speech recognition text information, the confidence level of each word/word in the speech recognition system is obtained;

If the confidence of each word/word exceeds the preset threshold, it is determined that the accuracy of the word/word is high, and at this time, it is detected and judged whether the corresponding word/word in the marked text information is consistent in content. If the content of the corresponding words/words is inconsistent, the corresponding word/word information in the marked text information will be replaced, for example, "the sky is very hot and the Antarctic glaciers on the earth are exposed" with "the sky is very hot and the Antarctic glaciers on the earth are sinking". Through this step, the amount of calculation in the above-mentioned edit distance algorithm can be reduced, thereby improving the operation efficiency.

In a possible implementation manner, obtaining sub-text training information in the marked text information and sub-audio training information in the audio information according to the second timestamp information, including:

Splitting the labeled text information into a plurality of sub-text training information according to the set characters or the specified number of characters, and respectively obtaining the start timestamp and end timestamp information corresponding to the plurality of sub-text training information from the second timestamp information;

The audio information is split into multiple sub-audio training information according to the start timestamp and end timestamp information corresponding to the multiple sub-text training information.

In this embodiment, the specific process of step 104 is:

After the second timestamp information is generated, the index position of the punctuation mark in the marked text information is detected or the index position to be cut is located according to the specified number of characters, and the marked text information is divided into multiple sub-text training information according to the index position, such as Divide "it is very hot, the glaciers in the south pole of the earth have collapsed" into "it is very hot" and "the glaciers in the south pole of the earth have collapsed".

Then, the starting timestamp information and the ending timestamp information in the training information of each sub-text are obtained from the second timestamp information, such as [19.83, 21.00] for "it's very hot".

Split the audio information according to the starting timestamp information and the ending timestamp information of the sub-text training information, obtain a plurality of word audio information corresponding to the sub-text training information, and use the sub-text training information and the corresponding word audio information as training data.

In a possible implementation manner, before generating the speech recognition text information and the first timestamp information corresponding to the audio information, the method further includes:

Input the marked text information into the language model in the speech recognition system for training, or dynamically increase the probability value of the marked text information when the speech recognition system decodes.

In this embodiment, considering that the text information of speech recognition obtained by the speech recognition system may not have high accuracy, before step 102 is executed, the acquired text information is input into the language model in the speech recognition system for training , or dynamically increase the probability value of generating marked text information during the decoding process of the audio information by the speech recognition system, so as to improve the accuracy of the audio information recognition by the speech recognition system.

FIG. 2 is a schematic structural composition diagram of an apparatus for generating training data according to an embodiment of the present invention.

As shown in FIG. 2, another aspect of the present invention provides a training data generating device, the device comprising:

an information receiving module 201, configured to receive audio information and corresponding marked text information;

a first information generation module 202, configured to generate speech recognition text information and first timestamp information corresponding to the audio information;

The second information generation module 203 is configured to match the marked text information with the speech recognition text information, and generate second timestamp information corresponding to the marked text information according to the first timestamp information;

The training data generating module 204 is configured to obtain sub-text training information in the marked text information and sub-audio training information in the audio information according to the second timestamp information.

In this embodiment, in the information receiving module 201, the audio information and the corresponding marked text information are preferably long audio and long marked text information, which can be audio books, speech audio, interview records, etc., and the acquisition method can be obtained through the crawler technology Fetch from the web or from a local database.

In the first information generation module 202, the speech recognition text information and the first time stamp information can be obtained by inputting the received audio information into an existing speech recognition system or by manual measurement; the first time stamp information includes corresponding Based on the start and end timestamp information of each word or word in the speech recognition text information, for example, the text information is marked as: "The sky is very hot, the glaciers in the South Pole of the earth have collapsed", assuming that the speech recognition text information is "The sky is very hot. The time stamp information of the earth's Antarctic glaciers is revealed" may be:

It is very hot: [day, 19.83, 20.49], [very, 20.49, 20.79], [hot, 20.79, 21.00];

Earth South Pole: [Earth, 21.90, 22.05], [South Pole, 22.05, 22.62];

Glacier Reveal: [Glacier, 23.67, 24.00], [ Reveal, 24.00, 24.24].

In the second information generation module 203, content matching is performed between the marked text information and the speech recognition text information, so that second time stamp information corresponding to the marked text information is generated.

Next, in the training data generating module 204, according to the second timestamp information, the sub-text training information in the marked text information and the sub-audio training information in the audio information are acquired.

Therefore, by obtaining the original audio information and the labeled text information, and using the timestamp information of the audio information to obtain a plurality of sub-audio training information and corresponding sub-text training information from the original audio information and the labeled text information, a large number of high-quality audio sub-text training information can be obtained. High-quality speech training data, the process is efficient and cost-effective.

In a possible implementation manner, the second information generation module 203 is specifically configured to:

Use edit distance algorithm to perform text similarity matching between labeled text information and speech recognition text information;

Based on the marked text information, text alignment processing is performed on the words/words in the matched voice text information.

In this embodiment, the edit distance algorithm refers to the minimum number of editing operations required to convert two character strings from one to the other, and the larger the distance between them, the lower the similarity is.

When the similarity is matched, the labeled text information and the speech recognition text information can be split into multiple long sentence information or word-level sentence information according to punctuation marks or word segmentation tools, respectively, and the long sentence information in the labeled text information and the speech recognition text information can be split. The sentence information or word-level sentence information is matched for pairwise similarity, and the two sentences with the highest similarity are selected as matching.

Then perform text alignment processing. In the processing process, you can first use the existing word segmentation tool to perform word segmentation processing on the matching voice text information to obtain multiple word/word information, and then use the marked text information as the benchmark to classify the voice text information. The words/words in the text correspond to the words/words in the marked text information, and the unaligned parts can be filled by adding specific symbols to complete content matching. For the above example, the alignment is expressed as:

Label text: It is very hot, and the glaciers in the south pole of the earth have collapsed;

Identifying text: It's hot __Earth's South Pole__ glaciers are all exposed__ (underscores indicate blanks).

In a possible implementation manner, the training data generation module 204 is specifically used for:

Splitting the labeled text information into a plurality of sub-text training information according to the set characters or the specified number of characters, and respectively obtaining the start timestamp and end timestamp information corresponding to the plurality of sub-text training information from the second timestamp information;

The audio information is split into multiple sub-audio training information according to the start timestamp and end timestamp information corresponding to the multiple sub-text training information.

In this embodiment, the training data generation module 204 is specifically used for:

After the second timestamp information is generated, the index position of the punctuation mark in the marked text information is detected or the index position to be cut is located according to the specified number of characters, and the marked text information is divided into multiple sub-text training information according to the index position, such as Divide "it is very hot, the glaciers in the south pole of the earth have collapsed" into "it is very hot" and "the glaciers in the south pole of the earth have collapsed".

Then, the starting timestamp information and the ending timestamp information in the training information of each sub-text are obtained from the second timestamp information, such as [19.83, 21.00] for "it's very hot".

Split the audio information according to the starting timestamp information and the ending timestamp information of the sub-text training information, obtain a plurality of word audio information corresponding to the sub-text training information, and use the sub-text training information and the corresponding word audio information as training data.

Another aspect of the present invention provides a computer-readable storage medium, the storage medium comprising a set of computer-executable instructions, when executed, for performing any of the above-mentioned training data generation methods.

In this embodiment of the present invention, the computer-readable storage medium includes a set of computer-executable instructions, which, when the instructions are executed, are used to receive audio information and corresponding marked text information; to generate speech recognition text information corresponding to the audio information and first timestamp information; the content matches the marked text information and the speech recognition text information, and generates second timestamp information corresponding to the marked text information, wherein the second timestamp information corresponds to the first timestamp information; according to the second timestamp information , to obtain the sub-text training information in the labeled text information and the sub-audio training information in the audio information. Therefore, by obtaining the original audio information and the labeled text information, and using the timestamp information of the audio information to obtain a plurality of sub-audio training information and corresponding sub-text training information from the original audio information and the labeled text information, a large number of high-quality audio sub-text training information can be obtained. High-quality speech training data, the process is efficient and cost-effective.

In the description of this specification, description with reference to the terms "one embodiment," "some embodiments," "example," "specific example," or "some examples", etc., mean specific features described in connection with the embodiment or example , structure, material or feature is included in at least one embodiment or example of the present invention. Furthermore, the particular features, structures, materials or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, those skilled in the art may combine and combine the different embodiments or examples described in this specification, as well as the features of the different embodiments or examples, without conflicting each other.

In addition, the terms "first" and "second" are only used for descriptive purposes, and should not be construed as indicating or implying relative importance or implying the number of indicated technical features. Thus, a feature delimited with "first", "second" may expressly or implicitly include at least one of that feature. In the description of the present invention, "plurality" means two or more, unless otherwise expressly and specifically defined.

The above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited to this. Any person skilled in the art who is familiar with the technical scope disclosed by the present invention can easily think of changes or replacements, which should cover within the protection scope of the present invention. Therefore, the protection scope of the present invention should be subject to the protection scope of the claims.

Claims (10)

1. A method of generating training data, the method comprising:

receiving audio information and corresponding labeled text information;

generating voice recognition text information and first time stamp information corresponding to the audio information;

the content is matched with the labeled text information and the voice recognition text information, and second timestamp information corresponding to the labeled text information is generated according to the first timestamp information;

and acquiring sub-text training information in the labeled text information and sub-audio training information in the audio information according to the second timestamp information.

2. The method of claim 1, wherein the content matches the annotation text information and speech recognition text information, comprising:

performing text similarity matching on the marked text information and the voice recognition text information by using an editing distance algorithm;

and performing text alignment processing on the characters/words in the matched voice text information by taking the marked text information as a reference.

3. The method of claim 2, wherein the generating second timestamp information corresponding to the annotation text information from the first timestamp information comprises:

acquiring start timestamp information and end timestamp information corresponding to each word/word information in the voice recognition text information from the first timestamp information;

and aiming at each character/word information in the labeled text information, copying the starting time stamp information and the ending time stamp information corresponding to the matched character/word information in the voice recognition text information, and generating second time stamp information corresponding to the labeled text information.

4. The method of claim 2, wherein prior to content matching the annotation text information and speech recognition text information, the method comprises:

acquiring a confidence coefficient corresponding to the character/word information in the voice recognition text information through a voice recognition system;

and detecting and replacing the corresponding character/word information in the labeled text information according to the confidence coefficient of each character/word information.

5. The method of claim 1, wherein the obtaining sub-text training information in the labeled text information and sub-audio training information in the audio information according to the second timestamp information comprises:

dividing the labeled text information into a plurality of sub-text training information according to the number of set characters or designated characters, and respectively acquiring the starting time stamp information and the ending time stamp information corresponding to the plurality of sub-text training information from the second time stamp information;

and splitting the audio information into a plurality of pieces of sub audio training information according to the start time stamp information and the end time stamp information corresponding to the plurality of pieces of sub text training information.

6. The method of claim 1, wherein prior to generating speech recognition text information and first timestamp information corresponding to the audio information, the method further comprises:

and inputting the tagged text information into a language model in a voice recognition system for training, or dynamically increasing the probability value of the tagged text information when the voice recognition system decodes.

7. An apparatus for generating training data, the apparatus comprising:

the information receiving module is used for receiving the audio information and the corresponding labeled text information;

a first information generating module for generating speech recognition text information and first time stamp information corresponding to the audio information;

the second information generation module is used for matching the content with the labeled text information and the voice recognition text information and generating second timestamp information corresponding to the labeled text information according to the first timestamp information;

and the training data generation module is used for acquiring sub-text training information in the labeled text information and sub-audio training information in the audio information according to the second timestamp information.

8. The apparatus of claim 7, wherein the second information generating module is specifically configured to:

performing text similarity matching on the marked text information and the voice recognition text information by using an editing distance algorithm;

and performing text alignment processing on the characters/words in the matched voice text information by taking the marked text information as a reference.

9. The apparatus of claim 8, wherein the training data generation module is specifically configured to:

dividing the labeled text information into a plurality of sub-text training information according to the number of set characters or designated characters, and respectively acquiring the starting time stamp information and the ending time stamp information corresponding to the plurality of sub-text training information from the second time stamp information;

and splitting the audio information into a plurality of pieces of sub audio training information according to the start time stamp information and the end time stamp information corresponding to the plurality of pieces of sub text training information.

10. A computer-readable storage medium comprising a set of computer-executable instructions that, when executed, perform the training data generation method of any of claims 1-6.

Images