JP6426971B2 - Learning data generation device and program thereof - Google Patents

Description

  The present invention relates to a learning data generation apparatus and program for generating learning data necessary for adaptation of an acoustic model used for speech recognition of a broadcast program by quasi-supervised learning.

  Currently, in some sports programs and information programs, subtitles are produced using the lispeak method. In this lispeak method, subtitles are produced by speech recognition of program sound reproduced by a subtitle production re-speaker called subtitle caster (for example, non-patent document 1). The respeak method requires special reproduction technology and requires time to produce subtitles through the re-speaker. Therefore, a method capable of recognizing program sound in real time is desired regardless of the response method.

  In order to realize this, it is necessary to have an acoustic model that can accurately recognize voices of broadcast programs of various genres such as sports programs and information programs. At this time, a large-scale speech language corpus is required as learning data for constructing an acoustic model. The speech language corpus is required to have high accuracy in order to construct a practical level acoustic model.

  Conventionally, quasi-supervised learning has been proposed as a method for generating a speech language corpus (for example, Non-Patent Document 2). The technology described in Non-Patent Document 2 performs alignment from the speech recognition result of program speech and subtitle text, determines whether the speech recognition result and subtitle text match each speech section, and the coincident speech section To extract Then, the technology described in Non-Patent Document 2 uses voice data and subtitle text corresponding to the extracted utterance section for learning of an acoustic model.

Matsui et al., Real-time Captioning of Sports Relay by Lith Peak System using Paraphrase, Journal of the Institute of Electronics, Information and Communication Engineers, D-11, Information and System Processing, II-Pattern Processing, Vol. 87, No. 2, pp. 427- 435, 2004-02-01 Lamel et. Al, Lightly Supervised and Unsupervised Acoustic Model Training, Computer Speech and Language, Vol 6, pp. 115-129, 2002

However, the technology described in Non-Patent Document 2 has a problem that it can not generate a necessary amount of learning data when applied to broadcast programs of other genres because it is a news program.
Specifically, in information programs, background music and noise are often included, and performers other than announcers often do not utter correctly. For this reason, the technology described in Non-Patent Document 2 lowers the speech recognition accuracy of the information program even when using the acoustic model learned in the news program, and the word matching section between the speech recognition result and the subtitle text decreases. I will. As a result, the technique described in Non-Patent Document 2 can not generate a necessary amount of learning data.

  An object of the present invention is to provide a learning data generation device capable of generating more highly accurate learning data and a program thereof.

  In view of the above problems, a learning data generation apparatus according to the present invention is a learning data generation apparatus that generates learning data necessary for adaptation of an acoustic model used for speech recognition of a broadcast program by quasi-supervised learning. And a third language model generation unit, a speech recognition unit, an alignment unit, a substitution unit, and a learning data generation unit.

  According to this configuration, the learning data generation device linearly interpolates the first language model generated in advance from the text corpus and the second language model generated in advance from the subtitle text of the broadcast program by the third language model generation means. To generate a third language model.

  The learning data generation apparatus performs speech recognition of the broadcast program by the speech recognition unit using the third language model and the acoustic model generated in advance. Then, the learning data generation apparatus performs alignment by using alignment means to associate words in the speech recognition text and subtitle text representing the speech recognition result of the broadcast program in time order.

  Here, it is considered that the accuracy of speech recognition is lower than that of subtitle production. In addition, when the word associated between the speech recognition text and the subtitle text is different, and the word sequence preceding and following the word matches, the word included in the speech recognition text is erroneously recognized as speech recognition The possibilities are very high.

  Therefore, the learning data generation device uses the substitution unit to change the word for each word associated between the speech recognition text and the subtitle text, and the number of words set in advance before and after the word Whether or not the word is a replacement target is determined based on whether or not the sequences match. Then, when the word is a replacement target, the learning data generation device replaces the word of the speech recognition text with the word of the subtitle text by the substitution means.

  As described above, since the learning data generation device substitutes the words of the speech recognition text even when the words of the speech recognition text and the subtitle text do not match because the accuracy of the speech recognition is low, the speech recognition text and the subtitle text The word match interval of can be increased.

  The learning data generation device determines, by the learning data generation means, whether or not the speech recognition text and the caption text replaced by the replacement means match each other during the speech interval of the broadcast program, and the speech intervals determined to match The words of the subtitle text corresponding to the speech section are added as labels to the voice data of. At this time, since the word matching section of the speech recognition text and the caption text increases, the learning data generation apparatus also increases the speech section determined to match.

According to the present invention, the following excellent effects can be obtained.
The learning data generation device according to the present invention replaces words in the speech recognition text even when the words in the speech recognition text and the subtitle text do not match because the accuracy of speech recognition is low. As a result, the learning data generation device can generate more accurate learning data because the word matching section between the voice recognition text and the subtitle text increases.

FIG. 1 is a block diagram showing the configuration of an acoustic model generation device according to a first embodiment of the present invention. It is explanatory drawing explaining the substitution of the word in the acoustic model production | generation apparatus of FIG. It is a flowchart which shows operation | movement of the acoustic model production | generation apparatus of FIG. It is a block diagram which shows the structure of the acoustic model production | generation apparatus based on 2nd Embodiment of this invention. It is a flowchart which shows operation | movement of the acoustic model production | generation apparatus of FIG. In Example 1 of this invention, it is a graph which shows the relationship between the number of words, and the number of patterns which are different. In Example 2, 3 and a comparative example, it is a graph which shows the relationship between the frequency | count of adaptation of "close-up modern", and a speech language corpus. In Example 2, 3 and a comparative example, it is a graph which shows the relationship between the frequency | count of adaptation of "Marutokuki magazine", and a speech language corpus. In Example 2, 3 and a comparative example, it is a graph which shows the relationship between the frequency | count of adaptation of "Science ZERO", and a speech language corpus.

  Hereinafter, each embodiment of the present invention will be described in detail with reference to the drawings as appropriate. In each embodiment, means having the same function are denoted by the same reference numeral, and the description thereof is omitted.

First Embodiment
[Configuration of acoustic model generation device]
The configuration of an acoustic model generation device (learning data generation device) 1 according to a first embodiment of the present invention will be described with reference to FIG.
The acoustic model generation device 1 generates learning data necessary for adaptation of the acoustic model by quasi-supervised learning, and adapts (generates) the acoustic model using the generated learning data.
This acoustic model is not limited to news programs, and can be used for speech recognition of broadcast programs of various genres such as sports programs and information programs.

  As shown in FIG. 1, the acoustic model generation device 1 includes an adaptive language model generation unit (third language model generation unit) 10, a speech recognition unit 20, an alignment unit 30, a substitution unit 40, and a learning data generation unit. 50 and acoustic model adaptation means 60.

  The adaptive language model generation unit 10 generates an adaptive language model (third language model) by interpolating the baseline language model (first language model) and the domain language model (second language model). It is a thing.

The baseline language model is a language model generated in advance from a large-scale text corpus.
The domain language model is a language model generated in advance from subtitle text attached to a broadcast program.

  The adaptive language model generation means 10 receives a baseline language model and a domain language model. Then, the adaptive language model generation unit 10 linearly interpolates the input baseline language model and domain language model to generate an adaptive language model. At this time, the adaptive language model generation unit 10 weights the domain language model more heavily than the baseline language model.

  For example, suppose that a baseline language model and a domain language model are trigram language models. Also, in the baseline language model and the domain language model, there are entries of the same word chain “Today”, “Ha” and “I” as described below, and their scores (probabilities) are '7. It shall be 0 'and' 5.0 '. Also, assuming that the interpolation coefficient (weighting coefficient) of the domain language model is “0.9” and the interpolation coefficient of the baseline language model is “0.1”, the following is obtained.

<Example of each language model>
Baseline language model: "Today", "Ha", "I" Score 7.0
Domain language model: "Today", "Ha", "I" Score 5.0
Adaptive Language Model: "Today", "Ha", "I" Score 5.2

  In this example, the adaptive language model generation unit 10 obtains a multiplication value '0.7' of the score '7.0' of the baseline language model and the interpolation coefficient '0.1' of the baseline language model. Furthermore, the adaptive language model generation unit 10 obtains a multiplication value “4.5” of the domain language model score “5.0” and the domain language model interpolation coefficient “0.9”. Then, the adaptive language model generation unit 10 obtains a score '5.2' obtained by adding the product of the product of the product of the baseline language model and the product of the domain language model. "," Add an entry of the word chain "I" to the adaptation language model.

Thereafter, the adaptive language model generation means 10 outputs the generated adaptive language model to the speech recognition means 20.
Note that the baseline language model, the domain language model, and the adaptation language model are not limited to the examples described above. Also, the interpolation coefficient is not limited to the above-described example.

  The speech recognition means 20 performs speech recognition of a broadcast program using the adaptation language model and the baseline acoustic model input from the adaptation language model generation means 10. Here, the voice recognition means 20 receives voice data in which the voice of the broadcast program is recorded and a baseline acoustic model generated in advance. Then, the speech recognition means 20 performs speech recognition of the speech data for each speech section using an arbitrary speech recognition decoder such as a one pass decoder or a two pass decoder, and generates a speech recognition text representing a speech recognition result.

Thereafter, the speech recognition means 20 outputs the generated speech recognition text and speech data (not shown) to the alignment means 30.
In the case of iterative processing to be described later, the speech recognition unit 20 updates the baseline acoustic model with the adapted acoustic model input from the acoustic model adaptation unit 60, and uses the adapted acoustic model and the adapted language model. Perform voice recognition of the broadcast program.

The alignment unit 30 aligns the speech recognition text input from the speech recognition unit 20 with the subtitle text.
The alignment is to associate words included in the speech recognition text and the subtitle text in time order.

  Here, the alignment means 30 receives the subtitle text attached to the broadcast program. Then, the alignment unit 30 associates the words included in the speech recognition text with the words included in the subtitle text in time order. Thereafter, the alignment means 30 outputs the aligned speech recognition text and subtitle text and the speech data to the substitution means 40.

  The substitution unit 40 determines whether the word is different for each word associated between the speech recognition text and the subtitle text input from the alignment unit 30, and that a word sequence preceding or following the word matches Thus, it is determined whether or not the word is a replacement target. Then, when the word is a replacement target, the replacement means 40 replaces the word of the speech recognition text with the word of the subtitle text.

<Word substitution>
The substitution of words by the substitution means 40 will be described with reference to FIG. 2 (see FIG. 1 as appropriate).
In FIG. 2, the words a to d, the words X, and the words Y included in the speech recognition text 100 and the subtitle text 200 are illustrated as “a” to “d”, “X”, and “Y”. Further, the words a,..., The word b and the words c,..., The word d are respectively a word sequence in which N words are continuous. In addition, between the speech recognition text 100 and the subtitle text 200, it is assumed that the words from the word a to the word b and the words from the word c to the word d match.

  As shown in FIG. 2, it is assumed that words a,..., Word b and words c,..., Word d are associated between the speech recognition text 100 and the subtitle text 200. Further, it is assumed that the word X of the speech recognition text 100 and the word Y of the subtitle text 200 are associated with each other.

  The substitution means 40 presets the number of words N with an arbitrary value. The number of words N is preferably set to '5' in order to suppress misalignment and increase the amount of learning data (see Example 1).

  Here, the substitution means 40 determines, in order from the head side of the voice recognition text 100 and the subtitle text 200, whether or not the associated words match. First, since the word a of the speech recognition text 100 matches the word a of the subtitle text 200, the substitution means 40 does not determine the word a as a substitution target. Similar to the word a, the replacement means 40 does not determine up to the word b as a replacement target.

  Further, the substitution unit 40 determines that the word X of the speech recognition text 100 and the word Y of the subtitle text 200 do not match because they are different words. Here, N identical words a,..., Word b continue in front of the word X of the speech recognition text 100 and the word Y of the subtitle text 200. Further, after the word X of the speech recognition text 100 and after the word Y of the subtitle text 200, N identical words c,. From this, the substitution means 40 determines that the word X in the speech recognition text 100 and the N word series preceding and following the word Y in the subtitle text 200 match. Therefore, the substitution unit 40 determines the word X of the speech recognition text 100 as the substitution target, and replaces the word X with the word Y of the subtitle text 200.

  That is, when the words used as the determination reference are different, and the word sequence before and after the words used as the determination reference matches, the replacement means 40 determines that the word in the voice recognition text 100 is erroneously recognized as voice, Replace with the word of subtitle text 200.

Subsequently, since the words c,..., And d match between the speech recognition text 100 and the subtitle text 200, the substitution means 40 does not determine the words c,.
After that, the substitution means 40 outputs the speech recognition text 100, subtitle text 200, and speech data that have been substituted to the learning data generation means 50.

  The learning data generation unit 50 determines, for each utterance section, whether or not the speech recognition text 100 input from the substitution unit 40 and the subtitle text 200 match in order to generate learning data.

  Here, the learning data generation unit 50 detects speech segments of the speech data and the speech recognition text 100 input from the substitution unit 40 as a determination unit of the speech recognition text 100 and the subtitle text 200. Then, the learning data generation unit 50 performs the determination for each detected utterance section, and adds the word of the subtitle text corresponding to the utterance section as a label to the voice data of the utterance section determined to be coincident as a label. Generate data.

  For example, in FIG. 2, word a to word d are assumed to be the same utterance section. In this case, since the word X of the speech recognition text 100 is replaced with the word Y, the learning data generation unit 50 matches the speech sections from the word a to the word d between the speech recognition text 100 and the subtitle text 200 Then, it determines and generates learning data from this utterance section.

  Thereafter, the learning data generation means 50 outputs the generated learning data to the acoustic model adaptation means 60. Furthermore, the learning data generation means 50 may output the generated learning data as a speech language corpus.

Returning to FIG. 1, the description of the configuration of the acoustic model generation device 1 will be continued.
The acoustic model adaptation means 60 uses the learning data input from the learning data generation means 50 to adapt the acoustic model. For example, the acoustic model adaptation means 60 can use a Hidden Markov Model (HMM) as the acoustic model. Further, the acoustic model adaptation means 60 may use a MAP (Maximum A. Posteriori estimation) method as an acoustic model adaptation method.

  Further, since the accuracy of the sound recognition is improved by using the adapted acoustic model (adapted acoustic model), the acoustic model adaptation means 60 determines whether or not to perform the iterative process. Specifically, the acoustic model adaptation means 60 increments the number of times of adaptation of the acoustic model (the number of adaptations), and determines whether the number of times of adaptation is equal to or less than a preset threshold.

Here, when the number of times of adaptation is equal to or less than the threshold value, the acoustic model adaptation means 60 determines that repetitive processing is to be performed, and outputs an adapted acoustic model to the speech recognition means 20.
On the other hand, when the number of times of adaptation exceeds the threshold value, the acoustic model adaptation means 60 determines that the iterative processing is not to be performed, outputs the adapted acoustic model to the outside, and ends the processing.

  In addition, in the iterative process, each means of the acoustic model generation device 1 performs the same process except that the speech recognition means 20 uses an adapted acoustic model instead of the baseline acoustic model, and therefore, further description will be omitted.

Moreover, since each means other than the substitution means 40 is described in the following reference 1 of the acoustic model production | generation apparatus 1, the description beyond this is abbreviate | omitted.
Reference 1: Lamel et. Al, Lightly Supervised and Unsupervised Acoustic Model Training, Computer Speech and Language, Vol 6, pp. 115-129, 2002

[Operation of acoustic model generation apparatus]
The operation of the acoustic model generation device 1 will be described with reference to FIG. 3 (see FIG. 1 as needed).
The acoustic model generation device 1 generates the adaptation language model by interpolating the baseline language model and the domain language model by the adaptation language model generation means 10 (step S1).

The acoustic model generation device 1 causes the speech recognition unit 20 to perform speech recognition of a broadcast program using the adaptive language model and the baseline acoustic model generated in step S1 (step S2).
The acoustic model generation device 1 causes the alignment unit 30 to align the sound-to-speech recognition text generated at step S2 with the subtitle text (step S3).

  The acoustic model generation device 1 determines whether the words of the speech recognition text and the subtitle text aligned in step S3 are different by the substitution means 40 and whether or not the word sequence of the number N of words preceding and following the word matches. It is determined whether the word is a replacement target. Then, when the word is a replacement target, the replacement means 40 replaces the word of the speech recognition text with the word of the subtitle text (step S4).

  The acoustic model generation device 1 causes the learning data generation means 50 to determine, for each utterance section, whether or not the speech recognition text replaced in step S4 matches the subtitle text. Then, the learning data generation unit 50 generates learning data by adding, as a label, the word of the subtitle text corresponding to the speech section to the voice data of the speech section determined to match (step S5).

The acoustic model generation device 1 causes the acoustic model adaptation means 60 to adapt the acoustic model using the learning data generated in step S5, and increments the number of times of adaptation (step S6).
The acoustic model generation device 1 determines whether or not the iterative process is to be performed based on whether or not the number of times of adaptation is equal to or less than the threshold value by the acoustic model adaptation means 60 (step S7).

When the iterative process is performed (Yes in step S7), the acoustic model generation device 1 returns to the process of step S2. In the process of step S2, the speech recognition means 20 performs speech recognition of the broadcast program using the acoustic model adapted in step S6 instead of the baseline acoustic model. After that, the acoustic model generation device 1 continues the processing after step S3.
When the repetitive processing is not performed (No in Step S7), the acoustic model adaptation means 60 outputs the acoustic model adapted in Step S6, and ends the processing.

  As described above, the acoustic model generation device 1 according to the first embodiment of the present invention does not match the words of the speech recognition text even when the words of the speech recognition text and the subtitle text do not match because the accuracy of speech recognition is low. Replace. As a result, the acoustic model generation device 1 can generate more highly accurate learning data by increasing the word matching section between the speech recognition text and the subtitle text.

Second Embodiment
[Configuration of acoustic model generation device]
With respect to the configuration of the acoustic model generation device 1B according to the second embodiment of the present invention with reference to FIG. 4, points different from the first embodiment will be described (see FIG. 1 as needed).
The second embodiment differs from the first embodiment in that the learning data and the speech language corpus are treated as different data.

As shown in FIG. 4, the acoustic model generation device 1B includes an adaptive language model generation unit 10, a speech recognition unit 20, an alignment unit 30B, a substitution unit 40, a learning data generation unit 50B, and an acoustic model adaptation unit. 60 and speech language corpus generation means 70.
The respective units other than the alignment unit 30B, the learning data generation unit 50B, and the speech language corpus generation unit 70 are the same as those in the first embodiment, and thus the description thereof will be omitted.

The alignment unit 30 B outputs the aligned speech recognition text and subtitle text to the substitution unit 40 and the speech language corpus generation unit 70. Since the alignment means 30B is the same as that of 1st Embodiment in other points, description is abbreviate | omitted.
The learning data generation unit 50B is the same as the first embodiment except that it does not output the speech language corpus, and thus the description thereof is omitted.

  The speech language corpus generation means 70 determines, for each speech section, whether or not the speech recognition text input from the alignment means 30B matches the subtitle text. Then, the speech language corpus generation unit 70 generates a speech language corpus by adding the words of the subtitle text corresponding to the speech section as a label to the speech data of the speech section determined to be coincident.

When generating learning data, the learning data generation means 50 of FIG. 1 uses speech recognition texts in which words are substituted (that is, speech recognition texts input from the substitution means 40). On the other hand, when generating the speech language corpus, the speech language corpus generation means 70 uses speech recognition text in which words are not substituted (that is, speech recognition text input from the alignment means 30B).
Since the speech language corpus generation means 70 is the same as the learning data generation means 50 of FIG. 1 in other points, the description will be omitted.

[Operation of acoustic model generation apparatus]
The operation of the acoustic model generation device 1B will be described with reference to FIG. 5 (see FIGS. 3 and 4 as appropriate).
The processes in steps S1 to S7 in FIG. 5 are the same as the steps in FIG.

The acoustic model generation device 1B causes the speech language corpus generation means 70 to determine, for each utterance section, whether the speech recognition text aligned in step S3 matches the subtitle text. Then, the speech language corpus generation unit 70 generates a speech language corpus by adding the words of the subtitle text corresponding to the speech section as a label to the speech data of the speech section determined to be coincident (step S8). .
The process of step S8 is not limited after step S5, and may be performed after step S3 to before step S7.

  As described above, in the acoustic model generation device 1B according to the second embodiment of the present invention, as in the first embodiment, the word matching section between the speech recognition text and the subtitle text is increased, so that highly accurate learning data can be obtained. More can be generated.

Example 1
Hereinafter, setting of the number of words N will be described as the first embodiment.
The subtitle text has sufficient accuracy and is unlikely to be erroneous.

  When there are a plurality of similar word chains in the subtitle text, misalignment of the words associated by alignment may occur. If the number of words N is set to a small value such as 1 or 2, misalignment may not be eliminated, and a word in speech recognition text may be replaced with an incorrect word in subtitle text. On the other hand, when the number of words N is set to a large value, although the misalignment is eliminated, the number of words determined to be replacement targets may decrease, and an utterance section available as an utterance label may not be detected.

As described above, in order to accurately detect the section (word) to be substituted for the subtitle text from the speech recognition text among the mismatched sections for the speech recognition text and the subtitle text, an appropriate number of words N is set There must be. Therefore, the number of patterns in which N word sequences before and after a certain word match and the word is different from the number of patterns was investigated from the broadcast program. If many different patterns occur in one broadcast, there is a possibility that an alignment deviation will occur, so that it is not possible to anticipate the generation of highly accurate learning data.
It should be noted that “a pattern in which N word chains before and after a certain word match and the word is different” is abbreviated as “different pattern”.

  Broadcast programs to be surveyed are 100 times broadcasts of "Close-up modern (26 minutes broadcast time)," Marutoku Magazine (5 minutes broadcast time), and "Science ZERO (30 minutes broadcast time)". Then, while changing the value of the word number N, the number of different patterns included in the broadcast program to be checked was investigated.

  The survey results are shown in FIG. The horizontal axis in FIG. 6 represents the number of words N, and the horizontal axis represents the average value of the number of different patterns per broadcast. Further, in FIG. 6, “’ ”represents the result of“ Close-up modern ”,“ 』” represents the result of “Marutoku Magazine”, and “得” represents the result of “Science ZERO”.

  In FIG. 6, the number of words N is such that the number of different patterns is “0”, and the minimum value may be set among them. Assuming that the number of words N = 5 for three types of broadcast programs to be investigated, the number of different patterns is “0”. From this, it is considered that no misalignment occurs when the number of words N is set to 5.

(Examples 2 and 3)
The following describes a speech language corpus generation experiment.
Here, a speech language corpus was generated using the acoustic model generation device 1 of FIG. 1, the acoustic model generation device 1B of FIG. 4 and the method described in reference 1 and the generated speech language corpus was verified. . Hereinafter, the acoustic model generation apparatus 1 of FIG. 1 is referred to as Example 2, the acoustic model generation apparatus 1B of FIG. 4 is referred to as Example 3, and the method described in Reference 1 is referred to as a comparative example.

  In Examples 2 and 3 and Comparative Example, learning data was generated from speech recognition texts and subtitle texts for two hours each of “Close-up Contemporary”, “Marudoku Magazine”, and “Science ZERO”. These three types of broadcast programs are broadcasted from February to June 2014, with broadcast times different from those in the first embodiment.

"Close-up Contemporary" is a live broadcast news program. Subtitles of "Close-up Contemporary" are produced by a speed word processor method, and often the program caster's uttered content is subtitled as it is, including slight errors.
"Marutoku Magazine" is an offline information program. "Science ZERO" is a literary program. Subtitles of these "Marutoku Magazine" and "Science ZERO" are produced in advance.

  The adapted language model was generated for each broadcast using a baseline language model with a vocabulary size of 100 kilobytes learned from the transcription of a broadcast program and a domain language model learned from subtitle text. At this time, interpolation coefficients of the baseline language model and the domain language model are '0.1' and '0.9', respectively.

The speech recognition decoder utilized the two-pass decoder described in reference 2 below. The two-pass decoder performs speech recognition using a gender-dependent HMM while making a gender determination.
Reference 2: Imai et al., Improvement of Speech Recognition Technology for Real-time Subtitle Production for Broadcast, 2nd Document Processing Workshop, pp. 113-120, 2008

  Baseline acoustic models were learned from news programs broadcast by the Japan Broadcasting Corporation. In this news program, a man speaks for 340 hours and a woman speaks for 240 hours. The gender-specific acoustic model is a tri-state HMM of 5-state 3-self-loop and has about 4000 states of 16 mixed distributions by state sharing. These male and female acoustic models were adapted by learning data extracted from the alignment result of speech recognition text and subtitle text.

The method described in Reference Document 3 below was used for detection of the utterance section. The method described in Reference 3 performs endless phoneme recognition using a gender-dependent, gender-dependent acoustic model, and detects an utterance interval from a cumulative phoneme likelihood ratio of speech / non-speech.
Reference 3: T. Imai et. Al, Online speech detection and dual-gender speech recognition for captioning broadcast news, IEICE Trans. Inf & Syst, Vol E90-D, no. 8, pp. 1286-1291, 2007

  7 to 9 show the relationship between the number of times of adaptation of the acoustic model (horizontal axis) and the extraction rate of the speech language corpus (vertical axis). FIG. 7 shows the experimental result of “Close-up modern”, FIG. 8 shows the experimental result of “Marutoku Magazine”, and FIG. 9 shows the experimental result of “Science ZERO”. Moreover, in FIG. 7-FIG. 9, "(triangle | delta)" represents Example 1, "(triangle | delta)" represents Example 2, and "(triple)" represents a comparative example.

  When the number of times of adaptation is five, the extraction rate is 1.3 times or more for all the broadcast programs compared to the comparative example in the first embodiment. In addition, in Example 2, the extraction rate was 1.2 times or more for all broadcast programs as compared to the comparative example.

  When the number of adaptations was 5, the accuracy of the speech label of the speech language corpus was verified. In the first embodiment, substitution for an incorrect subtitle text is performed, and errors increase more than the second embodiment. Here, it has been found that the error in the speech label in Example 1 is caused by the incoincidence "Ano", "Eh". Furthermore, in both of the first and second embodiments, since the accuracy of the speech language corpus exceeds 99%, it has sufficient accuracy for constructing an acoustic model.

  In addition, when the number of adaptations was 5, the extraction rates of the speech language corpus were compared for three types of broadcast programs. As a result, the extraction rate increased in the order of "Science ZERO", "Marutoku Magazine", and "Close-up Contemporary".

Here, it is considered that the “close-up modern” has the lowest extraction rate since subtitles have not been given immediately before the end of the broadcast program. In the same broadcast program, the program caster uttered until just before the end of several broadcast runs, so it was not possible to subtitle all program audio by the speed word processor method.
The speed word processor method is a subtitle production method using a special high speed input keyboard which is input by pressing a plurality of keys simultaneously.

  In addition, “Marutoku Magazine” had a higher percentage of background music in airtime than “Science ZERO”. For this reason, it is thought that "Science ZERO" has a higher extraction rate than "Marutoku Magazine".

  From this, in order to increase the extraction rate of the speech language corpus, (1) an off-line subtitle program in which program audio is subtitled until the end of the broadcast program; (2) a broadcast program with a small amount of background music Is preferred.

  As mentioned above, although each embodiment and each example of the present invention were explained in full detail, the present invention is not limited to each above-mentioned embodiment and each example mentioned above, The design change etc. which do not deviate from the gist of the present invention Also included.

  In the above-described embodiment, the baseline language model, the domain language model, and the baseline acoustic model are described as being externally input, but the present invention is not limited thereto. For example, the acoustic model generation device may be provided with a database that stores and manages each language model and each acoustic model, and the acoustic model may be adapted with reference to this database.

  In the above embodiment, the acoustic model generation device (the learning data generation device) is described as an independent hardware, but the present invention is not limited to this. For example, the present invention can also be realized by a learning data generation program that causes hardware resources of a computer such as a CPU, a memory, and a hard disk to cooperate as a learning data generation device. This program may be distributed via a communication line, or may be distributed by writing on a recording medium such as a CD-ROM or a flash memory.

1, 1 B Acoustic Model Generator (Learning Data Generator)
10 Adaptive Language Model Generation Means (Third Language Model Generation Means)
20 speech recognition means 30, 30B alignment means 40 substitution means 50, 50B learning data generation means 60 acoustic model adaptation means 70 speech language corpus generation means

Claims (5)

A learning data generation apparatus that generates learning data necessary for adaptation of an acoustic model used for speech recognition of a broadcast program by quasi-supervised learning,
Third language model generation means for generating a third language model by linearly interpolating a first language model generated in advance from a text corpus and a second language model generated in advance from subtitle text of the broadcast program;
Voice recognition means for voice recognition of the broadcast program using the third language model and a previously generated acoustic model;
Alignment means for performing alignment in which words of the speech recognition text representing the speech recognition result of the broadcast program and the subtitle text are associated in time order;
The word is different for each word associated between the voice recognition text and the subtitle text, and the word depends on whether or not the word sequence of the number of words set in advance before and after the word matches. A replacement unit that determines whether the word is a replacement target, and if the word is a replacement target, replacing the word of the speech recognition text with the word of the subtitle text;
It is determined whether or not the speech recognition text replaced by the replacement means matches the subtitle text for each speech zone of the broadcast program, and the speech zone of the speech zone determined to match is the speech zone Learning data generation means for generating the learning data by adding words of subtitle text corresponding to
A learning data generation apparatus comprising:   The learning data generation apparatus according to claim 1, wherein the replacement unit has five words set in advance.   The learning data generation apparatus according to claim 1 or 2, further comprising: an acoustic model adaptation unit that adapts the acoustic model using the training data. The acoustic model adaptation means determines whether or not the number of times of adaptation of the acoustic model is equal to or less than a preset threshold value, and when the number of times is equal to or less than the threshold value, Output to recognition means,
4. The learning data generation apparatus according to claim 3 , wherein the speech recognition unit performs speech recognition on the broadcast program using the third language model and the adapted acoustic model.   The learning data generation program for functioning a computer as a learning data generation apparatus as described in any one of Claims 1-4.

Images