JP3162832B2 - Subtitle super screen creation device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an apparatus for creating a superimposed subtitle screen used in a television program.

[Summary of the Invention] [0002] The present invention relates to an apparatus for creating a subtitle super screen at the time of producing a television program, and presupposes that voice recognition and subtitles will be superimposed. Conventionally, by automatically recognizing the raw voice uttered by the speaker, generating a subtitle screen as a television signal from a character string obtained based on this recognition result, and synthesizing it with an external video to create a subtitle super screen This makes it possible to create a superimposed subtitle screen more easily and more quickly than by using a keyboard or the like.

[0003]

2. Description of the Related Art Conventionally, a subtitle super screen at the time of producing a television program is created by the following three methods.

[0004] The first method uses photographic technology, in which characters are printed on paper with a photocomposer,
After shooting it with a camera, the subtitle screen obtained by this shooting processing is combined with an arbitrary external video to create a subtitle super screen.

A second method is a method using electronic technology. In this method, a device similar to a word processor called an electronic character generator is used, and characters are input from a keyboard provided in the device and a built-in device is used. A character font is called, a television signal for a subtitle screen is generated using the character font, and the television signal is combined with an arbitrary external video to create a subtitle super screen.

A third method is to use a program technique. In this method, a program for displaying characters and graphics to be superimposed on a subtitle is previously set in a personal computer, and a subtitle screen is displayed using a numeric keypad or a touch panel of a keyboard. Then, the subtitle screen is combined with an arbitrary external video to create a subtitle super screen.

[0007]

However, the conventional subtitle super screen creation methods have the following problems.

That is, in the method using photographic technology or the method using electronic technology, when inputting characters, it is necessary to use a typesetting machine or a word processor, so that skilled operators must be trained. In terms of security and cost, it was not a suitable method for creating a subtitle super screen.

[0009] Further, in the method using photographic technology and the method using electronic technology, a certain time is required for inputting characters. Therefore, when a subtitle super screen is used for broadcasting, it is necessary to record a broadcast program before recording. Subtitle screen creation must be completed.

[0010] In addition, in the method using photographic technology and the method using electronic technology, the input characters cannot be easily corrected after the subtitle super screen is created. There was a problem that it was difficult.

Although the method using the program technique has an advantage that a desired subtitle screen can be called immediately even during broadcasting, only a predetermined subtitle screen can be created. However, there is a problem that it is not suitable for creating a subtitle super screen incorporating arbitrary characters.

Therefore, in order to solve such a problem,
There has been proposed a method of automatically recognizing audio in a television program on the receiver side and creating a subtitle super screen based on the recognition result.

[0013] As a technique related to this technique, for example, Japanese Patent Application No. 60-106779 discloses a system for displaying superimposed subtitles on TV and monitor displays using an unspecified speaker's voice input device. Method "and a" television receiver "incorporating a speech recognition device disclosed in Japanese Utility Model Application No. 63-131212.

[0014] However, these techniques are mainly intended for a hearing-impaired viewer, and the receiver automatically recognizes the sound in the television program sent from the broadcasting station, and outputs the recognition result on the receiver. Since it is a method of superimposing subtitles on video, it seems to be extremely difficult to realize it from the following points.

[0015] As a first problem, in most cases, the sound in a television program transmitted from a broadcast station is BG-based.
Background sounds such as M and voices of other speakers are mixed, and it is difficult to extract voices of a specific speaker.

As a second problem, since the number of speakers in a broadcast program is unspecified, it is difficult to reliably recognize the voice of such unspecified speakers.

As a third problem, the vocabulary of words spoken in a broadcast program is enormous, and unless the program contents or words to be subtitled are specified, there are too many vocabularies to be recognized. The dictionary becomes huge.

The present invention has been made in view of the above circumstances, and enables a broadcast station to create a subtitle super screen of a television program more easily and more quickly than a conventional keyboard or the like. It is an object of the present invention to provide a subtitle super screen creation device that can eliminate various difficulties that occur when creating a subtitle supermarket.

[0019]

In order to achieve the above-mentioned object, a subtitle super-screen creation apparatus according to the present invention provides a voice recognition system for generating a character string data by performing voice recognition processing on an input voice signal of a specific speaker. Unit and a subtitle screen generation unit that creates a subtitle screen based on the character string data generated by the voice recognition unit and combines the subtitle screen with an external video to create a subtitle super screen It is characterized by.

[0020]

In the above arrangement, on the assumption that voice recognition or subtitle superimposition is performed, raw voices uttered by a plurality of specific speakers are subjected to voice recognition processing, and based on a character string obtained by this voice recognition processing. By creating a subtitle screen and generating a television signal, even a person who is not skilled in operation can easily and quickly create a superimposed subtitle screen when producing a television program, even if input is performed using a conventional keyboard or the like.

[0021]

FIG. 1 is a block diagram showing a subtitle super screen creation system to which an embodiment of a subtitle super screen creation apparatus according to the present invention is applied.

The system for creating a superimposed subtitle screen shown in FIG. 1 includes a speech recognition unit 1 for recognizing an input speech signal of a specific speaker to generate character string data, and a character string generated by the speech recognition unit 1. A subtitle screen generating unit 2 that generates a subtitle screen based on the data, and combines this subtitle screen with an external video to create a subtitle super screen, when a specific speaker's audio signal is input After capturing the audio signal and performing voice recognition, a subtitle screen is created based on the character string data obtained by the voice recognition processing, and the subtitle screen is synthesized with an external video to create a subtitle super screen.

As shown in FIG. 2, the speech recognition section 1 includes an A / D converter 3, a sound analysis section 4, a vowel recognition section 5, a consonant recognition section 6, a recognition dictionary 7, a language processing section 8, With
When a voice signal is input, the voice signal is fetched and digitized. Then, an acoustic feature is extracted based on the voice data obtained by the digitizing process, and vowel recognition is performed based on the acoustic feature. After generating hypotheses based on the recognition results and calculating the likelihood of the consonant for each hypothesis, the optimal hypothesis is determined based on each of these likelihoods, and this is used as the recognition result for the character. The column data is supplied to the subtitle screen generation unit 2.

The A / D converter 3 receives an audio signal picked up by a microphone, for example, when a specific operator inputs a word or phrase to be superimposed on subtitles or an audio signal when a sentence is read out, Then, the data is digitized at a preset sampling frequency, for example, a sampling frequency of 15 kHz sufficient to extract an acoustic feature value, to generate audio data, and this is supplied to the acoustic analysis unit 4.

The acoustic analyzer 4 takes in the audio data output from the A / D converter 3 and has a length of 20 m.
The voice data is divided into frames at a period of 5 ms using a Hamming window of s, and then linear prediction analysis and zero-crossing wave analysis are performed on the voice data of each frame to obtain an 18-dimensional LPC cepstrum coefficient, Sound parameters such as the number of intersections and power are obtained, and the sound parameters are supplied to the vowel recognition unit 5 and the consonant recognition unit 6.

The vowel recognition unit 5 takes in the acoustic parameters output from the acoustic analysis unit 4 and collects and learns in advance the acoustic parameters and the pre-learned vowel standard pattern, ie, the voice of the person to be recognized. A vowel standard pattern that has been adapted to a new speaker using the vowel standard pattern obtained or the vowel standard pattern of another speaker that has already been trained, and based on the comparison result, the vowel in the voice data Is detected to create vowel sequence data of the input voice, and this is supplied to the language processing unit 8.

The consonant recognition unit 6 takes in the acoustic parameters output from the acoustic analysis unit 4 and collects and learns in advance the HMM (hidden Markov model) that has been learned in advance, that is, the voice of the person to be recognized. The input when each hypothesis output from the language processing unit 8 is adopted using the HMM created by adapting to a new speaker using the HMM of another speaker already trained. The likelihood of a consonant part constituting the speech is obtained, and this is supplied to the language processing unit 8.

In the recognition dictionary 7, the independent words to be recognized are described in a text format, so that the language processing unit 8 can use a highly versatile phrase grammar to construct a phrase that can be composed from each independent word. All data can be automatically generated.

The linguistic processing unit 8 takes in the vowel sequence data output from the vowel recognition unit 5, searches the recognition dictionary 7 using the vowel sequence data as a key, and retrieves word data or phrase data including the vowel sequence data. Read, replace the uncertain vowels in the vowel sequence data with other vowels based on these word data or phrase data, delete or insert new vowels, etc.
Some candidates are generated as hypotheses and supplied to the consonant recognition unit 6. When the likelihood data (data indicating the certainty) of each hypothesis is output from the consonant recognition unit 6, the likelihood data and the vowel sequence data output from the vowel recognition unit 5 are integrated, The closeness between each hypothesis and the input voice is obtained, and the character string data of the closest hypothesis is supplied to the caption screen generation unit 2 as a recognition result.

The subtitle screen generation unit 2 includes a character font file 10 and a character string / subtitle screen conversion unit 11 as shown in FIG.
, A video RAM 12, a scan converter 13,
A synthesizing unit 14 for fetching character string data output from the voice recognition unit 1 and accessing a character font file 10 based on the character string data to fetch character font information to create a subtitle screen After that, this is converted into a television video signal and combined with an external video (external video) to create a superimposed subtitle screen.

The character font file 10 stores the font information of each character used in the subtitle screen, and stores the font information of the character specified in response to the read command from the character string / subtitle screen conversion unit 11. This is read and supplied to the character string / subtitle screen conversion unit 11.

The character string / subtitle screen conversion unit 11 takes in the character string data output from the voice recognition unit 1 and accesses each of the character font files 10 based on each character code constituting the character string data. Then, character font information corresponding to each of the character codes is read out, arranged at an optimum position on the screen to create caption screen data, and supplied to the video RAM 12.

The video RAM 12 takes in the subtitle screen data output from the character string / subtitle screen conversion unit 11 and stores it. When a read command is output from the scan converter 13, the stored subtitle screen data is stored. The screen data is read and supplied to the scan converter 13.

The scan converter 13 outputs the video R
The subtitle screen data output from the AM 12 is fetched, and the subtitle screen data is specified by a specified standard, for example, N
The video signal is converted into a television video signal of a standard such as TSC, PAL, SECAM, and HDTV, and supplied to the synthesizing unit 14.

The synthesizing unit 14 is provided with the scan converter 1
3 and a video signal (external video signal) supplied from outside and a television video signal output from an external device 3 to create and output a subtitle super screen.

Then, in order to measure the speech recognition rate of this embodiment, a circuit having the configuration shown in FIG. 4 was created as actual hardware for realizing the speech recognition unit 1 described above.

The circuit shown in FIG. 1 includes an A / D converter 3 for performing A / D conversion, nine transputers 20 to 28 each constituted by a processor for parallel processing, and storage of a transputer 20 for control. And a disk device 29 used as a device.

The transputer 20 for control
Performs the processing of the vowel recognition unit 5, controls all the transputers 21 to 28, and exchanges data with the subtitle screen generation unit 2, and the transputers 21 to 24 perform the processing of the acoustic analysis unit 4.

In this case, each of these transputers 21
Among them, the transputer 21 performs acoustic analysis of the even-numbered frame of the input audio data, and the transputer 22 performs vector quantization of the LPC cepstrum coefficient obtained by the transputer 21. The transputer 23 performs an acoustic analysis on the odd-numbered frames of the input audio data, and the transputer 24
Vector quantization of the LPC cepstrum coefficient obtained in step 3 is performed.

The transputers 25 to 28 perform the processing of the language processing unit 8 and the consonant recognition unit 6 by different hypothesis generation methods.

The disk device 29 is used as a storage area for storing independent words used in the recognition dictionary 7 in a text format.

When an experiment was conducted on the speech recognition rate of an actual speech signal using this circuit, the following remarkable results could be obtained.

That is, for the sumo wrestling program in the television program, the announcer uttered a sentence regarding the name of the wrestler and the determinant, and a superimposed subtitle screen was created based on the audio signal obtained at this time.

As a result, the sentence uttered by the announcer is a sentence that is divided almost in units of phrases, such as "Kihanata and Konishiki's approach", and "In the current approach, Kihanada has won by a close approach". At this time, a phrase recognition rate of 98% was obtained, and a sentence recognition rate of 100% was obtained.

At this time, the time required for the processing almost coincided with the real time, and it was confirmed that the device can be used without any practical problems.

As described above, in this embodiment, the voices of a plurality of specific speakers uttered during the production of a television program are to be recognized, and each hypothesis is obtained using a vowel sequence as a voice recognition processing method. Since the method of obtaining the character string of the voice signal using the likelihood of the consonant for each of these hypotheses is used, the input voice can be reliably recognized. It is possible to create a superimposed subtitle screen of a television program more easily and quickly on the broadcast station side, and as a result, it is possible to eliminate various difficulties that occur when creating a superimposed subtitle screen on the receiver side.

In the above-described embodiment, the sound analysis unit 4 constituting the speech recognition unit 1 divides the speech data into frames at a cycle of 5 ms using a Hamming window having a length of 20 ms. A sound analysis method is used to obtain sound parameters such as an 18-dimensional LPC cepstrum coefficient, the number of zero crossings, and power by performing a linear prediction analysis and a zero-crossing wave analysis on the audio data of In addition to the analysis method, for example, a frequency spectrum analysis such as an FFT analysis or another method may be used.

In the above-described embodiment, the vowel recognition unit 5 and the consonant recognition unit 6 constituting the speech recognition unit 1 are used as the speech recognition processing method.
(Hidden Markov Model)
Not only such a voice recognition method but also a voice recognition method using, for example, DP matching or a neural network may be used.

[0049]

As described above, according to the present invention, by utilizing voice recognition, even a person who is not skilled in operation can easily and quickly input a television program by using a conventional keyboard or the like. You can create a subtitle super screen during production.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a subtitle super screen creation system to which an embodiment of a subtitle super screen creation device according to the present invention is applied.

FIG. 2 is a block diagram illustrating a detailed circuit configuration example of a speech recognition unit illustrated in FIG. 1;

FIG. 3 is a block diagram illustrating a detailed circuit configuration example of a subtitle screen generation unit illustrated in FIG. 1;

FIG. 4 is a block diagram illustrating a specific hardware configuration example of a speech recognition unit illustrated in FIG. 2;

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 Speech recognition part 2 Subtitle screen generation part 3 A / D converter 4 Sound analysis part 5 Vowel recognition part 6 Consonant recognition part 7 Recognition dictionary 8 Language processing part 10 Character font file 11 Character string / subtitle screen conversion part 12 Video RAM 13 Scan converter 14 Synthesis unit 20-28 Transputer 29 Disk unit

──────────────────────────────────────────────────続 き Continuation of the front page (72) Inventor Eiichi Miyasaka 1-10-11 Kinuta, Setagaya-ku, Tokyo Japan Broadcasting Corporation Research Institute of Broadcasting Technology (56) References JP-A-61-264882 (JP, A) Hei 5-176232 (JP, A) JP-A 2-53670 (JP, U) JP-A 6-7372 (JP, U) (58) Fields surveyed (Int. Cl. ⁷ , DB name) H04N 5 / 278 G10L 3/00 551 G06F 3/16 320 G06F 15/20 503

Claims (1)

(57) [Claims] 1. A voice recognition unit that generates character string data by performing voice recognition processing on an input voice signal of a specific speaker, and generates a subtitle screen based on the character string data generated by the voice recognition unit. And a subtitle screen generating unit that generates a subtitle super screen by combining the subtitle screen with an external video.