JP2018045256A - Subtitle production device and subtitle production method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a technique for efficiently producing a subtitle.SOLUTION: A voice recognition part 30 performs voice recognition of object voice 10 or voice obtained by repeating the object voice 10 and converts the voice into a text. A text division/connection processing part 40 performs division processing of the text after the voice recognition, to create a subtitle text. A keyboard correction part 60 corrects the subtitle text. A delay part 82 outputs a plurality of delay voice obtained by delaying the object voice 10 by prescribed different time. A delay switching switch 84 switches the plurality of delay voice output by the delay part 82 and provides the delay voice to the keyboard correction part 60, on the basis of an instruction of a person who corrects the subtitle text.SELECTED DRAWING: Figure 2

Description

  The present invention relates to a technique for producing subtitles from audio.

  In recent years, subtitle broadcasting for hearing-impaired people has been implemented in television broadcasting such as terrestrial broadcasting, BS broadcasting, and CS broadcasting. Especially for real-time subtitles to be given to live broadcasts such as news and sports broadcasts, current methods for producing subtitles are mainly methods in which an expert inputs immediately using a keyboard. In this process, while listening to broadcast or studio sound, the operator immediately inputs what is being spoken using the keyboard and converts it into text. Such subtitle production requires a plurality of skilled specialized operators, and there is a problem of reducing the cost of subtitle production.

JP 2001-142482 A JP 2006-119534 A

  In the immediate input method, several operators divide the flowing voice by a plurality of people in time series and input the keyboard in order. However, skilled input is required for immediate input (shorthand writing) using a keyboard, so it is necessary to train the operator over a long period of time, and investment is required. In addition, since a plurality of persons input in order, it is necessary for the operators to take a breath of Aki, which is a factor that requires long-term training, and a price for the operator's skill is required.

  Although there is a text conversion technique using speech recognition as a method other than keyboard input, the recognition rate of speech recognition is not 100%, and speed is required for correcting speech recognition results.

  Whether it is an immediate input method or a text-based method using speech recognition, there is a problem of shortage of operators due to the special technology, and it also costs personnel training to newly train operators, The cost is inevitable for subtitle production.

  The present invention has been made in view of these problems, and an object thereof is to provide a technique for efficiently producing captions.

  In order to solve the above-described problem, a caption production device according to an aspect of the present invention includes a speech recognition unit that recognizes a target speech or a speech that is a repetition of the target speech and converts the speech into text, and a process for dividing the text after speech recognition A subtitle text generation unit, a subtitle text correction unit, a delay unit that outputs a plurality of delayed sounds obtained by delaying the target audio by a predetermined different time, and a subtitle text corrector And a switching unit that switches a plurality of delayed sounds output by the delay unit and provides them to the correction unit according to an instruction.

  Another aspect of the present invention is also a caption production device. This apparatus includes a speech recognition unit that recognizes a target speech or a speech that is a repetition of the target speech and converts the speech into text, a split processing unit that splits the text after speech recognition to generate subtitle text, A correction unit for correcting, a delay unit for outputting a delayed sound obtained by delaying the target sound by a predetermined time, an audio reproducing unit for reproducing an audio file in which the recognized speech is recorded, and a subtitle text corrector A switching unit that switches between the delayed sound output by the delay unit and the reproduced sound of the audio file output by the sound reproducing unit according to an instruction and provides the sound to the correction unit.

  Yet another aspect of the present invention is a caption production method. The method includes a speech recognition step of recognizing target speech or speech reiterating the target speech and converting it to text, a split processing step of splitting the text after speech recognition to generate subtitle text, and subtitle text A correction step for correcting, a delay step for outputting a plurality of delayed sounds obtained by delaying the target sound by a predetermined different time, and a plurality of delayed sounds output by the delay step according to an instruction from a subtitle text corrector. A switching step of switching and providing to the correction step.

  Yet another embodiment of the present invention is also a caption production method. The method includes a speech recognition step of recognizing target speech or speech reiterating the target speech and converting it to text, a split processing step of splitting the text after speech recognition to generate subtitle text, and subtitle text A correction step for correcting, a delay step for outputting a delayed sound obtained by delaying the target sound by a predetermined time, an audio reproducing step for reproducing an audio file in which the recognized speech is recorded, and a subtitle text modifier A switching step of switching between the delayed sound output by the delay step and the playback sound of the sound file output by the sound playback step according to an instruction and providing the same to the correction step.

  It should be noted that any combination of the above-described constituent elements and the expression of the present invention converted between a method, an apparatus, a system, a computer program, a data structure, a recording medium, and the like are also effective as an aspect of the present invention.

  According to the present invention, captions can be produced efficiently.

It is a block diagram of the closed caption production apparatus which concerns on 1st Embodiment. It is a block diagram of the closed caption production apparatus which concerns on 2nd Embodiment. It is a block diagram of the closed caption production apparatus which concerns on 3rd Embodiment. FIG. 3 is a diagram schematically illustrating a state in which a plurality of delayed sounds are switched and output by a delay unit and a delay switch in FIG. 2. It is a block diagram of the delay changeover switch of FIG. It is a figure which illustrates typically the mode of the synchronous reproduction | regeneration of an audio file in the caption production apparatus of FIG.

  FIG. 1 is a configuration diagram of a caption production device 100 according to the first embodiment.

  The target audio 10 is generally audio accompanied with video such as television broadcasting. The target voice 10 is input to the squirrel peak unit 20 or is directly input to the voice recognition unit 30 as a raw voice. Whether or not the configuration of the squirrel peak unit 20 is provided depends on the necessity of the squirrel peak by the re-speaker. For example, in the case of news broadcasting, when the announcer is uttering accurately, the configuration of the rispeak unit 20 may be omitted and the announcer's live voice may be directly input to the voice recognition unit 30. In addition, even when the re-speaker cannot be attached due to a budget or the like, the live voice is directly input to the voice recognition unit 30.

  In the squirrel peak unit 20, the re-speaker clearly repeats at a constant utterance speed while listening to the target sound 10 with headphones or the like and appropriately dividing the same content. The repeated voice is input to the microphone. The squirrel peak unit 20 outputs the sound of the re-speaker input to the microphone and supplies it to the voice recognition unit 30.

  The voice recognition unit 30 recognizes the rispeak voice or the live voice and converts it into text. The voice recognition unit 30 is configured by general voice recognition software on a personal computer (PC) (this PC is referred to as “PC1”). Since the speech recognition result is subjected to caption correction processing at a later stage, the speech recognition software may be a relatively inexpensive one that does not have a high recognition rate. The text after the speech recognition is input to the text division / combination processing unit 40.

  The text division / combination processing unit 40 performs processing to divide or combine the speech-recognized text so as to be within a predetermined number of subtitle characters. Further, the text division / combination processing unit 40 may perform processing for color-coding the text by a speaker. For example, the text color is different between the main caster and the sub-caster.

  Text splitting, combining, and coloring are displayed on the touch panel display etc. on the second PC (referred to as “PC2”), and the person in charge of splitting indicates the split / join position on the touch panel. It is done by doing. The text (“subtitle text”) after the division / combination / coloring processing by the text division / combination processing unit 40 is input to the subtitle time series management unit 50.

  The respeaker in the risk peak unit 20 and the division person in charge in the text division / combination processing unit 40 may be the same person. This is because a skilled re-speaker can perform text division / combination processing after speech recognition while performing a re-peak.

  The total time required for the preprocessing by the squirrel peak unit 20, the speech recognition unit 30, and the text division / combination processing unit 40 is P1 seconds. The preprocessing time P1 is measured in advance.

  The subtitle time series management unit 50 manages the subtitle text adjusted to an appropriate length in time series, and sequentially distributes the subtitle text to the plurality of keyboard correction units 60.

  The plurality of keyboard correction units 60 are terminals (referred to as “PC3” to “PCn”) used by each of a plurality of correctors. The delay unit 80 delays the target voice 10 by a predetermined time and outputs it. The delay unit 80 is a general analog audio delay device, and can delay the input audio by a specified time and output it. Here, the delay unit 80 delays and outputs the target voice 10 by a time slightly longer than the above-described preprocessing time P1. The delayed sound output from the delay unit 80 is input to the keyboard correction unit 60 or the headphones.

  In the keyboard correcting unit 60, the corrector performs an operation of correcting an error in the speech recognition result. Further, the corrector corrects the subtitle text while listening to the delayed sound output from the delay unit 80 through headphones. The corrector outputs the corrected subtitle text as soon as the correction of the subtitle text that he / she is responsible for is completed. The corrected subtitle text output by the plurality of keyboard correction units 60 is input to the transmission order control unit 70 asynchronously.

  The transmission order control unit 70 correctly replaces the order of the subtitle texts supplied asynchronously from the plurality of keyboard correction units 60 and transmits the final subtitles to the broadcast station.

  The caption time-series management unit 50 and the transmission order control unit 70 can be executed by the same server (referred to as “server 1”).

  In the caption production device 100 according to the present embodiment, even if the recognition accuracy of the speech recognition software is low, the corrector can correct the caption while listening to the delayed live sound. The re-speaker needs to be an expert, but the corrector does not need to be an expert. As a result, the total cost of subtitle production can be kept low.

  FIG. 2 is a configuration diagram of the caption production device 110 according to the second embodiment. The components common to the caption production device 100 according to the first embodiment are denoted by the same reference numerals and description thereof is omitted. The subtitle production apparatus 110 in FIG. 2 is different from the subtitle production apparatus 100 in FIG. 1 in the configuration of the delay unit 82 and the delay changeover switch 84.

  The delay unit 82 delays the target sound 10 by a plurality of different delay times and outputs a plurality of delayed sounds. The plurality of delayed sounds that have been output are input to the delay changeover switch 84. The delay changeover switch 84 selects and outputs one of a plurality of delayed sounds. The selected delayed sound is input to the keyboard correcting unit 60 or the headphones.

  FIG. 4 is a diagram schematically illustrating a state in which a plurality of delayed sounds are switched and output by the delay unit 82 and the delay switch 84.

  Reference numeral 200 indicates a section of the target speech 10, which is A seconds long. This is a phrase that the re-speaker appropriately divides when repeating. Reference numeral 250 indicates the time taken for “preprocessing” by the rispeaking unit 20, the speech recognition unit 30, and the text division / combination processing unit 40, which is B seconds here.

  Here, the delay unit 82 delays and outputs the raw voice by three delay times D1, D2, and D3. The first delay time D1 is slightly longer than the preprocessing time B. The second delay time D2 is a time obtained by adding one section of time A to the target speech 10 to the first delay time D1. The third delay time D3 is a time obtained by adding the time A of one section of the target speech 10 to the second delay time D2.

  When the first switch of the delay changeover switch 84 is pressed down, a delayed voice obtained by delaying the raw voice by the first delay time D1 is output for A second (reference numeral 210). Similarly, when the second switch and the third switch of the delay changeover switch 84 are pressed down, a delayed sound obtained by delaying the raw sound by the second delay time D2 and the third delay time D3 is output for A seconds. (Reference numerals 220, 230). However, the first switch of the delay changeover switch 84 may be omitted, and the first delayed sound may be automatically output without pressing the switch after the first delay time D1 has elapsed. In this case, when the corrector subsequently depresses the second switch and the third switch, the second and third delayed sounds are output.

  FIG. 5 is a configuration diagram of the delay changeover switch 84. The delay changeover switch 84 receives input of the delayed sounds 1 to n and outputs any delayed sound. If the internal switches SW1 to SWn-1 are provided and all the internal switches SW1 to SWn-1 are off, the delayed sound 1 is output, and if only SW1 is turned on, the delayed sound 2 is output and only SW2 is turned on. Is output, the delayed sound 3 is output, and when only SWn-1 is turned on, the delayed sound n is output.

  The delay changeover switch 84 is prepared for each corrector. The corrector can rehearse the second and third delayed sounds by operating the switch. In this case, if the correction of subtitles is not completed only by listening to the first delayed sound, the missed portion can be rehearsed several times.

  The delay changeover switch 84 may be realized by pressing a specific key of the keyboard, or may be realized by providing a hand switch separately from the keyboard. Or you may implement | achieve by a foot switch or a pedal so that a correction person may ensure a correction speed, without releasing a hand from a keyboard.

  As an example, a configuration has been described in which delayed speech can be output up to three times, but in general, a configuration in which delayed speech can be output up to n times can be employed. When there are few correction parts in the subtitle, the correction work may be completed by listening to the delayed sound once. In that case, only the delayed sound of the first delay time D1 is used, and the caption can be put out quickly. On the other hand, when there are many correction portions in the subtitle, the accuracy of the subtitle can be improved by repeatedly listening to the live audio up to n times. Therefore, the speed and accuracy of subtitles can be improved with a good balance.

  When the recognition rate of the voice recognition software is low, or when the quality of the rispeak voice by the re-speaker is low, the accuracy of subtitles can be improved by listening to the live voice up to n times. In other words, this means that the accuracy of subtitles can be improved in post-processing without using expensive speech recognition software or skilled re-speakers, and the cost of subtitle production can be kept low. it can.

  FIG. 3 is a configuration diagram of the caption production device 120 according to the third embodiment. The components common to the caption production device 100 according to the first embodiment are denoted by the same reference numerals and description thereof is omitted. In the caption production device 120 of FIG. 3, the keyboard correction / audio reproduction control unit 60 reproduces the audio file 32 stored by the audio recognition unit 30, and the audio mixer 90 includes the reproduced audio from the audio file 32 and the delay unit 80. 1 is different from the caption production device 100 shown in FIG.

  In the caption production device 100 according to the first embodiment in FIG. 1 and the caption production device 110 according to the second embodiment in FIG. 2, even when the corrector hears the delayed live speech, the live speech is not synchronized with the caption text. Therefore, unnecessary audio before and after the subtitle in charge is included, and the audio position of the assigned subtitle is searched. This is a factor that causes instability, such as waiting for the audio part that is the subject of the subtitle text to start or if the target audio part has already begun to play, causing the loss of time to the corrector, The result is a slower speed. Therefore, in the caption production device 120 according to the third embodiment, the voice recognized by the voice recognition unit 30 is stored in a file, and the keyboard correction / voice playback control unit 60 plays the voice file according to the instructions of the corrector. It can be so.

  The audio file 32 is a file in which the voice re-speaked by the re-speaker is recorded (however, when the re-speaker is not attached, a recorded live sound is used). Since the voice is temporarily saved when the voice recognition software performs voice recognition processing, the saved file can be used as the voice file 32. In order to synchronize the reproduction of the audio file 32 with the subtitle text handled by the corrector, the position (start position and end position) where the word is uttered in the audio file 32 for each word that has been voice-recognized in units of milliseconds. The recorded “audio reproduction information” is used. With this audio reproduction information, the subtitle character string and the audio file can be completely synchronized.

  Since the reproduction of the audio file can be synchronized with the subtitle text, audio that is not related to the subtitle text handled by the corrector does not enter before and after. Further, since it is an audio file, it can be easily reproduced any number of times, and the reproduction speed can be increased to 1.5 times, for example.

  The audio mixer 90 switches to either the live audio delayed by a predetermined time by the delay unit 80 or the reproduced audio from the audio file 32 synchronized with the subtitle text, and inputs it to the keyboard correcting unit 60 or the headphones, and the corrector's headphones Output from. Here, the audio mixer 90 is outside the keyboard correction / audio reproduction control unit 60, but the audio mixer 90 may be provided in the keyboard correction / audio reproduction control unit 60.

  FIG. 6 is a diagram for schematically explaining the state of synchronous reproduction of the audio file 32. Reference numeral 200 indicates a section of the target speech 10, which is A seconds long. Reference numeral 250 indicates a time required for preprocessing by the rispeak unit 20, the speech recognition unit 30, and the text division / combination processing unit 40, which is B seconds here. When the reproduction sound of the audio file 32 is selected by the audio mixer 90, the audio file 32 is reproduced with a delay of time P2 (reference numeral 241). Here, P2 = A + B, and the sound completely synchronized with the text being edited. Is played. Thereafter, the audio file 32 can be repeatedly reproduced (reference numerals 242 and 243).

  The audio mixer 90 is a switching unit such as a switch or a pedal, and the live audio delayed by the delay unit 80 is output the first time. However, if the corrector instructs, the reproduced audio of the audio file is output after the second time. The For simple subtitles with few corrections, the correction work can be completed simply by listening to the first delayed audio. However, for complicated subtitles with many corrections, the audio file can be played and listened to as many times as possible. The first delayed sound is live sound, while the second and subsequent playback sounds are re-speaker squirrel peak sounds, so the exact contents of the parts that are difficult to hear with the live sound must be grasped by the squirrel peak sound. Can do.

  In the above description, the audio file 32 is a recording of the voice replayed by the re-speaker. However, instead of the lispeak audio file or together with the lispeak audio file, a raw audio file that records the live audio that does not pass through the re-speaker is generated. The keyboard correction / audio reproduction control unit 60 may reproduce the raw audio file and provide it to the audio mixer 90. Unlike the Lispeak audio file, the live audio file does not synchronize with the subtitle text. However, if the quality of the Lispeak audio is not good, the corrector can edit the text subtitle from the Lispeak audio while editing the raw audio file. The accuracy of subtitles can be improved by playing back and listening to live audio.

  The present invention has been described based on the embodiments. The embodiments are exemplifications, and it will be understood by those skilled in the art that various modifications can be made to combinations of the respective constituent elements and processing processes, and such modifications are within the scope of the present invention. .

  It is also possible to realize multilingual subtitle broadcasting in real time by combining subtitle texts transmitted by the transmission order control unit 70 to generate a sentence and automatically translating the sentence into another language by automatic translation software. .

  Although the caption production device of the above embodiment generates the caption in real time based on the sound of the broadcasted program, the caption production apparatus of the above embodiment can also be used when generating the caption based on the recorded program. A caption production device can be used.

  10 target voices, 20 rispeak parts, 30 voice recognition parts, 40 text division / combination processing parts, 50 subtitle time series management parts, 60 keyboard correction parts, 70 transmission order control parts, 80 delay parts, 82 delay parts, 84 delay switching Switch, 90 audio mixer, 100, 110, 120 caption production device.

Claims (3)

A voice recognition unit for recognizing and converting the target voice or the squirrel voice obtained by reiterating the target voice into text and recognizing the target voice or the voice-recognized squirrel peak voice as a voice file;
A correction section for correcting subtitle text after speech recognition;
An output unit for outputting the target audio;
An audio reproduction unit that reproduces an audio file in which the target audio / risk peak audio is recorded in synchronization with a character string of the subtitle text;
According to an instruction from the corrector of the subtitle text, either the target audio output by the output unit or the target audio / risk peak audio synchronized with the character string of the subtitle text reproduced from the audio file output by the audio reproduction unit A subtitle production apparatus comprising: a switching unit that switches to provide to headphones worn by the corrector.   The switching unit outputs the target voice to the headphones for the first time, and outputs the target voice / risk peak voice synchronized with the character string of the subtitle text from which the audio file is played for the second time and thereafter. The caption production device according to claim 1. A speech recognition step of recognizing and converting the target speech or the squirrel speech reiterating the target speech into text and recognizing the target speech or speech-recognized squirrel peak speech as a speech file;
A correction step for correcting subtitle text after speech recognition;
An output step for outputting the target audio;
An audio reproduction step of reproducing an audio file in which the target audio / risk peak audio is recorded in synchronization with a character string of the subtitle text;
According to an instruction from the corrector of the subtitle text, either the target voice output by the output step or the target voice / risk peak voice synchronized with the character string of the subtitle text reproduced from the voice file output by the voice playback step A subtitle production method executed by a subtitle production apparatus, comprising: a switching step of switching between and providing to headphones worn by the corrector.

Images