KR101478918B1 - Apparatus and method for correcting caption subtitle - Google Patents

Abstract

The present invention relates to an apparatus for correcting captions of a broadcasting program. According to one aspect, the apparatus for correcting captions receives a caption created through stenography corresponding to a voice of broadcasting and organized schedule information of the broadcasting. A broadcasting program corresponding to the caption is searched among broadcasting programs included in the organized schedule information based on a time in which the caption is received. The information of the searched broadcasting program is matched with the caption to generate a caption file and data. A text format and a time synchronization value of each caption of the caption file are corrected based on a voice recognition result of the broadcasting and the utilization objective of the caption file.

Description

[0001] APPARATUS AND METHOD FOR CORRECTING CAPTION SUBTITLE [0002]

The examples described below relate to a device for correcting a caption subtitle of a broadcast program, and more particularly to a device for correcting a caption subtitle to be used in various applications.

Services such as closed caption subtitles, sign language, video description, etc. are being used for broadcasting so that people with a disability can use TV broadcast contents. Among them, closed caption subtitles are most widely used.

Although current broadcasting stations support closed captioning entirely only in terrestrial broadcasting, it is expected that closed captioning will be gradually supported in multimedia on the web or mobile environment. In addition, closed caption subtitles can be considered as a valuable resource as metadata of a program for keyword extraction or scene retrieval, and can be used as a translation for foreign language translation.

Therefore, broadcasters are faced with the need for a new system or a method for efficiently reusing closed caption subtitles. The method of reusing the closed caption caption and the new system should be designed considering the characteristics of the closed caption caption and the compatibility with the currently used system.

Since 2008, the right of people with disabilities to access broadcast content has been legally guaranteed in Korea. In 2011, specific guidelines were established for accessing broadcast content for persons with disabilities, such as classification of targets and program rate.

The present invention stores a closed caption subtitle in a file format of a program unit, changes the text format of the caption file captions to various forms according to the setting, corrects time synchronization, and reproduces digital television (DTV) , A device for generating caption subtitles for use in various applications such as reusing broadcast programs on a satellite platform or generating video on demand (VOD) subtitles or foreign language translation on the web or a mobile multimedia service platform to provide.

The present invention also provides an apparatus for generating high quality closed caption subtitles suitable for reuse in comparison with existing closed caption subtitles.

According to an aspect of the present invention, a caption caption correcting apparatus includes a receiver for receiving a caption caption generated through stenography corresponding to a voice of a broadcast and a composition schedule information of the broadcast, A subtitle data generation unit for searching a broadcast program corresponding to the caption subtitle from the broadcast programs included in the programming schedule information and for generating a subtitle file and data by matching information of the searched broadcast program with the caption subtitle, And a correction unit for correcting a text format and a time synchronization value of each caption subtitles of the subtitles file based on the purpose of utilization of the file and the speech recognition result of the broadcast.

The receiving unit receives the shorthand data via the dedicated modem, and stores the received time and text of the shorthand data in a log form or transmits the received time and text to the caption data generation unit in real time.

The caption file generated by the caption data generation unit includes a caption text of the corresponding program and a time stamp which is a time position at which each caption text appears. The caption data generated by the caption data generation unit may include at least one of a title of the broadcast program, a broadcast time of the broadcast program, a brief description of the broadcast program, a unique identifier of the broadcast program, a broadcast degree of the broadcast program, Version, a subtitle file format (e.g., various formats such as SRT, SMI, SUB, etc.), and a language code.

Wherein the correcting unit includes a voice recognition unit for recognizing a voice in the moving picture of the broadcast program and a time synchronization correction unit for comparing the time stamp of the recognized voice with the time stamp of the caption text to correct time synchronization of the caption subtitle .

Wherein the correcting unit comprises: a format changing unit for changing the format of the caption subtitle in a format set in accordance with the utilization purpose of the caption subtitle; and a speech dictionary composed of a text of a caption subtitle of the changed format, And a time synchronization correction unit for correcting the time synchronization of the caption subtitle using the time synchronization correction unit.

The speech recognition unit searches for a word in a speech dictionary made up of the text of the caption file generated by the caption data generation unit when the speech is recognized, adopts the matched word as a speech recognition result, Can be generated. In addition, when it is desired to recognize only one section of the speech, only the texts having a time stamp within a predetermined margin based on the utterance timing of the corresponding section of the text of the subtitle file generated by the subtitle data generation section, And then searching for only the words of the speech dictionary and adopting the matched word as the speech recognition result. The speech recognition unit may repeat the speech recognition process even when the speech recognition is performed on the entire image, thereby reducing the range of a dictionary to be referred to in speech recognition, thereby improving the accuracy of the entire speech recognition.

The speech recognition unit can detect a section in which music exists in addition to the voice.

The speech recognition unit may distinguish a speaker from a moving picture of a broadcast program and recognize a speech or voice uttered by a speaker.

The time synchronization correction unit may correct the time synchronization of the caption by using the new time stamp generated by the voice recognition unit and the text of the original caption file. The time synchronization can be corrected by replacing the new text timestamp assigned to the first appearing word of each line of the subtitle after changing the text line wrapping of the subtitle through the format changing unit of the original subtitle with the timestamp of the corresponding line.

The correcting unit may include a correcting unit for checking and correcting the spelling of text of the caption subtitle and the time synchronization correction error, and a text coloring unit for coloring the text of the caption subtitle so as to be distinguished by the speaker of the broadcast program.

In another aspect, the caption subtitle correction apparatus may further include a caption subtitle correction unit for correcting time synchronization of the caption subtitle included in the caption data if the video file name of the broadcast program included in the folder is the same as the caption data name, And a folder monitoring unit for monitoring the file name and the name of the caption data.

In another aspect, the caption subtitle correction apparatus may further include a schedule monitoring unit for monitoring the composition schedule for the purpose of correcting time synchronization of the caption subtitles at a predetermined period based on the program scheduling information of the broadcast.

In another aspect, the caption subtitle correction apparatus may further include a filtering unit for filtering the specific program in the program schedule information so that the correction unit does not operate in the specific program.

In another aspect, the caption subtitle correction apparatus further includes a manual correction unit for collectively changing the time synchronization and the sentence length of all the lines of the subtitles, or merging, dividing and changing the time stamps of individual lines according to the user's need can do.

According to an aspect of the present invention, there is provided a caption subtitle correction method, comprising: receiving a caption closed caption generated through stenography corresponding to a speech of a broadcast and schedule scheduling information of the broadcast; Searching for a broadcast program corresponding to the caption subtitle from the broadcast programs included in the program schedule information, generating subtitle file and data by matching information of the searched broadcast program with the caption subtitle, And correcting the text format and the time synchronization value of each of the caption subtitles of the caption file based on the speech recognition result of the caption file.

The step of correcting may include a step of recognizing the voice in the moving picture of the broadcast program and a step of comparing the time stamp of the recognized voice with the time stamp of the caption data to correct time synchronization of the caption subtitle .

Wherein the step of correcting comprises: changing the format of the caption subtitle in a format set in accordance with the utilization purpose of the caption subtitle; recognizing a voice in the moving picture of the broadcast program; And correcting the time synchronization by replacing the assigned new time stamp with a new time stamp generated through speech recognition.

The present invention stores a closed caption subtitle in a file format of a program unit, changes the text format of the caption file captions to various forms according to the setting, corrects time synchronization, and reproduces digital television (DTV) , A device for generating caption subtitles for use in a variety of applications such as reusing broadcast programs on a satellite platform or creating video on demand (VOD) subtitles or foreign language translation primitives on the web or a mobile multimedia service platform .

In addition, the present invention can provide an apparatus for generating high quality closed caption subtitles suitable for reuse in comparison with existing closed caption subtitles.

1 is a block diagram of a caption subtitle correction apparatus according to an embodiment of the present invention.
2 is a block diagram of a correction unit according to the first embodiment of the present invention.
3 is a block diagram of a correction unit according to a second embodiment of the present invention.
4 is a block diagram of a correction unit according to a third embodiment of the present invention.
FIG. 5 illustrates a system in which a caption caption correction apparatus according to an embodiment of the present invention is used.
6 is a flowchart of a caption subtitle correction method according to an embodiment of the present invention.
7A to 7C are views illustrating examples of speech recognition using a speech dictionary according to an embodiment of the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

In Korea, it is legally obliged to provide closed caption service in terrestrial broadcasting, and it is being produced in real-time stenography method. However, even after real-time broadcasting, it is increasingly necessary to provide closed caption service in other service platforms. In order to efficiently reuse existing closed caption subtitles, the present invention creates a closed caption subtitles file and corrects / transforms the closed caption subtitles file into a high-quality subtitle file that can be provided to applications of various service systems Can be provided. The present invention can automatically correct the time synchronization of the closed caption caption using speech recognition, thereby minimizing manual correction.

1 is a block diagram of a caption subtitle correction apparatus according to an embodiment of the present invention.

Referring to FIG. 1, the caption caption correcting apparatus according to an exemplary embodiment of the present invention may include a receiving unit 110, a caption data generating unit 120, and a correcting unit 130 as essential components. The caption subtitle correction apparatus according to another embodiment includes a folder monitoring unit 140, a schedule monitoring unit 150, a filtering unit 160, a manual correction unit 170, and a central management system 180 can do.

The receiving unit 110 receives caption caption and broadcasting schedule information generated through stenography corresponding to the audio of the broadcast. Here, the caption subtitle means a closed caption subtitle for the disabled. The caption subtitle includes all the text corresponding to the voice in the broadcast program.

If the caption data generated by the shorthanding in real time is transmitted through the dedicated modem connected to the broadcasting company, the receiving unit 110 branches to the stage before the corresponding data is transmitted to the DTV inserter and the encoder, And the text may be stored in a log form or transmitted to the caption data generation unit 120 in real time. Broadcast scheduling information is controlled by APC (Automatic Program Control), so APC most accurately reflects changes in schedule information and information. The receiving unit 110 may receive broadcast scheduling information from a schedule interface connected to the APC. The receiving unit 110 may receive the scheduling information from the scheduling interface in a SOAP (Simple Object Access Protocol) or a Restful web service scheme. For example, the broadcast schedule schedule information may include a title, a broadcast time, a unique identifier (Program ID) of a broadcast program, a brief description of a broadcast program, .

The caption data generation unit 120 searches the broadcast program corresponding to the caption caption among the broadcast programs included in the composition schedule information on the basis of the caption caption reception time and matches the information of the searched broadcast program with the caption caption, File can be generated. The caption file can be specified in various types of caption file, such as SAMI, SRT, and SUB, which are generally distributed. When the caption text is received in the receiving unit 110, do.

Further, the caption data (or metadata in other names) of the caption file may include at least one of a title of the broadcast program, a broadcast time, a brief description of the broadcast program, a unique identifier (Program ID) of the broadcast program, An edited version of the caption file format, a character encoding format, and a language code of the caption subtitle. Caption The language code of the subtitle indicates whether the caption caption is Korean, English or any other language. The caption data generation unit 120 may upload the generated caption file and the caption data to the central management system 180. [

The correcting unit 130 can correct the time synchronization of the caption subtitle of the subtitle file generated by the subtitle data generator 120 based on the speech recognition result of the broadcast image. The correcting unit 130 may correct the time synchronization of the caption subtitle so that the timing of the speech recognized in the video image of the broadcast program matched with the caption subtitle matches the time stamp of the caption subtitle. As a method of recognizing a voice in a video image of a broadcast program, various types of voice recognition methods known to those of ordinary skill in the art can be used. Also, the correction unit 130 may change the text format according to the purpose of using the subtitle file, check the accuracy of the text spelling or correction result, and display the text color differently according to the speaker.

If the video file name of the broadcasting program included in the pre-registered folder and the name of the subtitle file are the same, the folder monitoring unit 140 monitors the video file name and the subtitle file for the purpose of automatically starting the correction operation in the correcting unit 130 You can monitor the name. If the unique identifier (Program ID) of the broadcast program is included in the name of the video file, the caption of the corresponding video is requested with a unique identifier (Program ID) through the interface of the central management system 180 After downloading, you can start to work.

When registering a new folder for folder monitoring, it is also possible to specify in advance whether the automatic operation is performed. That is, a sentence formatting method corresponding to the purpose of future use of the subtitle to be corrected is set in advance for each folder to be monitored, and when the new work target is inputted, the folder monitoring unit 140 automatically You can request to perform a task. That is, it is possible to designate a method such as subtitle for DTV, VOD subtitles of 40 bytes width and 2 lines, and completed sentences for foreign language translation, at the time of registering a new monitoring folder.

The schedule monitoring unit 150 may monitor the scheduling schedule and perform the task scheduling for the purpose of correcting the time synchronization of the caption subtitles at a predetermined period based on the broadcast scheduling information. The correcting unit 130 can utilize only the schedule monitoring method without using the folder monitoring unit 140 as a method for starting automatic operation. For example, when the system operator views the schedule table provided by the schedule monitoring unit 150 and designates a task program target and an order, the schedule monitor 150 acquires a unique identifier (Program ID ) To the central management system 180, and requests the image file from the repository to allow the correction unit 130 to start the operation.

The filtering unit 160 may filter a specific program from the broadcast schedule schedule information so that the corrector 130 does not operate in a specific program. For example, in the case of a music broadcast or a broadcast relaying a concert, it is difficult to distinguish between voice and music, and the accuracy of the operation of the corrector 130 may be deteriorated. Thus, a particular program can be filtered. The specific program can be adjusted variously according to the user's setting. For example, if a specific program code is filtered and registered, even if a file having a unique identifier under the program code is newly detected in the folder registered in the folder monitoring unit 140, the job is not requested to the correction unit 130 Do not.

The manual correction unit 170 can correct the time synchronization of the caption subtitles based on the time synchronization input from the user among the caption subtitles, according to the needs of the user. The timestamp can be adjusted based on the user's input, either for the entire caption subtitle, or for each section or sentence. Not only time synchronization, but also the length, merging, and segmentation of sentences can be manually adjusted. In the passive correction unit 170, only the corrected spelling and timestamp correction details are corrected by the correction unit 130 so that the user can confirm and change the corrected contents.

The corrected caption subtitles in the correction unit 130 may be transmitted to the central management system 180. [ The corrected caption subtitles in the manual correction unit 170 can also be transmitted to the central management system 180. [ The central management system 180 can manage the corrected caption subtitles. The central management system 180 may provide stored caption subtitles according to the needs of various entities.

2 is a block diagram of a correction unit according to the first embodiment of the present invention.

Referring to FIG. 2, a correction unit 130A according to an embodiment may include a voice recognition unit 210 and a time synchronization correction unit 220. FIG.

The voice recognition unit 210 can recognize the voice in the moving picture of the broadcast program. The speech recognition unit 210 may use an acoustic phonetic method, a pattern recognition method, a neural network method, a SVM (Support Vector Machine) method, an artificial intelligence method, or the like. For example, pattern recognition methods may include templates, statistical techniques using Dynamic Time Warping (DTW), Vector Quantization (VQ), and HMM (Hidden Markov Model).

The speech recognition unit 210 can identify a speaker in a moving image of a broadcast program and recognize a speech or voice uttered by a speaker.

The speech recognition unit 210 can detect a section in which there is no human voice but a background music.

When recognizing the speech, the speech recognition unit 210 searches for a word in a speech dictionary made up of the text of the caption file generated by the caption data generation unit 120 of FIG. 1, adopts the matched word as a speech recognition result, The speech time point of the speech can be generated as a new time stamp.

For example, when recognizing only one section of a speech, the speech recognition unit 210 extracts a predetermined margin from the text of the subtitle file generated by the subtitle data generation unit 120, ), The speech dictionary is created using only the texts having the time stamps in the range of " 0 " to " 0 ", and only the words of the speech dictionary are searched to adopt the matched words as speech recognition results.

In another example, the speech recognition unit 210 can increase the accuracy of the entire speech recognition by reducing the range of dictionaries to be referred to in speech recognition by repeating the segment speech recognition process even when performing speech recognition on the entire image .

The time synchronization corrector 220 may compare the time stamp of the voice recognized by the voice recognition unit 210 with the time stamp of the caption data to correct the time synchronization of the caption caption included in the caption data. The time synchronization corrector 220 can correct the time synchronization by correcting the time stamp of the caption subtitle to the time stamp of the recognized voice.

The time synchronization correcting unit 220 performs the interval correction based on the portion matched in the speech dictionary in a predetermined margin range based on the time stamp of the speech recognized by the speech recognition unit 210 can do. In the speech dictionary, text recognized in advance by speech recognition may be stored together with a time stamp.

3 is a block diagram of a correction unit according to a second embodiment of the present invention.

Referring to FIG. 3, the correction unit 130B according to an exemplary embodiment may include a format changing unit 310, a voice recognizing unit 320, and a time synchronization correcting unit 330. FIG.

The format changing unit 310 can change the format of the caption subtitle in the format set in accordance with the use of the caption subtitle. The caption data can be used for various applications such as digital television (DTV), multimedia on the web, mobile multimedia, video on demand (VOD), and the like. However, it is necessary to change the format because the caption subtitle is displayed for each application or the space where the caption subtitle can be displayed on one line on the screen (for example, the width and height allocated to the caption subtitle) is different.

Table 1 shows the format of the caption subtitle according to each application.

[Table 1]

It is important to note how the caption subtitles are displayed (e.g., roll-up or pop-on) and how many bytes per line of the screen Information can be preset for each type of application. For example, the subtitles generated by the foreign language translation primitive should be formatted in a complete sentence unit without separation in the middle, taking into account the difference in word order between languages. The format changing unit 310 can change the display format of the caption subtitle and the line format based on the information set for each application.

When recognizing the speech, the speech recognition unit 320 searches for a word in a speech dictionary made up of the text of the caption file generated by the caption data generation unit 120 of FIG. 1, adopts the matched word as a speech recognition result, The speech time point of the speech can be generated as a new time stamp.

The voice recognition unit 320 may perform the exemplary method illustrated in FIG.

The time synchronization corrector 330 may compare the time stamp of the voice recognized in the moving picture with the time stamp of the caption data to correct the time synchronization of the caption subtitle of the changed format included in the caption data.

The time synchronization corrector 330 compares the time stamps of the speech uttered by the moving picture of the broadcast program with the timestamps of the speech dictionary composed of the text of the caption subtitle of the changed format in the format changing unit 310, Can be corrected. For example, the speech dictionary may consist of text of caption subtitles in a modified format. Based on the timing of the voice to be uttered, the time synchronization correcting unit 330 searches for a section matched with the timing in the voice dictionary, and searches for a portion matched with the uttered voice to correct time synchronization.

4 is a block diagram of a correction unit according to a third embodiment of the present invention.

4, the correcting unit 130C according to one embodiment includes a format changing unit 410, a voice recognizing unit 420, a time synchronizing correcting unit 430, a text coloring unit 440, and a correcting unit 450 ).

The description of the format changing unit 410, the voice recognizing unit 420 and the time synchronization correcting unit 430 may be applied as is in the description of FIG.

The text coloring unit 440 may color the text of the caption subtitles so that the caption subtitles are distinguished for each speaker of the broadcast program. For example, the text may be colored in yellow, white, red, or the like so that text is distinguished by the speaker.

The correcting unit 450 can check the spelling of the text of the caption subtitle and correct the error. The correcting unit 450 can check the spelling of the caption subtitles based on the text spelling dictionary, and correct the erroneous portion. In addition, when it is determined that the correction error is a correction error using the time stamp difference corrected by the time synchronization correction unit 430 and the music interval information detected by the voice recognition unit 420, the correction unit 450 further corrects the time stamp can do.

FIG. 5 illustrates a system in which a caption caption correcting apparatus according to another embodiment of the present invention is used. FIG. 5 illustrates an embodiment in terms of the name of a model that can be used in a broadcasting station and the compatibility with existing equipment using new equipment.

5, the system may include a caption modem 510, a caption subtitle correction unit 520, an APC (Automatic Program Control) 530, a central management system 540, and an external service 550.

The caption modem 510 may receive the caption caption generated through stenography corresponding to the audio of the broadcast and transmit the caption caption to the DTV caption inserter and the encoder.

The caption subtitle correction apparatus 520 can correct the caption subtitle received from the caption modem 510 in the manner described in FIGS. The caption subtitle correcting apparatus 520 can correct the time synchronization of the caption subtitle to match the actual voice of the broadcast and change the format of the caption subtitle for each application to generate high quality subtitle data for easy reuse in applications.

The APC (Automatic Program Control) 530 may provide the program scheduling information to the caption subtitle compensating apparatus 520. The caption subtitle correction apparatus 520 receives the composition schedule information, and can generate the caption file and the caption data including the corrected caption subtitle.

A Central Management System (CMS) 540 may store the caption data generated by the caption caption correcting apparatus 520. The CMS 540 may provide the caption data to the internal / external service 550 so that the internal / external service 550 reuses the caption data.

Internal and external services 550 may include re-broadcasting digital television (DTV), reusing broadcast programs on a cable or satellite platform, or generating video on demand (VOD) subtitles or foreign language translations on web or mobile multimedia services platforms And so on.

The caption subtitle correction apparatus 520 may include a closed caption recorder (CRS). The CRS may be configured to include the receiving unit 110, the caption data generating unit 120, and the schedule monitoring unit 150 of FIG. The CRS can convert the caption subtitle into an original subtitle file when the caption subtitle is received from the speeding device through the caption modem 510. [ The CRS may upload the original subtitle file to the central management system 540. [ For example, an original subtitle file may have a Synchronized Accessible Media Interchange (SAMI) format and have an extension of .smi.

The CRS can generate a relative time stamp on the text line of the subtitle file by comparing the reception time of the caption subtitle received from the subtitle modem 510 with the program schedule information using the daily program information received from the APC 530 have.

In the case of a prerecorded program, a subtitle file may be created immediately after the program is terminated and uploaded to the CMS 540. In the case of a real-time program, since the schedule information does not reflect an unexpected schedule, the subtitle file creation is completed after about 10 minutes after the program ends. When uploading a new subtitle file, the CRS may upload information such as unique program ID (PID), language code, etc. to the CMS 540 together.

Post-processing for correcting the caption subtitle can be performed more easily by using the original subtitle file. Every sentence in the original subtitle file contains an end mark. An end mark means, for example, a period, a question mark, an exclamation point, and so on. If the speaker changes, the sentence can start with a dash and the color of the sentence can change. And may be inserted as a parenthesis if the closed caption subtitles include non-speech elements (e.g., wind sounds).

Whether it is by means of read-only or speech recognition, there is a great likelihood of delays in online captioning. The delay time for each sentence can be different. Due to this irregular delay, the caption subtitles may be displayed on a screen that is not matched. Therefore, if an original subtitle file in which a delay occurs is used as a subtitle of a VOD or a DVD, inconvenience may be caused to the viewer. The time delay of closed caption subtitles has a large effect on visual satisfaction. In addition, the text format of the subtitle may be different depending on the type of application. However, there is a great inconvenience of cost and time in manually adjusting the delay time and changing the format.

Korean broadcasters do not use speech recognition for real-time closed captioning. This is because there is not a large number of channels or programs to generate closed caption subtitles and there is still a concern about the accuracy of speech recognition of Hangul. In a broadcast environment, more stringent accuracy is required.

 However, the method of using speech recognition for an original subtitle file in which an irregular time delay is reflected may be technically more useful than dictation. In addition, the cost added for correction can be minimized. Therefore, an automated server program needs to be implemented that can adjust time and change the format of the text. Of course, TV programs may not be suitable for speech recognition because they may include various genres. Therefore, an exceptional case which is not suitable for speech recognition needs to be distinguished separately, and passive correction must also be performed as a method for compensating the exception.

The automation server program (WiseSync) can be used in the caption subtitle correction apparatus 520. [ For example, the above program can be used in the process of performing the functions of the controller 130, the folder monitoring unit 140, the schedule monitoring unit 150, and the filtering unit 160 of FIG. The purpose of this program is to automatically change the format of the subtitles and to sort them. For this purpose, we decided to name it WiseSync.

For example, the input of WiseSync may be an original subtitle file generated by the CRS that stores DTV closed caption subtitles. DTV closed caption subtitles can be decoded and generated in a roll-up style. Subtitle words may appear from left to right until the maximum line length of 40 bytes is filled. Sometimes a line change occurs in the middle of words.

Before reordering subtitles using speech recognition in WiseSync, you can change the format of subtitles depending on the application's purpose. For example, in the case of VOD, WiseSync can change the format of subtitles in a pop-on style without losing the middle of words. The maximum length and height of the caption subtitles can be set in advance. In the draft for translation, the completed sentence must be included for each line.

The caption subtitle changed in format can acquire a new time stamp through voice recognition. The words of the caption subtitles whose format has been changed can be used to make a speech dictionary. The time of the utterance from the first word to the last word in the speech dictionary can be continuously matched. By using the time stamp of the original subtitle file, the search range for searching the portion of the speech dictionary matching the word of the subtitle can be narrowed. By narrowing the search range, the accuracy of the post-processing result can be maximized.

WiseSync can automatically perform the watch folder function, and can exchange caption data with the CMS 540. WiseSync can monitor the assigned folder to find subtitle files that need to be time synchronized. If the video file and the subtitle file have the same file name, the video file and the subtitle file can be regarded as a pair, and the post-processing operation can be performed. Alternatively, if the name of the video file includes the program ID, an operation of retrieving the original subtitle file having the same program ID from the CMS 540 can be performed.

WiseSync has a fairly good overall accuracy, but is exceptional in certain genres such as music shows or concerts (for example, Music Bank, Gag Concert). Therefore, WiseSync can perform a function of searching for a specific program from the program schedule to filter the programs corresponding to the exception from the post-processing. If a program that does not carry out a post-process is included in the broadcast program list, a post-process operation may not be performed for subsequent episodes of the program even when the post-process operation requirement is satisfied.

The terminal program (WiseCat) can be used in the caption subtitle correction apparatus 520. For example, the above program may be used in the course of performing the function of the manual correction unit 170 of FIG. The purpose of WiseCat is to complement the behavior of WiseSync. WiseCat can have a form for editing subtitles on a regular PC, and can perform some special functions.

WiseCat can be used when you want to connect to WiseSync remotely, and you can use WiseSync function through remote connection. WiseCat can manually shift the caption subtitle time according to the user's input. WiseCat also allows you to edit subtitles using other editing features such as customized checks or text coloring.

Various video versions may exist for one broadcast program. It can be edited or trimmed differently according to various image standards. For example, there may be an encoded image and an edited image. The encoded image may be stored in a video archive, and the edited image may be stored in an integrated CMS 540 hub. If the image is edited or trimmed, the previous synchronization can not be maintained. Therefore, it is necessary to update the subtitle version according to the new video version. Once the CRS creates an edited version of the subtitles, it can be uploaded to the CMS 540. Other systems, such as internal and external services 550, may download and use caption subtitles of appropriate caption editing versions according to the purpose. Therefore, when a subtitle file is uploaded to the CMS 540 in the CRS, a system ID, a unique ID, an edition version, a program ID, a language code, and the like can be uploaded together.

Closed caption subtitles with time synchronization corrected can be used as metadata. For example, closed caption subtitles can be used to retrieve a desired scene from content stored on a video archive. For example, a scene when a specific person or a specific place is referred to in an image can be retrieved through closed caption subtitles.

6 is a flowchart of a caption subtitle correction method according to an embodiment of the present invention.

In operation 610, the caption subtitle correction apparatus according to an exemplary embodiment may receive caption closed caption and broadcast schedule schedule information generated through stenography corresponding to a voice of a broadcast.

In operation 620, the caption subtitle correction apparatus according to an exemplary embodiment may search for broadcast programs corresponding to the caption subtitles in the broadcast programs included in the composition schedule information, based on the reception time of the caption subtitles.

In operation 630, the caption subtitle correction apparatus according to an exemplary embodiment may generate the subtitle file and the metadata by matching the information of the searched broadcast program with the caption subtitle. The metadata may be named caption data. The metadata may include a title of the broadcast program, a broadcast time of the broadcast program, a brief description of the broadcast program, a unique identifier of the broadcast program, a broadcast degree of the broadcast program, Format, and language code.

In operation 640, the caption subtitle correction apparatus according to an exemplary embodiment may correct the text format and the time synchronization value of each caption subtitles of the caption file based on the speech recognition result of the broadcast.

The caption subtitle correcting apparatus according to an embodiment can recognize the voice in the moving picture of the broadcast program, compare the time stamp of the recognized voice with the time stamp of the caption data, and correct time synchronization of the caption subtitle.

The caption subtitle correction apparatus according to an embodiment changes the format of the caption subtitle in the format set for the use of the caption subtitle, recognizes the voice in the moving picture of the broadcast program, By replacing the time stamp with a new time stamp generated through speech recognition, time synchronization can be corrected.

The method of FIG. 1 to FIG. 4 may be applied to the method performed by the caption subtitle correction apparatus in each step of FIG.

7A to 7C are views illustrating examples of speech recognition using a speech dictionary according to an embodiment of the present invention.

Referring to FIG. 7A, the speech recognition unit of FIGS. 2, 3 and 4 extracts a speech dictionary (a word dictionary of FIG. 7A) created by text of a caption file generated by the caption data generation unit 120 of FIG. It is possible to search for a word and employ the matched word as a speech recognition result, thereby generating a speech time point of the speech with a new time stamp.

Referring to FIG. 7B, when it is desired to recognize only one section of speech (for example, utterance 2), the utterance timing of the corresponding section of the text of the subtitle file generated by the subtitle data generator 120 of FIG. A speech dictionary may be created using only texts having a time stamp within a predetermined range of margins, and only the words of the speech dictionary may be searched to adopt a matched word as a speech recognition result.

Referring to FIG. 7C, the speech recognition unit of FIGS. 2, 3 and 4 can reduce the range of a dictionary to be referred to in speech recognition by repeating the speech recognition recognition process even when performing speech recognition on the entire image. The accuracy of the entire speech recognition can be improved.

The methods according to embodiments of the present invention may be implemented in the form of program instructions that can be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, and the like, alone or in combination. The program instructions recorded on the medium may be those specially designed and constructed for the present invention or may be available to those skilled in the art of computer software.

While the invention has been shown and described with reference to certain preferred embodiments thereof, it will be understood by those of ordinary skill in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims. This is possible.

Therefore, the scope of the present invention should not be limited to the described embodiments, but should be determined by the equivalents of the claims, as well as the claims.

Claims (14)

A receiving unit for receiving a caption subtitle generated through stenography corresponding to a voice of a broadcast and schedule schedule information of the broadcast;
Searching for a broadcasting program corresponding to the caption subtitle from the broadcasting programs included in the programming schedule information based on the reception time of the caption subtitle and matching the information of the searched broadcasting program with the caption subtitle, A subtitle data generating unit for generating a subtitle data; And
A correcting unit for correcting a text format and a time synchronization value of each caption subtitles of the subtitles file based on a purpose of using the subtitles file and a speech recognition result of the broadcast,
Lt; / RTI >
The correction unit
A voice recognition unit for recognizing a voice in a moving picture of the broadcast program; And
A time synchronizing corrector for comparing the time stamp of the recognized voice with the time stamp of the caption data to correct time synchronization of the caption subtitle;
/ RTI >
The time synchronization corrector
And performs a section correction based on a portion matched in a speech dictionary in a range of a predetermined margin based on a time stamp of the recognized speech. The method according to claim 1,
Wherein the caption file includes a caption text of a corresponding program and a time stamp which is a time position at which each caption text appears,
The caption data may include at least one of a title of the broadcast program, a broadcast time of the broadcast program, a brief description of the broadcast program, a unique identifier of the broadcast program, a broadcast degree of the broadcast program, an edited version of the caption subtitle, Language code that includes at least one of
Caption subtitle correction device. delete delete A receiving unit for receiving a caption subtitle generated through stenography corresponding to a voice of a broadcast and schedule schedule information of the broadcast;
Searching for a broadcasting program corresponding to the caption subtitle from the broadcasting programs included in the programming schedule information based on the reception time of the caption subtitle and matching the information of the searched broadcasting program with the caption subtitle, A subtitle data generating unit for generating a subtitle data; And
A correcting unit for correcting a text format and a time synchronization value of each caption subtitles of the subtitles file based on a purpose of using the subtitles file and a speech recognition result of the broadcast,
Lt; / RTI >
The correction unit
A format changing unit for changing a format of the caption subtitle in a format set in accordance with a use purpose of the caption subtitle;
A voice recognition unit for recognizing a voice in a moving picture of the broadcast program;
A time synchronization correcting unit for correcting time synchronization by replacing a new time stamp assigned to a first appearing word of each line of the caption subtitle with a new time stamp generated by the speech recognition unit;
A correcting unit for checking and correcting the spelling of the caption subtitle and the time synchronization correction error; And
A text coloring part for coloring the text of the caption subtitle so as to be distinguished by a speaker of the broadcast program,
Caption correction unit. delete A receiving unit for receiving a caption subtitle generated through stenography corresponding to a voice of a broadcast and schedule schedule information of the broadcast;
Searching for a broadcasting program corresponding to the caption subtitle from the broadcasting programs included in the programming schedule information based on the reception time of the caption subtitle and matching the information of the searched broadcasting program with the caption subtitle, A subtitle data generating unit for generating a subtitle data;
A correcting unit correcting a text format and a time synchronization value of each caption subtitles of the caption file based on a purpose of using the caption file and a speech recognition result of the broadcast; And
If the video file name of the broadcast program included in the folder is the same as the name of the caption data, the name of the video file name and the name of the caption data are monitored for the purpose of correcting the time synchronization of the caption caption included in the caption data Folder monitoring unit
Caption correction unit. A receiving unit for receiving a caption subtitle generated through stenography corresponding to a voice of a broadcast and schedule schedule information of the broadcast;
Searching for a broadcasting program corresponding to the caption subtitle from the broadcasting programs included in the programming schedule information based on the reception time of the caption subtitle and matching the information of the searched broadcasting program with the caption subtitle, A subtitle data generating unit for generating a subtitle data;
A correcting unit correcting a text format and a time synchronization value of each caption subtitles of the caption file based on a purpose of using the caption file and a speech recognition result of the broadcast; And
And a schedule monitoring unit for monitoring the program schedule for the purpose of correcting time synchronization of the caption subtitles at a predetermined period based on the program scheduling information of the broadcast,
Caption correction unit. A receiving unit for receiving a caption subtitle generated through stenography corresponding to a voice of a broadcast and schedule schedule information of the broadcast;
Searching for a broadcasting program corresponding to the caption subtitle from the broadcasting programs included in the programming schedule information based on the reception time of the caption subtitle and matching the information of the searched broadcasting program with the caption subtitle, A subtitle data generating unit for generating a subtitle data;
A correcting unit correcting a text format and a time synchronization value of each caption subtitles of the caption file based on a purpose of using the caption file and a speech recognition result of the broadcast; And
A filtering unit for filtering the specific program from the broadcast schedule schedule information so that the correction unit does not operate in the specific program,
Caption correction unit. 6. The method of claim 5,
A manual correction unit that corrects time synchronization of the caption subtitle and confirms and confirms a correction result of the caption subtitle based on time synchronization input from the user among the caption subtitles according to a user's need;
Further comprising a caption subtitle correction unit. Receiving a closed caption subtitle generated through stenography corresponding to a voice of a broadcast and schedule schedule information of the broadcast;
Retrieving a broadcast program corresponding to the caption subtitle from the broadcast programs included in the programming schedule information based on the reception time of the caption subtitle;
Generating subtitle file and subtitle data by matching information of the searched broadcasting program with the caption subtitle; And
Correcting the text format and the time synchronization value of each caption subtitles of the caption file based on the speech recognition result of the broadcast
Lt; / RTI >
The step of correcting the text format and the time synchronization value
Recognizing a voice in a moving picture of the broadcast program; And
Comparing the time stamp of the recognized voice with the time stamp of the caption data to correct time synchronization of the caption subtitle;
Lt; / RTI >
The step of correcting the time synchronization of the caption subtitle
And performing a section correction based on a portion matched in a speech dictionary in a range of a predetermined margin based on a time stamp of the recognized speech. delete delete 12. The method of claim 11,
The step of correcting
Changing a format of the caption subtitle in a format set in accordance with a use purpose of the caption subtitle; And
Recognizing a voice in a moving picture of the broadcast program; And
Correcting the time synchronization by replacing the new time stamp assigned to the first appearing word of each line of the caption subtitle with a new time stamp generated through speech recognition
And a caption subtitle correction method.

Images