KR101582574B1 - Apparatus and service method for providing many languages of digital broadcasting using real time translation - Google Patents

Abstract

The present invention provides a method and system for providing a multilingual subtitle service for digital broadcasting in which a subtitle is analyzed and a subtitle is analyzed in a desired language and synchronized with a selected digital broadcasting program while streaming a selected digital broadcasting program when a user selects a desired digital broadcasting program and a language Apparatus and method.

Description

TECHNICAL FIELD The present invention relates to a method and apparatus for providing a multi-language subtitle of a digital broadcasting through real-

The present invention relates to an apparatus and a method for providing a subtitle of a digital broadcasting, and more particularly, to an apparatus and method for providing a multilingual subtitle of a digital broadcasting capable of providing subtitles for digital broadcasting in various languages through real time translation.

Recently, with the development of a broadcasting signal transmission method, a digital TV adopting a digital transmission method has been spreading. In particular, high-quality and high-quality digital broadcasting broadcasted via satellite, terrestrial, and cable has many advantages, so that it can provide high quality, high quality, and various additional information as compared with conventional analog broadcasting.

As one of various supplementary services, there is a demand for a broadcast caption machine translation system which enables a digital broadcast to extract a caption signal from a broadcast signal and automatically translate the caption signal into a language desired by the viewer, It is a fact that it is increasing.

Here, the broadcast subtitle is a subtitle signal inserted into a video signal transmitted from a broadcasting company. Recently, many broadcasting companies have implemented such a subtitle broadcast for the hearing-impaired person. In the United States, subtitling functions have been mandated for TVs over 13 inches since 1990, and in Korea, caption broadcasting programs are being expanded in TV broadcasting companies and CATVs. The continual expansion of foreign subtitle broadcasting such as CNN, NHK and AFKN It is expected.

However, in TV broadcast subtitles, spoken language and written language are mixed and used in various broadcasting genres such as drama, literary current affairs, and amusement programs as well as news. In news, various terms and expressions appear in various fields, There is a technically significant difficulty in developing a broadcast subtitle machine translation system having translation performance.

In order to solve such technical difficulties, Korean Patent Laid-Open Publication No. 1997-56985 (published on July 31, 1997) discloses a Korean language and a foreign language translation unit separately in the TV so that broadcast subtitle data can be translated into a mother tongue or a foreign language And a TV having a broadcasting subtitle translation function capable of providing a convenient viewing to viewers.

However, since the television having the broadcast subtitle translation function extracts the broadcast subtitle data from the received broadcast signal, and translates the subtitle data according to the user's request and displays it on the TV screen, There is a drawback to buying a TV. Also, when a broadcast signal is input through a media other than a TV set such as a satellite set-top box, a video player, and a DMB terminal, a subtitle translation function can not be provided.

SUMMARY OF THE INVENTION Accordingly, the present invention has been made keeping in mind the above problems occurring in the prior art, and an object of the present invention is to provide a method and apparatus for decoding a digital broadcast program, The present invention provides an apparatus and method for providing a multilingual subtitle of a digital broadcast.

In order to achieve the above object, according to an embodiment of the present invention, there is provided an apparatus for providing a multilingual subtitle for digital broadcasting through real-time translation, including: a subtitle extracting unit for extracting and decoding a subtitle from digital broadcasting data streamed by a user terminal; A caption analyzing unit for analyzing the caption data extracted by the caption extracting unit on a morphological unit basis and reconstructing the caption data into one sentence form to be translated; A translation request unit for requesting translation of the reconstituted caption data from the caption analysis unit to a predetermined translation service apparatus; A translation result collecting unit for receiving and collecting the caption data translated by the translation request; And a translation subtitle transmission unit for receiving the input of the language to be translated from the user terminal and synchronizing the translated caption data collected through the translation result integration unit with the digital broadcast data.

The translation result collecting unit may match the reproduction time of the translated caption data based on the audio signal of the digital broadcast data.

The translation result collecting unit may calculate a time required for the translation process by confirming the time requested by the translation request unit and the time when the translated caption data is received, and if the time required for the translation process exceeds a predetermined time The user can request to delay the video data of the digital broadcast data by the elapsed time.

In addition, when a translation request is requested from the translation request unit, the apparatus for providing a multilingual caption in digital broadcasting through real-time translation according to an embodiment of the present invention receives the caption data reconstructed by the caption analyzing unit and performs automatic translation And a translation service device.

The digital broadcast data may include real-time broadcast data and Video On Demand (VOD) data.

According to another embodiment of the present invention, there is provided a method for providing a multilingual caption providing service of digital broadcasting through real-time translation, wherein a multilingual caption providing service device (hereinafter referred to as a service device) selects a digital broadcasting program and a caption language Receiving an input; Extracting and decoding the subtitles from the digital broadcasting data streamed by the user terminal at the request of the user terminal; Analyzing the subtitles decoded by the service apparatus on a morpheme-by-morpheme basis, and reconstructing the subtitles into one sentence requiring translation; Requesting the service apparatus to translate the reconstituted caption data into a predetermined translation service apparatus; Receiving the translated caption data from the translation service apparatus and collecting the translated caption data; And transmitting the translated caption data collected by the service apparatus in synchronization with the digital broadcast data.

Wherein the service device checks the time when the service device requests translation from the translation service device and the time when the translated device data is received from the translation service device, And request that the video data of the digital broadcast data be delayed by a time that has elapsed after a predetermined period of time has elapsed. At this time, when the request is generated, it can be reflected in the step of transmitting in synchronization with the digital broadcasting data.

The service device may request to delay the reproduction time of the video data based on the delay time between the reproduction time of the caption data extracted from the digital broadcast data and the reproduction time of the video data in the step of collecting the translated caption data.

According to the present invention, a user can simultaneously view a subtitle translated into a desired language when viewing a digital broadcasting program, thereby maximizing the user's convenience. Particularly, there is an effect that digital broadcasting can be viewed immediately without having to wait until a digital broadcasting program is translated and converted into a database.

1 is a network connection diagram of an apparatus for providing a multilingual subtitle according to an embodiment of the present invention;
FIG. 2 is a detailed configuration diagram of an apparatus for providing a multilingual subtitle according to an embodiment of the present invention; FIG.
3 is a flowchart illustrating a method of providing a multi-language subtitle according to an exemplary embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. The configuration of the present invention and the operation and effect thereof will be clearly understood through the following detailed description. Before describing the present invention in detail, the same components are denoted by the same reference symbols as possible even if they are displayed on different drawings. In the case where it is judged that the gist of the present invention may be blurred to a known configuration, do.

FIG. 1 is a network configuration diagram of an apparatus for providing a multi-language subtitle according to an embodiment of the present invention, and FIG. 2 is a detailed configuration diagram of an apparatus for providing a multi-language subtitle according to an embodiment of the present invention.

Referring first to FIG. 1, an apparatus 100 for providing a multilingual subtitle according to an exemplary embodiment of the present invention basically performs a function of translating subtitles of moving pictures into multiple languages in real time.

Particularly, the multilingual caption providing service apparatus 100 according to the embodiment of the present invention interworks between a broadcasting company and a user terminal 200 to stream digital broadcasting data provided by a broadcaster and at the same time, Translated into languages and provided in real time. As described above, the multilingual caption providing service apparatus 100 according to the embodiment of the present invention provides a real-time translation technology.

The broadcasting company plays the role of producing, editing, and modifying the broadcast contents and providing the broadcast contents to the broadcast receiving unit 1. A broadcasting company may include a program provider (PP), a terrestrial or a contents provider (CP), and the like.

Digital broadcasting data provided by a broadcasting company may include both existing broadcasting data and various moving picture data on the Internet. For example, real-time broadcast data, video on demand (VOD) data, cable broadcast data, and the like.

The digital broadcast data is generally received via the broadcast receiver 1 and converted into a format suitable for the environment of the user terminal 200 via the streaming processing unit 2. [ And transmitted to the user terminal 200 through the broadcast transmission unit 3 in the converted form. At this time, the transmission method to the user terminal 200 may use one of broadcasting or unicasting, or a combination thereof.

Alternatively, the digital broadcast data may be provided in advance in the VOD (Video On Demand) server 4, and may be extracted and provided according to the request of the user terminal 200.

The streaming processing unit 2 will be described in detail below with reference to FIG.

Based on this configuration, the multilingual caption providing service apparatus 100 according to the embodiment of the present invention includes a caption extracting unit 110, a caption analyzing unit 120, a translation request unit 130, a translation result collecting unit 140, A subtitle management unit 150, and a translation subtitle transmission unit 160. [

The subtitle extracting unit 110 extracts subtitles from digital broadcasting data streamed through the broadcast receiver 1 and decodes the subtitles. When caption data is included in the digital broadcast data, its extraction can be easily performed. That is, the digital broadcasting data received through the broadcasting receiver 1 includes normal image data and voice data, and when text subtitle data different therefrom exists, they are combined and provided as one file. Therefore, extraction of caption data can be easily performed by those skilled in the art.

1, the subtitle extracting unit 110 can directly extract digital broadcast data streamed from the rear end of the broadcast receiving unit 1 through the broadcast receiving unit 1, It may be extracted in association with the moving picture separating unit 21 of the streaming processing unit 2 for separating the streaming digital broadcasting data into video, audio and subtitles.

At the time of extraction, caption start time for synchronizing with caption data is also extracted.

At this time, when the subtitle start time is normally received together with the video data through the broadcast receiver 1, the playback time of the video data matches well with the playback time of the video data. However, a closed caption system If the subtitle data is additionally included, the reproduction time of the video data may be delayed by about several seconds.

The caption analyzing unit 120 analyzes the caption data extracted by the caption extracting unit 110 by morpheme units such as words, characters, and symbols, and reconstructs the caption data into a single sentence to be requested for translation. In addition, the caption analyzing unit 120 may analyze the caption data on a sentence-by-sentence basis including the phrase.

The translation request unit 130 determines a device to be translated (hereinafter referred to as a translation service apparatus) in advance and transmits the caption data analyzed by the caption analyzing unit 120 to the translation service apparatus 300 to request a translation.

The translation service apparatus 300 automatically translates the requested caption data into the desired language through the translation request unit 130. [ The translation service apparatus 300 can be divided into separate independent apparatuses in association with an external translation company and may be included in the multilingual caption providing apparatus 100 of the present invention. For example, the translation service apparatus 300 may be a professional translator, a translation site such as a google site, or the like.

In this embodiment, the translation service apparatus 300 needs to have database data for various languages so as to enable multilingual support. Database material may include word-to-word, sentence-to-sentence corresponding language modeling data. In addition, when analyzing original sentences, various analysis algorithms are constructed in database data as well as methods of statistical analysis based on word type, word sequence, and phrase type for each language to be translated.

The translation result collecting unit 140 receives the translated caption data (hereinafter, translated caption data) from the translation service apparatus 300 and collects the translated caption data.

At this time, the translation result collecting unit 140 can match the reproduction time of the translation caption data based on the audio signal of the digital broadcast data. That is, the caption data is matched and collected in a section in which the audio signal of the digital broadcasting data is generated, thereby synchronizing the translated caption data with the digital broadcasting data.

For synchronization, the translation result collecting unit 140 calculates the time required for the translation process by checking the time for receiving the translation subtitle data and the time requested by the translation request unit 130. Accordingly, when the time required for the translation process is delayed by a predetermined time, it is possible to request the delay of the video data of the digital broadcast data by the elapsed time.

In addition, in the case where the original film data is delayed by several seconds compared with the reproduction time of the video data in the digital broadcasting data (for example, in the case where the original caption data is closed caption data input by the human body in real time for protecting the right of view of the hearing- The translation result collecting unit 140 may request that the reproduction time of the video data be delayed based on the reproduction time of the caption data extracted from the digital broadcast data (i.e., the original caption data) and the reproduction time of the video data. That is, the translation result collecting unit 140 may request to delay the reproduction time of the image data by the delay time based on the value obtained by calculating the delay time between the reproduction time of the original film data and the reproduction time of the image data

The translation subtitle management unit 150 stores the original title data for the digital broadcast data and the translation subtitle data collected through the translation result collection unit 140 in a database and reconstructs the data into a file format suitable for future use .

The translation subtitle transmission unit 160 receives the input of the language to be translated from the user terminal 200 and transmits the translation subtitle data collected through the translation result collection unit 140 in synchronization with the digital broadcast data for streaming.

Particularly, when the translation result collecting unit 140 generates a delay request for the video data of the digital broadcasting data, the translation subtitle transmitting unit 160 delays and transmits the video data of the digital broadcasting data by the delay time.

Next, referring to FIG. 2, the streaming processing unit 2 may include a configuration for separating digital broadcasting data streamed through a broadcast receiving unit (1 in FIG.

The streaming processing unit 2 includes a moving picture separating unit 21, a voice signal characteristic analyzing unit 22, a voice start time recording unit 23 and an integrating unit 24.

The moving picture separating unit 21 separates the video data and the audio data from the digital broadcasting data streamed through the broadcasting receiver (1 in FIG. 1). The moving picture separating unit 21 may include an image extracting unit 21a for extracting image data from digital broadcasting data streamed through a broadcasting receiver, and a voice extracting unit 21b for extracting voice data.

The voice signal characteristic analyzing unit 22 analyzes the voice data extracted through the voice extracting unit 21b and detects the voice start time in the moving picture. The voice start time is used to determine the subtitle start time when synchronizing subtitle data.

Analysis of speech data can utilize frequency, peak, edge detection, noise analysis, etc. of speech signal.

The voice start time recording unit 23 records the times at which the voice start time is judged in the video based on the information analyzed through the voice signal characteristic analyzer 22.

The integrating unit 24 integrates the video data and audio data extracted through the video extracting unit 21a and the audio extracting unit 21b into one broadcast data. At this time, the voice data is integrated into the form including the voice start time via the voice start time recording unit 23.

In addition, the integrating unit 24 can receive the translation subtitle data translated in real time from the multilingual caption providing service apparatus (100 of FIG. 1) described above and integrate it with the video data.

With this configuration, the present embodiment can also translate real-time broadcasting data broadcasted by the airwaves into a language desired by the user and stream the broadcasting data together with real-time broadcasting data.

Hereinafter, a service method for providing multi-language subtitle data based on the configurations of Figs. 1 and 2 will be described in detail.

Referring to FIG. 3, in a first step S1, a multilingual caption providing service apparatus (hereinafter referred to as a service apparatus) 100 receives a selection input for digital broadcasting data and a language to be viewed from a user terminal.

In the next step S2, the service apparatus 100 extracts subtitles from the digital broadcast data streamed by the user terminal through the subtitle extracting unit 110. [

In the next step S3, the caption data extracted by the caption analyzing unit 120 of the service apparatus 100 is analyzed in units of words or phrases, and reconstructed into one sentence requested for translation.

In the next step S4, the translation request unit 130 of the service apparatus 100 requests translation of the reconstituted caption data into a predetermined translation service apparatus. It also conveys selection information of the language to be translated at the time of submission.

In the next step S5, the translation result collecting unit 140 of the service apparatus 100 receives and combines the translated caption data from the translation service apparatus. Then, the translation subtitle data collected can be stored in the database of the translation subtitle management unit 150, matching the original title data.

In the collecting step, the translation result collecting unit 140 of the service apparatus 100 can confirm the time required for the translation process from the time of requesting translation to the translation service apparatus and the time of receiving the translation subtitle data. As a result, if the time required for the translation process is delayed, the digital video data of the digital broadcast data may be requested to be delayed and transmitted by a delay time in order to compensate for the delay.

In the case where the translation result collecting unit 140 of the service apparatus 100 is a closed caption, it is possible to calculate the delay time between the reproduction time of the closed caption and the reproduction time of the video data and request the reproduction time of the video data to be delayed by the delay time have.

In the next steps (S6, S7), the translation subtitle transmission unit 160 of the service apparatus 100 synchronizes the translation subtitle data collected by the translation result collection unit with the digital broadcast data, and stores the digital broadcast data including the translation subtitle data . When a translation request is generated in the translation result collecting unit 140, the image data is delayed by a delay time for accurate synchronization.

The foregoing description is merely illustrative of the present invention, and various modifications may be made by those skilled in the art without departing from the spirit of the present invention. Accordingly, the embodiments disclosed in the specification of the present invention are not intended to limit the present invention. The scope of the present invention should be construed according to the following claims, and all the techniques within the scope of equivalents should be construed as being included in the scope of the present invention.

1: Broadcast receiver 2: Data processor
3: broadcasting transmission unit 4: VOD server
100: Multilingual subtitle service device
110: caption extracting unit 120: caption analyzing unit
130: Translation request section 140: Translation result gathering section
150: translation subtitle management unit 160: translation subtitle transmission unit
200: user terminal 300: translation service device

Claims (9)

A subtitle extracting unit for extracting and decoding a subtitle from digital broadcast data that is streamed in real time at the request of the user terminal;
A caption analyzing unit for analyzing the caption data extracted by the caption extracting unit on a morphological unit basis and reconstructing the caption data into one sentence form to be translated;
A translation request unit for requesting translation of the reconstituted caption data from the caption analysis unit to a predetermined translation service apparatus;
A translation result collecting unit for receiving and collecting the caption data translated by the translation request; And
And a translation subtitle transmission unit for inputting the translated caption data received from the user terminal for the language to be translated and collected through the translation result collecting unit in synchronism with the digital broadcasting data for real-
The translation result collecting unit,
The time required for the translation process is calculated by confirming the time requested by the translation request unit and the time for receiving the translated caption data,
Wherein the request for delaying the video data of the digital broadcasting data is delayed by an amount of time elapsed when the time required for the translation process is delayed after a predetermined time elapses.
The method according to claim 1,
The translation result collecting unit
And the playback time of the translated caption data is matched based on the audio signal of the digital broadcast data.
delete The method according to claim 1,
And a translating service unit for receiving the caption data reconstructed by the caption analyzing unit and performing automatic translation when a translation request is requested from the translation request unit
Wherein the digital broadcasting service providing apparatus provides the service for providing the multilingual subtitle of digital broadcasting through real-time translation.
The method according to claim 1,
Wherein the digital broadcasting data includes real-time broadcasting data and video-on-demand (VOD) data.
Receiving a selection of a digital broadcast program and a subtitle language to be watched from a user terminal;
Extracting and decoding the subtitles of the service device from digital broadcast data in which the multilingual caption providing service device streams in real time at the request of the user terminal;
Analyzing the subtitles decoded by the multilingual caption providing service device on a morpheme-by-morpheme basis and reconstructing the sentence into a single sentence requiring translation;
Requesting translation of the reconstituted caption data from the multilingual caption providing service device to a predetermined translation service device;
Receiving the translated caption data from the translation service apparatus by the multilingual caption providing service apparatus; And
Synchronizing the translation caption data collected by the multilingual caption providing service device with the digital broadcasting data for real-time streaming,
Wherein the multilingual caption providing service apparatus, when collecting the translated caption data from the translation service apparatus,
The time required for translating the translation service apparatus and the time for receiving the translated caption data from the translation service apparatus are calculated to calculate the time required for the translation process,
Wherein the request for delaying the video data of the digital broadcasting data is delayed by an amount of time elapsed when the time required for the translation process is delayed after a predetermined time elapses.
delete The method according to claim 6,
And when the request is generated, synchronizing the digital broadcast data with the digital broadcast data and transmitting the synchronized digital broadcast data.
The method according to claim 6,
In the step of collecting the translated caption data by the service apparatus,
And requesting to delay the reproduction time of the video data based on the reproduction time of the caption data extracted from the digital broadcast data and the reproduction time of the video data. .

Images