KR102552533B1 - Device and method for providing subtitle service - Google Patents

Abstract

An apparatus for providing a caption service is disclosed. An apparatus for providing a caption service according to the present invention includes a caption extracting module that obtains caption text from caption data in a broadcast stream provided from a broadcasting company, and a caption file of broadcast content broadcast in real time by the broadcast stream using the caption text. and a caption generation module for generating and distributing the caption file so that the caption of the caption file is inserted into VOD broadcast content provided by the broadcaster to a VOD service provider.

Description

Apparatus and method for providing subtitle service {DEVICE AND METHOD FOR PROVIDING SUBTITLE SERVICE}

The present invention relates to an apparatus and method for providing a caption service capable of extracting caption text from a real-time broadcasting stream and providing fast VOD captions using the extracted caption text.

IPTV (Internet Protocol Television) is an interactive broadcasting and communication service that provides various multimedia contents such as video contents such as movies and broadcasting programs and Internet searches to a television receiver using a high-speed Internet network. Viewers can easily search the Internet using a remote control, as well as enjoy various contents and additional services provided by the Internet, such as watching movies, home shopping, home banking, online games, and MP3, from IPTV.

Meanwhile, an Internet Protocol Television (IPTV) service not only provides service subscribers with a real-time broadcasting stream provided by a broadcasting company, but also provides broadcast contents broadcasted by a broadcasting company to service subscribers as VOD.

Meanwhile, the broadcasting company provides the broadcasting content that has been broadcasted to the VOD service provider, and this can be done on the same day at the earliest or the next day for ordinary people. However, since it takes several days for VOD service providers to provide subtitles, there is a time lag between the time when broadcasting contents can be provided as VOD and the time when subtitles can be provided, and accordingly, VOD subtitles cannot be provided quickly. there was

The present invention is intended to solve the above problems, and an object of the present invention is to provide an apparatus and method for providing a caption service capable of extracting caption text from a real-time broadcasting stream and providing fast VOD captions using the extracted caption text. is to provide

An apparatus for providing a caption service according to the present invention includes a caption extracting module that obtains caption text from caption data in a broadcast stream provided from a broadcasting company, and a caption file of broadcast content broadcast in real time by the broadcast stream using the caption text. and a caption generation module for generating and distributing the caption file so that the caption of the caption file is inserted into VOD broadcast content provided by the broadcaster to a VOD service provider.

In this case, when a request for generating a caption file for the broadcast content is received from the VOD service provider receiving the broadcast content for VOD, the caption generation module may generate the caption file using the caption text.

Meanwhile, the caption generation module may obtain audio text from audio in the VOD broadcast content, and match a keyword in the audio text with a keyword in the caption text to adjust a time stamp of the caption file.

Meanwhile, the subtitle translation module may further include a subtitle translation module for generating one or more foreign language subtitle files by translating a Korean subtitle file generated using the subtitle text, and the subtitle generating module may distribute the one or more foreign language subtitle files.

Meanwhile, when a request for a real-time foreign language caption is received from the media playback device, the media playback device may further include a caption transmission module that translates the subtitle stream of the channel being watched and transmits the translation to the media playback device.

In this case, the caption transmission module may include a plurality of translation agents each including a plurality of language gauges corresponding to a plurality of channels and storing session information of media playback devices that have applied for a corresponding language.

In this case, the plurality of translation agents include a first translation agent corresponding to a first channel, and the first translation agent translates the subtitle stream obtained from the first channel by the subtitle extraction module to translate the first channel. - Generates a first language subtitle stream and a first channel-second language subtitle stream, and transmits the first channel-first language subtitle stream to media playback devices corresponding to session information stored in a language gauge of the first language. The first channel-second language subtitle stream may be transmitted in a multicast manner to media reproducing devices corresponding to session information stored in the language gauge of the second language.

In this case, the plurality of translation agents may further include a second translation agent corresponding to a second channel, and the first translation agent may, when the first translation agent is idle, obtain a subtitle stream from the second channel. to generate a second channel-first language subtitle stream, and transmit the second channel-first language subtitle stream to media playback devices corresponding to the session information stored in the language gauge of the first language in the second translation agent. Streams can be transmitted in a multicast manner.

Meanwhile, the subtitle translation module extracts translated text corresponding to some text in the Korean subtitle file from memory caching to generate the foreign language subtitle file, and the translated text is updated based on the number of accesses of the subtitle translation module. can

Meanwhile, a method for providing a caption service according to the present invention includes obtaining caption text from caption data within a broadcast stream provided from a broadcasting company, and generating a caption file of broadcast content broadcast in real time by the broadcast stream using the caption text. and distributing the caption file so that the caption of the caption file is inserted into VOD broadcast content provided by the broadcaster to a VOD service provider.

As described above, according to the present invention, subtitle text is secured from a broadcast stream in advance, and when VOD broadcast content is received, the previously secured subtitle text can be simply processed and provided as a subtitle file. It has the advantage of providing VOD service with immediate subtitles (including foreign language subtitles).

According to the present invention, by first extracting subtitle text and then adjusting a time stamp using audio of VOD broadcast content, there is an advantage in providing a VOD service in which the subtitle and audio are synchronized. In particular, when a delay occurs due to the nature of broadcast subtitles input after the stenographer listens to the audio, there is an advantage in providing subtitles that are in sync even when actually broadcast broadcast content is edited and provided for VOD.

Due to the characteristics of the present invention in which subtitle text is extracted and translated for various channels, efficient management of various channels, various translation languages, and a large number of media playback devices is required. According to the present invention, there is an advantage in that real-time translation into various languages and transmission of translated subtitles can be efficiently managed by using a translation agent for each channel and a language gauge for each language.

In addition, according to the present invention, by effectively utilizing thread resources of idle translation agents, there is an advantage in that overload of other translation agents is prevented and limited resources are used in a balanced manner.

1 is a diagram for explaining a conventional VOD providing system and a VOD providing system of the present invention.
2 is a block diagram for explaining the configuration of a caption service providing device according to the present invention.
3 is a diagram for explaining a method of generating a VOD caption file among operating methods of a caption service providing apparatus according to the present invention.
4 is a diagram for explaining a method for adjusting sync of a subtitle file according to the present invention.
5 is a diagram for explaining a method of generating a foreign language subtitle file according to the present invention.
6 is a diagram for explaining a method of providing a real-time translation service according to the present invention.
7 is a diagram for explaining the operation of the caption transmission module 130 according to the present invention.
8A is a diagram for explaining a language gauge.
8B is a diagram for explaining a method of utilizing resources of an idle translation agent.

Hereinafter, the embodiments disclosed in this specification will be described in detail with reference to the accompanying drawings, but the same or similar elements are given the same reference numerals regardless of reference numerals, and redundant description thereof will be omitted. The suffixes "module" and "unit" for components used in the following description are given or used together in consideration of ease of writing the specification, and do not have meanings or roles that are distinct from each other by themselves. In addition, in describing the embodiments disclosed in this specification, if it is determined that a detailed description of a related known technology may obscure the gist of the embodiment disclosed in this specification, the detailed description thereof will be omitted. In addition, the accompanying drawings are only for easy understanding of the embodiments disclosed in this specification, the technical idea disclosed in this specification is not limited by the accompanying drawings, and all changes included in the spirit and technical scope of the present invention , it should be understood to include equivalents or substitutes.

Terms including ordinal numbers, such as first and second, may be used to describe various components, but the components are not limited by the terms. These terms are only used for the purpose of distinguishing one component from another.

It is understood that when an element is referred to as being "connected" or "connected" to another element, it may be directly connected or connected to the other element, but other elements may exist in the middle. It should be. On the other hand, when an element is referred to as “directly connected” or “directly connected” to another element, it should be understood that no other element exists in the middle.

Singular expressions include plural expressions unless the context clearly dictates otherwise. In this application, terms such as "comprise" or "have" are intended to designate that there is a feature, number, step, operation, component, part, or combination thereof described in the specification, but one or more other features It should be understood that the presence or addition of numbers, steps, operations, components, parts, or combinations thereof is not precluded.

In implementing the present invention, components may be subdivided for convenience of description, but these components may be implemented in one device or module, or one component may be divided into multiple devices or modules may be implemented in

1 is a diagram for explaining a conventional VOD providing system and a VOD providing system of the present invention.

A broadcaster may produce, edit, or change broadcast content before transmitting it. In this case, real-time broadcasting of broadcast content can be performed in a manner in which the broadcaster transmits a broadcast stream, and the IPTV service provider receives the broadcast stream and transmits the broadcast stream to IPTV service subscribers.

In addition, the broadcast stream may include caption data for the hearing impaired, and in this case, broadcast content including broadcast captions may be broadcast in real time. Such broadcast subtitles are produced by stenographers, and broadcasting companies are obliged to produce and transmit broadcast subtitles for the hearing impaired according to the Broadcasting Act.

IPTV service providers may include VOD service providers. Referring to FIG. 1A , a broadcasting company provides a VOD service provider with finished broadcasting content. That is, when broadcast content is received, the VOD service provider can provide the received broadcast content to service subscribers as video on demand (VOD). Also, the storage of broadcast content is typically performed on the next day after the broadcast content is broadcast.

Meanwhile, as a method for a VOD service provider to provide VOD subtitles, the above-described method of acquiring broadcast subtitles from a broadcasting company may be considered. However, unlike the broadcaster's duty to provide broadcast subtitles for the hearing impaired in real-time broadcasting, it is not the broadcaster's duty to provide VOD subtitles. Therefore, VOD service providers must provide VOD subtitles only when broadcasters provide broadcast subtitles. A problem arises that can provide

Also, as a method for a VOD service provider to provide VOD subtitles, a method of extracting audio of received broadcast content as text may be considered. However, this requires additional time and cost, and in the case of text extraction by a STT (Speech To Text) device, there is a problem in that additional processing is required because the accuracy is significantly lowered.

Therefore, several days are required for VOD subtitles to be completed, and VOD subtitles cannot be provided immediately even though broadcast contents have already been received (for example, the next day after broadcast contents were broadcast).

Meanwhile, FIG. 1B shows a VOD providing system according to the present invention.

In the present invention, a caption service providing apparatus extracts caption text from caption data in a broadcasting stream in advance, and when broadcast content is received thereafter, simply processes the extracted caption text to provide VOD captions. Accordingly, in the present invention, it is possible to provide VOD subtitles and translated subtitles generated using VOD subtitles within 30 minutes after VOD is received.

2 is a block diagram for explaining the configuration of a caption service providing device according to the present invention.

The caption service providing apparatus 100 may include a database 110, a caption extraction module 120, a caption transmission module 130, a caption translation module 140, and a caption generation module 150.

The database 110 may store subtitle text or other data necessary for the operation of the present invention.

The caption extracting module 120 may acquire caption text by joining a broadcasting company, receiving a broadcast stream, and extracting caption data from the broadcast stream. In this case, the caption extraction module 120 may store caption text in the database 110 .

The caption transmission module 130 may receive a foreign language caption request from a media playback device. Also, the caption transmission module 130 may receive a caption stream from the caption extraction module 120, translate the received caption stream, generate a foreign language caption stream, and transmit the stream to a media player. Here, the meaning that the caption transmission module 130 translates the caption stream means that the caption transmission module 130 directly translates the caption stream, as well as other modules or devices (eg, the caption translation module 140 or a translator). etc.) to transmit the subtitle stream to translate the subtitle stream.

The caption translation module 140 may translate caption streams or caption text stored in the database 110 . Here, the translation by the subtitle translation module 140 may mean that the subtitle translation module 140 not only translates directly, but also transmits a subtitle stream or subtitle text to another device (translator, etc.). there is.

The caption generation module 150 may generate a caption file using caption text and distribute the caption file so that the caption file is inserted into VOD broadcasting content. Also, the caption generation module 150 may adjust synchronization between VOD broadcast content and a caption file by adjusting a time stamp of the caption file.

Meanwhile, the caption service providing apparatus 100 may include a communication unit, a control unit, and a memory.

Here, the communication unit may provide an interface for connecting the caption service providing device 100 with other devices through a wired/wireless network. In this case, the caption service providing device 100 may transmit and receive data with other devices through a communication unit.

Meanwhile, the control unit may control overall operations of the caption service providing apparatus 100 . In addition, the above-described caption extraction module 120, caption transmission module 130, caption translation module 150, and caption generation module 150 may be detailed components of the control unit.

Meanwhile, the memory may store various data for the operation of the caption service providing apparatus 100, and the database 110 described above may be implemented by the memory.

3 is a diagram for explaining a method of generating a VOD caption file among operating methods of a caption service providing apparatus according to the present invention.

The caption extracting module 120 may obtain caption text from caption data within a broadcast stream provided from a broadcasting company (S305).

In detail, the caption extraction module 120 may receive a broadcast stream by joining IGMP to a router. Here, the broadcasting stream is for real-time broadcasting of broadcasting content, such as a broadcasting program, and may include caption data for the hearing impaired. Here, the caption data may be broadcast captions generated by a stenographer based on audio in broadcast contents inserted into a broadcast stream.

The caption extracting module 120 may obtain caption text by extracting caption data from the broadcast stream and decoding the extracted caption data. Also, the caption extraction module 120 may store caption text in the database 110 (S310).

Meanwhile, the caption extraction module 120 may store a time stamp together with caption text. Here, the time stamp is to indicate the point of time of elements constituting the caption text (sentence, word, morpheme, etc.), and may be matched with the caption text and stored.

Meanwhile, the caption extraction module 120 may include a plurality of caption extraction agents. Here, the caption extraction agent may join a corresponding channel, receive a broadcast stream, and acquire caption text from the broadcast stream.

For example, the first caption extraction agent may join the first channel, receive a broadcast stream of the first channel, and acquire caption text of the first channel from the broadcast stream of the first channel. Also, the second caption extraction agent may join the second channel, receive a broadcast stream of the second channel, and acquire caption text of the second channel from the broadcast stream of the second channel.

Also, the caption extraction module 120 may separate and store the acquired caption text for each broadcasting content. For example, the first caption extraction agent stores the caption text of the first broadcast content (eg, drama A) broadcast on the first channel in the database 110, and the second broadcast content (eg, drama A) broadcast on the first channel. For example, subtitle text of entertainment program B) may be stored in the database 110 .

Meanwhile, an IPTV service provider refers to an IPTV service providing entity or a system operated by an IPTV service providing entity, and includes a caption service providing device 100, a VOD service provider 200, and a caption download server 300. can do.

For the VOD service, the VOD service provider 200 may receive broadcast content broadcast in real time through a broadcast stream from a broadcasting company. For example, after the drama “Mr. Moonshine Episode 1” is broadcast on February 15, 2021, the VOD service provider 200 may receive broadcast content titled “Mr. Moonshine Episode 1” from the broadcasting company.

Hereinafter, broadcasting contents provided by a broadcasting company to the VOD service provider 200 for VOD service are referred to as VOD broadcasting contents.

Meanwhile, the caption generation module 150 may generate a caption file of broadcast content broadcasted in real time by a broadcast stream using caption text.

Specifically, when VOD broadcast content is received, the VOD service provider 200 may transmit a caption generation request for the received VOD broadcast content to the caption service providing device 100 (S315).

Further, when a request for generating a caption file for the corresponding broadcast content is received from the VOD service provider 200 receiving broadcast content for VOD, the caption generation module 150 uses the caption text stored in the database 110 to create a caption file. can create

In detail, the caption generation module 150 may read caption text of the broadcast content for which caption file generation is requested from the database 110 (S320), and convert the read caption text to generate a caption file (S325). Here, the caption file may have an extension such as smi, srt, or vtt, and may also include a pre-stored time stamp matched with caption text.

Meanwhile, a caption file generated by the caption generation module 150 may not be in sync with audio of broadcast content for VOD.

The reason why the caption file generated by the caption generation module 150 is out of sync with the audio in broadcast content for VOD is as follows.

First, it is difficult to match the start time of VOD broadcasting content and the start time of a subtitle file. The time stamp of the caption file generated by the caption generation module 150 is generated based on the broadcast caption created by the stenographer. Due to the nature of the broadcast caption input after the stenographer listens to the audio, a delay occurs.

In addition, there are many cases in which detailed differences occur between broadcasting content broadcast in real time by a broadcasting company and broadcasting content for VOD, even though they are the same broadcasting program. For example, in the broadcast content provided to the VOD service provider 200, since interstitial advertisements may be deleted or some scenes may be edited, even though the broadcast content is for the same broadcast program, the broadcast content broadcast in real time by the broadcaster A difference occurs between the broadcast contents for VOD, and therefore, the caption file generated by the caption generation module 150 and the audio in the broadcast contents for VOD are out of sync.

Therefore, the caption generation module 150 may adjust the sync of the caption file (S330).

This will be described in detail with reference to FIG. 4 .

4 is a diagram for explaining a method for adjusting sync of a subtitle file according to the present invention.

The caption generating module 150 may include a caption sync module 410 .

The caption sync module 410 may extract audio from broadcast content for VOD and extract a feature vector (eg, MFCC Feature) of the audio (410).

In this case, the caption sync module 410 may convert speech within the audio into text in units of phonemes using an acoustic model (420). Here, the acoustic model is an acoustic model based on a hidden Markov model and a deep neural network (DNN), and may be a model learned to classify the extracted feature vectors into corresponding phonemes. .

In addition, the subtitle sync module 410 uses a language model that probabilistically analyzes which words appear before and after a specific word using an N-gram probabilistic language model and word lexicon, which is a preprocessing collection of word dictionary data Lexicon) and performing decoding based on Weighted Finite State Transducer (WFST) to obtain audio text (430, 440, 450). That is, audio text may refer to extracting voice from audio in broadcast content for VOD and converting the extracted voice into text. In this case, the caption sync module 410 may store a time stamp together with audio text. Here, the time stamp is to indicate the point of time of an element constituting the audio text (sentence, word, morpheme, etc.), and may be matched with the audio text and stored.

Meanwhile, the caption sync module 410 may adjust the time stamp of the caption file by matching keywords in the audio text with keywords in the caption file (460).

In detail, the caption sync module 410 may extract identical keywords from the audio text and caption files based on whether the keywords are the same and the location of the keywords in the caption.

In addition, the caption sync module 410 may adjust the time stamp of the caption file so that the position of the keyword in the caption file is converted to the position of the same keyword in the audio text (470). Accordingly, a final caption file with an adjusted time stamp may be generated.

Meanwhile, a subtitle file of broadcast content broadcast in real time by a broadcast stream must be inserted into VOD broadcast content provided by a broadcaster to a VOD service provider. Here, the inserted caption file may include a final caption file whose time stamp is adjusted according to the processing described in FIG. 4 .

Also, the caption generating module 150 may distribute the caption file so that the caption file of the broadcast content broadcast in real time is inserted into the broadcast content for VOD provided by the broadcaster to the VOD service provider 200 .

Here, distributing a caption file may mean providing a caption file so that the VOD service provider 200 can provide a VOD service by inserting the caption file into VOD broadcast content.

For example, the caption generation module 150 may directly transmit a caption file to the VOD service provider 200 or upload the caption file to the caption download server 300 so that the VOD service provider 200 can use it. In this case, the VOD service provider 200 may provide a VOD service by inserting subtitles of a subtitle file into VOD broadcast content.

For another example, when the caption service providing device 100 directly provides a VOD service, a caption file may be stored in a database in the caption service providing device 100 .

Returning to FIG. 3 again, the caption service providing apparatus 100 may generate a foreign language caption file by translating the generated caption file (Korean caption file).

Specifically, when a caption file is generated, the caption generation module 150 may transmit a foreign language translation request for the generated caption file to the caption translation module 140 (S335). In this case, the subtitle translation module 140 may generate one or more foreign language subtitle files by translating the Korean subtitle file generated using the subtitle text (S340).

A method of generating a foreign language subtitle file will be described in detail with reference to FIG. 5 .

5 is a diagram for explaining a method of generating a foreign language subtitle file according to the present invention.

The caption translation module 140 may separate sentences in the Korean caption file using the caption corpus (S501). In this case, the subtitle translation module 140 separates sentences by using the corpus caching stored in the memory caching 1010, or if there is no corresponding corpus caching in the memory caching 1010, the caption translation module 140 uses the corpus caching stored in the data warehouse 1020. Sentences in the Korean subtitle file can be separated using the corpus data set.

On the other hand, a sentence is composed of a set of morphemes, and morphemes can be composed of words, stems, and endings.

In this case, the caption translation module 140 may provide morphemes constituting sentences, words constituting morphemes, stems, and endings to the N-gram probabilistic language model. In this case, the N-gram probabilistic language model analyzes morphemes, words, stems, and endings to obtain a sentence ending point, and can separate sentences in the Korean subtitle file to be suitable for translation.

Meanwhile, previously translated text may be stored in the memory caching 1010 by forming a subtitle data set. Here, the previously translated text may include sentences, morphemes, words, and the like, and Korean text and translated text corresponding to the Korean text may be matched and stored.

Meanwhile, the translated text stored in the memory caching 1010 may be updated based on the number of accesses of the subtitle translation module 140 .

Specifically, when the data in the memory caching 1010 is full, the memory caching 1010 may secure storage space by deleting the least accessed translation text by the caption translation module 140 . That is, translation texts most frequently accessed by the caption translation module 140 may be stored in the memory caching 1010 .

Also, the caption translation module 140 may check whether the text in the Korean caption file exists in the memory caching 1010 (S502). Also, when some text in the Korean subtitle file exists in the memory caching 1010, the caption translation module 140 may extract translated text corresponding to the partial text from the memory caching.

Meanwhile, when a dialect exists in a sentence, the caption translation module 140 may convert the dialect into a standard language (S503). In this case, the subtitle translation module 140 converts the dialect into standard language using the standard language caching stored in the memory caching 1010, or if the corresponding standard language caching does not exist in the memory caching 1010, the data warehouse 1020 The dialect in a sentence can be converted into standard language using the standard language-dialect data set stored in .

Meanwhile, when the preprocessed sentence 510 is generated through S501, S502, and S503, the caption translation module 140 may provide the preprocessed sentence 510 to the translator 1030.

In this case, the translator 1030 may translate the preprocessed sentences 510 into one or more foreign languages, and the subtitle translation module 1040 may generate one or more foreign language subtitle files based on the translation result.

Meanwhile, the generation and translation of the preprocessed sentence described above may be performed based on the AI learning model 1040.

Returning to FIG. 3 again, the caption translation module 1040 may transmit one or more foreign language caption files to the caption generation module 150 (S345).

In this case, the subtitle generation module 150 may distribute one or more foreign language subtitle files (S350).

As described above, according to the present invention, subtitle text is secured from a broadcast stream in advance, and when VOD broadcast content is received, the previously secured subtitle text can be simply processed and provided as a subtitle file. It has the advantage of providing VOD service with immediate subtitles (including foreign language subtitles).

In addition, according to the present invention, when broadcast content for VOD is received, processing such as generating a subtitle file and synchronizing subtitles is performed, thereby having an advantage of saving resources. That is, subtitle text extraction is continuously performed for 24 hours for various channels, whereas subtitle file generation and subtitle sink processing are performed only for broadcast content that is a VOD service target, thereby reducing unnecessary costs.

In addition, according to the present invention, by first extracting subtitle text and then adjusting the time stamp using the audio of VOD broadcast content, there is an advantage in providing a VOD service in which the subtitle and audio are synchronized. In particular, when a delay occurs due to the nature of broadcast subtitles input after the stenographer listens to the audio, there is an advantage in providing subtitles that are in sync even when actually broadcast broadcast content is edited and provided for VOD.

In addition, due to the characteristics of the present invention in which subtitle text is extracted and translated for various channels, a great deal of translation is performed. Furthermore, according to the present invention, memory caching is updated based on the number of accesses of the subtitle translation module, and redundant translation is prevented using the memory caching, thereby enabling fast translation.

6 is a diagram for explaining a method of providing a real-time translation service according to the present invention.

The caption extracting module 120 may obtain caption text from caption data within a broadcast stream provided from a broadcasting company (S605). Meanwhile, like a broadcast stream, caption data is also transmitted in the form of a stream, and hence, the caption data will be referred to as a caption stream.

In this case, the caption extraction module 120 may transmit the caption stream to the caption transmission module 130 (S610). In this case, the caption transmission module 130 may receive the caption stream of the channel being viewed in the media playback device 400 from the caption extraction module 120 and temporarily store the received caption stream in a task queue (S615).

Meanwhile, the media playback device 400 is a device that decodes a broadcast stream and outputs broadcast content in real time, and may include a TV or a set-top box.

Meanwhile, when an input requesting a foreign language subtitle is received from the user, the media playback device 400 may transmit a foreign language subtitle request including a translation language designated by the user to the subtitle transmission module 130 (S620).

Meanwhile, when a foreign language subtitle request is received from the media playback device 400, the subtitle transmission module 130 may check whether the translation agent is running (S625). Here, checking whether a translation agent is running may mean checking whether a translation agent corresponding to a corresponding channel can provide a translation service.

For example, a real-time foreign language subtitle request is received from the media playback device 400, but a subtitle stream may not exist because an advertisement is being broadcast on a channel viewed in the media playback device 400. In this case, the caption transmission module 130 may transmit a translation impossible message to the media playback device 400 or provide a translation service when a broadcast program is broadcasted and a caption stream is received later.

Meanwhile, when a real-time foreign language caption request is received from the media playback device 400, the caption transmission module 130 may translate the subtitle stream and transmit it to the media playback device 400 (S630, S635, S640).

Here, the meaning that the caption transmission module 130 translates the caption stream means that the caption transmission module 130 directly translates the caption stream, as well as other modules or devices (eg, the caption translation module 140 or a translator). 1030, etc.) to transmit the subtitle stream to translate the subtitle stream.

The operation of the caption transmission module 130 will be described in detail with reference to FIG. 7 .

7 is a diagram for explaining the operation of the caption transmission module 130 according to the present invention.

The caption transmission module 130 may include a plurality of translation agents 715 and 725. Here, the plurality of translation agents 715 and 725 may respectively correspond to a plurality of channels. For example, the first translation agent 715 is an agent that manages translation for a subtitle stream of a first channel, and the second translation agent 725 is an agent that manages translation for a subtitle stream of a second channel. can

Although two translation agents are shown in FIG. 7, it is not limited thereto, and translation agents may exist for each channel.

Meanwhile, each of the plurality of translation agents 715 and 725 may include a plurality of language gauges for storing session information of media playback devices that have applied for a corresponding language.

This will be described with reference to FIG. 8A together.

8A is a diagram for explaining a language gauge.

Taking the example of the first translation agent 715 among the plurality of translation agents, the first translation agent 715 may include a plurality of language gauges respectively corresponding to a plurality of languages.

For example, the first language gauge 810 is a language gauge corresponding to Chinese, the second language gauge 820 is a language gauge corresponding to English, and the third language gauge 830 is a language gauge corresponding to Korean. can

Meanwhile, a plurality of media playback devices connected to the caption service providing apparatus 100 may transmit a request for a foreign language caption including a translation language specified by a user.

In this case, requests for foreign language subtitles received from the set-top boxes watching the first channel may be transmitted to the first translation agent 715 .

In this case, the first translation agent 715 may obtain a translation language from the foreign language subtitle request and store session information (IP information of the media playback device) of the media playback device in a language gauge corresponding to the translation language.

For example, when the first media playback device watching the first channel transmits a request for a foreign language subtitle specifying Chinese, the first translation agent 715 converts session information of the first media playback device into a first language corresponding to Chinese. It can be stored in the gauge 810. Next, when the second media playback device watching the first channel transmits a request for a foreign language subtitle specifying English, the first translation agent 715 converts the session information of the second media playback device to a second language gauge corresponding to Chinese. (820). Next, when the third media playback device watching the first channel transmits a request for a foreign language subtitle specifying Chinese, the first translation agent 715 converts the session information of the third media playback device to a first language gauge corresponding to Chinese. (810).

Meanwhile, the first translation agent 715 may start translation into a corresponding language when session information exists in the language gauge.

For example, the first translation agent 715 does not start Chinese translation when session information does not exist in the first language gauge 810, and if at least one session information exists in the first language gauge 810 You can start translating Chinese.

The first translation agent 715 receives the subtitle stream obtained from the first channel by the subtitle extraction module 120 (ie, the subtitle stream extracted by the first subtitle extraction agent 710 joined to the first channel) and converts the subtitle stream into a corresponding subtitle stream. can be translated into languages. The specific translation method has been described with reference to FIG. 5 and will be omitted below.

Hereinafter, a situation in which session information exists in the first language gauge 810 and session information exists in the second language gauge 820 will be described.

The first translation agent 715 translates the subtitle stream obtained from the first channel by the subtitle extracting module 120 into a first language and a second language, respectively, so that the first channel-the first language subtitle stream and the first channel- A second language subtitle stream may be generated.

The first translation agent 715 may transmit the first channel-first language subtitle stream to media playback devices corresponding to the session information stored in the language gauge of the first language in a multicast manner (S645). .

For example, when session information of the 1st to 10th media playback devices is stored in the language gauge corresponding to Chinese, the first translation agent 715 uses the IP information of the 1st to 10th media playback devices. The first channel-Chinese subtitle stream may be transmitted to the first to tenth media playback devices in a multicast manner.

In addition, the first translation agent 715 may transmit the first channel-second language subtitle stream to media playback devices corresponding to the session information stored in the language gauge of the second language in a multicast manner (S645).

For example, when session information of the 11th to 25th media playback devices is stored in the language gauge corresponding to English, the first translation agent 715 uses the IP information of the 11th to 25th media playback devices to translate The first channel-English subtitle stream may be transmitted to the 11th to 25th media playback devices in a multicast manner.

In this case, the first translation agent 715 multicasts the first channel-Chinese subtitle stream to the first to tenth media playback devices through the first multicast channel, and transmits the subtitle stream in Chinese through the second multicast channel. The first channel-English subtitle stream may be transmitted to the 11th to 25th media playback devices in a multicast manner.

Meanwhile, session information of the 26th media player may be added to a language gauge corresponding to English while the first channel-English subtitle stream is transmitted in a multicast manner. In this case, the first translation agent 715 may transmit the first channel-English subtitle stream to the 11th to 26th media playback devices in a multicast manner using the IP information of the 11th to 26th media playback devices. .

In addition, while the first channel-Chinese subtitle stream is transmitted in a multicast method and the first channel-English subtitle stream is transmitted in a multicast method, session information of the 27th media playback device may be stored in a language gauge corresponding to French. there is. In this case, the first translation agent 715 starts translating the subtitle stream acquired from the first channel into a third language (French) by the subtitle extracting module 120, so that the first channel-third language (French) is translated. ) subtitle streams can be additionally created. In this case, the first translation agent 715 multicasts the first channel-Chinese subtitle stream to the first to tenth media playback devices through the first multicast channel, and transmits the subtitle stream in Chinese through the second multicast channel. The 1st channel-English subtitle stream is multicasted to the 11th to 26th media playback devices, and the 1st channel-French subtitle stream is multicasted to the 27th media playback device through a third multicast channel. can be sent

Next, a method of utilizing idle translation agent resources will be described with reference to FIG. 8B.

8B is a diagram for explaining a method of utilizing resources of an idle translation agent.

Each of the plurality of translation agents is configured independently and resources are allocated. Also, the idle translation agent may be a translation agent that is not currently translating (that is, not directly translating and not requesting translation from another module or device).

That is, the idle translation agent may refer to a translation agent that is not providing a translation service for a channel corresponding to the translation agent. For example, when the first translation agent is not providing a translation service for the first channel because the first channel is advertising, the first translation agent may be an idle translation agent. For another example, when media playback devices watching the first channel do not transmit a foreign language subtitle request, the first translation agent may be an idle translation agent.

Meanwhile, when the first translation agent is idle, the first translation agent 715 may translate subtitle streams of other channels.

Specifically, when the first translation agent 715 is idle, the first translation agent 715 may generate a second channel-first language subtitle stream by translating a subtitle stream obtained from the second channel.

That is, the second translation agent 725 performs translation on the subtitle stream originally obtained from the second channel. However, when the first translation agent 715 is idle, the first translation agent 715 can utilize idle resources by performing translation on the subtitle stream obtained from the second channel.

In this case, the first translation agent 715 may receive the second channel caption stream from the second channel caption extraction agent 720 . The first translation agent 715 may translate the subtitle stream of the second channel into the first language requested by the second translation agent 725 to generate a second channel-first language subtitle stream.

Meanwhile, the language gauge 840 corresponding to the first language in the second translation agent 725 may be shared with the first translation agent 715 .

The first translation agent 715 multicasts the second channel-first language subtitle stream to media playback devices corresponding to the session information stored in the language gauge of the first language in the second translation agent 725. can be sent to

For example, if session information of the 30th to 38th media playback devices is stored in the language gauge corresponding to Chinese of the second translation agent 725, the first translation agent 715 plays the 30th to 38th media. The second channel-Chinese subtitle stream may be multicast to the 30th to 38th media playback devices using the IP information of the devices.

Due to the characteristics of the present invention in which subtitle text is extracted and translated for various channels, efficient management of various channels, various translation languages, and a large number of media playback devices is required. According to the present invention, there is an advantage in that real-time translation into various languages and transmission of translated subtitles can be efficiently managed by using a translation agent for each channel and a language gauge for each language.

In addition, according to the present invention, by effectively utilizing thread resources of idle translation agents, there is an advantage in that overload of other translation agents is prevented and limited resources are used in a balanced manner.

The above-described present invention can be implemented as computer readable code on a medium on which a program is recorded. The computer-readable medium includes all types of recording devices in which data that can be read by a computer system is stored. Examples of computer-readable media include Hard Disk Drive (HDD), Solid State Disk (SSD), Silicon Disk Drive (SDD), ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, etc. there is Also, the computer may include a control unit. Accordingly, the above detailed description should not be construed as limiting in all respects and should be considered as illustrative. The scope of the present invention should be determined by reasonable interpretation of the appended claims, and all changes within the equivalent scope of the present invention are included in the scope of the present invention.

100: caption service providing device 110: database
120: caption extraction module 130: caption transmission module
140: subtitle translation module 150: subtitle generation module

Claims (10)

a caption extracting module that obtains caption text from caption data within a broadcast stream provided from a broadcasting company;
A caption file of broadcast content broadcast in real time by the broadcast stream is generated using the caption text, and the caption file is distributed so that the caption of the caption file is inserted into the broadcast content for VOD provided by the broadcaster to a VOD service provider. a subtitle generation module that does; and
Includes a subtitle transmission module;
The caption transmission module includes a first translation agent corresponding to a first channel among a plurality of channels;
The first translation agent,
translating the subtitle stream obtained from the first channel to generate a first channel-first language subtitle stream and a first channel-second language subtitle stream;
Transmitting the first channel-first language subtitle stream in a multicast method to media playback devices corresponding to session information stored in a language gauge of the first language;
Transmitting the first channel-second language subtitle stream in a multicast method to media playback devices corresponding to session information stored in a language gauge of a second language.
Caption service provider. According to claim 1,
The subtitle generation module,
Generating the caption file using the caption text when a request for generating a caption file for the broadcast content is received from the VOD service provider receiving the broadcast content for VOD
Caption service provider. According to claim 1,
The subtitle generation module,
Obtaining audio text from audio in the VOD broadcast content, and matching a keyword in the audio text with a keyword in the subtitle text to adjust a time stamp of the subtitle file
Caption service provider. According to claim 1,
A caption translation module for generating at least one foreign language caption file by translating the Korean caption file generated using the caption text;
The subtitle generating module distributes the one or more foreign language subtitle files.
Caption service provider. According to claim 1,
The subtitle transmission module,
When a request for real-time foreign language subtitles is received from the media playback device, the media playback device translates the subtitle stream of the channel being watched and transmits it to the media playback device.
Caption service provider. According to claim 5,
The subtitle transmission module,
A plurality of translation agents each including a plurality of language gauges corresponding to the plurality of channels and storing session information of media playback devices that have applied for a corresponding language.
Caption service provider. delete According to claim 6,
The plurality of translation agents further include a second translation agent corresponding to a second channel;
The first translation agent,
When the first translation agent is idle, translate the subtitle stream obtained from the second channel to generate a second channel-first language subtitle stream;
Transmitting the second channel-first language subtitle stream in a multicast method to media playback devices corresponding to session information stored in a language gauge of a first language in the second translation agent.
Caption service provider. According to claim 4,
The subtitle translation module,
Generating the foreign language subtitle file by extracting translated text corresponding to some text in the Korean subtitle file from memory caching;
The translated text is
Updated based on the number of accesses of the subtitle translation module
Caption service provider. obtaining caption text from caption data within a broadcast stream provided from a broadcasting company;
generating a caption file of broadcast content broadcast in real time by the broadcast stream using the caption text;
distributing the caption file so that the caption of the caption file is inserted into VOD broadcast content provided by the broadcaster to a VOD service provider;
A first translation agent corresponding to a first channel among a plurality of channels translates a subtitle stream obtained from the first channel to generate a first channel-first language subtitle stream and a first channel-second language subtitle stream. step;
transmitting, by the first translation agent, the first channel-first language subtitle stream to media playback devices corresponding to session information stored in a language gauge of the first language in a multicast manner; and
Transmitting, by the first translation agent, the first channel-second language subtitle stream in a multicast manner to media playback devices corresponding to session information stored in a language gauge of a second language;
How to provide closed captioning services.

Images