JP6461638B2 - Receiving machine - Google Patents

Description

  The present invention relates to a receiver that handles subtitle character strings that are displayed superimposed on video content.

In recent years, captioning services (closed captions) have been provided in digital television broadcasting, and by providing a character string superimposed on conventional video content, a hearing-impaired person can also enjoy television broadcasting. Non-Patent Document 1 discloses a technique for transmitting subtitle information in a PES (Packetized Elementary Stream). Also, subtitle information is converted to MMT (MPEG Media Transport) MFU (Media).
A method of accommodating and transmitting in a Fragment Unit) or MPU (Media Processing Unit) has been studied. The subtitle character string included in the subtitle information is displayed in synchronism with the program or time according to the time control mode defined by the subtitle information, or is displayed immediately after reception without being synchronized. The subtitle information described in Non-Patent Document 1 stores a subtitle character string that is displayed superimposed on the portion of the video content that is displayed at the timing at which the portion transmitted simultaneously with the subtitle information is displayed. Therefore, the receiver can display the subtitle character string corresponding to the video content by sequentially displaying the received subtitle information.

“Data Broadcast Coding and Transmission Standards for Digital Broadcasting”, ARIB STD-B24, 5.4 edition, Japan Radio Industry Association, December 16, 2009

In recent years, a receiver that executes an application written in conformity with HTML5, such as Hybridcast (registered trademark), has been proposed. Subtitle information is required to be used by an application executed by such a receiver. In order to display a subtitle character string using an application described in conformity with HTML5, it is considered to use information based on XML (Extensible Markup Language) as subtitle information. However, since the information based on XML is a text file, the amount of data is large, and there is a concern that the band of the broadcast wave may be compressed. On the other hand, when a binary file obtained by compressing a text file is used to reduce the amount of data, it takes time to decompress the file, and there is a concern that notification of information that requires immediacy may be delayed.
The objective of this invention is providing the receiver which solves the subject mentioned above.

The first aspect includes a stream reception unit that receives a video stream from the video stream received; and subtitles decoding unit for decoding the caption information including subtitle text to be displayed superimposed on the video content, the video said received from the stream, and a table decoder for decoding the caption information table including compression information indicating whether at least the subtitle information is compressed, among the caption information the subtitle decoder is decoded is compressed by the subtitle information table A subtitle decompression unit that decompresses what is shown to be, and the subtitle decompression unit includes information specifying the XML schema of the subtitle information in the subtitle information table decoded by the table decoding unit And a receiver that expands the caption information based on an XML schema specified by the information .

  According to at least one of the above aspects, the receiver determines whether or not caption information is compressed based on the caption information table, and determines whether or not to expand the caption information. As a result, the broadcaster compresses subtitle information for information that does not require immediacy and transmits it without compressing subtitle information for information that requires immediacy. It is possible to save the broadcast wave band while enabling prompt notification.

It is a schematic block diagram which shows the structure of the broadcast system 1 by 1st Embodiment. It is a figure which shows an example of caption information. It is a flowchart which shows the display method of the subtitle character string which concerns on 1st Embodiment. It is a schematic block diagram which shows the structure of the broadcast system 1 by 2nd Embodiment. It is a figure which shows the example of merge subtitle information. It is a figure which shows the example of the HTML5 application which acquires video content and subtitle information. It is a figure which shows the production | generation method of merged subtitle information.

<< First Embodiment >>
Hereinafter, the first embodiment will be described in detail with reference to the drawings.
FIG. 1 is a schematic block diagram showing a configuration of a broadcasting system 1 according to the first embodiment.
The broadcast system 1 is a system that includes a broadcast transmission device 100 and a receiver 300. The broadcast sending device 100 is owned by the broadcaster, and the receiver 300 is owned by the user.

The broadcast transmission apparatus 100 includes a video packet generation unit 101, a caption packet generation unit 102, and a broadcast unit 103.
The video packet generator 101 stores video content in MPU (Media Processing Unit) data.
The caption packet generation unit 102 stores the caption information in the MPU data. The caption information is information described in a TTML (Timed Text Markup Language) format or data obtained by compressing TTML with EXI. The caption information includes a caption character string displayed on the screen at one time. That is, subtitle information changes sequentially according to the content of video content that changes over time.

FIG. 2 is a diagram illustrating an example of caption information.
As shown in FIG. 2, the caption information is data in TTML format that stores a caption character string displayed on the screen at one time. In the example illustrated in FIG. 2, a caption character string displayed at a time from 00: 00: 11: 06 to 00: 00: 12: 14 is stored in the caption information. Thus, by storing the subtitle character string displayed on the screen at one time in the subtitle information, the data size of the subtitle information is reduced, and the broadcast transmission apparatus 100 reliably transmits the subtitle information in a short time. be able to. Further, different IDs are assigned to the caption information for each caption character string.

Also, the caption packet generation unit 102 stores a caption information table that stores metadata regarding caption information in the MPU data. The subtitle information table stores subtitle modes and schema URLs for all subtitle information being transmitted with the asset, and subtitle IDs, language codes, display modes, and compression modes for subtitles in multiple languages being transmitted. It is a table. In the subtitle mode, the subtitle information is basic subtitle information that transmits the subtitle character string to be displayed as a fragment each time, or the subtitle character string that is to be displayed in the program in a batch or subtitle character string. In addition, it is information indicating whether it is extended caption information such as caption information including a decoration method of the caption character string. The schema URL is information indicating a URL for acquiring the XML schema of the subtitle used for decompressing the subtitle information when the subtitle described in XML is compressed. The subtitle ID is an ID for indicating subtitle information in each language of subtitle information in multiple languages transmitted by the asset. For example, by designating a certain subtitle ID, the receiver 300 can display Japanese subtitles. The display mode is information indicating whether the subtitle character string display in the receiver 300 is automatically displayed, automatically hidden, selected, or switched between display and non-display according to a specific condition.
The compression mode is information indicating whether or not the caption information is compressed. Specifically, the value of the compression mode indicates “1” when the caption information is compressed, and the value of the compression mode indicates “0” when the caption information is not compressed. In the present embodiment, the compression mode is an example of compression information.

  The broadcasting unit 103 multiplexes an MPU storing video content, an MPU storing subtitle information, and an MPU storing a subtitle information table on a broadcast wave and sequentially transmits them. Specifically, an asset including MPU data storing video content, an MPU data storing subtitle information, and an asset including MPU data storing a subtitle information table are multiplexed on a broadcast wave and sequentially transmitted. . That is, the video stream is multiplexed on the broadcast wave. Note that, in an asset including subtitle information, a subtitle information table that stores metadata of the subtitle information is transmitted at a certain frequency so that the subtitle information can be appropriately acquired no matter what timing the broadcast is received.

The receiver 300 includes a broadcast receiving unit 301, a transport decoding unit 302, a video / audio decoding unit 303, a caption decoding unit 304, a table decoding unit 305, a caption expansion unit 306, a caption processing unit 307, and a display control unit 308.
The broadcast receiving unit 301 receives broadcast waves sent from the broadcast sending device 100. Thereby, the broadcast receiving unit 301 receives the video stream. In the present embodiment, the broadcast receiving unit 301 is an example of a stream receiving unit.
The transport decoding unit 302 separates the MMTP packet multiplexed on the broadcast wave into MPU data and section data, and outputs the MPU data to the video / audio decoding unit 303, the caption decoding unit 304, and the table decoding unit 305.

The video / audio decoding unit 303 acquires the video content by decoding the MPU data received from the transport decoding unit 302 into the video content.
The caption decoding unit 304 acquires the caption information by decoding the MPU data received from the transport decoding unit 302 into the caption information.
The table decoding unit 305 acquires the caption information table by decoding the MPU data received from the transport decoding unit 302 into the caption information table.

The caption expansion unit 306 expands the caption information decoded by the caption decoding unit 304 when the compression mode of the caption information table decoded by the table decoding unit 305 is “1”. The caption decompression unit 306 decompresses caption information based on the XML schema stored in the schema storage unit 309. When the schema URL is described in the caption information table, the caption decompression unit 306 writes the content of the XML schema obtained by accessing the schema URL into the schema storage unit 309 and then updates the XML. The subtitle information is expanded based on the schema.
The caption processing unit 307 controls the character color, arrangement, and display / non-display timing of the caption character string based on the caption information decoded by the caption decoding unit 304. The caption processing unit 307 is embedded software (resident application) of the receiver 300.

  The display control unit 308 outputs a display signal for displaying the video content decoded by the video / audio decoding unit 303 to a display device (not shown). Also, when there is an instruction to display subtitles by the user's operation, a display signal that superimposes the subtitle information processed by the subtitle processing unit 307 on the video content is displayed on the display device.

  The schema storage unit 309 stores an XML schema that defines the data structure of caption information. The schema storage unit 309 stores the XML schema in a rewritable manner in accordance with changes in the definition of caption information.

Here, a method of displaying a caption character string by the receiver 300 will be described.
FIG. 3 is a flowchart showing a subtitle character string display method according to the first embodiment.
When the broadcast receiving unit 301 receives a broadcast wave, the transport decoding unit 302 acquires MPU data from the broadcast wave. In the asset transmitted by the broadcast wave, since the caption information table storing the metadata of the caption information is transmitted before the caption information, the table decoding unit 305 first decodes the MPU data into the caption information table (step S1). ). Next, the caption decoding unit 304 decodes MPU data into caption information (step S2).

  Next, the caption expansion unit 306 reads the compression mode of the caption information table decoded by the table decoding unit 305 (step S3). Next, the subtitle expansion unit 306 determines whether or not the compression mode associated with the subtitle information decoded in step S2 indicates “1” in the subtitle information table (step S4). When the compression mode of the caption information table indicates “1” (step S4: YES), the caption expansion unit 306 determines whether or not the schema URL is included in the caption information table (step S5). When the caption decompression unit 306 determines that the schema URL is included in the caption information table (step S5: YES), the caption decompression unit 306 accesses the schema URL to acquire a new XML schema, and the schema storage unit uses the XML schema. The XML schema stored in 309 is updated (step S6).

  The subtitle decompression unit 306 reads the XML schema from the schema storage unit 309 when a new XML schema is recorded in the schema storage unit 309 or when the schema URL is not included in the subtitle information table (step S5: NO) ( Step S7). Then, the caption decompression unit 306 decompresses the caption information based on the read XML schema (step S8), and outputs the decompressed caption information to the caption processing unit 307.

  On the other hand, when the compression mode associated with the caption information decoded in step S2 indicates “0” in the caption information table read by the caption decompression unit 306 (step S4: NO), the caption information is stored in the caption decompression unit 306. Without being decompressed, it is output from the caption decoding unit 304 to the caption processing unit 307. The caption processing unit 307 controls the character color, arrangement, and display / non-display timing of the caption character string based on the caption information input from the caption decoding unit 304 or the caption decompression unit 306 (step S9).

  Thus, according to the present embodiment, the receiver 300 is based on the information in the caption information table when the compressed information and the uncompressed information are mixed in the caption information multiplexed on the broadcast wave. Determines whether or not expansion is possible. As a result, the broadcaster compresses subtitle information for information that does not require immediacy and transmits it without compressing subtitle information for information that requires immediacy. It is possible to save the broadcast wave band while enabling prompt notification. Further, even when the XML schema specification of the caption information is changed, the receiver 300 accesses the schema URL by the caption information table, acquires the latest XML schema, updates the caption information, and then expands the caption information. Can do.

<< Second Embodiment >>
Next, a second embodiment will be described.
FIG. 4 is a schematic block diagram showing the configuration of the broadcast system 1 according to the second embodiment.
The broadcasting system 1 according to the second embodiment is different from the first embodiment in the configuration of the receiver 300. In the second embodiment, an example in which a subtitle character string included in subtitle information is presented in an application described in accordance with HTML5 executed by the receiver 300 will be described.

  The receiver 300 further includes a caption merging unit 310 and an application execution unit 311 in addition to the configuration of the first embodiment.

  The caption merging unit 310 generates merged caption information obtained by merging the caption information that has not been presented with the caption character string, among the caption information decoded by the caption decoding unit 304 and the caption information expanded by the caption expansion unit 306. Here, the caption information for which the presentation of the caption character string has not ended is the caption information including the caption character string currently presented by the application execution unit 311 and the caption including the caption character string to be displayed after the current time. It is information.

FIG. 5 is a diagram illustrating an example of merged caption information.
As shown in FIG. 5, the merged caption information is TTML format data storing a plurality of caption character strings. The merged caption information shown in FIG. 5 has a div element obtained by merging p tags indicating the caption character strings included in the plurality of caption information.

  The application execution unit 311 executes an application described in conformity with HTML5. The application executed by the application control unit includes an application for processing caption information.

Here, a method in which the application execution unit 311 acquires video content and caption information will be described.
FIG. 6 is a diagram illustrating an example of an HTML5 application that acquires video content and caption information.
An application executed by the application execution unit 311 is described in conformity with HTML5. In the application, video content is referred to in the video element shown in FIG. 6A or the object element shown in FIG. Then, as shown in FIGS. 6A and 6B, the merged caption information is referred to in the track element included in the element that refers to the video content. The track element is an element defined in HTML5 as an element that explicitly specifies an external synchronized text track as a media element. When subtitle information in a plurality of languages exists, the merged subtitle information in each language is referred to using a plurality of track elements as shown in FIGS. Note that the multilingual subtitle character string reference method is not limited to this. For example, subtitle character strings of a plurality of languages may be stored in one merged subtitle information. In this case, the subtitle character string is divided for each language by the div element.

As described above, in the present embodiment, the application execution unit 311 acquires merged caption information obtained by merging caption information multiplexed on a broadcast wave using a track element that refers to information acquired via a normal network.
Then, as shown in FIG. 6C, an application execution unit is described in which an instruction for acquiring a cue (subtitle character string) of subtitle information referred to by a track element by JavaScript (registered trademark) is described in the application. 311 can process the merged caption information.

  Then, by describing an event handler that activates an update of merged caption information (for example, an oncuechange event) as an event in the application, the application execution unit 311 allows the merged caption information to be updated when the merged caption information is updated. It is possible to execute the process of re-acquiring. Thereby, the application execution unit 311 can present all the subtitle character strings included in the sequentially transmitted subtitle information.

Here, a method of generating merged caption information by the caption merger 310 will be described.
FIG. 7 is a diagram illustrating a method for generating merged caption information.
In the example illustrated in FIG. 7, caption information is received and decoded in the order of A, B, C, D, and E in the receiver 300. Here, it is assumed that the caption information B and the caption information D are compressed.
In the example illustrated in FIG. 7, when the caption information B is decoded by the caption decoding unit 304 and further expanded by the caption expansion unit 306, the application execution unit 311 displays the caption character string included in the caption information A. . Note that which subtitle character string is currently displayed can be understood by referring to the display time associated with the subtitle character string included in the subtitle information and the merged subtitle information. At this time, the subtitle merging unit 310 includes subtitle information A that is subtitle information including the subtitle character string that is currently displayed, and a subtitle character string that includes subtitle character strings that are displayed after the subtitle character string included in the subtitle information A. A certain subtitle information B is merged to generate merged subtitle information. Note that the caption information B to be merged is expanded by the caption expansion unit 306.

  The subtitle merging unit 310 removes elements with overlapping IDs when the header of the subtitle information to be merged includes a plurality of style elements and region elements, or when the elements have different IDs. Merge in header and sort each in ascending ID order. Also, the subtitle merging unit 310 merges each p element into one div element when a plurality of p elements are included in the div element of the subtitle information to be merged or has different IDs. Sort elements in ascending ID order. Also, the subtitle merging unit 310 merges each div element into a body element when the subtitle information to be merged includes a plurality of div elements or has different IDs, and the div elements are in ascending order of ID. Sort. Accordingly, the caption merging unit generates merged caption information.

Then, the caption merging unit 310 updates the merged caption information by overwriting the merged caption information on the merged caption information generated in the past. The caption merging unit 310 notifies the application execution unit 311 of the update of the merged caption information.
When the application execution unit 311 receives an update notification to the application being executed, if an event handler that starts updating the merged caption information as an event is described, the application execution unit 311 executes the event handler and reacquires the merged caption information. To do. At this time, the application execution unit displays the subtitle character string (the subtitle character string included in the subtitle information A) included in the merged subtitle information before the update, but the subtitle character string being displayed by reacquisition of the merged subtitle information There is no problem that cannot be displayed. This is because the subtitle character string included in the subtitle information A is also included in the updated merged subtitle information.

  Similarly, when the subtitle character string displayed when the subtitle information C is decoded is included in the subtitle information B, the subtitle information B and the subtitle information C are merged. Further, when the subtitle character string displayed when the subtitle information D is decoded and further expanded is included in the subtitle information C, the subtitle information C and the subtitle information D are merged. Further, when the subtitle character string displayed when the subtitle information E is decoded is included in the subtitle information C, the subtitle information C, the subtitle information D, and the subtitle information E are merged.

  As described above, the broadcasting system 1 according to the second embodiment generates merged caption information by merging the caption information for which the presentation of the caption character string has not ended, and refers to the merged caption information in the track element of HTML5. Let Thereby, the broadcasting system 1 according to the second embodiment can present a caption character string included in caption information in an application described in conformity with HTML5. At this time, the merged subtitle information is merged with the subtitle information decoded by the subtitle decoding unit 304 for subtitle information that is not compressed at the time of reception, and the subtitle expansion unit 306 is expanded for subtitle information that is compressed at the time of reception. Merge subtitle information. Accordingly, the merged subtitle information can be processed in the application execution unit 311 regardless of whether or not the subtitle information is compressed.

As described above, the embodiment has been described in detail with reference to the drawings. However, the specific configuration is not limited to that described above, and various design changes and the like can be made.
For example, in the above-described embodiment, the video stream including the caption information table and the caption information is multiplexed on the broadcast wave, and the broadcast receiving unit 301 receives the broadcast wave. However, the present invention is not limited to this. For example, the receiver 300 may receive a video stream including a caption information table and caption information via a network such as the Internet. That is, the caption information and the caption information table may be distributed via the Internet.

  In the second embodiment, the case where the caption merging unit 310 generates merged caption information by merging a plurality of caption information is described, but the present invention is not limited to this. For example, the caption merging unit 310 may generate the merged caption information by merging the previously generated merged caption information with the caption information newly decoded by the caption decoding unit 304. In this case, the generated subtitle information includes subtitle information for which presentation of all subtitle character strings has already been completed. In this case, the data amount of the merged caption information is increased, but the utility value as metadata is increased, such as searching for a program using the merged caption information.

DESCRIPTION OF SYMBOLS 1 ... Broadcasting system 100 ... Broadcast transmission apparatus 101 ... Video packet generation part 102 ... Subtitle packet generation part 103 ... Broadcasting part 200 ... Server apparatus 201 ... Extended subtitle transmission part 3
00 ... Receiver 301 ... Broadcast receiving unit (stream receiving unit) 302 ... Transport decoding unit 303 ... Video / audio decoding unit 304 ... Subtitle decoding unit 305 ... Table receiving unit 306
... Subtitle expansion unit 307 ... Subtitle processing unit 308 ... Display control unit 309 ... Schema storage unit 3
10 ... Subtitle merger 311 ... Application execution unit

Claims (1)

A stream receiver for receiving a video stream;
A subtitle decoding unit that decodes subtitle information including a subtitle character string to be displayed superimposed on the video content from the received video stream;
A table decoding unit for decoding a caption information table including compressed information indicating whether or not at least the caption information is compressed from the received video stream;
Of the subtitle information decoded by the subtitle decoding unit, a subtitle expansion unit that expands what is shown to be compressed by the subtitle information table ;
A schema storage unit for storing the XML schema ,
When the subtitle information table decoded by the table decoding unit includes information specifying the XML schema of the subtitle information, the subtitle expansion unit expands the subtitle information based on the XML schema specified by the information Receiving machine.

Images