WO2016199527A1

WO2016199527A1 - Transmission device, transmission method, reception device, and reception method

Info

Publication number: WO2016199527A1
Application number: PCT/JP2016/063788
Authority: WO
Inventors: 義治出葉; 北里　直久
Original assignee: ソニー株式会社
Priority date: 2015-06-11
Filing date: 2016-05-09
Publication date: 2016-12-15

Abstract

Provided is an application for updating a display in real time on the basis of data transmitted as general-purpose data. In order to update data in an HTML browser in real time, AngularJS, which is one framework of JavaScript(R), is used in an HTML document describing a data broadcast application. Also, an object Navigator.genobjs for delivering information about individual display objects to the HTML browser is prepared, and the data is read in JSON format from the general-purpose data stream and delivered to the HTML browser.

Description

Transmitting apparatus, transmitting method, receiving apparatus, and receiving method

The technology disclosed in this specification includes a transmission apparatus and a transmission method for transmitting an application and a data stream used by the application, and a reception apparatus and a reception method for receiving the application and the data stream used by the application and displaying the application. About.

Digital broadcasting services that employ MPEG2-TS as the media transport method are in operation around the world. In the next-generation digital broadcasting standard, a change from MPEG2-TS to an MMT (MPEG Media Transport) system is being studied (for example, see Patent Document 1). The MMT system can be easily used in a combination of different transmission paths, and can be used in common for a plurality of broadcast and communication transmission paths.

In digital broadcasting, a data broadcasting application is transmitted on a broadcast wave together with components such as video, audio, and captions (see, for example, Patent Document 2). The data broadcasting application is content described using a description language such as BML (Broadcast Markup Language) or HTML (Hyper Text Markup Language) 5. On the broadcast receiver side, content is interpreted using a browser for data broadcasting, and text information is presented to the user, access to the receiver's built-in functions, and bidirectional communication with the server are performed.

JP 2014-200054 A JP 2011-166441 A

An object of the technology disclosed in the present specification is to provide a transmission apparatus and a transmission method for transmitting an application and a data stream used by the application.

A further object of the technology disclosed in the present specification is to provide a receiving apparatus and a receiving method for receiving an application and a data stream used by the application and displaying the application.

The technology disclosed in the present specification has been made in consideration of the above-mentioned problems, and the first aspect thereof is
An application including a first description part describing a procedure for obtaining data from a data stream and a second description part describing a procedure for performing drawing processing based on data obtained by executing the first description part An application transmitter that transmits
A data stream transmission unit for transmitting a data stream including data used in the application;
It is the transmitter which comprises.

According to a second aspect of the technology disclosed in this specification, the application transmission unit of the transmission device according to the first aspect uses the object that reads data from the data stream in a predetermined format. The application including the first description part is configured to be transmitted.

According to a third aspect of the technology disclosed in the present specification, the application transmission unit of the transmission device according to the second aspect describes a procedure for repeatedly drawing processing based on data delivered by the object. The application including the second description part is configured to be transmitted.

According to the fourth aspect of the technology disclosed in this specification, the data stream transmission unit of the transmission device according to the first aspect displays the display object that moves in synchronization with the motion of the broadcast video by the application. For transmitting the data to be transmitted.

According to a fifth aspect of the technology disclosed in this specification, the data stream transmission unit of the transmission device according to the fourth aspect includes a position and a size of the display object on a screen, and a display of the display object. It is configured to transmit the data including information about the data.

According to the sixth aspect of the technology disclosed in this specification, the transmission device according to the first aspect further includes a control information transmission unit that transmits control information including a time stamp indicating a display time of the data. Yes. The data stream transmission unit is configured to intermittently transmit the data.

According to a seventh aspect of the technology disclosed in this specification, the data stream transmission unit of the transmission device according to the sixth aspect performs interpolation for obtaining an interpolation position of the display object until the next timing. The data including information is transmitted intermittently.

According to the eighth aspect of the technology disclosed in this specification, the application transmission unit of the transmission device according to the first aspect is an HTML5 that enables monitoring of a script variable such as a function provided by AngularJS, for example. The application is configured to be transmitted in the form of a document.

Further, the ninth aspect of the technology disclosed in this specification is:
An application including a first description part describing a procedure for obtaining data from a data stream and a second description part describing a procedure for performing drawing processing based on data obtained by executing the first description part An application sending step for sending
A data stream transmission step of transmitting a data stream including data used in the application;
Is a transmission method.

The tenth aspect of the technology disclosed in this specification is:
An application including a first description part describing a procedure for obtaining data from a data stream and a second description part describing a procedure for performing drawing processing based on data obtained by executing the first description part An application receiver for receiving
A data stream receiver for receiving a data stream including data used in the application;
A processing unit for displaying the application using the received data stream;
It is the receiver which comprises.

According to an eleventh aspect of the technology disclosed in this specification, the processing unit of the reception device according to the tenth aspect reads the data read from the data stream received according to the description of the first description unit. Is used to display an application in accordance with the description of the second description section.

According to a twelfth aspect of the technology disclosed in this specification, the processing unit of the reception device according to the eleventh aspect is predetermined from the received data stream by an object used in the first description unit. The data is read in the following format.

According to a thirteenth aspect of the technology disclosed in this specification, the processing unit of the receiving device according to the twelfth aspect is based on the data passed by the object as the process of the second description unit. Thus, the drawing process is repeatedly performed.

According to a fourteenth aspect of the technology disclosed in this specification, the data stream receiving unit of the receiving device according to the tenth aspect displays a display object that moves in synchronization with the motion of a broadcast video. The data is received, and the processing unit is configured to display the display object based on the data read from the data stream in accordance with the description of the first description unit.

According to a fifteenth aspect of the technology disclosed in this specification, the data stream reception unit of the reception device according to the fourteenth aspect includes a position and a size of the display object on a screen, and display of the display object. Receiving the data stream including information relating to data, wherein the processing unit displays the display data of the display object based on the position and size read from the data stream in accordance with the description of the first description unit; It is configured.

According to a sixteenth aspect of the technology disclosed in this specification, the receiving device according to the fourteenth aspect further includes a control information receiving unit that transmits control information including a time stamp indicating a display time of the data. Yes. The data stream receiving unit intermittently receives the data stream including the interpolation information for obtaining the interpolation position of the display object until the next timing, and the processing unit is configured to display data other than the display timing. The display object is displayed at the interpolation position based on the interpolation information.

According to a seventeenth aspect of the technology disclosed in this specification, the application transmission unit of the reception device according to the tenth aspect uses the HTML5 document format application that enables a function to monitor a change in script variables. The processing unit is configured to display the application based on the function of the HTML5 browser.

The eighteenth aspect of the technology disclosed in this specification is:
An application including a first description part describing a procedure for obtaining data from a data stream and a second description part describing a procedure for performing drawing processing based on data obtained by executing the first description part Receiving the application, and
A data stream receiving step for receiving a data stream including data used in the application;
Processing to display the application using the received data stream;
Is a receiving method.

According to the technology disclosed in this specification, it is possible to provide a transmission apparatus and a transmission method for transmitting an application and a data stream used by the application.

Further, according to the technology disclosed in this specification, it is possible to provide a receiving apparatus and a receiving method for receiving an application and a data stream used by the application and displaying the application.

In addition, the effect described in this specification is an illustration to the last, and the effect of this invention is not limited to this. In addition to the above effects, the present invention may have additional effects.

Other objects, features, and advantages of the technology disclosed in the present specification will become apparent from a more detailed description based on embodiments to be described later and the accompanying drawings.

FIG. 1 is a diagram schematically illustrating a configuration example of a digital broadcasting system 10. FIG. 2 is a diagram showing a protocol stack 200 in the digital broadcasting system 10 using the MMT method. FIG. 3 is a diagram illustrating a configuration example of the broadcast transmission system 11. FIG. 4 is a diagram illustrating a configuration example of the receiver 12 that receives a broadcast signal from the broadcast transmission system 11. FIG. 5 is a diagram showing an image of a broadcast signal 500 transmitted by the MMT method. FIG. 6 is a diagram showing a mechanism for designating each asset related to the broadcast service from the MP table in the PA message. FIG. 7 is a diagram for explaining a mechanism for acquiring a file related to a data broadcast transmitted by MMT. FIG. 8 is a diagram for explaining a mechanism for generating a video signal for output. FIG. 9 is a diagram showing a display example of a broadcast screen on which a display object synchronized with the video of the broadcast program is displayed. FIG. 10 is a diagram showing the configuration of MPU and MFU of general-purpose data. FIG. 11 is a diagram showing the structure of a general-purpose data stream in which moving object list description documents describing information on display objects for the number of objects and display data referenced from the documents are packaged. FIG. 12 is a diagram showing a display example of the display object described by the moving object / list description document shown in FIG. FIG. 13 is a diagram illustrating a syntax example of the moving object list description document. FIG. 14 is a diagram for explaining a mechanism for interpolating the display position of the display object based on the interpolation information. FIG. 15 is a diagram illustrating an HTML part of a data broadcasting application. FIG. 16 is a diagram illustrating a JavaScript (registered trademark) part of a data broadcasting application. FIG. 17 is a diagram showing a description example of the JavaScript (registered trademark) part assumed from the HTML part. FIG. 18 is a diagram schematically illustrating a functional configuration for displaying an object that moves according to a broadcast video. FIG. 19 is a diagram illustrating a scenario in which display object information used by a data broadcasting application is based on general-purpose data.

Hereinafter, embodiments of the technology disclosed in this specification will be described in detail with reference to the drawings.

A. System Configuration FIG. 1 schematically shows a configuration example of a digital broadcasting system 10. The illustrated digital broadcast system 10 includes a broadcast transmission system 11 and a receiver 12.

The broadcast transmission system 11 applies the MMT system for transmission of broadcast signals, and transmits each component constituting the broadcast service as an IP (Internet Protocol) packet. Specifically, the broadcast transmission system 11 uses the MMTP (MMT Protocol) payload with the video signal and audio signal codes of the broadcast program, and the content (data broadcast application, etc.) related to the broadcast program and subtitle signals. It is put into MMTP packets and then converted into IP packets, which are transmitted in the form of TLV (Type Length Value) packets on the broadcast transmission path. Here, components related to the broadcast program main body such as video, audio, and subtitles are synchronized media. Further, content, that is, a data broadcasting application encoded in HTML (Hyper Text Transfer Protocol) 5 format or the like, is a non-timed media.

The broadcast transmission system 11 also performs MMT-SI signaling (notification), which is control information related to a broadcast service that transmits synchronous and asynchronous media. The MMT-SI includes a message, a table, and a descriptor. The table is transmitted as a container for messages. Some parameters shown in messages and tables are described in descriptor format.

On the other hand, the receiver 12 receives the TLV packet sent from the broadcast transmission system 11 through the broadcast transmission path. The receiver 12 then decodes a synchronous transmission medium constituting a broadcast program such as video, audio, and subtitles from the received packet, and presents images, audio, and subtitles. Further, when the receiver 12 acquires each data file for data broadcasting from the received packet, the receiver 12 starts an application engine such as an HTML browser and displays the data broadcast linked to the broadcast program and the video of the TV program. Display in part or all of the screen area.

The receiver 12 also receives MMT-SI that is control information related to the broadcast service. Based on the MMT-SI, the receiver 12 performs reception control of transmission media such as video, audio, captions, and data broadcasting, and output (display, audio output) control on the receiver 12.

FIG. 2 shows a protocol stack 200 in the digital broadcasting system 10 using the MMT method.

One broadcast service is composed of video 201, audio 202, subtitle 203, application 204, and content / download 205 components. The video 201 is encoded 211 in the HEVC (High Efficiency Video Coding) format, the audio 202 is encoded 212 in the AAC (Advanced Audio Coding) format, and the subtitle 203 is subtitle encoded 213. The application 204 includes EPG (Electric Program Guide), but is encoded in, for example, the HTML5 format 214.

On the MMT layer 220, the video program and audio

signal encoding components

211 and 212 of the broadcast program are MFU (Media Fragment Unit) and MPU (Media Processing Unit), which are put on the MMTP payload and converted into an MMTP packet. In addition, the subtitle signal encoding component 213 of the broadcast program and the application (data / content) encoding component 214 related to the broadcast program are also MFU and MPU, and are put in the MMTP payload and converted into an MMTP packet. In addition, MMT-SI 221 which is control information related to MMT which is a media transport system (indicating the configuration of a broadcast program) is also put into an MMTP packet on the MMTP payload. Details of the control information MMT-SI related to the MMT will be described later. MPU is a unit of processing, and MFU is a unit smaller than MPU.

The data transmission method 215 of the content download 205 includes four types: a caption / text super transmission method, an application transmission method, an event / message transmission method, and a general-purpose data transmission method. Among these, the application transmission method is used for a data transmission service asynchronous with a broadcast program. The general-purpose data transmission method is a method for transmitting various types of data in a synchronous or asynchronous manner. Data used by a player that presents data other than video, audio, and subtitles and data used in multimedia services. It can be applied to streaming.

In the UDP (User Datagram Protocol) / IP layer 230, the MMTP packet is converted into an IP packet. An NTP (Network Time Protocol) packet 206 including information on the current time for the synchronous media is also converted into an IP packet. Further, these IP packets are converted into TLV packets in the TLV layer 240 and transmitted through the broadcast transmission path 250 which is the lowermost physical layer. A TLV-SI 241 related to a TLV multiplexing format for multiplexing IP packets is also converted into a TLV packet and transmitted through the broadcast transmission path 250. The transmission slot in which the TLV packet is multiplexed is specified by using TLV stream identification information (TLV_stream_id) from a TMCC (Transmission and Multiplexing Configuration Control) signal 251 of the transmission path.

FIG. 3 shows a configuration example of a broadcast transmission system 11 that transmits a broadcast signal having a protocol stack structure as shown in FIG. The broadcast transmission system 11 shown in the figure includes a clock unit 301, a signal transmission unit 302, a video encoder 303, an audio encoder 304, a caption / text super encoder 305, a signaling encoder 306, and a file encoder 307. An electronic data processing system (Electronic Data Processing System: EDPS) 308, a TLV signaling encoder 309, an IP service multiplexer (MUX) 310, a TLV multiplexer (MUX) 311, and a modulation / transmission unit 312. I have.

The signal transmission unit 302 is, for example, a recording / playback device such as a studio of a TV broadcast station or a VTR, and stream data such as video, audio, subtitles, and superimpose characters, which are synchronous media, respectively, with a video encoder 303, an audio The data is sent to the encoder 304 and the subtitle / character super encoder 305. In addition, the signal transmission unit 302 sends the data broadcast application, which is an asynchronous medium, and synchronous and asynchronous general-purpose data to the file encoder 307.

The electronic data processing system 308 is a TV broadcast station scheduler and a file supply source. The electronic data processing system 308 sends a data broadcast application, which is an asynchronous medium, and synchronous and asynchronous general-purpose data to the file encoder 307. The electronic data processing system 308 also sends control information indicating the configuration of the broadcast service to the signaling encoder 306. Also, the electronic data processing system 308 sends control information related to multiplexing of IP packets to the TLV signaling encoder 309.

The video encoder 303 HEVC-encodes the video signal transmitted from the signal transmission unit 302, further packetizes it, and sends an IP packet including the MMT packet of the video signal to the IP service multiplexer 310. Also, the audio encoder 304 AAC-encodes the voice signal sent from the signal sending unit 302, further packetizes it, and sends an IP packet including the MMT packet of the voice signal to the IP service multiplexer 310. The subtitle / character super encoder 305 encodes the subtitle signal and the character super signal transmitted from the signal transmission unit 302, further generates an MPU in units of presentation processing, and generates an IP packet including the MMT packet of the subtitle. Send to IP service multiplexer 310.

When the broadcast transmission system 11 transmits a plurality of broadcast services (that is, components of a plurality of broadcast channels), HEVC encoding of the video signal, AAC encoding of the audio signal, and subtitle / text super encoding are performed for each broadcast service. The processing is performed and sent to the IP service multiplexers 310-1,..., 310-N corresponding to the respective broadcast services # 1 to #N.

The signaling encoder 306 generates control information (MMT-SI) indicating the configuration of the broadcast signal based on information transmitted from the electronic data processing system 308, and an MMT packet in which the MMT-SI is arranged in the payload portion Is sent to the IP service multiplexer 310. MMT-SI includes messages, tables, and descriptors (described above). The signaling encoder 306 generates MMT-SI for each broadcasting service, and also generates MMT-SI for broadcasting services.

The file encoder 307 encodes general data used in the data broadcasting application in HTML5 format or the data broadcasting application transmitted from the signal transmission unit 3102 or the electronic data processing system 308, further packetizes the MMT packet, and The containing IP packet is sent to the IP service multiplexer 310.

The broadcast transmission system 11 is equipped with a plurality of IP service multiplexers 310-1,..., 310-N for each broadcast service (broadcast channel) # 1 to #N to be transmitted. The IP service multiplexer 310 of the channel for each broadcast service is an IP including each of video, audio, subtitles, (broadcast service unit) MMT-SI, data broadcast application, and general-purpose data sent from each encoder 303-307. The packet is multiplexed to generate a TLV packet including a broadcast signal for each broadcast service and an MMT-SI across broadcast services.

Based on the information sent from the electronic data processing system 308, the TLV signaling encoder 309 generates a TLV packet in which the control information (TLV-SI) related to the multiplexing of the IP packet is arranged in the payload portion.

The TLV multiplexer 311 multiplexes TLV packets generated by the IP service multiplexers 310-1 to 310-N and the TLV signaling encoder 309, and generates a TLV stream identified by the TLV stream identification information. To do.

The modulation / transmission unit 312 performs RF modulation processing on the TLV stream generated by the TLV multiplexer 311 and sends it to the broadcast transmission path.

The operation of the broadcast transmission system 11 shown in FIG. 3 will be described.

The video signal transmitted from the signal transmission unit 302 is supplied to the video encoder 303. In the video encoder 303, the video signal is HEVC encoded and further packetized to generate an IP packet including an MMT packet of the HEVC encoded video signal. This IP packet is sent to the IP service multiplexer 310.

Also, the same processing is performed on the audio signal sent from the signal sending unit 302, the caption, and the character super signal. That is, an IP packet including the MMT packet of the AAC encoded audio signal generated by the audio encoder 304 is sent to the IP service multiplexer 310 and the subtitle encoded signal generated by the subtitle / character super encoder 305 is transmitted. An IP packet including the MMT packet is sent to the IP service multiplexer 310.

In addition, the signaling encoder 306 generates control information (MMT-SI) indicating the configuration of the broadcast signal based on information transmitted from the electronic data processing system 308, and the MMT-SI is arranged in the payload portion. An IP packet including the MMT packet is generated. These IP packets are sent to the IP service multiplexer 310.

In addition, data broadcasting applications transmitted from the signal transmission unit 302 or the electronic data processing system 308, and general-purpose data used in the data broadcasting applications are supplied to the file encoder 307. The file encoder 307 encodes the data broadcasting application in HTML5 format and general-purpose data, further packetizes it, and generates an IP packet including this MMT packet. This IP packet is sent to the IP service multiplexer 310.

In each IP service multiplexer 310-1,..., 310-N, video, audio, subtitle, MMT-SI (broadcast service unit and service crossing), data broadcasting application (from each encoder 303 to 307) HTML5 document) and IP packets including each of the general-purpose data are multiplexed to generate a TLV packet constituting one channel.

The TLV signaling encoder 309 generates a TLV packet in which the control information (TLV-SI) related to the multiplexing of the IP packet is arranged in the payload portion based on the information transmitted from the electronic data processing system 308.

In the TLV multiplexer 311, the TLV packets generated by the IP service multiplexers 310-1 to 310 -N and the TLV signaling encoder 309 are multiplexed to generate a TLV stream. The modulation / transmission unit 312 performs RF modulation processing on the TLV stream generated by the TLV multiplexer 311 and sends the RF modulation signal to the broadcast transmission path.

FIG. 4 shows a configuration example of the receiver 12 that receives a broadcast signal from the broadcast transmission system 11. The illustrated receiver 12 includes a tuner / demodulator 401, an MMT demultiplexer (DEMUX) 402, a clock recovery unit 403, a video decoder 404, an audio decoder 405, a character super decoder 406, and a subtitle. Decoder 407, multimedia (MM) cache 408, SI cache 409, broadcast system control unit 410, application (App) engine 411, communication interface (IF) 412, scaler 414, synthesis unit 415˜ 418.

The tuner / demodulator 401 receives a broadcast signal, performs demodulation processing, and obtains a TLV stream. The MMT demultiplexer 402 performs demultiplex processing and depacketization processing on this TLV stream. The demultiplexer 402 includes a packet filter 402-1 and an SI filter 402-2.

The packet filter 402-1 performs filtering of the IP packet based on the TLV stream identification information and the IP address, and further filters the MMTP packet from the IP packet based on the information in the MMTP header so that the video, audio, text Synchronous transmission media such as super and subtitles are distributed to a video decoder 404, an audio decoder 405, a character super decoder 406, and a subtitle decoder 407, respectively.

The packet filter 402-1 distributes each encoding component of multimedia (data broadcasting application) and general-purpose data to the multimedia (MM) cache 408.

Further, the packet filter 402-1 distributes the MMTP packet carrying the signaling information to the SI filter 402-2. Then, the SI filter 402-2 filters the signaling information SI and caches it in the SI cache 409.

Based on the current time information included in the NTP packet filtered by the packet filter 402-1 in the MMT demultiplexer 402, the clock recovery unit 403 generates time information synchronized with this time information, and The video data is output to a video decoder 404, an audio decoder 405, a character super decoder 406, and a subtitle decoder 407, which decode the synchronous transmission media.

The video decoder 404 decodes the encoded video signal obtained by the MMT demultiplexer 402 to obtain a baseband video signal. The audio decoder 405 decodes the encoded audio signal obtained by the MMT demultiplexer 402 to obtain a baseband audio signal. The character super decoder 406 and the caption decoder 407 decode the character super and the caption encoded signal obtained by the MMT demultiplexer 402, respectively, and obtain the character super and caption display signals, respectively.

The broadcast system control unit 410 controls the broadcast service reception operation of the entire receiver 12 based on the MMT-SI cached in the SI cache 409. For example, when the broadcast system control unit 410 analyzes the application execution control information (MH-AIT) included in the MMT-SI and finds a data broadcast application that is instructed to start, the broadcast system control unit 410 Instructs the data broadcasting presentation process.

The application engine 411 is, for example, an HTML browser or the like, and processes a data broadcast application (HTML5 document or the like) cached in the multimedia cache 408 to generate a data broadcast display signal and an audio signal. In addition, the application engine 411 acquires data files (media data used for data broadcast display, link destination applications, etc.) necessary for data broadcast display from the IP network via the communication interface 412. You can also.

In the present embodiment, it is assumed that general-purpose data used in the data broadcasting application is streamed. The application engine 411 acquires the general-purpose data to be streamed and reflects it on the display of the application.

The scaler 414 performs a scaling process on the video signal (broadcast video) decoded by the video decoder 404 in accordance with the screen size of the receiver 12.

The synthesizing unit 415 synthesizes the audio signal decoded by the audio decoder 405 and the audio signal for data broadcasting reproduced by the application engine 411 to generate an audio signal for output.

The composition unit 416 synthesizes the character super display decoded by the character super decoder 406 and the subtitle display decoded by the subtitle decoder 407. The combining unit 417 combines the broadcast video scaled by the scaler 414 and the data broadcast display signal generated by the application engine 411. Further, the synthesizing unit 418 at the subsequent stage synthesizes the broadcast video on which the display of the data broadcast is superimposed, the character super display and the subtitle display output from the synthesizing unit 416, and generates an output video signal.

The operation of the receiver 12 shown in FIG. 4 will be described.

The tuner / demodulator 401 receives a broadcast signal, performs demodulation processing, and obtains a TLV stream. The MMT demultiplexer 402 performs demultiplexing processing and packetization processing on the TLV stream, and encodes each of NTP time information, video, audio, text super and subtitles, data broadcasting application, and general-purpose data. After the signal and signaling information are extracted, the video filter 404, the audio decoder 405, the character super decoder 406, the subtitle decoder 407, the multimedia (MM) cache 408, the SI filter are extracted by the packet filter 402-1. Each is assigned to 402-2.

Also, the NTP packet extracted by the demultiplexer 402 is distributed to the clock recovery unit 403. The clock recovery unit 403 generates time information synchronized with the time information based on the time information put on the NTP packet. That is, the clock recovery unit 403 generates time information that matches the time information generated by the clock unit 301 on the broadcast transmission system 11 side. The generated time information is output to the video decoder 404, the audio decoder 405, the character super decoder 406, and the subtitle decoder 407, respectively, for decoding each synchronous transmission medium.

The encoded video signal extracted by the MMT demultiplexer 402 is sent to the video decoder 404 and decoded to obtain a baseband video signal. The character super encoded signal extracted by the demultiplexer 402 is sent to the character super decoder 406 to be decoded to obtain a character super display signal. Further, the caption encoded signal extracted by the demultiplexer 402 is sent to the caption decoder 407 and decoded, so that a caption display signal is obtained.

The broadcast system control unit 410 controls the entire reception processing operation by the receiver 12 including the processing of the data broadcast application based on the MMT-SI received via the SI filter 402-2 and the SI cache 409.

The application engine 411 processes the data broadcast application (HTML5 document or the like) cached in the multimedia cache 408, and generates a data broadcast display signal and an audio signal. In the present embodiment, it is assumed that general-purpose data used in the data broadcasting application is streamed. In the application engine 411, the acquired general-purpose data is reflected in the display of the application.

The scaler 414 performs a scaling process on the video signal (broadcast video) decoded by the video decoder 404. The synthesizing unit 416 synthesizes the character super display decoded by the character super decoder 406 and the subtitle display decoded by the subtitle decoder 407. Subsequently, the combining unit 417 combines the broadcast video scaled by the scaler 414 and the display signal of the data broadcast generated by the application engine 411. Further, in the subsequent synthesis unit 418, the broadcast video on which the display of the data broadcast is superimposed, the character super display and the caption display output from the synthesis unit 416 are synthesized, and an output video signal is generated.

Figure 8 illustrates the mechanism for generating video signals for output. The scaler 414 performs a scaling process on the video signal (broadcast video) decoded by the video decoder 404 according to the screen size of the receiver 12. The combining unit 417 combines the broadcast video 811 scaled by the scaler 414 and the display signal (graphics) 812 of the data broadcast generated by the application engine 411 to generate a multimedia plane 802. The synthesizing unit 416 synthesizes the character super plane 804 including the character super display decoded by the character super decoder 406 and the subtitle plane 803 including the normal subtitle display decoded by the subtitle decoder 407. Further, the combining unit 418 combines a plane obtained by combining the multimedia plane 802 in which the data broadcast display is superimposed on the broadcast video, the character super plane 804 output from the combining unit 416 and the normal subtitle plane 803 into a background plane. A video signal for output, that is, a display screen is generated by superimposing the data on 801 in order.

The synthesizing unit 415 synthesizes the audio signal decoded by the audio decoder 405 and the audio signal for data broadcasting reproduced by the application engine 411 to generate an output audio signal.

B. Configuration of Broadcast Signal FIG. 5 shows an image of a broadcast signal 500 transmitted from the broadcast transmission system 11 toward the receiver 12 to the broadcast transmission path according to the MMT method.

One broadcast service is a package composed of a plurality of components such as video, audio, captions, and data broadcast applications (provided that one broadcast service corresponds to one channel, that is, a broadcast program). The video, audio, subtitles, etc. related to the main part of the broadcast program are synchronous media, and the data broadcast application is an asynchronous media. In the MMT system, it is easy to use these synchronous and asynchronous media in a combination of different transmission paths.

In the MMT transmission method, the component is defined as “asset”. Each asset is identified by unique asset identification information (asset_id). Asset identification information uniquely corresponds to a component tag that identifies the component. The video asset corresponds to the video component, the audio asset corresponds to the audio component, the subtitle asset corresponds to the subtitle component, and the data file asset for the data broadcasting application corresponds to the data file component. In the example shown in FIG. 5, the MMT transmission paths 501 to 505 for each asset such as video, audio, caption, data broadcasting application, general-purpose data, etc. are used as the broadcast signal 500. In addition, an MMT transmission path 506 for control information (MMT-SI) related to a broadcast service is also used.

Each asset is composed of a set (logical group) of one or more MPUs sharing the same asset identification information. The MPU is a format that is a transmission unit in the MMT system, and is composed of one or more MFUs. The MPU is transmitted to each asset through a dedicated ES (Elementary Stream), that is, an MMT transmission path. Therefore, each MPU is uniquely identified by asset identification information and the sequence number of the MPU on the corresponding MMT transmission path.

In each MMT transmission path, the encoded media data is converted into an MMTP packet in the MPU format and transmitted as an IP packet. Each MMT transmission line corresponds to one IP data flow. The IP data flow referred to here is that the values of the five types of fields of the IP header and the UDP header in the source IP address, destination IP address, IP header protocol type, source port number, and destination port number are all the same. Is a set of IP packets.

In the example shown in FIG. 5, the transmission path 501 transmits an MMTP packet of an encoded video signal composed of an MPU logical group having common asset identification information for video signals. Similarly, an MMT packet of an encoded audio signal composed of an MPU logical group having common asset identification information for audio signals is transmitted on the transmission path 502, and has common asset identification information for subtitle signals on the transmission path 503. An MMTP packet of an encoded subtitle signal composed of an MPU group is transmitted. In addition, the transmission path 504-1 transmits an MMTP packet of an encoding application including an MPU logical group having common asset identification information for a data broadcasting application, and the transmission path 504-2 is used for another data broadcasting application. An MMTP packet of an encoded application consisting of an MPU logical group having common asset identification information is transmitted. In addition, the transmission path 505 transmits the encoded general-purpose data signal MMTP packet including the MPU group having the common asset identification information.

∙ Multiple assets of the same type may be transmitted within one package (broadcasting service). In the example shown in FIG. 5, the assets of the two data broadcasting applications are assigned different asset identification numbers and are transmitted through dedicated ES, that is, MMT transmission paths 504-1 and 504-2, respectively. For example, an asset of a program-linked data broadcasting application linked to a broadcast program is transmitted on one MMT transmission path 504-1, and an unlinked program-related data broadcasting application (not linked to a broadcast program) is transmitted on the other MMT transmission path 504-2. For example, assets such as weather forecasts and news are transmitted. It is also assumed that the assets of the data broadcasting application are provided independently from two or more delivery segments.

Also, the MMT method can use a plurality of broadcast and communication transmission paths in common. For example, asynchronous media such as data broadcasting (HTML5 document, etc.) are transmitted together with synchronous media of the same broadcast service using transmission paths 504-1 and 504-2 of the broadcast signal 500 as shown in FIG. In addition, it can be transmitted via a communication transmission path (not shown) such as an IP network.

The transmission path 505 is used to transmit various data by a general-purpose data transmission method in a broadcasting system using MMT. General-purpose data transmission can be used for streaming data used by HTML5 format data broadcasting applications.

The transmission path 506 is used for MMT signaling, that is, transmission of MMT-SI that is control information indicating information related to the configuration of the MMT package and the broadcast service. On the transmission path 506, the MMT message including the MMT-SI is converted into an MMTP packet and repeatedly transmitted by the carousel method. In FIG. 5, the transmission line for TLV-SI is not shown.

MMT-SI has messages, tables, and descriptors. Examples of messages transmitted on the transmission line 506 include a PA message 510, an M2 section message 520, and a data transmission message 530.

The PA message 510 is control information indicating the configuration of a broadcast program and the like, and is a container that stores an MP (MMT Package) table 511 describing information constituting a package such as an asset list and its position.

The MP (MMT Package) table 511 is a table showing basic reception control information for each broadcast service, and specifically provides information constituting the package, such as an asset list and asset location information. The MP table 511 includes MPT descriptors such as application service descriptors. The application service descriptor indicates entry information of the data broadcasting application (for example, data transmission message and entry information of each M2 section message that transmits MH-AIT and EMT). In the MP table 511, an MPU time stamp descriptor that provides the presentation time of the MPU is arranged.

PA message 510 is an entry point of the broadcast service, and fixed packet identification information (for example, 0x0000) is assigned to the MMTP packet that transmits PA message 510. Therefore, on the receiver side, the PA message 510 can be obtained by directly specifying the fixed packet identification information on the MMT transmission path 506. Then, as shown in FIG. 6, referring to the MP table 511 transmitted by the PA message 510, each asset (video, audio, caption, data broadcasting application, general-purpose data) constituting the package (broadcast program) and others Indirect signaling messages can be specified. In the MP table, an MH-data encoding scheme descriptor is arranged to identify a data encoding scheme for each asset other than video and audio.

Although omitted in FIG. 5, examples of the table stored in the PA message 510 include a PL (Package List) table (PLT) and an LC (Layout Configuration) table (LCT) in addition to the MP table. PLT indicates a list of IP data flows and packet IDs for transmitting PA messages of MMT packages provided as broadcasting services, and IP data flows for transmitting IP services. The LCT is used to associate layout information for presentation with a layout number. The PA message is a container that can store up to three tables: an MP table, a PL table, and an LC table.

The M2 section message 520 is a message that transmits the section extension format of MPEG-2 Systems, and is a container that stores the section format signaling table one by one. In one M2 section message 520, tables such as MH-AIT (Application Information Table) 521 and EMT (Event Message Table) 522 are stored one by one.

MH-AIT 521 is a table that specifies dynamic control information related to applications and additional information necessary for execution. Specifically, the MH-AIT 521 indicates the processing method of each application to be controlled in the table by application_control_code in the application information loop, and arranges various application information descriptors describing application information. Here, application_control_code is the application processing method such as autostart (automatic application startup), present (application executable state), kill (application termination), prefetch (application acquisition and retention (pre-cache)). Show. In addition, the transmission protocol descriptor, which is one of the application information descriptors, specifies the transmission method of the application and the location information corresponding to the transmission method (common in the case of HTTP / HTTPS and MMT asynchronous transmission). URL (Uniform Resource Locator) format for specifying location information).

The EMT 522 is a signaling table used for an event message transmission method, and stores information (event message descriptor) related to an event message (synchronous / asynchronous message from a broadcasting station to an application on a receiver). The event / message transmission system provides a means for sending message information from a broadcasting station to a data broadcasting application operating on a receiver immediately or at a specified time.

The data transmission message 530 is a message for transmitting control information related to transmission of the data broadcasting application. The data transmission message 530 includes a data directory management table (Data Directory management Table: DDMT) 531, a data asset management table (Data Asset Management Table: DAMT) 532, and a data content management table (Data Content ConfigurationTable 33: This is a container that can store up to three tables at the same time. The data directory management table and the data asset management table are indispensable for the data broadcasting service, but the data / content management table is optional.

Here, the data directory management table 531 is a table for managing data broadcasting applications in directory units (in other words, production units of data broadcasting applications). The table describes the directory structure for items (directories and subdirectories and files included in a directory) included in one package, so that the file structure of the application and the structure for file transmission can be separated. it can. The table shows the correspondence between the path name of each node, such as a directory or file, and the node tag that identifies each node in the data transmission message.

Also, the data / asset management table 532 is a table for managing the data broadcasting application in units of assets, and describes version information for each MPU with the configuration of the MPU in the asset. Specifically, the table shows download identification information of each component, and a node tag and node on the MMT transmission path that identify a node (file or directory) included in each MPU that transmits the component in the data transmission message. The correspondence with the item identification information to identify is shown.

Also, the data / content management table 533 is a table for managing the data broadcasting application for each presentation unit (Presentation Unit: PU). The table shows presentation unit identification information for identifying a data broadcasting application for each presentation unit, and a node tag for identifying a node (file or directory) included in each presentation unit in a data transmission message. In addition, the table can include link destination presentation unit identification information for identifying a presentation unit linked to the presentation unit, and information on cache control of each presentation unit. Can be used for flexible and effective cache control.

In the transmission system of the data broadcasting application by MMT, by utilizing the above three kinds of signaling tables 531 to 533 that are transmitted by the data transmission message, the transmission data structure of the file unit and the directory structure in contents (data broadcasting application) production Can independently represent the data structure of usage units such as application units and presentation units. The configuration of each of the signaling tables 531 to 533 is defined by ARIB STD B-60.

In MH-AIT, location information of each application is described in a URL format, in other words, a path name composed of a combination of a directory node name and a file name. On the other hand, the asset path name can be converted into identification information (packet identification information) on the MMT transmission path by using the data directory management table and the data asset management table transmitted in the data transmission message. With reference to FIG. 7, a mechanism for acquiring a file related to a data broadcast transmitted by MMT will be briefly described.

The receiver refers to the application information loop in the MH-AIT transmitted in the M2 section message, and detects an application to be started (for example, an application whose automatic start is specified by the application control code). be able to. In addition, the receiver can acquire the location information of the application, that is, the path name, from the transmission protocol descriptor arranged in the MH-AIT. In the case of an application transmitted by broadcasting, as indicated by reference numeral 701, the node tag of the file corresponding to the specified path name can be obtained from the data directory management table in the data transmission message.

Then, as indicated by reference numeral 702, the component tag of the data asset in which the item having the node tag obtained in the data directory management table is transmitted from the data asset management table in the data transmission message, Download identification information, MPU sequence number, and item identification information can be obtained.

Further, as indicated by reference number 703, when the location information of the asset having the component tag obtained in the data asset management table is acquired from the MP table, the corresponding file is actually stored as indicated by reference number 704. The data asset to be transmitted can be identified.

Then, within the identified data asset, the file identification corresponding to the carousel is repeated using the download identification information obtained from the data asset management table and the download identification information described in the header area of the MMTP packet for transmitting the item. The unit of transmission can be uniquely identified. As indicated by a reference number 705, an item having an MPU sequence number and item identification information obtained from the data / asset management table can be designated as a desired file among items repeatedly transmitted.

C. A data broadcast application synchronized with the movement of a broadcast video is generally a structured document file encoded in a format such as HTML5. As described with reference to FIG. 4, in the receiver 12, the application engine 411 equipped with an HTML browser or the like analyzes the encoded application to generate a data broadcast display signal, which is converted into a video decoder The composition unit 417 superimposes on the broadcast video decoded in 404. In some cases, a display signal for subtitles or text superimpose is further synthesized on a broadcast video obtained by synthesizing a data broadcast.

Some data broadcasting applications include independent data broadcasting that is not directly related to the main part of the broadcast program and is completed by itself, but many are program-linked data broadcasting applications that are linked to the broadcast program. The present specification discloses a technique for providing a data broadcast application that displays graphics that move in synchronization with the motion of a broadcast video. In the example shown in FIG. 8, the graphics that move in synchronization with the movement of the broadcast video means that the graphics 812 that is superimposed on the broadcast video 811 moves in synchronization with the movement in the broadcast video 811. .

FIG. 9 shows a display example of a broadcast screen on which display objects synchronized with the broadcast program video are displayed. This figure is a soccer game broadcast screen, and displays a balloon-like display object that displays information about the players participating (eg, player name, position, number of goals, etc.) and follows each player's movement. Overlay is displayed.

However, FIG. 9 is only one example of displaying a display object on a broadcast program screen in an overlay manner. The same overlay display of display objects can be applied to broadcast programs that relay various sports such as baseball, basketball, golf, athletics, judo, and car races other than soccer. Of course, the same overlay display of display objects can be applied to broadcast programs of various genres such as dramas and music programs other than sports.

For example, a moving object list description (MOLD) document described below can be used as a method for expressing information related to a display object synchronized with video. The moving object list description document has a document structure that describes a list of information on display objects for the number of objects displayed on the screen at the same time. In the moving object list description document, as the information for each display object, the position and size of the display object on the screen, and the display data itself or display data acquisition destination information (data reference number to be referred to or URL) do it.

Also, for the data broadcasting application in the HTML5 format, data for changing the display object every moment is transmitted as a general-purpose data stream. As shown in FIG. 9, when it is desired to display a display object synchronized with the video of the broadcast program, the control information of the display object that changes from time to time may be transmitted in a synchronized general-purpose data stream synchronized with the video.

Also, as information about the display object, interpolation information for obtaining the interpolation position of each display object until the next timing is also described in the moving object list description document. The interpolation information includes information for obtaining an interpolation position for each time obtained by subdividing the time until the next timing. Transmission of a general-purpose data stream for each video frame is not practical because of the large amount of information. The general-purpose data stream is transmitted intermittently, and the receiver side displays the display object at the interpolation position obtained based on the interpolation information described in the moving object list description document, until the next timing. The display object can be smoothly moved during the interval. In the broadcast program display example shown in FIG. 9, the display object can be displayed in an overlay manner by interpolation so as to smoothly follow the soccer player constantly moving around on the pitch. For example, by describing the velocity vector information of the display object as the interpolation information, the interpolation position of the display object can be linearly interpolated. Furthermore, by including the angular velocity information, it becomes possible to interpolate the arc of the interpolation position of the display object. The velocity vector and the angular velocity can be obtained by analyzing the movement of the corresponding player.

Further, URL information of hyperlinks may be further described in the moving object list description document as information on display objects. On the receiver side, when a video viewer operates a display object, information can be acquired and displayed from a link destination specified based on the URL specified for the display object.

Also, the display data of the display object that is commonly referenced from multiple moving object list description documents is not packaged with the moving object list description document and transmitted every time. It may be transmitted in a packaged form with a description document, and may be commonly referred to from a moving object list description document transmitted thereafter. By transmitting general-purpose data consisting only of a moving object list description document that refers to external display data, the amount of transmission information can be reduced (that is, the transmission band can be saved). If the display data is updated and packaged with the moving object list description document and transmitted, the band saving effect is enhanced. However, it is preferable to transmit general-purpose data in which display data is packaged at regular intervals so that a display object can be displayed even by a receiver that has been tuned in the middle of a program.

Further, the general-purpose data stream may be intermittently transmitted at a constant cycle, or the transmission cycle may be variably controlled. For example, the transmission cycle of the general-purpose data stream may be controlled according to the movement of the display object. For example, when the movement of the display object is fast, it is transmitted at intervals of 1 second, so that a more accurate interpolation position of the display object can be easily obtained. Conversely, when the movement of the display object is slow, the transmission information amount may be reduced (that is, the transmission band is saved) at intervals of 5 seconds.

From the transmission side (broadcasting station), the moving object list description document and the display data (image file) of the display object to be referred to are packaged, and a time stamp indicating the display time of each package as a general-purpose data stream. Broadcast intermittently. The time stamp information indicating the display time of the package unit can be indicated by an MPU time stamp descriptor that provides the presentation time of the MPU. Then, on the receiver side, when the information for drawing each display object is extracted from the moving object list description document, the display object synchronized with the video is displayed on the screen of the broadcast program based on the display time specified by the time stamp. Can be overlaid. An application engine that executes a data broadcasting application sequentially draws an object that follows the movement of a video by sequentially accessing control information received as a general-purpose data stream and updating the display of the object.

In general data transmission processing, MPU is the unit of processing. The MPU includes data used at a certain timing and is a unit that can perform processing such as presentation in units of MPU. FIG. 10 shows the configuration of MPU and MFU of general-purpose data. The MPU of general-purpose data is composed of one or more MFUs without arranging MPU metadata and movie fragment metadata. Each MFU includes a general-purpose data header part and a data part. Data of an arbitrary size can be stored in the data portion of each MFU. For example, one file can be sent by one MFU.

FIG. 11 schematically shows the structure of a general-purpose data stream in which a moving object list description document describing information about display objects for the number of objects and display data referenced from the document are packaged. Yes.

This general-purpose data stream is transmitted as a synchronous MPU. In general-purpose data transmission processing, the MPU is a unit of processing, and the MPU is composed of a plurality of MFUs. When information about a display object is transmitted as general-purpose data, a moving object list description document and a display data package referenced from the document are transmitted as a processing unit by one MPU.

In FIG. 11, reference numeral 1100 is an MPU that transmits one package. The moving object list description document is stored in the first MFU indicated by reference numeral 1110 and is referenced from this moving object list description document. Display data (image file) of each display object is stored in subsequent MFUs 1120-1,..., 1120-N.

The moving object list description document stored in the data part of the top MFU 1110 is composed of a list of information on display objects for N objects to be displayed on the screen. As shown in FIG. 12, when four display objects # 1 to # 4 are simultaneously displayed on the screen, N = 4, and the moving object list description document lists information on the four display objects. .

In the moving object list description document, tag information (Tag) for identifying a display object and information for displaying the display object on the screen (Position) as information about the display object # 1 in the information block indicated by reference numeral 1111. ) And size (Size), velocity vector information (Vector) and angular velocity information (θ) as interpolation information, and display object display data information (MediaData). As described above, hyperlink URL information may be further included as display object information (not shown in FIG. 11). Although omitted for simplification of the drawings, it should be understood that the other information blocks also describe similar information regarding the corresponding display objects.

The display data information (MediaData) of the display object can be represented by display data acquisition destination information (such as a data reference number to be referred to or a URL). In the example shown in FIG. 11, it is assumed that display data (image file) of a display object is also packaged with a moving object / list description document, and acquisition source information such as URL is transmitted within the same MPU. The location information of the MFU storing the corresponding image file is designated. Alternatively, the display data itself, that is, binary data can be described in the information block of the moving object list description document.

Also, reference number 1130 is an MP table, which designates asset identification information and location information of each asset (video, audio, caption, data broadcasting application, etc.) constituting a broadcast signal. The display time of the MPU 1100 is designated by the MPU time stamp descriptor 1131 arranged for the asset of the general-purpose data stream in the second loop of the MP table. That is, the MPU time stamp descriptor 1131 specifies the sequence number of the MPU 1100 that transmits the moving object list description document by mpu_sequence_number, and the display time by mpu_presentation_time.

When the same function is realized in the MPEG TS system broadcasting system instead of the MMT system, the same information is transmitted in a PES (Packetized Elementary Stream) packet and the display time is indicated by PTS (Presentation TimeStamp). What should I do?

FIG. 13 shows a syntax example of the moving object list description document. When “mld” is designated by the parameter data_type_byte in the general-purpose data header part of the MFU, a moving object list description document as shown is stored in the data part. In the figure, parameters prefixed with @ are attribute information, and parameters without @ indicate elements. Hereinafter, the meaning of each parameter will be described.

“Mold” indicates that the data is a moving object list description document. At the same time, the information of the moving object mo is arranged in the document by the number of objects to be displayed on the screen.

Each moving object mo is identified by an object tag. An object tag as an attribute value is described in the moTag field.

As information on each moving object mo, object position information (position), object size information (size), object motion vector information (mv), information on object display data (media), link destination information at the time of object operation (Link) is included.

The object position information (position) includes the horizontal coordinate x and the vertical coordinate y of the display position of the object on the screen (upper left corner in the case of a rectangular area) as attribute values, and each indicates the pixel position as an integer value.

The object size information (size) includes the width x and height y of the rectangular area indicating the size of the object to be displayed on the screen as attribute values, each represented by a pixel value (integer value).

The object motion vector information (mv) is an attribute value including a velocity vector composed of the horizontal velocity x and the vertical velocity y of the object, and angular velocity information θ (provided that the movement of the object is a curved motion expressed by an arc). An integer value indicating each is described. On the receiver side, the motion vector information (mv) can be used to determine the interpolation position of the object up to the timing of the next MPU (that is, the next moving object list description document) (described above).

Information related to display data (media) indicates the type of display data of the object (JPEG or PNG) and the acquisition destination information of the display data of the object. The acquisition source of the display data is roughly classified into any of the same MPU, the previous MPU, or on the Internet, and is designated by the parameter material_location in the additional information (mold_info) of the general data header part (described above). In the display data acquisition destination information, an acquisition destination at a location specified by material_location is specified in url format, and an image file in a format specified by the type of display data is placed in the acquisition destination specified by url. That is the premise. When 0 is designated as the display data acquisition destination information, the display data binary data may be stored in the display data type area.

The link destination information (link) at the time of object operation specifies the link destination information of the object in url format. When a video viewer operates a display object, information can be acquired and displayed from a link destination specified by link destination information (link).

FIG. 14 illustrates a mechanism for interpolating the display position of the display object based on the interpolation information specified by the moving object list description document. Reference numeral 1401 indicates the display position (X1, Y1) of the display object specified in the moving object list description document transmitted at a certain timing T1. Reference numeral 1402 indicates the display position (X2, Y2) of the same display object specified in the moving object list description document transmitted at the next timing T2. Reference numerals 1411 to 1414 are interpolation positions for each time when the timing interval is subdivided. In the moving object list description document transmitted at the timing T1, it is assumed that the velocity vector (x, y) and the angular velocity θ are specified as the interpolation information of the display object. Based on the velocity vector (x, y), the interpolation positions 1411,... At each subdivided time can be linearly interpolated, and the interpolation positions 1411,. Is possible.

An application engine 411 that executes an application that displays a display object on a video in an overlay manner sequentially acquires moving object list description documents transmitted in a general-purpose data stream, and interpolates based on interpolation information. It is possible to display a display object that moves smoothly according to the movement of the display.

Next, a process for displaying graphics moving in synchronization with the motion of the broadcast video using the general-purpose data stream as described above by the application engine, that is, the HTML browser will be described.

In the display example of the application as shown in FIG. 12, it is necessary to move the display object in real time following the movement of each soccer player on the pitch in the video of the soccer game. For this reason, a moving object list description document describing information for rendering each display object that changes from moment to moment is transmitted as a general-purpose data stream (described above). Then, the HTML browser needs to sequentially access the information of each display object from the general-purpose data stream and update the display object in real time.

In this embodiment, AngularJS, which is one of the frameworks of JavaScript (registered trademark), is used in an HTML document that describes a data broadcasting application in order to update data in an HTML browser in real time. AngularJS has a function of monitoring changes in script variables (well known). In addition, an object for transferring information of each display object to the HTML browser is prepared, and data is read from the general-purpose data stream in the JSON (Java Script (registered trademark) Object Notation) format, and is written in the HTML document. The data is transferred to the HTML browser as if the data had been stored. In this specification, an object for passing data is referred to as Navigator. It will be called genobjs. The JSON format is a format in which key / value pairs are paired with a colon, these pairs are listed separated by commas, and the whole is enclosed in curly braces {.

FIG. 15 illustrates the HTML part of the data broadcasting application.

The HTML part is an HTML document described in the HTML5 format, and consists of a head document type declaration (Document Type Definition: DTD) followed by an html element. In the example shown, <! “doctype html>” corresponds to the document type declaration, and declares that the document is created in HTML5.

The html element is composed of a head element and a body element. The head element is delimited by <head> to </ head>, and clearly shows the header information of the document. The body element is delimited by <body> to </ body>, and clearly indicates the content of the document, that is, the main body of the document displayed by the browser.

First, the ng-app attribute is specified in the html element to declare that this HTML document is an application using AngularJS (that is, the AngularJS function is enabled). Incidentally, the attribute used in AngularJS is prefixed with ng- like this ng-app attribute. The ng-app attribute specifies a range in which AngularJS operates. As shown in the figure, when <html ng-app> is set, the entire HTML page becomes the operation range of AngularJS. A <script> tag is arranged in the head element, that is, <head> to </ head>, and the URI of AngularrJS, which is an external script file, is specified by the src attribute. In addition, a <script> tag is further placed in <head> to </ head>, and the file sample.script.script describing the script is placed in the same directory as the document with the src attribute. Specifies reading js.

In the <body> tag, the controller SampleController for the view is specified by the ng-controller attribute. The controller defined in the view (HTML) is described in a JavaScript (registered trademark) file (described later).

<P> tag is a {{item. length}} specifies that object is one paragraph. The <ul> tag instructs the display of an unordered list. Then, the list item is specified by the subsequent <li> tag.

And the ng-repeat directive specified in the <li> tag executes repeated drawing of items in the object genobjs. genobjs is an object Navigator. prepared for reading data from a general-purpose data stream in the JSON format. It is a reserved word of genobjs, and the moving object tag (mottag), the position of the moving object (positionx and positiony), etc. specified by double parentheses {{...}} are sequentially read into genobjs in JSON format and changed. The object is repeatedly displayed in the browser. Therefore, the body element of the HTML part is stored in the object Navigator. It can be said to be a “second description part” that describes a procedure for drawing processing based on data acquired by genobjs.

Describe the view controller SampleController specified in the HTML part (ng-controller attribute) in the JavaScript (registered trademark) file. FIG. 16 illustrates the JavaScript (registered trademark) part of the data broadcasting application.

Object Navigator prepared for the transfer of general-purpose data from the general-purpose data stream (data reading in JSON format). Genobjs is described as an argument $ scope of the controller SampleController. Navigator. Since genobjs sequentially reads each parameter value of the moving object list description document from the general-purpose data stream in the JSON format, it is assumed that the data as shown in FIG. 17 is included in the JavaScript (registered trademark) part. Thus, an HTML part as shown in FIG. 15 can be described. Therefore, the JavaScript (registered trademark) part shown in FIG. 16 can be said to be “a first description part describing a procedure for a browser to acquire data from a data stream”.

FIG. 18 schematically shows a functional configuration for displaying an object that moves in accordance with the broadcast video. Reference numeral 1810 denotes an HTML browser (application engine 411) that executes a data broadcasting application described in an HTML document as shown in FIG. The browser 1810 includes a data reading unit 1811 that reads data from the lower-layer general-purpose data reception processing unit 1820, and a display processing unit 1812 that repeatedly displays objects based on the latest general-purpose data read by the data reading unit 1811. It has.

The substance of the data reading unit 1811 is an object Navigator. genobjs, and the latest parameter value related to the display object is written in the reserved word genobjs. Therefore, from the display processing unit 1812, it is assumed that data is included in the JavaScript (registered trademark) unit as shown in FIG.

The value of general-purpose data read by the data reading unit 1811 is passed to the display processing unit 1812 using genobjs.

The display processing unit 1812 checks whether or not it coincides with the MPU presentation time specified by the MPU time stamp descriptor every time the display timing of the video frame arrives.

When the current display timing is the MPU presentation time specified by the MPU time stamp descriptor, the display processing unit 1812 describes the moving object list description document transmitted by the first MFU of the corresponding MPU. The display object is overlaid on the video frame based on the position information (positionx, positiony) of the object with reference to genobjs.

If the current display timing is not the MPU presentation time specified by the MPU time stamp descriptor, the display processing unit 1812 refers to genobjs and refers to element information of the motion vector (mv) (that is, velocity vector information). In addition, based on the angular velocity information), the interpolation position is calculated at the current time, and the display of the object is updated.

FIG. 19 exemplifies a scenario in which display object information used by the data broadcasting application is based on general-purpose data. In the figure, a broadcast signal transmitted from a certain broadcaster (broadcast station) is illustrated, and the horizontal axis is a time axis. The broadcast signal includes a video asset 1910, an audio asset 1920, a data (application) asset 1930, and a general-purpose data asset 1940 as components constituting a broadcast program. The broadcast signal may further include other assets, but is omitted for simplification of the drawing.

Also, the broadcast signal includes a control signal 1950 for transmitting control information (MMT-SI) such as a signaling table such as an MP table and a signaling message serving as a container of the signaling table.

In the figure, the data asset 1930 transmits a data broadcasting application (see FIGS. 15 and 16) for displaying a display object on the broadcast program video, and the general data asset 1940 displays the data broadcasting application. It is assumed that object control information, that is, a moving object list description document, is intermittently transmitted as a general-purpose data stream. The display time of each MPU transmitted by the general-purpose data asset 1940 is specified by the MPU time stamp descriptor of the MP table (described above).

Reference numeral 1941 indicates an MPU in which a moving object list description document and display data (image file) of a display object to be referred to are packaged. A moving object list description document is stored in the data portion of the first MFU, and display data referred to from the moving object list description document is stored one by one in the second and subsequent MFUs (see FIG. 11). See

Reference numeral 1942 indicates an MPU that does not package display data. This MPU transmits only one MFU storing the moving object list description document. From this moving object list description document, display data already transmitted by the preceding MPU 1941 is referenced. Similarly, the MPU indicated by the reference number 1943 transmits only the MFU of the moving object list description document. Reduces the amount of transmission information (ie, saves transmission bandwidth) by packaging and transmitting display data that is commonly referenced in multiple moving object list description documents to a single moving object list description document. can do.

Reference numeral 1944 indicates an MPU in which a moving object list description document and display data of a display object to be referred to are packaged. Similar to the MPU indicated by the reference number 1941, the moving object list description document is stored in the data portion of the first MFU, and the display data referenced from this moving object list description document is stored in the second and subsequent MFUs. One by one is stored (same as above).

When the display data is updated, the MPU 1944 including the updated display data and the packaged moving object list description document may be transmitted. Alternatively, in order to be able to display a display object even in a receiver that has been tuned in the middle of a program, MPU 1944 in which display data is packaged is transmitted after a certain period of time has passed since MPU 1941 was transmitted.

The MPU time stamp descriptor in the MP table indicated by the reference number 1951 specifies the display time T10 of the display object by the moving object list description document transmitted by the MPU indicated by the reference number 1941. Similarly, the MPU time stamp descriptor in the MP table indicated by reference number 1952 specifies the display time T20 of the MPU indicated by reference number 1942, and the MPU time stamp descriptor in the MP table indicated by reference number 1953 is the reference number. The MPU display time T30 indicated by 1943 is specified, and the MPU time stamp descriptor in the MP table indicated by reference number 1954 specifies the MPU display time indicated by reference number 1944.

On the receiver side, when MPU 1941 transmitted by broadcast waves is received, the moving object list description document transmitted by the first MFU is analyzed, and the display data transmitted by the second and subsequent MFUs is converted to general data, Retain only the retention period specified in the header part.

When the display time T10 specified by the MPU time stamp descriptor of the MP table 1951 arrives, the receiver displays each display as indicated by reference numeral 1961 according to the description content of the moving object list description document received by the MPU 1941. The object is overlaid on the broadcast program video.

In addition, the receiver designates each display object in the moving object list description document at a time T11 obtained by subdividing the time interval until the display time T20 designated for the next MPU 1942 arrives. An interpolation position is obtained based on the interpolated information (velocity vector, angular velocity), and the display of each display object is updated as indicated by reference numeral 1962.

Thereafter, when the display time T20 specified by the MPU time stamp descriptor of the MP table 1952 arrives, the receiver precedes as indicated by the reference number 1963 according to the description content of the moving object list description document received by the MPU 1942. With reference to the display data transmitted by the MPU 1941, each display object is displayed as an overlay on the video of the broadcast program.

As described above, the technology disclosed in this specification has been described in detail with reference to specific embodiments. However, it is obvious that those skilled in the art can make modifications and substitutions of the embodiments without departing from the scope of the technology disclosed in this specification.

In the present specification, the embodiment in which the technology disclosed in this specification is applied to a broadcasting system adopting the MMT method for media transmission has been mainly described. However, the gist of the technology disclosed in this specification is limited to this. It is not something. The technology disclosed in the present specification can be similarly applied to various types of broadcasting systems having a mechanism for streaming data used in a data broadcasting application. For example, when a similar function is realized in an MPEG TS broadcast system, display object information may be similarly transmitted using a PES packet, and a display time may be indicated by PTS.

Further, in this specification, an HTML document using AngularJS, which is one of the JavaScript (registered trademark) frameworks, has been described as an example. However, in order to realize the technology disclosed in this specification, data broadcasting is performed. The application is not limited to such a description format, and a framework having a similar function for monitoring changes in script variables may be used.

In short, the technology disclosed in the present specification has been described in the form of exemplification, but the content described in the present specification should not be interpreted in a limited manner. In order to determine the gist of the technology disclosed in this specification, the claims should be taken into consideration.

Note that the technology disclosed in the present specification can also be configured as follows.
(1) A first description part describing a procedure for acquiring data from a data stream, and a second description part describing a procedure for performing drawing processing based on data acquired by executing the first description part An application transmitter that transmits an application including
A data stream transmission unit for transmitting a data stream including data used in the application;
A transmission apparatus comprising:
(2) The application transmission unit transmits the application including the first description unit using an object that reads data from the data stream in a predetermined format.
The transmission device according to (1) above.
(3) The application transmission unit transmits the application including the second description unit that describes a procedure for repeatedly performing drawing processing based on data passed by the object.
The transmission device according to (2) above.
(4) The data stream transmission unit transmits the data for the application to display a display object that moves in synchronization with the motion of the broadcast video.
The transmission device according to (1) above.
(5) The data stream transmission unit transmits the data including the position and size of the display object on the screen and information regarding display data of the display object.
The transmission device according to (4) above.
(6) a control information transmission unit that transmits control information including a time stamp indicating a display time of the data;
The data stream transmission unit transmits the data intermittently.
The transmission device according to (1) above.
(7) The data stream transmission unit intermittently transmits the data including interpolation information for obtaining an interpolation position of the display object until the next timing.
The transmission device according to (6) above.
(8) The application transmission unit transmits the application in the form of an HTML5 document that enables monitoring of script variables such as functions provided by AngularJS, for example.
The transmission device according to any one of (1) to (7) above.
(9) A first description part describing a procedure for acquiring data from a data stream, and a second description part describing a procedure for performing drawing processing based on data acquired by executing the first description part An application sending step for sending an application including:
A data stream transmission step of transmitting a data stream including data used in the application;
A transmission method.
(10) A first description part describing a procedure for obtaining data from a data stream, and a second description part describing a procedure for performing drawing processing based on data obtained by executing the first description part An application receiver that receives an application including
A data stream receiver for receiving a data stream including data used in the application;
A processing unit for displaying the application using the received data stream;
A receiving apparatus comprising:
(11) The processing unit displays an application according to the description of the second description unit by using the data read from the data stream received according to the description of the first description unit.
The receiving device according to (10) above.
(12) The processing unit reads the data in a predetermined format from the received data stream by the object used in the first description unit.
The receiving device according to (11) above.
(13) The processing unit repeatedly performs a drawing process based on the data delivered by the object as the process of the second description unit.
The receiving device according to (12) above.
(14) The data stream receiving unit receives the data for displaying a display object that moves in synchronization with the motion of the broadcast video,
The processing unit displays the display object based on the data read from the data stream in accordance with the description of the first description unit.
The receiving device according to (10) above.
(15) The data stream receiving unit receives the data stream including information on the position and size of the display object on the screen and display data of the display object,
The processing unit displays the display data of the display object based on the position and size read from the data stream according to the description of the first description unit;
The receiving device according to (14) above.
(16) a control information receiving unit that transmits control information including a time stamp indicating a display time of the data;
The data stream receiving unit intermittently receives the data stream including interpolation information for obtaining an interpolation position of the display object until the next timing,
The processing unit displays a display object at an interpolation position based on the interpolation information other than the display timing.
The receiving device according to any one of (14) and (15).
(17) The application transmission unit receives the application in an HTML5 document format that enables a function of monitoring changes in script variables,
The processing unit displays the application based on the function of the HTML5 browser.
The receiving device according to any one of (10) to (16).
(18) A first description part describing a procedure for acquiring data from a data stream, and a second description part describing a procedure for performing drawing processing based on data acquired by executing the first description part Receiving an application including: an application receiving step;
A data stream receiving step for receiving a data stream including data used in the application;
Processing to display the application using the received data stream;
Receiving method.

DESCRIPTION OF SYMBOLS 10 ... Digital broadcasting system 11 ... Broadcast transmission system, 12 ... Receiver 301 ... Clock part, 302 ... Signal transmission part 303 ... Video encoder, 304 ... Audio encoder 305 ... Subtitle / text super encoder 306 ... Signaling encoder 307 ... File encoder 308 ... Electronic data processing system,
309 ... TLV signaling encoder 310 ... IP service multiplexer 311 ... TLV multiplexer 312 ... Modulation / transmission unit 401 ... Tuner demodulation unit 402 ... MMT demultiplexer 403 ... Clock recovery unit 404 ... Video decoder 405 ... Audio decoder 406 ... Character super decoder, 407 ... Subtitle decoder 408 ... Multimedia cache, 409 ... SI cache 410 ... Broadcast system control unit 411 ... Application engine 412 ... Communication interface 414 ... Scaler 415 to 418 ... Composition Part

Claims

An application including a first description part describing a procedure for obtaining data from a data stream and a second description part describing a procedure for performing drawing processing based on data obtained by executing the first description part An application transmitter that transmits
A data stream transmission unit for transmitting a data stream including data used in the application;
A transmission apparatus comprising:
The application transmission unit transmits the application including the first description unit using an object for reading data in a predetermined format from the data stream;
The transmission device according to claim 1.
The application transmission unit transmits the application including the second description unit describing a procedure for repeatedly drawing processing based on data passed by the object.
The transmission device according to claim 2.
The data stream transmission unit transmits the data for the application to display a display object that moves in synchronization with the motion of the broadcast video.
The transmission device according to claim 1.
The data stream transmission unit transmits the data including information on a position and a size of the display object on the screen and display data of the display object.
The transmission device according to claim 4.
A control information transmission unit that transmits control information including a time stamp indicating a display time of the data;
The data stream transmission unit transmits the data intermittently.
The transmission device according to claim 1.
The data stream transmission unit intermittently transmits the data including interpolation information for obtaining an interpolation position of the display object until the next timing.
The transmission device according to claim 6.
The application transmission unit transmits the application in the form of an HTML5 document that enables a function to monitor a change in script variables.
The transmission device according to claim 1.
An application including a first description part describing a procedure for obtaining data from a data stream and a second description part describing a procedure for performing drawing processing based on data obtained by executing the first description part An application sending step for sending
A data stream transmission step of transmitting a data stream including data used in the application;
A transmission method.
An application including a first description part describing a procedure for obtaining data from a data stream and a second description part describing a procedure for performing drawing processing based on data obtained by executing the first description part An application receiver for receiving
A data stream receiver for receiving a data stream including data used in the application;
A processing unit for displaying the application using the received data stream;
A receiving apparatus comprising:
The processor uses the data read from the data stream received according to the description of the first description unit to display an application according to the description of the second description unit;
The receiving device according to claim 10.
The processing unit reads the data in a predetermined format from the received data stream by the object used in the first description unit.
The receiving device according to claim 11.
The processing unit repeatedly performs a drawing process based on the data delivered by the object as the process of the second description unit.
The receiving device according to claim 12.
The data stream receiving unit receives the data for displaying a display object that moves in synchronization with the motion of the broadcast video,
The processing unit displays the display object based on the data read from the data stream in accordance with the description of the first description unit.
The receiving device according to claim 10.
The data stream receiving unit receives the data stream including information on a position and size of the display object on the screen and display data of the display object;
The processing unit displays the display data of the display object based on the position and size read from the data stream according to the description of the first description unit;
The receiving device according to claim 14.
A control information receiving unit for transmitting control information including a time stamp indicating a display time of the data;
The data stream receiving unit intermittently receives the data stream including interpolation information for obtaining an interpolation position of the display object until the next timing,
The processing unit displays a display object at an interpolation position based on the interpolation information other than the display timing.
The receiving device according to claim 14.
The application transmission unit receives the application in an HTML5 document format that enables a function to monitor a change in script variables,
The processing unit displays the application based on the function of the HTML5 browser.
The receiving device according to claim 10.
An application including a first description part describing a procedure for obtaining data from a data stream and a second description part describing a procedure for performing drawing processing based on data obtained by executing the first description part Receiving the application, and
A data stream receiving step for receiving a data stream including data used in the application;
Processing to display the application using the received data stream;
Receiving method.