JP2017188891A - Broadcast service retransmission system and viewing portable terminal - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a broadcast service retransmission technology optimal for a portable terminal.SOLUTION: A broadcast service retransmission server extracts each of a video/voice stream, a data stream synchronized with the video/voice stream, and a data content not synchronized with the video/voice stream, from a broadcast stream. The video/voice stream is transmitted to a terminal, using a HTTP streaming system. The data stream synchronized with the video/voice stream is received by the terminal, using an index file transmitted using HTTP long polling. The data content not synchronized with the video/voice stream is transmitted as with the data stream synchronized with the video/voice stream or transmitted on demand.SELECTED DRAWING: Figure 2

Description

  The present invention relates to a broadcast service retransmission system and a portable terminal for viewing that retransmit a television broadcast using a communication network.

  Services are provided to retransmit digital television broadcasts using a communication network. In this retransmission service, in addition to video / audio broadcasting services, services such as multimedia data broadcasting, subtitles, character supervision, etc. are provided, and the services comply with television receivers. By the way, a streaming service that enables viewing of video / audio using a mobile terminal (for example, a smartphone) is widely provided. In addition to viewing video / audio data, it is desired to use digital television broadcasting services including data broadcasting on mobile terminals.

  Note that IP retransmission of broadcast services is described in Patent Document 1 (Japanese Patent Laid-Open No. 2007-274623) and Patent Document 2 (Japanese Patent Laid-Open No. 2012-231383). Japanese Patent Application Laid-Open No. 2004-228561 discloses appropriately recording a broadcast program retransmitted by IP using a standard time server and error information. Patent Document 2 discloses that synchronization with related information is performed using close caption information in video content data distributed by an IPTV service.

  The above-described prior art and its problems are described only to explain a part of the background of the present invention. It should be noted that the present invention is not limited to the above-described prior art and problems.

JP 2007-274623 A JP 2012-231383 A

  The present invention has been made in view of the above circumstances, and an object thereof is to provide a broadcast service retransmission technique optimized for a mobile terminal.

  According to this invention, in order to achieve the above-mentioned object, the configuration as described in the claims is adopted. Here, prior to explaining the invention in detail, supplementary explanation will be given for the claims.

  That is, according to one aspect of the present invention, a broadcast service retransmission apparatus includes: a video / audio stream transmitting means for transmitting a video / audio stream extracted from a broadcast stream to a terminal via a network in a streaming scheme; Data stream transmitting means for transmitting the data stream extracted from the terminal to the terminal via the network, and further comprising the data stream transmitting means: a data stream storage means for storing the data stream in units of packets An index file storage means for storing an index file that makes the data stream stored in units of packets accessible in units of packets; and the index file is sent via the network in response to polling from the terminal. Means for transmitting to the terminal; the data stream stored in the data stream storage means in packet units in response to a transfer request in packet units from the terminal using the index file transmitted to the terminal; Means for transmitting to the terminal.

  According to this configuration, the data stream can be efficiently distributed independently of the video / audio stream.

  The data content synchronized with the video / audio stream includes subtitle data, text super, and event message. Content data asynchronous to the video / audio stream typically includes a page written in BML and an electronic program guide (EPG).

  The index file includes an entry that enables a data stream stored in units of packets to be extracted in units of packets, and is hereinafter also referred to as message management data, message management file, message, and message list.

  In this configuration, the broadcast service retransmission apparatus may further include means for extracting each of the video / audio stream and the data stream from the broadcast stream.

  In this configuration, the data stream transmitted by the data stream transmitting means may include a data stream synchronized with the video / audio stream.

  Further, the data stream transmitted by the data stream transmission means may or may not include a data stream of data content asynchronous to the video / audio stream. When the data stream transmitted by the data stream transmission means does not include asynchronous data content, the data content is reconstructed from the data stream of the data content asynchronous to the video / audio stream, and the reconstructed data content is In response to a transmission request based on the operation in, it may be transmitted through the network on demand.

  In this configuration, the streaming method is a streaming method compliant with the HTTP protocol. For example, it may be HLS (HTTP Live Streaming, Apple), MPEG-DASH, Smooth Streaming (Microsoft), or HTTP Dynamic Streaming (Adobe Systems). Each method other than HTTP streaming such as RTS (Real Time Streaming Protocol) and RTMP (Real Time Messaging Protocol) may be adopted.

  In this configuration, the polling is HTTP long polling. Other pseudo push transmission systems may be adopted.

  According to another aspect of the present invention, a terminal that can be connected to a network: a video / audio stream that is extracted from a broadcast stream and transmitted via the network in a streaming manner from a transmission device; An audio stream receiving unit; and a data stream extracted from the broadcast stream and transmitted via the network, the data stream receiving unit receiving from the transmitting device, the data stream receiving unit: Index file receiving means for receiving, from polling, an index file that makes the data stream stored in the transmitting device accessible in units of packets; and the data stream stored in the transmitting device The above Using the index file received by DEX file receiving means, and configured to include a means for receiving a packet-by-packet basis.

  Also in this configuration, the data stream can be efficiently distributed independently of the video / audio stream.

  In this configuration, the terminal may be a terminal that performs communication based on mobile communication.

  According to another aspect of the present invention, a terminal that can be connected to a network: a video / audio stream that is extracted from a broadcast stream and transmitted via the network in a streaming manner from a transmission device; A computer program configured to include: an audio stream receiving unit; and a data stream receiving unit that receives a data stream extracted from the broadcast stream and transmitted via the network from the transmitting device. A stream receiving means: an index file receiving means for receiving, from polling, an index file that makes the data stream stored in the transmitting apparatus accessible in units of packets; and stored in the transmitting apparatus Have The data stream using the index file received by the index file receiving means, to provide the computer program configured to include a means for receiving a packet-by-packet basis.

  Also in this configuration, the data stream can be efficiently distributed independently of the video / audio stream.

  According to still another aspect of the present invention, a broadcast service retransmission apparatus includes: a means for transmitting a video / audio stream extracted from a broadcast stream to a terminal via a network in a streaming manner; Means for transmitting a data stream synchronized with the video / audio stream to the terminal via the network in response to polling from the terminal; asynchronous with the video / audio stream extracted from the broadcast stream; Means for transmitting data content via the network in response to a transmission request based on an operation at the terminal.

  In this configuration, since the broadcast content is transmitted by a method optimized according to the type of the broadcast content without being constrained by the basic broadcast system, the processing load of the broadcast service retransmission apparatus can be reduced. In addition, the load on the viewing terminal for the retransmission service can be reduced. This retransmission system may obtain video / audio streams, synchronized data streams, and asynchronous data content from one or more broadcaster facilities, and the playback system itself extracts these from the broadcast streams. May be.

  As described above, the data content synchronized with the video / audio stream includes caption data, character superimpose, and an event message. Content data asynchronous to the video / audio stream typically includes a page written in BML and an electronic program guide (EPG).

  In this configuration, the broadcast service retransmission apparatus includes means for extracting a broadcast stream from a video / audio stream, a data stream synchronized with the video / audio stream, and data content asynchronous with the video / audio stream, respectively. It may be configured to include.

  According to another aspect of the present invention, a terminal connectable to a network: means for receiving a video / audio stream extracted from a broadcast stream and transmitted via the network in a streaming manner; from the broadcast stream; Means for receiving a data stream synchronized with the video / audio stream extracted and transmitted in response to polling from the terminal via the network; and extracted from the broadcast stream and via the network, Means for receiving the video / audio stream and asynchronous data content transmitted in response to a transmission request based on an operation in the terminal.

  In this configuration, the load on the terminal can be reduced.

  In this configuration, the terminal is typically a portable terminal or a mobile communication terminal that performs communication based on mobile communication, but may be a fixed terminal. A portable terminal or mobile communication terminal is typically a mobile phone, but includes a portable computer, a tablet computer, a media player, personal digital assistants, and combinations of two or more of these products. Any device may be used.

  According to another aspect of the present invention, in order to achieve the above-described object, a terminal connectable to a network: a video / audio stream extracted from a broadcast stream and transmitted via the network in a streaming manner Means for receiving; means for receiving a data stream synchronized with the video / audio stream, extracted from the broadcast stream and transmitted in response to polling from the terminal via the network; and from the broadcast stream Provided is a computer program configured to include means for receiving the video / audio stream and asynchronous data content extracted and transmitted in response to a transmission request based on an operation in the terminal via the network To do.

  In this configuration, the load on the terminal can be reduced.

  The present invention can be realized not only as an apparatus or a system but also as a method. Of course, a part of the invention can be configured as software. Of course, software products used to cause a computer to execute such software are also included in the technical scope of the present invention.

  These and other aspects of the invention are set forth in the appended claims and will be described in detail below with reference to examples.

  According to this invention, the processing load of the broadcast service retransmission apparatus can be reduced. In addition, the load on the viewing terminal for the retransmission service can be reduced.

It is a figure explaining the environment where the IP retransmission system of the Example of this invention is performed. It is a functional block diagram explaining the outline | summary of the IP retransmission system of FIG. It is a figure explaining the message management file of FIG. It is a figure explaining caption data. It is a block diagram which shows the structural example of the portable terminal of FIG. It is a functional block diagram of the portable terminal of FIG. It is a figure explaining the example of a display of the portable terminal of FIG. It is a figure which shows the example of the channel information which can be acquired from an authentication server. It is a figure explaining the detail of the channel information of FIG. It is a block diagram which shows the structural example of the synchronous data reproduction | regeneration part of FIG. It is a schematic diagram explaining the operation | movement of the synchronous data reproduction | regeneration part of FIG. FIG. 12 is a schematic diagram for explaining an operation example of obtaining a correction PCR by the synchronous data reproducing unit in FIG. 11. It is a flowchart explaining the button selection operation | movement of the user interface part of FIG. It is a figure which shows the example of a screen display explaining the button selection operation | movement of the user interface part of FIG. It is a figure which shows the example of BML description explaining the button selection operation | movement of the user interface part of FIG. It is a flowchart explaining operation | movement of the two-dimensional barcode process part of the user interface part of FIG. It is a figure which shows the example of a screen display explaining operation | movement of the two-dimensional barcode process part of the user interface part of FIG. It is a functional block diagram explaining the modification of the above-mentioned Example. It is a schematic diagram explaining the processing flow of subtitle data. It is a figure explaining the message in a message management file. It is a figure explaining the entity classification of the message in a message management file.

  Examples of the present invention will be described below.

  FIG. 1 shows an overall environment for implementing an embodiment applied to a digital television retransmission system for performing IP retransmission of digital television broadcasting (terrestrial digital television broadcasting, BS, CS digital television broadcasting) according to the present invention. In this figure, the broadcaster system 100 may be a system operated by a basic broadcaster, obtains a TS stream of a broadcast service before uplink, or demodulates a received broadcast radio wave to obtain a TS stream. Further, it is demultiplexed into a video / audio stream and a data stream, and is transmitted to the distribution provider system 200 using a predetermined line. The distribution company system 200 processes the transmitted video / audio stream and data stream for redistribution. As described later, the video / audio stream (MPEG-2 TS) is used for IP redistribution. Data that is transcoded into a predetermined format (HLS, MPEG-DASH, etc.) and distributed to the mobile terminal 300, and is synchronized with video / audio streams such as subtitle data, character super data, and event messages in the data stream. And other asynchronous data (data broadcasting described in BML, electronic program guide, ie EPG, etc.) are individually managed, the former is distributed using HTTP long polling, and the latter is on demand. It is supposed to be delivered. As will be described later, both synchronous data synchronized with the video / audio stream and other asynchronous data may be distributed using HTTP polling. Distribution company system 200 is connected to communication network 500, typically the Internet. At least a part of the communication network 500 may be constituted by a CDN (content delivery network) service. The portable terminal 300 is connected to the communication network 500 via the gateway 400 of the mobile communication carrier. Further, the authentication server 600 may be connected to the communication network 500 to authenticate a request from the mobile terminal 300 regarding the IP retransmission service. The broadcast provider system 100 and the distribution provider system 200 may be integrated.

  FIG. 2 illustrates configuration examples of the broadcast provider system 100, the distribution provider system 200, and the mobile terminal 300, and other configuration examples may be employed. In FIG. 2, a broadcaster system 100 includes a broadcast video / audio extraction unit 110 and a broadcast data extraction unit 120. The broadcast video / audio extraction unit 110 receives video / audio TS packets from a broadcast TS stream. Extract. The broadcast data extraction unit 120 extracts data-related TS packets other than video / audio from the broadcast TS stream. Distribution company system 200 includes video / audio transcoding unit 210, data receiving unit 220, data management DB unit 230, video / audio distribution unit 240, synchronous data distribution unit 250, asynchronous data distribution unit 260, and message management file holding. Part 270 and the like. The video / audio transcoding unit 210 converts video / audio MPEG-2 TS packets into a streaming file of an HTTP streaming method (typically an HLS method). The video / audio distribution unit 240 distributes the video / audio stream to the mobile terminal 300 using the HTTP streaming method. The data receiving unit 220 typically receives data-related TS packets other than video / audio, and synchronizes synchronous data synchronized with video / audio streams such as subtitle data, character super data, and event messages. The synchronization data distribution unit 250 distributes the synchronization data to the mobile terminal 300 by HTTP long polling. In addition, the data receiving unit 220 reconstructs the BML document and EPG from the asynchronous data packet and stores them in the data management DB 230, and the asynchronous data distribution unit 260 turns on these BML document and EPG from the portable terminal 300. Delivery to the mobile terminal 300 based on an HTTP request for demand, for example, an HTTP request made by operating a default button display displayed on the display unit of the mobile terminal 300.

  The broadcast provider system 100 or the distribution provider system 200 sends an index file (also referred to as index data, index list, message management data, message management file, or message list) for managing TS packets of a broadcast TS stream as a message. It is held in the message management file holding unit 270 as a management file. The broadcaster system 100 or the distribution provider system 200 creates and manages an index (message entry, message) shown in FIG. 3 every time a TS packet is received. With respect to the caption data, the caption packet as schematically shown in FIG. 4 is retained in the data storage management unit, for example, the data management DB 230, the data main body storage management unit 280 of FIG. It can be accessed with the path specified by the message's entity file name attribute.

  The mobile terminal 300 displays video distributed by HTTP streaming on the display unit 300A (FIG. 2), and subtitle data (300B) distributed using a message management file transmitted by HTTP long polling. ) On the display unit 300A and displayed on the display unit 300A, and a BML document or EPG distributed on demand is displayed on the display unit 300C. Note that the display unit 300 </ b> A and the display unit 300 </ b> C are partitioned according to the layout, and may be a part of the display area of the mobile terminal 300 or may be the whole. Display unit 300A may be displayed over the entire display area based on selectable display modes. Further, based on selectable display modes, the caption data and / or the character super data may be displayed in different areas (windows) that do not overlap with or partially overlap the display unit 300A.

  An example of HTTP long polling will be described later.

  FIG. 5 shows a configuration example of the mobile terminal 300. In this figure, the mobile terminal 300 includes a processing system 310, a memory 320, an input / output subsystem 330, a high-frequency circuit unit 340, and an audio circuit unit 350. Each of the components can be realized as appropriate by hardware including a signal processing integrated circuit and an application specific integrated circuit, software, or a combination thereof. The high-frequency circuit unit 340 is used to transmit / receive information to / from other devices over a wireless link or network, and to perform this function, an antenna system, an RF transceiver, an amplifier, a tuner, an oscillator, Including, but not limited to, digital signal processors, CODEC chip sets, memories, and other known circuit configurations. The high-frequency circuit unit 340 includes mobile communication such as time division multiple access (TDMA) and code division multiple access (CDMA), wireless LAN (IEEE 802.11b, IEEE 802.11a, IEEE 802.11g and / or IEEE 802.11N), Bluetooth, and the like. (IEEE802.15.1, “Bluetooth” is a trademark), communication protocols such as Wi-MAX (IEEE802.16) can be used, but are not limited thereto. The audio circuit unit 350 is connected to the audio speaker 351 and the microphone 352. The processing system 310 includes a processor 311, a memory controller 312, and a peripheral device interface 313, and the memory 320 stores codes and data executed by the processor 311. The memory 320 may include any storage device such as RAM, ROM, FLASH, disk drive, magnetic tape and the like. The processor 311 executes various software components stored in the memory 320. For example, the software components include an operating system 321, a communication module 322, a contact / operation module 323, a graphics module 324, an application 325, a timer module 326, and a web browser module 327. The application 325 includes a location determination function such as a browser, an address book, email, encryption, a global positioning system (GPS), and a two-dimensional barcode decoder. The Internet retransmission television broadcast viewing application 325A (FIG. 6) of this embodiment is typically realized as one of the applications 325. The input / output subsystem 330 controls the touch panel 331 and the like. In this portable terminal 300, a function for viewing a television broadcast service distributed by Internet retransmission is implemented as an application. By installing a corresponding application in the memory 320, the mobile terminal 300 is realized as an Internet retransmitted television broadcast viewing terminal.

  FIG. 6 schematically shows functions realized by the Internet retransmission television broadcast viewing application 325A. In the figure, the mobile terminal 300 includes a video / audio stream reproduction unit 360, a synchronous data reproduction unit 370, an asynchronous data browsing unit. 380 includes a user interface unit 390 as a function. The user interface unit 390 includes a button selection unit 391 and a barcode processing unit 392.

  The video / audio stream reproduction unit 360 decodes the video / audio packet file distributed from the video / audio distribution unit 240 of the distribution company system 200 by HTTP streaming, and, for example, as shown in FIG. The moving image is displayed in the video viewing area A of the display area. Note that in order to start viewing a television broadcast, a corresponding application may be selected and activated, and a channel may be selected from an on-screen display or the like similar to that of a normal television receiver. Channel switching may be performed based on swipe. In some cases, an authentication dialog is executed with the authentication server 600 (FIG. 1). Note that the application 325A can acquire channel information for each broadcast channel that can be viewed from the authentication server 600, for example, and FIG. 8 shows a specific example of this channel information. The channel information includes information as shown in FIG. In FIG. 7, the “d” button is used for switching data broadcasting, the “character” button is used for switching subtitles, the “main” button is used for switching audio, and there are other buttons for switching the display vertically and horizontally. In addition, in connection with subtitle display, buttons for displaying subtitles, etc. in different areas, and buttons for advancing or delaying the timing of subtitles, etc. are provided so that subtitle display mode and timing can be finely adjusted. Also good.

  Further, in FIG. 6, a synchronization data reproduction unit 370 reproduces synchronization data such as captions. The synchronous data reproduction unit 370 acquires the synchronous data from the synchronous data distribution unit 250 of the distribution company system 200 using HTTP long polling, and displays subtitles, character supermarkets, etc. at the timing synchronized with the video / audio streaming. It is supposed to be. The synchronous data reproduction unit 370 processes an event message that is also synchronous data at a timing synchronized with video / audio streaming. The BML document content can be switched by the event message.

  FIG. 10 illustrates a corrected PCR generation unit 372 that controls the synchronization timing in the synchronization data reproduction unit 370. In FIG. 10, the synchronization data reproduction unit 370 includes a caption display unit 371, a corrected PCR generation unit 372, a message management file holding unit 373, a caption data holding unit 374, and a polling control unit 375. The corrected PCR generation unit 372 generates a PCR (program clock reference) corresponding to the playback status of the video / audio stream playback unit 360. The polling control unit 375 polls the synchronous distribution unit 250 of the distribution provider system 200, takes out a new entry (message) from the message management file holding unit 270, and stores the message management file holding unit of the synchronous data reproduction unit 370. In addition, new subtitle data is added to the subtitle data holding unit 374. The message management file holding unit 373 and the caption data holding unit 374 are configured to delete entries that are no longer needed. The message management file holding unit 270 of the distribution company system 200 also deletes entries that are no longer needed. The subtitle display unit 371 refers to the display timing (PST) of the subtitle packet in the subtitle data holding unit 374 based on the correction PCR calculated by the correction PCR generation unit 372 and starts displaying subtitles (same for character superimpose). Then, the subtitle display is terminated with reference to the erase timing.

  The correction PCR generation unit 372 includes a date and time PRSTS (date and time compliant with the time of the server that creates the message management file) when the program switching (change in the stream configuration of the broadcast program in one channel) is detected, and playback The difference between the recorded video / audio stream date and time (date and time on the playback time axis; date and time that conforms to the time of the server that performs TS transcoding) is calculated, and the elapsed time from the program switching time to the date and time on the playback time axis ( The elapsed time of PCR) is obtained, and the resolution of the program reference clock (for example, 90000 / second) is multiplied by this to obtain the elapsed clock. Furthermore, the correction PCR generation unit 372 holds the PCR (PCRP) when the program switching is detected, and calculates the correction PCR by adding the elapsed clock with reference to this. Such a calculation method is shown in FIG. A specific calculation example is shown in FIG. The upper limit of PCR is “85889935102”. Here, the exact time of the video is set to EXT-PROGRAM-DATE-TIME, and preferably the time of the server that creates the HLS and TS transcodes and the server that creates the message management file is in units of one frame. It is assumed that there is no deviation. For example, the program switching (the stream configuration of the broadcast program in one channel has changed) is detected by the portable terminal 300 detecting that the value of the version number of the PMT (program map table) packet has changed. Detect. In a typical example, “pmt_changed” (see FIGS. 21 and 20 described later) is a message notifying that the PMT data has changed, and the detection time of the server is transferred to the message execution time in this message. Is done. The server also receives the PMT data extracted from the tuner, acquires the PCR from the PCR description location described in the PMT, and transcribes it as additional information in the fourth column of the “pmt_changed” message. When the PMT indicates program switching (the broadcast program stream configuration in one channel has changed), the previous message management data may be cleared. FIG. 12 schematically shows an example of the operation for calculating the correction PCR.

  In addition, the difference between the PTS (Presentation Time Stamp) and the real time conversion place of the PCR is obtained, and the real time is converted by adding this difference to the time when the PCR is received, and this is output to the message management file. The display / erase of subtitles may be controlled in real time instead of based on the clock. In addition, when a button for advancing or delaying the timing of subtitles or the like is provided so that the subtitle display mode and timing can be finely adjusted, for example, every time the “+” button is operated, A constant clock increment (ΔPCR, for example, 90000 clocks per second) corresponding to a certain time increment is added to the value, and each time the “−” button is operated, a constant time is added to the current value of the correction PCR. A fixed clock decrement (−ΔPCR) corresponding to the decrement may be added (subtracted), and when the “reset” button is operated, the original correction PCR may be restored. Not only the correction PCR but also the reference PCR set by a predetermined reference may be increased or decreased in the same manner.

  Next, the button selection unit 391 of the user interface unit 390 in FIG. 6 will be described. FIG. 14 shows an example of a television broadcast display, except for the video part, which is a document described in BML. “A”, “B”, and “C” are operation buttons. In relation to a normal television receiver, the focus position is changed from the button to the button by using the cursor movement key of the remote controller, and finally the remote controller is used. Press the OK button to select an item. A key corresponding to the cursor movement key of the remote controller may be provided in the user interface unit 390 of the portable terminal 300. In this embodiment, the item is selected when the target button is touched. This will be described in detail below.

  FIG. 15 shows the BML document description of the display example of FIG. As can be seen, the description of the button element includes a style attribute of “nav-index”. That is, in the example of FIG. 14, the description for displaying the buttons shown in (1), (2), and (3) includes the “nav-index” style attribute enclosed in (1) of FIG. In this example, when the style attribute of “nav-index” is included in the description of the touched area, it is determined that it can be selected, and the content is selected.

  FIG. 13 shows an operation example of the button selection unit 391. As shown in FIG. 13, when a touch is detected, it is determined whether or not an element is detected in the area (step S01). If no element is detected, the process ends as it is (an on-screen display may be displayed). ), If it is detected, it is determined whether or not a focus movement description (“nav-index” style attribute description) is included (step S02). If it is not detected, the process ends. Executes the instruction in the button description (step S03). After that, the button selection process is terminated.

  Next, the barcode processing unit 392 of the user interface unit 390 in FIG. 6 will be described. FIG. 17 shows a state in which a two-dimensional barcode image is displayed on the display unit of the mobile terminal 300. In a normal television receiver usage mode, a two-dimensional barcode image displayed on the screen of the television receiver is captured by a camera function of a mobile terminal or the like, and this is decoded by a barcode decoding function to obtain a URL. Then, the browser is activated based on the acquired URL, and the content of the URL is browsed. In this embodiment, by touching the image display part (part (1) in FIG. 17) of the two-dimensional barcode on the display unit of the portable terminal 300, the URL content can be browsed as it is. Yes.

  As shown in FIG. 16, when a touch is detected, it is determined whether or not an element is detected in the area (step S11), and if it is not detected, the process is terminated (an on-screen display may be displayed). If detected, the element image is extracted and decoded by the two-dimensional decoding function, and it is determined whether or not the decoding is successful (step S12). If the decoding is unsuccessful, the process ends as it is. If the decoding is successful, the contents can be browsed by the browser using the decoded contents (URL). Thereafter, the barcode processing is terminated.

  Of course, when the display unit is touched, the button selection process of FIG. 13 and the barcode process of FIG.

  This is the end of the description of the embodiment.

  Next, a modification of the above embodiment will be described.

  FIG. 18 corresponds to FIG. 2, and corresponding portions are denoted by corresponding reference numerals. In FIG. 18, not only synchronous data but also asynchronous data (BML document) is distributed to the mobile terminal 300 by long polling. That is, the data receiving unit 220 receives TS packets related to data other than video and audio, generates a processing management message for each TS packet, holds the message in the message management file holding unit 270, and stores the TS packet body. The data body storage management unit 280 stores the address that can be specified by the message. The data distribution unit 250A uses a message management file (also referred to as message management data) to synchronize data synchronized with video / audio streams such as subtitle data, character super data, and event messages, BML documents, and EPGs. Asynchronous data is distributed to the portable terminal 300 by HTTP long polling.

  FIG. 19 illustrates an example of a processing flow using subtitle data as an example. In the modification of FIG. 18, other data-related TS packets are processed in the same manner. In FIG. 19, the tuner of the broadcast wave receiving apparatus 100A of the broadcaster system 100 receives TS packets and supplies the TS packets to the TS separation program 100B of the broadcaster system 100 (X01). The TS packet is transmitted from the data distribution server 200A of the distribution company system 200 (X02). The data distribution server 200A stores the subtitle data in the data body storage management unit 280, and in response to this, adds a new message to the message management data (message management file) in the message management file holding unit 270, Update (X03). The application 325A of the portable terminal 300 first retrieves the message management data in the long polling processing part X04 (X05), and in response to this, the data distribution server 200A returns the latest data of the message management data to the application 325A. (X06). The data distribution server 200A repeats returning the updated message management data to the application 325A each time it is updated (X07). The application 325A interprets the message management data acquired in steps X06 and X07 for each line, and if it is a caption message, the application 325A retrieves the caption data acquired in step X03 (X08), and the data distribution server 200A responds accordingly. The subtitle data is sent to the application 325A (X09). The application 325A performs display processing at an appropriate timing from the contents of the message management data and the caption data, and the caption is displayed to the user (X10, X11).

  FIG. 20 shows an example of a message. As described above with reference to FIG. 3, the message is the message execution time in the first column (also called the firing time), the message type (substance in the second column) It includes the entity file name in the third column and additional information in the fourth column (also referred to as type) (FIG. 20A). In the specific example of FIG. 20B, the message execution time is “2016-04-20T13: 02: 15.588 + 0900”, the message type is “cap_packet”, and the entity file name is “/ tvmock / 20160420 / 7fe4”. .7fe4.0420 / cap / 00055002 "and no additional information is included. The message type “cap_packet” indicates that the message relates to subtitles, and that the subtitle data is stored in the entity file name “/tvmock/20160420/7fe4.7fe4.0420/cap/00055002” . The main message types are as shown in FIG. Although the message is formed for each TS packet of the data stream, the TS packet of the video / audio stream may be generated and used for management.

  “Pmt_changed” in FIG. 21 is used for controlling the display timing of caption data as described above.

  In addition, this invention is determined based on description of a claim, and is not limited to the specific structure of the Example, a subject, and an effect. The present invention is not limited to the above-described embodiments, and various modifications can be made without departing from the spirit of the present invention.

  For example, although the above embodiment has been described in relation to a mobile terminal, the present invention may be widely applied to a fixed terminal and a television receiver. For example, a button selection operation and a barcode process are provided with a touch panel. The present invention can also be applied to a fixed terminal or a television receiver with a touch panel connected to the Internet.

The technical features described here are listed below.
[Broadcast retransmission method]
[Technical features 1]
A video / audio stream transmitting means for transmitting a video / audio stream extracted from the broadcast stream to the terminal via a network in a streaming manner;
Data stream transmission means for transmitting the data stream extracted from the broadcast stream to the terminal via the network,
The data stream transmission means includes:
Data stream storage means for storing the data stream in units of packets;
Index file storage means for storing an index file that makes the data stream stored in packet units accessible in packet units;
Means for transmitting the index file to the terminal via the network in response to polling from the terminal;
Means for transmitting the data stream stored in the data stream storage means to the terminal in packet units in response to a transfer request in packet units from the terminal using the index file transmitted to the terminal; A broadcast service retransmission apparatus comprising:
[Technical feature 2]
The broadcast service retransmission apparatus according to Technical Feature 1, further comprising means for extracting a video / audio stream and a data stream, respectively, from the broadcast stream.
[Technical feature 3]
3. The broadcast service retransmission apparatus according to technical feature 1 or 2, wherein the data stream transmitted by the data stream transmission means is a data stream synchronized with the video / audio stream.
[Technical feature 4]
4. The broadcast service retransmission apparatus according to technical feature 1, 2, or 3, wherein the data stream transmitted by the data stream transmission means is data content asynchronous with the video / audio stream.
[Technical feature 5]
The broadcast service retransmission apparatus according to any one of the technical features 1 to 4, wherein the streaming method is a streaming method compliant with the HTTP protocol.
[Technical feature 6]
The broadcast service retransmission apparatus according to any one of the technical features 1 to 5, wherein the polling is HTTP long polling.
[Technical feature 7]
A terminal that can be connected to a network,
A video / audio stream receiving means for receiving a video / audio stream extracted from the broadcast stream and transmitted via the network in a streaming manner from a transmission device;
A data stream receiving means for receiving a data stream extracted from the broadcast stream and transmitted via the network from the transmitting device;
The data stream receiving means includes
Index file receiving means for receiving, from polling, an index file that makes the data stream stored in the transmitting device accessible in units of packets;
And a means for receiving the data stream stored in the transmitting apparatus in units of packets using the index file received by the index file receiving means.
[Technical feature 8]
The terminal according to Technical Feature 8, which performs communication based on mobile communication.
[Technical feature 9]
A device that can be connected to the network
Video / audio stream receiving means for receiving, from a transmission device, a video / audio stream extracted from a broadcast stream and transmitted via the network in a streaming manner; and
A computer program used to cause a data stream extracted from the broadcast stream and transmitted via the network to be executed as data stream receiving means for receiving from the transmission device,
The data stream receiving means includes
Index file receiving means for receiving, from polling, an index file that makes the data stream stored in the transmitting device accessible in units of packets;
The computer program comprising: means for receiving the data stream stored in the transmitting device in units of packets using the index file received by the index file receiving means.
[Element selection method]
[Technical features 1]
A display for displaying a document written in a markup language;
A touch panel corresponding to the display unit;
A document browsing apparatus comprising: means for executing a command relating to the display element when there is an instruction relating to focus movement in the description of the document relating to the display element at a position corresponding to a touch operation on the touch panel.
[Technical feature 2]
The document browsing apparatus according to the technical feature 1, wherein the instruction relating to the focus movement corresponds to a cursor movement key of a remote controller.
[Technical feature 3]
The document browsing apparatus according to any one of the technical features 1 and 2, wherein the markup language is BML.
[Technical feature 4]
A mobile terminal having a display unit for displaying a document described in a markup language and a touch panel corresponding to the display unit is related to focus movement in the description of the document regarding a display element at a position corresponding to a touch operation on the touch panel. A computer program used for realizing as a means for executing an instruction related to the display element when there is an instruction.
[Barcode processing method]
[Technical features 1]
A display unit;
A touch panel corresponding to the display unit;
Decoding means for decoding the barcode;
A display device comprising: means for extracting an image at a position corresponding to a touch operation on the touch panel and decoding the image by the decoding means.
[Technical feature 2]
A display unit;
A touch panel corresponding to the display unit;
A display device comprising: means for extracting an image at a position corresponding to a touch operation on the touch panel and causing the barcode decoding means to decode the image.
[Technical feature 3]
3. The display device according to technical feature 1 or 2, wherein the barcode is a two-dimensional barcode.
[Technical feature 4]
4. The display device according to any one of the technical features 1, 2, or 3 that executes processing based on the decoding result when decoding by the decoding unit is successful.
[Technical feature 5]
The display device according to any one of technical features 1 to 4, wherein the decryption result is a URL, and the URL request is transmitted.
[Technical feature 6]
The display device according to any one of the technical features 1 to 5 configured by a mobile communication terminal.
[Technical feature 7]
The display device according to any one of technical features 1 to 6 for displaying a document described in BML.
[Technical feature 8]
A portable terminal having a display unit and a touch panel corresponding to the display unit,
The computer program used in order to implement | achieve as a means to extract the image of the position corresponding to the touch operation with respect to the said touch panel, and to make a barcode decoding means decode.
[Synchronization method]
[Technical features 1]
A terminal that can be connected to a broadcast service retransmission apparatus via a network,
Means for receiving a video / audio stream extracted from the broadcast stream and transmitted via the network from the broadcast service retransmission apparatus in a first scheme;
Means for reproducing the video / audio stream;
Means for obtaining a date and time (playback time axis date and time) compliant with the broadcast service retransmission device at the time of the current playback of the video and audio stream being played back;
When the program map table (PMT) indicates that the stream configuration of the broadcast program in the channel has changed, the program reference clock (PCRP) acquired from the description of the PMT and the broadcast service transmitting apparatus that processes the PMT Means for obtaining a date and time (PRSTS) compliant with
Means for receiving a data stream synchronized with the video / audio stream extracted from the broadcast stream and transmitted via the network in a second manner;
Means for calculating the number of clocks passed between the PRSTS and the playback time axis date and time, and adding the calculated clock number to the PCRP to predict a program clock reference of the playback time axis date and time; ,
And a processing execution unit configured to execute processing of a data stream synchronized with the video / audio stream based on the predicted program clock reference.
[Technical feature 2]
The terminal according to technical feature 1, wherein a data stream synchronized with the video / audio stream includes subtitle data, and the processing execution means reproduces the subtitle data.
[Technical feature 3]
The terminal according to technical feature 1 or 2, wherein the data stream synchronized with the video / audio stream includes an event message, and the processing execution means processes the event message.
[Technical feature 4]
The terminal according to any one of technical features 1 to 3, wherein the video / audio stream is an HTTP live streaming stream.
[Technical feature 5]
5. The terminal according to technical feature 4, wherein the playback time axis date and time is calculated by HPDT + MCT from EXT-PROGRAM-DATE-TIME (HPDT) and playback elapsed time (MCT) of the HLS file forming the video / audio stream.
[Technical feature 6]
When the resolution of the program clock reference is FHz, the program clock reference of the playback time axis date and time is
PCRP + (HPDT + MCT-PRSTS) × F
The terminal according to technical feature 5, which is calculated by:
[Technical feature 7]
The terminal according to technical feature 6, wherein F is 90000.
[Technical feature 8]
A terminal that can be connected to the broadcast service retransmission device via a network.
Means for receiving a video / audio stream extracted from the broadcast stream and transmitted via the network from the broadcast service retransmission apparatus in a first scheme;
Means for reproducing the video / audio stream;
Means for obtaining a date and time (playback time axis date and time) compliant with the broadcast service retransmission device at the time of the current playback of the video and audio stream being played back;
When the program map table (PMT) indicates that the stream configuration of the broadcast program in the channel has changed, the program reference clock (PCRP) acquired from the description of the PMT and the broadcast service transmitting apparatus that processes the PMT Means for obtaining a date and time (PRSTS) compliant with
Means for receiving a data stream synchronized with the video / audio stream extracted from the broadcast stream and transmitted via the network in a second manner;
Means for calculating the number of clocks passed between the PRSTS and the playback time axis date and time, and adding the calculated clock number to the PCRP to predict a program clock reference of the playback time axis date and time; ,
A computer program used for realizing processing execution means for executing processing of a data stream synchronized with the video / audio stream based on the predicted program clock reference.
[Technical feature 9]
A terminal that can be connected to a broadcast service retransmission apparatus via a network,
Means for receiving a video / audio stream extracted from the broadcast stream and transmitted via the network from the broadcast service retransmission apparatus in a first scheme;
Means for reproducing the video / audio stream;
Means for obtaining a date and time (playback time axis date and time) compliant with the broadcast service retransmission device at the time of the current playback of the video and audio stream being played back;
Means for acquiring a program reference clock (PCRP) associated with the predetermined event and a date and time (PRSTS) compliant with the broadcast service transmitting apparatus in which the predetermined event is processed;
Means for receiving a data stream synchronized with the video / audio stream extracted from the broadcast stream and transmitted via the network in a second manner;
Means for calculating the number of clocks passed between the PRSTS and the playback time axis date and time, and adding the calculated clock number to the PCRP to predict a program clock reference of the playback time axis date and time; ,
And a processing execution unit configured to execute processing of a data stream synchronized with the video / audio stream based on the predicted program clock reference.

100 Broadcaster System 110 Broadcast Video / Audio Extraction Unit 120 Broadcast Data Extraction Unit 200 Distribution Business System 210 Video / Audio Transcoding Unit 220 Data Receiving Unit 230 Data Management DB Unit 240 Video / Audio Distribution Unit 250 Synchronous Data Distribution Unit 260 Asynchronous data distribution unit 270 Message management file holding unit 280 Data body storage management unit 300 Mobile terminal 310 Processing system 320 Memory 325 Application 360 Video / audio stream reproduction unit 370 Synchronous data reproduction unit 371 Subtitle display unit 372 Correction PCR generation unit 373 Message management File holding unit 374 Subtitle data holding unit 375 Polling control unit 380 Asynchronous data browsing unit 390 User interface unit 391 Button selection unit 392 Barcode processing unit

Claims (13)

A video / audio stream transmitting means for transmitting a video / audio stream extracted from the broadcast stream to the terminal via a network in a streaming manner;
Data stream transmission means for transmitting the data stream extracted from the broadcast stream to the terminal via the network,
The data stream transmission means includes:
Data stream storage means for storing the data stream in units of packets;
Index file storage means for storing an index file that makes the data stream stored in packet units accessible in packet units;
Means for transmitting the index file to the terminal via the network in response to polling from the terminal;
Means for transmitting the data stream stored in the data stream storage means to the terminal in packet units in response to a transfer request in packet units from the terminal using the index file transmitted to the terminal; A broadcast service retransmission apparatus comprising:   The broadcast service retransmission apparatus according to claim 1, further comprising means for extracting a video / audio stream and a data stream from the broadcast stream.   3. The broadcast service retransmission apparatus according to claim 1, wherein the data stream transmitted by the data stream transmission means includes a data stream synchronized with the video / audio stream.   4. The broadcast service retransmission apparatus according to claim 1, wherein the data stream transmitted by the data stream transmission means includes data content asynchronous with the video / audio stream.   The broadcast service retransmission apparatus according to any one of claims 1 to 4, wherein the streaming system is a streaming system conforming to an HTTP protocol.   The broadcast service retransmission apparatus according to claim 1, wherein the polling is HTTP long polling. A terminal that can be connected to a network,
A video / audio stream receiving means for receiving a video / audio stream extracted from the broadcast stream and transmitted via the network in a streaming manner from a transmission device;
A data stream receiving means for receiving a data stream extracted from the broadcast stream and transmitted via the network from the transmitting device;
The data stream receiving means includes
Index file receiving means for receiving, from polling, an index file that makes the data stream stored in the transmitting device accessible in units of packets;
And a means for receiving the data stream stored in the transmitting apparatus in units of packets using the index file received by the index file receiving means.   The terminal according to claim 8, wherein communication is performed based on mobile communication. A device that can be connected to the network
Video / audio stream receiving means for receiving, from a transmission device, a video / audio stream extracted from a broadcast stream and transmitted via the network in a streaming manner; and
A computer program used to cause a data stream extracted from the broadcast stream and transmitted via the network to be executed as data stream receiving means for receiving from the transmission device,
The data stream receiving means includes
Index file receiving means for receiving, from polling, an index file that makes the data stream stored in the transmitting device accessible in units of packets;
The computer program comprising: means for receiving the data stream stored in the transmitting device in units of packets using the index file received by the index file receiving means. Means for transmitting a video / audio stream extracted from a broadcast stream to a terminal via a network in a streaming manner;
Means for transmitting a data stream extracted from the broadcast stream and synchronized with the video / audio stream to the terminal via the network in response to polling from the terminal;
A broadcast service retransmission comprising: means for transmitting data content asynchronous with the video / audio stream extracted from the broadcast stream via the network in response to a transmission request based on an operation at the terminal; apparatus.   11. The broadcast service retransmission apparatus according to claim 10, further comprising means for extracting a broadcast stream from a video / audio stream, a data stream synchronized with the video / audio stream, and data content asynchronous to the video / audio stream. A terminal that can be connected to a network,
Means for receiving a video / audio stream extracted from the broadcast stream and transmitted via the network in a streaming manner;
Means for receiving a data stream synchronized with the video / audio stream extracted from the broadcast stream and transmitted in response to polling from the terminal via the network;
A terminal for receiving data content asynchronous with the video / audio stream, which is extracted from the broadcast stream and transmitted in response to a transmission request based on an operation in the terminal via the network; . A device that can be connected to the network
Means for receiving a video / audio stream extracted from the broadcast stream and transmitted via the network in a streaming manner;
Means for receiving a data stream synchronized with the video / audio stream extracted from the broadcast stream and transmitted in response to polling from the terminal via the network;
Used as a means for receiving data content that is extracted from the broadcast stream and transmitted via the network in response to a transmission request based on an operation at the terminal, asynchronous with the video / audio stream. Computer program.

Images