JP6596363B2 - Time mapping information generation apparatus, synchronized playback system, time mapping information generation method, and time mapping information generation program - Google Patents

Description

  The present invention relates to a synchronized playback system that synchronizes broadcasting and communication.

  Conventionally, multimedia contents composed of multiple components such as video, audio, and subtitles are distributed via multiple different paths for each component, and the playback timing between the components is presented in synchronization by one or multiple receivers. System is used.

  In such a system, in order to synchronize the reproduction timing between components at the receiver, a common time stamp is added to each component on the transmission side. This time stamp is added to each component distribution unit (distribution unit such as a video / audio frame), and indicates the time at which data of this distribution unit should be presented by the receiver.

  In addition, when a plurality of components have different transmission paths, the time required to reach the receiver differs. For this reason, a technique has been proposed in which the receiver synchronizes the presentation timing by buffering or skipping components based on the above-described time stamp (see, for example, Patent Documents 1 to 3).

JP 2012-10009 A JP 2012-205075 A JP, 2015-207873, A

However, when a recorded program is distributed by video on demand (VOD), this communication stream may have a time system different from that of broadcasting.
For example, MPEG-DASH (Dynamic Adaptive Streaming over HTTP) handles time information in relative time in milliseconds, where the content start time is 0 milliseconds. On the other hand, in broadcasting, PCR (Program Clock Reference) which is a clock value of 27 MHz is used as a time system.
As described above, when the time systems of the broadcast stream and the communication stream are different from each other, it is difficult to reproduce them in synchronization with each other.

  An object of the present invention is to provide a time mapping information generation device, a synchronized playback system, a time mapping information generation method, and a time mapping information generation program for synchronizing contents having different time systems.

  The time mapping information generating device according to the present invention includes a time information acquisition unit that acquires reference time information at a specific time in a time system used in communication content synchronized with broadcasting, and a broadcast corresponding to the reference time information. A clock value acquisition unit that acquires a reference clock value in the time system used in the broadcast stream, and an output unit that outputs mapping information in which the reference time information and the reference clock value are associated with each other.

  The time information acquisition unit acquires a specific type of packet including the reference time information from a broadcast stream, and the clock value acquisition unit acquires a packet including a clock value transmitted before and after the specific type of packet. The reference clock value may be calculated by linear interpolation.

  The time information acquisition unit acquires a specific type of packet including program information from a broadcast stream, extracts a start time of the switched program as the reference time information at the time of program switching, and the clock value acquisition unit May acquire a packet including a clock value transmitted before and after the specific type of packet, and calculate the reference clock value by linear interpolation.

  The time information acquisition unit receives an input of the reference time information in which a specific image appears in a broadcast stream, and the clock value acquisition unit determines a frame in which the specific image appears and is given to the frame The reference clock value may be calculated based on the time stamp.

  A synchronized playback system according to the present invention is a synchronized playback system including the time mapping information generation device and a receiver that receives broadcast and communication content, the receiver from the time mapping information generation device. Based on the acquired mapping information, the broadcast clock value and the time information of the communication content are associated and reproduced.

  The time mapping information generation method according to the present invention includes a time information acquisition step of acquiring reference time information at a specific time in a time system used in communication content synchronized with broadcast, and a broadcast corresponding to the reference time information. The computer executes a clock value acquisition step of acquiring a reference clock value in the time system used in the broadcast stream from a broadcast stream, and an output step of outputting mapping information in which the reference time information and the reference clock value are associated with each other.

  The time mapping information generation program according to the present invention includes a time information acquisition step of acquiring reference time information at a specific time in a time system used in communication content synchronized with broadcasting, and a broadcast corresponding to the reference time information. The computer executes a clock value acquisition step of acquiring a reference clock value in the time system used in the broadcast stream, and an output step of outputting mapping information in which the reference time information and the reference clock value are associated with each other.

  According to the present invention, contents having different time systems can be synchronized.

It is a figure which shows the function structure of the time mapping information generation apparatus which concerns on 1st Embodiment. It is a conceptual diagram which illustrates the calculation method of the reference clock value which concerns on 1st Embodiment. It is a figure which shows the function structure of the time mapping information generation apparatus which concerns on 2nd Embodiment. It is a figure which shows the function structure of the time mapping information generation apparatus which concerns on 3rd Embodiment. It is a figure which shows the structural example of the synchronous reproduction | regeneration system which concerns on 1st-3rd embodiment.

[First Embodiment]
The first embodiment of the present invention will be described below.
FIG. 1 is a diagram illustrating a functional configuration of the time mapping information generation device 1a according to the present embodiment.

  The time mapping information generation device 1a is a server device or an information processing device (computer) such as a PC (Personal Computer) provided with various interfaces such as a communication unit and an input / output device in addition to a storage unit and a control unit. The time mapping information generating device 1a realizes various functions by causing the control unit to execute predetermined software (time mapping information generating program) stored in the storage unit.

  The control unit of the time mapping information generation device 1a includes a packet analysis unit 11a, a JST value acquisition unit 12a (time information acquisition unit), a PCR value acquisition unit 13 (clock value acquisition unit), a mapping information generation unit 14, A mapping information distribution unit 15 (output unit).

  The packet analysis unit 11a sequentially receives TS packets of the broadcast stream and analyzes the packet type to obtain a specific type of packet including time information. Specifically, the packet analysis unit 11a acquires a TOT (Time Offset Table) packet.

  The JST value acquisition unit 12a acquires the JST value (JST_time) included in the TOT packet as reference time information at a specific time point in the time system used for communication content.

  The PCR value acquisition unit 13 acquires a packet including a clock value (PCR value) sent before and after the TOT packet, and obtains a reference clock value in a time system used in broadcasting corresponding to the reference time information by linear interpolation. calculate.

FIG. 2 is a conceptual diagram illustrating a reference clock value calculation method according to this embodiment.
In this example, a TOT packet indicating time “10:00” is acquired, and PCR values “100” and “104” are detected before and after this TOT.

At this time, the PCR value acquisition unit 13 linearly interpolates the PCR value based on the number of packets from the PCR to the TOT, and sets the PCR value corresponding to the acquired TOT reference time information “10:00” as “101”. calculate.
When a fraction is generated by linear interpolation, a value rounded to an integer may be calculated.

  The mapping information generation unit 14 generates and stores mapping information in which reference time information (JST value) in a time system used in communication content is associated with a reference clock value (PCR value) in a time system used in broadcasting. To do.

  The mapping information distribution unit 15 outputs the stored mapping information to an external device, that is, a receiver that synchronously reproduces broadcast content and communication content in response to a request.

According to the present embodiment, the time mapping information generating device 1a uses the TOT packet to correspond to the reference time information (JST value) at a specific time point in the time system used in the communication content and the reference time information. Mapping information in which a reference clock value (PCR value) in a time system used in broadcasting is associated is output.
Therefore, a receiver that reproduces content can synchronize a plurality of contents having different time systems based on the distributed mapping information.

[Second Embodiment]
Hereinafter, a second embodiment of the present invention will be described.
In addition, the same code | symbol is attached | subjected to the structure similar to 1st Embodiment, and description is abbreviate | omitted or simplified.

FIG. 3 is a diagram illustrating a functional configuration of the time mapping information generation device 1b according to the present embodiment.
The time mapping information generation device 1b is an information processing device (computer) similar to the time mapping information generation device 1a of the first embodiment, and controls predetermined software (time mapping information generation program) stored in the storage unit as a control unit. By executing this, various functions are realized.

  The control unit of the time mapping information generation device 1b includes a packet analysis unit 11b, a program switching determination unit 16, a JST value acquisition unit 12b (time information acquisition unit), a PCR value acquisition unit 13 (clock value acquisition unit), A mapping information generation unit 14 and a mapping information distribution unit 15 (output unit) are provided.

  The packet analysis unit 11b sequentially receives TS packets of the broadcast stream and analyzes the packet type to obtain a specific type of packet including program information. Specifically, the packet analysis unit 11b acquires an EIT (Event Information Table) p / f packet.

  The program switching determination unit 16 monitors the event_id of the current program included in the EIT p / f packet, and specifies the EIT p / f packet when the program is switched and updated to the event_id of the next program.

  The JST value acquisition unit 12b uses the start_time (JST value) of the updated program described in the specified EIT p / f packet as reference time information at a specific time in the time system used for communication content. get.

  As in the first embodiment, the PCR value acquisition unit 13 acquires a packet including a clock value (PCR value) sent before and after the EIT p / f packet, and corresponds to the reference time information by linear interpolation. A reference clock value in a time system used in broadcasting is calculated.

  The mapping information generation unit 14 generates and stores mapping information in which reference time information (JST value) in a time system used in communication content is associated with a reference clock value (PCR value) in a time system used in broadcasting. To do.

  The mapping information distribution unit 15 outputs the stored mapping information to an external device, that is, a receiver that synchronously reproduces broadcast content and communication content in response to a request.

According to the present embodiment, the time mapping information generation device 1b uses the EIT p / f packet to reference time information (JST value) at a specific time point in the time system used in the communication content, and the reference time. The mapping information corresponding to the information and the reference clock value (PCR value) in the time system used in broadcasting is output.
Therefore, a receiver that reproduces content can synchronize a plurality of contents having different time systems based on the distributed mapping information.

[Third Embodiment]
Hereinafter, a third embodiment of the present invention will be described.
In addition, the same code | symbol is attached | subjected to the structure similar to 1st Embodiment or 2nd Embodiment, and description is abbreviate | omitted or simplified.

FIG. 4 is a diagram illustrating a functional configuration of the time mapping information generation device 1c according to the present embodiment.
The time mapping information generation device 1c is an information processing device (computer) similar to the time mapping information generation device 1a of the first embodiment, and controls predetermined software (time mapping information generation program) stored in the storage unit. By executing this, various functions are realized.

  The control unit of the time mapping information generation device 1c includes an image decoding unit 17, a correlation determination unit 18, a PTS value acquisition unit 19, a mapping information generation unit 14 (time information acquisition unit and clock value acquisition unit), and mapping information. And a delivery unit 15 (output unit).

  The image decoding unit 17 extracts image data for each frame from the TS packet and decodes it.

The correlation determination unit 18 calculates a correlation value between the decoded image and the determination image, and determines that a specific image has appeared when the correlation value exceeds a predetermined threshold.
Here, the determination image is an image that is determined to appear at a specific time (reference time information) in the broadcast stream, and for example, a call sign for each broadcast station can be employed.

  The PTS value acquisition unit 19 acquires a PTS (Presentation Time Stamp) value assigned to the image frame determined by the correlation determination unit 18.

  The mapping information generation unit 14 receives input of reference time information in which a specific image appears. Further, the mapping information generation unit 14 calculates a PCR value corresponding to the reference time information based on the acquired PTS value. The mapping information generation unit 14 generates and stores mapping information in which reference time information (JST value) in a time system used in communication content is associated with a reference clock value (PCR value) in a time system used in broadcasting. To do.

  The mapping information distribution unit 15 outputs the stored mapping information to an external device, that is, a receiver that synchronously reproduces broadcast content and communication content in response to a request.

According to the present embodiment, the time mapping information generation device 1c specifies a frame presented at the reference time using a determination image in which the time (reference time information) that appears in the broadcast is determined in advance. Extract the time stamp (PTS). The time mapping information generating apparatus 1c converts the time stamp into a reference clock value, thereby corresponding to the reference time information (JST value) at a specific time point in the time system used in the communication content and the reference time information. The mapping information in which the reference clock value (PCR value) in the time system used in broadcasting is associated can be output.
Therefore, a receiver that reproduces content can synchronize a plurality of contents having different time systems based on the distributed mapping information.

  The value of the mapping information generated according to the present embodiment is obtained directly without performing the calculation by linear interpolation as in the first embodiment or the second embodiment. For this reason, it can be expected that the distributed mapping information is more accurate.

[System configuration example]
FIG. 5 is a diagram illustrating a configuration example of the synchronized playback system 100 including the time mapping information generation device 1 (1a, 1b, or 1c) of the first to third embodiments.

  The synchronized playback system 100 includes a time mapping information generation device 1, a broadcast station device 2, a content server 3, a modulator 4, a receiver 5, and a mobile terminal 6.

  The broadcast station apparatus 2 transmits a broadcast signal from the TS output unit 21 by MPEG2-TS. This broadcast signal is branched to the time mapping information generating apparatus 1 and input to the modulator 4.

  In the content server 3, the start time distribution unit 31 distributes the VOD content start time, and the VOD distribution unit 32 distributes the VOD content itself to the receiver 5 and the mobile terminal 6.

  The modulator 4 modulates an MPEG2-TS standard broadcast signal into an ISDB-T standard signal and provides the signal to the receiver 5.

The receiver 5 is, for example, a television or emulator created in accordance with “IPTV standard broadcast communication cooperation system specification IPTVFJ STD-0010 version 2.0” and “IPTV standard HTML5 browser specification IPTVFJ STD-0011 version 2.1”. is there.
The receiver 5 includes a TS receiver 51, a PCR value acquisition unit 52, a PCR value distribution unit 53, a synchronization control unit 54, and a VOD reception / playback unit 55.

  The TS receiver 51 receives the TS signal transmitted from the broadcast station device 2 as a signal of the ISDB-T standard via the modulator 4. At this time, the TS receiver 51 corrects the system clock managed by the receiver 5 based on the PCR value included in the TS signal.

  The PCR value acquisition unit 52 acquires a system clock managed by the receiver 5, that is, a PCR value, and provides it to the PCR value distribution unit 53 and the synchronization control unit 54.

  The PCR value distribution unit 53 distributes the PCR value acquired by the PCR value acquisition unit 52 to the mobile terminal 6.

  The synchronization control unit 54 receives the mapping information from the time mapping information generation device 1 and converts the PCR value into a JST value, thereby reproducing the playback position (time information) of the VOD content played back by the VOD reception / playback unit 55 and the broadcast. Control to synchronize with the clock value.

  The VOD reception / playback unit 55 receives and plays back the VOD content from the content server 3. At this time, the VOD reception / playback unit 55 notifies the synchronization control unit 54 of the current playback position (time information) of the VOD content, and receives synchronization control information indicating the time difference between the playback position and the broadcast from the synchronization control unit 54. Change the playback position (skip or wait). As a result, the VOD reception / playback unit 55 performs synchronous playback by associating the clock value of the broadcast with the time information of the VOD content.

The mobile terminal 6 is an example of a receiver that receives and reproduces VOD content in cooperation with the receiver 5 that receives a broadcast.
The mobile terminal 6 includes a PCR value receiving unit 61, a synchronization control unit 62, and a VOD reception / playback unit 63.

  The PCR value receiving unit 61 receives the PCR value distributed from the receiver 5 and provides it to the synchronization control unit 62.

  Similar to the synchronization control unit 54 of the receiver 5, the synchronization control unit 62 receives the mapping information from the time mapping information generation device 1 and converts the PCR value into a JST value, thereby being reproduced by the VOD reception / reproduction unit 63. Control is performed to synchronize the playback position (time information) of the VOD content and the broadcast clock value.

  The VOD reception / playback unit 63 receives and plays back the VOD content from the content server 3 in the same manner as the VOD reception / playback unit 55 of the receiver 5. In addition, the VOD reception / playback unit 63 notifies the synchronization control unit 62 of the current playback position (time information) of the VOD content, and receives the synchronization control information indicating the time difference between the playback position and the broadcast from the synchronization control unit 62 and plays it back. Change the position (skip or wait). As a result, the VOD reception / play unit 63 performs synchronous playback by associating the clock value of the broadcast with the time information of the VOD content.

  Here, the synchronization control processing in the synchronization control unit 54 of the receiver 5 and the synchronization control unit 62 of the mobile terminal 6 is executed by the following procedures (1) to (5).

(1) The synchronization control unit (54 or 62) acquires a PCR value, obtains a program elapsed time Tpass corresponding to a broadcast time system, and acquires video data (segment file) of VOD content corresponding to Tpass. Tpass is derived from the following equation.
Tpass = β (Tb−Tb0) + Tc0−Tstart
Tb = PCR value Tb0 = PCR value of mapping information Tc0 = JST value of mapping information Tstart = Program start time [JST]
β = 1/27000000

(2) The synchronization control unit (54 or 62) acquires the PCR value, and calculates the time difference δ between the broadcast and the VOD content by the following equation.
δ = β (Tb−Tb0) − (Tstart + TcurrentTime−Tc0)
TcurrentTime = VOD content playback elapsed time [s]

  (3) The synchronization control unit (54 or 62) skips or waits for the playback position of the VOD content for δ seconds.

  (4) The synchronization control unit (54 or 62) repeats the steps (3) and (4) until δ falls within a predetermined synchronization adjustment threshold range, and stabilizes the synchronization state.

(5) After the step (5), the synchronization control unit (54 or 62) periodically calculates δ, and when it exceeds a pre-synchronization threshold value, the synchronization process, that is, the procedure (2) ) To (4) are executed.
Note that the re-synchronization threshold value of the procedure (5) is set to a value larger than the synchronization adjustment threshold value of the procedure (4), so that video disturbance due to frequent synchronization adjustment can be suppressed.

  As mentioned above, although embodiment of this invention was described, this invention is not restricted to embodiment mentioned above. Further, the effects described in the present embodiment are merely a list of the most preferable effects resulting from the present invention, and the effects of the present invention are not limited to those described in the present embodiment.

  The mapping information may be periodically updated at a predetermined timing. For example, the mapping information may be updated every time a TOT packet according to the first embodiment or an EIT p / f packet according to the second embodiment is detected.

  Further, the present invention may be realized by recording a program for realizing the function of the time mapping generation apparatus on a computer-readable recording medium, causing the computer system to read and execute the program recorded on the recording medium. Good.

  The “computer system” here includes an OS and hardware such as peripheral devices. The “computer-readable recording medium” refers to a storage device such as a portable medium such as a flexible disk, a magneto-optical disk, a ROM, a CD-ROM, or a hard disk built in a computer system.

  In addition, “computer-readable recording medium” means that a program is dynamically stored for a short time, like a communication line when a program is transmitted via a network such as the Internet or a communication line such as a telephone line. It is also possible to include one that holds a program for a certain time, such as a volatile memory inside a computer system that becomes a server or client in that case. Further, the program may be for realizing a part of the above-described functions, and may be capable of realizing the above-described functions in combination with a program already recorded in the computer system. .

1, 1a, 1b, 1c Time mapping information generating device 5 Receiver 6 Mobile terminal 11a, 11b Packet analysis unit 12a, 12b JST value acquisition unit 13 PCR value acquisition unit 14 Mapping information generation unit 15 Mapping information distribution unit 16 Program switching determination Unit 17 Image decoding unit 18 Correlation determination unit 19 PTS value acquisition unit 100 Synchronous playback system

Claims (10)

A time information acquisition unit that acquires reference time information at a specific time in a time system used in communication content to be synchronized with broadcast;
A clock value acquisition unit for acquiring a reference clock value in a time system used in broadcasting corresponding to the reference time information from a broadcast stream;
An output unit that outputs mapping information in which the reference time information and the reference clock value are associated with each other .
The time information acquisition unit acquires a specific type of packet including the reference time information from a broadcast stream,
The clock value acquisition unit, the obtains the packet containing the sent clock value before and after the specific type of packet, the linear interpolation by the time the mapping information generating device you calculate the reference clock value. A time information acquisition unit that acquires reference time information at a specific time in a time system used in communication content to be synchronized with broadcast;
A clock value acquisition unit for acquiring a reference clock value in a time system used in broadcasting corresponding to the reference time information from a broadcast stream;
An output unit that outputs mapping information in which the reference time information and the reference clock value are associated with each other .
The time information acquisition unit acquires a specific type of packet including program information from a broadcast stream, and extracts the start time of the program after switching as the reference time information at the time of program switching,
The clock value acquisition unit, the obtains the packet containing the sent clock value before and after the specific type of packet, the linear interpolation by the time the mapping information generating device you calculate the reference clock value. A time information acquisition unit that acquires reference time information at a specific time in a time system used in communication content to be synchronized with broadcast;
A clock value acquisition unit for acquiring a reference clock value in a time system used in broadcasting corresponding to the reference time information from a broadcast stream;
An output unit that outputs mapping information in which the reference time information and the reference clock value are associated with each other .
The time information acquisition unit accepts input of the reference time information in which a specific image in a broadcast stream appears,
The clock value acquiring unit determines a frame the particular image appears, the time mapping information generating device you calculate the reference clock value based on the time stamp is given to the frame. A synchronized playback system comprising the time mapping information generating device according to any one of claims 1 to 3 , and a receiver for receiving broadcast and communication content,
The receiver is a synchronous reproduction system that reproduces a broadcast clock value and communication content time information in association with each other based on the mapping information acquired from the time mapping information generation device. A time information acquisition step of acquiring reference time information at a specific time in a time system used in communication content to be synchronized with broadcasting;
A clock value obtaining step for obtaining a reference clock value in a time system used in broadcasting corresponding to the reference time information from a broadcast stream;
The computer executes an output step of outputting mapping information in which the reference time information and the reference clock value are associated with each other,
In the time information acquisition step, a specific type of packet including the reference time information is acquired from a broadcast stream,
A time mapping information generation method in which, in the clock value acquisition step, a packet including a clock value transmitted before and after the specific type of packet is acquired, and the reference clock value is calculated by linear interpolation . A time information acquisition step of acquiring reference time information at a specific time in a time system used in communication content to be synchronized with broadcasting;
A clock value obtaining step for obtaining a reference clock value in a time system used in broadcasting corresponding to the reference time information from a broadcast stream;
The computer executes an output step of outputting mapping information in which the reference time information and the reference clock value are associated with each other,
In the time information acquisition step, a specific type of packet including program information is acquired from a broadcast stream, and at the time of program switching, the start time of the program after switching is extracted as the reference time information,
A time mapping information generation method in which, in the clock value acquisition step, a packet including a clock value transmitted before and after the specific type of packet is acquired, and the reference clock value is calculated by linear interpolation . A time information acquisition step of acquiring reference time information at a specific time in a time system used in communication content to be synchronized with broadcasting;
A clock value obtaining step for obtaining a reference clock value in a time system used in broadcasting corresponding to the reference time information from a broadcast stream;
The computer executes an output step of outputting mapping information in which the reference time information and the reference clock value are associated with each other,
In the time information acquisition step, an input of the reference time information in which a specific image in the broadcast stream appears is received,
In the clock value acquisition step, a time mapping information generation method for determining a frame in which the specific image appears and calculating the reference clock value based on a time stamp given to the frame . A time information acquisition step of acquiring reference time information at a specific time in a time system used in communication content to be synchronized with broadcasting;
A clock value obtaining step for obtaining a reference clock value in a time system used in broadcasting corresponding to the reference time information from a broadcast stream;
Outputting the mapping information in which the reference time information and the reference clock value are associated with each other ;
In the time information acquisition step, a specific type of packet including the reference time information is acquired from a broadcast stream,
In the clock value acquisition step, wherein to acquire a packet containing a clock value which has been sent before or after a particular type of packet, time mapping information generating program order to calculate the reference clock value by linear interpolation. A time information acquisition step of acquiring reference time information at a specific time in a time system used in communication content to be synchronized with broadcasting;
A clock value obtaining step for obtaining a reference clock value in a time system used in broadcasting corresponding to the reference time information from a broadcast stream;
Outputting the mapping information in which the reference time information and the reference clock value are associated with each other ;
In the time information acquisition step, a specific type of packet including program information is acquired from a broadcast stream, and at the time of program switching, a start time of the program after switching is extracted as the reference time information,
In the clock value acquisition step, wherein to acquire a packet containing a clock value which has been sent before or after a particular type of packet, time mapping information generating program order to calculate the reference clock value by linear interpolation. A time information acquisition step of acquiring reference time information at a specific time in a time system used in communication content to be synchronized with broadcasting;
A clock value obtaining step for obtaining a reference clock value in a time system used in broadcasting corresponding to the reference time information from a broadcast stream;
Outputting the mapping information in which the reference time information and the reference clock value are associated with each other ;
In the time information acquisition step, the input of the reference time information in which a specific image in the broadcast stream appears is accepted,
In the clock value acquisition step, wherein to determine a frame specific image appears, the time mapping information generating program order to calculate the reference clock value based on the time stamp is given to the frame.

Images