JP2017220713A - Unicast distribution device, reproduction synchronous system, unicast distribution method, and unicast distribution program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a unicast distribution device, reproduction synchronous system, unicast distribution method, and unicast distribution program that can perform synchronous reproduction of video distributed by unicast and video distributed by multicast.SOLUTION: A unicast distribution server 4 returns a sequence number of an MPU which a client terminal 6 should receive next on the basis of absolute time in an http response to an http request. In addition, it stores an asset MPU and a signaling message in an MPU construction buffer 42 and a signaling message construction buffer 43 respectively, associates them with a signal message after completion of MPU construction, temporarily stores them in a distribution buffer 44, and distributes them according to the http request.SELECTED DRAWING: Figure 3

Description

  The present invention relates to a unicast distribution apparatus, a reproduction synchronization system, a unicast distribution method, and a unicast distribution program.

  MMT (MPEG Media Transport) has been standardized as the next-generation transport protocol, providing a framework that can be played back synchronously even when multi-content delivery via multiple media and multiple routes such as broadcasting and communication, and simultaneous delivery of the same content. ing.

  On the other hand, for video distribution of events, etc., there is a service that allows users to select and view not only the uniform images such as 4k / 8k provided by SP (Service Provider) but also cameras from various angles. Is provided. Furthermore, there is a need for a style that allows users to view a video that suits the user's preference in synchronism with a large screen, such as cutting and enlarging the area that each user wants to see.

  Various distribution forms such as a multicast network that allows all users to receive the same content, and a unicast network that allows users to receive individual content are assumed. It is desired to enable content viewing.

Mitsuhiro Hirabayashi Next-Generation Video Distribution Technology “MPEG-DASH” Technical Overview, Standardization and Related Technology Trends Journal of the Institute of Image Information and Television Engineers Vol.67 No.2 (2013) ISO / IEC 23008-1: 2014 Information technology-High efficiency coding and media delivery in heterogeneous environments-Part 1: MPEG media transport (MMT) ISO / IEC 23008-11: 2015 Information technology-High efficiency coding and media delivery in heterogeneous environments-Part 11: MPEG media transport composition information ISO / IEC 23009-1: 2014 Information technology-Dynamic adaptive streaming over HTTP (DASH)-Part 1: Media presentation description and segment formats

  However, the conventional technique has a problem related to synchronization as described below.

  In addition, as described below, the related art has a problem related to identification of received content.

  That is, as described above, there is MPEG-DASH as a video distribution using unicast. In the case of MPEG-DASH, an Adaptation Set and Representation, a playback time, a URL, and the like are described in an MPD (Media Presentation Description) file, and the content to be received by the client can be specified. However, when performing unicast delivery with MMT, a means for specifying the received content is required.

  In view of the above-described conventional technology, the present invention provides a unicast distribution apparatus, a reproduction synchronization system, a unicast distribution method, and a unicast distribution apparatus capable of performing synchronous reproduction of video distributed by unicast and video distributed by multicast. The purpose is to provide a unicast distribution program.

  In order to achieve the above object, the invention according to the first aspect is a unicast delivery apparatus, which is received by an MMTP receiving function unit that receives an MMTP (MPEG Media Transport Protocol) packet and the MMTP receiving function unit An MPU construction buffer that manages an MPU (Media Processing Unit) included in the MMTP packet, a signaling message construction buffer that manages a signaling message included in the MMTP packet received by the MMTP reception function unit, and an MMTP packet to be distributed A distribution buffer managed in units of MPU, and an http distribution server function unit for a client that acquires an MMTP packet of a specified asset from the distribution buffer and distributes it to the client terminal in response to an http request from a client terminal In the http response to the http request, based on the absolute time, The serial client terminal returns the sequence number of the MPU to be received is summarized as.

  The invention according to the second aspect is a unicast delivery apparatus, and is included in an MMTP reception function unit that receives an MMTP (MPEG Media Transport Protocol) packet and an MMTP packet received by the MMTP reception function unit An MPU construction buffer that manages an MPU (Media Processing Unit), a signaling message construction buffer that manages a signaling message included in an MMTP packet received by the MMTP reception function unit, and an MMTP packet to be distributed are managed in units of MPU An MPU construction buffer comprising: a delivery buffer; and an http delivery server function unit for a client that obtains an MMTP packet of a specified asset from the delivery buffer and delivers it to the client terminal in response to an http request from a client terminal And MPU of the asset and signaling in the signaling message construction buffer The store each message, performs a correlation between the signal message after MPU construction completed, the temporarily stores the distribution buffer, and summarized in that the delivery in accordance with the http request.

  The invention according to the third aspect is a reproduction synchronization system, which is a delay correction device for adding an offset time, receives content from the delay correction device, performs transmission control of the content, and performs multicast distribution 3. The unicast distribution server according to claim 1, which receives content from the packet transmission server and the delay correction device and performs unicast distribution in response to an http request from a client terminal, and multicast from the packet transmission server. The gist of the present invention is to provide a client terminal capable of synchronously reproducing the received content and the content received by unicast from the unicast distribution server.

  The invention according to the fourth aspect is a unicast delivery method, wherein an MMTP reception step of receiving an MMTP (MPEG Media Transport Protocol) packet, and an MPU (included in the MMTP packet received in the MMTP reception step) An MPU construction step for managing a Media Processing Unit) using an MPU construction buffer; a signaling message construction step for managing a signaling message included in the MMTP packet received in the MMTP reception step using a signaling message construction buffer; A distribution step for managing a target MMTP packet using a distribution buffer in units of MPUs, and in response to an http request from a client terminal, obtains an MMTP packet of a specified asset from the distribution buffer and distributes it to the client terminal Http delivery step for the client, In the http response to the quest, the gist is to return the MPU sequence number to be received next by the client terminal based on the absolute time.

  The invention according to the fifth aspect is a unicast delivery method, wherein an MMTP reception step of receiving an MMTP (MPEG Media Transport Protocol) packet, and an MPU (included in the MMTP packet received in the MMTP reception step) An MPU construction step for managing a Media Processing Unit) using an MPU construction buffer; a signaling message construction step for managing a signaling message included in the MMTP packet received in the MMTP reception step using a signaling message construction buffer; A distribution step for managing a target MMTP packet using a distribution buffer in units of MPUs, and in response to an http request from a client terminal, obtains an MMTP packet of a specified asset from the distribution buffer and distributes it to the client terminal Http delivery step for clients, and the MPU structure The MPU and signaling message of the asset are stored in the buffer and the signaling message construction buffer, respectively, are associated with the signal message after completion of construction of the MPU, temporarily stored in the delivery buffer, and delivered in response to the http request This is the gist.

  Moreover, the invention according to the sixth aspect is a unicast distribution program, and the gist thereof is to cause a computer to realize each functional unit according to the first or second aspect.

  According to the present invention, there are provided a unicast distribution apparatus, a reproduction synchronization system, a unicast distribution method, and a unicast distribution program capable of performing synchronous reproduction of video distributed by unicast and video distributed by multicast. Is possible.

It is a block diagram of the reproduction | regeneration synchronization system in embodiment of this invention. It is a sequence diagram which shows the detail of step S5, S6 shown by FIG. It is a functional block diagram of the unicast delivery server in embodiment of this invention. It is explanatory drawing of the MPU construction buffer with which the unicast delivery server in embodiment of this invention is provided. It is explanatory drawing which shows the relationship between the MPU construction buffer with which the unicast delivery server in embodiment of this invention is provided, the signaling message construction buffer, and the delivery buffer.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. The following embodiments exemplify an MMT synchronization system for embodying the technical idea of the present invention, and the configuration of the apparatus and the configuration of data are limited to the following embodiments. is not.

≪Overview≫
The embodiment of the present invention is a technology related to video content distribution. MPEG-DASH, which is a conventional technology, is a technology that distributes video using unicast. However, since the playback time is specified as a relative time, video that is distributed by unicast and video that is distributed by multicast are used. It was very difficult to perform synchronized playback. In the embodiment of the present invention, an asset (content) to be reproduced and a signal message designating the reproduction time are stored in the data body (payload) of the http packet and distributed. In addition, the MPU and signal message of the asset are stored in the MPU construction buffer and signaling message construction buffer of the unicast delivery server, respectively, and are associated with the signal message after the completion of construction of the MPU, and temporarily stored in the delivery buffer. Delivered upon request from. As a result, it is possible to perform synchronized playback of video distributed by unicast and video distributed by multicast.

  In MMT, several signaling messages are defined. In the present embodiment, an MPT (MMT Package Table) message is used as a message for designating the reproduction time, but the present invention is not limited to this. For example, a PA (Package Access) message can also be used as a message for specifying the playback time.

≪Overall flow≫
FIG. 1 is a configuration diagram of a reproduction synchronization system according to an embodiment of the present invention. This reproduction synchronization system is, for example, an MMT synchronization system based on unicast distribution, and as shown in FIG. 1, a content transmission method conversion server 3, a unicast distribution server 4, a packet transmission server 5, and a client terminal 6 With. The content transmission method conversion server 3 is a device that converts the content transmission method (here, a delay correction device that adds an offset time). The unicast distribution server 4 is a device that receives content from the content transmission method conversion server 3 and performs unicast distribution in response to an http request from the client terminal 6. The packet transmission server 5 is a device that receives content from the content transmission method conversion server 3, performs transmission control of the content, and performs multicast distribution. The client terminal 6 is an apparatus capable of reproducing the content received from the packet transmission server 5 by multicast and the content received from the unicast distribution server 4 in synchronization with each other.

  The overall flow of such a reproduction synchronization system will be described below with reference to FIG.

  Step S1: The video stream from the camera 1 is converted into an MMTP packet by the MMT transmission device 2 and input to the content transmission method conversion server 3. The content transmission method conversion server 3 performs processing for replacing the MPU time stamp and the MMTP time stamp to which a fixed offset time is added.

  Step S2: MMT packets corresponding to a plurality of video streams whose time stamps have been changed are transmitted to the unicast distribution server 4 and the packet transmission server 5.

  Step S3: The packet transmission server 5 performs transmission control of the MMT packet according to the MMTP time stamp, and performs multicast distribution.

  Step S4: The unicast distribution server 4 buffers the received MMTP packet into MPUs and signaling messages. After the MPU construction is completed, the MPU is moved to the subsequent distribution buffer in association with the signaling message.

  Step S5: The client terminal 6 transmits to the unicast distribution server 4 an http request including the Asset_ID and MPU_SN information to be received. Asset_ID is an ID for identifying an asset (content) such as video or audio. MPU_SN is an MPU sequence number.

  Step S6: The unicast distribution server 4 returns an http response storing the MMTP packet including the MPU_SN to be received next.

  Step S7: By interpreting the MMT, the client terminal 6 can reproduce the content received by multicast and unicast in synchronization.

  As described above, in the playback synchronization system according to the embodiment of the present invention, the MPU storing the content and the MPT message corresponding to the MPU are stored in the http payload and distributed to the client (steps). S4). The MPT message is a signaling message that specifies the asset to be played back and the playback time thereof, and the playback time specified in the signaling message is an absolute time. As a result, it is possible to perform synchronized playback of video distributed by unicast and video distributed by multicast.

  In the playback synchronization system according to the embodiment of the present invention, Asset_ID and MPU_SN to be acquired are stored in the URL and http header in the request / response between the server and the client based on http (step) S5, S6). As described above, Asset_ID is an ID for identifying an asset (content) such as video or audio. MPU_SN is an MPU sequence number. As a result, even when MMTP packets are unicasted, the MPU-SN to be acquired can be acquired from the unicast distribution server 4 (identified received content) based on the URL and Asset_ID acquired by another means. Is possible.

<< Details of Steps S5 and S6 >>
FIG. 2 is a sequence diagram showing details of steps S5 and S6 shown in FIG. As will be described below, a signaling message that specifies an asset to be reproduced and its reproduction time is stored in a payload of http and distributed.

  Step S11: The client terminal 6 stores the URL of the unicast distribution server 4 and the information of Asset_ID to be received acquired by another means in the URL, and transmits the URL to the unicast distribution server 4 as an http request. Specifically, as shown by a symbol M1 in the figure, a URL including asset_ID to be acquired is sent.

  Steps S12 and S13: Based on the system time, the unicast distribution server 4 stores the MPU_SN to be received by the client terminal 6 in the http response header and transmits it to the client terminal 6. Specifically, as indicated by a symbol M2 in the figure, the packet ID of the first MPU_SN, MPU, and MPT to be acquired is included.

  Step S <b> 14: The client terminal 6 transmits an http request including Asset_ID and MPU_SN to the unicast distribution server 4 based on the information of the http response header received from the unicast distribution server 4. Specifically, as indicated by a symbol M3 in the figure, the Asset_ID and MPU_SN acquired based on the response from the unicast distribution server 4 are requested.

  Step S15: The unicast distribution server 4 acquires an MMTP packet based on the request from the client terminal 6. Specifically, the MMTP packet specified by the http request received from the client terminal 6 is acquired from the distribution buffer.

  Step S16: The unicast distribution server 4 includes the MPU_SN to be received next in the http response header, and returns an http response storing the acquired MMTP packet to the client terminal 6. Specifically, as indicated by reference numerals M4 and M5 in the figure, the head MPU_SN to be acquired next and actual data are sent.

≪Details of step S4≫
-Construction method on Unicast distribution server 4 (outline)-
FIG. 3 is a functional block diagram of the unicast distribution server 4. As shown in this figure, the unicast delivery server 4 includes an MMTP reception function unit 41, an MPU construction buffer 42, a signaling message construction buffer 43, a delivery buffer 44, and an http delivery server function unit 45 for clients. .

  The MMTP reception function unit 41 receives the MMTP stream, accumulates MPU packets in the MPU construction buffer 42, and accumulates signaling messages in the signaling message construction buffer 43. The MPU construction buffer 42 manages MPU packets for each MPU_sequence_number, and manages MMTP packets for each MPU in the order of packet_sequence_number. In order to perform completion determination based on the MPU construction waiting time, the MMTP timestamp of the first MMTP packet and the last MMTP packet is held for each MPU. The signaling message construction buffer 43 arranges and manages signaling messages in order of packet_sequence_number for each packet_id. The distribution buffer 44 manages MMTP packets (MPUs and signaling messages) in units of MPUs. The http delivery server function unit 45 for the client returns the MMTP packet of the specified asset in response to the http request from the client terminal 6.

  Here, since the unicast distribution server 4 stores and distributes the asset and the signaling message in the payload of http / TCP, it should be temporarily stored and distributed by the MMTP storage function unit 46 in the unicast distribution server 4. Sending processing is performed in response to a request from the client terminal 6 in consideration of timing. Here, the part including the MPU construction buffer 42, the signaling message construction buffer 43, and the distribution buffer 44 is called an MMTP accumulation function unit 46.

-MPU construction processing-
FIG. 4 is an explanatory diagram of the MPU construction buffer 42. Here, it is assumed that the MPU construction wait offset time is 1 sec.

  First, the unicast distribution server 4 manages MMTP packets for each MPU. At that time, monitoring is performed by arranging packets_sequence_number in the order of the MMTP packet header.

  In addition, the unicast distribution server 4 determines whether the MPU construction has been completed. In the completion determination, an MPU construction time is held as an offset setting value, and an MPU that satisfies the following comparison with the MMTPTimestamp of the first MMTP packet header of the MPU is determined to be complete.

First MMTP timestamp ≤ (System time + MPU construction wait offset time)
MMTP timestamp: Time included in the MMTP packet header (UTC)
System time: UTC time synchronized with NTP server
MPU construction waiting offset time: estimated time for MPU construction The unicast distribution server 4 moves the MPU determined to be completed to the distribution buffer 44. In the example of FIG. 4, MPU # 1 is moved to the distribution buffer 44 at 11: 59: 59: 000, MPU # 2 is moved to the distribution buffer 44 at 11: 59: 59.500, and MPU # 3 is moved to 10:00: Move to the distribution buffer 44 at 00.000, and move MPU # 4 to the distribution buffer 44 at 10: 00: 00.500.

  The unicast distribution server 4 moves to the distribution buffer 44 that satisfies the completion condition even when there is an unarrived MMTP packet. Although there is a possibility that an error may occur for a moment during video reproduction on the client terminal 6 side, the processing is continued. If it arrives after moving to the distribution buffer 44, an error log is output and discarded.

-Correspondence between MPU and signaling message-
FIG. 5 is an explanatory diagram showing the relationship among the MPU construction buffer 42, the signaling message construction buffer 43, and the distribution buffer 44. The “buffer” in this specification includes not only a storage device that stores data but also a functional unit that manages the storage device.

  The unicast distribution server 4 associates the MPU whose MPU construction is completed in the MPU construction buffer 42 with the corresponding signaling message.

  Here, for example, the MMTP stream received by the MMTP reception function unit 41 is adjusted so that a signaling message including MPT information is transmitted while the corresponding MPU is transmitted as an MMTP packet, and the corresponding A message having a timestamp equal to or less than the last MMTPTimestamp including the MPU is associated with a signaling message corresponding to the corresponding MPU.

  If the processing time and load for analyzing the MPT are allowed, the MPU time stamp descriptor in the MPT can be analyzed and the MPU sequence number can be extracted for association.

-Processing in distribution buffer 44-
Unicast distribution is started by a request from the client terminal 6, but MMTP packet groups (MPUs and signaling messages) whose MMTPTimestamp is older than the system time-asset storage time are deleted from the distribution buffer 44.

  By setting the asset accumulation time, MMTP packets are accumulated in the distribution buffer 44 for the set time, and can be provided even when there is a request for unicast asset distribution from the client terminal 6 in the past time.

  As described above, the unicast delivery server 4 in the embodiment of the present invention includes the MMTP reception function unit 41, the MPU construction buffer 42, the signaling message construction buffer 43, the delivery buffer 44, and the http delivery server function for clients. Part 45. The MMTP reception function unit 41 receives an MMTP packet. The MPU construction buffer 42 manages the MPU included in the MMTP packet received by the MMTP reception function unit 41. The signaling message construction buffer 43 manages a signaling message included in the MMTP packet received by the MMTP reception function unit 41. The distribution buffer 44 manages distribution target MMTP packets in units of MPUs. In response to an http request from the client terminal 6, the client http distribution server function unit 45 acquires the MMTP packet of the designated asset from the distribution buffer 44 and distributes it to the client terminal 6. Then, in the http response to the http request, the sequence number of the MPU to be received next by the client terminal 6 is returned based on the absolute time. As a result, it is possible to perform synchronized playback of video distributed by unicast and video distributed by multicast.

  In addition, the unicast distribution server 4 in the embodiment of the present invention includes an MMTP reception function unit 41, an MPU construction buffer 42, a signaling message construction buffer 43, a distribution buffer 44, a client http distribution server function unit 45, Is provided. The MMTP reception function unit 41 receives an MMTP packet. The MPU construction buffer 42 manages the MPU included in the MMTP packet received by the MMTP reception function unit 41. The signaling message construction buffer 43 manages a signaling message included in the MMTP packet received by the MMTP reception function unit 41. The distribution buffer 44 manages distribution target MMTP packets in units of MPUs. In response to an http request from the client terminal 6, the client http distribution server function unit 45 acquires the MMTP packet of the designated asset from the distribution buffer 44 and distributes it to the client terminal 6. Then, the MPU and the signaling message of the asset are respectively stored in the MPU construction buffer 42 and the signaling message construction buffer 43, are associated with the signal message after the MPU construction is completed, temporarily stored in the distribution buffer 44, and sent to the http request. Deliver accordingly. This makes it possible to specify received content when unicasting MMTP packets.

  Although not particularly mentioned in the above description, the content transmission method conversion server 3 performs processing for synchronizing the delay time due to the delivery delay between the unicast delivery server 4 and the client terminal 6. It has become.

  In MPEG-DASH, MPD files are interpreted on the client side, and contents are acquired from the server. On the other hand, in the embodiment of the present invention, the client terminal 6 performs synchronous reproduction by following the designation from the unicast distribution server 4 if only MMT mounting is basically performed. For example, in steps S11 to S16 of FIG. 2, the unicast distribution server 4 and the client terminal 6 are reciprocated twice, but this part also receives the designation of MPU_SN to be received from the unicast distribution server 4 and follows it. It has become.

  In addition, the embodiment of the present invention can be realized not only as such a unicast distribution server 4 but also as a unicast distribution method in which a characteristic function unit provided in such a unicast distribution server 4 is a step. Or can be realized as a unicast distribution program that causes a computer to execute these steps. Needless to say, such a program can be distributed via a recording medium such as a CD-ROM or a transmission medium such as the Internet.

DESCRIPTION OF SYMBOLS 1 ... Camera 2 ... MMT transmission apparatus 3 ... Content transmission system conversion server (delay correction apparatus)
4 ... Unicast distribution server 5 ... Packet transmission server
6 ... Client terminal 41 ... MMTP reception function part 42 ... MPU construction buffer 43 ... Signaling message construction buffer 44 ... Delivery buffer 45 ... Client http delivery server function part 46 ... MMTP storage function part

Claims (6)

MMTP reception function unit that receives MMTP (MPEG Media Transport Protocol) packets;
An MPU construction buffer that manages an MPU (Media Processing Unit) included in the MMTP packet received by the MMTP reception function unit;
A signaling message construction buffer for managing a signaling message included in the MMTP packet received by the MMTP reception function unit;
A delivery buffer that manages MMTP packets to be delivered in MPU units;
In response to an http request from a client terminal, an MMTP packet for a specified asset is obtained from the distribution buffer and is delivered to the client terminal.
A unicast distribution apparatus, which returns a sequence number of an MPU to be received next by the client terminal based on an absolute time in an http response to the http request. MMTP reception function unit that receives MMTP (MPEG Media Transport Protocol) packets;
An MPU construction buffer that manages an MPU (Media Processing Unit) included in the MMTP packet received by the MMTP reception function unit;
A signaling message construction buffer for managing a signaling message included in the MMTP packet received by the MMTP reception function unit;
A delivery buffer that manages MMTP packets to be delivered in MPU units;
In response to an http request from a client terminal, an MMTP packet for a specified asset is obtained from the distribution buffer and is delivered to the client terminal.
The MPU construction buffer and the signaling message construction buffer store the asset MPU and the signaling message, respectively, associate with the signal message after the MPU construction is completed, temporarily store in the delivery buffer, and respond to the http request A unicast distribution device characterized by distribution. A delay correction device for adding an offset time;
A packet transmission server that receives content from the delay correction device, performs transmission control of the content, and performs multicast distribution;
The unicast distribution server according to claim 1 or 2, wherein content is received from the delay correction device and unicast distribution is performed in response to an http request from a client terminal.
A reproduction synchronization system comprising: a client terminal capable of synchronizing and reproducing content received by multicast from the packet transmission server and content received by unicast from the unicast distribution server. An MMTP reception step of receiving an MMTP (MPEG Media Transport Protocol) packet;
MPU construction step for managing an MPU (Media Processing Unit) included in the MMTP packet received in the MMTP reception step using an MPU construction buffer;
A signaling message construction step for managing a signaling message included in the MMTP packet received in the MMTP reception step using a signaling message construction buffer;
A distribution step for managing distribution target MMTP packets in units of MPU using a distribution buffer;
In response to an http request from a client terminal, an MMTP packet for a specified asset is obtained from the delivery buffer and delivered to the client terminal.
In the http response to the http request, the MPU sequence number to be received next by the client terminal is returned based on the absolute time. An MMTP reception step of receiving an MMTP (MPEG Media Transport Protocol) packet;
MPU construction step for managing an MPU (Media Processing Unit) included in the MMTP packet received in the MMTP reception step using an MPU construction buffer;
A signaling message construction step for managing a signaling message included in the MMTP packet received in the MMTP reception step using a signaling message construction buffer;
A distribution step for managing distribution target MMTP packets in units of MPU using a distribution buffer;
In response to an http request from a client terminal, an MMTP packet for a specified asset is obtained from the delivery buffer and delivered to the client terminal.
The MPU construction buffer and the signaling message construction buffer store the asset MPU and the signaling message, respectively, associate with the signal message after the MPU construction is completed, temporarily store in the delivery buffer, and respond to the http request A unicast distribution method characterized in that the distribution is performed.   A unicast distribution program for causing a computer to realize each functional unit according to claim 1.

Images