JP2019536312A - System and method for enabling communication associated with digital media distribution - Google Patents

Abstract

The device is configured to signal a frame header indicating a dynamic adaptive streaming over hypertext transfer protocol message type, and to signal one or more given arguments corresponding to the message type as JavaScript Object Notation encoding parameters. You may.

Description

  The present disclosure relates to the field of digital media distribution.

  Digital media playback functions include digital TVs including so-called “smart” TVs, set-top boxes, laptop or desktop computers, tablet computers, digital recording devices, digital media players, video gaming devices, mobiles including so-called “smart” phones. It can be incorporated into a wide range of devices, including telephones, dedicated video streaming devices, and the like. Digital media content (eg, video and audio programs) can originate from multiple sources, such as online media service providers including wireless television providers, satellite television providers, cable television providers, so-called streaming service providers, and the like. Digital media content may be delivered over packet-switched networks including interactive networks such as Internet Protocol (IP) networks and unidirectional networks such as digital broadcast networks.

  Digital media content may be transmitted from a source to a receiving device (eg, digital television or smartphone) according to a content delivery protocol model. The content delivery protocol mode can be based on one or more transmission standards. Examples of transmission standards include the Digital Video Broadcasting (DVB) standard, the Integrated Services Digital Broadcasting (ISDB) standard, and, for example, the ATSC 3.0 series of standards currently being created And standards developed by the Advanced Television Systems Committee (ATSC). Current techniques for transmitting digital media content from a source to a receiver device may not be ideal.

  According to one embodiment of the present disclosure, a method for signaling information associated with a media presentation includes signaling a frame header indicating a dynamic adaptive streaming over hypertext transfer protocol message type, and a JavaScript Object Notation encoding parameter. Signaling one or more given arguments corresponding to the message type.

FIG. 1 is a conceptual diagram illustrating an example content delivery protocol model in accordance with one or more techniques of this disclosure. FIG. 2 is a block diagram illustrating an example system that can implement one or more techniques of this disclosure. FIG. 3 is a block diagram illustrating an example computing device that can implement one or more techniques of this disclosure. FIG. 4 is a block diagram illustrating an example of a media distribution engine that can implement one or more techniques of this disclosure. FIG. 5 is a conceptual diagram illustrating an example communication flow according to one or more techniques of this disclosure. FIG. 6A is a computer programming list showing an exemplary schema for each of the media presentation document request messages. FIG. 6B is a computer programming list showing an exemplary schema for each of the media presentation document request messages. FIG. 7A is a computer programming list showing an exemplary schema for each of the media presentation document request response messages. FIG. 7B is a computer programming list showing an exemplary schema for each of the media presentation document request response messages. FIG. 8A is a computer programming list showing an exemplary schema for each of the segment request messages. FIG. 8B is a computer programming list showing an exemplary schema for each of the segment request messages. FIG. 9A is a computer programming list showing an exemplary schema for each of the segment request response messages. FIG. 9B is a computer programming list showing an exemplary schema for each of the segment request response messages. FIG. 9C is a computer programming list showing an exemplary schema for each of the segment request response messages. FIG. 9D is a computer programming list showing an exemplary schema for each of the segment request response messages. FIG. 10A is a computer programming list showing an exemplary schema for each of the cancellation request messages. FIG. 10B is a computer programming list showing an exemplary schema for each of the cancellation request messages. FIG. 11A is a computer programming list showing an exemplary schema for each of the resource change notification messages. FIG. 11B is a computer programming list showing an exemplary schema for each of the resource change notification messages.

  In general, this disclosure describes techniques that enable communication associated with distribution of digital media content. It should be noted that the digital media content may be included as part of an audiovisual service, or in some embodiments may be included as a dedicated audio service. Although some embodiments describe the techniques of this disclosure for specific transmission standards and specific digital media formats, the techniques described herein generally apply to various transmission standards and digital media formats. Note that it may be applicable. For example, the techniques described in this specification are generally DVB standards, ISDB standards, ATSC standards, Digital Terrestrial Multimedia Broadcast (DTMB) standards, Digital Multimedia Broadcast (DMB) standards, HybridBandB Of the World Wide Web Consortium (W3C) standards, Universal Plug and Play (UPnP) standards, and Moving Pictures Expert Group (MPEG) standards May be applicable to any of the above. Further, it is noted that the incorporation of documents herein by reference is for purposes of explanation and should not be constructed to limit and / or create ambiguity with respect to the terms used herein. Should. For example, if an incorporated reference provides a definition of a term that is different from another incorporated reference, and / or the term is used herein, that term broadly includes each respective definition. Should be construed in a manner and / or in a manner that includes each of the specific definitions in the alternative.

  According to another embodiment of the present disclosure, a device signaling information associated with a media presentation signals a frame header indicating a dynamic adaptive streaming over hypertext transfer protocol message type, and a JavaScript Object Notation encoding parameter. As one or more processors configured to signal one or more given arguments corresponding to the message type.

  According to another embodiment of the present disclosure, an apparatus for signaling information associated with a media presentation includes means for signaling a frame header indicating a dynamic adaptive streaming over hypertext transfer protocol message type, and JavaScript Object Notation encoding. Means for signaling one or more given arguments corresponding to the message type as parameters.

  According to another embodiment of the present disclosure, a non-transitory computer readable storage medium, when executed, causes one or more processors of the device to signal a frame header indicating a dynamic adaptive streaming over hypertext transfer protocol message type. And an instruction for signaling one or more given arguments corresponding to the message type as a JavaScript Object Notation encoding parameter.

  The details of one or more embodiments are set forth in the accompanying drawings and the description below. Other features, objects, and advantages will be apparent from the description and drawings, and from the claims.

  The computing device and / or transmission system may be based on a model that includes one or more abstraction layers, where each abstraction layer data is represented according to a particular structure, eg, packet structure, modulation scheme, and so on. An example of a model that includes a defined abstraction layer is a so-called Open Systems Interconnection (OSI) model. The OSI model defines a seven layer stack model that includes an application layer, a presentation layer, a session layer, a transport layer, a network layer, a data link layer, and a physical layer. It should be noted that the use of the terms upper and lower for the description of the layers in the stack model may be based on the application layer being the top layer and the physical layer being the bottom layer. is there. Further, in some cases, the term “layer 1” or “L1” can be used to refer to the physical layer, and the term “layer 2” or “L2” can be used to refer to the link layer; The term “layer 3” or “L3” or “IP layer” may be used to refer to the network layer.

  A physical layer can generally refer to a layer in which electrical signals form digital data. For example, the physical layer can refer to a layer that defines how modulated radio frequency (RF) symbols form a frame of digital data. The data link layer, which may also be referred to as the link layer, can refer to an abstraction used before physical layer processing on the transmitting side and after receiving the physical layer on the receiving side. As used in the present invention, the link layer is used for transporting data from the network layer to the physical layer on the transmitting side, and used for transporting data from the physical layer to the network layer on the receiving side. Note that the sender and receiver are logical roles, and a single device can act as both the sender of one instance and the receiver of another instance. It is. The link layer processes various types of data (eg, video files, audio files, or application files) encapsulated in a specific packet type (eg, Internet Protocol Version 4 (IPv4) packet) by the physical layer. Can be abstracted into a single generic format for A network layer can generally refer to a layer where logical addressing occurs. That is, the network layer can generally provide addressing information (eg, IP addresses), which can deliver data packets to specific nodes (eg, computing devices) in the network. As used herein, the term network layer can refer to a layer above the link layer and / or a layer having data structured such that it can be received for link layer processing. The transport layer can refer to a layer that enables so-called interprocess communication services. Each of the session layer, presentation layer, and application layer can define how data is delivered for use by a user application. Transmission standards, including transmission standards currently in development, can include a content delivery protocol model that defines the protocols supported for each layer, and can further define one or more specific layer implementations.

FIG. 1 shows an example of a content delivery protocol model according to the techniques described herein. The techniques described herein may be implemented in a system configured to operate based on the content delivery protocol model 100.
The content delivery protocol model 100 can enable distribution of digital media content. Accordingly, the content delivery protocol model 100 can be implemented in a system that allows a user to access digital media content from a media service provider (eg, a so-called streaming service). The term media service or media presentation can be used to refer to a collection of media components (eg, moving image components, audio components, and subtitle components) that are collectively presented to a user, where a component is a plurality of media. The service can be either continuous or intermittent, and the service can be a real-time service (eg, a multimedia presentation corresponding to a live event) or a non-real-time service (eg, video-on Demand service).

  As shown in FIG. 1, the content delivery protocol model 100 includes a physical layer and a data link layer. As described above, the physical layer can generally refer to a layer in which electrical signals form digital data, and the data link layer can include a general format for processing by the physical layer. In the example shown in FIG. 1, in some examples, the physical layer includes a physical layer frame structure that includes a defined preamble and a data payload structure that includes one logical structure within the RF channel or portion of the RF channel. And can be included. In the embodiment shown in FIG. 1, the data link layer may be a data structure used to abstract various specific packet types into a single generic format for processing by the physical layer, As described above, a network layer can generally refer to a layer where logical addressing occurs. Referring to FIG. 1, the content delivery protocol model 100 can utilize an IP addressing scheme, eg, IPv4 and / or IPv6 as a network layer. As mentioned above, the transport layer can generally refer to a layer that enables so-called interprocess communication services. As shown in FIG. 1, the content delivery protocol model 100 can use a transport control protocol (TCP) as a transport layer.

  As described above, the content delivery protocol model 100 can enable distribution of digital media content. In the example of FIG. 1, the application can include an application that allows a user to render digital media content. For example, the content delivery protocol model can include a video player application that runs on a mobile device or the like. In the embodiment shown in FIG. 1, the media codec includes a base codec (ie, a multimedia encoder / decoder implementation) that allows a digital media content format (eg, MPEG format, etc.) to be rendered by an application. be able to. In the embodiment shown in FIG. 1, the content delivery protocol model 100 utilizes dynamic adaptive streaming over HTTP (DASH) to allow a logical client to receive digital media content from a logical server. To. DASH is, MPEG DASH ISO / IEC: ISO / IEC 23009~1: 2014, "Information technology_Dynamic adaptive streaming over HTTP (DASH) -Part1: Media presentation description and segment formats", International Organization for Standardization, 2nd Edition, 5/15 / 2014 (hereinafter “ISO / IEC 23009-1: 2014”) is described in the DASH-IF profile, which is incorporated herein by reference. As shown in FIG. 1, a DASH media presentation can include a data segment, a video segment, and an audio segment. In some embodiments, a DASH media presentation is a linear service or part of a linear service of a given period defined by a service provider (eg, a single TV program or a series of linear TV programs over a period of time). Set). In accordance with DASH, a media presentation document (MPD) is a metadata fragment that contains a formalized description of the profile of a DASH media presentation. The fragments can include a set of extensible markup language (XML) encoded metadata fragments. The MPD content provides a resource identifier for the segment and a context for the identified resource in the media presentation. The data structure and semantics of MPD fragments are described with respect to ISO / IEC 23009-1: 2014. Furthermore, it should be noted that a draft version of ISO / IEC 23009-1 is currently proposed. Thus, in the example shown in FIG. 1, the MPD may include MPD as described in ISO / IEC 23209-1: 2014, currently proposed MPD, and / or combinations thereof.

  In ISO / IEC 23009-1: 2014, a media presentation described in MPD can include a sequence of one or more periods, each period being one or more adaptation sets. Can be included. It should be noted that if the adaptation set includes multiple media content components, each media content component may be described individually. Each adaptation set can include one or more representations. In ISO / IEC 23009-1: 2014, each representation is provided as follows. (1) As a single segment, sub-segments are aligned across the representation of the adaptation set, and (2) As a sequence of segments, each segment is generated by a universal resource locator (URL) generated in the template. Addressable. The properties of each media content component may be described by an adaptation set element and / or an element in an adaptation set including, for example, a content component element.

  As shown in FIG. 1, DASH may operate over HTTP and WebSocket protocols. That is, the DASH media presentation is described in the Internet Engineering Task Force (IETF) Request for Comment (RFC) 7540 Hypertext Transfer Protocol Version 2 (HTTP / 2) in NTP / Ent 2 in May / 2015. Task Force, RFC 6455 IETF: “The WebSocket Protocol” (December 2011) and the WebSocket API (World Wide Web Consortium (W3C) Recommendation Candidate, September 20, 2012) A full-duplex HTTP-compatible protocol, including the WebSocket protocol May be transported via a protocol, each of which is incorporated herein by reference.

  The WebSocket protocol allows mutual communication between a logical client and a logical server (sometimes called a remote host) through the establishment of a bidirectional socket. It should be noted that in some cases, the side or endpoint of a bidirectional socket may be referred to as a terminal. A bidirectional socket based on the WebSocket protocol exchanges character data encoded in Universal Transformation Format-8 (UTF-8) between a logical client and a logical server via TCP. Including a bi-directional socket. 23009-6: DASH with Server Push and WebSockets (ISO / IEC JTC1 / SC29 / WG11 / W16232, June 2016) (hereinafter referred to as “W16232”) (this is incorporated herein by reference) Defines signaling and message formats for driving delivery of MPEG-DASH media presentations using bi-directional sockets based on the WebSocket protocol.

In W16232, the transmission of a segment from the server to the client (referred to as push in W16232) is based on a push strategy, which defines how the segment can be transmitted from the server to the client. In W16232, a push directive is a request qualifier sent from the client to the server, which allows the client to express its expectation regarding the server's push strategy to process the request. Furthermore, in W16232, a push acknowledgment (also called Push Ack) is a response modifier sent from the server to the client, which states that the server describes the push strategy used in processing the request. it can. At W16232, push directives and push acknowledgments may be communicated by sending a message that includes a DASH subprotocol frame header and one or more given arguments. The semantics of the DASH subprotocol frame header as provided in W16232 are provided in Table 1.

In W16232, the following definitions are provided for each of STREAM_ID, MSG_CODE, E, F, EXT_LENGTH, and Extension.
STREAM_ID: an identifier of the current stream, which allows multiple requests / responses to be multiplexed over the same WebSocket connection. The response to a specific request shall use the same STREAM_ID as that request. The appearance of a new STREAM_ID indicates that a new stream has been started. It is assumed that the stream identified by the conveyed STREAM_ID is closed by the cancellation request, the end of the stream, or the reception of an error.

MSG_CODE: indicates an MPEG-DASH message represented by this frame. Available message codes are defined in Table 2.

E: This field is an error flag. If this field is set, the receiver can interpret the message as an error. Additional information regarding the error may be available in the extension header.
F: Reserved EXT_LENGTH: Provides extension data of 4 bytes length preceding application data including padding.
Extension: Extension headers are additional information applicable to requests / responses in accordance with IETF RFC 7158 [The JavaScript Object Notation (JSON) Data interchange format, March 2013, which is incorporated herein by reference]. It must be a JavaScript Object Notation (JSON) encoding of the field. To align on a 4-byte boundary, 0-byte padding can be added after the extension header. Assume that the content is encoded in the UTF-8 format. The JSON encoding of the extension header shall consist of a single root level JSON object containing zero or more name / value pairs.

As shown in Table 2, each message type includes the given arguments shown in Tables 3-8. In each of Tables 3 to 8, Type indicates a data type, URI is a character string including a Uniform Resource Identifier (URI) defined by RFC3986, and String is UTF- An 8-character string, an integer, and Boolean is the value of true or false. In each of Tables 3-8, the radix indicates the number of required instances of the parameter in the message instance, and 0 indicates that parameter value signaling is optional.

  Referring to Tables 3 and 4, W16232 provides the following format for the PushDirective parameter.

The format of PushDirective in the format ABNF [IETF RFC 5234, Augmented BNF [Backus-Nur Form] for Syntax Specifications: ABNF, January 2008] is as follows:
PUSH_DIRECTIVE = PUSH_TYPE [OWS OWS QVALUE]
PUSH_TYPE = <PushType defined in [below]>
QVALUE = <qvalue described in RFC7231>
“OWS” is defined in RFC 7230 [IETF RFC 7230, Hypertext Transfer Protocol (HTTP / 1.1): Message Syntax and Routing, June 2014], section 3.2.3, and represents an optional blank. .

  When multiple push directives are applied to a request, the client can apply a quality value (“qvalue”) as described for use in RFC 7231 content negotiation. Clients can apply higher quality values to directives that they prefer over alternative directives that have lower quality values. Note that these values are hints to the server and do not mean that the server always chooses the strategy with the highest quality value.

Referring to Tables 5 and 6, W16232 provides the following format for PushAck.
The format of PushAck in ABNF format is as follows:
PUSH_ACK = PUSH_TYPE
In the formula, PUSH_TYPE is defined by [below].
W16232 provides the following formats for PushType:
The format of PushType in ABNF format is as follows:
PUSH_TYPE = PUSH_TYPE_NAM [OWS]; “OWS PUSH_PARAMS] PUSH_TYPE_NAME = DQUATE <URN> DQUATE
PUSH_PARAMS = PUSH_PARAM * (OWS ”;“ OWS PUSH_PARAM)
PUSH PARAM = 1 * VCHAR
Where
-The “<URN>” syntax is defined in RFC 2141 [IETF RFC 2141, Uniform Resource Names (URN) Syntax, May 1997].

  The messages defined for signaling push directives and push acknowledgments at W16232 may not be ideal.

  FIG. 2 is a block diagram illustrating an example system that can implement one or more of the techniques described in this disclosure. System 200 may be configured to communicate data in accordance with the techniques described herein. In the example shown in FIG. 2, the system 200 includes one or more computing devices 202A-202N, one or more content provider sites 204A-204N, a database 206, one or more media distribution engines 208A-208N, and A wide area network 210 is included. System 200 can include software modules. Software modules are stored in memory and can be executed by a processor. The system 200 can include one or more processors and a plurality of internal and / or external memory devices. Examples of memory devices include file servers, file transfer protocol (FTP) servers, network attached storage (NAS) devices, local disk drives, or any other type of data that can store data. A device or a storage medium may be mentioned. The storage medium may include a Blu-ray® disk, DVD, CD-ROM, magnetic disk, flash memory, or any other suitable digital storage medium. If the techniques described herein are partially implemented in software, the device stores software instructions in a suitable non-transitory computer-readable medium and uses one or more processors. Instructions can be executed in hardware.

  System 200 is configured to allow digital media content such as, for example, movies, live sporting events, and data and applications and their associated media presentations to be distributed to and accessed by multiple computing devices. 1 represents an example of a system that can be done. In the embodiment shown in FIG. 2, computing devices 202A-202N can include any device equipped for wired and / or wireless communication, such as a television, set top box, digital video recorder, desktop , Laptop or tablet computers, gaming consoles, eg, “smart” phones, mobile phones, and mobile devices including personal gaming devices. Although system 200 is illustrated as having a separate site, it should be noted that such illustration is for illustrative purposes and does not limit system 200 to a particular physical architecture. The functions of the system 200 and the sites included therein may be implemented using any combination of hardware implementation, firmware implementation, and / or software implementation.

  FIG. 3 is a block diagram illustrating an example computing device that can implement one or more techniques of this disclosure, wherein the computing device 300 receives data from a communication network and allows a user to access multimedia content. 2 is an example of a computing device that can be configured to: Arithmetic device 300 includes central processing unit (s) 302, system memory 304, system interface 312, audio decoder 314, audio output system 316, video decoder 318, display system 320, and I / O device (s). 322 and a network interface 324. Central processing unit (s) 302, system memory 304, system interface 312, audio decoder 314, audio output system 316, video decoder 318, display system 320, I / O device (s) 322, and network interface Each of 324 may be interconnected (physically, communicatively and / or operatively) for inter-component communication, one or more microprocessors, digital signal processors (DSPs), Various such as application specific integrated circuits (ASIC), field programmable gate arrays (FPGA), discrete logic, software, hardware, firmware, or combinations thereof It can be implemented with any suitable circuitry. Although computing device 300 is illustrated as having separate functional blocks, it should be noted that such illustration is for illustrative purposes and does not limit computing device 300 to a particular hardware architecture. . The functions of the computing device 300 can be implemented using any combination of hardware implementation, firmware implementation, and / or software implementation.

  The CPU (s) 302 may be configured to perform functions and / or process instructions for execution on the computing device 300. The CPU (s) 302 may include single core and / or multi-core central processing units. The CPU (s) 302 may be capable of retrieving and processing instructions, code, and / or data structures for performing one or more of the techniques described herein. The instructions may be stored on a computer readable medium such as system memory 304. The system memory 304 can be described as a non-transitory or tangible computer readable storage medium. In some embodiments, the system memory 304 can provide temporary and / or long-term storage. In some embodiments, system memory 304 or a portion thereof may be described as non-volatile memory, and in other embodiments, a portion of system memory 304 may be described as volatile memory. The system memory 304 may be configured to store information that may be used by the computing device 300 during operation. The system memory 304 can be used to store program instructions for execution by the CPU (s) 302, and the program running on the computing device 300 can temporarily store information during program execution. May be used to store. Further, the system memory 304 may be configured to store a number of digital media content files.

  As shown in FIG. 3, the system memory 304 includes an operating system 306, an application 308, and a browser application 310. Application 308 can include an application that is implemented within or executed by computing device 300 and is implemented or included within and operational by a component of computing device 300. Can be implemented and / or operatively / communicatively coupled. The application 308 can include instructions that can cause the CPU (s) 302 of the computing device 300 to perform a particular function. The application 308 may include an algorithm expressed by computer programming statements such as a for loop, a while loop, an if statement, a do loop. Application 308 may be developed using a specific programming language. Examples of programming languages include Java (registered trademark), Jini (registered trademark), C, C ++, Objective C, swift, Perl (registered trademark), Python (registered trademark), PhP, UNIX (registered trademark) Shell, Visual Basic and Visual Basic Script. Browser application 310 includes a logical structure that allows computing device 300 to retrieve data using resource identifiers and / or to render the retrieved data. Browser application 310 can include one or more web browsers. Browser application 310 is configured to process documents that generally correspond to website content, such as, for example, markup language documents (eg, HTML5, XML, etc.) and script language documents (eg, JavaScript (eg, JavaScript)). May be. Application 308 and browser application 310 may execute in conjunction with operating system 306. That is, the operating system 306 may be configured to facilitate application interaction with the CPU (s) 302 of the computing device 300 and other hardware components. In some embodiments, operating system 306 may be an operating system designed to be installed on a set-top box, digital video recorder, television, etc. Further, in some embodiments, operating system 306 may be an operating system designed to be installed on a dedicated computing device and / or a mobile computing device. It should be noted that the techniques described herein may be utilized by a device configured to operate using any and all combinations of software architectures.

  The system interface 312 may be configured to allow communication between components of the computing device 300. In one example, the system interface 312 includes a structure that allows data to be transferred from one peer device to another peer device or storage medium. For example, the system interface 312 is an accelerated graphics port (AGP) -based protocol, such as a peripheral component interconnect such as a PCI Express® (PCIe) bus specification managed by the Peripheral Component Interconnect Interest Group. (Peripheral Component Interconnect (PCI)) may include a chipset corresponding to a bus-based protocol, or any other form of structure that can be used to interconnect with a peer device (eg, a proprietary bus protocol) .

  As described above, the computing device 300 is configured to receive data from a communication network and extract digital media content therefrom. The extracted digital media content may be encapsulated in various packet types. The network interface 324 may be configured to allow the computing device 300 to send and receive data via a local area network and / or a wide area network. Network interface 324 may include a network interface card, such as an Ethernet card, an optical transceiver, a radio frequency transceiver, or any other type of device configured to send and receive information. The network interface 324 may be configured to perform physical signaling, addressing, and channel access control according to the physical layer and media access control (MAC) layer utilized in the network. The computing device 300 may be configured to analyze a signal generated according to any of the techniques described herein.

  The audio decoder 314 may be configured to receive and process audio packets. For example, the audio decoder 314 can include a combination of hardware and software configured to implement aspects of the audio codec. That is, audio decoder 314 may be configured to receive audio packets and provide audio data to audio output system 316 for rendering. The audio data may be encoded using a multi-channel format such as that developed by Dolby and Digital Theater Systems. The audio data may be encoded using an audio compression format. Examples of audio compression formats include the Motion Picture Experts Group (MPEG) format, Advanced Audio Coding (AAC) format, DTS-HD format, and Dolby Digital (AC-3) format. The audio output system 316 may be configured to render audio data. For example, the audio output system 316 can include an audio processor, a digital / analog converter, an amplifier, and a speaker system. The speaker system can include any of a variety of speaker systems such as headphones, an integrated stereo speaker system, a multi-speaker system, or a surround sound system.

  Video decoder 318 may be configured to receive and process video packets. For example, video decoder 318 may include a combination of hardware and software used to implement aspects of the video codec. In one example, the video decoder 318 is an ITU-T H.264 standard. 262 or ISO / IEC MPEG-2 Visual, ISO / IEC MPEG-4 Visual, ITU-T H.264. H.264 (also known as ISO / IEC MPEG-4 Advanced Video Coding (AVC)) and high-efficiency video coding (HEVC) video data encoded according to any number of video compression standards. It may be configured to decrypt. Display system 320 may be configured to retrieve and process video data for display. For example, the display system 320 can receive pixel data from the video decoder 318 and output the data for visual presentation. Further, the display system 320 may be configured to output graphics (eg, a graphical user interface) associated with moving image data. The display system 320 may be a liquid crystal display (LCD), a plasma display, an organic light emitting diode (OLED) display, or another type of display device that can present moving image data to the user. One of a variety of display devices. The display device may be configured to display standard definition content, high definition content, or ultra-high accuracy content.

  The I / O device (s) 322 may be configured to receive input and provide output during operation of the computing device 300. That is, the I / O device (s) 322 allows the user to select the multimedia content to be rendered. The input can be, for example, a pushbutton remote control, a device that includes a touch-sensitive screen, a motion-based input device, a voice-based input device, or any other type of device configured to receive user input, etc. It can be generated from an input device. The I / O device (s) 322 may be a standardized communication protocol such as Universal Serial Bus Protocol (Universal Serial Bus, USB), Bluetooth (registered trademark), ZigBee (registered trademark), or May be operatively coupled to computing device 300 using a proprietary communication protocol, such as the Infrared communication protocol.

Referring back to FIG. 2, the wide area network 210 is an example of a network configured to be able to distribute digital media content. Wide area network 210 includes a packet-based network and can operate according to a combination of one or more telecommunications protocols. The telecommunications protocol can include a dedicated aspect and / or can include a standardized telecommunications protocol.
Examples of standardized telecommunications protocols, pan-European digital mobile telephone system (Global System Mobile Communications, GSM) standard, Code Division Multiple Access (code division multiple access, CDMA) ^{standard, 3 rd Generation Partnership Project (3GPP} ) One or more of the standards, the European Telecommunications Standards Institute (ETSI) standard, the European Standard (EN), the IP standard, the Wireless Application Protocol (WAP) standard, and, for example, the IEEE 802 standard Institute of Electrical and Electronics Engineers (IEEE) standards such as (for example, Wi-Fi). Wide area network 210 may include any combination of wireless and / or wired communication media. The wide area network 210 is a coaxial cable, optical fiber cable, twisted pair cable, Ethernet (registered trademark) cable, wireless transmitter and receiver, router, switch, repeater, base station, or communication between various devices and sites. Any other device that may be useful to facilitate can be included. In one example, wide area network 210 may generally correspond to the Internet and its sub-networks.

  Further, over wide area network 210 may include radio and television service networks, including public wireless transmission networks, public or subscription-based satellite transmission networks, and public or subscription-based cable networks, and / or over-the-top (over-the-top). the top) or an internet service provider. Wide area network 210 may operate according to a combination of one or more telecommunications protocols associated with radio and television service networks. The telecommunications protocol can include a dedicated aspect and / or can include a standardized telecommunications protocol. Examples of standardized telecommunications protocols include DVB standard, ATSC standard, ISDB standard, DTMB standard, DMB standard, data over cable service interface standard (DOCSIS) standard, HbbTV standard, W3C standard. , And UPnP standards.

  Referring back to FIG. 2, content provider sites 204A-204N represent examples of sites that can transmit multimedia content. For example, a content provider site may include a studio having one or more studio content servers configured to provide multimedia files and / or streams to a media distribution engine and / or database. Database 306 may include a storage device configured to store, for example, multimedia content and associated data, such as data including descriptive data and executable interactive applications. Data associated with multimedia content can be formatted according to defined data formats such as, for example, Hypertext Markup Language (HTML), Dynamic HTML, XML, and JSON, with URLs and URIs Can be included.

  Media distribution engines 208 </ b> A- 208 </ b> N may be configured to distribute digital media content over wide area network 210. For example, media distribution engines 208A-208N may be included in one or more broadcast stations, cable television provider sites, satellite television provider sites, internet-based television provider sites, and / or so-called streaming media service sites. Media distribution engines 208A-208N may be configured to receive data including, for example, multimedia content, interactive applications, and messages and distribute the data to computing devices 202A-202N through wide area network 210.

  FIG. 4 is a block diagram illustrating an example of a media distribution engine that can implement one or more techniques of this disclosure. The media distribution engine 400 may be configured to receive data and output signals representing data for distribution over a communication network, such as the wide area network 210. For example, the media distribution engine 400 may be configured to receive digital media content and output one or more data streams. A data stream can generally refer to data encapsulated in a set of one or more data packets. As shown in FIG. 4, the media distribution engine 400 includes a central processing unit (s) 402, a system memory 404, a system interface 412, a media presentation description generation unit 414, a segment generation unit 416, and transport / network packet generation. Part 418 and a network interface 420. Each of central processing unit (s) 402, system memory 404, system interface 412, media presentation generation unit 414, segment generation unit 416, transport / network packet generation unit 418, and network interface 420 are each configured for inter-component communication. One or more microprocessors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays, which may be interconnected (physically, communicatively and / or operatively) for (FPGA), discrete logic, software, hardware, firmware, or any combination of various circuits such as combinations thereof. Although media delivery engine 400 is illustrated as having separate functional blocks, it should be noted that such illustration is for purposes of illustration and does not limit media delivery engine 400 to a particular hardware architecture. It is. The functionality of the media delivery engine 400 can be implemented using any combination of hardware implementation, firmware implementation, and / or software implementation.

  The central processing unit (s) 402, system memory 404, system interface 412 and network interface 420 are each a central processing unit (s) 302, system memory 304, system interface 312 and network interface 324 described above. It may be the same. CPU (s) 402 may be configured to implement functions and / or process instructions for execution in media distribution engine 400. The system interface 412 may be configured to allow communication between components of the media distribution engine 400. The network interface 420 may be configured to allow the media distribution engine 400 to send and receive data via a local area network and / or a wide area network. System memory 404 may be configured to store information that may be used by media distribution engine 400 during operation. The system memory 404 may include individual memory elements contained within each of the component encapsulation unit 402, transport / network packet generation unit 404, link layer packet generation unit 406, and frame builder and waveform generation unit 408. It should be noted that. For example, system memory 404 includes one or more buffers (eg, first-in first-out (FIFO) buffers) configured to store data for processing by components of media distribution engine 400. be able to.

  As shown in FIG. 4, the system memory includes an operating system 406, media content 408, and server application 410. The operating system 406 may be configured to facilitate application interaction with the CPU (s) 402 of the media distribution engine 400 and other hardware components. Media content 408 may include data that forms a media presentation (eg, a video component, audio component, subtitle component, etc.). The media presentation generator 414 may be configured to receive one or more components and generate a media presentation based on DASH. Further, the media presentation generator 414 may be configured to generate a media presentation description fragment. The segment generator 416 may be configured to receive the media component and generate one or more segments for inclusion in the media presentation. The transport / network packet generation unit 418 receives the transport package, and transport layer packets corresponding to the transport package (eg, UDP, transport control protocol (TCP), etc.) and network layer packets (eg, Ipv4, Ipv6, compressed IP packet, etc.) may be encapsulated.

  Referring back to FIG. 4, the system memory 404 includes a server application 410. As described above, the WebSocket protocol enables mutual communication between a logical client and a logical server through establishment of a bidirectional socket. The server application 410 can include instructions that can cause the CPU (s) 402 of the media distribution engine 400 to perform certain functions associated with the WebSocket protocol. FIG. 5 is a conceptual diagram illustrating an example communication flow according to one or more techniques of this disclosure. FIG. 5 shows an example message flow for carrying an MPEG-DASH media presentation over a full-duplex WebSocket session. As previously mentioned, the messages defined for signaling push directives and push acknowledgments at W16232 may not be ideal. Computing device 300 and media distribution engine 400 may be configured to exchange messages in accordance with the techniques described herein.

  In the example shown in FIG. 5, at 502, the browser application 310 sends an MPD request to the server application 410. In some embodiments, the browser application 310 may be referred to as a client, or an HTTP client, or a DASH client. Referring to Table 3, W16232 cannot define a normative JSON schema for the given arguments in Table 3. 6A-6B are computer programming lists showing exemplary JSON schemas for each of the media presentation document request messages. In one example, the browser application 310 may be configured to send an MPD request to the server application 410 based on the exemplary schema shown in FIGS. 6A-6B.

Referring back to FIG. 5, at 504, the server application 410 sends an MPD request response to the browser application 310. Referring to Table 5, W16232 cannot define a normative JSON schema for the given arguments in Table 5. Further, in Table 5, the parameter status includes the “Number” type in W16232. The JSON data type “Number” corresponds to an integer and a floating point number. For HTTP status codes, no floating point numbers are required. Thus, in some embodiments, the server application 410 may be configured to send a response message that includes a parameter status that uses the JSON data type “Integrer”. It should be noted that in this case, the JSON data type may be referred to as “Integrer” or “integer”. Referring back to Table 5, W16232 provides “mpd” as a parameter of type “MPD”. Since it does not define which JSON data type is used, it does not completely specify how MPD is encoded as a JSON encoded parameter. In one example, for the parameter “mpd”, the server application 410 may use the “string” JSON data type with encoding based on one of the following exemplary requirements.
Media presentation description (MPD) data returned by the server, after all generated line feeds (0x0A) are deleted and all generated double quotes (0x22) are replaced with single quotes (0x27) Are included in the JSON character string.
Or media presentation description (MPD) data returned by the server is included in the JSON string after all generated line feeds (0x0A) have been deleted or escape encoded.
Alternatively, the media presentation description (MPD) data returned by the server is included in the JSON character string after escape encoding.

Thus, in one example, the server application 410 can send an MPD request response to the browser application 310 and the format of the MPD request response is defined as provided in Table 9.

  In addition, FIGS. 7A-7B are computer programming lists showing exemplary schemas for each of the media presentation document request response messages. In one example, the server application 410 may be configured to send an MPD request response to the browser application 310 based on the exemplary schema shown in FIGS. 7A-7B.

  Referring back to FIG. 5, at 506, the browser application 310 sends a segment request to the server application 410. Referring to Table 4, W16232 cannot define a normative JSON schema for the given arguments in Table 4. 8A-8B are computer programming lists showing an exemplary JSON schema for each of the segment request messages. In one example, the browser application 310 may be configured to send a segment request to the server application 410 based on the exemplary schema shown in FIGS.

  Referring back to FIG. 5, at 508, the server application 410 sends a segment request response to the browser application 310. Referring to Table 6, W16232 includes “segment” as a parameter of type “Segment”. Since it does not define which JSON data type is used, it does not completely specify how the segment is encoded as a JSON encoded parameter. In one example, the server application 410 may be configured to send a segment request response, and the parameter “Segment” is not a given argument. In one example, the server application 410 may be configured to send segment data directly as binary data without JSON encoding after JSON encoded parameters including “push_ack”, “headers”, “status”. Good. 9A-9B are computer programming lists showing exemplary schemas for each of the segment request response messages. As shown in FIGS. 9A-9B, the exemplary schema does not include the parameter “Segment”. It should be noted that in each of the embodiments shown in FIGS. 9A-9B, status can have an integer data type. In one example, the server application 410 may be configured to send a segment request response to the browser application 310 based on the exemplary schema shown in FIGS. 9A-9B.

In another example, the server application 410 may be configured to send a “segment” parameter as a JSON, and the segment parameter may be based on the description provided in Table 10, so The entry to be replaced may be replaced.

  9C-9D are computer programming lists showing exemplary schemas for each of the segment request response messages. As shown in FIGS. 9C-9D, the exemplary schema includes a parameter “Segment”. It should be noted that in each of the embodiments shown in FIGS. 9C-9D, status can have an integer data type. In one example, the server application 410 may be configured to send a segment request response to the browser application 310 based on the exemplary schema shown in FIGS. 9C-9D. In one example, the schema in FIGS. 9C-9D can be used with Table 10. Referring again to FIG. 5, the session is terminated by the browser application 310 sending a cancel request to the server application 410 (510) or by the server application 410 sending a resource change notification to the browser application (512). be able to. Referring to Table 8, W16232 cannot define a normative JSON schema for the given arguments in Table 8. FIGS. 10A-10B are computer programming lists showing exemplary JSON schemas for each of the cancel request messages. In one example, the browser application 310 may be configured to send a cancel request message to the server application 410 based on the exemplary schema shown in FIGS. 10A-10B.

  Referring to Table 7, W16232 cannot define a normative JSON schema for the given arguments in Table 7. 11A-11B are computer programming lists showing exemplary schemas for each of the resource change notification messages. In one example, the server application 410 may be configured to send a cancel request message to the browser application 310 based on the exemplary schema shown in FIGS. 11A-11B. As shown in Table 7, including the header parameter may be optional. In one example, if the headers parameter is not included in an instance of the end_of_stream message, EXT_LENGTH shall be set to 0 in the DASH subprotocol frame (or in some embodiments, should be set) and the EXT_LENGTH field Assume that there is no empty JSON parameter encoding later. It should be noted that including the empty JSON string data with the additional required padding to the 4-byte boundary is a waste of bandwidth and requires unnecessary processing if the headers parameter is not present. It is. In this manner, computing device 300 and media distribution engine 400 may be configured to exchange messages in accordance with the techniques described herein.

As described above, in W16232, the format of PushType in ABNF format is as follows:
PUSH_TYPE = PUSH_TYPE_NAME [OWS]; “OWS PUSH_PARAMS] PUSH_TYPE_NAME = DQUATE <URN> DQUATE
PUSH_PARAMS = PUSH_PARAM * (OWS ”;“ OWS PUSH_PARAM)
PUSH_PARAM = 1 * VCHAR

  In some cases, PUSH_PARAM may conflict with JSON reserved characters. For example, VCHAR includes the character DQUATE ("). JSON uses DQUATE as a reserved character. In some cases, PushDirect is signaled as a JSON property but may require complex analysis. In one example, the PUSH_PARAM grammar can be defined with some simple modifications so as not to disturb the JSON encoding: In one example, the browser application 410 and the server application 510 are preceded by a DQUAUTE if the PUSH_PARAM includes a DQUATE. It may be configured to be escaped with “\” (ie,% × 5C).

In another example dealing with the double quotes described above, the following changes can be made to W16232:
Add the following to section “3.2 Convention”: PCHAR =% × 21 /% × 23-7E
Replace generated VCHAR with PPCHAR (section 6.1.2 PushType, 6.1.5 URLList, 6.1.6 URLTemplate, and 6.1.7 FastStartParams).
In 6.1.5 (URL Template), change the two DQUATEs to '(single quotes, i.e.% x27). Also, in Table 2 (Valid attributes for FastStartParams), the generated DQUATE is changed to '(single quote, ie,% x27), and''is changed to' (single quote, ie,% x27).
The above change uses single quotes instead of double quote characters. For this reason, PCHAR is defined to exclude DQUATE characters (% x22).

It should be noted that in W16232, the following sub-protocol identifiers are used for the WebSocket sub-protocol.
Sub-protocol identifier: “mpeg-dash”

In some embodiments, browser 410 and server application 510 may be configured to provide future extensibility by including version and / or year in the sub-protocol identifier. In one example, the tag-based sub-protocol identifier may be defined as follows:
Sub-protocol identifier: “tag: mpeg.chigliation.org-dash, 2016: 23009-6”
In one example, the URN-based sub-protocol identifier may be defined as follows:
Sub-protocol identifier: “urn: mpeg: dash: fdh: 2016”
Or Sub-protocol identifier: “urn: mpeg: dash: fdh: 2016: 23009-6”
In one example, the sub-protocol identifier and sub-protocol generic name may be defined as follows:
Sub-protocol identifier: “2016.fdh.dash.mpeg.chiariglione.org”
Sub-protocol general name: "MPEG-DASH-FDH-23009-6"

  In this manner, computing device 300 and media distribution engine 400 may be configured to exchange messages in accordance with the techniques described herein.

  In one or more examples, the functions described can be implemented in hardware, software, firmware, or any combination thereof. When implemented in software, this function can be stored or transmitted as one or more instructions or code on a computer-readable medium and executed by a hardware-based processing unit. The computer-readable medium corresponds to a tangible medium, such as a data storage medium or communication medium, including any medium that facilitates transfer of a computer program from one place to another, for example, according to a communication protocol. A storage medium can be included. In this manner, computer-readable media generally may correspond to (1) non-transitory tangible computer-readable storage media or (2) a communication medium such as a signal or carrier wave. Any data storage medium may be accessed by one or more computers or one or more processors to retrieve instructions, code, and / or data structures for introduction of the techniques described in this disclosure. It can be an available medium. The computer program product can include a computer-readable medium.

  By way of example, and not limitation, such computer-readable storage media can be RAM, ROM, EEPROM, CD-ROM, or other optical disk storage device, magnetic disk storage device, other magnetic storage device, flash memory, or Any other medium can be used, i.e. any other medium that can be used to store the desired program code in the form of instructions or data structures and that is accessible by the computer. Any connection is also properly termed a computer-readable medium. For example, instructions use a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technology such as infrared, wireless and microwave from a website, server, or other remote source As such, coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, wireless and microwave are included in the definition of media. However, it should be understood that computer-readable media and data storage media do not include connections, carrier waves, signals, or other transient media, but instead are directed to non-transitory tangible storage media. When used in the present invention, a disk and a disc are a compact disc (Compact Disc, CD), a laser disc, an optical disc, a digital versatile disc (Digital Versatile Disc, DVD, floppy disk, and Blu-ray® disc, which normally reproduces data magnetically and the disc emits a laser Used to optically reproduce data. Combinations of the above should also be included within the scope of computer-readable media.

  The instructions are one or more of one or more digital signal processors (DSPs), general purpose microprocessors, application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), or other equivalent integrated or discrete logic circuits. Can be executed by any processor. Thus, as used herein, the term “processor” can refer to any of the foregoing structures or any other structure suitable for the implementation of the techniques described herein. In addition, in some aspects, the functionality described herein is provided in a dedicated hardware module and / or software module configured to encode and decode, or embedded in a composite codec. Can be. This technique can also be fully implemented in one or more circuits or logic elements.

  The techniques of this disclosure may be implemented in a wide variety of devices or apparatuses, including a wireless handset, an integrated circuit (IC), or a set of ICs (eg, a chip set). Although various components, modules, or units have been described in this disclosure to highlight functional aspects of a device configured to perform the disclosed techniques, realization with different hardware units is possible. It is not always necessary. Rather, as described above, the various units may be combined with codec hardware units, or by a collection of interoperating hardware units including one or more processors as described above, along with suitable software and / or firmware. Can be provided.

  Furthermore, each functional block and various functions of the base station device and terminal device (moving image decoder and moving image encoder) used in each of the above-described implementations are generally an integrated circuit or a circuit that is a plurality of integrated circuits. Can be implemented or implemented. Circuits designed to perform the functions described herein include general purpose processors, digital signal processors (DSPs), application specific or general purpose application integrated circuits (ASICs), field programmable gate arrays (FPGAs) or other It may comprise a programmable logic device, discrete gate or transistor logic, or individual hardware components, or combinations thereof. A general purpose processor may be a microprocessor, or a processor may be a conventional processor, controller, microcontroller, or state machine. The general-purpose processor or each circuit described above may be configured with a digital circuit or an analog circuit. Furthermore, if an integrated circuit technology that replaces the current integrated circuit appears as a result of advances in semiconductor technology, an integrated circuit using this technology can also be used.

  Various embodiments have been described. These and other embodiments are within the scope of the following claims.

  This application is related to US Provisional Patent Application No. 62 / 408,008, filed October 13, 2016, claims priority from this provisional application, and is hereby incorporated by reference herein. The whole is incorporated.

Claims (13)

A method for signaling information associated with a media presentation, the method comprising:
Signaling a frame header indicating a dynamic adaptive streaming over hypertext transfer protocol message type, and signaling one or more given arguments corresponding to the message type as a JavaScript Object Notation encoding parameter. ,Method.   The method of claim 1, wherein the frame header indicates a media presentation description response message type and the provided argument indicates a media presentation description parameter.   The method of claim 2, wherein the media presentation description parameter comprises an encoded string.   The method of claim 1, wherein the frame header indicates a segment request response message type and the provided argument indicates a status parameter.   The method of claim 4, wherein the given argument includes a segment parameter, and the segment parameter includes an encoded string.   A device for signaling information associated with a media presentation, the device comprising one or more processors configured to perform any and all combinations of the steps included in claims 1-5. ,device.   A device for analyzing information associated with a media presentation, wherein the device is configured to analyze a signal generated according to any and all combinations of the steps included in claims 1-5. A device comprising the above processor.   The device according to claim 6 or 7, wherein the device comprises a media distribution engine.   The device according to claim 6, wherein the device includes a computing device.   The device is selected from the group consisting of a desktop or laptop computer, a mobile device, a smartphone, a mobile phone, a personal data assistant (PDA), a television, a tablet device, or a personal gaming device. Device described in. The device of claim 8;
10. A system comprising: the device of claim 9.   An apparatus for signaling information associated with a media presentation, the apparatus comprising means for performing any and all combinations of the steps included in claims 1-5.   A non-transitory computer readable storage medium having stored therein instructions that, when executed, cause one or more processors of the device to perform any and all combinations of the steps included in claims 1-5.