JP2023519378A - Method and apparatus for media scene description - Google Patents

Abstract

A system, method, and device for managing media storage and delivery, wherein a Media Access Facility (MAF) obtains a Graphics Language Transmission Format (glTF) file corresponding to a scene; Obtaining from the glTF file a uniform resource locator (URL) parameter pointing to the binary data blob, determining that the binary data blob has a Concise Binary Object Representation (CBOR) format, and a CBOR parser function implemented by MAF and retrieving media content corresponding to a scene based on the object, including converting a binary data blob into an object having JavaScript Object Notation (JSON) format using device.

Description

Embodiments of the present disclosure use 3D modeling syntax to support media objects, implement media syntax to support various media codecs, containers, and formats, and implement media storage via predefined programming interfaces. and system design for managing delivery methods and providing media buffer control and rendering capabilities.

Graphics Language Transmission Format (glTF) is an API-neutral runtime asset 3D modeling delivery format. Compared to traditional 3D modeling tools, glTF provides a more efficient, extensible and interoperable format for transmitting and loading 3D content. glTF 2.0 is the latest version of the glTF specification written by the Khronos 3D Group. This format supports simple scenegraph formats that can generally support static (non-timed) objects in the scene, including the 'png' and 'jpeg' image formats. glTF 2.0 supports simple animation and supports translation, rotation, and scaling of primitive shapes, ie geometric objects, described using glTF primitives. glTF 2.0 does not support timed media and therefore neither video nor audio.

''Information technology - Coding of audiovisual objects - Part 12: ISO base media file format'', ISO/IEC 14496-12 (December 2015), ''Draft of FDIS of ISO/IEC 23000-19 Common Media Application Format for Segmented Media'', ISO/IEC JTC1/SC29/WG11 MPEG117/16819 (April 2017) and ''Text of ISO/IEC FDIS 23009-1 4th edition'', ISO/IEC JTC 1/SC 29/ WG 11 N18609 (August 2019), as well as the glTF 2.0 specification, are hereby incorporated by reference in their entireties.

According to one embodiment, a method for managing storage and delivery of media is implemented by at least one processor and configured by a media access function (MAF) to provide a graphics language transmission format corresponding to a scene. (glTF) file; obtaining from the glTF file a uniform resource locator (URL) parameter pointing to a binary data blob; CBOR) format, and using a CBOR parser function implemented by MAF to convert the binary data blob into an object having JavaScript Object Notation (JSON) format; and obtaining media content corresponding to the scene based on the object.

According to one embodiment, a device for managing storage and distribution of media includes at least one memory configured to store program code, reads the program code and operates when instructed by the program code. wherein the program code causes the at least one processor to retrieve a Graphics Language Transmission Format (glTF) file corresponding to the scene via a Media Access Facility (MAF). a first retrieving code configured to cause at least one processor to retrieve uniform resource locator (URL) parameters pointing to binary data blobs from a glTF file; and at least determination code configured to cause one processor to determine that the binary data blob has a Concise Binary Object Representation (CBOR) format; and at least one processor, using a CBOR parser function implemented by MAF, Conversion code configured to convert a binary data blob into an object having a JavaScript Object Notation (JSON) format and configured to cause at least one processor to obtain media content corresponding to a scene based on the object. and at least one processor including a third retrieval code.

According to one embodiment, the non-transitory computer-readable medium, when executed by at least one processor of a device for managing storage and delivery of media, is transferred to at least one processor by a media access function (MAF). , to get the Graphics Language Transmission Format (glTF) file corresponding to the scene, and from the glTF file, to get the Uniform Resource Locator (URL) parameter pointing to the binary data blob, and the binary data blob to be the Concise Binary Object Representation ( CBOR) format, and use the CBOR parser function implemented by MAF to convert the binary data blob into an object with JavaScript Object Notation (JSON) format, and respond to the scene based on the object. Store instructions, including one or more instructions configured to cause acquisition of media content.

Further features, properties and various advantages of the disclosed subject matter will become more apparent from the following detailed description and accompanying drawings.

1 is a diagram of an environment in which the methods, apparatus, and systems described herein may be implemented, according to embodiments; FIG. 2 is a block diagram of exemplary components of one or more devices of FIG. 1, according to an embodiment; FIG. FIG. 4 is a schematic diagram of a glTF scene description object, according to an embodiment; 1 is a schematic diagram of a media scene description system reference architecture, according to an embodiment; FIG. 1 is an example glTF JavaScript Object Notation (JSON) format representation, according to an embodiment. 1 is an example MPEG glTF extension, according to an embodiment. FIG. 4 illustrates a file with JSON format, according to an embodiment; FIG. 4 illustrates a file with CBOR format, according to an embodiment; FIG. 4 illustrates an example glTF syntax, according to an embodiment; FIG. 4 is an illustration of an exemplary process for managing storage and distribution of media, according to an embodiment; FIG. 4 is an illustration of an exemplary process for managing storage and distribution of media, according to an embodiment; FIG. 4 is an illustration of an exemplary process for managing storage and distribution of media, according to an embodiment;

FIG. 1 is a diagram of an environment 100 in which the methods, apparatus, and systems described herein may be implemented, according to embodiments. As shown in FIG. 1, environment 100 may include user device 110, platform 120, and network . Devices in environment 100 may be interconnected via wired connections, wireless connections, or a combination of wired and wireless connections.

User device 110 includes one or more devices capable of receiving, generating, storing, processing, and/or providing information associated with platform 120 . For example, user device 110 can be a computing device (e.g., desktop computer, laptop computer, tablet computer, handheld computer, smart speaker, server, etc.), mobile phone (e.g., smart phone, wireless phone, etc.), wearable device (e.g., a pair of smart glasses or a smartwatch), or similar device. In some implementations, user device 110 may receive information from platform 120 and/or transmit information to the platform.

Platform 120 includes one or more devices as described elsewhere herein. In some implementations, platform 120 may include a cloud server or group of cloud servers. In some implementations, platform 120 may be designed to be modular so that software components can be interchanged according to specific needs. This allows the platform 120 to be easily and/or quickly restored for different uses.

In some implementations, the platform 120 may be hosted within a cloud computing environment 122 as depicted. In particular, although implementations described herein describe platform 120 as being hosted within cloud computing environment 122, in some implementations platform 120 may not be cloud-based. (ie, may be implemented outside of a cloud computing environment) and may be partially cloud-based.

Cloud computing environment 122 includes the environment that hosts platform 120 . A cloud computing environment 122 provides computing, software, data access, storage, etc. that does not require end-user (e.g., user device 110) knowledge of the physical location and configuration of systems and/or devices hosting platform 120. can provide services. As depicted, cloud computing environment 122 may include a group of computing resources 124 (collectively referred to as "computing resources 124" and individually referred to as "computing resources 124").

Computing resources 124 include one or more personal computers, workstation computers, server devices, or other types of computing and/or communication devices. In some implementations, computing resource 124 may be hosted by platform 120 . Cloud resources may include computing instances running within computing resources 124, storage devices provided within computing resources 124, data transfer devices provided by computing resources 124, and the like. In some implementations, computing resources 124 may communicate with other computing resources 124 via wired connections, wireless connections, or a combination of wired and wireless connections.

As further shown in FIG. 1, computing resources 124 may include one or more applications (“APP”) 124-1, one or more virtual machines (“VM”) 124-2, virtualized storage (“VS”) 124-3, one or more hypervisors (“HYP”) 124-4.

Applications 124 - 1 include one or more software applications that may be provided to user device 110 and/or platform 120 or accessed by user device 110 and/or platform 120 . Application 124 - 1 may eliminate the need to install and run software applications on user device 110 . For example, application 124 - 1 may include software associated with platform 120 and/or any other software that may be provided via cloud computing environment 122 . In some implementations, one application 124-1 can send and receive information to and from one or more other applications 124-1 via virtual machine 124-2. For example, application 124-1 may provide media streaming including, but not limited to, audio streaming, visual streaming, object description streams, scene description streams, and the like. A scene description generally refers to a descriptor that describes a scene. A scene can generally refer to any 2D, 3D, and/or immersive objects and their associated properties, commands, and/or behaviors. A scene description may be transmitted in the form of a scene graph, which is a hierarchical representation of audio, video and graphic objects. Note that scene descriptions may be sent independently of other types of streams, eg, audio streams, visual streams, object description streams, and the like.

A virtual machine 124-2 comprises a software implementation of a machine (eg, a computer) that executes programs like a physical machine. Virtual machine 124-2 may be either a system virtual machine or a process virtual machine, depending on the application by virtual machine 124-2 and the degree of correspondence with any real machine. A system virtual machine can provide a complete system platform that supports running a complete operating system (“OS”). A process virtual machine can run a single program and support a single process. In some implementations, a virtual machine 124-2 may run on behalf of a user (eg, user device 110) and perform functions of cloud computing environment 122, such as data management, synchronization, or long-term data transfer. Can manage infrastructure.

Virtualized storage 124-3 includes one or more storage systems and/or one or more devices that employ virtualization techniques within the computing resource 124 storage system or device. In some implementations, in the context of storage systems, types of virtualization may include block virtualization and file virtualization. Block virtualization can refer to the abstraction (or separation) of logical storage from physical storage such that the storage system can be accessed regardless of physical storage or heterogeneous structure. Separation may allow flexibility in how storage system administrators manage storage for end users. File virtualization can eliminate dependencies between data accessed at the file level and where the file is physically stored. This may allow optimizing storage usage, server consolidation, and/or performing smooth file migrations.

Hypervisor 124-4 may provide hardware virtualization techniques that allow multiple operating systems (eg, “guest operating systems”) to run simultaneously on a host computer such as computing resource 124. . Hypervisor 124-4 can present virtual operating platforms to guest operating systems and can manage the execution of guest operating systems. Multiple instances of different operating systems can share virtualized hardware resources.

Network 130 includes one or more wired and/or wireless networks. For example, network 130 may include cellular networks (e.g., fifth generation (5G) networks, long term evolution (LTE) networks, third generation (3G) networks, code division multiple access (CDMA) networks, etc.), public area mobile networks (PLMN), local area networks (LAN), wide area networks (WAN), metropolitan area networks (MAN), telephone networks (e.g. Public Switched Telephone Network (PSTN)), private networks, ad-hoc networks, intranets, Internet, It may include fiber optic based networks, etc., and/or combinations of these or other types of networks.

The number and placement of devices and networks shown in FIG. 1 are provided as an example. In practice, there may be more devices and/or networks, fewer devices and/or networks, different devices and/or networks, or different arrangements of devices and/or networks than those shown in FIG. good. Additionally, two or more devices shown in FIG. 1 may be implemented within a single device, or the single device shown in FIG. 1 may be implemented as multiple distributed devices. good. Additionally or alternatively, a set of devices (e.g., one or more devices) in environment 100 can perform one or more functions described as being performed by another set of devices in environment 100. .

FIG. 2 is a block diagram of exemplary components of one or more devices of FIG. Device 200 may correspond to user device 110 and/or platform 120 . Device 200 may comprise bus 210, processor 220, memory 230, storage component 240, input component 250, output component 260, and communication interface 270, as shown in FIG.

Bus 210 includes components that enable communication between components of device 200 . Processor 220 is implemented in hardware, firmware, or a combination of hardware and software. Processor 220 may be a central processing unit (CPU), graphics processing unit (GPU), accelerated processing unit (APU), microprocessor, microcontroller, digital signal processor (DSP), field programmable gate array (FPGA), application specific integration A circuit (ASIC), or another type of processing component. In some implementations, processor 220 includes one or more processors that can be programmed to perform functions. Memory 230 may be random access memory (RAM), read only memory (ROM), and/or another type of dynamic or static storage device that stores information and/or instructions for use by processor 220 (e.g., , flash memory, magnetic memory, and/or optical memory).

Storage component 240 stores information and/or software related to the operation and use of device 200 . For example, storage component 240 may include hard disks (e.g., magnetic disks, optical disks, magneto-optical disks, and/or solid-state disks), compact disks (CDs), digital versatile disks (DVDs), floppy disks, floppy disks, along with corresponding drives. May include cartridges, magnetic tapes, and/or other types of non-transitory computer-readable media.

Input component 250 includes components that enable device 200 to receive information, such as through user input (eg, touch screen display, keyboard, keypad, mouse, buttons, switches, and/or microphone). . Additionally or alternatively, input components 250 may include sensors (eg, global positioning system (GPS) components, accelerometers, gyroscopes, and/or actuators) for sensing information. Output components 260 include components that provide output information from device 200 (eg, a display, speakers, and/or one or more light emitting diodes (LEDs)).

Communication interface 270 is a transceiver-like component (e.g., transceiver and/or separate receiver and transmitter). Communication interface 270 may allow device 200 to receive information from and/or provide information to another device. For example, communication interface 270 may include an Ethernet interface, an optical interface, a coaxial interface, an infrared interface, a radio frequency (RF) interface, a universal serial bus (USB) interface, a Wi-Fi interface, a cellular network interface, and the like.

Device 200 is capable of performing one or more processes described herein. Device 200 may perform these processes in response to processor 220 executing software instructions stored by non-transitory computer-readable media, such as memory 230 and/or storage component 240 . A computer-readable medium is defined herein as a non-transitory memory device. A memory device may include memory space within a single physical storage device or spread across multiple physical storage devices.

The software instructions may be read into memory 230 and/or storage component 240 from another computer-readable medium or from another device via communication interface 270 . The software instructions stored in memory 230 and/or storage component 240, when executed, can cause processor 220 to perform one or more of the processes described herein. Additionally or alternatively, hard-wired circuitry may be used in place of or in combination with software instructions to perform one or more processes described herein. Thus, implementations described herein are not limited to any specific combination of hardware circuitry and software.

The number and arrangement of components shown in FIG. 2 are provided as an example. In practice, device 200 may include more, fewer, different, or differently arranged components than those shown in FIG. Additionally or alternatively, a set of components (e.g., one or more components) of device 200 may perform one or more functions described as being performed by another set of components of device 200. .

Referring to Figure 3, Graphics Language Transmission Format (glTF) is an application programming interface (API) neutral runtime asset 3D modeling delivery format. Compared to traditional 3D modeling tools, glTF provides a more efficient, extensible and interoperable format for transmitting and loading 3D content.

A glTF scene can be a combination of multiple glTF assets. glTF assets include, for example, scene objects 301, nodes 302, cameras 303, meshes 304, lighting 305, animations 306, accessors 307, materials 308, skins 309, buffer views 310, techniques 311, textures 312, buffers 313, programs 314, It can be a JavaScript Object Notation (JSON) formatted file containing a full scene description that can include images 315, samplers 316, shaders 317, and supporting external data.

glTF also supports external data sources that can be referenced in any of the above scene objects. In embodiments, binary files may be used for animations 306 or other buffer-based data 313 . Image files can be used for object textures 312 .

Referring to Figure 5, glTF scenes can be organized in JSON format, as described above. A glTF asset may contain zero or more scenes 503, which may be a set of visual objects to render. A scene may be defined in a scene array. In the example shown in FIG. 5, there is a single scene 506 at a single node 501, but embodiments are not so limited. Various parameters that can be associated with each node object. For example, name 502 may specify the name of a node object and scene name 504 may specify the name of a single scene.

A glTF scene asset may be consumed by a presentation engine to render a 3D or immersive scene to the user. Existing glTF syntax only supports 3D objects with static or computer-generated animation. There is no support for media types such as video or audio, let alone rendering video/audio media types.

On the other hand, existing glTF cannot describe scenes using a geographic coordinate system, and such a feature is desired for some media presentation scenarios.

Therefore, it is necessary to extend glTF to support media types including traditional 2D flat video, immersive media content such as virtual reality (VR), augmented reality (AR), extended reality (XR), and spatial audio. It is said that This may require extensions to support video/audio syntax and systems for media delivery and rendering.

The Moving Picture Experts Group (MPEG) has defined several extensions to the glTF specification to support immersive media content. Referring to FIG. 3, the new extensions are MPEG_media 330, MPEG_scene_dynamic 331, MPEG_texture_video 333, MPEG_animation_timing 332, MPEG_audio_spatial 334, MPEG_accessor_timed 335, MPEG_buffer_circular 336. In FIG. 3, in general, elements with rounded contours, such as elements 301-317, may be glTF elements, and elements with square contours, such as elements 330-336, are MPEG-based extensions of the glTF specification. , but embodiments are not limited to this.

MPEG_media may be supported if MPEG_media 330 is specified as the root identifier. Referring to Figure 6, the syntax that supports MPEG media can be declared as a high-level JSON syntax. The syntax from 601 to 604 in Figure 6 can be presented exactly as shown, if supported.

Scene updates may be expressed using the JSON patch protocol and MPEG_scene_dynamic 331 may be used to support the JSON patch protocol.

MPEG texture video extensions identified by MPEG_texture_video 333 can provide the possibility to link glTF texture objects to MPEG media and their corresponding tracks listed by the MPEG_media object. The MPEG texture video extension may also provide a reference to MPEG_accessor_timed 335, where decoded timed textures are made available.

The MPEG_audio_spatial 334 extension may support multiple audio types.

To support timed data access, the buffer element can be extended to provide circular buffer functionality. This extension is named MPEG_buffer_circular 336 and may be included as part of a glTF "buffer" object, such as buffer 313 .

The above MPEG extensions may enable the creation of immersive experiences using glTF. Finally, glTF assets with MPEG extensions can be used and loaded into the rendering engine for visualization.

Referring to FIG. 4, a reference media scene description architecture 400 illustrates an example of how MPEG extensions can be used to support media types such as audio/video. Media content is retrieved from external sources such as the media cloud 401 using media search engines and media access facilities (MAF) 402 and processed using video decoders 403, audio decoders 404, and other data compressors 405. , buffered in video buffer 406 , audio buffer 407 , and other buffers 408 and rendered by presentation engine 409 . In some cases, media content may be stored in local storage 410 . MAF provides a framework for integrating elements from several MPEG standards into a single specification suitable for specific but widely available applications. For example, MAF can specify how metadata is combined with timed media information for presentation in well-defined formats that facilitate exchange, management, editing, and presentation of media. The presentation may be "local" to the system or accessible via a network or other streaming mechanism.

Referring to FIG. 4, MPEG scene description extensions may separate presentation engine 409 from media search engine 402 . Presentation engine 409 and media search engine 402 may communicate via a predefined programming interface that allows presentation engine 409 to request media data needed to render a scene. Media search engine 402 may retrieve requested media and make it available in a timely manner and in a format that can be processed immediately by presentation engine 409 . For example, the requested media assets may reside on the network in a compressed form, so that the media search engine 402 retrieves and decodes the assets and presents the resulting media data to the presentation engine for rendering. Pass to 409. Media data may be passed from media search engine 402 to presentation engine 409 in the form of buffers. Requests for media data may be passed from presentation engine 409 to media search engine 402 via the media search API. A video decoder 403 may be used for flexible use of video decoding resources. When video decoder 403 is used, presentation engine 409 may provide information regarding input and output formats to video decoder 403 via application configuration APIs.

As mentioned above, the glTF syntax may be expressed in a JSON file. The Internet Engineering Task Force (IETF) Concise Binary Object Representation (CBOR) can represent a compact data format compared to the traditional JSON format. CBOR relates to JSON-like data objects in name/value pair format, but expressed in a binary and compact way, and has much more support by key-value types. The size of files in CBOR format may be smaller than corresponding files in JSON format. In some cases, CBOR files may be more than 50% smaller than the corresponding JSON files. CBOR is registered as "application/cbor" with the Internet Assigned Numbers Authority (IANA).

CBOR can be used as one of the similarly widely supported glTF interchangeable compressed file formats due to its compact data size and compatibility with JSON.

Information in CBOR is stored in binary form. Binary data formats may need to be parsed each time a computer or machine is used to understand the stored data, as many use cases for information involve machines to understand data, JSON or It can have speed advantages over human-readable data formats such as XML.

FIG. 7A shows an example of a file in JSON format, and FIG. 7B shows an example of a corresponding file in CBOR format. For example, the character "a" (711) in the JSON formatted file of Figure 7A may correspond to 0x61 (721) in the CBOR formatted file of Figure 7B. Similarly, the character 'b' (712) in the JSON formatted file in Figure 7A corresponds to 0x62 (722) in the CBOR formatted file in Figure 7B, and the character 'c' (713) in the JSON formatted file in Figure 7A. ) may correspond to 0x63 (723) in the CBOR formatted file of FIG. 7B.

Compared to JSON, using CBOR for scene description can bring advantages in terms of smaller data size and support for multiple key-value types instead of just string objects in JSON. A function programming interface may be used in the presented media scene description reference architecture, more precisely in the media access function module.

As support for CBOR by glTF is becoming more prevalent, such support could, for example, increase glTF file format interoperability, reduce file size in local storage or cache, and enable glTF file transfer with minimal processing power in MAF 402. May be added to MPEG scene descriptions to reduce latency.

CBOR parser functionality according to embodiments may be implemented by MAF 402 to convert CBOR input to glTF natively supported JSON format, and as a file compressor for saving large glTF files to local storage or cache 410. may be used.

The CBOR Parser API provides one of the methods such as cbor2Json(), json2Cbor, and save(), as shown in Table 1 below.

A detailed interface description can be as follows.
interface InputFileParser {
readonly attribute FILE inputFileName;
readonly attribute FILE outputFileName;
readonly attribute CBOR cborDataBlob;
FILE cbor2Json()(FILE cborInput);
FILE json2Cbor(FILE jsonInput);
FILE cbor2Json(CBOR cborDataBlob);
bool save();
};

The functionality proposed above can be used in various scenarios, for example:

Referring to Figure 8, the glTF 'url' or 'uri' syntax can point to a CBOR binary data blob (802). In embodiments, there may be two ways of identifying whether a binary is actually in CBOR data format. According to Example 1, Multipurpose Internet Mail Extensions (MIME) types specifying 'mimeTypes' in 'application/cbor' (801) may be signaled.
According to example 2, the prefix "application/cbor" may be included before the actual binary data. Examples 1 and 2 may be used together. In either case, you may parse the CBOR file format into JSON by calling a function called "cbor2Json(Object)" that takes the CBOR binary data.

If the input glTF is in CBOR format, the output may be glTF by using the cbor2Json() API.

If the input is in native glTF format, no conversion may be necessary.

For local storage or caching purposes, glTF files may be saved as CBOR using the json2Cbor() and save() interfaces.

Accordingly, embodiments may relate to methods for providing glTF file format interoperability with CBOR, reducing file sizes for local storage or caching, increasing data transfer rates, and reducing file transfer latency.

Processes 900A, 900B, and 900C for managing media storage and distribution are described below with reference to FIGS. 9A-9C.

FIG. 9A is a flowchart of an exemplary process 900A for managing media storage and distribution.

As shown in FIG. 9A, process 900A may include obtaining a glTF file corresponding to a scene (block 911) by a Media Access Facility (MAF). In embodiments, the MAF may correspond to MAF 402 .

As further shown in FIG. 9A, the process 900A may include obtaining uniform resource locator (URL) parameters pointing to the binary data blob from the glTF file (block 912).

As further shown in FIG. 9A, process 900A may include determining that the binary data blob has a CBOR format (block 913).

As further shown in FIG. 9A, process 900A may include converting the binary data blob into an object having JSON format (block 914) using the CBOR parser function implemented by MAF.

As shown in FIG. 9A, process 900A may include obtaining media content corresponding to the scene based on the object (block 914).

In embodiments, an object with JSON format may be larger than a binary data blob with CBOR format.

In embodiments, the binary data blob may be determined to have a CBOR format based on the Multipurpose Internet Mail Extensions (MIME) type signaled in the glTF file.

In embodiments, a binary data blob may be determined to have a CBOR format based on a prefix included at the beginning of the binary data blob.

In embodiments, a binary data blob may be determined to have a CBOR format based on the Multipurpose Internet Mail Extensions (MIME) type signaled in the glTF file and the prefix included at the beginning of the binary data blob.

In embodiments, the MAF may be included in the Moving Picture Experts Group (MPEG) Scene Description Architecture.

In embodiments, the CBOR parser functionality may be implemented using an application programming interface associated with MAF.

FIG. 9B is a flowchart of an exemplary process 900B for managing media storage and distribution. In embodiments, one or more blocks of process 900B may be performed in combination with one or more blocks of process 900A. For example, one or more blocks of process 900B may occur after one or more blocks of process 900A.

As further shown in FIG. 9B, process 900B may include determining that the glTF file has a CBOR format (block 921).

As shown in FIG. 9B, process 900B may include converting the glTF file into a converted glTF file having JSON format (block 922) using a CBOR parser function implemented by MAF. In embodiments, this CBOR parser function may be different than the CBOR parser function used at block 914 .

In embodiments, a converted glTF file with JSON format may be larger than a glTF file with CBOR format.

FIG. 9C is a flowchart of an exemplary process 900C for managing media storage and distribution. In embodiments, one or more blocks of process 900C may be performed in combination with one or more blocks of process 900A and/or process 900B. For example, one or more blocks of process 900C may occur after one or more blocks of process 900A or after one or more blocks of process 900B.

As shown in FIG. 9C, process 900C reconverts the converted glTF file into reconverted glTF having CBOR format using the JSON parser function implemented by MAF (block 931). can include

As further shown in FIG. 9C, process 900C may include storing the retranslated glTF file in local storage and/or cache (block 932).

Although Figures 9A-9C show exemplary blocks of processes 900A, 900B, and 900C, in some implementations, blocks of processes 900A, 900B, and 900C are shown in Figures 9A-9C. It may include more blocks, fewer blocks, different blocks, or a different arrangement of blocks than those shown. Additionally or alternatively, two or more of the process blocks of processes 900A, 900B, and 900C may be performed in parallel. In embodiments, any one or more blocks of processes 900A, 900B, and 900C may be combined in any order with any other one or more blocks of processes 900A, 900B, and 900C. , processes 900A, 900B, and 900C may be divided or combined as desired.

Furthermore, the proposed methods may be implemented by processing circuitry (eg, one or more processors or one or more integrated circuits). In one example, one or more processors execute a program stored in a non-transitory computer-readable medium to perform one or more of the proposed methods.

The foregoing disclosure, while providing illustration and description, is not exhaustive and is not intended to limit implementations to the precise forms disclosed. Modifications and variations are possible in light of the above disclosure, or may be acquired from practice of the implementation.

It will be appreciated that the systems and/or methods described herein may be implemented in different forms in hardware, firmware, or a combination of hardware and software. The actual specialized control hardware or software code used to implement these systems and/or methods is not limiting of implementations. Therefore, it should be appreciated that software and hardware may be designed to implement the systems and/or methods based on the description herein.

Even though specific combinations of features are recited in the claims and/or disclosed herein, these combinations are not intended to limit the disclosure of possible embodiments. Indeed, many of these features may be combined in ways not specifically recited in the claims and/or disclosed herein. Each dependent claim listed below may be directly dependent on only one claim, but the disclosure of possible implementations includes each dependent claim in combination with every other claim in the claim set. include.

No element, act, or instruction used herein should be construed as essential or essential unless explicitly stated as such. Also, as used herein, the articles "a" and "an" are intended to include one or more and may be used interchangeably with "one or more." Further, as used herein, the term "set" includes one or more items (e.g., related items, unrelated items, combinations of related and unrelated items, etc.) may be used interchangeably with "one or more". Where only one item is covered, the term "one" or similar language is used. Also, the terms "has," "have," "having," etc., as used herein are intended to be open-ended terms. Further, the phrase "based on" means "based at least in part on," unless stated otherwise.

100 environment
110 user devices
120 platforms
122 Cloud Computing Environment
124 computing resources
124-1 Applications
124-2 Virtual Machine
124-3 Virtual storage
124-4 Hypervisor
130 network
200 devices
210 bus
220 processor
230 memory
240 storage components
250 input components
260 output components
270 communication interface
301 scene objects, elements
302 nodes, elements
303 camera, element
304 mesh, elements
305 lighting, elements
306 animations, elements
307 accessors, elements
308 Materials, Elements
309 skins, elements
310 buffer view, element
311 techniques, elements
312 object textures, elements
313 buffers, elements
314 programs, elements
315 images, elements
316 samplers, elements
317 shaders, elements
330 MPEG_media, element
331 MPEG_scene_dynamic, element
332 MPEG_animation_timing, element
333 MPEG_texture_video, element
334 MPEG_audio_spatial, element
335 MPEG_accessor_timed, element
336 MPEG_buffer_circular, element
400 Reference Media Scene Description Architecture
401 Media Cloud
402 Media Search Engine and Media Access Facility (MAF)
403 video decoder
404 audio decoder
405 other data compressors
406 video buffer
407 Audio Buffer
408 other buffer
409 Presentation Engine
410 local storage, cache
501 single node
502 Name
503 scenes
504 scene name
506 single scene
601 syntax
602 syntax
603 Syntax
604 Syntax
711 characters
712 characters
713 characters
721 0x61
722 0x62
723 0x63
801 application/cbor
802 CBOR Binary Data Blob
900A process
900B process
900C process

Referring to Figure 5, glTF scenes can be organized in JSON format, as described above. A glTF asset may contain zero or more scenes 503, which may be a set of visual objects to render. A scene may be defined in a scene array. In the example shown in FIG. 5, there is a single scene 506 at a single node 501, but embodiments are not so limited. Various parameters may be associated with each node object. For example, name 502 may specify the name of a node object and scene name 504 may specify the name of a single scene.

As shown in FIG. 9A, process 900A may include obtaining media content corresponding to the scene based on the object (block 915 ).

Claims (20)

1. A method of managing media storage and distribution, said method being implemented by at least one processor,
obtaining a Graphics Language Transmission Format (glTF) file corresponding to the scene by a Media Access Facility (MAF);
obtaining uniform resource locator (URL) parameters pointing to binary data blobs from the glTF file;
determining that the binary data blob has a Concise Binary Object Representation (CBOR) format;
converting the binary data blob into an object having a JavaScript Object Notation (JSON) format using a CBOR parser function implemented by the MAF;
and obtaining media content corresponding to the scene based on the object. 2. The method of claim 1, wherein said object having said JSON format is larger than said binary data blob having said CBOR format. 2. The method of claim 1, wherein the binary data blob is determined to have the CBOR format based on a Multipurpose Internet Mail Extensions (MIME) type signaled in the glTF file. 2. The method of claim 1, wherein said binary data blob is determined to have said CBOR format based on a prefix included at the beginning of said binary data blob. 2. The binary data blob of claim 1, wherein the binary data blob is determined to have the CBOR format based on a Multipurpose Internet Mail Extensions (MIME) type signaled in the glTF file and a prefix included at the beginning of the binary data blob. the method of. 2. The method of claim 1, wherein the MAF is included in a Moving Picture Experts Group (MPEG) Scene Description Architecture. 2. The method of claim 1, wherein the CBOR parser functionality is implemented using an application programming interface associated with the MAF. A device for managing storage and distribution of media, said device comprising:
at least one memory configured to store program code;
at least one processor configured to read the program code and operate when instructed by the program code, the program code comprising:
first retrieving code configured to cause the at least one processor to retrieve a Graphics Language Transmission Format (glTF) file corresponding to a scene via a Media Access Facility (MAF);
a second retrieval code configured to cause the at least one processor to retrieve uniform resource locator (URL) parameters indicative of binary data blobs from the glTF file;
determination code configured to cause the at least one processor to determine that the binary data blob has a Concise Binary Object Representation (CBOR) format;
transformation code configured to cause the at least one processor to transform the binary data blob into an object having JavaScript Object Notation (JSON) format using CBOR parser functionality implemented by the MAF;
a third acquisition code configured to cause the at least one processor to acquire media content corresponding to the scene based on the object; and at least one processor, comprising: 9. The device of claim 8, wherein the object having the JSON format is larger than the binary data blob having the CBOR format. 9. The device of claim 8, wherein the binary data blob is determined to have the CBOR format based on a Multipurpose Internet Mail Extensions (MIME) type signaled in the glTF file. 9. The device of claim 8, wherein said binary data blob is determined to have said CBOR format based on a prefix included at the beginning of said binary data blob. 9. The binary data blob of claim 8, wherein the binary data blob is determined to have the CBOR format based on a Multipurpose Internet Mail Extensions (MIME) type signaled in the glTF file and a prefix included at the beginning of the binary data blob. device. 9. The device of claim 8, wherein the MAF is included in a Moving Picture Experts Group (MPEG) Scene Description Architecture. 9. The device of claim 8, wherein the CBOR parser functionality is implemented using an application programming interface associated with the MAF. A non-transitory computer-readable medium storing instructions which, when executed by at least one processor of a device for managing storage and distribution of media, causes the at least one processor to:
Acquire the Graphics Language Transmission Format (glTF) file corresponding to the scene by the Media Access Facility (MAF),
Obtain uniform resource locator (URL) parameters pointing to binary data blobs from the glTF file;
determining that the binary data blob has a Concise Binary Object Representation (CBOR) format;
converting the binary data blob into an object having a JavaScript Object Notation (JSON) format using a CBOR parser function implemented by the MAF;
A non-transitory computer-readable medium comprising one or more instructions configured to cause acquisition of media content corresponding to the scene based on the object. 16. The non-transitory computer-readable medium of claim 15, wherein said object having said JSON format is larger than said binary data blob having said CBOR format. 16. The non-transitory computer-readable medium of claim 15, wherein said binary data blob is determined to have said CBOR format based on a Multipurpose Internet Mail Extensions (MIME) type signaled in said glTF file. 16. The non-transitory computer-readable medium of claim 15, wherein said binary data blob is determined to have said CBOR format based on a prefix included at the beginning of said binary data blob. 16. The non-transitory computer-readable medium of claim 15, wherein the MAF is included in a Moving Picture Experts Group (MPEG) Scene Description Architecture. 16. The non-transitory computer-readable medium of claim 15, wherein the CBOR parser functionality is implemented using an application programming interface associated with the MAF.

Images