WO2017195206A1

WO2017195206A1 - Systems and methods for adapting multi-media content objects

Info

Publication number: WO2017195206A1
Application number: PCT/IL2017/050524
Authority: WO
Inventors: Doron Segev; Mike BOGDANOVSKY; Efraim Atad; Michael Birnboim
Original assignee: Showbox Ltd.
Priority date: 2016-05-11
Filing date: 2017-05-11
Publication date: 2017-11-16

Abstract

There is provided a method comprising: accessing a rich video document (RVD) comprising a multi-layer hierarchical data structure mapping hierarchical associations of rendered content items arranged such that a node at a relatively higher layer nests rendered content item(s) of node(s) at a relatively lower layer; receiving instructions to adapt rendering instructions stored by a certain node at a certain layer of the multi-layer hierarchical data structure; rendering content item(s) according to the adapted instructions, and storing the rendered content item(s) in a designated storage location of the certain node; rendering other content item(s) according to instructions of the other node(s) that nest(s) the certain node, and storing the rendered other content item(s) in respective designated storage location of the other node(s); and providing the rendered other content item(s) for presentation on the display by the player during presentation of the multi-media content object.

Description

SYSTEMS AND METHODS FOR ADAPTING MULTI-MEDIA

CONTENT OBJECTS

BACKGROUND

The present invention, in some embodiments thereof, relates to multi-media content objects and, more specifically, but not exclusively, to systems and methods for adapting multi-media content objects.

Software packages allow users, which may be professionals or amateurs, to create videos and other computer graphic based multi-media objects themselves. The software tool provides the user with the ability, for example, to insert content items, remove content items, edit content items, apply special effects (e.g., change the colors, apply special filters), and assemble the content items into a sequence (e.g., slide show).

SUMMARY

According to an aspect of some embodiments of the present invention there is provided a computer implemented method for adapting a multi-media content object, comprising: accessing a rich video document (RVD) representing a multi-media content object, the RVD comprising instructions for rendering by rendering code for presentation on a display by a player, the RVD comprising a multi-layer hierarchical data structure mapping hierarchical associations of content items rendered according to rendering instructions of at least one content item, the hierarchical data structure arranged such that a node at a relatively higher layer nests at least one content item rendered using rendering instructions of at least one node at a relatively lower layer; accessing a storage device including allocated storage comprising at least one storage location, wherein each of the at least one storage location is allocated to store a rendering of respective content items rendered according to the instructions of the associated respective node of the RVD; receiving instructions to adapt the rendering instructions stored by a certain node at a certain layer of the multi-layer hierarchical data structure of the RVD; rendering at least one content item according to the adapted instructions of the certain node, and storing the rendered at least one content item in a designated storage location of the certain node; rendering at least one other content item according to instructions of the at least one other node that nests the certain node, and storing the rendered at least one other content item in respective designated storage location of the at least one other node; and providing the rendered at least one other content item for presentation on the display by the player during presentation of the multi-media content object.

Optionally, the multi-media content object comprises a video clip, wherein at least some higher layer nodes represent at least a scene or a sub-scene of the video clip, and wherein at least some lower layer nodes store content items selected from the group consisting of: audio, animation, video clip, overlay, and title.

Optionally, at least one node of the multi-layer hierarchical data structure comprises instructions for rendering text.

Optionally, the multi-layer hierarchical data structure comprises a tree or graph with respective layers represented by nodes comprising branching-nodes and leaf- nodes, wherein relatively higher nodes of the tree nest content objects of relatively lower nodes of the tree, and a root node of the tree represents the multi-media content object by nesting all lower nodes and leafs of the tree.

Optionally, the RVD instructions include at least one of: content items for embedding in the multi-media content object, references to content items, and metadata.

Optionally, instructions of respective nodes of the RVD are stored in respective storage locations within the allocated storage.

Optionally, the RVD instructions stored at each node comprise at least one of a spatial and a temporal association of the at least one content item associated with the respective node, with the multi-media content object. Optionally, the RVD instructions are selected from the group consisting of: placement coordinates, in time marker, out time marker, and z-index representing a GUI-layer.

Optionally, the RVD instructions at each node comprise instructions for insertion of the rendered content item into the multi-media content object. Optionally, the instructions for insertion of the rendered content item are selected from the group consisting of: a link or address to a source media file storing the content item, offset to start, duration to play, and cropping information.

Optionally, in response to the received instructions to adapt the rendering instructions, rendering the at least one content item, rendering the at least one other content item, and providing the rendered at least one other content item, are iterated. Optionally, the rendering code creates the multi-media content object by retrieving the rendered at least one content item from the respective storage location of the allocated storage.

Optionally, the instructions to adapt rendering instructions are selected from the group consisting of: adding at least one new node to the multi-layer hierarchical data structure, removing at least one existing node from the multi-layer hierarchical data structure, modifying a temporal phase, modifying a GUI layer, modifying a spatial placement in a frame, and modifying the content item.

Optionally, at least one node stores at least one member selected from the group consisting of: segmentation data, classification data, and audio data.

Optionally, the method further comprises clearing the storage location of the allocated storage storing the rendered content item associated with the adapted certain node storing the adapted rendering instructions; and clearing at least one other storage location storing rendered content items associated with at least one other node at a relatively higher layer than the certain node, the at least one other node nesting the certain node. Optionally, the clearing at least one other storage location is performed recursively from the certain node to the next higher layer node nesting the current node. Alternatively or additionally, clearing the storage location and clearing the at least one other storage location is performed according to a set-of-rules that define at least one of: which stored components of the respective node to clear and which stored components of the respective node to maintain. Optionally, the set-of-rules define at least one of: clearing data that is adapted by the instructions to adapt rendering instructions, and maintaining data that is not adapted by the instructions to adapt rendering instructions. Alternatively or additionally, the set-of-rules define the at least one of clearing and maintaining the stored components according to at least one of: spatial and temporal adaptation defined by the instructions to adapt rendering instructions.

According to an aspect of some embodiments of the present invention there is provided a system for adapting a multi-media content object, comprising: a database storing a rich video document (RVD) representing a multi-media content object, the RVD comprising instructions for rendering by rendering code for presentation on a display by a player, the RVD comprising a multi-layer hierarchical data structure mapping hierarchical associations of content items rendered according to rendering instructions of at least one content item, the hierarchical data structure arranged such that a node at a relatively higher layer nests at least one content item rendered using rendering instructions of at least one node at a relatively lower layer; a storage device including allocated storage comprising at least one storage location, wherein each of the at least one storage location is allocated to store a rendering of respective content items rendered according to the instructions of the associated respective node of the RVD; a program store storing code; and at least one processor coupled to the database, the storage device, and the program store for implementing the stored code, the code comprising: code to receive instructions to adapt the rendering instructions stored by a certain node at a certain layer of the multi-layer hierarchical data structure of the RVD; code to render at least one content item according to the adapted instructions of the certain node, and storing the rendered at least one content item in the designated storage location of the certain node, and to render at least one other content item according to instructions of the at least one other node that nests the certain node, and storing the rendered at least one other content item in respective designated storage location of the at least one other node; and code to provide the rendered at least one other content item for presentation on the display by the player during presentation of the multi-media content object.

Optionally, the database, the storage device, and the program store are stored by a server in network communication with a client terminal in communication with the display, wherein the client terminal stores the Player that presents the multi-media content object on the display, and wherein the instructions to adapt the rendering instructions are received from a multi-media authoring tool stored on the client terminal.

According to an aspect of some embodiments of the present invention there is provided a computer program product comprising a non-transitory computer readable storage medium storing program code thereon for implementation by a processor of a system for adapting a multi-media content object, comprising: instructions to access a rich video document (RVD) representing a multi-media content object, the RVD comprising instructions for rendering by rendering code for presentation on a display by a player, the RVD comprising a multi-layer hierarchical data structure mapping hierarchical associations of content items rendered according to rendering instructions of at least one content item, the hierarchical data structure arranged such that a node at a relatively higher layer nests at least one content item rendered using rendering instructions of at least one node at a relatively lower layer; instructions to access a storage device including allocated storage comprising at least one storage location, wherein each of the at least one storage location is allocated to store a rendering of respective content items rendered according to the instructions of the associated respective node of the RVD; instructions to receive instructions to adapt the rendering instructions stored by a certain node at a certain layer of the multi-layer hierarchical data structure of the RVD; instructions to render at least one content item according to the adapted instructions of the certain node, and storing the rendered at least one content item in the designated storage location of the certain node; instructions to render at least one other content item according to instructions of the at least one other node that nests the certain node, and storing the rendered at least one other content item in respective designated storage location of the at least one other node; and instructions to provide the rendered at least one other content item for presentation on the display by the player during presentation of the multi-media content object.

Unless otherwise defined, all technical and/or scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the invention pertains. Although methods and materials similar or equivalent to those described herein can be used in the practice or testing of embodiments of the invention, exemplary methods and/or materials are described below. In case of conflict, the patent specification, including definitions, will control. In addition, the materials, methods, and examples are illustrative only and are not intended to be necessarily limiting.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

Some embodiments of the invention are herein described, by way of example only, with reference to the accompanying drawings. With specific reference now to the drawings in detail, it is stressed that the particulars shown are by way of example and for purposes of illustrative discussion of embodiments of the invention. In this regard, the description taken with the drawings makes apparent to those skilled in the art how embodiments of the invention may be practiced. In the drawings:

FIG. 1 is a flowchart of a method for creating an adapted multi-media content object by re-using pre-rendered content item(s), in accordance with some embodiments of the present invention;

FIG. 2 is a block diagram of a system that includes an allocated storage that defines allocated storage locations for storing the pre-rendered content item(s), in accordance with some embodiments of the present invention;

FIG. 3 is an exemplary representation of a multi-layer hierarchical data structure, in accordance with some embodiments of the present invention;

FIG. 4 is a flowchart of an exemplary method for processing nodes of the multilayer hierarchical data structure according to the adaptation instructions, in according to some embodiments of the present invention;

FIG. 5 is a schematic depicting creation of the multi-media content object (or portion thereof) based on re-using rendered content item(s), in accordance with some embodiments of the present invention; and

FIG. 6 is a flowchart depicting an iterative process of creating a multi-media content object, in accordance with some embodiments of the present invention.

DETAILED DESCRIPTION

An aspect of some embodiments of the present invention relates to systems and/or methods (e.g., processor(s) executing code instructions based on the described method) that use a multi-layer hierarchical data structure that defines the storage of rendered content items used in creation of an adapted version of a multi-media content object (e.g., video clip), by optionally clearing the storage location(s) of existing data, rendering a new version of content item(s) based on adapted rendering instructions, and storing the adapted content item(s) in the storage location. Other content items that are not associated with (or not impacted by) the adaptation instructions are maintained in their respective storage locations (i.e., without clearing and re -rendering). The multimedia content object may be rendered (e.g., by rendering code) by retrieving and re- using the rendered content items from respective storage locations. The created multimedia content object may be presented on a display (e.g., of a client terminal) by a player, optionally an application designed to present and play video clips, optionally implemented as a graphical user interface (GUI)-player. Updated versions of the multi- media content object may be created by rendering the content items according to adaptation instructions, without necessarily re-rendering content items that are not associated and/or not impacted by the rendering instructions. The unaffected content items may be retrieved from respective storage locations storing stored content items that were previously rendered (i.e., before the current adaptation instructions). Using the multi-layer hierarchical data structure, users may quickly adapt multi-media content objects, without having to wait for re-rendering of the entire (or part of the) multi-media content object, since (optionally only) the content items impacted by the adaptation instructions are re -rendered, while (optionally only) the rendered content items not impacted by the adaptation instructions are already available and are not re-rendered. For example, when a user fixes a typo in a title of the video, the title is re-rendered and the video is re-created using the previously rendered other content items, instead of, for example, re-rendering the entire video or scene.

The multi-media content object is represented by a rich video document (RVD) that includes instructions for rendering of the multi-media content object by rendering code for presentation on a display by the player.

The RVD is represented based on a multi-layer hierarchical data structure that defines hierarchical associations of content items rendered according to rendering instructions stored by the nodes. Nodes located at relatively higher layers nest the content item(s) rendered using rendering instructions of the lower layer nodes.

The content of each node is stored in a storage location of allocated storage (i.e., stored by a storage device, for example, such as a memory, or hard drive). Each allocated storage location stores at least a rendering of the content item(s) associated with the respective node, for example, the memory allocated to each storage location is selected to be sufficiently large to store the rendered content item(s).

Based on instructions to adapt one or more content items of certain node(s) of the multi-media content object, the stored content of the allocated storage location defined by the certain node(s) affected by the instructions is cleared (or marked to allow overwrite of the memory contents). The adapted content item(s) are rendered according to the adapted rendering instructions, and stored in the cleared allocated storage location.

The clearing and rendering is iterated for nodes at higher layers that nest lower layer node(s), optionally until the root node. Nodes that are not directly linked to the certain nodes(s) affected by the rendering instructions are not processed (i.e., the stored content items are not cleared and/or re-rendered). For example, when the multi-layer hierarchical data structure is a graph or tree, sub-trees or sub-graphs that do not include the affected certain nodes are not cleared and/or re-rendered.

The rendering code creates the multi-media content object based on the adaptation, by using the stored previously rendered content items, in addition to the newly updated content items.

The systems and/or methods (e.g., implemented by the processor(s) executing code instructions) described herein provide a technical solution to the technical problem of improving the process of adapting multi-media content objects. Rendering of multimedia content objects (e.g., video clips) is a computationally intensive task, requiring substantial processing resources, which may take a significantly long time. Multiple connected processors may be required to render and re -render video clips quickly enough to allow a user to create and edit videos. When the user repeatedly edits the video (e.g., in the process of creating the video), the computational intensive tasks are iterating to show the intermediate video results to the user, further increasing the demand for computational and/or processing resources. Exemplary computationally intensive tasks in rendering of video clips include:

• Decoding of raw media (also referred to as a reel) such as video clips, audio clips and images. The raw media is decoded to a raw uncompressed format that enables its manipulation to create the output clip.

• Rendering of two dimensional (2D) or three dimensional (3D) computer generated graphics or animation.

• Video compositing, where multiple video sources are superimposed temporally and spatially to achieve a desired composition, video homography and

2D/3D projection of images. • Audio processing, such as filtering, adding effects and modulating the volume of various sources, as well as audio mixing of multiple sources into the soundtrack of the output clip.

• Video effects.

· Segmentation of objects in sources such as by using a green-screen technique, or via algorithmic methods.

The systems and/or methods (e.g., implemented by the processor(s) executing code instructions) described herein improve performance of a computer (e.g., client terminal, and/or server) and/or a network. The multi-layer hierarchical data structure defines the storage of rendered content items, which reduces the processing resources required to create an updated adapted version of the multi-media content object, by reusing the existing rendered content items instead of re-rendering content items unaffected by adaptation instructions. The multi-layer hierarchical data structure, which maps nesting associations between nodes, defines the nodes that are affected by the adaption instructions, which allows for rendering the affected nodes, while re-using the previously rendered content items of unaffected nodes. The improvements to the computer include, for example, reduced computation time for applying the adaptation instructions to create the updated multi-media content object (e.g., using existing resources). The improvements to the network include, for example, reduced network traffic when a client terminal running tools for creating and/or amending multi-media content objects is communicating with a server running the rendering code.

The systems and/or methods (e.g., implemented by the processor(s) executing code instructions) described herein create new data, in the form of the multi-layer hierarchical data structure that defines the allocation of storage resources of allocated storage hosted by a storage device and/or memory, as described herein.

Accordingly, the systems and/or methods described herein are necessarily rooted in computer technology to overcome an actual technical problem arising in multi-media content object processing, and/or network communication.

Before explaining at least one embodiment of the invention in detail, it is to be understood that the invention is not necessarily limited in its application to the details of construction and the arrangement of the components and/or methods set forth in the following description and/or illustrated in the drawings and/or the Examples. The invention is capable of other embodiments or of being practiced or carried out in various ways.

The present invention may be a system, a method, and/or a computer program product. The computer program product may include a computer readable storage medium (or media) having computer readable program instructions thereon for causing a processor to carry out aspects of the present invention.

The computer readable storage medium can be a tangible device that can retain and store instructions for use by an instruction execution device. The computer readable storage medium may be, for example, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. A non- exhaustive list of more specific examples of the computer readable storage medium includes the following: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, and any suitable combination of the foregoing. A computer readable storage medium, as used herein, is not to be construed as being transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission media (e.g., light pulses passing through a fiber-optic cable), or electrical signals transmitted through a wire.

Computer readable program instructions described herein can be downloaded to respective computing/processing devices from a computer readable storage medium or to an external computer or external storage device via a network, for example, the Internet, a local area network, a wide area network and/or a wireless network. The network may comprise copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers and/or edge servers. A network adapter card or network interface in each computing/processing device receives computer readable program instructions from the network and forwards the computer readable program instructions for storage in a computer readable storage medium within the respective computing/processing device. Computer readable program instructions for carrying out operations of the present invention may be assembler instructions, instruction- set- architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state-setting data, or either source code or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, C++ or the like, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The computer readable program instructions may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet Service Provider). In some embodiments, electronic circuitry including, for example, programmable logic circuitry, field-programmable gate arrays (FPGA), or programmable logic arrays (PLA) may execute the computer readable program instructions by utilizing state information of the computer readable program instructions to personalize the electronic circuitry, in order to perform aspects of the present invention.

Aspects of the present invention are described herein with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer readable program instructions.

These computer readable program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks. These computer readable program instructions may also be stored in a computer readable storage medium that can direct a computer, a programmable data processing apparatus, and/or other devices to function in a particular manner, such that the computer readable storage medium having instructions stored therein comprises an article of manufacture including instructions which implement aspects of the function/act specified in the flowchart and/or block diagram block or blocks.

The computer readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process, such that the instructions which execute on the computer, other programmable apparatus, or other device implement the functions/acts specified in the flowchart and/or block diagram block or blocks.

The flowchart and block diagrams in the Figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems that perform the specified functions or acts or carry out combinations of special purpose hardware and computer instructions.

As used herein, the term render may refer to processing of audio data and/or other non-image data, for example, mixing audio tracks.

As used herein, the term layer of the multi-layer hierarchical data structure refers to the association between nodes of the hierarchical data structure (e.g. nesting), which may be implemented using other relationships, for example, a directed graph. The higher layer node means the node that is associated with the lower layer node, for example, a node that nests and/or inherits other node(s).

Reference is now made to FIG. 1, which is a flowchart of a method for creating an adapted multi-media content object by re-using pre-rendered content item(s) and rendering content item(s) associated with the adaptation instruction, in accordance with some embodiments of the present invention. Reference is also made to FIG. 2, which is a block diagram of a system 200 that includes an allocated storage 202 that defines allocated storage locations for storing the pre-rendered content item(s) for creation of the adapted multi-media content object, in accordance with some embodiments of the present invention. System 200 may implement the acts of the method of FIG. 1, for example, by a computing unit 204 that includes a processing unit 206 that executes code instructions (based on the method) stored in a program store 208.

Computing unit 204 may be implemented as, for example, a client terminal, a server, a mobile device, a desktop computer, a thin client, a Smartphone, a Tablet computer, a laptop computer, a wearable computer, glasses computer, and a watch computer. It is noted that in another implementation (as described herein in additional detail), one or more functions described herein as being performed by computing 204 (acting as a server) may be performed by a client terminal 212 in network communication with server 204 over a network 214 using a network interface 216.

Processing unit 206 may be implemented, for example, as a central processing unit(s) (CPU), a graphics processing unit(s) (GPU), field programmable gate array(s) (FPGA), digital signal processor(s) (DSP), and application specific integrated circuit(s) (ASIC). Processing unit(s) 206 may include one or more processors (homogenous or heterogeneous), which may be arranged for parallel processing, as clusters and/or as one or more multi core processing units.

Processing unit 206 may be implemented, for example, as a central processing unit(s) (CPU), a graphics processing unit(s) (GPU), field programmable gate array(s) (FPGA), digital signal processor(s) (DSP), and application specific integrated circuit(s) (ASIC). Processing unit(s) 206 may include one or more processors (homogenous or heterogeneous), which may be arranged for parallel processing, as clusters and/or as one or more multi core processing units. Program store 208 stores code instructions implementable by processing unit 206, for example, a random access memory (RAM), read-only memory (ROM), and/or a storage device, for example, non-volatile memory, magnetic media, semiconductor memory devices, hard drive, removable storage, and optical media (e.g., DVD, CD- ROM).

Computing unit 204 may include a data repository 210 for storing data, for example, one or more of: an RVD repository 210A storing RVDs (e.g., a database), a multi-layer hierarchical data structure repository 210B storing multi-layer hierarchical data structures, a player 2 IOC, optionally a GUI-player, that presents the multi-media content object, a multi-media editing and/or creation tool 210D that allows a user to create and/or edit multi-media content objects (e.g., videos), and rendering code 210E that creates (i.e., renders) the multi-media content object. It is noted that player 210C and tool 210D may be a single application, or two independent applications. Data repository 210 may be implemented as, for example, a memory, a local hard-drive, a removable storage unit, an optical disk, a storage device, and/or as a remote server and/or computing cloud (e.g., accessed using a network connection). It is noted that player 2 IOC and/or tool 210D and/or rendering code 210E may be stored on client terminal 212, for example, in an implementation of system 200 in which client terminal 212 uses network 214 to access server 204 to remotely perform one or more functions of the method described with reference to FIG. 1.

Allocated storage 202 includes storage space allocated for storing renderings of content items, as described herein. Allocated storage 202 may be implemented as, for example, temporary memory, permanent memory, high-speed memory, low-speed memory, storage, a memory device integrated with the computer, an external storage unit (e.g., accessed by a cable, over a wireless link, and/or over a network connection), a primary storage, a main memory, semiconductor memory, a hard-drive, an optical drive, a random access memory, volatile memory, non-volatile memory, a central processing unit (CPU) memory, a cache, a solid-state drive, built-in memory, removable memory, and/or other data storage hardware implementations. Allocated storage 202 may be implemented, for example, within program store 208, within data repository 210, by an external (e.g., permanent or removable) storage unit 220, and/or by server(s) 218 acting as a remote storage device (e.g., located as a network node and/or within a computing cloud).

Network interface 216 may be implemented, for example, as one or more of, a network interface card, a wireless interface to connect to a wireless network, a physical interface for connecting to a cable for network connectivity, a virtual interface implemented in software, network communication software providing higher layers of network connectivity, and/or other implementations.

Computing unit 204 may access one or more remote servers 218 using network 214, for example, when content item(s) referenced in the RVD are remotely stored by server(s) 218.

Computing unit 204 includes or is in communication with one or more user interfaces 222 that provide a user with the ability to enter data and/or view presented data. Exemplary user interfaces 222 include, for example one or more of, a touchscreen, a display, a keyboard, a mouse, and voice activated software using speakers and microphone.

Computing unit 204 acting as a server may communicate with client terminal(s) 212, for example, to providing software as a service (SaaS) to client terminal(s) 212, provide an application that performs the clearing and rendering of content items (as described herein) for local download to client terminal(s) 212, and/or providing functions using a remote access session to client terminal(s) 212 such as through a web browser.

At 102, computing unit 204 accesses a rich video document (RVD), for example, stored in a database, stored in RVD repository 21 OA, or provided by code that dynamically creates the RVD on the fly. Each RVD represents a multi-media content object, for example, a video clip. The RVD includes instructions for rendering the multi-media content object, for example, by rendering code 210Efor presentation on a display (e.g., user interface 222) by player 210C.

The RVD instructions include content item(s) that embedded in the created multi-media content object, references to content items to insert into the created multi- media content object, and metadata. Example content items include text, still images, video clips, audio, animations, overlay, title, and/or pictures. The content item may include a video clip including an alpha channel, for example, using Portable Network Graphics (PNG) encoding used for alpha compositing when superimposed in the containing node (e.g., animated title with transparency). Metadata may include, for example, a font, a color, a common look (e.g., vintage), and applied special effects (e.g., a filter, lighting).

The RVD instructions may be adapted (as described herein), for example, by a user using multi-media editing and/or creation tool 210D.

The RVD instructions for rendering the multi-media content object may include, for example, placement coordinates, in time marker, out time marker, and z-index representing a GUI-layer.

The RVD may be implemented, for example, based on JavaScript Object

Notation (JSON), Extensible Markup Language (XML), Structured Query Language (SQL) database records, document based database records, and other implementations.

As used herein, the term RVD may sometimes be interchanged with the term multi-layer hierarchical data structure, for example, when the RVD is implemented using the multi-layer hierarchical data structure described herein.

The RVD is arranged as a multi-layer hierarchical data structure mapping hierarchical associations between content items (and/or other elements) of the multimedia content object stored at respective nodes of respective layers. The hierarchical data structure is arranged such that nodes at relatively higher layers nest content items (and/or other elements) of nodes at relatively lower layers. The multi-layer hierarchical data structure may be implemented, for example, as a graph (e.g., directed and/or acyclic graph), a tree, and using pointers between data objects.

Optionally, when the multi-media content object is a video clip, the nodes of the RVD represent a scene or a sub-scene of the video clip (arranged in a hierarchy where each scene nests sub-scenes). Each scene or sub-scene node stores content items used in the respective scene.

Optionally, the multi-layer hierarchical data structure is implemented as a tree or graph. Layers are represented by nodes. Higher layers include branching-nodes. The lowest layer includes leaf-nodes. Relatively higher nodes of the tree nest content objects of relatively lower nodes of the tree. A root node of the tree represents the multi-media content object by nesting all lower nodes and leafs of the tree. Content items may be rendered (e.g., by rendering code 210E) according to rendering instructions stored in the node, defined using a suitable computer graphic representation, for example, Text Markup Language (HTML), HTML5, Document Object Model (DOM) manipulations using HTML, JavaScript, and Cascading Style Sheets (CSS) code (e.g., CSS3), according to the World Wide Web Consortium (W3C) specifications, and Adobe After Effects. Optionally, one or more nodes store instructions for rendering text.

Node(s) may store rendering instructions to combine video and audio (which may be stored by the respective node, or by other nodes, such as lower-layer nodes that may be nested by the respective node. The rendering instructions may include instructions to multiplex the audio (e.g., based on AC-3 encoding) and video data (e.g., based on H.264 encoding), for example, in a manner that maintains audiovisual synchronization (sometimes termed lip sync). The multiplexing instructions may be defined according to a rendering format, for example, MP4.

Reference is now made to FIG. 3, which is an exemplary representation of a multi-layer hierarchical data structure 300, in accordance with some embodiments of the present invention. Multi-layer hierarchical data structure 300 may be represented as a tree or graph. Root node 302 represents the multi-media content object defined by the RVD. Higher layer branching nodes (one exemplary node 304 is shown for clarity) represent the main elements of the video, organized as the highest layer (i.e., one below root node), for example, main scenes, main audio, main overlay (e.g., image(s) and/or video(s) that are shown over another image(s) and/or video(s). The overlayed image(s) and/or video(s) may be partially transparent such that the underlying image(s) and/or video(s) are at least partially visible), and main title (e.g., title of the video, which may appear as written text presented in an opening scene). Relatively lower branching nodes (one exemplary node 306 is shown for clarify) represent sub-elements of a main element, for example, sub-scenes of a main scene, and animation (e.g., created using a computer graphic application) and/or audio (e.g., recorded audio and/or created using a sound application. Audio may be, for example, music, human voice, sound effects) used in a main overlay. The lowest layer leaf-nodes (one exemplary leaf-node 308 is shown for clarity) represent content items used by the higher layer branching node, for example, images, video clips (e.g., recorded video), animation, title, and audio. As described herein, leaf-nodes (e.g., 308) are nested by higher layer branching-nodes (e.g., 306), which are nested (optionally iteratively) by higher layer branching-nodes (e.g., 304), which are nested by root node 302. The higher layer branching-nodes may store rendering instructions for rendering a portion of the video using content items stored in lower layer nodes, for example, instructions to segment an item(s) and combine with a background, instructions to multiplex audio and video, and instructions to combine multiple content items (e.g., temporally and/or spatially).

Referring now back to block 102 of FIG. 1, optionally, the RVD instructions stored at each node include spatial and/or temporal association of the content item associated with the respective node with the multi-media content object. For example, placement coordinates of the content item within the multi-media content object, mark in/out time markers, and z-index (i.e., GUI layer). Alternatively or additionally, the RVD instructions at each node include instructions for insertion of the rendered content item into the multi-media content object. For example, the location of the content item (e.g., a file path, a uniform resource locator (URL), and a network address), offset to start, duration to play, and cropping information. Optionally, content items (and/or metadata and/or other rendering instructions) of relatively higher nodes are rendered as the superimposition of content items of relatively lower nodes that are nested by the higher node. The rendering based on superimposition may be performed recursively according to the defined hierarchy of the nodes.

Alternatively or additionally, the RVD instructions stored at respective nodes include segmentation data. The segmentation data may include instructions to segment an image or video, and/or may include the segmented content item. The segmentation data may include the background for rendering with the segmented content item. The segmentation data may include instructions for rendering the background with the segmented content item.

Alternatively or additionally, the RVD instructions stored at respective nodes include classification data. The classification data may be based on identification of objects in the image or video scene, for example, a person, a male, a female, a child, a dog, a bike, and a tree. The classification data may be based on identification of other parameters of the image and/or scene, for example, dominant colors, an indoor or outdoor scene, and a day or night scene. The classification data may be automatically generated by image analysis code, obtained from metadata associated with the image or video, and/or manually entered by a user.

Alternatively or additionally, the RVD instructions stored at respective nodes include audio data. The audio data may define one or more parameters for an audio content item, which may be stored by the respective node, and/or by another node (e.g., a lower layer node that is optionally nested by the respective node) for example, loudness, a classification of the audio content item (e.g., speech, music, other), and silence intervals.

As used herein, the term content item may sometimes broadly refer to the data (all the data or subset of data) stored in association with each node of the multi-layer hierarchical data structure described herein, for example, media items (e.g., audio, video, animation, title), metadata, rendering instructions, location of media items, segmentation data, and superimposition data.

At 104, computing unit 204 accesses allocated storage 202, where storage location(s) are allocated to store a rendering of respective content items rendered according to the rendering instructions of the associated respective node of the RVD.

Optionally, each node of the RVD is mapped to a respective memory location of allocated storage 202. Alternatively, some nodes of the RVD are mapped to respective memory locations of allocated storage 202, for example, according to a set-of-rules. The set-of-rules may define, for example, mapping nodes of a certain type, for example, mapping nodes representing scenes (e.g., and not mapping other node types, such as nodes representing audio).

As described herein, allocated storage serves the function of a cache that stores rendered content items for re-use in creation of an adapted version of the multi-media content object. Respective memory locations may be cleared when the stored rendered content item is no longer relevant to the adapted version of the multi-media content object. The memory location may store the newly rendered version of the content item based on the current version of the adaptation instructions.

Optionally, rendering instructions of respective nodes of the multi-layer hierarchical representation of the RVD are stored in respective allocated locations of allocated storage 202. Allocated storage 202 may be arranged according to classification categories, for example, storing spatial data, temporal data, content items according to type, metadata, and/or other classifications. The storage by classification categories allows for selective clearance of data of a certain classification category (e.g., according to a set-of-rules) and/or maintenance of data according to classification category.

At 106, computing unit 204 receives instructions to adapt the multi-media content object. The adaptation instructions may be received, for example, from multimedia editing and/or creation tool 210D. The adaptation instructions include instructions to adapt the rendering instructions stored by one or more nodes at respective layers of the multi-layer hierarchical data structure of the RVD. The adaptation instructions represent, for example, the user fixing a typo in a title, the user changing the order of scenes, the user adding a new sub-scene of a main scene, the user adding a content item after an existing scene, or other video editing functions.

Code may map the adaptation function the user enters using tool 210D to the node(s) of the multi-layer hierarchical data structure storing the rendering instructions corresponding to the user entered adaptation function. Optionally, the lowest layer node is affected. For example, when the user changes a content item of a sub-scene of a main scene of a video, the node storing the content item is identified.

Exemplary adaptation instructions to adapt rendering instructions of the multi- layer hierarchical data structure include: adding at least one new node to the multi-layer hierarchical data structure (e.g., when creating a new RVD, when adding additional scenes to the video), removing at least one existing node from the multi-layer hierarchical data structure, and modifying existing nodes. Exemplary modification of existing nodes include: modifying a temporal phase (e.g., mark in/out), modifying a GUI layer (e.g., z-index), modifying a spatial placement in a frame, and modifying the content item associated with the node.

Optionally, at 108, the allocated storage location of allocated storage 202 that stores the rendered content item(s) associated with the adapted node storing the adapted rendering instructions is cleared. Optionally, the cleared node is the lowest affected layer. The entire content of the node being adapted may be cleared, or the affected content item(s) may be cleared. The clearing of the rendered content item(s) may be performed, for example, by setting the allocated memory location to a cleared state, or setting an indication representing that overwriting of the allocation memory location is allowed (i.e., indirect clearing), or other memory clearance methods.

Alternatively, clearing of the rendered content item(s) is not necessarily performed. The existing rendered content item(s) may be maintained and stored in memory. Newly created rendered content item(s) (as described herein) may be stored in another allocated memory location, for example, maintaining a history of the rendered content items. The clearing may be performed, for example, of the oldest versions of the rendered content item(s) when the allocated memory location is filled with newer versions. Alternatively, clearing is not necessarily performed, and rendered content items are stored in designated locations in the allocated memory. As used herein, the term cleared (e.g., cleared allocated storage locations) is exemplary, and may be interchanged with the term designated (e.g., designated allocated storage locations), for example, when the clearing is not necessarily performed.

Optionally, one or more other allocated storage locations storing rendered content items associated with other node(s) are cleared. The cleared other nodes represent a relatively higher layer than the initially cleared node according to the multilayer hierarchical data structure. The other cleared node may nest the initially cleared node. The clearing of the other allocated storage locations (associated with the other higher layer nodes) may be performed recursively, from the initially cleared node to the next higher layer nesting the current node, according to the hierarchical associations defined by the multi-layer hierarchical data structure. The clearing may be performed sequentially, or substantially simultaneously.

Optionally, the clearing of the allocated storage location associated with the initially node and/or clearing of the other allocation storage locations associated with the higher layer nodes may be performed according to a set-of-rules. The set-of-rules may define which stored components of the respective node to clear and/or which stored components of the respective node to maintain. For example, clearing the node may preserve the segmentation data stored by the node and clear other data. The set-of-rules may define clearing of the data that is adapted by the instructions to adapt rendering instructions, and maintaining data that is not adapted by the instructions to adapt rendering instructions. The set-of-rules may define the clearing and/or maintaining of the stored components according to spatial and/or temporal adaptation defined by the instructions to adapt rendering instructions. For example, when the adaptation instructions relate to time, for example, adapting one or more sub-intervals of the time duration stored by the node, data related to the affected sub-interval(s) is cleared, while data related to other sub-intervals is maintained. The set-of-rules may be applied during the recursive clearing of higher layer nodes. In another example, when the adaptation instructions related to space, for example, adapting one or more areas of rendered frame(s), data related to the adapted frame(s) is cleared, while data related to other areas is maintained.

At 110, content item(s) are rendered according to the adapted instructions of the adapted initial node (e.g., that was cleared in block 108). The rendering may be performed by rendering code 210E. The rendered content item is stored in the allocated storage location associated with the certain node, which is (optionally) cleared.

Other content item(s) associated with other node(s) (i.e., higher layer nodes that nest the adapted initial node, and/or other nodes that are associated with the content item of the adapted initial node) are rendered according to the rendering instructions stored by the respective other node. The rendered other content item(s) are stored in the respective (optionally cleared) allocated storage locations of allocated storage 202 associated with the respective other node(s).

Reference is now made to FIG. 4, which is a flowchart of an exemplary method for processing nodes of the multi-layer hierarchical data structure according to the adaptation instructions, in according to some embodiments of the present invention. The method described with reference to FIG. 4 processes each node of the multi-layer hierarchical data structure, optionally recursively, optionally starting from the leaf- nodes. Storage locations of allocated storage 202 associated with each node are cleared, and content item(s) rendered according to adapted rendering instructions are stored in the cleared storage location, available for use in creation of the adapted multi-media content object. The method described with reference to FIG. 4, represents an exemplary implementation of blocks 108-110 of FIG. 1. The method described with reference to FIG. 4 may be implemented by processing unit 206 executing code stored in program store 208. At 402, an analysis is performed to determine when data in the node (e.g., content item(s), metadata, and other data) is being adapted based on the adaptation instructions. Higher layer nodes may be indirectly adapted when lower layer nodes are adapted by the adaptation instructions.

At 404, when the analysis determines that the data in the node is being adapted, an analysis is performed to determine when the storage location of allocated storage 202 associated with the node stores a rendered content item(s).

At 406, when the storage location stores the rendered content item(s), the allocated storage location is cleared.

At 408, an analysis is performed to determine when the storage location of allocated storage 202 associated with a higher layer node nesting the current node stores a rendered content item(s).

At 410, when the storage location of the higher layer node stores the rendered content item(s), the allocated storage location of the higher layer node is cleared.

It is noted that blocks 408-410 are iterated in a recursive manner, from the lowest layer leaf-node to the highest layer root node.

At 412, when the analysis of block 402 determines that the current node is not being adapted, and/or when the storage location of the higher layer node does not storage a rendered content item(s), and/or when the storage location of the current node and higher layer node(s) is cleared, an analysis is performed to determine when the storage location of the node (i.e., the node at the current layer being processed recursively) stores a rendered content item(s). It is noted that new nodes do not store rendered content item(s) in the associated storage location of allocated storage 202.

At 414, when the storage location of allocated storage 202 associated with the node does not store a rendered content item(s), the content item(s) is rendered according to the rendering instructions associated with the node, optionally the adapted instructions.

At 416, the rendered content item(s) is stored in the storage location of allocated storage 202 associated with the node. The stored rendered content item(s) may be re- used during a next iteration, to avoid re-rendering of the stored rendered content item(s).

At 418, when the storage location of allocated storage 202 associated with the node stores the rendered content item(s), the rendered content item(s) are retrieved from the storage location. The stored rendered content item(s) may be previously rendered content item(s) that were rendered based on adaptation instructions provided in an earlier iteration.

At 420, the retrieved rendered content item(s) are provided for creation of the multi-media content object(s). The retrieved rendered content item(s) may be provided recursively to higher layer nodes, for example, for superimposition.

Referring now back to FIG. 1 at 112, the rendered content items associated with the adapted multi-media content object are retrieved from the respective allocated locations of allocated storage 202. The content items may be retrieved according to the multi-layer hierarchical data structure, for example, retrieved recursively starting from the leaf-nodes, retrieved by following links starting from the root node, retrieved simultaneously from all (or some) nodes, or based on other methods.

At 114, the retrieved rendered content items may be used in creating the multimedia content object, optionally the video clip, (e.g., by rendering code 210E, or other code) for presentation on the display (e.g., user interface 222) by player 210C. For example, rendering code 210E may follow the rendering instructions defined by the RVD to render the multi-media content object using the retrieved content items. The multi-media content object may be created using the rendered content items, for example, recursively following the multi-layer hierarchical data structure, or using other methods. The rendered content items may be inserted into the multi-media content object. The rendered content items are provided for display by player 210C during presentation of the multi-media content object.

The adapted multi-media content object is created using the previously stored content items (i.e., which were not re-rendered or otherwise adapted based on the adaptation instructions) and the newly rendered content items (i.e., adapted and re- rendered according to the adaptation instructions, as described herein).

The process of creating the adapted multi-media content object is improved, by reducing the overall processing time required to create the adapted multi-media content object according to the adaptation instructions, by re-using existing rendered content items rather than re-rendering the content items, since rendering is a computationally intensive process, as described herein. Reference is now made to FIG. 5, which is a schematic depicting creation of the multi-media content object (or portion thereof) based on re-using rendered content item(s) stored in allocated storage 202, in accordance with some embodiments of the present invention. Image (or clip) 502 includes a person 504 that is segmented from a scene having a background of a tree and birds. Image (or clip) 506 includes another background of a volcano. The rendering instructions define segmenting person 504 from image 502, which is stored as segmented data 508 (e.g., as an image or clip) in allocated storage 202. The rendering instructions define superimposition of the person in segmented data 508 on to background 506, to create a new image (or clip) 510.

Background 506 may be replaced with a different background (not shown), optionally multiple different background defined by different adaptation instructions. The same rendered segmented data 508 is retrieved from the respective location of allocated storage 202, and used to create different images with the superimposition of the person on the different background images. The computationally heavy task of segmentation of person 504 from the background of image 502 is avoided in each iterations (apart from the first iteration to create segmented data 508) by reusing the stored rendered segmented data 508.

It is noted that the content items and/or rendering instructions described with reference to FIG. 5 may be stored in a single node, or a tree of nodes (or sub-tree of nodes).

Referring now back to 114 of FIG. 1, another example of re-using rendered content items is provided. Audio metadata stored in storage locations of allocated storage 202 associated with respective nodes may be reused. The audio metadata may be reused when rendering and/or audio-mixing according to instructions stored in a higher layer node using data stored in lower layer node(s). For example, when a music file in a lower layer node is adapted (e.g., replaced with another music file), and another music file in another lower layer node is not adapted, the audio-mixing performed by the higher layer node that nests both lower layer nodes may reuse the audio metadata of the non-adapted lower layer node to achieve desired mixing results, without a need to reprocess and re-extract the metadata from the non-adapted node.

At 116, one or more acts described with reference to blocks 106-114 are iterated. The iteration may be performed as part of the process of the user using tool 210D to create the multi-media content object, by viewing the multi-media content object at intermediate creation stages, for example, to fix errors, to remove items, to add items, to change the order of scenes, and/or to add other effects.

Reference is now made to FIG. 6, which is a flowchart depicting an iterative process of creating a multi-media content object, in accordance with some embodiments of the present invention. The systems and/or methods described herein avoid the iterative process of re-rendering the entire clip after each adaptation for viewing by the user, by re-using rendered content items stored in allocated storage 202 (as described herein), which reduces the computational time and/or the computations resources for creation of the adapted multi-media content object.

At 602, a user uses authoring tools (e.g., tool 210D) to created and/or adapt a multi-media content object, for example, a video.

At 604, an RVD is created and/or adapted based on the user entered instructions. The RVD may be created and/or adapted by code stored in program store 208 executed by processing unit 206.

At 606, the adapted multi-media content object is rendered by re-using rendered content items stored in allocated storage 202, as described herein.

A video clip 608A or sub video clip 608B is created, and at 610 is provided to a player (e.g., player 210C) for presentation on display (e.g., user interface 222).

Blocks 602-610 are iterated as the user adapts and views the multi-media content object.

The descriptions of the various embodiments of the present invention have been presented for purposes of illustration, but are not intended to be exhaustive or limited to the embodiments disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terminology used herein was chosen to best explain the principles of the embodiments, the practical application or technical improvement over technologies found in the marketplace, or to enable others of ordinary skill in the art to understand the embodiments disclosed herein.

It is expected that during the life of a patent maturing from this application many relevant multi-media content objects will be developed and the scope of the term multimedia content object is intended to include all such new technologies a priori. As used herein the term "about" refers to ± 10 %.

The terms "comprises", "comprising", "includes", "including", "having" and their conjugates mean "including but not limited to". This term encompasses the terms "consisting of" and "consisting essentially of".

The phrase "consisting essentially of" means that the composition or method may include additional ingredients and/or steps, but only if the additional ingredients and/or steps do not materially alter the basic and novel characteristics of the claimed composition or method.

As used herein, the singular form "a", "an" and "the" include plural references unless the context clearly dictates otherwise. For example, the term "a compound" or "at least one compound" may include a plurality of compounds, including mixtures thereof.

The word "exemplary" is used herein to mean "serving as an example, instance or illustration". Any embodiment described as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments and/or to exclude the incorporation of features from other embodiments.

The word "optionally" is used herein to mean "is provided in some embodiments and not provided in other embodiments". Any particular embodiment of the invention may include a plurality of "optional" features unless such features conflict.

Throughout this application, various embodiments of this invention may be presented in a range format. It should be understood that the description in range format is merely for convenience and brevity and should not be construed as an inflexible limitation on the scope of the invention. Accordingly, the description of a range should be considered to have specifically disclosed all the possible subranges as well as individual numerical values within that range. For example, description of a range such as from 1 to 6 should be considered to have specifically disclosed subranges such as from 1 to 3, from 1 to 4, from 1 to 5, from 2 to 4, from 2 to 6, from 3 to 6 etc., as well as individual numbers within that range, for example, 1, 2, 3, 4, 5, and 6. This applies regardless of the breadth of the range.

Whenever a numerical range is indicated herein, it is meant to include any cited numeral (fractional or integral) within the indicated range. The phrases "ranging/ranges between" a first indicate number and a second indicate number and "ranging/ranges from" a first indicate number "to" a second indicate number are used herein interchangeably and are meant to include the first and second indicated numbers and all the fractional and integral numerals therebetween.

It is appreciated that certain features of the invention, which are, for clarity, described in the context of separate embodiments, may also be provided in combination in a single embodiment. Conversely, various features of the invention, which are, for brevity, described in the context of a single embodiment, may also be provided separately or in any suitable subcombination or as suitable in any other described embodiment of the invention. Certain features described in the context of various embodiments are not to be considered essential features of those embodiments, unless the embodiment is inoperative without those elements.

Although the invention has been described in conjunction with specific embodiments thereof, it is evident that many alternatives, modifications and variations will be apparent to those skilled in the art. Accordingly, it is intended to embrace all such alternatives, modifications and variations that fall within the spirit and broad scope of the appended claims.

All publications, patents and patent applications mentioned in this specification are herein incorporated in their entirety by reference into the specification, to the same extent as if each individual publication, patent or patent application was specifically and individually indicated to be incorporated herein by reference. In addition, citation or identification of any reference in this application shall not be construed as an admission that such reference is available as prior art to the present invention. To the extent that section headings are used, they should not be construed as necessarily limiting.

Claims

WHAT IS CLAIMED IS:

1. A computer implemented method for adapting a multi-media content object, comprising:

accessing a rich video document (RVD) representing a multi-media content object, the RVD comprising instructions for rendering by rendering code for presentation on a display by a player, the RVD comprising a multi-layer hierarchical data structure mapping hierarchical associations of content items rendered according to rendering instructions of at least one content item, the hierarchical data structure arranged such that a node at a relatively higher layer nests at least one content item rendered using rendering instructions of at least one node at a relatively lower layer; accessing a storage device including allocated storage comprising at least one storage location, wherein each of the at least one storage location is allocated to store a rendering of respective content items rendered according to the instructions of the associated respective node of the RVD;

receiving instructions to adapt the rendering instructions stored by a certain node at a certain layer of the multi-layer hierarchical data structure of the RVD;

rendering at least one content item according to the adapted instructions of the certain node, and storing the rendered at least one content item in a designated storage location of the certain node;

rendering at least one other content item according to instructions of the at least one other node that nests the certain node, and storing the rendered at least one other content item in respective designated storage location of the at least one other node; and providing the rendered at least one other content item for presentation on the display by the player during presentation of the multi-media content object.

2. The computer implemented method of claim 1, wherein the multi-media content object comprises a video clip, wherein at least some higher layer nodes represent at least a scene or a sub-scene of the video clip, and wherein at least some lower layer nodes store content items selected from the group consisting of: audio, animation, video clip, overlay, and title.

3. The computer implemented method of claim 1, wherein at least one node of the multi-layer hierarchical data structure comprises instructions for rendering text.

4. The computer implemented method of claim 1, wherein the multi-layer hierarchical data structure comprises a tree or graph with respective layers represented by nodes comprising branching-nodes and leaf-nodes, wherein relatively higher nodes of the tree nest content objects of relatively lower nodes of the tree, and a root node of the tree represents the multi-media content object by nesting all lower nodes and leafs of the tree.

5. The computer implemented method of claim 1, wherein the RVD instructions include at least one of: content items for embedding in the multi-media content object, references to content items, and metadata.

6. The computer implemented method of claim 1, wherein instructions of respective nodes of the RVD are stored in respective storage locations within the allocated storage.

7. The computer implemented method of claim 1, wherein the RVD instructions stored at each node comprise at least one of a spatial and a temporal association of the at least one content item associated with the respective node, with the multi-media content object.

8. The computer implemented method of claim 7, wherein the RVD instructions are selected from the group consisting of: placement coordinates, in time marker, out time marker, and z-index representing a GUI-layer.

9. The computer implemented method of claim 1, wherein the RVD instructions at each node comprise instructions for insertion of the rendered content item into the multi-media content object.

10. The computer implemented method of claim 9, wherein the instructions for insertion of the rendered content item are selected from the group consisting of: a link or address to a source media file storing the content item, offset to start, duration to play, and cropping information.

11. The computer implemented method of claim 1, wherein in response to the received instructions to adapt the rendering instructions, rendering the at least one content item, rendering the at least one other content item, and providing the rendered at least one other content item, are iterated.

12. The computer implemented method of claim 1, wherein the rendering code creates the multi-media content object by retrieving the rendered at least one content item from the respective storage location of the allocated storage.

13. The computer implemented method of claim 1, wherein the instructions to adapt rendering instructions are selected from the group consisting of: adding at least one new node to the multi-layer hierarchical data structure, removing at least one existing node from the multi-layer hierarchical data structure, modifying a temporal phase, modifying a GUI layer, modifying a spatial placement in a frame, and modifying the content item.

14. The computer implemented method of claim 1, wherein at least one node stores at least one member selected from the group consisting of: segmentation data, classification data, and audio data.

15. The computer implemented method of claim 1, further comprising:

clearing the storage location of the allocated storage storing the rendered content item associated with the adapted certain node storing the adapted rendering instructions; and

clearing at least one other storage location storing rendered content items associated with at least one other node at a relatively higher layer than the certain node, the at least one other node nesting the certain node.

16. The computer implemented method of claim 15, wherein the clearing at least one other storage location is performed recursively from the certain node to the next higher layer node nesting the current node.

17. The computer implemented method of claim 15, wherein the clearing the storage location and clearing the at least one other storage location is performed according to a set-of-rules that define at least one of: which stored components of the respective node to clear and which stored components of the respective node to maintain.

18. The computer implemented method of claim 17, wherein the set-of-rules define at least one of: clearing data that is adapted by the instructions to adapt rendering instructions, and maintaining data that is not adapted by the instructions to adapt rendering instructions.

19. The computer implemented method of claim 17, wherein the set-of-rules define the at least one of clearing and maintaining the stored components according to at least one of: spatial and temporal adaptation defined by the instructions to adapt rendering instructions.

20. A system for adapting a multi-media content object, comprising:

a database storing a rich video document (RVD) representing a multi-media content object, the RVD comprising instructions for rendering by rendering code for presentation on a display by a player, the RVD comprising a multi-layer hierarchical data structure mapping hierarchical associations of content items rendered according to rendering instructions of at least one content item, the hierarchical data structure arranged such that a node at a relatively higher layer nests at least one content item rendered using rendering instructions of at least one node at a relatively lower layer; a storage device including allocated storage comprising at least one storage location, wherein each of the at least one storage location is allocated to store a rendering of respective content items rendered according to the instructions of the associated respective node of the RVD; a program store storing code; and

at least one processor coupled to the database, the storage device, and the program store for implementing the stored code, the code comprising:

code to receive instructions to adapt the rendering instructions stored by a certain node at a certain layer of the multi-layer hierarchical data structure of the RVD;

code to render at least one content item according to the adapted instructions of the certain node, and storing the rendered at least one content item in the designated storage location of the certain node, and to render at least one other content item according to instructions of the at least one other node that nests the certain node, and storing the rendered at least one other content item in respective designated storage location of the at least one other node; and

code to provide the rendered at least one other content item for presentation on the display by the player during presentation of the multi-media content object.

21. The system according to claim 20, wherein the database, the storage device, and the program store are stored by a server in network communication with a client terminal in communication with the display, wherein the client terminal stores the player that presents the multi-media content object on the display, and wherein the instructions to adapt the rendering instructions are received from a multi-media authoring tool stored on the client terminal.

22. A computer program product comprising a non-transitory computer readable storage medium storing program code thereon for implementation by a processor of a system for adapting a multi-media content object, comprising:

instructions to access a rich video document (RVD) representing a multi-media content object, the RVD comprising instructions for rendering by rendering code for presentation on a display by a player, the RVD comprising a multi-layer hierarchical data structure mapping hierarchical associations of content items rendered according to rendering instructions of at least one content item, the hierarchical data structure arranged such that a node at a relatively higher layer nests at least one content item rendered using rendering instructions of at least one node at a relatively lower layer; instructions to access a storage device including allocated storage comprising at least one storage location, wherein each of the at least one storage location is allocated to store a rendering of respective content items rendered according to the instructions of the associated respective node of the RVD;

instructions to receive instructions to adapt the rendering instructions stored by a certain node at a certain layer of the multi-layer hierarchical data structure of the RVD; instructions to render at least one content item according to the adapted instructions of the certain node, and storing the rendered at least one content item in the designated storage location of the certain node;

instructions to render at least one other content item according to instructions of the at least one other node that nests the certain node, and storing the rendered at least one other content item in respective designated storage location of the at least one other node; and

instructions to provide the rendered at least one other content item for presentation on the display by the player during presentation of the multi-media content object.