MXPA01005521A - Method and system for controlling multimedia streams using dynamic prototypes - Google Patents

Abstract

A method and system for encoding multimedia content using dynamic interfaces is disclosed. By providing an extendable, programmatic interface to the content of a scene, a scene may be more efficiently encoded than a scene in which each characteristic of each object in the scene is explicitly specified in the video stream for each frame. (FIG. 1)

Description

METHOD AND SYSTEM TO CONTROL MULTIMEDIA FLOWS USING DYNAMIC PROTOTYPES CROSS REFERENCE TO CO-PENDING REQUESTS The present application is related to the following co-pending applications, both filed on the same day: 1. "Method and System for Predictive Encoding of Data Arrays" (Method and System for the Predictive Coding of Data Arrangements), Proxy number 2167-0106-2, serial number 09 / 205,191 and 2. "Method and System for Encoding Rotations and Norms in 3D Generated Scenes" (Method and System for Coding Rotations and Normal in Scenes Generated in Three-Dimensional Shapes), number of proxy 2167-0105-2, serial number 09 / 205,190. Both applications name Julien Signes and Olivier Ondet as inventors, and these applications are incorporated here by reference.

BACKGROUND OF THE INVENTION FIELD OF THE INVENTION The present invention relates to an improved method for encoding sequences of computer-generated images, and more particularly, to the use of prototypes to encode how portions of a video stream are quantized, animated or otherwise updated.

DISCUSSION OF THE ANTECENDETES Although many computer-generated images were previously generated by specifying each pixel in each frame together, sequences of images by modern computers are usually created by extracting plural multimedia objects (for example, video and primitive two-dimensional or three-dimensional graphics), sometimes with one or more sound tracks. In the multimedia area, images and / or scenes are composed using a language. For example, The Hypertext Descriptor Language (HTML) specifies the two-dimensional (2D) composition of text and graphics. Similarly, the virtual reality descriptor (VRLM) language describes three-dimensional (3D) worlds. Multimedia languages now also include extensions for the dynamic alteration of composite images. The HTML language now includes dynamic extensions of HTLM (DHTML), as described in Dynamic HTML: The Definitive Reference, by Danny Goodman, published by O'Reilly & Associates Inc.1998, the contents to which they are incorporated herein by references. In addition, the XML extensions for the HTML language provide a meta script language to describe how the native language will be modified or extended. Similarly, the VRML language includes support for "PROTO" definitions, which allow new primitives to be "Prototyped". Part of a definition is a block of code or plural code blocks that are executed by the VRML language environment to provide the newly prototyped functionality. The Motion Picture Experts Group (MPEG) was formed to investigate the technologies required for the coding and decoding of image streams. The resulting standard (now referred to as "MPEG-1") has served as a basis for two additional MPEG standards: MPEG-2 and MPEG-4. The MPEG-4 is a standard that is "in progress". The final committee drafts are ISO / IEC FCD 14496-1 MPEG-4 systems and the visual system-2MEPG-4, the final committee drafts are incorporated here by references. Those drafts include several methods for encoding a multimedia content. Such methods include: 1. Quantization methods for scene parameters, 2. Animation streams to encode and transmit multimedia scenes, 3. Update streams to modify multimedia information over time. The completed draft of MPEG-4 includes support for: 1. Quantification in Binary Format for Scenes (hereinafter "BIFS"), 2. Animation using BIFS (hereinafter "BIFS" -Anim) and 3. Control of update using BIFS (hereinafter "BIFS Command"). The use of quantification under BIFS includes dialing nodes and fields with a quantization category. Each field of each node is assigned a category of quantification, which is applied to the field when a quantification parameter structure is given to give parameters during the quantification process. BIFS-Anim defines a coding method for animation based on multimedia data files and flows. BIFS-Anim includes 3 basic components: 1. a scene graphic, 2. an animation mask, 3. animation frames. The scene includes, a) Unmarked objects that are static and b) Marked objects that are going to be modified. The animation mask establishes the properties of the marked objects that are to be modified by the animation flow. The BIFS command is a protocol for updating BIFS nodes and a scene graphic. Allows the transportation of changes to the scene by sending command to replace the whole scene with another, removing some of the marked objects, or changing property values of objects marked in the scene.

COMPENDIUM OF THE INVENTION It is an object of the present invention to provide dynamic programming interfaces and / or extensions for a multimedia content language and / or a multimedia positioning language. It is another object of the present invention to provide links for animation, modification and quantization methods and to provide new prototypes for scenes, so that the scenes can be represented, animated and modified efficiently. These and other objects of the present invention are directed by one or more of, I. A method implemented by computer to encode multimedia content using dynamic interfaces, 2. a system for encoding multimedia content using dynamic interfaces, 3. a product of computer program to encode multimedia content using dynamic interfaces.

BRIEF DESCRIPTION OF THE DRAWINGS A more complete appreciation of the invention and many of its advantages will be readily apparent by reference to the following detailed descriptions, particularly when considered together with the accompanying drawings which: Figure 1 is a schematic illustration of a computer for implementing the method of the present invention; Figure 2 is a block diagram of the inter-relation between an interface declaration and a scene declaration; and Figure 3. is an example of a pseudo-code of an interface declaration.

DESCRIPTION OF THE PREFERRED MODALITY Referring now to the drawings, in which like reference numbers designate identical or corresponding parts throughout the various views, Figure 1 is a schematic illustration of a computer system for encoding multimedia content using dynamic interfaces. A computer 100 implements the method of the present invention, wherein the computer housing 102 houses a motherboard 104, which contains a CPU 106, memory 108 (eg, DRAM, ROM, EPROM, EEPROM, SRAM, and Flash RAM). ), and other optional special-purpose logic devices (for example, ASICs) or configurable logic devices, (GAL and FPGA reprogrammable). The computer 100 also includes plural input devices (e.g., a keyboard 122 and a mouse 124), and a display card 110 for controlling the monitor 120. In addition, the computer system 100 further includes a floppy disk drive 114; other removable media devices (e.g., compact disc 119, tape and removable magneto-optical media (not shown)); and a hard disk 112, or other fixed high density media units, connected using an appropriate device bus (for example a SCSI busbar, an improved IDE busbar or an Ultra DMA busbar). Also connected to the same busbar of the device or other busbar of the device, the computer 100 may further include a compact disk reader 118, a compact disc reader / writer unit (not shown) or a storage device for multiple sets of compact disc (not shown). Although the compact disc 119 is shown in a CD case, compact disk 119 can be inserted directly into CD-ROM drives, which do not require boxes. In an alternative embodiment, the present invention is implemented as one or as a combination of: 1. a portable cable box, 2. a video card, and / or 3. a receiving / reproducing unit. In another alternative embodiment, a printer (not shown) provides printed lists of interfaces for encoding multimedia contents. The system includes at least one computer-readable medium. Examples of computer-readable media are CDs 119, hard disks 112, floppy disks, tapes, magneto-optical disks, PROMs (EPROM, EEPROM, Flash EPROM), DRAM, SRAM, SDRAM, etc. Stored in any one or a combination of computer-readable media, the present invention includes software to control both the hardware of the computer 100 and to allow the computer 100 to interact with a human user. Such software may include, but is not limited to, device units, operating systems, and user applications, such as development tools. Said computer readable medium further includes the computer program product of the present invention for encoding multimedia content using dynamic interfaces. The computer coding devices of the present invention can be any interpreted or executable coding mechanism, including but not limited to, manuscripts, interpreters, dynamic link collections, Java classes and complete executable programs. As shown in Figure 2, a multimedia stream is received by the present invention and decoded into scene components and interface components. The scene components include information of both 1. digital objects in the scene and 2. how these objects are updated or animated. The interface components include interface definitions which, after decoding, are stored internally, and, in one embodiment, the definitions are stored, at least partially, in an Interface Coding Table. In particular, the Interface Coding Table is attached to a new interface and buffer of coding elements of a given interface definition in order to apply methods (eg, quantization, animation and update methods) of the content language digital (for example, MPEG-4BIFS or VRLM) to scene components. An Interface Coding Table includes the following information: 1. an update mode, 2. an animation mode (or an Anim mode), 3. a quantization category, 4. quantization parameters, 5. an animation category . Using the update mode, any parameter can be declared as an "input" type. In languages (for example, VRML) that support defining the types of parameters, either implicitly or explicitly, the parameters do not need to be defined separately as the "input" type. Specifically, the VRML language allows parameters to be defined as event entry, exposed field, event output, and field. Both the event entry and the exposed field are, by definition, "input" type parameters. When used, a type of "entries" indicates that the interface can receive more updates from the multimedia stream (for example, through an update stream such as a BIFS command stream). The animation mode can operate on any parameter in the interface, marking a parameter with the corresponding animation category. The quantization mode can also operate on any parameter in the interface, marking a parameter with the corresponding quantization category, and transmitting additional optional quantization parameters. Multimedia streams can use the information from the Interface Coding Table (ICT) as follows; 1. to control quantization, 2. to update a portion of a corresponding interface, and 3. to perform the animation through the interface. Specifically, a category of quantification, qc [i], associated with an interface parameter, ai, is applied using the value of the Interface Coding Table, qp [i] as well as other contextual information of the scene. This allows the interface declaration to be compressed almost optimally. Any given interface parameter, ai, marked in "input mode" can be modified through an update stream that will modify the values of the interface at a given time point. The interface can be animated using a continuous animation stream to continuously update some of the values. The animation category, ac [i], is used in that case. The Interface Coding Table is part of the Interface Declaration and has the structure shown, for example in Figure 3. The particular parameters are described below. ICT Mask: Boléanos values mask to determine what information is available for each parameter. Use of quantification: Boolean to determine if a quantization parameter is sent for each parameter. Update use: Boolean to set whether the information in the "Entry mode" is sent for each parameter as well as the update category information. As described above, in languages that already support independently define whether a parameter can be updated, this parameter is optional. Usage animation: Boolean to set whether the information in the "dynamic node" and animation category is sent for each parameter. Additional processing was performed for each parameter, ai at the interface. As described below with reference to the Interface Definition, PROTO, the number of parameters is specified in the variable "numeric fields". In this way, for each parameter, the following information is specified (optionally): In mode data: Information on whether this parameter of the interface is "updatable". This information can also come from the interface declaration. Quantification category data: The quantification category that will be used. These data are used when "quantification of use" is true. Quantification parameter data: The quantification parameters specific to this category: Minimum and maximum values, in information used for compression, according to the method of specific quantification / inverse quantification of the category. These data are used when "quantification of uses" is true. Animation parameter data: The animation scheme selected for the specific field. These data are used when "uses animation" is true. In the context of MPEG-4 and VRML, the present invention allows new interfaces to be defined through PROTOs and EXTERNPROTO. The only difference between PROTOS and EXTERNPROTOs is that the code that implements the interface is provided in the first case within the same description (PROTO), while in the second case it is provided in an external resource. In both cases, the use of an ICT table with the prototype declaration allows the field declarations to be marked with all the necessary parameters for the coding, animation and updating of the node in the previous form. In MPEG-4, the ICT table includes the following information: 1. a quantification category (see table 1), 2. an animation category (see table 2), and 3. quantification parameters (including minimum and maximum values) for the field).

Table 1 Categories of Quantification Table 2 Categories of animation With this information along with the PROTO definition, an MPEG-4 terminal will be able to generate a Node Coding Table needed to encode, update and animate a node. The following is a typical example of a Node Coding Table for the Directional Light node. The minimum and maximum values as well as the quantization types (Q) and animation (A) are provided. All identifications are automatically deducted from the field declaration. The Node Data Types (SF World Node, SF3D node) and their corresponding identifications are used to determine the context in which the node can be used. The PROTOS receive a specific category.

Table 4 Types MPEG-4 Binary Syntax The following is the binary syntax for PROTO definition with the corresponding Interface Coding table. The binary bitstream syntax is specified following MPEG-4 MSDL language. 1.0 PROTO PROTO Declaration (). { PROTOinterf aseDef inición (); PROTOcode (); PROTOTabladecode (); } 2.0 Definition of the PROTO interface PROTOinterf class of fInition. { bit (idBits) id; // The ID used to refer to the PROTO Bit (5) type; // Type of context, using them last bits, Table 1. bit (4) field bits; // Number to specify number of fields. bit (Field bits) numTimes; // Number of fields and events in proto. for (i = 0; i <numFields; ++ i). { bit (2) event type; // eventent, eventofuera, campo or // Campoexpues to bit (6) type field; // the type of field, using table 1 3. 0 Code PROTO class codePROTO. { bit (1) esExtern // is it an external proto? Yes (is External). { // Read URL to locate from f s. Proto. MFUrl locations; } so . { do . { SFNode node (TypeNode FWor Id, true); bit (1) plusNode; } while (moreNodos); } } 4.0 SFNode class SFNode (int nodeDataType, ISED boléano). { bit (1) is broken; yes (unused) { bit (BIFSConfig.nodoIDbits) nodeID; } so . { bit (GetNDTnBits (dataTypeNode)) localNodeType; nodotipo = GetNodoTipo (tipoDatosType, localNodeType); bit (1) is Updateable; yes (it acts 1 i zable). { bit (BIFSConf ig, nodeIDbits) nodeID; bit (1) AccessMask; yes (AccessMask). { yes (ISED) node isedMore caraNodeDescription (MakeNode (dataType node, TypeNode)); then More car aNode s cr ipci ón mnodo (MakeNode (dataNodeType, NodeType)); } so . { if (ISED) i sedLis mNodeDescription (nodeNode (nodeType, nodeType)); then ListNodeDescription lnodo (MakeNode (dataType node, TypeType)); } } . 0 isedMaskNodeDescripci ny class i sedMá s caraNodeDescription (NodeData node). { for (i = 0; i <node. numde Fcampo s; i ++). { bit (1) Mask; s i (More expensive). { yes (node, nodeType == PROTO). { bit (1) IsedCampo; yes (I sedCampo) bit (node. numDEFcampos;) protoCampo; then field value (node .field [node. from f 2 to 11 [i]]); so . { Valosr node field. field [nodo.def2all [i]]); } } 6.0 ised NodeListDescription class idesLis taNodeDescription (NodeData node). { bit (1) finBandera; while (FinBandera). { int (node. nDEFb its) fieldRef; yes (node, nodeType == PROTO). { bit (1) IsedCampo; yes (IsedCampo) bit (node. nymDEFcampos;) protoCampo; then Value Field (node .field [node. from f 2 to 11 [i]] so . { field value (node .field [node. from f 2 to 11 [i]]); bit (1) finBandera; } } 7.0 protoCodificationTable I nter PhaseCodi fi cationTable (). { InterfaceCodificationMask; InterphaseCodificationParameters; } 8.0 protoCodificationMáscara entre faseCodiFicaciónMáscara (). { UsoCuant boléano; UseBocean update; UsoAnim boléano; 9. 0 protoCodification Inter-phase ParametersCodification Parameters (). { for (i = 0; i <proto. NbdeCampos; i ++ { si (usoCuant) { ui nt (4) quantCategory; if (quantCategory == 13) unit (5) nbBits; bit (1) hasMinMax; if (tHisMinMore car) { S FCampo (fieldType) minCampoVa 1 or; Sfcampo (campotype) ma s xCampoVa 1 or;.}..}. // REM: Here you do not need "esUpdate" // since intentionally comes // from the "type of event and field" if (useAnim). {boolean isDyn; if (is Dyn) {uint (4) animCategory; } An illustrative use of the present invention uses PROTO prototypes in an MPEG-4 application to implement a collaborative professional application that allows multiple engineers to work simultaneously on a model / design (eg, the model of a boat) of remote sites and follow the changes in real time. Engineers who perform design experiments and want all sites to obtain the results of the experiment simultaneously, although using the most efficient compressed transmission to adjust a low bit rate network connection. In addition, the system must be generic enough to adapt to various designs. By using the MPEG-4 framework, you get a generic and efficient framework for the entire communication and application design, instead of designing a specific application and potentially suboptimizing each case. For this example, a model of a three-dimensional boat including 10, 000 polygons made with 20,000 vertices is presented. In the experiment, engineers want to analyze and control the deformation of the boat that results from certain times. Generally, the deformation can be controlled through several parameters, for example, wind speed and direction, and the height of the waves. The boat after is defined as a PROTO improved by MPEG-4, and after receiving new values a manuscript within the PROTO calculates the new parameters of vertices. In this way, engineers can carry live simulations and transits, using the BIFS animation protocol, only a few parameters are optimally encoded, instead of forwarding the 20,000 vertices. In addition, each time the new results are tested the engineers only need to transmit a new group of parameters using protocols of the BIFS command. Based on the sample, the following PROTO prototype can be defined: PROTO Windboat [exposedField SFVecef indspeed exposedField SFVec3f windOrientation exposedfield SFFoat waveHeight] [ Script { # Change vertices according to parameters Indexedf acSet. { # 3D model of the boat PARAMETERS OF THE ICT TABLE The following coding parameter can be associated in the ICT table.

In addition to adding functionality, the present invention produces more efficiently encoded bit streams. Comparisons of bit rates can be made by comparing the information transmitted without and with the use of the PROTO interface.

The gain can be measured for a PROTO illustration (placing a boat in a scene), for a PROTO update (changing values remotely), or for animation. The size in bits for each of the data that will be transmitted is compared. Note that if ordinary PROTOs are used if the ICT table then the update and animation functionality is not available.

As a second example as in the present invention it can be used to animate a bouncing ball. Instead of sending a command stream to update each polygon of the ball in each frame, an interface is created that controls the animation. The interface defines a parameter to control the height of the ball from the ground. In order to animate the ball only the height parameter needs to be changed for each frame, the manuscript of the interface performs the actual update of the vertices. Similarly, if the interphase of the ball includes a second parameter for color, the color of each polygon can be changed through a manuscript instead of animating the colors of the polygons individually. Although the present invention has been described above in terms of an interface extending the native methods provided by a language, the present invention is further directed to provide interphase interfaces. Just as objects can inherit their behavior as super classes, so can the interfaces and inherit interface functionalities initiated earlier. As will be apparent to one skilled in the art, the method and system of the present invention may be practiced in a manner other than that explicitly described herein without departing from the spirit of the invention. Therefore, the specification is not intended to be limiting, and only within the scope of the claims defines the limits of the invention.

Claims (3)

1. A multimedia system to extend the native capabilities of a digital content language, the system comprises: a processor; a non-volatile memory including a digital content language with a group of native capabilities; and a reprogrammable memory to receive digital objects and a prototype of an interface that will be executed by the processor to extend the group of native capabilities of the digital content language, where the prototype of the interface defines a method to use at least one of (a) a quantification method to quantify values, (b) animation of digital objects, and (c) updating of digital objects.

2. The system according to claim 1, wherein the digital content language is a virtual reality descriptor language that includes a prototype extension. The system according to claim 2, further comprising a decoder for an MPEG-4 BIFS protocol, wherein the decoder transmits decoded values to the prototype of the interface.