TW200803523A - Frame level multimedia decoding with frame information table - Google Patents

Abstract

Apparatus and method to decode video data while maintaining a target video quality using an integrated error control system including error detection, resynchronization and error recovery are described. Robust error control can be provided by a joint encoder-decoder functionality including multiple error resilience designs. In one aspect, error recovery may be an end-to-end integrated multi-layer error detection, resynchronization and recovery mechanism designed to achieve reliable error detection and error localization. The error recovery system may include cross-layer interaction of error detection, resynchronization and error recovery subsystems. In another aspect, error handling of a scalable coded bitstream is coordinated across a base-layer and enhancement layer of scalable compressed video.

Description

200803523 IX. Description of the invention: [Technical field to which the invention pertains] This disclosure is directed to multimedia signal processing, and more specifically, to video encoding and decoding. [Prior Art] A multimedia signal processing system such as a video encoder can encode a multimedia φ material using an encoding method based on an international standard such as the MPEG_X& H.26X standard. These encoding methods are typically directed to compressing multimedia material for transmission and/or storage. In summary, compression is the process of removing redundancy from the data. The video signal can be described in terms of a sequence of pictures, including a picture frame (complete picture) or a field (e.g., an interlaced video signal containing fields of alternating odd or even lines of pictures). As used herein, the term "frame" refers to a picture, a frame, or a field. The frame may consist of various = small portions of video material including individual primitives, tuples commonly referred to as tiles, and block groups commonly referred to as slices. The video coding method compresses the video signal by compressing each frame by using a lossless or lossy compression algorithm. In-frame coding (referred to herein as intra-coding) refers to the use of a frame to encode the frame. Inter-frame coding (referred to herein as inter-coding) refers to encoding a frame based on other ("reference") frames. For example, a video signal typically exhibits spatial redundancy, wherein portions of the video frame samples that are close to each other in the same frame have at least portions that match each other or substantially match. In addition, the frame typically exhibits temporal redundancy, and temporal redundancy can be removed using techniques such as motion compensated prediction. A meta-stream (such as a video bitstream) that is targeted to a single application can be encoded (eg, using scalable coding) into two, such as a base layer and one or more enhancement layers, or More than two independent layers. These layers can then be used to provide scalability, such as time and/or SNR (signal to noise ratio) scalability. Scalable coding is used for dynamic channels, where the scalable bit stream can be adapted to match the fluctuations in the network bandwidth. In channels that are prone to errors, scalable coding can increase robustness by unequal error protection of the base layer and the enhancement layer. ^ Wireless channels are prone to errors, including bit errors and packet loss. Because video compression inherently removes redundancy, the compressed data becomes critical. The loss of any part of this data in the transmission _ affects the reconstruction of the decoder at the D state. If the missing data is part of the reference part of the motion compensated prediction and/or spatial prediction, the effect is exacerbated, causing time and/or spatial error propagation. In addition, scalable coding can also aggravate the error propagation + instance and σ. If the enhancement layer data depends on the base layer, the loss of the base layer can make the correctly received enhancement layer data useless. Similarly, synchronization can be lost at the decoder if it can be re-stepped due to content-related coding and predictive coding that results in an even larger portion of the displayable lost video. Errors (4), _, and recovery may be difficult or possible for decoding n applications if most of the video is lost due to an error. A system-reliable error control system that includes (at least in part) error (four), resynchronization, and/or maximum use of received information. [Invention] The systems, methods, and devices of the present disclosure are each There are several aspects, where 120028.doc 200803523 does not have a single - individually negative for its required attributes #. Without limiting the scope of the present disclosure as expressed by the following claims, the more essential features thereof will now be briefly discussed. After considering this discussion, and especially after reading the title "Detailed Descripticm Qf"

After the section of Aspects), it will be appreciated how the example features of the present disclosure provide advantages to multimedia encoding and decoding, including, for example, improved error concealment and/or improved efficiency. • Provide a way to process multimedia materials. The method includes: receiving multimedia material; and synthesizing descriptive information about the multimedia material in the first layer, wherein the descriptive information is related to processing of the multimedia material in the second layer, and at least partially based on the descriptive information providing and the second layer Instructions related to the processing of multimedia materials. 'Provides a device for processing multimedia material. The apparatus includes: a receiver configured to receive multimedia material; an information organizer configured to organize descriptive information about the multimedia material in the first layer, wherein the descriptive information and the second The processing of the multimedia data in the layer and the error control decision subsystem are configured to provide at least part of the foundation: the binary information provides instructions related to the processing of the multimedia material in the second layer. A machine readable medium containing a code is provided. When the code is executed on one or more machines, the one or more machines are caused to perform program operations. The code includes: a code for receiving multimedia data; a code for organizing descriptive information about the multimedia material in the first layer, wherein the descriptive information is related to processing of the multimedia material in the second layer; And a code for providing instructions related to the processing of the 12202.doc 200803523 of the multimedia material in the second layer based at least in part on the descriptive information. A method of processing multimedia material is provided. The method includes receiving multimedia material; processing multimedia material in an upper layer; indicating a layer based at least in part on information associated with processing of the multimedia material in the upper layer; and based at least in part on processing associated with the multimedia material in the upper layer The information is processed in the lower layer. An apparatus for processing multimedia material is provided. The apparatus includes: a φ receiver configured to receive multimedia material; an upper decoder subsystem configured to process multimedia material in an upper layer and based at least in part on processing associated with the multimedia material in the upper layer The associated information indicates the layer, and the lower layer decoder subsystem, which is configured to process the multimedia material in the lower layer based at least in part on the information associated with the processing of the multimedia material in the upper layer. A machine readable medium containing a code is provided. When the code is executed on one or more machines, the one or more machines are caused to perform program operations. • The code includes: a code for receiving multimedia material, a code for processing multimedia material in an upper layer; and a layer for indicating, at least in part, information associated with processing of the media file in the upper layer And a code for processing the multimedia material in the lower layer based at least in part on the information associated with the location of the multimedia material in the upper layer. Provide a method of processing multimedia materials. The method includes: receiving multimedia material; receiving descriptive information about the multimedia material from the first layer, wherein the descriptive information is related to processing of the multimedia material in the second layer; and based at least in part on the descriptive information received The second layer handles the multi-media 12028.doc 200803523 body data. An apparatus for processing multimedia material is provided. The apparatus includes a receiver configured to receive multimedia material, and a decoder configured to receive descriptive information about the multimedia material from the first layer (where the 'descriptive information is in the second layer The processing of the multimedia material is related, and the multimedia material is processed in the second layer based at least in part on the descriptive information received. Φ provides a machine readable medium containing the code. When the code is executed on one or more machines, the one or more machines are caused to perform program operations. The code includes: a code for receiving multimedia data; a code for receiving descriptive information about the multimedia material from the first layer, wherein the descriptive information is related to processing of the multimedia material in the second layer; The code of the multimedia material is processed in the second layer based at least in part on the received descriptive information. [Embodiment] The following [Embodiment] is directed to some specific exemplary aspects of the present disclosure. Phrase, an aspect, ",,, another aspect", ",, an additional aspect,", "situation,", some sadness "some aspects" and the use of their analogues It is not intended to imply that the various aspects of the elements in the various aspects are mutually exclusive. Therefore, various aspects and various aspects of the elements may be eliminated and/or combined and still in the scope of the application. The various aspects of the present invention can be embodied in a number of different ways as defined and covered by the scope of the claims. In this description, reference is made to the drawings in which the System and method for processing 12028.doc -10· 200803523 in the system, encoder and decoder. Multimedia data can be transmitted, still images or any other suitable type of motion picture, sound, etc. The aspect includes use - including - In the bedding - or more integrated error control system to decode the error and the method: can be found by including a number of error elastic settings - the error 7:; For example, the root

The end-to-end phasing of the measurement and error localization is achieved by a reliable error. It has also been found to be beneficial in terms of error price measurement, resynchronization and recovery of the current performance of a certain cross-layer interaction = during processing. In another recording, the error processing of the scalable coded bit lines is coordinated across the base layer and the bare layer of the scalable video. BRIEF DESCRIPTION OF THE DRAWINGS Fig. 1 is a functional block diagram showing a multimedia communication system according to an item aspect. The system 1 includes an encoder device 1 that communicates with a decoder device 15 via a network 14A. In one example, the encoder device receives a multimedia signal from an external source 102 and encodes the signal for transmission over the network 140. In this example, the encoder device 11A includes a processor 112 coupled to a memory 114 and a transceiver 116. The processor 112 encodes the data from the multimedia data source and provides it to the transceiver 1 i 6 for communication via the network 140. In this example, the decoder device 150 includes a processor 152 coupled to a memory 154 and a transceiver 156. The processor 152 can include a general purpose processor and/or a digital signal processor and/or a special application hardware processor. 120028.doc -11- 200803523

One or more of them. Memory 154 can include one or more of a solid state or disk based storage or any readable and writable random access memory device. Transceiver 156 is configured to receive multimedia material via network 140 and make it available to processor 152 for decoding. In one example, transceiver 156 includes a wireless transceiver. The network 14 may include one or more of a wired or wireless communication system including an Ethernet, a telephone (eg, P0TS), a cable, a wire, and an optical fiber _ 糸" 'One or more of the first, the wireless system includes a code division multiple access (CDMA or CDMA2000) communication system, a frequency division multiple access (Fdma) system, such as GSM/GPRS (General Packet Radio Service) / EDGE Time-division multiple access (TDMA) system (enhanced data GSM environment), tetra (land, 麈 radio) mobile phone system, wideband code division multiple access (WCdma) system, high data rate (IxEV-DO Or one or more of a 金£^00 Gold Medal Multicast) system, an IEEE 802.11 system, a Media FL® system, a DMB system, an Orthogonal Frequency Division Multiple Access (OFDM) system, or a DVB-Η system. φ Because the wireless channel experiences random bit errors and burst errors, the error recovery design is designed to effectively handle both of these error types. It has been found that by using an integrated multi-layer error control system, two types of error classes I can be effectively handled. By using space or time error concealing T at the application layer, the video portion is effectively processed (including ( For example, one or more primitives, or even random bit errors that include the loss of one or more physical layer packets (PLPs). However, burst errors that result in the loss of multiple consecutive PLPs can be handled more efficiently by means of error control modules embedded in the transport and synchronization layers as discussed below. 120028.doc •12- 200803523

2 is a block diagram of an example of a multi-layer protocol stack for partitioning tasks including a cross-layer error control system, such as one of encoder device 110 and decoder device 150 in the system illustrated in FIG. Referring to Figures 2 and 2, a communication device such as encoder device 110 and decoder device 150 can use a multi-layer protocol stack for distributing processing tasks. The upper layer components of encoder device 110 and decoder device 150 may include multiple applications, such as video or audio encoders and/or decoders. Some embodiments may include multiple streams of information that are intended to be decoded simultaneously. In these situations, multiple stream synchronization tasks can also be performed in the upper component. In the encoder device 11, an upper layer component can provide code timing information in the bit stream transmitted over the wireless network and/or the wired network 14〇. In decoder device 150, the upper layer component can parse multiple streams of information so that the associated application decodes it at about the same time. The upper layer components of the encoder device 110 are distributed in one or more of the application layer 2〇5 and the synchronization layer 210. The lower layer components of the encoder device 11 are distributed among one or more of the transport layer 215, the stream and/or the medium access control (MAC) layer 22, and the physical layer. Similarly, the upper layer components of the decoder device 15 are distributed among one or more of the application layer 230 and the synchronization layer 235. The lower layer components of the decoder device 150 are distributed among one or more of the transport layer 240, the stream and/or the medium access control (MAC) layer 245, and the physical layer 250. Those skilled in the art will be aware of these layers and are familiar with the assignment of various tasks. It should be noted that the terms "upper layer" and "lower layer" as used herein are relative terms. For example, the synchronization layer 235 may be referred to as the lower layer according to the application layer 230, but may be referred to according to the transport layer 240. For the upper layer. Encoder device is provided for each of the layers in this example. i 〇120028.doc -13- 200803523

One of the errors in the elastic system 255. The lower layer of the encoder device ιι〇 can include various mechanisms for providing error resilience. Such error resilience mechanisms provided in the underlying components may include - erroneous (four) recipes, interlacing mechanisms, and other mechanisms known to those skilled in the art. The lower component of the decoder device 15 can include an error-correcting component that enables the guessing and correction of errors. Some errors introduced via the wired and/or wireless network 14G may not be corrected by the lower layer components of the decoder device 15. For their inability to correct: a solution such as requesting a recompilation portion by a component below the encoder device 110 may not be feasible in some cases. The layer component above the encoder device m can attach descriptive information about the packets of the multimedia material to the various layers of the communication in the header. In some instances 'encapsulation is performed at various layers to allow multiple streams to be separated (parsed) during encoding and to be at least partially used during decoding by header information added by various layers of the encoder Regroup. For example, the 'synchronization layer 2U' may add header information identifying a plurality of types of packets associated with a plurality of decoding H components that can simultaneously decode multiple types of packets. The synchronization layer header information may include fields of identification-data sequence time, data sequence duration, target decoder components (e.g., audio, video, and closed captions) frame number, number of packets, and other information. In some instances, the synchronization layer packet can be of variable length. This can be attributed to various encoding mechanisms, such as digital compression mechanisms including variable length encoding mechanisms. Transport layer 215 may also attach descriptive information to the transport layer packet in the transport header. The transport layer packet can be fixed length to support various error coding: mechanism, modulation mechanism and other machines using fixed length packets _, 卜 transmit standard 12028.doc -14- 200803523 header can contain identification by a single synchronization layer packet parsing Information on the number of transport layer packets. If the synchronization layer packet is of variable length, the number of transport layer packets that need to contain data may also be variable. In one aspect, at least some of the information included in the transmit and/or sync headers can be included in a directory. The directory may include header information associated with various layers such as application layer 205, synchronization layer 210, transport layer 215, and other layers. This directory can be communicated to the decoder. The information can be used by the decoder device to recover various errors (including identifying the size of the error packet received by the error, identifying the next available packet for resynchronization, and others). Header information from the header directory can be used to replace missing or erroneous original header information in the data stream. Further details of the header directory can be found in application No. ll/527,022, filed on September 25, 2006, entitled "VIDEO ENCODING METHOD ENABLING HIGHLY EFFICIENT PARTIAL DECODING OF H.264 AND OTHER TRANSFORM CODED INFORMATION" The application is assigned to its assignee and is hereby incorporated by reference in its entirety. In this example, error recovery system 260 in decoder device 150 is provided across each layer. Decoder device 150 may include providing error recovery Various mechanisms. These error recovery mechanisms may include underlying error detection and correction components (such as Reed-Solomon coding and/or Turbo-coding) and for replacement and/or Or hiding upper error recovery and/or error concealment mechanisms of data that are not calibrated by the underlying method. The various error recovery components in application layer 230 may benefit from information that may be used, for example, between synchronization layer 235 and transport layer 240. Included in the transport layer standard 12028.doc -15- 200803523 header layer header, header directory (if available), or the information can be decoded in the pirate Based on the estimation of the received data. As discussed above, the error resilience system in the encoder device 110 and the error recovery system in the decoder set m5G form an end-to-end integrated multi-layer error referred to herein as an error system. Detection, resynchronization and recovery mechanisms. Details of the error control system will now be discussed. It should be noted that the encoder device can be omitted, reconfigured, separated and/or combined with the encoder and the decoder shown in Figure 2. One or more of the elements of the device 15A. Figure 3A is a functional block diagram illustrating one aspect of a decoder device 150 that may be used in a system such as the one illustrated in Figure j. In this aspect The decoder 150 includes a receiver component 〇2, an information organizer component 〇4, an error control decision component 〇6, and a multimedia decoder component 308. The receiver 302 receives the encoded video material ( For example, the data encoded by the encoder 110 of Figure 1. The receiver 〇2 can receive the encoded material via a wired or wireless network, such as the network 140 of Figure 1. In one aspect, the data received package Transform coefficients representing source multimedia data. Transform transform coefficients into a domain in which the correlation of adjacent samples is significantly reduced. For example, images typically exhibit a high degree of spatial correlation in the spatial domain. Orthogonal to each other to exhibit zero correlation. Some examples of transforms that can be used for multimedia data include (but are not limited to): DCT (Discrete Cosine Transform), DFT (Discrete Fourier Transform), Hadmai (d) (Hadamai:d) Or Walsh-Hadamard transformation, 120028.doc -16- 200803523 Discrete wavelet transform, DST (discrete sine transform), Hal (fine) transform, oblique transform, KL (card, The Karhunen_L〇eve transform and the integer transform used in H.264, n, are used to transform a matrix or array of multimedia samples. A two-dimensional matrix is usually used, but a dimensional array can also be used. The information received also includes information indicating how to encode the coded block. This information may include inter-frame coded reference information such as motion vectors and frame numbers, as well as in-frame reference information including block size and spatial prediction directional indicators and others. The certain-received data includes a quantization parameter indicating how to estimate the per-transform coefficient from a certain number of bits, a non-zero indicator indicating how many transform coefficients in the transformed matrix are not zero, and others. The poor organizer component 304 collects descriptive information about the multimedia material from the bitstream. In the "item" aspect, the f-organizer 3〇4 interprets the transmission and synchronization layer header data for further processing. The transfer header can be processed to determine the frame and superframe boundaries, where the superframe is a group frame that can usually be decoded independently. The superframe may include a cover-frame of a fixed time period ranging from about Q2 seconds to about 2 seconds. The super frame size can be selected to allow - reasonable time. The transmit header can also be processed to determine the frame length and the block offset of the frame in the bitstream to handle the error PLP received by the self-flow/m A c layer. The synchronization layer header can be processed to: extract the frame number and interpret the base and enhance the frame, extract the frame rate required for the presentation time stamp and/or insert and obtain in the case of an error. The pTs of the frame inserted via frame rate up conversion (FRUC). The synchronization header can also be processed to: extract the presentation timestamp of the video frame to synchronize with the audio frame associated with the phase 12028.doc • 17·200803523; extract random access in the event of an error that causes synchronization in the decompressor Point location to mark the next resynchronization point. If the header directory as discussed above is available, the information organizer 3〇4 can also collect information from the header directory.

In addition to collecting information from the header and header catalogs, the information organizer 3〇4 can also generate descriptive information about the video material. Various header sum check codes, payload sum check codes, and error control mechanisms can be used to identify which part of the data is incorrect. The information generated may include the identification of such erroneous portions of the data. The error data can be - error distribution amount (four) - error rate measurement. Error data can be organized on any layer from the frame layer to the slice layer (cut # is a group coded block), a primitive block layer, or even a primitive layer. These types of descriptive information about the error data can be used to locate and determine the scope of the error. Details of the types of information that can be collected, maintained, flagged, or generated by the information organizer, J., Bianze, will be discussed below. In one aspect, the error control decision element 〇6 uses descriptive information collected and/or generated by the information organizer 〇4 (eg, stored in a table format) to provide instructions related to the processing of the multimedia material. . The error control decision component 306 analyzes the descriptive information to locate errors and determine which portions of the video are affected and to what extent these partial errors are. Using this information, the error control decision element 3〇6 can determine an error control method for handling such error conditions. In another aspect, the error control decision component receives feedback information from the upper layer. The feedback information may include information associated with the processing of multimedia in the upper layer. This feedback may include up-to-date information in the descriptive information of the upper layer of 120028.doc -18· 200803523. This information can be used to correct the tables stored in the lower layer. In addition, feedback information can include processing time, processing actions, processing status, and other information. Such information can be analyzed by the error control decision element 3〇6 to determine how to indicate the upper layer. The error control decision component 306 analyzes the collected information to determine how the multimedia material should be processed when the multimedia material is forwarded to the upper layer. The decision may include selecting one or more of a number of error control methods. The error control _ method may include spatial and/or temporal error concealment of the error portion of the video material. The error control method can also include an error recovery technique in which the error data is analyzed to be remedied in a manner that can be based on content or other information available to the upper application. The available time error concealed pole shape (4) (10) mail form) is known as frame rate up-conversion or fruc. Fruc builds a new frame based on other graphs (two through; ^ two frames across the frame to be constructed). The error control decision element 3〇6 may indicate that the upper layer uses $ when the wrong portion of the bead material (for example, a picture P-single frame or a large number of frames that are determined to be visually hidden) is in a manageable range. Inter- and/or time error concealment, error recovery or FRUC, and other error control mechanisms. However, if the scope of the erroneous data is too wide, the error control element can refer to the decoding of the upper part skipping the wrong part. The error control decision element 306 is discussed below to determine how to use the details of the upper layer. The eve media decoding benefit 3〇8 performs and can include audio, video closed words = two of his eve media bit stream solution. Multimedia Decoding Secondary: Corresponds to the inverse of the encoding operation used to encode the material. Encoded may be inter-frame coding (for example, time prediction data) and/or framed stone information in the box 120028.doc -19 - 200803523. Referring to Fig. 2, the power b performed by the multimedia decoder 〇8 can be performed at a plurality of layers such as the transfer layer 240/synchronization layer 235 and the application layer 25A. The communication layer function can include error detection and correction mechanisms for correcting errors and identifying uncorrectable errors. The identified uncorrectable error can be communicated to the information organizer 304 to include it in the descriptive resources as discussed above. The synchronization layer function can include buffering the receipt of the plurality of bitstreams directly with v > The material is ready to be decoded. At this time, all the synchronization data φ # is passed to the application layer decoder for almost simultaneous decoding. Application layer functions include decompression of audio, video, and closed caption bitstreams. The various decompressions can include dequantization and inverse transformations used to reconstruct the video data. In one aspect, after the information organizer 304 and the error control decision element 〇6 have performed the functions discussed above, the application layer of the video decoder component 308 receives the videogram at a frame in decoding order. frame. In some aspects, one or more of the elements of decoder 100 may be omitted, reconfigured, and/or combined. The elements can be implemented in the form of a hard body, a soft body, an intermediate, a microcode, or any combination thereof. Details of the actions performed by the elements of the decoder 15 will be discussed below in accordance with the methods illustrated in Figures 5A-5C. Figure 3B is a block diagram showing an example of a computer processor system that can be used in a decoder such as the one illustrated in Figure i. The decoder component 150 of this example includes a preprocessor element 32A, a random access memory (RAM) component 322, a digital signal processor (DSp) component μ4, and a video core component 326. The pre-processor 320 is used in an aspect to perform - or multiple actions in the actions performed by the various 120028.doc 200803523 from the figure. The preprocessor parses the video stream and writes the data to read 322. In addition, in one aspect, the preprocessor 320 implements the information organizer 234, the error control decision element 3〇6 = the pre-processing part of the multimedia decompressor 3G8 (for example, error concealment, error after the original action, etc.) Performing such more efficient, less computational intensive actions in the pre-processor 32g can perform more video decoding of the intensity in the causal order in the efficient video core 326. The DSP 324 retrieves the memory stored in the RAM 322. The parsed video data is: "re-', and is processed by the video core 326. The video core W6 performs de-living (also known as re-scaling or scaling), inverse transform and deblocking functions, and other video decompression. Function: The video core is usually implemented in a highly optimized and pipelined manner. Due to this, when the video sequence is recorded, the video data can be decoded in the fastest way. By performing disorder in the preprocessor Parsing, error detection, information organization, or error control maintains a causal order for decoding in the video core, taking into account the improved overall decoding performance. As discussed above, for the sake of error Control, the information organizer component collects descriptive information, organizes it into a table and forwards the table to the upper layer. One source of descriptive poverty is the various headers attached to the packets of various packetized layers. Figure 4 shows Description of an example of a multi-layered packet mechanism. This example packetization mechanism is used to explain some aspects of the error control system, but other packet mechanisms can also be used. The transport layer and the synchronization layer are framing & checksum protocol The layers provide a layering mechanism at various levels (including, for example, at a super frame (selected number of processed frames) level 12,026.doc 200803523, in a video access unit (VAU) Detecting errors at the level or at a PLP level. Therefore, effective error location can be performed at any or all of these layers. The application layer of the VAU containing a single video frame above the synchronization layer packet An additional level of integrity check is provided. In this example, application layer packets 405 A and 405B may be fixed and/or variable length packets. Application layer packets 405A and 405B may each be a The entire video frame or VAU. The synchronization layer adds a synchronization layer header (SH) 410 to each application layer packet 405A and 405B, resulting in synchronization layer packets 406 and 4063 (the synchronization layer packet 406 in FIG. 4) And 4066 includes a synchronization layer header 410 and includes application layer packets 405 A and 405B respectively. The synchronization layer packets 406 A and 406B are then input to the transport layer. In this example, the transport layer packet is of a fixed length. The synchronization layer packet is split into portions corresponding to the size of the transport layer packet and a transport layer header (TH) 415 is attached to the resulting transport layer packet. The transport layer may also attach a frame sum check code (FCS) to each sync. Layer package (not shown). FCS can be used to detect errors in the synchronization layer packet. In this example, the synchronization layer packet 406A containing the application layer packet 405A is divided into two transport layer packets 420A and 420B, wherein the packet 4206 includes the remaining portion 4258 of the synchronization layer packet 406 and the first portion 425C of the synchronization layer packet 406B. In this example, an additional transport layer header 415 is appended to portion 425C of transport layer packet 420B prior to the start of next sync layer packet 406B. A third transport layer packet 420D contains a portion 425D below the sync layer packet 406B. Synchronization layer header 410 and transport layer header 415 may contain similar information targeted to enable a decoder to recombine synchronization layer packets and application layer packets. A 120028.doc -22- 200803523 & head y includes such as packet size, number of packets, location of the header in the packet, data sequence time, data sequence duration, frame time, frame w horse & machine access Information on the number of points, the frame rate, and/or the number of associated packets in a group. Additionally, the header information can include stream identification information identifying the associated packet as belonging to the video bitstream, the audio bitstream, and/or the closed caption bitstream. A specific example of a transport and synchronization layer header will now be discussed. One of the functions of the φ transport layer is to provide a packet service that is superior to the octet-based service of the stream/MAC layer. The transport layer also provides a mechanism for determining the edge I of its payload packet (the VAU in the example shown in Figure 4) in the presence of a physical layer error. Multiple framing protocols can be used in association with the transport layer. The framing protocol associated with the transport layer specifies a protocol for combining its payload packets to establish a packet to be delivered to the decoder in the application layer. The boxing agreement is also used to deal with PLP errors and the rules of behavior that can be expected by the decoder. • An exemplary format for some of the fields in the transport layer header 415 in Table 1. In this example, the framing protocol provides a 122-character fixed-length PLP de-assert payload (this In addition to the beginning and end of the VAU in the example, the 'transfer header' is also used to transport the wrong PLP to the upper layer. Table 1 Block LENGTH LAST The label in Table 1 Type UNIT (7) The BIT(1) header is a subgroup length. Range 0-121 0/1 Seven-bit LENGTH field refers to 12028.doc -23- 200803523 shows the length of the payload in words and has a number from 0 to 121 words (since the PLP is 122 words long and The header is a block, so the maximum is 121). The LAST check bit set to one indicates that this transport layer packet contains the last fragment of the VAU. In this example, if the PLP is determined to be erroneous (as determined by one or more of the sum check code and/or the error correction mechanism), the transport layer sets the value of the LENGTH field to 122, thereby The PLP is marked as unavailable for the upper layer to which it is forwarded. An illustrative format for certain fields in the synchronization layer header 410 is given in Table 2. The synchronization layer packet forms a payload corresponding to the transport layer of the video. In one example, the video frame forms a synchronization layer packet. In the example shown in Table 2, the sync layer packet header 410 is a fixed 4-word header and the corresponding sync layer packet is a variable length payload corresponding to a video frame. The information included in the sync header field of Table 2 may include information such as video frame type, frame rate, presentation time stamp, random access flag, frame number in the super frame, and Whether the data is associated with a base or enhancement layer bit stream and others. Table 2 Block Name Block Type Description Stream-ED UNIT(2) 00-Video; 01-Audio; 10-Closed Caption PTS UINT(14) Presentation Timestamp Frame-ID FRAMEJD-TYPE (7) Frame-Number and Enhancement-Flag Frame—Number : Current frame number in SF Enh. Flag: 0-base; 1-Enhancement layer RAP-FLAG BIT(l) Random access point 1 PAD FRAME_RATE UINT(3) 1-KAJr 000-15 fps,001 -30 〇>s and other RESERVED UINT(5) reserved bits 12028.doc -24- 200803523

The Stream-ID field is used to privately display the <one of the payload data associated with the plurality of multimedia streams (e.g., audio, video, closed, etc.). The PTS field is used to indicate the presentation time available for synchronizing audio, ’ 疋 疋 礼, etc. The Frame_ID block includes a loop frame number (e.g., a bit representing the frame 0-127) and an enhancement bit indicating that the data is the base layer or the strong layer data. If no scalable encoding is used, the enhancement bit can be omitted. The RAP-FLAG field is used to indicate whether the frame can be decoded by a device.

Used as a - random access point. The random access point can be decoded by referring to any other previous or future picture or other part of the video. Intestines—reverse placements indicate one of a number of possible frame rates. The frame rate range can be changed from about 15 frames per frame or lower to about 60 frames per second or higher. The SERVED field can be used to convey other types of information that can be seen by those skilled in the art. In addition to the confession of 'head greed and synchronization header information, another source of descriptive information for the information organization element= can be the header directory (as described above) as a directory (in one instance) Medium) A set of copies of the header information separated by the video and/or audio bit stream, and a copy of the header information transmitted by the side information. Header directory information is listed in Table 3. 120028.doc -25- 200803523 Table 3 Block Name MESSAGEJD MEDIA TYPE NUM_VSL^RECORDS VSL_REC0RDs

Intercept Type UINT(8) UINT(2) UINT(l) VSL-RECORD-TYPE RAP-FLAG_BITS BIT(60) B_FRAME_FLAG_BITS BIT(60) RESERVED BIT(3)

Block value 5: Video sync directory 0: Video sync directory message 0: 1 VSL-records; 1:2 VSL-records VSL_record contains, 1. Frame rate 2. Frame number 3. First frame PTS 4. The last frame PTS SF in the RAP frame position bit map SF B frame position bit map TBD

The header directory can be transmitted as a variable length payload. Information in the message _ Multi-information is the copying of information in various headers of the packet mechanism (eg, _ frame rate, presentation timestamp, random access point). However, additional information may be included. This additional information may include the B-FRAME_FLAG-BITS block, which refers to the position of the B-frame in the superframe. The super frame usually begins with an independently decodable frame such as an in-frame coded frame. Other frames in the superframe usually contain a unidirectional prediction part (referred to herein as a P-frame part _ simply referred to as a P-frame) and a bi-directional prediction part (referred to herein as a B-frame part or simply referred to as B frame). In the example of Table 3, the random access points in the superframe are mapped into the RAP_FLAG_BITS field. The header directory provides header information and additional information about the location of certain frames (for example, B-Block) in the Super Frame. This information can be used to replace my lost header information (missed due to errors) and to enable the information organizer component 3〇4 to determine the possible identity of the wrong part of the otherwise unrecognizable material. Figure 5A shows A flow chart of an example of a method of processing multimedia 12028.doc -26-200803523 in a system such as that illustrated in FIG. Process 500 begins at block 5〇5, where the decoder device receives the encoded multimedia material. The encoded multimedia material may exist in the form of compressed information associated with the multimedia bitstream. The decoder device can receive the multimedia material via a wired and/or wireless network such as the network 140 shown in FIG. The multimedia material may include a plurality of synchronized and/or unsynchronized bitstreams including, but not limited to: audio, video, closed captioning, and the like. The multimedia material may include multiple packet layers including application layer packets, synchronization layer packets, and transport layer packets. The plurality of layers may each include header information as discussed above. The header information may include information such as those listed in Tables i and 2 above. Video data can be placed in sections such as frames, slices, primitive blocks, and the like. Frames can be used to form a super frame of multiple frames. The multimedia material received may also include - a header directory as discussed above. The received multimedia material can be encoded in a scalable layer such as a base layer and an enhancement layer. The header directory can contain information such as those listed in Table 3. The receiver component 3〇2 of the decoder device 15 in Fig. 3A can perform these functions at block 5〇5. After receiving the multimedia material at block 505, process 5 continues to block. At block 51G, the decoder device organizes descriptive information about the received multimedia poor. As discussed above with reference to FIG. 3A, at block 505, the information organizer 自 〇 白 自 自 自 自 自 自 自 自 自 自 自 自 自 自 自 自 自 自 。 。 。 。 。 。 。 。 。 。 。 。 The transmission header can be traced to (4) 4 to the (four) box and the super frame boundary. The header can also be processed to determine the length of the frame and the stream in the bit stream. The sync layer header can be processed to extract the frame number and resolve the base like " _ (for example, material scalable, (4) bit, 120028.doc 200803523 to extract the frame rate and / or insert and get the PTS of the frame. The sync header can also be processed to extract the presentation timestamp or extract the random access point/. If the header directory as discussed above is received at block 505, the block 51 is identified, compiled, collected, and maintained. The information indicated or generated by the flag may also be obtained from the header directory. The descriptive information organized at block 5 10 may also include information about the erroneous data. The erroneous data may include an error distribution measurement or an error rate. Measurement. Error data can be organized on any layer from a frame layer to a slice layer (slices are a set of coded primitive blocks), a primitive block layer, or even a primitive layer. These types of descriptive information can be used to locate and determine the scope of the error. An example of a table of descriptive information that can be organized at block 510 is now discussed. Table 4 lists the frames that can be generated at block 5 1 〇. An example of a news form. Similar tables can also be organized at other layers such as slices, primitive blocks, etc. Table 4 Frame Codes

Frame degree length UL2L3L4 Basic base layer Foundation PTS Frame type RAP Flag PLP error distribution PLP Error rate action PTS1 I 1 Error—dist_l 15% TBD PTS2 P 0 Error—dist2 10% TBD PTS3 P 0 Error A dist -3 0% TBD PTS4 P 0 Error_dist 4 40% TBD frame number, layer (for example, base or enhancement), frame length, PTS, frame type, RAP-FLAG field can be self-known as not wrong The sync layer header is obtained. If the header directory is received at block 505, such fields may also be obtained from the header directory. If several error frames are concatenated together (for example, 'caused by the synchronization header'), the frame length field can be set to 12028.doc -28- 200803523 to be equal to the word of the tandem frame and then The value of the number of w for the shield, and the number of w. For example, the frame type block: can be used to indicate [frame, P frame or B frame. Some of these fields cannot be filled due to washing errors. The PLP Error Distribution field is used to provide descriptive information about the location of the error data in the detected frame. Each frame may consist of several PLPs as described above with reference to Figure #. "Error-dist-η," the variable contains an indication of which part of pLp contains the error data. Several methods indicating the error distribution can be used. For example, the error distribution can be rounded up to 1/16 of the frame and can be two The word "Error-dist-η" variable is used to indicate that each bin or bit of the two-word variable indicates the presence of an error pLp corresponding to the 1/16 portion of the frame. A value of 1 indicates that there is a corresponding The range of errors pLp, and "〇,, indicates an error-free PLP part. If several frames are concatenated together, the pLp error distribution captures the total error distribution of all PLPs in the tandem frame. Here, in process 500 The last block "Action'' of the frame information table listed in Table 4 is not filled and can be determined at block 515 based on other information contained in the frame information table. It can be stored in the memory element 154 of the decoder device 150 in Figure i. The information organizer 304 of the decoder device 15 of Figure 3A can perform such functions at block 510. The organization description at block 5 10 After the sexual information, process 5 continues Passing to block 515, at block 515, the decoder device provides instructions related to the processing of the multimedia material in the second layer. The second layer can be an upper layer or a lower layer. The examples discussed above have been directed to providing The instructions are to the lower layer (eg, the transport layer and/or the synchronization layer) of the upper layer (eg, the application layer). However, the method discussed below will show that the upper layer can also be based on the descriptive capital obtained in the upper layer 120028.doc -29- 200803523 provides instructions to the underlying layer. In one aspect, the decoder device provides instructions related to the error control method to be performed in another layer (eg, the application layer). The error control method may include various error recovery techniques. In the error recovery technique, an attempt is made to remediate the variable values contained in the error f. These methods may include using the header directory discussed above (if the header directory is received at block 505) to identify the sequence layer packet map. The size of the box payload ^The header directory may contain information identifying the type of encoding, the number and size of the transport layer packets, timing information, etc. Executable error control It is hidden for errors. Error concealment techniques usually involve estimating the value of the graph from other values that have been received and/or decoded. Error concealment techniques can be hidden using time and/or space. For example, the part of the ''framelet' is Incorrect, the error concealment can be selected as time concealment based on the previous frame that has been decoded. If the part of the n frame is wrong, time prediction from two other received and/or decoded frames can be used. Another form of executable error control is FRUC. In the simple c technique, a complete frame is constructed based on one or more other frames. The C technology can use time similar to those used for the image. Hidden technology, (iv) is simply executed on the full frame. In one aspect, the error elements of decoder device 15A of Figure 3A perform such actions throughout block 515. Error Control Decision Element Which of the mis-distribution special error control techniques used in the block is used to _^ = (4) recommend the method of the various error control techniques. In some cases, error control 120028.doc -30- 200803523

The decision component 306 can determine that no error control techniques are feasible and can recommend skipping the error control of one or more frames. In this case, the last frame of the successful decoding can be displayed instead. In one aspect, the error control method determined at block 515 is stored in the "action block of the frame information table as shown in Table 3. Pass the frame information table to the layer in which the error control method is executed. The video decoder takes the corresponding frame, action, and item from the frame information table and uses it as the starting point of the guided decoding process. It should be noted that certain blocks of the process may be combined, omitted, reconfigured, or any combination thereof. Figure 5 is a flow chart illustrating another example of a method 520 of processing multimedia material in a system such as that illustrated in Figure i. The method 52 can be performed in an application layer having a decoder device that performs the method_lower layer of the ®5A in the lower layer. The method 520 begins at block 525 where the multimedia material is received at the layer of the execution method. The multimedia material can be part of a multimedia material such as a frame, slice or primitive block. In one aspect, portions of the multimedia material received at block 525 are compiled at a lower level, such as a transport layer and/or a combined transport layer packet to form a synchronization layer of a complete sync layer packet. The full sync layer packet can be either a full frame or some other part of the video that can be decoded. In some aspects, portions of the multimedia material received at block 525 are received in an order in which the portions of the multimedia material are displayed in the good media column. The multimedia decoding n subsystem of the decoder device 15 shown in Figure i can perform the operation at block 525. After receiving the multimedia material at block 525, the decoding of the process 52 is performed. 120028.doc • 31 - 200803523 The layer receives descriptive information about the multimedia material from the first layer at block 530. The first layer can be a lower layer (eg, a transport layer or a sync layer). The descriptive information received at block 530 can be identified, compiled, collected, maintained, flagged, or generated at block 5 10 of process 5 discussed above. The descriptive information received at block 530 may include the form of a frame information table such as entries for the entries shown in Table 3 or Table 4 above. The frame information table may include a "action" associated with the processing of the multimedia material. The multimedia decoder subsystem 308 of the decoder device 15A shown in Figure 1 can perform the action at block 530. After receiving the multimedia material at block 525 and receiving descriptive information about the multimedia material at block 53, the process 52 continues at block 535 where the second layer is based, at least in part, on the The descriptive information received is processed by the received multimedia material. If the descriptive information contains a recommended "action, then the decoder subsystem executing process 520 may or may not use the recommended action. As discussed above, the recommended actions may include one or more error control techniques including, but not limited to, error recovery techniques, error concealment techniques, or skip decoding. The decoder device may or may not follow the recommended actions depending on what data can be recovered during error recovery. For example, the underlying process of organizing the repetitive information received at block 53 may fail to identify how many frames are in the wrong data/knife. The upper error recovery technique may be able to identify the number of frames in the portion of the error message and may choose to perform certain error recovery or hiding techniques that are not recommended in the action field of the frame information table. The multimedia decoder subsystem 3〇8 of the decoder device 150 shown in the figure 可执行 can perform the action of the block 120028.doc -32- 200803523 5 3 5 . It should be noted that certain blocks of process 520 may be combined, omitted, reconfigured, or any combination thereof. Figure 5C is a flow diagram illustrating another example of a method 540 of processing multimedia material in a system such as that illustrated in Figure 1. Method 540 begins at block 545. At block 545, the decoder device receives the encoded multimedia material. The actions performed at block 545 may be similar to the actions performed at block 505 of process 500 illustrated in Figure 5a. Receiver component 302 of decoder device 150 in Figure 3A can perform the functions at block 545. The remaining actions of process 540 include actions 55 performed at the lower layer and actions 570 performed at the upper layer. The lower layer action 550 includes certain actions that may be similar to certain actions performed in the process 50G illustrated in Figure 5A. Similarly, the upper layer action 570 includes certain actions that may be similar to certain actions performed in the process 52 of Figure 5b. The method 540 illustrated in Figure 5C can be performed by a multi-layer multimedia decoder subsystem such as that shown in Figure 6. In one aspect, a multimedia decoder 600 includes a lower layer media module subsystem 6〇5 in the transport layer and the synchronization layer. The multimedia decoder 600 also includes an upper layer subsystem located in the application layer. The media module subsystem 605 can include the information organizer 3〇4 and the error control decision subsystem 306 illustrated in FIG. 3A. The application layer includes a multimedia decoder including a video decoding layer (VDL) 610 and an error control subsystem 615. As indicated by the up arrow 620, the underlying media module provides descriptive information and/or instructions to the upper layer. As indicated by arrow 625, upper subsystems 610 and 615 can provide feedback to the lower layers. Referring to Figure 5C', after receiving the encoded multimedia material 120028.doc -33 - 200803523 at block 545, process 540 continues at block 555 where the lower layer organizes the received multimedia. Descriptive information about the data. The actions performed at block 555 may be similar to the actions performed at block 510 of process 500 illustrated in Figure 5A. Descriptive information may include any or all of the information discussed above, such as the information described in Tables 3 and 4. After the descriptive information is organized at block 555, process 540 continues at block 560 where an instruction associated with the processing of the multimedia material is determined. The instructions may be determined based on the error distribution and other descriptive information organized at block 555. Additionally, in process 540, the lower layer receives feedback from the upper layer. The feedback may include information related to the processing of the multimedia material in the upper layer. The feedback may include information such as processing time for a particular portion of the multimedia material, processing actions performed in the upper layer (e.g., error control actions), and processing status (e.g., which frames have been decoded and displayed). Feedback can be used to reorganize descriptive information at block 555. Details of the method for determining instructions associated with the processing of multimedia material at block 560 are discussed below. The error control decision subsystem 306 of the decoder device 150 of Figure 5A can perform the actions at block 560. At block 565, the underlying subsystem provides descriptive information and/or instructions related to the processing of the multimedia material to the upper subsystem. At block 575, the upper subsystem receives descriptive information and/or instructions. The multimedia decoder subsystem 308 can perform the actions at blocks 565 and 575. After receiving the descriptive information and/or instructions at block 575, the process 54 continues at block 580 where the upper subsystem is based on instructions 12028.doc -34-200803523 and/or a description Sexual information to process multimedia materials. The actions performed at the block may be similar to the actions performed by the other block milks of the method illustrated in Figure 5B. If the descriptive information contains a recommended "action", then the decoder subsystem performing process 540 may or may not use the recommended one. As discussed above, the recommended actions may include - or multiple erroneous techniques, including (but limited to) error recovery techniques, error concealment techniques, or skip decoding. The decoder device may rely on or may not follow the recommended action 2 example in the event that the error recovery φ ^ can be recovered. [At the block 555, the descriptive information below the organization process Z can fail to identify the error data. How many frames are in the section. The upper-level error recovery technique may be able to identify the number of frames in the error data and optionally perform some error recovery or hiding techniques that are not pushed in the "(4)" field of the frame information table. The multimedia decoder subsystem 308 of the decoder device 15 can perform the action at block 58. Process 540 continues at block 585, where the upper multimedia player is based on the upper layer action The processing performed in 570 is referred to as feedback information, and the lower layer feedback may include processing time required to decode a portion of the multimedia material or processing time of a portion of the fully decoded data. By comparing the processing time and the processing at block 545 The presentation timestamp of the received new multimedia material is 5, and if the upper processing time is displayed based on the past processing performance, the lower layer process may instruct the upper layer to skip some frames (for example, 'B frame'). The feedback information received at the lower layer is organized into descriptive information organized at block 555. The feedback may also include information about the processing actions performed in the upper layer. In the case of δ, feedback may refer to specific error control techniques and/or normal decoding actions that are not performed on a particular frame or other portion of the multimedia material. Feedback may also include processing spoofs (eg, The frame is successfully decoded or unsuccessfully decoded. By including the processing action and processing state feedback information in the data organized at block 555, the lower layer can be adjusted at block 560 based on the updated descriptive information. If the process is backed up, the lower layer may instruct the upper layer to skip decoding of certain frames such as frames or enhancement layer data. The multimedia decoder subsystem 3〇8 of the decoder device 150 shown in Figure i The actions at block 585 may be performed. It should be noted that certain blocks of process 540 may be combined, omitted, reconfigured, or any combination thereof. Figure 7 is a diagram illustrating an organization that may be used to perform the methods illustrated in Figures 5A and 5C. A flowchart of an example of a method of descriptive information for certain actions in the process. The process 700 can be performed at block 5 1〇 of process 500 illustrated in FIG. 5A or process 54 illustrated in FIG. 5C. Descriptive information is organized at block 555. Process 700 begins at block 705 where a supermap of multimedia material received, for example, at block 5〇5 in process 500 is performed. The box is stored in a memory buffer. The super frame is a set of frames that can usually be independently decoded. The super frame can include a frame covering a fixed time period ranging from about 2 seconds to about 2 seconds. The size of the super frame can also be determined according to a fixed number of constituent frames to have a variable time period. The super frame size can be selected to allow a reasonable acquisition time. The super frame of the multimedia material is stored at block 705. Thereafter, process 700 continues at block 710 where it is determined whether the material includes multiple layers (eg, a base layer and one or more enhancement layers). If a single data layer is encoded only in the superframe, then 120028.doc • 36-200803523 Process 700 continues at block 715A. If two or more data layers are encoded in the superframe, then process 700 continues at block 715B. The superframe header can contain a flag indicating whether there are multiple layers in the superframe. At block 715A or 715B, the frame information table (fit) is initialized. Initializing FIT can be performed to set the field to some preset value. After initializing the FIT, process 700 proceeds to block 720A or block 720B, depending on whether the superframe contains multiple layers. In either case, the information contained in the optional header directory is entered at block 720A or block 720B. The header directory may contain any of the information as discussed above. The fit is initialized at block 715A or block 715B, respectively, and after the optional header directory is entered at block 720A or block 720B, process 7 advances to block at block 72 5-740 or block 745- At 760, the super frame is processed with a loop. At block 730 and block 750, the decoder device identifies the complete video access unit (VAU), and the decoder device can identify the complete video access unit via the available header information. The header information may include, for example, any of the transmission headers or synchronization headers (or any other headers) shown in Tables 1 and 2. Information in the optional header directory can also be used. The VAU in process 7 is assumed to be a frame, but other portions such as slices or blocks may also be identified at block 730 or block 75A. After identifying a complete VAU, an error score for the video material in the identified VAU is identified at block 735 or block 755, respectively. The error portion can be identified by the failure of the header sum check code or the failure of the transfer layer sum check code. Many techniques for detecting erroneous data are known to those skilled in the art. The error section can be used to compile FIT error distribution information (see PLP Error Distribution and PLP Error Rate Fields in Table 4). 120028.doc -37- 200803523 After identifying the cone error portion of the VAU at block 735 or block 755, the FIT information is organized at block 740 or block 760, respectively. Information in FIT can include

Any of the information in the information discussed in Table 4. Process 7 continues to process the superframe (blocks 725-740 or blocks 745-760) with loops until the end of the hyperframe is identified at decision block 725 or block 745. When the end of the superframe is identified, process 700 proceeds to block 800 where an error control action is determined. The information organizer component 304 of the decoder device 150 of Figure 3A can perform the acts of the process 700. It should be noted that certain blocks of process 700 may be combined, omitted, reconfigured, or any combination thereof. 8A and 8B are flow charts illustrating an example of a method of determining an error control action in the method illustrated in Fig. 7. Process 8 can also be performed to determine the error control action and at block 515 of process 5, illustrated in Figure 5-8, or at blocks 56 and 565 of process 54, illustrated in Figure 5C. Provide the appropriate instructions. In one aspect, the process 8〇〇 is used to determine the erroneous control action to be provided to the upper layer. For example, the error control action is determined based on the error distribution of the multimedia frame and the error rate. In the other, the process, the process identifies the other parts of the multimedia that the upper layer may use when performing the recommended error control actions. The description can be coded = media data in multiple layers such as 即 又 一 I I I I I I I I I I I I I 鬲 多个 多个 多个 多个 多个 多个 多个 多个 多个 多个 多个 多个 多个 多个 例如 例如 例如 例如 = =. The enhancement layer may also contain all the information in the (iv) box. Any of the FITs contains a portion of the base layer and the enhancement layer and is described in the layers; in:: is erroneous. Figure 9 illustrates a structure including an example of a physical layer seal 120028.doc-38-200803523 package for use in, for example, Figure 1 M, a scaled code base layer, and an enhancement layer. The base layer 900 contains a plurality of PLp 9i〇, pLp 9i〇

The transport layer header 915, the sync layer header 920, and the transport layer sum check code tail P 925 base layer 905 can be combined with, for example, frame 93 (the work frame labeled ^ and such as frame 935 (labeled F3rp map) The most significant bit of the box. The enhancement layer 95G also has a PLP 91G, a transmission header 915, a synchronization header 92, and a transport layer sum check code 925. The enhancement layer in this example is shown in Figure 93 (labeled Fr). And P frame 935 (labeled as the least significant bit. Additionally, enhancement layer 950 includes a synchronization layer packet of full B frame 940 (labeled F2) bi-predicted from I frame 93〇 and p frame 935, The base layer and enhancement layer pairs, Fi, and h, I are combined and decoded prior to constructing the B-frame & the process is considered to consider scalable coding of this form. Process 800 begins at block 805. At block 8.5, the decoder device integrates the header information into portions of the FIT containing the erroneous VAUs (such as those identified at blocks 735 and 755 in process 700 as illustrated in Figure 7). Header information can be obtained from the correct receiving transmission and/or synchronization layer header or obtained from the header directory (if received Header directory). If the error data cannot be isolated from the data of the special PLP (for example, due to loss of synchronization), the header directory information can be used to identify the PLP boundary and possibly fill the FIT "pLp error" and/or "PLP error rate" block. If the header directory is not available, the PLP error rate can be set to 1QQ%. The exemplary method $(9) uses the "pLp error when determining which error control is recommended" action Rate, however, those skilled in the art will understand how to use the 1>1^Error Distribution information and other forms of error data when determining the "action". The block 805 is populated with the error frame. After the FIT is blocked, the process proceeds to 12028.doc -39 - 200803523. The process 800 continues to process the frame in the superframe with the loop at block 810. At decision block 810, the decoder device checks the FIt. PLP error rate data and determine if the number of consecutive lost (ie, error) frames is greater than the threshold ''lost-th". If the gray of the consecutive lost frames exceeds the threshold, then process 800 is in the block 815 continues in the area At block 815, the 'action' field of the missing frame is set to a value that is recommended to skip the decoded frame of the lost frame. The "lost-th" threshold can be set to other error control techniques. The number of frames that are determined to be invalid or fully degraded so as not to be guaranteed. The threshold ''lost-th'' can range from about 3 frames to about 6 frames. When one is greater than one corresponding to each When the time interval of the three frames of the frame rate of 30 frames per second is executed, the performance of the time concealment technique is usually degraded. The faster frame rate allows for greater margins, such as from about 6 frames to about 12 frames at a frame rate of 6 frames per second. After the "action" for the missing frame is set to skip at block 815, process 8 continues to decision block 820. If the end of the superframe has been reached, the process continues to the remainder of the process illustrated in Figure 8B. If more frames are left to process, then process 800 continues to return to decision block 81. At decision block 810, if the number of consecutively lost frames does not exceed the threshold (including the condition of the completely error-free frame), then process 8 continues at decision block 825 where the decision block 8 is continued. However, the FIT "Frame Type" block is used to determine whether the current frame is a B-frame. The error control action performed on the B-frame is different from the one performed on the P-frame and the frame in this example. If the current frame is not a B-frame, then process 800 continues at decision block 830 where the PLP error rate (PLPJERR) and the threshold are 120028.doc -40-200803523 The value Ρ_ΤΗ is compared. The threshold ρ_τη sets a limit on the PLP error rate, and the normal error concealment technique (for example, space and time error concealment) is valid. The P_TH threshold can be about 2〇% to Within a range of approximately 4%. If the PLP error rate exceeds the P_TH threshold, then at block 835 the "action" for the current frame is set equal to skip. If the pLp error rate does not exceed = threshold Value, then at block 840, the "action" for the current frame is set to A value indicating the execution of normal error concealment (EC). After the "action" for the current frame is set at block 835 or block φ MO, process 8 continues to decision block 820 and as above If the more frames remain in the super frame, then the loop returns to block 8 1 〇. Return to decision block 825 'If the current frame is determined to be B frame, then

The process continues to decision block 845. In the example shown, it is assumed that the B frame is between the I frame and the P frame, or between the two p frames. If the previous frame, action &quot; is determined to be a &quot;action&quot;, then at block 850, process 800 sets the &quot;action&quot; of the current B frame to skip. Since the data from the ## prediction current B frame is not available, the normal construction of the B frame is not feasible and can also downgrade other error concealment options. Returning to decision area 堍 845, Ge 1 , right is not the first frame, action &quot; determined to skip, then process _ proceeds to block (5), where the PLP error rate and the other A threshold value b-twist comparison. If PL? is wrong, then at block 86, for the current frame, the action &quot; is again FRUC, otherwise at block (6), the action will be for the current frame. It is also hidden for the wrong hanging. The normal error for the close box in the & instance is hidden as a time prediction from the two decoded frames. These frames usually include 120028.doc -41- 200803523 with a frame before the beta frame and a frame after the B frame. However, two previous or two subsequent B-frames can also be used. The space concealment using the error-free portion of the current b-frame can also be used with the hang-up error concealment determined at block 86〇. Since there are two reference frames to choose from and there is no need to use both in the prediction, the threshold B_TH can be higher than the threshold P-TH for the p-frame. However, in some cases, fruC may be more robust and may be better hidden than normal error concealment and thus the value of B_τη may be set to a value lower than P-TH. The values of B-TH and Ρ-ΤΉ may depend on the type of channel conditions and how to introduce the wrong condition D. The hiding of FRUC may be similar to the normal B-frame error concealment, but it is performed on all frames. After the &quot;action has been made to all of the frames in the superframe, after decision, at decision block 820, process 8 continues to block 87 of Figure 8B. The loop from blocks 875 to 895 fills the FIT table with the resources available to the upper layer to perform the determined &quot;action,&quot;. At block 870, process 800 initiates another pass via FIT (starting at the beginning) and processes all of the base and enhancement layer frames with a loop. At decision block 875, if the current frame is not a beta frame or a frame to be hidden using FRUC, then the process continues at block 88, where a time error concealment is used. The variable skip one num to fill the FIT table. The skip-num variable indicates the number of frames in the current frame from the time frame, and the current frame is predicted by using the time error to hide the number of frames. Figure 10A graphically illustrates the position of the current p-frame 1〇〇5 and a previously decoded p-frame 1〇1〇 of the two frames preceding the current frame. In the example 120028.doc -42- 200803523, the skip-num variable is set equal to three. Therefore, P frames 1〇15 and 1020 skipped by the decoder will not be used. Instead, the motion vector 1025 of the current p-frame 1005 (see the scaled motion vector ι〇3〇) can be scaled to point to the previously decoded P-frame 1010. Figure 10A illustrates the frame as one-dimensional, but it is actually two-dimensional and the motion vector 1025 points to the two-dimensional position in the previous frame. In the example of Fig. 10A, the object 1〇35 in the frame 1〇10 is moved upward in the frame 1005. If the motion of the object is relatively constant, the linear extrapolation of the motion vector 1025 can be accurately pointed to the correct position in the frame , 1 , so that the object 1035 is repositioned upward in the frame 1 〇〇 5 . The display position of the object 1 〇 35 can be kept in the skipped frames 1015 and 1020. Returning to Figure 8B, after determining the skip-num variable of the current frame, process 800 continues at block 885, where the flag MV_FLAG of the frame indicated by the skip-num variable is set to A value indicating that the upper decoder should save the decoded value of the frame for future error concealment. Figure 10B graphically illustrates the flag representation of the decoded frame for error concealment of other error frames. In the example of Figure 10B, the decoded frame 1040 is flagged for hiding the error frame 1045 using normal error concealment. The decoded frames 1050 and 1055 are all flagged for execution of the FRUC for block 1060. These other combinations of only the examples and the preceding and/or following frames can be used for normal error concealment and FRUC. Returning to Figure 8B, after setting the MV_FLAG to be used to hide the frame of the current frame, process 800 continues at block 895. At block 895, the decoder checks to see if the end of the superframe has been reached. If the end of the superframe has been detected, the process 800 ends with the current superframe. If the more 120028.doc -43- 200803523 frame remains in the super frame, then process 800 returns to decision block 875 to process the remaining frames with a loop. At block 875, the right current frame is a b frame or a fruC: hidden frame is used. Then the process 800 continues at block 890, where the location is determined. The positions of the two frames are used to perform bidirectionally predicted variables B-NUM and b-num. Figure 〇C illustrates, by way of illustration, the variables used to indicate the location of the two decoded frames used to locate the error frame of the FRUC 13 (the same variable can be used for an erroneous B-frame). The current error frame 1〇65 has been confirmed to be hidden using FRUC. The variable b_num is set to two to indicate that a previous frame 1070 located outside of the two frames is the first reference frame. In the illustrated example, block 1075 is predicted from block 1070 using the received motion vector 1 〇 85. The variable B-NUM is set equal to 3 to indicate that the frame 1 〇 75 of one of the three frames before the frame 1070 is the second reference frame. In the illustrated example, the decoded frame 1075 is predicted from block 1 070 using the received motion vector 1085. The received motion vector 1085 (resulting in a scaled motion vector 1〇9〇) can be scaled to point to error frame 1065. The decoded portion of motion vector 1 〇 85 received by frames 1075 and 1070 can then be used to hide a portion of frame 1065 that is positioned by the scaled motion vector i 〇 9 。. In this example, b frame 1〇8〇 is not used to hide frame 1065 (B frame is usually not used for prediction). Normally, the closest correctly decoded frame will be used to perform error concealment. Both decoded frames can also be in front of or behind the hidden error frame. After filling the FIT with the B-NUM and b-num variables at block 890, the process continues to process the block 875-895 with the loop until the FIT of the entire super frame is filled with 120028.doc •44-200803523. Process 800 ends. In one aspect, before transferring individual frames and FITs to the upper layer for decoding, use Process 7〇〇 and Process 8〇〇 to populate FIT for all frames in the Super Frame. In this way, the frames can be forwarded in the order in which they were decoded. In addition, frames that should be skipped may or may not be forwarded. In another aspect, once both process 700 and process 800 are completed for a frame, the corresponding entries of the frame and FIT can be forwarded to the upper layer. The error control decision subsystem 〇6 of the decoder device 150 in FIG. 3A can perform the actions of the process 800. The illustrative processes 700 and 800 use the frame as a VAU. However, the VAU can also be a block of slices or primitives and can be filled with FIT corresponding to these parts instead of the frame. It should be noted that certain blocks of processes 700 and 800 may be combined, omitted, reconfigured, or any combination thereof. 11 is a functional block diagram illustrating another example of a decoder device 150 that can be used to process multimedia material in a system such as that illustrated in FIG. The malicious sample includes: means for receiving multimedia material; means for organizing descriptive information about the multimedia material in the first layer, wherein the descriptive information is related to processing of the multimedia material in the second layer; and Means for providing instructions related to processing of multimedia material in the second layer based at least in part on the descriptive information. Some examples of this aspect include: wherein the receiving component includes a receiver 1102; wherein the tissue component includes an information organizer subsystem 1104; and wherein the providing component includes an error control decision subsystem 1106. Figure 12 is a functional block diagram illustrating another example of a decoder device 150 that can be used to process multimedia material in a system such as that illustrated in Figure 1. The 120028.doc -45-200803523 aspect includes: means for receiving multimedia material; means for organizing descriptive information about the multimedia material in the first layer, wherein the descriptive information and the multimedia material in the second layer Processing associated; and means for providing instructions related to processing of the multimedia material in the second layer based at least in part on the descriptive information. Some examples of this aspect include: wherein the receiving component includes a module 1202 for receiving; wherein the tissue component includes a module 1204 for organizing information; and wherein the providing component includes a Module 1206 of the instruction. Figure 13 is a functional block diagram illustrating another example of a decoder device 15 that can be used to process multimedia material in a system such as that illustrated in Figure 1. The malicious sample includes: means for receiving multimedia material; means for processing multimedia material in the upper layer; means for indicating a layer based at least in part on information associated with processing of the multimedia material in the upper layer; The means for processing the multimedia material in the lower layer based at least in part on the information associated with the processing of the multimedia material in the upper layer. Some examples of this aspect include: # wherein the receiving component includes a receiver 1302; wherein the upper layer processing component includes an application layer multimedia decoder subsystem 13A8, wherein the means for indicating includes application layer multimedia decoding The subsystem 1308; and the lower processing element thereof includes a transport/synchronization layer multimedia decoder subsystem 1306 °. FIG. 14 is a diagram illustrating another decoder device 150 that can be used to process multimedia material in a system such as that illustrated in FIG. A functional block diagram of an example. This aspect includes: means for receiving multimedia material; means for processing multimedia material in an upper layer; for at least in part based on information associated with processing of multimedia data in the upper layer of the medium 12028.doc • 46-200803523 Denoting the components of the layer; and means for processing the multimedia material in the lower layer based at least in part on the information associated with the processing of the multimedia material in the upper layer. Some examples of this aspect include: wherein the edge receiving component includes a module i 4 〇 2 for receiving; wherein the upper layer processing component includes a module 1408 for processing multimedia in the upper layer; wherein the The component includes a module 1408' for processing multimedia in the lower layer and the lower processing component includes a module 1406 for processing multimedia material in the lower layer. 15 is a functional block diagram illustrating another example of a decoder device 150 that can be used to process multimedia material in a system such as that illustrated in FIG. The aspect includes: means for receiving multimedia data; means for receiving descriptive information about the multimedia material from the first layer, wherein the descriptive information is related to processing of the multimedia material in the second layer; and The means for processing the multimedia material in the second layer based at least in part on the received descriptive information. Some examples of such aspects include: wherein the means for receiving multimedia material comprises a receiver 1502; wherein the means for receiving descriptive information comprises a multimedia decoder 1508; and wherein the processing component comprises a multimedia decoder 1508. 16 is a functional block diagram illustrating another example of a decoder device 150 that can be used to process multimedia material in a system such as that illustrated in FIG. The aspect includes: means for receiving multimedia data; means for receiving descriptive information about the multimedia material from the first layer, wherein the descriptive information is related to processing of the multimedia material in the second layer; and The 12028.doc-47-200803523 component of the multimedia material is processed in the second layer based at least in part on the received descriptive information. Some examples of the aspect include: the means for receiving multimedia material includes a module 1602 for receiving; wherein the means for receiving descriptive information comprises a module for decoding multimedia 16 8 And the processing component therein includes a module 1608 for decoding multimedia. Those of ordinary skill in the art will appreciate that information and signals can be represented using any of a variety of different technologies and methods. For example, data, instructions, commands, information, signals, bits, etc., which may be referred to by the above description, may be represented by voltages, currents, electromagnetic waves, magnetic fields or particles, light fields or particles, or any combination thereof. Symbols and chips. It will be further appreciated by those skilled in the art that various illustrative logic blocks, modules, and algorithm steps described in connection with the examples disclosed herein can be implemented as electronic hardware, firmware, computer software, intermediates, Microcode or a combination thereof. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. This functionality is implemented as hardware or software depending on the specific application and the design constraints imposed on the overall system by Φ. Those skilled in the art can implement the described functionality in a different manner for each particular application, but this implementation decision should not be construed as causing a departure from the scope of the disclosed method. The various illustrative logic blocks, components, modules, and circuits described in the examples disclosed herein may be a general purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), A programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein for implementation or execution . The general purpose processor may be a microprocessor 120028.doc -48 - 200803523' or the processor may be any conventional processor, controller, microcontroller or state machine. The processor can also be implemented as a combination of computing devices, such as 'DSP and micro-combination, multiple microprocessors n, combined with a DSP core ASIC core—or multiple microprocessors, or any other such configuration. The steps of a method or algorithm described in connection with the examples disclosed herein may be embodied in a hardware, a software module executed by a processor, or a combination of both. The software module can reside in Ram memory, flash memory, ROM memory, EPR 〇M memory, 腾 r〇m memory, scratchpad; · hard disk, removable disk, (3)-, Optical storage media or any other form of storage medium known in the art. An exemplary storage medium is interfaced to the processor such that the processor can read information from the health storage medium and write the poor message to the storage medium. Alternatively, the health media can be integrated into the processor. The processor and library media can reside in a special application integrated circuit (ASK) where the ASIC can reside in the wireless data machine. Alternatively, the processor and the storage medium can reside in the wireless data unit as discrete components. The previous description of the disclosed examples is provided to enable a person skilled in the art to utilize or use the disclosed methods and apparatus. Various modifications to these examples will be readily apparent to those skilled in the art, and the principles defined herein may be applied to other examples and additional elements may be added. Thus, methods and apparatus for performing highly efficient and robust error control of multimedia material have been described. BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a block diagram showing a multimedia communication system according to an aspect. 120028.doc -49- 200803523 DEVICE DEVICE 110 </ RTI> </ RTI> </ RTI> Figure 2 is an example of a multi-layer protocol stack that includes the use of one of the inter-layer error control systems of the encoder and decoder device 150 in the system illustrated in Figure 1. Block diagram. Decoder Setup Figure 3A is a block diagram illustrating one aspect of the system that can be used, such as illustrated in Figure 1. Decoding Setup Figure 3B is a block diagram illustrating one example of a computer processor system that may be used in a system such as that illustrated in Figure i. φ Figure 4 shows an illustration of an example of a multi-layered packeting mechanism. Figure 5A is a flow chart illustrating an example of a method of processing multimedia material in a system such as that illustrated in Figure 。. Figure 5B is a flow chart illustrating another example of a method of processing multimedia material in a system such as that illustrated in Figure i. Figure 5C is a flow chart illustrating another example of a method of processing multimedia material in a system such as that illustrated in Figure i. 6 is a block diagram of an example of a multi-layer multimedia coder unit that can be used to perform the method illustrated in FIG. 5C. Figure 7 is a flow diagram illustrating an example of a method of organizing descriptive information for performing certain of the methods illustrated in Figures 5 and 5C. 8A and 8B are flow charts illustrating an example of a method of determining an error control action in the method illustrated in Fig. 7. 9 is a diagram showing the structure of an example of a physical layer packet including a scalable stencil base layer and an enhancement layer for a system such as that illustrated in FIG. Figure 1 〇 A graphically illustrates the position of a current p-frame and a previously decoded p-frame of two frames located in the current frame. 120028.doc -50· 200803523 Figure 1 〇B graphically illustrates the flag representation of the decoded frame for error concealment of other error frames. Figure 10C graphically illustrates variables used to indicate the location of two decoded frames for hiding an error frame using FRXJC. 11 is a functional block diagram illustrating another example of a decoder device 150 that may be used in a system such as that illustrated in FIG. FIG. 12 is a functional block diagram illustrating another example of a decoder device 150 that may be used in a system such as that illustrated in FIG. 1. FIG. 13 is a functional block diagram illustrating another example of a decoder device 150 that may be used in a system such as that illustrated in FIG. 1. FIG. 14 is a functional block diagram illustrating another example of a decoder device 150 that may be used in a system such as that illustrated in FIG. 1. Figure 15 is a functional block diagram illustrating another example of a decoder device 150 that may be used in a system such as that illustrated in Figure 1. Figure 16 is a functional block diagram illustrating another example of a decoder device 150 that may be used in a system such as that illustrated in Figure 1. [Major component symbol description] 100 Multimedia communication system 102 External source 110 Encoder device 112 Processor 114 Memory 116 Transceiver 140 Network 120028.doc _51_ 200803523 150 Decoder device 152 Processor 154 Memory 156 Transceiver 205 Application layer ~ 210 Synchronization Layer 215 Transport Layer 220 Stream and/or Media Access Control (MAC) Layer 225 Physical Layer 230 Application Layer 235 Synchronization Layer 240 Transport Layer 245 Stream and/or Media Access Control (MAC) Layer 250 Physical Layer 255 Error Elastic System 260 Error Recovery System 302 Receiver Element 304 Information Organizer Element 306 Error Control Decision Element 308 Multimedia Decoder Element 320 Preprocessor Element 322 Random Access Memory (RAM) Element 324 Digital Signal Processor (DSP) Element 326 Video core component 12028.doc -52- 200803523

405A, 405B application layer packet 406A, 406B synchronization layer packet 410 synchronization layer header 415 transport layer header 420A, 420B, transport layer packet 420D 425B synchronization layer packet 406A remaining portion 425C synchronization layer packet 406B first portion 425D synchronization layer packet 406B part 600 multimedia decoder 605 lower layer media module subsystem 610 video decoding layer / VDL 615 error control subsystem 620 arrow 625 arrow 905 base layer 910 PLP 915 transport layer header 920 synchronization layer header 925 transport layer sum check Code tail 930 frame 935 frame 940 B frame 950 enhancement layer 12028.doc -53- 200803523

1005 Current P Frame 1010 Decoded P Frame 1015 &gt; 1020 Skipped P Frame 1025 Motion Vector 1030 Scaled Motion Vector 103 5 Object 1040 Decoded Frame 1045 Error Frame 1050, 1055 Decoded Figure Block 1060 Frame 1065 Current Error Frame 1070 Previous Frame 1075 Decoded Frame 1080 B Frame 1085 Motion Vector 1090 Scaled Motion Vector 1102 Receiver 1104 Information Organizer Subsystem 1106 Error Control Decision Subsystem 1202 for Reception The module 1204 is used to organize the information module 1206 for providing the module of the instruction. 1302 Receiver 1306 Transmission/synchronization layer multimedia decoder subsystem 120028.doc -54- 200803523 1308 Application layer multimedia decoder subsystem 1402 For receiving modules, 1406 is used to process multimedia data in the lower layer, module 1408 is used to process multimedia in the upper layer, 1502, receiver 1508, multimedia decoder 1602 is used to receive module 1608, which is used to decode multimedia. Group 120028.doc -55-

Claims (1)

200803523 X. Patent application scope: 1 · A method for processing multimedia data, comprising: receiving the multimedia material; organizing descriptive information about the multimedia material in a first layer, wherein the descriptive information and a second The processing of the multimedia material in the layer is related; and providing instructions related to the processing of the multimedia material in the second layer based at least in part on the descriptive information. 2. The method of claim 1, further comprising communicating the descriptive information to the second layer. 3. The method of claim 1, wherein the descriptive information includes one of frame feature information, basic or enhanced data identification information, timing information, a coding type, a frame type, synchronization information, and prediction coding related information. Or more. 4· 2 'Method of claim 1' wherein the multimedia material contains an error message method further includes organizing the descriptive information to include information indicating the error-error distribution in the multimedia material. 5 · If the method of claim 4 is used, f is a misclassification ▲ seven, and 乂匕 is based at least in part on the error, Bessie determines the instructions. 6. If requested! The eve of the instruction (4) is further based on, at least in part, the processing of the multimedia material. 8_If the request is for: Eight claims that the descriptive information contains metadata. The method, the further information of the distribution is confirmed 佘 ^ 3 based at least in part on the error confirmation - error control method, wherein the second layer of the 120028.doc 200803523 these instructions are related to the determined error control method. 9. The method of claim 8, wherein the determined error control method comprises one or more of error recovery, error concealment, and insertion of a frame. 10. An apparatus for processing multimedia material, comprising: a receiver configured to receive the multimedia material; a poor organizer configured to organize the multimedia poor in a first layer Descriptive information relating to the processing of the multimedia material in a second layer; and an error control decision subsystem configured to provide the same based at least in part on the descriptive information The processing related instructions of the multimedia material in the second layer. 11. The device of claim 10, wherein the information organizer is further configured to pass the descriptive information to the second layer. 12. The device of claim 1, wherein the descriptive information comprises a frame feature information base or enhanced data identification information, timing information, a coding type, a frame type, synchronization information, and prediction coding related information. One or more. 13. The device of claim 10, wherein the multimedia material includes an error message and the &quot;Chai Media Data Processor is further configured to organize the descriptive information to include information indicative of an incorrect distribution of the error material In the multimedia material. 14. The apparatus of claim 13, wherein the error control decision subsystem is further configured to determine the instructions based at least in part on the error distribution information. 15. The device of claim 1 further comprising a multimedia decoder, the 120028.doc 200803523 multimedia decoder configured to change the multimedia material in the second layer based at least in part on the instructions deal with. 16. The device of claim 10, wherein the descriptive information comprises metadata. 17. The apparatus of claim 10, wherein the error control decision subsystem is further configured to determine an error control method based at least in part on the error distribution information, wherein the instructions to the one of the layers are incorrectly determined by the disc Control method related. _ I8. The apparatus of claim 17, wherein the determined error control method comprises one or more of error recovery, error concealment, and insertion of a frame. 19. An apparatus for processing multimedia material, comprising: means for receiving the multimedia material; means for organizing descriptive information about the multimedia material in a first layer, wherein the descriptive information and The processing of the multimedia material in the second layer is related; and means for providing an instruction related to the processing of the multimedia material in the second layer based at least in part on the descriptive information. 2. The device of claim 19, further comprising means for communicating the descriptive information to the second layer. 21. The device of claim 19, wherein the descriptive information comprises frame feature information, basic or enhanced data identification information, timing information, a coding type, a frame type, synchronization information, and prediction coding related information. One or more. 22. The device of claim 19, wherein the multimedia material comprises an error message, wherein the organizational component organizes the descriptive information to include information indicative of an error distribution of one of the errors 120028.doc 200803523 in the multimedia material. 23. The device of claim 22, further comprising means for determining the instructions based at least in part on the error distribution information. 24. The apparatus of claim 19, further comprising means for altering the processing of the multimedia material in the second layer based at least in part on the instructions. 25. The device of claim 19, wherein the descriptive information comprises metadata. 26. The device of claim 19, further comprising means for determining an error control method based at least in part on error distribution information, wherein the instructions provided to the second layer are associated with the determined error control method. 27. The device of claim 26, wherein the determined error control method comprises one or more of error recovery, error concealment, and insertion of a frame. 28. A machine readable medium comprising code for causing the one or more machines to perform program operations when the code is executed on one or more machines H, the code comprising: _ for receiving multimedia material a code for organizing descriptive information about the multimedia material in the first layer, wherein the descriptive information is related to the processing of the multimedia material in a second layer, and is used to be based, at least in part, on The descriptive information provides a code of an instruction associated with the processing of the multimedia poor material in the second layer. 29. The machine readable medium of claim 28, further comprising code for communicating the descriptive information to the second layer. 3. The machine readable medium of claim 28, wherein the descriptive information includes a map 120028.doc 200803523 1 匡 feature information, basic or enhanced data identification information, timing information, a 'type of exhaustion', type of frame, One or more of synchronization information and predictive coding related information. 31. The machine-readable medium of claim 28, wherein the multimedia material comprises an error=bedding, and further comprising information for organizing the descriptive information to misinterpret one of the errors&gt; The code in the multimedia material. The machine readable medium of claim 31, further comprising code for determining the instructions based, at least in part, on the error distribution information. The machine of claim 28, wherein the medium is commercially available, further comprising code for at least partially changing the processing of the multimedia material in the second layer based on the instructions. 34. The machine readable medium of claim 28, wherein the descriptive information comprises metadata. 35. The machine readable medium of claim 28, further comprising code for determining an error control method based at least in part on error distribution information, wherein the instructions provided to the second layer and the determined error control Method related. 36. The machine readable medium of claim 35, wherein the determined error control method comprises one or more of error recovery, error concealment, and insertion of a frame. 37. A method of processing multimedia material, comprising: receiving the multimedia material; processing the multimedia material in an upper layer; 120028.doc 200803523 based at least in part on an information indication associated with the processing of the multimedia material in the upper layer The lower layer; and processing the multimedia material in the lower layer based at least in part on the information associated with the processing of the multimedia material in the upper layer. The method of claim 37, further comprising organizing descriptive information about the multimedia material in the lower layer based at least in part on the information associated with the processing of the multimedia material in the upper layer. 39. The method of claim 38, further comprising providing an instruction related to the processing of the multimedia material in the upper layer based at least in part on the descriptive poorness. 4. The method of claim 38, wherein the descriptive information comprises metadata. 41. The method of claim 38, wherein the descriptive information comprises one of a frame feature basis or an enhanced data identification information, timing information, a coding type, a frame type, synchronization information, and prediction coding related information. Or more. 42. The method of claim 37, wherein the instructing the lower layer comprises: transmitting information including one or more of a processing time, a processing action, and a processing state. 43. An apparatus for processing multimedia material, comprising: a receiver configured to receive the multimedia material; an upper decoder subsystem configured to process the multimedia material in an upper layer, And indicating, at least in part, based on information associated with the processing of the multimedia material in the upper layer; and a lower layer decoder subsystem configured to be based at least in part on the processing of the multimedia material in the upper layer The associated information is processed in the next 12228.doc 200803523 layer. 44. The apparatus of claim 43, wherein the step-by-step includes a poor organizer that is at least partially based on the information associated with the multimedia 'processing in the upper layer Descriptive information about the multimedia material is organized in the lower layer. The device of claim 44, which further includes an error control decision subsystem, the error control decision subsystem is configured to provide at least in part based on the • descriptive information The apparatus of claim 44, wherein the descriptive information comprises metadata. 47. The apparatus of claim 44, wherein the descriptive information includes One or more of frame feature information, basic or enhanced data identification information, time series information, a code type, a frame type, synchronization information, and predictive coding related information. ~ 48. The device of claim 43, wherein the upper layer The decoder subsystem further sets the state to indicate the lower layer by passing information including one or more of processing time, processing action, and processing status. An apparatus for processing multimedia material, comprising: means for receiving the multimedia material; means for processing the multimedia material in an upper layer; for, at least in part, relating to the processing of the multimedia material in the upper layer The associated message refers to a component of the lower layer; and a component for processing the multimedia material in the lower layer based at least in part on the information associated with the processing of the multimedia material in the upper layer. The apparatus of claim 49, further comprising, for at least in part, the information associated with the processing of the layered data, and the descriptive information about the multimedia material member. The device of claim 5, further comprising means for providing an instruction related to the processing of the multimedia material in the upper layer based at least in part on the description. The device of claim 50, wherein the descriptive information includes metadata. A device for asking for 50, wherein the descriptive information includes one of frame feature a:, basic or enhanced data identification information, timing information, a coding type, a type of box, synchronization information, and prediction coding related information. More. The apparatus of claim 49, wherein the means for indicating the lower layer comprises means for transmitting information including one or more of processing time, processing action, and processing status. 55. A machine-readable medium comprising a code for causing the one or more machines to perform program operations when the code is executed on one or more machines II, the code comprising: for receiving multimedia material a code for processing the multimedia material in the upper layer; for indicating a layer of code based at least in part on information associated with the processing of the multimedia material in the upper layer; and for at least scoring Program for processing the multimedia material in the lower layer based on the information associated with the processing of the multimedia material in the upper layer. 120028.doc 200803523 code 0 56. The machine readable medium of claim 55, further A Thai-style stone horse for organizing, at least in part, based on the processing associated with the processing of the multimedia material in the upper layer, the descriptive information about the multimedia material in the lower layer. 57. The machine readable medium of claim 56, further comprising at least The blade provides a code for the instruction associated with the processing of the multimedia in the upper layer based on the descriptive information. 58. The machine readable medium of claim 5, wherein the descriptive information comprises metadata. 59. The machine-readable medium of claim 56, wherein the descriptive information comprises frame-specific information, basic or enhanced data identification information, timing information, a coding type, a frame type, synchronization information, and prediction coding correlation. One or more of the information. 60. The machine readable medium of claim 55, further comprising instructions for indicating the underlying code by transmitting information including one or more of processing time, processing action, and processing status. 61. A method for processing multimedia material, comprising: receiving the multimedia material; receiving descriptive information about the multimedia material from a first layer, wherein the descriptive information and the processing of the multimedia material in a second layer Correlation; and processing the multimedia material in the second layer based at least in part on the received descriptive information. The method of claim 61, further comprising: in the second layer of the second layer, receiving at least one instruction 'where the instruction is based at least in part on the descriptive information; and at least in part This is based on the processing of the received command body data. &amp; "The multimedia 63 in a layer, as in the method of claim 62, the receiving/receiving method in the basin/, is associated with an error control method: 64. The method of claim 63, the error control method Included in the method of claim 61, wherein the descriptive information includes metadata. 66. The method of claim 61, wherein the descriptive information includes a map The frame feature &amp; base or enhanced data identification information, timing information, a coding type, a frame type, synchronization information, and predictive coding related information. 67. - A device for processing multimedia data, The method includes: a receiver configured to receive the multimedia material, and a decoder configured to receive descriptive information about the multimedia material from a first layer, wherein the descriptive information The processing of the multimedia material in the second layer is related and configured to process the multimedia material in the second layer based at least in part on the received descriptive information. 68. The decoder is further configured to receive at least one instruction in the second layer, wherein the instruction is based at least in part on the descriptive information, and the decoder is further configured to be based at least in part on the received instruction Change the location of the multimedia material in the second layer to 12028.doc -10- 200803523. 69. If the request method is related to ., , and set the instruction of the receiving and an error control: :::69 The device, wherein the error control method includes one or more of an error complex, a bite, a 'day hidden, and a frame insertion. 72 Rujing: Item: 7 device' wherein the descriptive information includes a meta The device of , , 7 , wherein the descriptive information includes one of the features of the frame feature:: or enhanced data identification information, time series information, a coding type, a type of synchronization, and a prediction code Or a device for processing multimedia materials, comprising: means for receiving the multimedia material; and means for receiving descriptive information about the multimedia material from the first layer Wherein the descriptive information is associated with the processing of the multimedia material in a second layer; and means for processing the multimedia material in the second layer based at least in part on the received descriptive information. The device of item 73, further comprising: means for receiving at least one instruction in the first layer, wherein the instruction is based at least in part on the descriptive information; and for changing the at least in part based on the received instruction a component of the processing of the multimedia material in the second layer. The device of the sprayer 74, wherein the received command is associated with an error control method. 120028.doc • 11-200803523 The device of claim 75, wherein the error control The method includes one or more of the error recovery error and the insertion of a frame. = The device of claim 73, wherein the descriptive information comprises metadata. The device of the staff member 73, wherein the descriptive information includes the frame feature 2 or the enhanced data identification information, the time series information, the frame type of the stone horse class, the synchronization information, and the prediction coding related information _ or more . 79. A machine-readable medium comprising a code for causing the one or more machines to perform program operations when the code executes τ on one or more of the machines, the granular code comprising: for receiving a code of multimedia data; a #王码' for receiving descriptive information about the multimedia material from a first layer, wherein the descriptive information is related to the processing of the multimedia material in a second layer; and The code of the multimedia material is processed in the second layer based at least in part on the received descriptive information. &lt; 80. The machine-readable medium of claim 79, further comprising: a code for receiving at least one instruction in the second layer, wherein the instruction is based at least in part on the descriptive information; The code of the processing of the multimedia material in the second layer is changed based at least in part on the received instruction. 81. The machine readable medium of claim 80, wherein the received instruction is related to an error control method. 82. The machine readable medium of claim 81, wherein the error control method comprises 120028.doc -12-200803523 error recovery, 83. request item 79 data. Error concealment and a frame twist: One or more of the insertions. A machine-readable medium, wherein the 描β-descriptive information comprises a machine readable medium of the ninth aspect, wherein the descriptive information includes a frame feature: information, basic or enhanced material identification information, ~ or more of timing information, a code_type, a frame type, synchronization information, and predictive coding related information. 120028.doc -13 -