TWI416962B - Method and apparatus,and computer readable medium for frame prediction in hybrid video compression to enable temporal scalability - Google Patents

Description

Method, apparatus, and computer readable medium for frame prediction to enable temporal scalability in concurrent video compression [35U.S.C priority request under §119]

The patent application of the present invention has priority over the method and apparatus for the purpose of "frame prediction in the hybrid video compression to achieve temporary scalability" on April 7, 2004 (METHOD AND APPARATUS FOR PREDICTION FRAME IN HYBRID VIDEO COMPRESSION TO ENABLE TEMPORAL SCALABILITY) Provisional Application No. 60/560,433 and November 4, 2004, entitled "Methods and Equipment for Frame Prediction in Hybrid Video Compression for Temporary Scalability" (METHOD) </ RTI> <RTIgt; </ RTI> <RTIgt; </ RTI> <RTIgt; </ RTI> <RTIgt; </ RTI> <RTIgt; </ RTI> <RTIgt; </ RTI> </ RTI> </ RTI> <RTIgt;

The present invention relates to a method, apparatus and system for allocating digital data encoded in a manner that provides temporal scalability.

Due to the explosive growth and great success of the Internet and wireless communications, and the increasing demand for multimedia services, streaming media over the Internet and mobile/wireless channels has received considerable attention. In heterogeneous Internet Protocol (IP) networks, video is provided by a server and can be streamed by one or more clients. Wired connections include: dial-up, integrated services digital network (ISDN), cable, digital subscriber line protocol (collectively referred to as xDSL), fiber optics, local area network (LAN), wide area network (WAN), and other networks. The transmission mode can be unicast or multicast. Various individual client devices, including personal digital assistants (PDAs), laptops, desktops, video converters, TVs, HDTVs, mobile phones, and other devices, need to be used for different bandwidth bits of the same content. flow. This connection bandwidth can change rapidly over time (from 9.6 kbps to 100 Mbps and above) and can be faster than a server response.

Similar to heterogeneous IP network users are mobile/wireless communication. Transporting multimedia content via mobile/wireless channels is extremely challenging, and as such channels are often severely compromised by multipath fading, shadowing, intersymbol interference, and noise interference. Certain other reasons, such as mobile and competitive traffic, can also cause bandwidth changes and losses. Channel noise and the number of users served determine the nature of the time variation of the channel environment. In addition to environmental conditions, the destination network will become a broadband-only data network from the second to third generation cellular networks due to geographic location and mobile roaming. All of these variability affecting the available bandwidth requires adaptive rate adjustment of the multimedia content transmission, even at the time of the job. Therefore, the successful transmission of video over a heterogeneous wired/wireless network requires efficient coding and adaptability to changing network conditions, device characteristics and user preferences, as well as flexibility for loss.

In order to meet the requirements of different users and adapt to channel changes, multiple independent versions of the bit stream can be generated. Each bit stream can satisfy a level limit based on the transmission bandwidth, user display and computing power, but the servo pair Device storage and multicast applications are invalid. A single large bit stream containing one of the high-end users is built into the scalable coding at the server, and only the bit stream for the low-end application is embedded as a subset of the large bit stream. As such, a single bit stream can be adapted to various application environments by selectively transmitting sub-bitstreams. Another advantage provided by scalable coding is for robust video transmission over error prone channels. Error protection or error elimination can be easily implemented. A more stable transmission channel or better error protection can be applied to the base layer bits containing the most efficient information.

In a hybrid encoder like MPEG-1, MPEG-2, MPEG-4 (collectively referred to as MPEG-x), H.261, H.262, H.263, and H.264 (collectively referred to as H.26x) There is spatial, temporal, and signal-to-noise ratio (SNR) scalability. In hybrid coding, temporal redundancy can be removed by motion compensated prediction (MCP). A video is usually divided into a series of picture groups (GOPs), each of which starts with an internal coded frame (I) followed by a forward predictive frame (P) and a bidirectional predictive frame (B). Arrangement. Both the P-frame and the B-frame are intermediate frames. The B frame is the key to the temporary scalability of most MPEG-like encoders. However, some specifications such as MPEG-4 Simple Profile and H.264 Baseline Profile do not support B-frames.

In MPEG-4, specifications and levels provide a means of defining a subset of linguistic and semantics based on the decoder capabilities required to decode a particular bit stream. A specification is a subset of the entire lexical bit stream that is defined. A set of limits defined by a quasi-system that is imposed on parameters within the bit stream. For any given specification, the level typically corresponds to the processing load and memory capabilities of the decoder. As such, the specification and leveling rules limit the bit stream and thus impose limits on the ability to decode the bit stream. In general, a decoder should be considered to conform to one of the established specifications of a given location if it is able to correctly decode all of the allowed values of all of the linguistic elements specified by the specification.

It is an object of the present invention to provide a method and apparatus that provides simple and effective temporary scalability that also conforms to MPEG-4 Simple Profile and H.264 Baseline Profile. The MPEG-4 standard is explained in ISO/IEC 14496-2. The H.264 standard is explained in [ISO/IEC 14496-10].

This document describes a coding scheme and delivery scheme (such as MPEG-x) that provides temporary scalability in a video compression and provides temporary scalability for devices that conform to MPEG-4 Simple Profile and H.264 Baseline Profile. Or H.26x).

In one example, an encoder or transcoder can generate a single bit stream suitable for providing variable data speed and video quality to multiple users. The single bit stream can be formed during operation or stored in memory. For example, to meet bandwidth requirements, meet channel conditions such as ambient noise, or deliver variable quality video, the temporary zoom frame can be omitted from the video stream.

In another example, a decoder may choose to omit decoding of the temporarily scaled frame to, for example, save battery power or decode time.

In several communication systems, the data to be transmitted is compressed to use the available bandwidth more efficiently. For example, the Moving Picture Experts Group (MPEG) has developed several standards for digital data delivery systems. The MPEG-4 standard was developed for low to high data rate channels that typically experience high data loss. A similar standard is H.264 developed by the ITU-T Video Coding Experts Group (VCEG) together with ISO/IEC MPEG.

These MPEG-x and H.26x standards describe data processing and mediation techniques that are preferably suitable for compressing and delivering video, audio and other information using fixed or variable length source coding techniques. In particular, the above standards and other hybrid coding standards and techniques will use intra-frame coding techniques (eg, run length coding, Huffman coding, and the like) and inter-frame coding techniques (such as, for example, , forward and backward predictive coding, motion compensation, and the like) to compress video information. Specifically, in the case of a video processing system, the hybrid video processing system is characterized in that the video frame is subjected to predictive compression coding by means of intra-frame and/or inter-frame motion compensation coding.

This document describes a method and apparatus for encoding a video stream that includes an intra-coded frame, forward and backward prediction frames, and a unidirectionally predicted temporarily scaled frame. During video delivery, temporary scaling may occur at an originating device, at an intermediate device, or at a receiving device.

In-frame coding means that an image (a field or a frame) is encoded without reference to any other image, but the internal coded frame can be used as a reference for other frames. The terms "intra-frame", "intra-coded frame" and "I-frame" are all video objects formed by the internal coding method used throughout this patent application. Example.

Inter-frame or predictive coding refers to encoding an image (a field or a frame) with reference to another image. Compared with the inner coded frame, the intermediate code or prediction frame can be encoded with higher efficiency. An example of an inter-frame that will be used throughout this patent application is a predictive frame (either forward or backward prediction, also known as a "P-frame"), a bi-directional prediction frame (also known as a bi-predictive frame). As a "B frame", and a one-way prediction temporary zoom frame (also known as "P ^* frame"). Other terms used for inter-coding include: high pass encoding, residual encoding, motion compensated interpolation, and other methods known to those of ordinary skill in the art.

In a typical MPEG decoder, the block of the predicted coded block can be decoded (ie, including one or more) with respect to a reference frame (where an intra frame or another prediction frame can be used as a reference frame) Motion vectors and a block of residual error components). 1A is a schematic diagram illustrating a conventional MPEG-4 Simple Profile data stream depicting frame interdependence of a GOP. The GOP 10 is composed of an initial I frame 12 followed by a number of forward prediction P-frames 14. The dependency of the P-frame on a previous I or P frame may limit the provisional scalability to a system that only supports forward prediction frames, such as systems that conform to MPEG-4 Simple and H.264 Baseline Profile. Removing any of the P-frames 14 can result in loss of information that may be critical to decoding other P-frames. The removal of the P frame may result in, for example, video jitter or the decoder being unable to continue decoding before marking the I frame 16 under the start of the next GOP.

One solution to this temporary scalability problem is the bi-directional prediction frame used in the prior art. FIG. 1B is a schematic diagram illustrating a conventional encoded data stream that achieves temporary scalability, depicting frame interdependence of a GOP. The GOP 20 is composed of an I frame 22A, a forward prediction P frame 24, and a bidirectional prediction B frame 26. Each B frame can combine the forward and reverse motion vectors and remaining of the reference I frame 22A or the forward prediction P frame 24 (which can also use the backward prediction P frame, but not shown in this example). error. I frame 22B marks the beginning of the next GOP. As shown in FIG. 1B, only one B frame 26 is included between the I frame 22A and the P frame 24 or between the two P frames 24. Several B frames can be inserted between the reference frames to give greater flexibility to temporary scalability. Since no other frame can rely on the B frame as a reference frame, the B frame 26 can be removed without loss of information about other frame decoding. This feature of B-frame 26 allows B-frame 26 to be inserted into a bit stream, where the encoder, transcoder, or decoder can optionally remove B-frame 26 to accommodate channel conditions, bandwidth limitations, battery power, and Other considerations. For example, if there are three B frames between the reference frames, all three B frames can be removed and the frame rate can be reduced by three quarters, or the B frame in the middle can be retained and the other frame removed. Two frames to reduce the frame rate by half. The data rate is also reduced accordingly.

Although bi-directional prediction provides improved compression for forward (unidirectional) prediction alone, it has a downward trend. Bidirectional prediction requires increased computational requirements. Bidirectional prediction frames can lead to additional coding complexity, since two macroblock blocks (the most computationally intensive coding process) must be performed for each target macroblock, once with the past reference frame. Another time is to use the future reference frame. The introduction of a B frame also increases the computational complexity of the decoder and complicates scheduling. This increase in complexity is one of the main reasons why MPEG-4 Simple Profile and H.264 Baseline Profile do not support bidirectional prediction. These specifications were developed for devices that require efficient use of batteries and processing power, such as mobile phones, PDAs, and the like. The present invention provides an efficient way to provide temporary scalability for such power limited devices.

The present invention relates to a one-way predictive temporary zoom frame to provide temporary scalability without changing any of the MPEG-4 Simple Profile and H.264 Baseline Profile. Unidirectional prediction Temporary scaling frames use only one forward or backward prediction, rather than the two types of predictions used by traditional B frames. In addition, no other prediction frames can refer to the one-way prediction temporary scaling frame. Since no other frames can rely on the temporary zoom frame, the temporary zoom frame can be removed from the bit stream without affecting the remaining frames. As a result, there is no need to introduce any additional corpus into the MPEG-4 Simple Profile or H.264 Baseline Profile. A frame can be identified as a one-way predictive temporary zoom frame that is different from a normal predictive frame by the addition of a single additional bit.

2 is a schematic diagram illustrating an example of a forward prediction temporary scalability scheme in accordance with one aspect of the present invention. The GOP 200 includes an I frame 210A, a P frame 212, and a temporary scalability frame 214. As shown in FIG. 2, a single forward prediction frame can be used to temporarily scale the P ^* frame 214 as a one-way prediction between successive P frames 212. It should be appreciated that a plurality of one-way temporary zoom frames may be dependent on a single reference frame. Having multiple temporary zoom frames between successive P-frames 212 achieves better adaptability to meet data rate requirements. I-frame 210B marks the beginning of the next GOP.

Figure 3 is a schematic diagram illustrating an example of a reverse prediction temporal scalability scheme in accordance with one aspect of the present invention. The GOP 300 includes an I frame 310A, a P frame 312, and a temporary zoom frame 314. As shown in FIG. 3, a single reverse prediction frame can be used to temporarily scale the P ^* frame 314 as a one-way prediction between successive P frames 312. I frame 310B marks the beginning of the next GOP. As seen in both the reverse and forward directions, no other frames refer to the temporary zoom frames 214 and 314, respectively. Since the frameless reference temporarily zooms the frame, the temporary zoom frames can be omitted from the encoding, transmission or decoding without affecting any other frames. This can achieve a gradual reduction in quality and/or data rate based on the number of unidirectionally predicted temporary zoom frames excluded from transmission/decoding.

Since the unidirectional prediction temporarily zooms the frame to require less computation than the B frame, the unidirectional prediction temporary scaling frame can be advantageously used in a power limited or computationally constrained device. Since the unidirectional prediction temporarily zooms the frame to predict the subsequent P frame, the coding efficiency of the P frame is reduced compared to using only the P frame. This reduction in coding efficiency can be tolerated by the added benefit of having temporary scalability. The example of the unidirectional prediction temporary scaling frame provided in Figures 2 and 3 refers to only one frame. However, it should be understood that a one-way prediction temporary zoom frame can refer to more than one frame. Involving more than one previous or subsequent frame will increase the complexity of the operation, but it can also reduce the size of the remaining errors.

In addition to the operational benefits, a short delay can be achieved when using a forward-predictive one-way temporary zoom frame instead of a two-way frame. The two-way frames are encoded after the frame of the two-way frame is predicted backward. This can mean that there is an extra delay before the B frame can be displayed. 4 is an illustration of an example of a frame order for displaying and encoding a forward-predicted one-way temporary zoom frame using the present invention. As shown in FIG. 4, unlike the bi-predictive frame, the one-way predictive temporary zoom frame of the present invention can be encoded and transmitted in the same order as would be displayed at the remote device. The ability to sequentially encode and transmit forward-predicted one-way temporary scaling frames avoids the additional delays encountered when using B-frames, which can be an added benefit for applications such as video conferencing.

Figure 5 is a block diagram of a general communication system for encoding and decoding streaming images. System 500 includes an encoder device 505 and a decoder device 510. Encoder device 505 further includes an inner coding component 515, a predictive coding component 520, a temporary scaling component 525, and a memory component 530. Encoder device 505 is capable of accessing data from external source 535. For example, external source 535 can be external memory, the Internet, or a live video and/or voice feed. The material contained within external source 535 can be in an original (uncoded) or encoded state. The inner coding component 515 can be used to encode an inner coded frame. Predictive coding component 520 can be used to encode all types of predictive frames, including unidirectionally predictively temporarily scaled frames. In addition to the logic for encoding the prediction frame, the predictive coding component 520 also includes logic for selecting the reference frame and logic for excluding the temporary zoom frame from being referenced by other frames. Predictive coding component 520 can access the original or encoded data for encoding. The encoded data can be accessed to replace the normal P-frame or the I-frame with a one-way predictive temporary zoom frame. When the encoded data (or internal coded or intermediate coded material) is accessed, the logic contained within internal coding component 515 and predictive coding component 520 decodes the encoded data to produce reconstructed original data. The reconstructed original data is then encoded into a one-way predictive temporary zoom frame (or any other type of frame).

After encoding, the encoded frame is stored in memory element 530 or external memory. The external memory can be the same as external source 535 or be a separate memory component (not shown). The encoded frames are transmitted (Tx) by the network 540. Network 540 can be in wired or wireless form. Temporary scaling component 525 contains logic to determine if temporary scaling is desired prior to transmission. Temporary scaling component 525 can also include logic to identify the temporarily scaled frame and omit it from transmission if it is determined that temporary scaling is desired. The encoding process implemented by the encoder device will be explained more fully below.

The decoder device 510 includes elements similar to the encoder device 505, including: an internal decoding component 545, a predictive decoding component 550, a temporary scaling component 555, and a memory component 560. The decoder device 510 can receive encoded data that has been transmitted over the network 540 or receive encoded data from the external storage 565. Internal decoding component 545 can be used to decode the internally encoded material. The predictive decoding component 550 can be used to decode the predictive data, including unidirectionally predicting the temporarily scaled frame. Temporary scaling component 555 contains logic to determine if temporary scaling is desired prior to decoding. In this example, the temporary scaling component 555 also includes logic to identify the temporarily scaled frame and to omit it from being decoded if it is determined that temporary scaling is desired. After decoding, the decoded frames can be displayed on display element 570 or stored in internal memory 560 or external storage 565. Display component 570 can be an integrated portion of the decoding device, such as a display on a telephone or PDA. Display element 570 can also be an external peripheral device. The decoding process implemented by the decoder device will be explained more fully below.

The modifications implemented to provide a decoder device to support unidirectional prediction of temporarily scaling frames are minor. Since H.264 can support multiple reference coding, if the basic decoder can support at least two reference frames, it may not be necessary to modify the decoder to support one-way prediction of temporarily scaling the frame. The decoder conforming to the MPEG-4 Simple Profile only allows a reference frame to be stored in the buffer. Therefore, after decoding a one-way forward prediction temporary zoom frame, the reference frame buffer can be maintained for the subsequent P frame. The reference frame, instead of replacing the reference frame in the buffer with the temporarily decoded frame that was just decoded.

In addition to the encoding and decoding devices, temporary scaling can occur at an intermediate device called a transcoder. Referring to Figure 6, a block diagram of a transcoder device is shown. Transcoder device 600 is located between first network 605 and second network 620. Transcoder device 600 receives the encoded material from a device (such as encoder device 505 depicted in FIG. 5) via first network 605. Transcoder device 600 stores the received data in a memory component 615. The transcoder device also includes a temporary scaling element 610. Temporary scaling component 610 includes logic to determine if temporary scaling is desired prior to transmission on second network 620. Temporary scaling component 610 can also include logic to identify the temporarily scaled frame and omit it from transmission if it is determined that temporary zooming is desired. The transcoding process implemented by transcoder device 600 will be more fully explained below.

Figure 7 is a flow chart illustrating an example of an encoding process including temporary scaling in accordance with one aspect of the present invention. The encoding process takes place in an encoder such as the device 505 depicted in FIG. The digital video data is encoded into GOPs composed of a plurality of frames. A GOP begins with encoding an internal coded frame at 720. The intra-coded frame is used as at least some subsequent intermediate frames (or a previous intermediate frame in the case of a reverse prediction with an open GOP, wherein an open GOP can refer to a frame from another GOP) A reference point. Encoding process 700 also includes encoding a prediction frame (730) that can include a forward or backward prediction frame. The prediction frames may include motion compensation data such as motion vectors and residual errors that may refer to a previous internal coding or prediction frame. The predictive frame can also be used as a reference frame for both predictive frames (both normal and temporarily scaled frames). Encoding one-way prediction temporarily scaling the frame can achieve temporary scalability (740). The frames (730) can be computed in a manner similar to the predictive frames, as the frames can include motion compensation with reference to an internal coded or predictive frame. However, the temporary zoom frames themselves cannot be referenced by another frame (ie, the temporary zoom frame cannot be used to predict any other frame). The temporary zoom frame data may also include additional information that identifies the frame as a temporary zoom frame. Since the other frames are not dependent on the presence of the temporarily zoomed frame, the temporary zoom frame can be removed without adversely affecting other frames. The encoded frame can be stored in memory for subsequent delivery (750). The encoded frame can also be dispatched after encoding without the need to store step 750.

The encoding process 700 can continue to encode the GOP until the video material 710 is exhausted. To meet different goals, the GOP can be composed of different numbers of frames of different frame types. Encoding a large number of temporary scaling frames into a GOP will provide greater flexibility in adjusting delivery quality or complexity or decoding a GOP.

Figure 8 is a flow diagram of an example of a video delivery process including temporary zooming in accordance with the present invention. The left side of Figure 8 corresponds to a process in a video source, such as the encoder device 505 depicted in Figure 5, while the right side corresponds to a process in a destination device, such as the decoder device 510 depicted in Figure 5. . A wired/wireless network can be connected to the two sides and can be a combination of wired or wireless networks. The transfer to the new network may include a transcoder device, such as the transcoder device 600 depicted in FIG. Process 800 in FIG. 8 begins by capturing video frame data from memory 810. The memory may be a previously formed permanent memory or it may be a dynamic memory to maintain the frame data computed during transmission.

A decision is made as to whether to temporarily scale the video material (820). The factors considered in the decision may be, for example, providing a lower quality level, lowering the data rate to less than one of the networks, controlling the traffic, and maintaining a source. Or the battery power of a destination device or the time to limit encoding and/or decoding. If a temporary zoom is to be performed, the temporary zoom frame is identified and selectively removed from the data stream (830). Since there is no frame to refer to the temporary zoom frame, the removal of any one-way prediction temporary zoom frame will not affect any other frame. The identification can take a variety of forms, including, for example, a single additional item or flag, and when set equal to one, the additional item or flag identifies the frame as a temporary zoom frame. This additional item or flag can be encoded using a standard compliant law or in a patented manner. If the bit stream conforms to the standard (or specification), it can be communicated by a mutual a priori encoder-server (in the case of network adaptation) or a mutual a priori encoder-decoder identifier. The temporary zoom frames are identified (in the case of device complexity/power adaptation). The mutual a priori identifiers may be, for example, a frame position (e.g., an odd or even frame number), a decoded or visual time stamp, or a frame order. Another form of identification involves the decoder using information in the bitstream as to whether a frame is referenced by another frame. The unremoved video frame can be transmitted to the destination device (840) by the (and the like) wired and/or wireless network. In the case of multicast delivery, there may be multiple destination devices or, in the case of unicast delivery, there may be a single destination device.

At the destination device (such as a decoder such as decoder device 510 of Figure 5), or at an intermediate network device (such as a router or transcoder of device 600 of Figure 6), Coded video material (850). After the data is retrieved, the destination device or intermediate network device can determine whether to provide temporary scaling (860), respectively. The reason for the temporary zoom can be similar to the reason for the video source, especially for an intermediate network router, regarding network capabilities or network load. Reasons for temporary zooming may also include, for example, battery power conservation, particularly for resource-constrained devices such as PDA's, mobile phones, and the like. If temporary scaling is selected, the temporary zoom frame is identified and omitted to satisfy a target parameter, such as a data rate or a decoding time. After omitting the temporary scaling frame, the remaining frames (e.g., inner code decoding, forward predictive decoding, etc.) are decoded in a manner determined by their type (880).

The above may be implemented at an encoder such as encoder device 505 (Fig. 5), a transcoder such as transcoder device 600 (Fig. 6), or a decoder such as decoder device 510 (Fig. 5). The process of making a temporary zoom decision and removal is discussed. One or more of the three devices may participate in a decision to remove the temporary zoom frame within the same bit stream.

Although the various methods illustrated in FIGS. 7-8 are shown and described as a series of acts for the purpose of simplifying the explanation, it should be understood and appreciated that the invention is not limited to the order of the acts, as In the present invention, certain actions may occur in different orders and/or concurrently with other acts as illustrated and described herein.

Although the present invention has been fully described in conjunction with the use of internal frames and forward predictive frames as reference frames for such unidirectionally predicted temporally scaled frames, it should be understood that other frames such as reverse predictive frames may also be used. As a reference frame.

Although the invention has been fully described in connection with MPEG-x and H.26x type compression schemes, it should be appreciated that other video compression schemes may also implement the methods of the present invention.

Various aspects of the invention include, but are not limited to, the following description.

A method for encoding a multimedia frame, comprising encoding a removable temporary zoom frame by unidirectionally predicting a removable temporary zoom frame, wherein the removable temporary zoom frame cannot be used to predict any Other frames.

An apparatus for encoding a multimedia frame, comprising: means for encoding a removable temporary zoom frame by unidirectionally predicting a removable temporary zoom frame, wherein the removable temporary zoom frame Cannot be used to predict any other frame.

An electronic device for encoding a multimedia frame, the electronic device configured to encode a removable temporary zoom frame by unidirectionally predicting a removable temporary zoom frame, wherein the removable temporary frame The zoom frame cannot be used to predict any other frame.

A computer readable medium having instructions for causing a computer to perform a method of encoding a multimedia frame, comprising: encoding an internal coded frame that is not predicted by another frame; encoding a predictive frame, wherein At least one internal coding or prediction frame to predict the prediction frame; and encoding the removable temporary zoom frame by unidirectionally predicting a removable temporary zoom frame, wherein the removable temporary zoom frame Frames cannot be used to predict any other frame.

A method for decoding a multimedia frame, comprising: receiving encoded frame data; identifying any unidirectionally predictable removable temporary zoom frame, wherein the removable temporary zoom frame cannot be used for prediction a further frame; and decoding the received encoded frame to omit at least one removable temporary zoom frame without decoding.

An apparatus for decoding a multimedia frame, comprising: means for receiving encoded frame data; means for identifying any removable temporary zoom frame that is unidirectionally predicted, wherein the movable The divide-type temporary zoom frame cannot be used to predict any other frame; and the means for decoding the received encoded frame to omit at least one removable temporary zoom frame without decoding.

An electronic device for decoding a multimedia frame, the electronic device configured to: receive encoded frame data; identify any removable temporary zoom frame that is unidirectionally predicted, wherein the removable The temporary zoom frame cannot be used to predict any other frame; and the received encoded frame data is decoded to omit at least one removable temporary zoom frame without decoding.

A computer readable medium having instructions for causing a computer to perform a method of decoding a media frame, comprising: receiving encoded frame data; identifying any removable temporary zoom that is unidirectionally predicted a frame, wherein the removable temporary zoom frame cannot be used to predict any other frame; and the received encoded frame data is decoded to omit at least one removable temporary zoom frame from being decoded.

A method for temporarily scaling a multimedia frame, comprising: receiving an encoded frame by a first network; receiving, by the first network, a removable temporary zoom frame, wherein the at least one encoded The frame is used to predict the removable temporary zoom frame in one direction and the removable temporary zoom frame cannot be used to predict any other frame; the received encoded signal is transmitted by a second network transmission The box; and the removable temporary zoom frame is omitted from the transmission.

An apparatus for temporarily scaling a multimedia frame, comprising: means for receiving an encoded frame by a first network; for receiving a removable temporary zoom by the first network a component of the frame, wherein the removable temporary zoom frame is unidirectionally predicted according to the at least one encoded frame and the removable temporary zoom frame cannot be used to predict any other frame; The second network transmits the component of the encoded frame accessed; and the means for omitting the removable temporary zoom frame from the transmission.

Those of ordinary skill in the art will appreciate that information and signals can be represented using any of a variety of different technologies and techniques. For example, the materials, instructions, commands, information, signals, bits, symbols, and chips that may be referred to throughout the text may be voltage, current, electromagnetic waves, magnetic fields or particles, light fields or particles, or any of them. Combined to represent.

It will be further appreciated by those skilled in the art that the various illustrative logical blocks, modules, and algorithm steps described in connection with the examples disclosed herein can be constructed as electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. The ability to build this functionality as a hardware or soft system depends on the specific application and design constraints imposed on the overall system. Those skilled in the art can construct the functionality in a different manner for each particular application, but such implementation decisions should not be considered as a departure from the scope of the invention.

Various illustrative logic blocks, modules, and circuits set forth in connection with the examples disclosed herein may use a general purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), or Other programmable logic devices, discrete gate or transistor logic, discrete hardware components, or any combination of functions designed to perform the functions described herein are constructed or executed. A general purpose processor may be a microprocessor, but in the alternative, the processor may be any conventional processor, controller, microcontroller or state machine. The processor can also be constructed as a combination of computing devices, such as a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration.

The steps of the method or algorithm described in connection with the examples disclosed herein may be directly implemented in a hardware, in a software module executed by a processor, or in a combination of both. The software module can reside in random access memory (RAM), flash memory, read-only memory (ROM), erasable programmable read-only memory (EPROM), and electronic erasable programmable programming. Read only memory (EEPROM), scratchpad, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor such that the processor can read information from or write information to the storage medium. Alternatively, the storage medium can be integrated into the processor. The processor and the storage medium can reside in an application specific integrated circuit (ASIC). The ASIC can reside in the modem. Alternatively, the processor and the storage medium can reside as discrete components in the modem.

The above description of the disclosed examples is intended to enable any person skilled in the art to make or utilize the invention. A variety of modifications to the examples will be apparent to those skilled in the art, and the general principles defined herein may be applied to other examples without departing from the spirit or scope of the invention.

This paper describes a method, apparatus, and system for encoding, transcoding, and decoding a video stream that includes an inner coded frame, forward and backward predictive frames, and a unidirectionally predicted temporarily scaled frame.

10. . . Image group (GOP)

12. . . Initial I frame

14. . . Forward prediction P frame

16. . . Next I frame

22A. . . I frame

22B. . . I frame

twenty four. . . Forward prediction P frame

26. . . Bidirectional prediction B frame

200. . . Image group (GOP)

210A. . . I frame

210B. . . I frame

212. . . P frame

214. . . Temporarily zoom the frame

300. . . Image group (GOP)

310A. . . I frame

310B. . . I frame

312. . . P frame

314. . . Temporarily zoom the frame

500. . . system

505. . . Encoder device

510. . . Decoder device

515. . . Internal coding component

520. . . Predictive coding component

525. . . Temporary scaling component

530. . . Memory component

535. . . External source

540. . . network

545. . . Internal decoding component

550. . . Predictive decoding component

555. . . Temporary scaling component

560. . . Memory component (internal memory)

565. . . External storage

600. . . Transcoder device

605. . . First network

610. . . Temporary scaling component

615. . . Memory component

620. . . Second network

1A is a schematic diagram illustrating a conventional MPEG-4 Simple Profile data stream, and FIG. 1B is a schematic diagram illustrating a conventional encoded data stream that can achieve temporary scalability. FIG. 2 is a schematic diagram illustrating the present invention according to the present invention. A schematic diagram of an example of a forward prediction temporary scalability scheme, FIG. 3 is a schematic diagram illustrating an example of a reverse prediction temporary scalability scheme according to the present invention, and FIG. 4 is a schematic diagram for direct prediction using the present invention. FIG. 5 is a block diagram of a general communication system for encoding and decoding a streamed image, and FIG. 6 is a block diagram of a conversion decoder device. 7 is a flow chart illustrating an example of an encoding process including temporary scaling in accordance with the present invention, and FIG. 8 is a

10. . . Image group (GOP)

12. . . Initial I frame

14. . . Forward prediction P frame

16. . . Next I frame

22A. . . I frame

22B. . . I frame

twenty four. . . Forward prediction P frame

26. . . Bidirectional prediction B frame

Claims (33)

A method of encoding a multimedia frame in a single layer of bitstreams in an encoder, comprising: predicting, by unidirectionally, all of the shifts in the single layer of bitstreams in a display order Dividing the multimedia frame temporarily to encode the removable temporary zoomed multimedia frames in the single layer bit stream; by means of a decoder, the additional item data for identification of a multimedia frame is The removable temporarily zoomed multimedia frame is encoded as a removable temporarily zoomed multimedia frame, wherein the removable temporary zoomed multimedia frame is used to temporarily scale a data rate of the single layer bit stream Encoded to be removable by the decoder based on the additional item data; and to encode all of the multimedia frames including the removable temporarily scaled multimedia frames to the single layer bit stream The multimedia frame is temporarily scaled without any of the removable types to predict the multimedia frames, wherein the method is performed by one or more processors of the encoder. According to the method of claim 1, the method further includes: encoding at least one of the other multimedia frames as an internal coded frame, the internal coded frame not being predicted according to another frame. The method of claim 2, further comprising: encoding at least one of the other multimedia frames as a prediction frame, wherein the prediction frame is predicted according to the at least one internal coding or prediction frame. According to the method of claim 3, wherein the encoding of the prediction frame includes a forward direction Predict the prediction frame. The method of claim 1, further comprising: storing the encoded frames in a memory. The method of claim 3, further comprising: transmitting the encoded frames via a network. The method of claim 3, further comprising: transmitting the encoded intra-coded frame and the encoded prediction frame via a network, and omitting the encoded removable temporary zoom from the transmission Multimedia frame. The method of claim 3, further comprising: encoding the prediction frame by means of a dynamic vector and residual error data; and encoding the removable temporary zoomed multimedia frames by means of a dynamic vector and residual error data. The method of claim 6, further comprising: receiving the transmitted frames; and decoding the received frames. The method of claim 6, further comprising: receiving the transmitted frames; decoding the received internal coded frame and the received prediction frame, and omitting the received removable temporary zoomed multimedia Frame. The method of claim 6, further comprising: receiving the received frames; and identifying each of the received removable temporarily zoomed multimedia frames by means of a prior identifier. An electronic device for encoding a multimedia frame in a single layer of bitstreams, the electronic device being configured to: by unidirectionally backward predicting all of the single layer bitstreams in a display order Removably temporarily zooming the multimedia frame to encode the removable temporary zoomed multimedia frames in the single layer bitstream, with a decoder having additional data for identification of a multimedia frame The removable temporary zoomed multimedia frame code is a removable temporarily zoomed multimedia frame, wherein the removable temporary zoomed multimedia message is used to temporarily scale a data rate of the single layer bit stream The frame is encoded to be removable by the decoder based on the additional item data, and to encode all of the multimedia frames including the removable temporarily zoomed multimedia frames, without using any such The removable multimedia frame is temporarily zoomed to predict the multimedia frames. According to the electronic device of claim 12, it is further configured to encode at least one of the other multimedia frames as an internal coded frame, the internal coded frame not being predicted according to another frame. The electronic device of claim 13, further configured to encode at least one of the other multimedia frames as a prediction frame, wherein the prediction frame is predicted based on the at least one internal coding or prediction frame. According to the electronic device of claim 14, it is further configured to encode the prediction frame by using forward prediction. According to the electronic device of claim 12, it is further configured to store the encoded frames in a memory. According to the electronic device of claim 14, it is further configured to transmit the encoded frames via a network. According to the electronic device of claim 14, further configured to transmit the encoded intra-coded frame and the encoded predictive frame via a network, and the encoded removable form is omitted from the transmission Temporarily zoom the multimedia frame. According to the electronic device of claim 14, it is further configured to encode the predictive frame by means of a motion vector and residual error data, and to encode the removable temporary zoomed multimedia frames by means of a dynamic vector and residual error data. A non-transitory computer readable medium having instructions for causing a computer to perform a method of encoding a multimedia frame in accordance with the method of claim 1. A method of decoding a multimedia frame in a decoder, comprising: receiving one or more removable temporarily scaled multimedia frames encoded in a single layer of bitstreams and at the single layer Frame data of other multimedia frames in the meta stream, wherein all of the removable temporary zoomed multimedia frames in the single layer bit stream for a display order are unidirectionally reverse predicted, wherein the The removable temporary zooming multimedia frame is encoded as a removable temporary zooming multimedia frame by using the additional item data for identifying the multimedia frame, including the removable temporary zooming multimedia frame The multimedia frames are encoded without using any of the removable temporary zoomed multimedia frames to predict the multimedia frames, and wherein to temporarily scale a data rate of the single layer bit stream, such The removable temporary zoom multimedia frame is encoded to be removable; Identifying at least one of the unidirectionally predicted removable temporarily zoomed multimedia frames based on the additional item data; and decoding the received encoded frame data to omit the removable temporary zoomed multimedia At least one of the frames is not decoded, wherein the method is performed by one or more processors of the decoder. According to the method of claim 21, the method further includes: receiving, by the encoded frame data, at least one internal code frame, the at least one internal code frame is not predicted according to another frame; and decoding the internal code frame. The method of claim 22, further comprising: receiving the at least one prediction frame with the encoded frame material, wherein the prediction frame is predicted according to the at least one encoded frame; and decoding the prediction frame. The method of claim 21, wherein the receiving step comprises receiving via a wireless network. According to the method of claim 23, further comprising: receiving one of the forward predicted prediction frames. According to the method of claim 21, further comprising: identifying, by means of a prior identifier, each of the received removable temporarily zoomed multimedia frames. An electronic device for decoding a multimedia frame, the electronic device configured to: receive encoded frame data comprising one or more removable temporarily scaled multimedia frames in a single layer bit stream, One-way reverse Detecting all of the removable temporary zoom frames for a display order and other multimedia frames in the single layer bit stream, wherein the removable temporarily zoomed multimedia frames are a multimedia frame The additional item data for identifying is encoded as a removable temporarily zoomed multimedia frame, wherein the multimedia frames including the removable temporary zoomed multimedia frames are encoded without using any of the Removably temporarily zooming the multimedia frame to predict the multimedia frames, and wherein in order to temporarily scale a data rate of the single layer bit stream, the removable temporary zoomed multimedia frames are encoded to Removable; identifying at least one of the unidirectionally predictable removable temporary zoomed multimedia frames based on the additional item data, and decoding the received encoded frame data to omit the removable Temporarily scaling at least one of the multimedia frames without decoding. The electronic device of claim 27, further configured to receive the at least one internal code frame by the encoded frame data, the at least one internal code frame not predicting according to another frame, and decoding the internal code frame. The electronic device of claim 28, further configured to receive the at least one prediction frame with the encoded frame material, wherein the predicted frame is predicted according to the at least one encoded frame; and the prediction frame is decoded. According to the electronic device of claim 27, it is further configured to receive the encoded frame material via a wireless network. According to the electronic device of claim 29, it is further configured to receive one of the forward predicted prediction frames. According to the electronic device of claim 29, it is further configured with a prior knowledge A unique identifier identifies each of the received removable temporary zoom frames. A computer readable medium having instructions for causing a computer to perform a method of decoding a multimedia frame in accordance with a request item 21.