US6233356B1 - Generalized scalability for video coder based on video objects - Google Patents

Generalized scalability for video coder based on video objects Download PDF

Info

Publication number
US6233356B1
US6233356B1 US09111215 US11121598A US6233356B1 US 6233356 B1 US6233356 B1 US 6233356B1 US 09111215 US09111215 US 09111215 US 11121598 A US11121598 A US 11121598A US 6233356 B1 US6233356 B1 US 6233356B1
Authority
US
Grant status
Grant
Patent type
Prior art keywords
video
layer
object
vop
data
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
US09111215
Inventor
Barin Geoffry Haskell
Atul Puri
Robert Lewis Schmidt
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
AT&T Intellectual Property II LP
Original Assignee
AT&T Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Grant date

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/593Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving spatial prediction techniques
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/20Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using video object coding
    • H04N19/23Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using video object coding with coding of regions that are present throughout a whole video segment, e.g. sprites, background or mosaic
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/20Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using video object coding
    • H04N19/29Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using video object coding involving scalability at the object level, e.g. video object layer [VOL]
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/30Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using hierarchical techniques, e.g. scalability
    • H04N19/31Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using hierarchical techniques, e.g. scalability in the temporal domain
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/30Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using hierarchical techniques, e.g. scalability
    • H04N19/33Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using hierarchical techniques, e.g. scalability in the spatial domain
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/44Decoders specially adapted therefor, e.g. video decoders which are asymmetric with respect to the encoder
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television, VOD [Video On Demand]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234Processing of video elementary streams, e.g. splicing of content streams, manipulating MPEG-4 scene graphs
    • H04N21/23412Processing of video elementary streams, e.g. splicing of content streams, manipulating MPEG-4 scene graphs for generating or manipulating the scene composition of objects, e.g. MPEG-4 objects
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television, VOD [Video On Demand]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234Processing of video elementary streams, e.g. splicing of content streams, manipulating MPEG-4 scene graphs
    • H04N21/2343Processing of video elementary streams, e.g. splicing of content streams, manipulating MPEG-4 scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
    • H04N21/234327Processing of video elementary streams, e.g. splicing of content streams, manipulating MPEG-4 scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements by decomposing into layers, e.g. base layer and one or more enhancement layers
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television, VOD [Video On Demand]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234Processing of video elementary streams, e.g. splicing of content streams, manipulating MPEG-4 scene graphs
    • H04N21/2343Processing of video elementary streams, e.g. splicing of content streams, manipulating MPEG-4 scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
    • H04N21/234363Processing of video elementary streams, e.g. splicing of content streams, manipulating MPEG-4 scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements by altering the spatial resolution, e.g. for clients with a lower screen resolution
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television, VOD [Video On Demand]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234Processing of video elementary streams, e.g. splicing of content streams, manipulating MPEG-4 scene graphs
    • H04N21/2343Processing of video elementary streams, e.g. splicing of content streams, manipulating MPEG-4 scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
    • H04N21/234381Processing of video elementary streams, e.g. splicing of content streams, manipulating MPEG-4 scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements by altering the temporal resolution, e.g. decreasing the frame rate by frame skipping
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television, VOD [Video On Demand]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network, synchronizing decoder's clock; Client middleware
    • H04N21/44Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs
    • H04N21/44012Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs involving rendering scenes according to scene graphs, e.g. MPEG-4 scene graphs
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television, VOD [Video On Demand]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network, synchronizing decoder's clock; Client middleware
    • H04N21/44Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs
    • H04N21/4402Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
    • H04N21/440227Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display by decomposing into layers, e.g. base layer and one or more enhancement layers
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television, VOD [Video On Demand]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network, synchronizing decoder's clock; Client middleware
    • H04N21/44Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs
    • H04N21/4402Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
    • H04N21/440263Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display by altering the spatial resolution, e.g. for displaying on a connected PDA
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television, VOD [Video On Demand]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network, synchronizing decoder's clock; Client middleware
    • H04N21/44Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs
    • H04N21/4402Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
    • H04N21/440281Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display by altering the temporal resolution, e.g. by frame skipping
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television, VOD [Video On Demand]
    • H04N21/60Selective content distribution, e.g. interactive television, VOD [Video On Demand] using Network structure or processes specifically adapted for video distribution between server and client or between remote clients; Control signaling specific to video distribution between clients, server and network components, e.g. to video encoder or decoder; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client
    • H04N21/63Control signaling related to video distribution between client, server and network components; Network processes for video distribution between server and clients or between remote clients, e.g. transmitting basic layer and enhancement layers over different transmission paths, setting up a peer-to-peer communication via Internet between remote STB's; Communication protocols; Addressing
    • H04N21/631Multimode Transmission, e.g. transmitting basic layers and enhancement layers of the content over different transmission paths or transmitting with different error corrections, different keys or with different transmission protocols
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00Details of television systems
    • H04N5/44Receiver circuitry
    • H04N5/4401Receiver circuitry for the reception of a digital modulated video signal

Abstract

A video coding system that codes video objects as scalable video object layers. Data of each video object may be segregated into one or more layers. A base layer contains sufficient information to decode a basic representation of the video object. Enhancement layers contain supplementary data regarding the video object that, if decoded, enhance the basic representation obtained from the base layer. The present invention thus provides a coding scheme suitable for use with decoders of varying processing power. A simple decoder may decode only the base layer of video objects to obtain the basic representation. However, more powerful decoders may decode the base layer data of video objects and additional enhancement layer data to obtain improved decoded output. The coding scheme supports enhancement of both the spatial resolution and the temporal resolution of video objects.

Description

RELATED APPLICATION

The present invention benefits from priority of U.S. patent application Ser. No. 60/069,888, filed Jul. 8, 1997, the disclosure of which is incorporated herein by reference. The invention also relates to the invention of U.S. patent application Ser. No. 08/827,142, filed Mar. 21, 1997, the disclosure of which is incorporated herein by reference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a video coding system in which image data is organized into video objects and coded according to a scalable coding scheme. The coding scheme provides spatial scalability, temporal scalability or both.

2. Related Art

Video coding is a field that currently exhibits dynamic change. Video coding generally relates to any method that represents natural and/or synthetic visual information in an efficient manner. A variety of video coding standards currently are established and a number of other coding standards are being drafted. The present invention relates to an invention originally proposed for use in the Motion Pictures Experts Group standard MPEG-4.

One earlier video standard, known as “MPEG-2,” codes video information as video pictures or “frames.” Consider a sequence of video information to be coded, the sequence represented by a series of frames. The MPEG-2 standard coded each frame according to one of three coding methods. A given image could be coded according to:

Intra-coding where the frame was coded without reference to any other frame (known as “I-pictures”),

Predictive-coding where the frame was coded with reference to one previously coded frame (known as “P-pictures”), or

Bi-directionally predictive coding where the frame was coded with reference to as many as two previously coded frames (known as “B-pictures”).

Frames are not necessarily coded in the order in which they appear under MPEG-2. It is possible to code a first frame as an I-picture then code a fourth frame as a P-picture predicted from the I-picture. Second and third frames may be coded as B-pictures, each predicted with reference to the I- and P-pictures previously coded. A time index is provided to permit a decoder to reassemble the correct frame sequence when it decodes coded data.

MPEG-4, currently being drafted, integrated the concept of “video objects” to I-, P- and B-coding. Video object based coders decompose a video sequence into video objects. An example is provided in FIGS. 1 (a)-(d). There, a frame includes image data including the head and shoulders of a narrator, a suspended logo and a background. An encoder may determine that the narrator, logo and background are three distinct video objects, each shown separately in FIGS. 1 (b)-(d). The video coder may code each separately.

Video object-based coding schemes recognize that video objects may remain in a video sequence across many frames. The appearance of a video object on any given frame is a “video object plane” or “VOP”. VOPs may be coded as I-VOPs using intra coding techniques, as P-VOPs using predictive coding techniques or B-VOPs using bi-directionally predictive coding techniques. For each VOP, additional administrative data is transmitted with the coded VOP data that provides information regarding, for example, the video objects location in the displayed image.

Coding video information on a video object-basis may improve coding efficiency in certain applications. For example, if the logo were a static image, an encoder may code it as an initial I-VOP. However, for subsequent frames, coding the logo as a P- or B-VOP would yield almost no image data. The P- or B-coding essentially amounts to an “instruction” that the original image information should be redisplayed for successive frames. Such coding provides improved coding efficiency.

One goal of the MPEG-4 standard is to provide a coding scheme that may be used with decoders of various processing power. Simple decoders should be able to decode coded video data for display. More powerful decoders should be able to decode the coded video data and obtain superior output such as improved image quality or attached functionalities. As of the priority date of this application, no known video object-based coding scheme provides such flexibility.

MPEG-2 provides scalability for its video picture-based coder. However, the scalability protocol defined by MPEG-2 is tremendously complicated. Coding of spatial scalability, where additional data for VOPs is coded into an optional enhancement layer, is coded using a first protocol. Coding of temporal scalability, where data of additional VOPs is coded in the enhancement layer, is coded using a second protocol. Each protocol is separately defined from the other and requires highly context specific analysis and complicated lookup tables in a decoder. The scalability protocol of the MPEG-2 is disadvantageous because its complexity makes it difficult to implement. Accordingly, there is a further need in the art for a generalized scalability protocol.

SUMMARY OF THE INVENTION

The present invention provides a video coding system that codes video objects as video object layers. Data of each video object may be segregated into one or more layers. A base layer contains sufficient information to decode a basic representation of the video object. Enhancement layers contain supplementary data regarding the video object that, if decoded, enhance the basic representation obtained from the base layer. The present invention thus provides a coding scheme suitable for use with decoders of varying processing power. A simple decoder may decode only the base layer to obtain the basic representation. However, more powerful decoders may decode the base layer data and additional enhancement layer data to obtain improved decoded output.

BRIEF DESCRIPTION OF THE DRAWINGS

FIGS. 1 (a)-(d) provide an example of video data and video objects that may be derived therefrom.

FIG. 2 is an organizational chart illustrating a video coding hierarchy established by the present invention.

FIG. 3 illustrates an object based video coder constructed in accordance with an embodiment of the present invention.

FIG. 4 is a block diagram of a video object encoder constructed in accordance with an embodiment of the present invention.

FIG. 5 illustrates an application of temporal scalability provided by the present invention.

FIG. 6 illustrates an application of spatial scalability provided by the present invention.

FIG. 7 is a block diagram of a video object decoder constructed in accordance with an embodiment of the present invention.

FIG. 8 is a block diagram of a scalability preprocessor constructed in accordance with an embodiment of the present invention.

FIG. 9 is a block diagram of an enhancement layer encoder constructed in accordance with an embodiment of the present invention.

FIG. 10 is a block diagram of a midprocessor constructed in accordance with an embodiment of the present invention.

FIG. 11 is a block diagram of an enhancement layer decoder constructed in accordance with an embodiment of the present invention.

FIG. 12 is a block diagram of a scalability post-processor constructed in accordance with an embodiment of the present invention.

DETAILED DESCRIPTION

The present invention introduces a concept of “video object layers” to the video object-based coding scheme. Data of each video object may be assigned to one or more layers of the video object and coded. A base layer contains sufficient information to represent the video object at a first level of image quality. Enhancement layers contain supplementary data regarding the video object that, if decoded, improve the image quality of the base layer. The present invention thus provides an object based a coding scheme suitable for use with decoders of varying processing power. A simple decoder may decode only the base layer of objects to obtain the basic representation. More powerful decoders may decode the base layer data and additional enhancement layer data of objects to obtain improved decoded output.

FIG. 2 illustrates an organizational scheme established by the present invention. An image sequence to be coded is a video session. The video session may be populated by a number of video objects. Each video object may be populated by one or more video object layers. A video object layer is an organizational artifact that represents which part of the coded bitstream output by the video coder carries certain image information related to the video object. For example, base layer data may be assigned to a first video object layer (layers VOL1 for each video object VOØ, VO1 and VO2 in FIG. 2). Enhancement layer data may be assigned to a second video object layer, such as VOL2 in each of VO1 and VO2. The video object layers are themselves populated by video object planes.

Enhancement layers need not be provided for every video object. For example, FIG. 2 illustrates a video session that provides only a single video object layer for video object VOØ.

There is no limit to the number of video object layers that may be provided for a single video object. However, each video object layer added to a video object will be associated with a certain amount of administrative information required to code the video object layer. The overhead administrative data can impair coding efficiency.

FIG. 3 illustrates a video coding system constructed in accordance with an embodiment of the present invention. The coding system includes an encoder 100 and a decoder 200 separated by a channel 300. The encoder 100 receives input video objects data and codes the video objects data according to the coding scheme described above with respect to FIG. 2. The encoder 100 outputs coded data to the channel 300. The decoder 200 receives the coded data from the channel 300 and decodes it using techniques complementary to those used at the encoder 100. The decoder outputs decoded video data for display, storage or other use.

The channel 300 may be a real time data medium in which coded data output from the encoder 100 is routed directly to the decoder 200. As such, the channel 300 may be represented by a data communication channel provided by the Internet, a computer network, a wireless data network or a telecommunication network. The channel 300 may also be a storage medium, such as a magnetic, optical or electrical memory. In these applications, the encoder 100 and decoder 200 need not work contemporaneously. The encoder 100 may store coded data in the channel 300 where the coded data may reside until retrieved by the decoder 200.

The encoder 100 includes a video object segmenter/formatter 400, plurality of video object encoders 500 a-n and a systems multiplexer (“MUX”) 600. In a typical application, the encoder 100 may be a microprocessor or digital signal processor that is logically divided into these components 400-600 by program instructions. Alternatively, the components 400-600 may be populated by hardware components adapted to perform these functions.

The video objects segmenter/formatter 400 receives input video data and identifies video objects therefrom. The process of decomposing an image sequence into video objects is well known and described in “Coding of Moving Pictures and Video,” ISO/IEC 14496-2 (July 1997). The video object segmenter/formatter 400 outputs VOP data to each of the video object encoders 500 a-n.

The video object encoders 500 a-n receive the VOP data of their respective video objects and code the VOP data according to the structure shown in FIG. 2. That is, the video object encoder (say, 500 a) determines how many video object layers to use in coding the video object data. It determines what part of the input VOP data is coded as base layer data and what part is coded as enhancement layer data. The video object encoder codes the base layer data and any enhancement layer data as coded VOPs of each video object layer. It outputs coded video object data to the MUX 600.

The MUX 600 organizes the coded video object data received from each of the video object encoders 500 into a data stream and outputs the data stream to the channel 300. The MUX 600 may merge data from other sources, such as audio coders (not shown), graphics coder (not shown), into the unitary signal stream.

The decoder 200 includes a systems demultiplexer (“DEMUX”) 700, a plurality of video object decoders 800 a-n and a video objects compositor 900. As with the encoder 100, the decoder 200 may be a microprocessor or digital signal processor that is logically divided into these components 700-900 by program instructions. Alternatively, the components 700-900 may be populated by hardware components adapted to perform these functions.

The DEMUX 700 retrieves the unitary coded signal from the data stream channel 300. It distinguishes the coded data of the various video objects from each other. Data for each video object is routed to a respective video object decoder 800 a-n. Other coded data, such as graphics data or coded audio data, may be routed to other decoders (not shown).

The video object decoders 800 a-n decode base layer data and any enhancement layer data using techniques complementary to those applied at the video object encoders 500 a-n. The video object decoders 800 a-n output decoded video objects.

The video objects compositor 900 assembles a composite image from the decoded VOP data of each video object. The video objects compositor 900 outputs the composite image to a display, memory or other device as determined by a user.

FIG. 4 is a block diagram of a video object encoder 500 a of the present invention. The video object encoder includes a scalability pre-processor 510, a base layer encoder 520, a midprocessor 530, an enhancement layer encoder 540 and an encoder multiplexer 550. Again, the components of the video object encoder 500 a may be provided in hardware or may be logical devices provided in a microprocessor or a digital signal processor.

VOP data of a video object is input to the scalability pre-processor 510. The scalability pre-processor 510 determines which data is to be coded in the base layer and which data is to be coded in the enhancement layer. It outputs a first set of VOPs to the base layer encoder 520 and a second set of VOPs to the enhancement layer encoder 540.

The base layer encoder 520 codes base layer VOPs according to conventional techniques. Such coding may include the nonscalable coding techniques of the MPEG-4 standard. Base layer VOPs are coded by intra coding, predictive coding or bi-directionally predictive coding and output on line 522 to the encoder multiplexer MUX 550. The base layer encoder also outputs locally decoded VOPs on line 524. The base layer encoder obtains locally decoded VOPs by decoding the coded base layer data. Effectively, the locally decoded VOPs mimic decoded base layer data that is obtained at the decoder 200.

The midprocessor 530 receives the locally decoded VOPs and depending on its mode of operation, outputs up sampled, down sampled or unchanged VOP data to the enhancement layer encoder 540.

The enhancement layer encoder 540 receives VOP data from the scalability preprocessor 510 and locally decoded VOP data possibly having been modified by the midprocessor 530. The enhancement layer encoder 540 codes the VOP data received from the scalability preprocessor using the locally decoded VOP data as a basis for prediction. It outputs coded enhancement layer data to the encoder multiplexer 550.

The encoder multiplexer MUX 550 outputs coded base and enhancement layer video object data from the video object encoder.

FIG. 5 illustrates an example of object based temporal scalability that may be achieved by the present invention. There, a first sequence of VOPs 1010, 1030, 1050, are coded by the base layer encoder 520 and a second sequence of VOPs 1020, 1040, are coded by the enhancement layer encoder 540. In time order, the VOPs appear in the order: 1010, 1020, 1030, 1040, 1050, . . . .

The base layer encoder 520 codes VOP 1010 first as an I-VOP. Second, it codes VOP 1050 as a P-VOP using VOP 1010 as a basis for prediction. Third, it codes VOP 1030 as a B-VOP using VOPs 1010 and 1050 as bases for prediction.

The enhancement layer encoder 540 codes VOP 1020 using base layer locally decoded VOPs 1010 and 1030 as bases for prediction. It also codes VOP 1040 using base layer locally decoded VOPs 1030 and 1050 as bases for prediction. Although not shown in FIG. 5, an enhancement layer VOP (such as 1040) can look to another enhancement layer VOP as a basis for prediction. For example, VOP 1040 could be coded using VOPs 1020 as a basis for prediction.

On decoding, a simple decoder decodes only the coded base layer data. It decodes and displays VOPs 1010, 1030, 1050, . . . providing a video sequence for display having a first frame rate. A power decoder, however, that decodes both base layer and enhancement layer data obtains the entire VOP sequence 1010, 1020, 1030, 1040, 1050, . . . . It decodes a video sequence having a higher frame rate. With a higher frame rate, an observer would perceive more natural motion.

FIG. 6 illustrates an example of object based spatial scalability that may be achieved by the present invention. There, VOPs 1110-1140 are coded by the base layer encoder 520. Spatially, larger VOPs 1210-1240 are coded by the enhancement layer encoder 540. Enhancement layer VOPs 1210-1240 coincide, frame for frame, with the base layer VOPs 1110-1140.

The base layer encoder 520 codes the base layer VOPs in the order 1110, 1130, 1120, . . . . VOP 1110 is coded as an I-VOP. VOP 1130 is coded as a P-VOP using VOP 1110 as a basis for prediction. VOP 1120 is coded third as a B-VOP using VOPs 1110 and 1130 as a basis for prediction. VOP 1140 is coded sometime thereafter using VOP 1130 and another VOP (not shown) as a basis for prediction.

The enhancement layer encoder 540 codes the enhancement layer VOPs in the order 1210, 1220,1230, 1240, . . . . As shown in FIG. 6, VOP 1210 is a P-VOP coded using VOP 1110 as a basis for prediction. VOP 1220 is coded as a B-VOP using base layer VOP 1120 and enhancement layer VOP 1210 as a basis for prediction. VOPs 1230 and 1240 are coded in a manner similar to VOP 1220; they are coded as B-VOPs using the temporally coincident VOP from the base layer and the immediately previous enhancement layer VOP as a basis for prediction.

On decoding, a simple decoder that decodes only the coded base layer data obtains the smaller VOPs 1110-1140. However, a more powerful decoder that decodes both the coded base layer data and the coded enhancement layer data obtains a larger VOP. On display, the decoded video object may be displayed as a larger image or may be displayed at a fixed size but may be displayed with higher resolution.

Scalability also provides a graceful degradation in image quality in the presence of channel errors. In one application, the coded base layer data may be supplemented with error correction coding. As is known, error correction coding adds redundancy to coded information. Error coded signals experience less vulnerability to transmission errors than signals without error coding. However, error coding also increases the bit-rate of the signal. By providing error correction coding to the coded base layer data without providing such coding to the coded enhancement layer data, an intermediate level of error protection is achieved without a large increase in the bit rate. Enhancement layer VOPs are not error coded, which would otherwise reduce the transmitted bit rate of the unified signal. When channel errors occur, the coded base layer data is protected against the errors. Thus, at least a basic representation of the video object is maintained. Graceful signal degradation is achieved in the presence of channel errors.

FIG. 7 illustrates a block diagram of a video object decoder 800 a constructed in accordance with an embodiment of the present invention. The video object decoder 800 a includes a decoder demultiplexer (DEMUX) 810, a base layer decoder 820, a midprocessor 830, an enhancement layer decoder 840 and a scalability post-processor 850. The components of the video object decoder 800 a may be provided in hardware or may be logical devices provided in a microprocessor or a digital signal processor.

The DEMUX 810 receives the coded video object data from the system demultiplexer 700 (FIG. 3). It distinguishes coded base layer data from coded enhancement layer data and routes each type of data to the base layer decoder 820 and enhancement layer decoder 840 respectively.

The base layer decoder 820 decodes the coded base layer data to obtain base layer VOPs. It outputs decoded base layer VOPs on output 822. In the absence of channel errors, the decoded base layer VOPs should represent identically the locally decoded VOPs output on line 524 from the base layer encoder 520 to the midprocessor 530 (FIG. 4). The decoded base layer VOPs are input to the scalability post processor 850 and to the midprocessor 830 (line 524).

The decoder midprocessor 830 operates identically to the encoder midprocessor 530 of FIG. 4. If midprocessor 530 had up sampled locally decoded VOPs, midprocessor 830 up samples the decoded base layer VOPs. If midprocessor 530 had down sampled or left unchanged the locally decoded VOPs, midprocessor 830 also down samples or leaves unchanged the decoded base layer VOPs. An output of the midprocessor 830 is input to the enhancement layer decoder 840.

The enhancement layer decoder 840 receives coded enhancement layer data from the DEMUX 810 and decoded base layer data (possibly modified) from the midprocessor 830. The enhancement layer decoder 840 decodes the coded enhancement layer data with reference to the decoded base layer data as necessary. It outputs decoded enhancement layer VOPs to the scalability post-processor 850.

The scalability post-processor 850 generates composite video object data from the decoded base layer data and the decoded enhancement layer data. In the case of temporal scalability, the scalability post-processor 850 reassembles the VOPs in the correct time ordered sequence. In the case of spatial scalability, the scalability post-processor outputs the decoded enhancement layer data. The decoded base layer data is integrated into the decoded enhancement layer VOPs as part of the decoding process.

FIG. 8 illustrates a block diagram of the scalability pre-processor 510 (FIG. 4). The scalability pre-processor 510 includes a temporal decimator 511, a horizontal and vertical decimator 512 and a temporal demultiplexer 513. It can perform spatial resolution reduction (horizontal and/or vertical) and temporal resolution reduction by dropping intermediate pictures or VOPs as necessary. VOPs input to the scalability pre-processor are input on line 514. The scalability pre-processor outputs VOPs to the base layer decoder on line 515 and other VOPs to the enhancement layer decoder on line 516.

The temporal decimator 511 reduces the VOP rate of both the base layer and the enhancement layer by dropping predetermined VOPs.

The temporal demultiplexer is used for temporal scalability. For a given VOP input to it, the temporal demultiplexer 513 routes it to either the base layer decoder (over output 515) or to the enhancement layer decoder (over output 516).

The horizontal and vertical decimator 512 may be used for spatial scalability. Each VOP input to the scalability pre-processor (or, at least, those output from the temporal decimator) is output directly to the enhancement layer decoder over line 516. The VOPs are also input to the horizontal and vertical decimator where image data of each VOP is removed to shrink them. The shrunken VOPs output from the horizontal and vertical decimator are output to the base layer encoder over line 515.

FIG. 9 is a block diagram of an enhancement layer encoder 540 for video objects constructed in accordance with the present invention. The enhancement layer encoder 540 includes a VOP Motion Compensated DCT Encoder 541, a VOP Interlayer Motion Estimator 542 (“VIME”) and a VOP Interlayer Motion Compensated Predictor 543. It receives the enhancement layer VOPs from the scalability pre-processor 510 at input 544 and the locally decoded base layer VOPs (possibly modified) at input 545. The enhancement layer encoder outputs the coded enhancement layer data on output 546.

The enhancement layer encoder 540 receives the enhancement layer VOPs from the scalability pre-processor 510 on input 544. They are input to the VOP Motion Compensated DCT Encoder 541 and to the VOP Interlayer Motion Estimator 542. The VOP Motion Compensated DCT Encoder 541 is a motion compensated transform encoder that is adapted to accept a predicted VOP and motion vectors as inputs. The motion vectors are generated by VIME 542, a normal motion estimator that has been adapted to accept enhancement layer VOPs from input 544.

VIME 542 performs motion estimation on an enhancement layer VOP with reference to a locally decoded base layer VOP. It outputs motion vectors to the VOP Interlayer Motion Compensated Predictor 543 and, selectively, to the VOP Motion Compensated DCT Encoder 541.

The VOP Interlayer Motion Compensated Predictor 543 is a normal motion compensated predictor that operates on the locally decoded base layer VOPs received from the midprocessor 530. It obtains a prediction from one or two possible sources of prediction. In a first prediction, prediction is made with reference to a first VOP. In a second prediction, prediction is made with reference to a second VOP. A third prediction obtains an average of the first and second predictions. The source of predictions, the first and second VOPs, may be located in either the base layer or enhancement layer. Arrows in FIGS. 5 & 6 illustrate exemplary prediction directions.

In an MPEG-4 system image data of video objects is organized into blocks of image data. Prediction according to the three predictions described above may be performed on a block by block basis. Thus a first block of a VOP may be predicted using prediction 1 (First VOP), a second block may be predicted using prediction 2 (second VOP), and a third block may be predicted using prediction 3 (both VOPs). In the embodiment, the first and second VOPs are properly viewed as possible sources for prediction because they may be used as sources for prediction but are not necessary used.

The VOP Interlayer Motion Compensated Predictor 543 outputs predicted VOPs. The output of the VOP Interlayer Motion Compensated Predictor 543 or the locally decoded base layer VOPs are input to the VOP Motion Compensated DCT Encoder 541.

FIG. 10 is a block diagram of a midprocessor 530, 830 constructed in accordance with an embodiment of the present invention. The midprocessor 530, 830 includes a horizontal interpolator 531 and a vertical interpolator 532 on a first processing path, a horizontal decimator 533 and a vertical decimator 534 on a second processing path and a third, shunt path 535. It receives VOPs on input 536 and outputs VOPs on an output 537.

The horizontal interpolator 531 and vertical interpolator 532 are enabled when the midprocessor 530, 830 operates in an up sampling mode. For each VOP, the horizontal interpolator 531 and vertical interpolator 532 enlarge the VOP and calculate image data for data point(s) between original data points.

The horizontal decimator 533 and vertical decimator 534 are enabled when the midprocessor 530, 830 operates in down sampling mode. The horizontal decimator 533 and vertical decimator 534 reduce the VOP and remove image data for certain of the original data points.

The shunt path 535 outputs untouched the VOPs input to the midprocessor 530, 830.

FIG. 11 is a block diagram of the enhancement layer decoder of video objects 840 of FIG. 7. The enhancement layer decoder 840 includes a VOP Motion Compensated DCT Decoder 841 and a VOP Interlayer Motion Compensated Predictor 842. The coded enhancement layer data is input to the enhancement layer decoder on input 843. Decoded base layer VOPs received from the midprocessor 830 are input to the enhancement layer decoder on input 844. The enhancement layer decoder 840 outputs decoded enhancement layer VOPs on output 845.

The VOP Motion Compensated DCT Decoder 841 decodes motion vectors as well as the prediction mode from the coded enhancement layer data and outputs them to the VOP Interlayer Motion Compensated Predictor 842 along with decoded enhancement layer previous VOP. The VOP Interlayer Motion Compensated Predictor 842 also receives the decoded base layer VOPs from line 844. The VOP Interlayer Motion Compensated Predictor 842 outputs predicted VOPs back to the VOP Motion Compensated DCT Decoder 841. Based upon either the enhanced layer previous decoded VOPs or the decoded base layer VOPs, or their combination, the VOP Motion Compensated DCT Decoder 841 generates the decoded enhancement layer VOPs. Among the combinations allowed at the encoder are one-half of previous decoded enhancement layer VOP and one-half of the base layer VOP, as well as one-half of a previous and a next decoded VOP of base layer.

FIG. 12 is a block diagram of the scalability post-processor 850. It includes a temporal multiplexer 851 and a temporal interpolator 852. The scalability post-processor 850 receives decoded base layer data on input 853 and decoded enhancement layer VOPs on input 854. It outputs composite video object data on output 855.

The temporal multiplexer 851 reassembles the VOPs from the base layer and the enhancement layer into a single stream of VOPs. The temporal interpolator 852 is used for temporal scalability to rearrange VOPs into the correct time ordered sequence. For spatial scalability, the decoded base layer VOPs may be ignored; the decoded enhancement layer data bypasses the temporal multiplexer 851.

The temporal interpolator 852 increases the frame rate of the VOPs in a manner that complements the temporal decimator 511 of the video object encoder 500 a (FIG. 8). If the temporal decimator 511 was bypassed for encoding, the temporal interpolator 852 may be bypassed during decoding.

As has been shown, the present invention provides a system providing scalability, either temporal scalability, spatial scalability or both. VOPs are separated into base layer VOPs and enhancement layer VOPs and coded as such. On decoding, a specific decoder may decode the coded base layer data with or without the coded enhancement layer data, depending on it processing power and channel conditions.

The present invention also provides a general scalability syntax while coding.

Generalized scalability allows predictions to be correctly formed at the decoder by embedding the necessary codes indicating the specific type of temporal scalability or spatial scalability to be derived. The reference VOPs for prediction are selected by reference_select_code as described in Tables 1 and 2. In coding P-VOPs belonging to an enhancement layer, the forward reference can be one of the following three: the most recent decoded VOP of enhancement layer, the most recent VOP of the lower layer in display order, or the next VOP of the lower layer in display order.

In B-VOPs, the forward reference can be one of the two: the most recent decoded enhancement VOP or the most recent lower layer VOP in display order. The backward reference can be one of the three: the temporally coincident VOP in the lower layer, the most recent lower layer VOP in display order, or the next lower layer VOP in display order.

TABLE 1
Prediction Reference Choices For P-VOPs
in The Object-Based Temporal Scalability
ref_select
code Forward Prediction Reference
00 Most recent decoded enhancement VOP belonging
to the same layer.
01 Most recent VOP in display order belonging to
the reference layer.
10 Next VOP in display order belonging to the
reference layer.
11 Temporally coincident VOP in the reference
layer (no motion vectors)

TABLE 2
Prediction Reference Choices For B-VOPs
In The Case of Scalability
ref_select Forward Temporal Backward Temporal
code Reference Reference
00 Most recent decoded Temporally coincident VOP in
enhancement VOP of the reference layer
the same layer (no motion vectors)
01 Most recent decoded Most recent VOP in display
enhancement VOP of order belonging to the
the same layer. reference layer.
10 Most recent decoded Next VOP in display order
enhancement VOP of belonging to the
the same layer. reference layer.
11 Most recent VOP in Next VOP in display
display order belonging order belonging to the
to the reference layer. reference layer.

The enhancement layer can contain P or B-VOPs, however, in scalability configurations of FIG. 4 and FIG. 5, the B-VOPs in the enhancement layer behave more like P-VOPs at least in the sense that a decoded B-VOP can be used to predict the following P or B-VOPs.

When the most recent VOP in the lower layer is used as reference, this includes the VOP that is temporally coincident with the VOP in the enhancement layer. However, this necessitates use of lower layer for motion compensation which requires motion vectors.

If the coincident VOP in the lower layer is used explicitly as reference, no motion vectors are sent and this mode can be used to provide spatial scalability. Spatial scalability in MPEG-2 uses spatio-temporal prediction, which is accomplished as per FIG. 5 more efficiently by simply using the three prediction modes: forward prediction (prediction direction 1), backward prediction (prediction direction 2), interpolated prediction (prediction directions 1 and 2) available for B-VOPs.

Since the VOPs can have a rectangular shape (picture) or an irregular shape, both the traditional as well as object based temporal and spatial scalabilities become possible. We now provide some details by which scalability can be accomplished for arbitrary shaped VOPs by extending the technique of chroma-keying known in the art. Normally, scalable coding of arbitrary shaped objects requires explicit transmission of shape information of each VOP, however, by use of a simpler technique of chroma-keying in which only rectangular VOPs containing arbitrary shaped VOP are coded such that in the region outside of arbitrary shape of interest a key color (not present anywhere in the VOP) is inserted by the encoder and specified in the bitstream allowing deletion by the decoder, the only caveat is that the key color insertion/deletion is performed not only on arbitrary shape VOPs of lower (here, a base) layer but also in enhancement layer. Thus it becomes possible at the decoder to recover VOPs of scalable arbitrary shape since coding is really performed on rectangular VOP windows in the same manner as coding of pictures.

The class hierarchy introduced in FIG. 2 can be used to implement a practical bitstream representation that may allow ease of access for object manipulation and editing functionalities. For illustrative purposes, they are described with reference to syntax elements from “MPEG-4 Video Verification Model Version 2.1,” ISO/IEC JTC1/SC29/WG11, MPEG 96/776 (March 1996) (herein, “VM 2.1”). Tables 3-6 illustrate by example some bitstream details of video syntax class and meaning of various syntax elements in each class, particularly for reorganized or new syntax elements.

TABLE 3
Video Session
No. of
Syntax bits
VideoSession() {
video_session_start_code 32
do {
do {
VideoObject()
} while (nextbits() = =
video_object_start_code)
if (nextbits() != session_end_code)
video_session_start_code 32
} while (nextbits() != video_session_end_code)
video_session_end_code 32
}

TABLE 4
Video Object
No. of
Syntax bits
VideoObject() {
video_object_start_code 24 + 3
object_id  5
do {
VideoObjectLayer()
} while (nextbits() = =
video_object_layer_start_code)
next_start_code()
}
object_id: It uniquely identifies a layer. It is a 5-bit quantity with values from 0 to 31.

TABLE 5
Video Object Layer
No. of
Syntax bits
VideoObjectLayer() {
video_object_layer_start_code 28 
layer_id 4
layer_width 10 
layer_height 10 
quant_type_sel 1
if (quant_type_sel) {
load_intra_quant_mat 1
if (load_intra_quant_mat)
intra_quant_mat[64] 8*64
load_nonintra_quant_mat 1
if (load_nonintra_quant_mat)
nonintra_quant_mat[64] 8*64
}
intra_dcpred_disable 1
scalability 1
if (scalability) {
ref_layer_id 4
ref_layer—sampling_direc 1
hor_sampling_factor_n 5
hor_sampling_factor_m 5
vert_sampling_factor_n 5
vert_sampling_factor_m 5
enhancement_type 1
}
do {
VideoObjectPlane()
} while (nextbits() = =
video_object_plane_start_code}
next_start_code()
}
layer_id: It uniquely identifies a layer. It is a 4-bit quantity with values from 0 to 15. A value of 0 identifies the first independently coded layer.
layer_width, layer_height: These values define the spatial resolution of a layer in pixels units.
Scalability: This is a 1-bit flag that indicates if scalability is used for coding of the current layer.
ref_layer_id: It uniquely identifies a decoded layer to be used as a reference for predictions in the case of scalability. It is a 4-bit quantity with values from 0 to 15.
ref_layer_sampling_direc: This is a 1-bit flag whose value when “0” indicates that the reference layer specified by ref_layer_id has the same or lower resolution as the layer being coded. Alternatively, a value of “1” indicates that the resolution of reference layer is higher than the resolution of layer being coded resolution.
hor_sampling_factor_n, hor_sampling_factor_m: These are 5-bit quantities in range 1 to 31 whose ratio hor_sampling_factor_n/ hor_sampling_factor_m indicates the resampling needed in horizontal direction; the direction of sampling is indicated by ref_layer_sampling_direc.
vert_sampling_factor_n, vert_sampling_factor_m: These are 5-bit quantities in range of 1 to 31 whose ratio vert_sampling_factor_n/vert_sampling_factor_m indicates the resampling needed in vertical direction; the direction of sampling is indicated by ref_layer_sampling_direc.
enhancement_type: This is a 1-bit flag that indicates the type of an enhancement structure in a scalability. It has a value of ì1î when an enhancement layer enhances a partial region of the base layer. It has a value of ì0î when an enhancement layer enhances entire region of the base layer. The default value of this flag is ì0î.

TABLE 6
Video Object Plane
No. of
Syntax bits
VideoObjectPlane() {
video_object_plane_start_code 32
vop_temp_ref 16
vop_visibility  1
vop_of_arbitrary_shape  1
if (vop_of_arbitrary_shape) {
vop_width 10
vop_height 10
if (vop_visibility) {
vop_composition_order  5
vop_hor_spatial_ref 10
marker_bit  1
vop_vert_spatial_ref 10
vop_scaling  3
:
:
}
:
/* syntax to derive shapes by
deleting key color */
:
}
vop_coding_type  2
if (vop_coding_type = = 1 ||
vop_coding_type = = 2)
{
vop_fcode_forward  2
if (vop_coding_type = = 2) {
vop_fcode_backward  2
vop_dbquant  2
}
else {
vop_quant  5
}
if (!scalability) {
separate_motion_texture  1
if (!separate_motion_texture)
combined_motion_texture_coding()
else {
motion_coding()
texture_coding()
}
}
else {
:
/* syntax to derive forward and backward shapes by  1
deleting key color */
:
}
ref_select_code  2
if (vop_coding_type = = 1 ||
vop_coding_type = =
2) {
forward_temporal_ref 10
if (plane_coding_type = = 2) {
marker_bit  1
backward_temporal_ref 10
}
}
combined_motion_texture_coding()
}
}

The meaning of the syntax elements of video object planes is specified in VM2.1.

Accordingly, the present invention provides a video coding system and syntax supporting generalized scalability. The system finds application with limited or noisy channels and with decoders of varying processing power.

Claims (26)

We claim:
1. A method of coding video information, comprising:
receiving the video information,
identifying a video object in the video information, for the video object,
coding a first part of the video information associated with the one video object as a first video object layer, the first video object layer including a first plurality of video object planes, and
coding a second part of the video information associated with the one video object as a second video object layer, the second video object layer including a second plurality of video object planes.
2. The method of claim 1, wherein the first video object layer is a base layer, representing the video information at a first level of image quality.
3. The method of claim 2, wherein the second video object layer is an enhancement layer representing a portion of the video information not represented by the base layer.
4. The method of claim 1 wherein the second video object layer represents the video object at a greater spatial resolution than the first video object layer.
5. The method of claim 1, wherein the second video object layer represents the video object at different times than the first video object layer.
6. A method of decoding coded video data, the coded video data including coded first and second video object layers for a video object, the method comprising:
receiving the coded video data,
decoding the coded first video object layer the first video object layer including a first plurality of video object planes,
decoding the coded second video object layer, the second video object layer including a second plurality of video object planes, and
generating a decoded video object based upon the decoded first and second video object layers.
7. The method of claim 6, wherein the decoded first video object layer is a base layer representing source video information at a first level of image quality.
8. The method of claim 6, wherein the decoded video object includes video information of the video object at a greater temporal rate than would be obtained by decoding only the coded first video object layer.
9. The method of claim 6, wherein the decoded second video object layer represents the video object at different times than the decoded first video object layer.
10. The method of claim 6, wherein the decoded second video object layer represents the video object at a greater spatial resolution would be obtained by decoding only the coded first video object layer.
11. A method of decoding coded video data, the coded video data including coded first and second video object layers, the method comprising:
receiving the coded video data,
distinguishing the coded first video object layer from the coded video data the first video object layer including a first plurality of video object planes,
decoding the coded first video object layer the second video object layer including a second plurality of video object planes, and
generating a decoded video object based upon the decoded first video object layer.
12. A method of coding video information, comprising:
identifying a video object in the video information,
representing the video object as a series of video object planes,
coding a first part of the video object planes as a base video object layer, and
coding a second part of the video object planes as an enhancement video object layer.
13. The method of claim 12, wherein a first number of the video object planes are coded in the base video object layer and a second number of the video object planes are coded in the enhancement video object layer.
14. The method of claim 12, wherein a video object plane in the enhancement video object layer is coded based upon a prediction made from a video object plane in the base video object layer.
15. The method of claim 12, wherein a video object plane in the enhancement video object layer is coded based on two candidates for prediction: a video object plane in base video object layer and a second video object plane in enhancement layer.
16. The method of claim 12, wherein a video object plane in the enhancement video object layer is coded based upon a prediction made from a second video object plane in the enhancement video object layer.
17. The method of claim 12,
wherein the coded base and enhancement video object layers provide spatial scalability,
wherein at least one video object plane is coded in the base video object layer at a first size and the same video object plane is coded in the enhancement video object layer at a second, larger size.
18. The method of claim 12, wherein coding of the one video object plane in the enhancement video object layer is made as a prediction based upon the coding of the one video object plane in the base video object layer.
19. The method of claim 12, wherein coding of the one video object plane in the enhancement video object layer is made as a prediction based upon the coding of the one video object plane in the base video object layer.
20. The method of claim 12, wherein the coded base and enhancement video object layers provide spatial scalability wherein at least one video object plane is coded in the base video object layer at a first resolution and the same video object plane is coded in the enhancement video object layer at a second, larger resolution.
21. A scalable video coding method providing generalized scalability, comprising:
identifying a video object from the video information,
representing the video object as a series of video object planes,
coding a first part of the video object planes as a base video object layer, and
coding a second part of the video object planes as an enhancement video object layer, the coding of the coded base video object layer as a candidate for prediction using a single syntax applicable for both temporal and spatial scalability.
22. A method for decoding coded video data, comprising:
decoding a first part of the video data as a base video object layer, the base video object layer including a first plurality of video object planes, and
decoding a second part of the video data as an enhancement video object layer, the enhancement video object layer including a second plurality of video object planes, the decoding made as a prediction based upon the decoded base video object layer and with reference to a syntax in the coded video data identifying whether temporal and spatial scalability coding is present in the coded video data.
23. The method of claim 22, wherein the decoding of the enhancement video object layer comprises generating, for the video object layer, one of three predictions from the group of:
a prediction from a first video object plane,
a prediction from a second video object plane, and
a prediction obtained by averaging the first and second predictions.
24. The method of claim 22, wherein the coded video data identifies which of the three predictions should be used for decoding the enhancement video object layer.
25. The method of claim 22, wherein the coded video information represents blocks of data for the enhancement video object layer and the predictions are made independently on a block-by-block basis.
26. The method of claim 22, wherein the coded video data identifies, on a block-by-block basis, which of the three predictions should be used for decoding the enhancement video object layer.
US09111215 1997-07-08 1998-07-07 Generalized scalability for video coder based on video objects Active US6233356B1 (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
US6988897 true 1997-07-08 1997-07-08
US09111215 US6233356B1 (en) 1997-07-08 1998-07-07 Generalized scalability for video coder based on video objects

Applications Claiming Priority (8)

Application Number Priority Date Filing Date Title
US09111215 US6233356B1 (en) 1997-07-08 1998-07-07 Generalized scalability for video coder based on video objects
US09814061 US6526177B1 (en) 1997-07-08 2001-03-22 Generalized scalability for video coder based on video objects
US10336709 US6707949B2 (en) 1997-07-08 2003-01-06 Generalized scalability for video coder based on video objects
US10761518 US6993201B1 (en) 1997-07-08 2004-01-20 Generalized scalability for video coder based on video objects
US11197700 US7457474B1 (en) 1997-07-08 2005-08-04 Generalized scalability for video coder based on video objects
US12253307 US8385667B2 (en) 1997-07-08 2008-10-17 Generalized scalability for video coder based on video objects
US13776500 US8824815B2 (en) 1997-07-08 2013-02-25 Generalized scalability for video coder based on video objects
US14473023 US9420309B2 (en) 1997-07-08 2014-08-29 Generalized scalability for video coder based on video objects

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US09814061 Continuation US6526177B1 (en) 1997-07-08 2001-03-22 Generalized scalability for video coder based on video objects

Publications (1)

Publication Number Publication Date
US6233356B1 true US6233356B1 (en) 2001-05-15

Family

ID=26750528

Family Applications (7)

Application Number Title Priority Date Filing Date
US09111215 Active US6233356B1 (en) 1997-07-08 1998-07-07 Generalized scalability for video coder based on video objects
US09814061 Active US6526177B1 (en) 1997-07-08 2001-03-22 Generalized scalability for video coder based on video objects
US10336709 Active US6707949B2 (en) 1997-07-08 2003-01-06 Generalized scalability for video coder based on video objects
US11197700 Active 2019-08-30 US7457474B1 (en) 1997-07-08 2005-08-04 Generalized scalability for video coder based on video objects
US12253307 Active 2021-07-27 US8385667B2 (en) 1997-07-08 2008-10-17 Generalized scalability for video coder based on video objects
US13776500 Active US8824815B2 (en) 1997-07-08 2013-02-25 Generalized scalability for video coder based on video objects
US14473023 Active US9420309B2 (en) 1997-07-08 2014-08-29 Generalized scalability for video coder based on video objects

Family Applications After (6)

Application Number Title Priority Date Filing Date
US09814061 Active US6526177B1 (en) 1997-07-08 2001-03-22 Generalized scalability for video coder based on video objects
US10336709 Active US6707949B2 (en) 1997-07-08 2003-01-06 Generalized scalability for video coder based on video objects
US11197700 Active 2019-08-30 US7457474B1 (en) 1997-07-08 2005-08-04 Generalized scalability for video coder based on video objects
US12253307 Active 2021-07-27 US8385667B2 (en) 1997-07-08 2008-10-17 Generalized scalability for video coder based on video objects
US13776500 Active US8824815B2 (en) 1997-07-08 2013-02-25 Generalized scalability for video coder based on video objects
US14473023 Active US9420309B2 (en) 1997-07-08 2014-08-29 Generalized scalability for video coder based on video objects

Country Status (1)

Country Link
US (7) US6233356B1 (en)

Cited By (65)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6337710B1 (en) * 1997-06-12 2002-01-08 Lsi Logic Corporation Graphical video editing system for regeneration of bitstreams
US20020071492A1 (en) * 2000-11-23 2002-06-13 Yves Ramanzin Video decoding method and corresponding decoder
US6459732B2 (en) * 1997-02-14 2002-10-01 At&T Corp. Chroma-key for efficient and low complexity shape representation of coded arbitrary video objects
US20020168086A1 (en) * 2001-04-18 2002-11-14 Takayuki Sugahara Encoding, producing and decoding methods of object data, and apparatuses for encoding, producing and decoding the object data, and programs for encoding and decoding the object data, and recording medium for the object data
US20020176025A1 (en) * 2001-03-05 2002-11-28 Chang-Su Kim Systems and methods for encoding redundant motion vectors in compressed video bitstreams
US6493387B1 (en) * 2000-04-10 2002-12-10 Samsung Electronics Co., Ltd. Moving picture coding/decoding method and apparatus having spatially scalable architecture and signal-to-noise ratio scalable architecture together
US6507618B1 (en) * 2000-04-25 2003-01-14 Hewlett-Packard Company Compressed video signal including independently coded regions
US6535559B2 (en) * 1997-04-01 2003-03-18 Sony Corporation Image encoder, image encoding method, image decoder, image decoding method, and distribution media
US20030064258A1 (en) * 2001-09-28 2003-04-03 Pan Alfred I-Tsung Fuel additives for fuel cell
US6553150B1 (en) 2000-04-25 2003-04-22 Hewlett-Packard Development Co., Lp Image sequence compression featuring independently coded regions
US20030118113A1 (en) * 2001-12-20 2003-06-26 Comer Mary Lafuze Fine-grain scalable video decoder with conditional replacement
US20030118099A1 (en) * 2001-12-20 2003-06-26 Comer Mary Lafuze Fine-grain scalable video encoder with conditional replacement
US20030142872A1 (en) * 2002-01-29 2003-07-31 Masahiko Koyanagi Image processing apparatus, decoding device, encoding device, image processing system, image processing method, and encoding method
FR2835386A1 (en) * 2002-01-31 2003-08-01 Daniel Lecomte Device for the distribution securisee, conditional access, the controlled viewing, private copying and management of audiovisual content rights MPEG-4 Type
US6606329B1 (en) * 1998-07-17 2003-08-12 Koninklijke Philips Electronics N.V. Device for demultiplexing coded data
US6618444B1 (en) 1997-02-14 2003-09-09 At&T Corp. Scene description nodes to support improved chroma-key shape representation of coded arbitrary images and video objects
US20030169930A1 (en) * 2002-03-11 2003-09-11 Rong-Fuh Shyu Method and apparatus for digital signal processing
WO2003084236A1 (en) * 2002-03-29 2003-10-09 Koninklijke Philips Electronics N.V. Video encoding method and corresponding device and signal
US20040013308A1 (en) * 2002-07-18 2004-01-22 Lg Electronics Inc. Calculation method for prediction motion vector
US20040017852A1 (en) * 2002-05-29 2004-01-29 Diego Garrido Predictive interpolation of a video signal
US6707944B1 (en) * 1997-09-04 2004-03-16 Electronics And Telecommunications Research Institute Computational graceful degradation method using priority information in multiple objects case
US20040114183A1 (en) * 2002-12-12 2004-06-17 Wei-Jen Lee Method and apparatus for image data storage management
US20040131015A1 (en) * 2002-12-20 2004-07-08 Ho-Sang Sung System and method for transmitting and receiving wideband speech signals
US20040158878A1 (en) * 2003-02-07 2004-08-12 Viresh Ratnakar Power scalable digital video decoding
US6798977B2 (en) * 1998-02-04 2004-09-28 Canon Kabushiki Kaisha Image data encoding and decoding using plural different encoding circuits
US6798838B1 (en) * 2000-03-02 2004-09-28 Koninklijke Philips Electronics N.V. System and method for improving video transmission over a wireless network
FR2854531A1 (en) * 2003-05-02 2004-11-05 Medialive Method and system for securiser scrambling, descrambling and distribution of vector visual sequences
US6847683B2 (en) * 2000-11-23 2005-01-25 Koninklijke Philips Electronics N.V. Video coding method and corresponding encoder
US20050111558A1 (en) * 2002-01-30 2005-05-26 Medialive Secure device that is used to process high-quality audiovisual works
US20050175093A1 (en) * 2004-02-06 2005-08-11 Haskell Barin G. Target bitrate estimator, picture activity and buffer management in rate control for video coder
US20050175092A1 (en) * 2004-02-06 2005-08-11 Atul Puri H.264/AVC coder incorporating rate and quality controller
US20050180502A1 (en) * 2004-02-06 2005-08-18 Atul Puri Rate control for video coder employing adaptive linear regression bits modeling
US6934906B1 (en) 1999-07-08 2005-08-23 At&T Corp. Methods and apparatus for integrating external applications into an MPEG-4 scene
US20050232364A1 (en) * 1997-10-20 2005-10-20 Shinichi Kuroda Image encoding device and image decoding device
US6970510B1 (en) 2000-04-25 2005-11-29 Wee Susie J Method for downstream editing of compressed video
US6973130B1 (en) 2000-04-25 2005-12-06 Wee Susie J Compressed video signal including information for independently coded regions
US20060062294A1 (en) * 2002-11-25 2006-03-23 Shu Lin Two-layer decoding for hybrid high-definition dvd
US7039107B1 (en) * 1998-09-29 2006-05-02 Siemens Aktiengesellschaft Method and array for processing a digitized image with image points
US20060120459A1 (en) * 2004-11-29 2006-06-08 Park Seung W Method for coding vector refinement information required to use motion vectors in base layer pictures when encoding video signal and method for decoding video data using such coded vector refinement information
US20060120454A1 (en) * 2004-11-29 2006-06-08 Park Seung W Method and apparatus for encoding/decoding video signal using motion vectors of pictures in base layer
US20060133499A1 (en) * 2004-11-29 2006-06-22 Park Seung W Method and apparatus for encoding video signal using previous picture already converted into H picture as reference picture of current picture and method and apparatus for decoding such encoded video signal
US20060159176A1 (en) * 2004-12-16 2006-07-20 Park Seung W Method and apparatus for deriving motion vectors of macroblocks from motion vectors of pictures of base layer when encoding/decoding video signal
US7095444B1 (en) * 2000-11-06 2006-08-22 Sri International Method and apparatus for generating, distributing and reconstructing deconstructed video
US20070091997A1 (en) * 2003-05-28 2007-04-26 Chad Fogg Method And Apparatus For Scalable Video Decoder Using An Enhancement Stream
US20070165842A1 (en) * 2004-03-16 2007-07-19 Medialive, A Corporation Of France Process and highly secure system for the distribution of audiovisual streams
US20070180481A1 (en) * 2004-03-23 2007-08-02 Hee-Kyung Lee Enhanced model of relation with quantitative representation, and tv anytime service method and system employing the same
US20070183499A1 (en) * 2004-08-16 2007-08-09 Nippon Telegraph And Telephone Corporation Picture encoding method, picture decoding method, picture encoding apparatus, picture decoding apparatus, picture encoding program, and picture decoding program
US20080001950A1 (en) * 2006-06-30 2008-01-03 Microsoft Corporation Producing animated scenes from still images
US20080063085A1 (en) * 2006-09-11 2008-03-13 Apple Computer, Inc. Post-processing for decoder complexity scalability
US20080062018A1 (en) * 2006-09-11 2008-03-13 Apple Computer, Inc. Complexity-aware encoding
US20080095238A1 (en) * 2006-10-18 2008-04-24 Apple Inc. Scalable video coding with filtering of lower layers
US20080214176A1 (en) * 2005-01-11 2008-09-04 Peter Amon Methods and Devices for the Transmission of Scalable Data
US20090304086A1 (en) * 2008-06-06 2009-12-10 Apple Inc. Method and system for video coder and decoder joint optimization
US7650058B1 (en) * 2001-11-08 2010-01-19 Cernium Corporation Object selective video recording
US7869503B2 (en) 2004-02-06 2011-01-11 Apple Inc. Rate and quality controller for H.264/AVC video coder and scene analyzer therefor
US7889792B2 (en) 2003-12-24 2011-02-15 Apple Inc. Method and system for video encoding using a variable number of B frames
US20110194026A1 (en) * 2005-08-08 2011-08-11 Koninklijke Philips Electronics, N.V. Method and system for video copyright protection
US8026945B2 (en) 2005-07-22 2011-09-27 Cernium Corporation Directed attention digital video recordation
US20120224630A1 (en) * 2001-01-09 2012-09-06 Wen-Hsiao Peng Method and apparatus for providing prediction mode fine granularity scalability
US20120230424A1 (en) * 2005-03-16 2012-09-13 Duvivier Christian L Multithread processing of video frames
US20130322530A1 (en) * 2012-05-14 2013-12-05 Luca Rossato Encoding and reconstruction of residual data based on support information
US8824815B2 (en) 1997-07-08 2014-09-02 At&T Intellectual Property Ii, L.P. Generalized scalability for video coder based on video objects
US8976856B2 (en) 2010-09-30 2015-03-10 Apple Inc. Optimized deblocking filters
EP2055106B1 (en) * 2006-08-16 2015-06-17 Microsoft Technology Licensing, LLC Techniques for variable resolution encoding and decoding of digital video
US9215467B2 (en) 2008-11-17 2015-12-15 Checkvideo Llc Analytics-modulated coding of surveillance video

Families Citing this family (52)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CA2281538C (en) * 1997-02-14 2006-01-03 Alexandros Eleftheriadis Object-based audio-visual terminal and bitstream structure
US7199836B1 (en) * 1998-02-13 2007-04-03 The Trustees Of Columbia University In The City Of New York Object-based audio-visual terminal and bitstream structure
US6292512B1 (en) * 1998-07-06 2001-09-18 U.S. Philips Corporation Scalable video coding system
JP4541482B2 (en) * 2000-02-29 2010-09-08 キヤノン株式会社 Image processing apparatus and image processing method
US6697426B1 (en) * 2000-03-17 2004-02-24 Koninklijke Philips Electronics N.V. Reduction of layer-decoding complexity by reordering the transmission of enhancement layer frames
JP4224662B2 (en) * 2000-08-09 2009-02-18 ソニー株式会社 The image coding apparatus and method, image decoding apparatus and method, and image processing apparatus
US20020080878A1 (en) * 2000-10-12 2002-06-27 Webcast Technologies, Inc. Video apparatus and method for digital video enhancement
US6907070B2 (en) * 2000-12-15 2005-06-14 Microsoft Corporation Drifting reduction and macroblock-based control in progressive fine granularity scalable video coding
US7480252B2 (en) * 2002-10-04 2009-01-20 Koniklijke Philips Electronics N.V. Method and system for improving transmission efficiency using multiple-description layered encoding
US8204079B2 (en) 2002-10-28 2012-06-19 Qualcomm Incorporated Joint transmission of multiple multimedia streams
US20040083495A1 (en) * 2002-10-29 2004-04-29 Lane Richard D. Mulitmedia transmission using variable gain amplification based on data importance
US20060133475A1 (en) * 2003-02-17 2006-06-22 Bruls Wilhelmus H A Video coding
US8437347B2 (en) * 2003-10-14 2013-05-07 Qualcomm Incorporated Scalable encoding for multicast broadcast multimedia service
US7899059B2 (en) * 2003-11-12 2011-03-01 Agere Systems Inc. Media delivery using quality of service differentiation within a media stream
US20050135478A1 (en) * 2003-12-18 2005-06-23 Van Der Schaar Mihaela Reduction of layer-decoding complexity by reordering the transmission of enhancement layer frames
DE102004059978B4 (en) * 2004-10-15 2006-09-07 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Apparatus and method for providing an encoded video sequence, and for decoding an encoded video sequence using a inter-layer residual values ​​prediction and a computer program and a computer readable medium
WO2006042612A1 (en) * 2004-10-15 2006-04-27 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Device and method for generating a coded video sequence and for decoding a coded video sequence while using an inter-layer residual value prediction
WO2006042611A1 (en) * 2004-10-15 2006-04-27 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Device and method for generating a coded video sequence while using an inter-layer movement data prediction
US20060156363A1 (en) * 2005-01-07 2006-07-13 Microsoft Corporation File storage for scalable media
KR100703746B1 (en) * 2005-01-21 2007-04-05 삼성전자주식회사 Video coding method and apparatus for predicting effectively unsynchronized frame
US7725799B2 (en) * 2005-03-31 2010-05-25 Qualcomm Incorporated Power savings in hierarchically coded modulation
US8619860B2 (en) * 2005-05-03 2013-12-31 Qualcomm Incorporated System and method for scalable encoding and decoding of multimedia data using multiple layers
JP4779435B2 (en) * 2005-05-17 2011-09-28 ソニー株式会社 Image processing apparatus, image processing method, and computer program
US20070003499A1 (en) * 2005-06-30 2007-01-04 The Gillette Company Particulate enhanced efficacy antiperspirant salt with raised pH
US20070014346A1 (en) * 2005-07-13 2007-01-18 Nokia Corporation Coding dependency indication in scalable video coding
KR100763196B1 (en) * 2005-10-19 2007-10-04 삼성전자주식회사 Method for coding flags in a layer using inter-layer correlation, method for decoding the coded flags, and apparatus thereof
WO2007080223A1 (en) * 2006-01-10 2007-07-19 Nokia Corporation Buffering of decoded reference pictures
US8937997B2 (en) * 2006-03-16 2015-01-20 Apple Inc. Scalable video coding/multiplexing compatible with non-scalable decoders
EP1848218A1 (en) * 2006-04-20 2007-10-24 THOMSON Licensing Method for deriving motion data for high resolution pictures from motion data of low resolution pictures and coding and decoding devices implementing said method
US8773494B2 (en) * 2006-08-29 2014-07-08 Microsoft Corporation Techniques for managing visual compositions for a multimedia conference call
US8990305B2 (en) * 2006-10-18 2015-03-24 Microsoft Corporation Techniques for virtual conferencing servers
US20080100694A1 (en) * 2006-10-27 2008-05-01 Microsoft Corporation Distributed caching for multimedia conference calls
US8503524B2 (en) * 2007-01-23 2013-08-06 Sharp Laboratories Of America, Inc. Methods and systems for inter-layer image prediction
US8665942B2 (en) 2007-01-23 2014-03-04 Sharp Laboratories Of America, Inc. Methods and systems for inter-layer image prediction signaling
US20090262838A1 (en) * 2008-04-21 2009-10-22 Qualcomm Incorporated Method and apparatus for encoding transmission data blocks for enhanced switching performance
US8218633B2 (en) * 2008-06-18 2012-07-10 Kiu Sha Management Limited Liability Company Bidirectionally decodable Wyner-Ziv video coding
US8539092B2 (en) * 2008-07-09 2013-09-17 Apple Inc. Video streaming using multiple channels
US8514931B2 (en) * 2009-03-20 2013-08-20 Ecole Polytechnique Federale De Lausanne (Epfl) Method of providing scalable video coding (SVC) video content with added media content
KR20100123075A (en) * 2009-05-14 2010-11-24 삼성전자주식회사 Appratus and method for supporting scalability scheme in a video telephony system
JP5455547B2 (en) * 2009-10-19 2014-03-26 キヤノン株式会社 Image processing apparatus and image processing method
US8428364B2 (en) 2010-01-15 2013-04-23 Dolby Laboratories Licensing Corporation Edge enhancement for temporal scaling with metadata
US8611414B2 (en) * 2010-02-17 2013-12-17 University-Industry Cooperation Group Of Kyung Hee University Video signal processing and encoding
JP5667773B2 (en) * 2010-03-18 2015-02-12 キヤノン株式会社 Information creating device and control method thereof
US8731152B2 (en) 2010-06-18 2014-05-20 Microsoft Corporation Reducing use of periodic key frames in video conferencing
EP2429192A1 (en) * 2010-08-17 2012-03-14 Streamworks International S.A. Video signal processing
US20130188740A1 (en) * 2010-09-13 2013-07-25 Industry-Academic Coooperation Foundation Hanbat National University Method and apparatus for entropy encoding/decoding
US9380324B2 (en) 2013-01-23 2016-06-28 Steven Schoenwald Video content distribution package
US9767854B2 (en) 2013-01-23 2017-09-19 Steven Schoenwald Video content distribution package
US9241128B2 (en) 2013-02-14 2016-01-19 Warner Bros. Entertainment Inc. Video conversion technology
CN105308962A (en) * 2013-04-05 2016-02-03 三星电子株式会社 Multi-layer video coding method and device, and multi-layer video decoding method and device
US20150373341A1 (en) * 2014-06-23 2015-12-24 Cisco Technology, Inc. Techniques for Interactive Region-Based Scalability
WO2016003221A1 (en) * 2014-07-03 2016-01-07 한국전자통신연구원 Signal multiplexing device and signal multiplexing method using layered division multiplexing

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5999189A (en) * 1995-08-04 1999-12-07 Microsoft Corporation Image compression to reduce pixel and texture memory requirements in a real-time image generator
US6005623A (en) * 1994-06-08 1999-12-21 Matsushita Electric Industrial Co., Ltd. Image conversion apparatus for transforming compressed image data of different resolutions wherein side information is scaled
US6026183A (en) * 1995-10-27 2000-02-15 Texas Instruments Incorporated Content-based video compression

Family Cites Families (33)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4958226A (en) 1989-09-27 1990-09-18 At&T Bell Laboratories Conditional motion compensated interpolation of digital motion video
US5465119A (en) 1991-02-22 1995-11-07 Demografx Pixel interlacing apparatus and method
EP1330129A3 (en) 1991-04-10 2006-03-15 Mitsubishi Denki Kabushiki Kaisha Encoder and Decoder
US5467136A (en) 1991-05-31 1995-11-14 Kabushiki Kaisha Toshiba Video decoder for determining a motion vector from a scaled vector and a difference vector
US5414469A (en) 1991-10-31 1995-05-09 International Business Machines Corporation Motion video compression system with multiresolution features
US5408328A (en) 1992-03-23 1995-04-18 Ricoh Corporation, California Research Center Compressed image virtual editing system
JPH05316360A (en) 1992-05-14 1993-11-26 Fuji Xerox Co Ltd Coding/decoding device for picture signal
US5270813A (en) * 1992-07-02 1993-12-14 At&T Bell Laboratories Spatially scalable video coding facilitating the derivation of variable-resolution images
KR970003102B1 (en) 1993-09-17 1997-03-14 배순훈 Half pixel motion compensation circuit for video decoder
US6026088A (en) * 1993-10-20 2000-02-15 Lsi Logic Corporation Network architecture
JP2956464B2 (en) 1993-12-29 1999-10-04 日本ビクター株式会社 Image information compression and decompression apparatus
JP3732867B2 (en) 1995-03-09 2006-01-11 株式会社ルネサステクノロジ Image expansion apparatus
US5742892A (en) * 1995-04-18 1998-04-21 Sun Microsystems, Inc. Decoder for a software-implemented end-to-end scalable video delivery system
US5612735A (en) * 1995-05-26 1997-03-18 Luncent Technologies Inc. Digital 3D/stereoscopic video compression technique utilizing two disparity estimates
US5619256A (en) * 1995-05-26 1997-04-08 Lucent Technologies Inc. Digital 3D/stereoscopic video compression technique utilizing disparity and motion compensated predictions
JPH09182083A (en) 1995-12-27 1997-07-11 Matsushita Electric Ind Co Ltd Video image encoding method and decoding method and device therefor
US5778097A (en) 1996-03-07 1998-07-07 Intel Corporation Table-driven bi-directional motion estimation using scratch area and offset valves
CA2208950A1 (en) * 1996-07-03 1998-01-03 Xuemin Chen Rate control for stereoscopic digital video encoding
JP3263807B2 (en) 1996-09-09 2002-03-11 ソニー株式会社 The image coding apparatus and image coding method
KR100235347B1 (en) 1996-09-19 1999-12-15 전주범 Method and apparatus for encoding a video signal of a contour of an object
WO1998017068A1 (en) * 1996-10-11 1998-04-23 Sarnoff Corporation Stereoscopic video coding and decoding apparatus and method
US5978509A (en) 1996-10-23 1999-11-02 Texas Instruments Incorporated Low power video decoder system with block-based motion compensation
US5886736A (en) * 1996-10-24 1999-03-23 General Instrument Corporation Synchronization of a stereoscopic video sequence
US6173013B1 (en) * 1996-11-08 2001-01-09 Sony Corporation Method and apparatus for encoding enhancement and base layer image signals using a predicted image signal
US6043846A (en) * 1996-11-15 2000-03-28 Matsushita Electric Industrial Co., Ltd. Prediction apparatus and method for improving coding efficiency in scalable video coding
US6148026A (en) 1997-01-08 2000-11-14 At&T Corp. Mesh node coding to enable object based functionalities within a motion compensated transform video coder
US6005980A (en) 1997-03-07 1999-12-21 General Instrument Corporation Motion estimation and compensation of video object planes for interlaced digital video
US6057884A (en) * 1997-06-05 2000-05-02 General Instrument Corporation Temporal and spatial scaleable coding for video object planes
US6233356B1 (en) * 1997-07-08 2001-05-15 At&T Corp. Generalized scalability for video coder based on video objects
US6993201B1 (en) * 1997-07-08 2006-01-31 At&T Corp. Generalized scalability for video coder based on video objects
US6351563B1 (en) * 1997-07-09 2002-02-26 Hyundai Electronics Ind. Co., Ltd. Apparatus and method for coding/decoding scalable shape binary image using mode of lower and current layers
US6731811B1 (en) * 1997-12-19 2004-05-04 Voicecraft, Inc. Scalable predictive coding method and apparatus
US6055623A (en) * 1998-04-07 2000-04-25 International Business Machine Corporation Specialized millicode instruction for editing functions

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6005623A (en) * 1994-06-08 1999-12-21 Matsushita Electric Industrial Co., Ltd. Image conversion apparatus for transforming compressed image data of different resolutions wherein side information is scaled
US5999189A (en) * 1995-08-04 1999-12-07 Microsoft Corporation Image compression to reduce pixel and texture memory requirements in a real-time image generator
US6026183A (en) * 1995-10-27 2000-02-15 Texas Instruments Incorporated Content-based video compression

Non-Patent Citations (11)

* Cited by examiner, † Cited by third party
Title
"International Organization fof Standardisation-Generic Coding of Moving Pictures and Associated Audio Information: Video, Recommendation H.262," ISO/IEC 13818-2 JTC1/SC29/WG11 N0702 (revised), incorporating N702 Delta of Mar. 24 and further editorial corrections May 10, 1994.
"International Organization for Standardisation-MPEG-4 Video Verification Model Version 2.1," ISO/IEC JTC1/SC29/WG11 XXXX of May 3, 1996.
"International Organization for Standardisation-Working Draft 4.0 of ISO/IEC 14496-2," ISO/IEC JTC1/SC29/WG11 N1797, MPEG97/Stockholm, Jul. 1997.
"International Organization fof Standardisation—Generic Coding of Moving Pictures and Associated Audio Information: Video, Recommendation H.262," ISO/IEC 13818-2 JTC1/SC29/WG11 N0702 (revised), incorporating N702 Delta of Mar. 24 and further editorial corrections May 10, 1994.
"International Organization for Standardisation—MPEG-4 Video Verification Model Version 2.1," ISO/IEC JTC1/SC29/WG11 XXXX of May 3, 1996.
"International Organization for Standardisation—Working Draft 4.0 of ISO/IEC 14496-2," ISO/IEC JTC1/SC29/WG11 N1797, MPEG97/Stockholm, Jul. 1997.
Index of Web Search.
Web site material, "MPEG-4 Video and Its Potential for Future Multimedia Services," http://wwwam.hhi.de/mpeg-video/papers/sikora/iscas.htm (Jan. 27, 1997).
Web site material, "MPEG-4: A Flexible Coding Standard for the Emerging Mobile Multimedia Applications", http://www.tnt.uni-hannover.de/project/eu/momusys/docs/Paper /IST/pimrc/pimrc.html (Oct. 9, 2000).
Web site material, "Object-Based Coding," http://www.ece.concordia.ca/-sorial/seminar 1/node18.html. (Jul. 14, 1999).
Web site material, "The Structure of the MPEG-4 Video Coding Algorithm", http://www.ee.iisc.ernet.in/people/students/gupta/Mpeg4/Coding/fmpeg4vm.htm (Mar. 1, 1998).

Cited By (159)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6459732B2 (en) * 1997-02-14 2002-10-01 At&T Corp. Chroma-key for efficient and low complexity shape representation of coded arbitrary video objects
US6618444B1 (en) 1997-02-14 2003-09-09 At&T Corp. Scene description nodes to support improved chroma-key shape representation of coded arbitrary images and video objects
US6535559B2 (en) * 1997-04-01 2003-03-18 Sony Corporation Image encoder, image encoding method, image decoder, image decoding method, and distribution media
US6643328B2 (en) * 1997-04-01 2003-11-04 Sony Corporation Image encoder, image encoding method, image decoder, image decoding method, and distribution media
US6337710B1 (en) * 1997-06-12 2002-01-08 Lsi Logic Corporation Graphical video editing system for regeneration of bitstreams
US9420309B2 (en) 1997-07-08 2016-08-16 At&T Intellectual Property Ii, L.P. Generalized scalability for video coder based on video objects
US8824815B2 (en) 1997-07-08 2014-09-02 At&T Intellectual Property Ii, L.P. Generalized scalability for video coder based on video objects
US20040114811A1 (en) * 1997-09-04 2004-06-17 Lim Young Kwon Computational graceful degradation method using priority information in multiple objects case
US6707944B1 (en) * 1997-09-04 2004-03-16 Electronics And Telecommunications Research Institute Computational graceful degradation method using priority information in multiple objects case
US20060222248A1 (en) * 1997-09-04 2006-10-05 Lim Young K Computational graceful degradation method using priority information in multiple objects case
US7362906B2 (en) 1997-09-04 2008-04-22 Electronics And Telecommunications Research Institute Computational graceful degradation method using priority information in multiple objects case
US7062097B2 (en) 1997-09-04 2006-06-13 Electronics And Telecommunications Research Institute Computational graceful degradation method using priority information in multiple objects case
US7876818B2 (en) 1997-10-20 2011-01-25 Mitsubishi Denki Kabushiki Kaisha Image encoding device and image decoding device
US8824561B2 (en) 1997-10-20 2014-09-02 Mitsubishi Electric Corporation Image encoding device and image decoding device
US8737463B2 (en) 1997-10-20 2014-05-27 Mitsubishi Electric Corporation Image encoding device and image decoding device
US20080049833A1 (en) * 1997-10-20 2008-02-28 Shinichi Kuroda Image encoding device and image decoding device
US20080044092A1 (en) * 1997-10-20 2008-02-21 Shinichi Kuroda Image encoding device and image decoding device
US6983014B1 (en) * 1997-10-20 2006-01-03 Mitsubishi Denki Kabushiki Kaisha Image encoding device and image decoding device
US20050232365A1 (en) * 1997-10-20 2005-10-20 Shinichi Kuroda Image encoding device and image decoding device
US7356078B2 (en) 1997-10-20 2008-04-08 Mitsubishi Denki Kabushiki Kaisha Image encoding device and image decoding device
US20050232364A1 (en) * 1997-10-20 2005-10-20 Shinichi Kuroda Image encoding device and image decoding device
US6798977B2 (en) * 1998-02-04 2004-09-28 Canon Kabushiki Kaisha Image data encoding and decoding using plural different encoding circuits
US6606329B1 (en) * 1998-07-17 2003-08-12 Koninklijke Philips Electronics N.V. Device for demultiplexing coded data
US7039107B1 (en) * 1998-09-29 2006-05-02 Siemens Aktiengesellschaft Method and array for processing a digitized image with image points
US9473770B2 (en) 1999-07-08 2016-10-18 At&T Intellectual Property Ii, L.P. Methods and apparatus for integrating external applications into an MPEG-4 scene
US8566692B2 (en) 1999-07-08 2013-10-22 At&T Intellectual Property Ii, L.P. Methods and apparatus for integrating external applications into an MPEG-4 scene
US6934906B1 (en) 1999-07-08 2005-08-23 At&T Corp. Methods and apparatus for integrating external applications into an MPEG-4 scene
US20090092190A1 (en) * 1999-07-08 2009-04-09 At&T Corp. Methods and apparatus for integrating external applications into an mpeg-4 scene
US7461330B1 (en) 1999-07-08 2008-12-02 At&T Intellectual Property Ii L.P. Methods and apparatus for integrating external applications into an MPEG-4 scene
US6798838B1 (en) * 2000-03-02 2004-09-28 Koninklijke Philips Electronics N.V. System and method for improving video transmission over a wireless network
US6493387B1 (en) * 2000-04-10 2002-12-10 Samsung Electronics Co., Ltd. Moving picture coding/decoding method and apparatus having spatially scalable architecture and signal-to-noise ratio scalable architecture together
US6553150B1 (en) 2000-04-25 2003-04-22 Hewlett-Packard Development Co., Lp Image sequence compression featuring independently coded regions
US6507618B1 (en) * 2000-04-25 2003-01-14 Hewlett-Packard Company Compressed video signal including independently coded regions
US6970510B1 (en) 2000-04-25 2005-11-29 Wee Susie J Method for downstream editing of compressed video
US6973130B1 (en) 2000-04-25 2005-12-06 Wee Susie J Compressed video signal including information for independently coded regions
US7095444B1 (en) * 2000-11-06 2006-08-22 Sri International Method and apparatus for generating, distributing and reconstructing deconstructed video
US6847683B2 (en) * 2000-11-23 2005-01-25 Koninklijke Philips Electronics N.V. Video coding method and corresponding encoder
US20020071492A1 (en) * 2000-11-23 2002-06-13 Yves Ramanzin Video decoding method and corresponding decoder
US7944966B2 (en) 2000-11-23 2011-05-17 Trident Microsystems (Far East) Ltd. Video decoding method and corresponding decoder
US20120224630A1 (en) * 2001-01-09 2012-09-06 Wen-Hsiao Peng Method and apparatus for providing prediction mode fine granularity scalability
US20020176025A1 (en) * 2001-03-05 2002-11-28 Chang-Su Kim Systems and methods for encoding redundant motion vectors in compressed video bitstreams
US7003033B2 (en) * 2001-03-05 2006-02-21 Intervideo, Inc. Systems and methods for encoding redundant motion vectors in compressed video bitstreams
US20020168086A1 (en) * 2001-04-18 2002-11-14 Takayuki Sugahara Encoding, producing and decoding methods of object data, and apparatuses for encoding, producing and decoding the object data, and programs for encoding and decoding the object data, and recording medium for the object data
US20030064258A1 (en) * 2001-09-28 2003-04-03 Pan Alfred I-Tsung Fuel additives for fuel cell
US7650058B1 (en) * 2001-11-08 2010-01-19 Cernium Corporation Object selective video recording
US20030118113A1 (en) * 2001-12-20 2003-06-26 Comer Mary Lafuze Fine-grain scalable video decoder with conditional replacement
US20030118099A1 (en) * 2001-12-20 2003-06-26 Comer Mary Lafuze Fine-grain scalable video encoder with conditional replacement
US20030142872A1 (en) * 2002-01-29 2003-07-31 Masahiko Koyanagi Image processing apparatus, decoding device, encoding device, image processing system, image processing method, and encoding method
US20050111558A1 (en) * 2002-01-30 2005-05-26 Medialive Secure device that is used to process high-quality audiovisual works
US7327936B2 (en) * 2002-01-30 2008-02-05 Medialive Secured device for the processing of high-quality audiovisual works
CN100397901C (en) 2002-01-31 2008-06-25 梅迪亚利夫公司 Device that is used for secure diffusion controlled display private copying and management of and conditional access to MPEG-4-type audiovisual content rights
FR2835386A1 (en) * 2002-01-31 2003-08-01 Daniel Lecomte Device for the distribution securisee, conditional access, the controlled viewing, private copying and management of audiovisual content rights MPEG-4 Type
WO2003065731A1 (en) * 2002-01-31 2003-08-07 Medialive Device that is used for secure diffusion, controlled display, private copying and management of, and conditional access to, mpeg-4-type audiovisual content rights
JP2005516562A (en) * 2002-01-31 2005-06-02 メディアライブMedialive Mpeg-4 type of secure broadcast of audio-visual content rights, conditional access, control display, private copy, and apparatus for management
US20050084018A1 (en) * 2002-01-31 2005-04-21 Daniel Lecomte Device that is used for secure diffusion controlled display private copying and management of and conditional access to mpeg-4-type audiovisual content rights
US7428755B2 (en) 2002-01-31 2008-09-23 Medialive Device that is used for secure diffusion controlled display private copying and management of and conditional access to MPEG-4-type audiovisual content rights
US7313282B2 (en) * 2002-03-11 2007-12-25 Winbond Electronics Corporation Method and apparatus for digital signal processing
US20030169930A1 (en) * 2002-03-11 2003-09-11 Rong-Fuh Shyu Method and apparatus for digital signal processing
US20050141768A1 (en) * 2002-03-29 2005-06-30 Koninklijka Philips Electronics N.V. Video encoding method and corresponding device and signal
WO2003084236A1 (en) * 2002-03-29 2003-10-09 Koninklijke Philips Electronics N.V. Video encoding method and corresponding device and signal
US7386049B2 (en) * 2002-05-29 2008-06-10 Innovation Management Sciences, Llc Predictive interpolation of a video signal
US20040022318A1 (en) * 2002-05-29 2004-02-05 Diego Garrido Video interpolation coding
US7397858B2 (en) 2002-05-29 2008-07-08 Innovation Management Sciences, Llc Maintaining a plurality of codebooks related to a video signal
US20070230914A1 (en) * 2002-05-29 2007-10-04 Diego Garrido Classifying image areas of a video signal
US20040017853A1 (en) * 2002-05-29 2004-01-29 Diego Garrido Maintaining a plurality of codebooks related to a video signal
US7656950B2 (en) * 2002-05-29 2010-02-02 Diego Garrido Video interpolation coding
US20040017852A1 (en) * 2002-05-29 2004-01-29 Diego Garrido Predictive interpolation of a video signal
US7715477B2 (en) 2002-05-29 2010-05-11 Diego Garrido Classifying image areas of a video signal
US8023561B1 (en) 2002-05-29 2011-09-20 Innovation Management Sciences Predictive interpolation of a video signal
US8467622B2 (en) 2002-07-18 2013-06-18 Lg Electronics Inc. Method of determining motion vectors and a reference picture index for a current block in a picture to be decoded
US20080044093A1 (en) * 2002-07-18 2008-02-21 Jeon Byeong M Apparatus for determining motion vectors and a reference picture index for a current block in a picture to be decoded
US20080044094A1 (en) * 2002-07-18 2008-02-21 Jeon Byeong M Method of determining motion vectors and a reference picture index for a current block in a picture to be decoded
US20080037886A1 (en) * 2002-07-18 2008-02-14 Jeon Byeong M Apparatus for determining motion vectors and a reference picture index for a current block in a picture to be decoded
US20080037636A1 (en) * 2002-07-18 2008-02-14 Jeon Byeong M Method of determining motion vectors and a reference picture index for a current block in a picture to be decoded
US8649622B2 (en) 2002-07-18 2014-02-11 Lg Electronics Inc. Method of determining motion vectors and a reference picture index for a current block in a picture to be decoded
US8644630B2 (en) 2002-07-18 2014-02-04 Lg Electronics Inc. Apparatus for determining motion vectors and a reference picture index for a current block in a picture to be decoded
US8649621B2 (en) 2002-07-18 2014-02-11 Lg Electronics Inc. Apparatus for determining motion vectors and a reference picture index for a current block in a picture to be decoded
US8655089B2 (en) 2002-07-18 2014-02-18 Lg Electronics Inc. Apparatus for determining motion vectors and a reference picture index for a current block in a picture to be decoded
US8644631B2 (en) 2002-07-18 2014-02-04 Lg Electronics Inc. Method of predicting a motion vector for a current block in a current picture
US8712172B2 (en) 2002-07-18 2014-04-29 Lg Electronics Inc. Method of predicting a motion vector for a current block in a current picture
US8467620B2 (en) 2002-07-18 2013-06-18 Lg Electronics Inc. Method of determining motion vectors and a reference picture index for a current block in a picture to be decoded
US20040013308A1 (en) * 2002-07-18 2004-01-22 Lg Electronics Inc. Calculation method for prediction motion vector
US8639048B2 (en) 2002-07-18 2014-01-28 Lg Electronics Inc. Method of determining motion vectors and a reference picture index for a current block in a picture to be decoded
US20060262981A1 (en) * 2002-07-18 2006-11-23 Jeon Byeong M Calculation method for prediction motion vector
US8634666B2 (en) 2002-07-18 2014-01-21 Lg Electronics Inc. Apparatus for determining motion vectors and a reference picture index for a current block in a picture to be decoded
US8634667B2 (en) 2002-07-18 2014-01-21 Lg Electronics Inc. Method of predicting a motion vector for a current block in a current picture
US8908983B2 (en) 2002-07-18 2014-12-09 Lg Electronics Inc. Method of predicting a motion vector for a current block in a current picture
US8571335B2 (en) * 2002-07-18 2013-10-29 Lg Electronics Inc. Calculation method for prediction motion vector
US8565544B2 (en) 2002-07-18 2013-10-22 Lg Electronics Inc. Apparatus for predicting a motion vector for a current block in a picture to be decoded
US8428373B2 (en) 2002-07-18 2013-04-23 Lg Electronics Inc. Apparatus for determining motion vectors and a reference picture index for a current block in a picture to be decoded
US8463058B2 (en) * 2002-07-18 2013-06-11 Lg Electronics Inc. Calculation method for prediction motion vector
US8467621B2 (en) 2002-07-18 2013-06-18 Lg Electronics Inc. Method of determining motion vectors and a reference picture index for a current block in a picture to be decoded
US8548264B2 (en) 2002-07-18 2013-10-01 Lg Electronics Inc. Apparatus for predicting a motion vector for a current block in a picture to be decoded
US8509550B2 (en) 2002-07-18 2013-08-13 Lg Electronics Inc. Apparatus for determining motion vectors and a reference picture index for a current block in a picture to be decoded
US9544589B2 (en) 2002-07-18 2017-01-10 Lg Electronics Inc. Method of predicting a motion vector for a current block in a current picture
US9560354B2 (en) 2002-07-18 2017-01-31 Lg Electronics Inc. Method of predicting a motion vector for a current block in a current picture
US9544590B2 (en) 2002-07-18 2017-01-10 Lg Electronics Inc. Method of predicing a motion vector for a current block in a current picture
US9544591B2 (en) 2002-07-18 2017-01-10 Lg Electronics Inc. Method of predicting a motion vector for a current block in a current picture
US20080037885A1 (en) * 2002-07-18 2008-02-14 Jeon Byeong M Apparatus for predicting a motion vector for a current block in a picture to be decoded
US8472738B2 (en) 2002-07-18 2013-06-25 Lg Electronics Inc. Apparatus for determining motion vectors and a reference picture index for a current block in a picture to be decoded
US8130831B2 (en) 2002-11-25 2012-03-06 Thomson Licensing Two-layer encoding for hybrid high-definition DVD
US20060062294A1 (en) * 2002-11-25 2006-03-23 Shu Lin Two-layer decoding for hybrid high-definition dvd
US7519118B2 (en) * 2002-11-25 2009-04-14 Thomson Licensing Two-layer decoding for hybrid high-definition DVD
US20060062295A1 (en) * 2002-11-25 2006-03-23 Shu Lin Two-layer encoding for hybrid high-definition dvd
US20040114183A1 (en) * 2002-12-12 2004-06-17 Wei-Jen Lee Method and apparatus for image data storage management
US7649856B2 (en) * 2002-12-20 2010-01-19 Electronics And Telecommunications Research Institute System and method for transmitting and receiving wideband speech signals
US8259629B2 (en) 2002-12-20 2012-09-04 Electronics And Telecommunications Research Institute System and method for transmitting and receiving wideband speech signals with a synthesized signal
US20040131015A1 (en) * 2002-12-20 2004-07-08 Ho-Sang Sung System and method for transmitting and receiving wideband speech signals
US20100082335A1 (en) * 2002-12-20 2010-04-01 Electronics And Telecommunications Research Institute System and method for transmitting and receiving wideband speech signals
US20040158878A1 (en) * 2003-02-07 2004-08-12 Viresh Ratnakar Power scalable digital video decoding
CN100544434C (en) 2003-05-02 2009-09-23 梅迪亚利夫公司 Process and system for securing the distribution of visual sequences
FR2854531A1 (en) * 2003-05-02 2004-11-05 Medialive Method and system for securiser scrambling, descrambling and distribution of vector visual sequences
WO2004100552A1 (en) * 2003-05-02 2004-11-18 Medialive Method and system for ensuring secure scrambling and de-scrambling and the distribution of vectoral visual sequences
US7957530B2 (en) 2003-05-02 2011-06-07 Querell Data Limited Liability Company Method and system for ensuring secure scrambling and de-scrambling and the distribution of vectoral visual sequences
US20070091997A1 (en) * 2003-05-28 2007-04-26 Chad Fogg Method And Apparatus For Scalable Video Decoder Using An Enhancement Stream
US7889792B2 (en) 2003-12-24 2011-02-15 Apple Inc. Method and system for video encoding using a variable number of B frames
US7492820B2 (en) 2004-02-06 2009-02-17 Apple Inc. Rate control for video coder employing adaptive linear regression bits modeling
US20090103610A1 (en) * 2004-02-06 2009-04-23 Apple Inc. Rate control for video coder employing adaptive linear regression bits modeling
US7986731B2 (en) 2004-02-06 2011-07-26 Apple Inc. H.264/AVC coder incorporating rate and quality controller
US7869503B2 (en) 2004-02-06 2011-01-11 Apple Inc. Rate and quality controller for H.264/AVC video coder and scene analyzer therefor
US20050175093A1 (en) * 2004-02-06 2005-08-11 Haskell Barin G. Target bitrate estimator, picture activity and buffer management in rate control for video coder
US8036267B2 (en) 2004-02-06 2011-10-11 Apple, Inc. Rate control for video coder employing adaptive linear regression bits modeling
US20050175092A1 (en) * 2004-02-06 2005-08-11 Atul Puri H.264/AVC coder incorporating rate and quality controller
US20050180502A1 (en) * 2004-02-06 2005-08-18 Atul Puri Rate control for video coder employing adaptive linear regression bits modeling
US7453938B2 (en) 2004-02-06 2008-11-18 Apple Inc. Target bitrate estimator, picture activity and buffer management in rate control for video coder
US8270598B2 (en) * 2004-03-16 2012-09-18 Querell Data Limited Liability Company Highly secured method and device for distributing audio-visual streams
US20070165842A1 (en) * 2004-03-16 2007-07-19 Medialive, A Corporation Of France Process and highly secure system for the distribution of audiovisual streams
US20070180481A1 (en) * 2004-03-23 2007-08-02 Hee-Kyung Lee Enhanced model of relation with quantitative representation, and tv anytime service method and system employing the same
US20160330473A1 (en) * 2004-08-16 2016-11-10 Nippon Telegraph And Telephone Corporation Picture encoding method, picture decoding method, picture encoding apparatus, picture decoding apparatus, picture encoding program, and picture decoding program
US9402087B2 (en) * 2004-08-16 2016-07-26 Nippon Telegraph And Telephone Corporation Picture encoding method, picture decoding method, picture encoding apparatus, picture decoding apparatus, picture encoding program, and picture decoding program
US20070183499A1 (en) * 2004-08-16 2007-08-09 Nippon Telegraph And Telephone Corporation Picture encoding method, picture decoding method, picture encoding apparatus, picture decoding apparatus, picture encoding program, and picture decoding program
US20060120459A1 (en) * 2004-11-29 2006-06-08 Park Seung W Method for coding vector refinement information required to use motion vectors in base layer pictures when encoding video signal and method for decoding video data using such coded vector refinement information
US20060133499A1 (en) * 2004-11-29 2006-06-22 Park Seung W Method and apparatus for encoding video signal using previous picture already converted into H picture as reference picture of current picture and method and apparatus for decoding such encoded video signal
US20060120454A1 (en) * 2004-11-29 2006-06-08 Park Seung W Method and apparatus for encoding/decoding video signal using motion vectors of pictures in base layer
US20060159176A1 (en) * 2004-12-16 2006-07-20 Park Seung W Method and apparatus for deriving motion vectors of macroblocks from motion vectors of pictures of base layer when encoding/decoding video signal
US20080214176A1 (en) * 2005-01-11 2008-09-04 Peter Amon Methods and Devices for the Transmission of Scalable Data
US8948739B2 (en) * 2005-01-11 2015-02-03 Siemens Aktiengesellschaft Methods and devices for the transmission of scalable data
US8804849B2 (en) * 2005-03-16 2014-08-12 Apple Inc. Multithread processing of video frames
US20120230424A1 (en) * 2005-03-16 2012-09-13 Duvivier Christian L Multithread processing of video frames
US8587655B2 (en) 2005-07-22 2013-11-19 Checkvideo Llc Directed attention digital video recordation
US8026945B2 (en) 2005-07-22 2011-09-27 Cernium Corporation Directed attention digital video recordation
US20110194026A1 (en) * 2005-08-08 2011-08-11 Koninklijke Philips Electronics, N.V. Method and system for video copyright protection
US7609271B2 (en) * 2006-06-30 2009-10-27 Microsoft Corporation Producing animated scenes from still images
US20080001950A1 (en) * 2006-06-30 2008-01-03 Microsoft Corporation Producing animated scenes from still images
EP2055106B1 (en) * 2006-08-16 2015-06-17 Microsoft Technology Licensing, LLC Techniques for variable resolution encoding and decoding of digital video
US20080063085A1 (en) * 2006-09-11 2008-03-13 Apple Computer, Inc. Post-processing for decoder complexity scalability
US20110234430A1 (en) * 2006-09-11 2011-09-29 Apple Inc. Complexity-aware encoding
US8731064B2 (en) 2006-09-11 2014-05-20 Apple Inc. Post-processing for decoder complexity scalability
US20080062018A1 (en) * 2006-09-11 2008-03-13 Apple Computer, Inc. Complexity-aware encoding
US7456760B2 (en) 2006-09-11 2008-11-25 Apple Inc. Complexity-aware encoding
US20090073005A1 (en) * 2006-09-11 2009-03-19 Apple Computer, Inc. Complexity-aware encoding
US8830092B2 (en) 2006-09-11 2014-09-09 Apple Inc. Complexity-aware encoding
US7969333B2 (en) 2006-09-11 2011-06-28 Apple Inc. Complexity-aware encoding
US20080095238A1 (en) * 2006-10-18 2008-04-24 Apple Inc. Scalable video coding with filtering of lower layers
US20090304086A1 (en) * 2008-06-06 2009-12-10 Apple Inc. Method and system for video coder and decoder joint optimization
US9215467B2 (en) 2008-11-17 2015-12-15 Checkvideo Llc Analytics-modulated coding of surveillance video
US8976856B2 (en) 2010-09-30 2015-03-10 Apple Inc. Optimized deblocking filters
US9961345B2 (en) * 2012-05-14 2018-05-01 V-Nova International Limited Encoding and reconstruction of residual data based on support information
US20130322530A1 (en) * 2012-05-14 2013-12-05 Luca Rossato Encoding and reconstruction of residual data based on support information

Also Published As

Publication number Publication date Type
US20130170563A1 (en) 2013-07-04 application
US6707949B2 (en) 2004-03-16 grant
US20030165274A1 (en) 2003-09-04 application
US20140362907A1 (en) 2014-12-11 application
US8385667B2 (en) 2013-02-26 grant
US9420309B2 (en) 2016-08-16 grant
US8824815B2 (en) 2014-09-02 grant
US6526177B1 (en) 2003-02-25 grant
US7457474B1 (en) 2008-11-25 grant
US20090041123A1 (en) 2009-02-12 application

Similar Documents

Publication Publication Date Title
Vetro et al. Overview of the stereo and multiview video coding extensions of the H. 264/MPEG-4 AVC standard
Moccagatta et al. Error-resilient coding in JPEG-2000 and MPEG-4
US5278647A (en) Video decoder using adaptive macroblock leak signals
US5633683A (en) Arrangement and method for transmitting and receiving mosaic video signals including sub-pictures for easy selection of a program to be viewed
US6493385B1 (en) Image encoding method, image encoder, image decoding method, and image decoder
US5822460A (en) Method and apparatus for generating chrominance shape information of a video object plane in a video signal
US5784572A (en) Method and apparatus for compressing video and voice signals according to different standards
US6825885B2 (en) Motion information coding and decoding method
US6243418B1 (en) Method and apparatus for encoding a motion vector of a binary shape signal
US20070064800A1 (en) Method of estimating disparity vector, and method and apparatus for encoding and decoding multi-view moving picture using the disparity vector estimation method
US6647061B1 (en) Video size conversion and transcoding from MPEG-2 to MPEG-4
US6081295A (en) Method and apparatus for transcoding bit streams with video data
US20030215011A1 (en) Method and apparatus for transcoding compressed video bitstreams
US20070025444A1 (en) Coding Method
US6459812B2 (en) Picture encoding and/or decoding apparatus and method for providing scalability of a video object whose position changes with time and a recording medium having the same recorded thereon
US7627034B2 (en) Method for scalably encoding and decoding video signal
US20070162922A1 (en) Apparatus and method for processing video data using gaze detection
US20080219351A1 (en) Apparatus of Predictive Coding/Decoding Using View-Temporal Reference Picture Buffers and Method Using the Same
US6005980A (en) Motion estimation and compensation of video object planes for interlaced digital video
US20090060034A1 (en) Method and apparatus for scalably encoding and decoding video signal
US20100098157A1 (en) method and an apparatus for processing a video signal
US5974184A (en) Intra-macroblock DC and AC coefficient prediction for interlaced digital video
US5612735A (en) Digital 3D/stereoscopic video compression technique utilizing two disparity estimates
US7421127B2 (en) Spatial scalable compression scheme using spatial sharpness enhancement techniques
US5652616A (en) Optimal disparity estimation for stereoscopic video coding

Legal Events

Date Code Title Description
AS Assignment

Owner name: AT&T CORP, NEW YORK

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:HASKELL, BARIN GEOFFRY;PURI, ATUL;SCHMIDT, ROBERT LEWIS;REEL/FRAME:009498/0008

Effective date: 19980731

FPAY Fee payment

Year of fee payment: 4

FPAY Fee payment

Year of fee payment: 8

AS Assignment

Owner name: AT&T PROPERTIES, LLC,NEVADA

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:AT&T CORP.;REEL/FRAME:024168/0722

Effective date: 20100225

Owner name: AT&T PROPERTIES, LLC, NEVADA

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:AT&T CORP.;REEL/FRAME:024168/0722

Effective date: 20100225

AS Assignment

Owner name: AT&T INTELLECTUAL PROPERTY II, L.P.,NEVADA

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:AT&T PROPERTIES, LLC;REEL/FRAME:024176/0709

Effective date: 20100225

Owner name: AT&T INTELLECTUAL PROPERTY II, L.P., NEVADA

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:AT&T PROPERTIES, LLC;REEL/FRAME:024176/0709

Effective date: 20100225

FPAY Fee payment

Year of fee payment: 12