WO2000064148A1 - Method and apparatus for efficient video processing - Google Patents

Method and apparatus for efficient video processing Download PDF

Info

Publication number
WO2000064148A1
WO2000064148A1 PCT/US2000/010451 US0010451W WO0064148A1 WO 2000064148 A1 WO2000064148 A1 WO 2000064148A1 US 0010451 W US0010451 W US 0010451W WO 0064148 A1 WO0064148 A1 WO 0064148A1
Authority
WO
WIPO (PCT)
Prior art keywords
encoder
decoder
information
frame
segment
Prior art date
Application number
PCT/US2000/010451
Other languages
English (en)
French (fr)
Other versions
WO2000064148A9 (en
Inventor
Adityo Prakash
Eniko F. Prakash
Original Assignee
Pulsent Corporation
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Pulsent Corporation filed Critical Pulsent Corporation
Priority to AU44685/00A priority Critical patent/AU4468500A/en
Publication of WO2000064148A1 publication Critical patent/WO2000064148A1/en
Publication of WO2000064148A9 publication Critical patent/WO2000064148A9/en

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00Television systems
    • H04N7/12Systems in which the television signal is transmitted via one channel or a plurality of parallel channels, the bandwidth of each channel being less than the bandwidth of the television signal
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/59Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving spatial sub-sampling or interpolation, e.g. alteration of picture size or resolution
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/103Selection of coding mode or of prediction mode
    • H04N19/107Selection of coding mode or of prediction mode between spatial and temporal predictive coding, e.g. picture refresh
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/12Selection from among a plurality of transforms or standards, e.g. selection between discrete cosine transform [DCT] and sub-band transform or selection between H.263 and H.264
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/189Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the adaptation method, adaptation tool or adaptation type used for the adaptive coding
    • H04N19/196Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the adaptation method, adaptation tool or adaptation type used for the adaptive coding being specially adapted for the computation of encoding parameters, e.g. by averaging previously computed encoding parameters
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/46Embedding additional information in the video signal during the compression process
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/503Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
    • H04N19/507Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction using conditional replenishment
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/503Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
    • H04N19/51Motion estimation or motion compensation
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/503Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
    • H04N19/51Motion estimation or motion compensation
    • H04N19/513Processing of motion vectors
    • H04N19/517Processing of motion vectors by encoding
    • H04N19/52Processing of motion vectors by encoding by predictive encoding
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/60Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
    • H04N19/61Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding in combination with predictive coding
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/80Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation

Definitions

  • the present invention relates to the compression of motion video data, and more
  • Compression is the means by which digital motion video
  • the first type of video compression focuses on the reduction of spatial redundancy.
  • Spatial redundancy refers to taking advantage of the correlation among neighboring pixels in order to derive a more efficient representation of the important information in an image frame.
  • These methods are more appropriately termed still image compression routines, as they do not attempt to address the issue of temporal, or frame to frame, redundancy, as explained in section 2.2. They work reasonably well on individual video image frames.
  • a critical element in video compression is reducing
  • One of the first commonly used methods of image compression was the DCT, or
  • DCT operates by representing each digital image frame as a series of cosine waves or frequencies. Afterwards, the coefficients of the cosine series are quantized. The higher frequency coefficients are quantized more harshly than those of the lower
  • the wavelet transformation compression scheme was devised. This system is similar to the DCT. The only substantial difference is that the image frame is represented as a series of wavelets, or windowed oscillations, instead of as a series of cosine waves.
  • the goal of fractal compression is to take an image and determine the single
  • a fractal is an object that is self-similar at different scales, or resolutions, i.e. no matter what resolution you look at, the object remains the same. Theoretically, fantastic compression ratios could occur as simple equations describe complex images.
  • Fractal compression is not a viable method of general compression. The high
  • Block Based Motion Estimation Block Matching is the process by which a block ' of the image is subdivided into uniform size blocks and each block is tracked from one frame to another and represented by a motion vector instead of having the block re-coded and placed into the bitstream for a second time. Examples of compression routines that use block matching include MPEG, and all its variants.
  • MPEG operates by performing a still image compression on the first frame and transmitting it. It then divides the same frame into 16 pixel by 16 pixel square blocks and attempts to find each block within the next frame. For each block that still exists in the subsequent frame, MPEG needs only transmit the motion vector, or movement, of the
  • residue frame which is coded using JPEG and sent to the receiver to complete the image frame.
  • the encoder divides the second image frame into blocks and the routine
  • a keyframe is an image frame which is
  • block method is inherently crude, as the blocks do not have any relationship with real objects in the image.
  • a given block may comprise a part of an object, a whole object, or even multiple dissimilar objects with unrelated motion.
  • neighboring objects will have similar motion.
  • blocks do not correspond to real objects, block based systems cannot use this information to further reduce the bitstream
  • This invention represents a novel approach to the problem of video compression.
  • the goal of video compression is to represent accurately a sequence
  • the smart decoder also makes the same predictions about the subsequent images in the related sequence of images as the encoder.
  • the encoder can simply send the difference between the prediction and the actual values, thus also reducing the bitstream,
  • Compression of digital motion video is the process by which superfluous or redundant information, both spatial and temporal, contained within a sequence of related video frames (frames) is removed.
  • Video compression allows the sequence of frames to
  • This invention is novel in that it. uses a smart decoder to take much of the
  • absent from the bitstream is the information regarding the structural information inherent within the image frame, such as geometry, color, and brightness, which, in a complex frame is a significant amount of video information. Further, absent from the bitstream is information regarding any decision made by the encoder such as segment ordering, segment association and disassociation, etc.
  • Fig. 1 is an overview drawing of the encoder for use with a compatible decoder as will be described later with respect to Fig. 2.
  • the encoder works as follows:
  • the encoder obtains a reference image frame
  • the encoder encodes the image frame from step 1;
  • step 3 The encoded image from step 2 is reconstructed by the encoder, in the same manner as the decoder will;
  • the encoder segments the reconstructed image from step 4; Alternatively, the encoder segments the original reference image frame from step 1;
  • step 4 The segments determined in step 4 are ordered by the encoder, in the same manner as the decoder will;
  • the encoder obtains a new image frame
  • step 6 is determined by motion matching
  • the encoder encodes the kinetic information
  • the encoder attempts to fill each of the background residues from step 9 and 10.
  • the encoder determines the difference between the predicted fill and the actual fill for each of the background residue areas.
  • the encoder determines the local residue areas in the second image frame, from the segment motion information
  • the encoder orders the local residues from step 13, in the same manner as the, decoder will;
  • the encoder encodes the local residues from step 13.
  • the encoder determines any special instructions associated with the segment information
  • the encoder transmits the following information, and reconstructs the second frame, and continues at step 6:
  • Fig 2 is an overview drawing of the decoder system with a compatible encoder as described in Fig 1.. The decoder system works as follows:
  • the decoder receives a first encoded image frame from step 3 of the encoder description
  • step 2 The encoded image frame from step 1 is reconstructed by the decoder in the same manner as the encoder;
  • the reconstructed image frame from step 2 is segmented by the decoder.
  • the reconstructed image frame is not segmented by the decoder
  • the decoder receives a flag from the encoder stating whether the second frame from step 19 and 20 of the encoder description is a keyframe, i.e. not represented in relation to any other frame. If so, then the decoder returns to step 1.
  • the decoder receives motion information regarding the segments determined
  • step 3 from the encoder
  • the decoder begins to reconstruct a subsequent image frame using the
  • the decoder determines where areas, previously hidden, are now revealed, also known as the background residue
  • the decoder attempts to fill the background residue locations from step 6;
  • the decoder receives additional background residue information plus flags denoting the coding method for the additional background residue information from step 8 from the encoder;
  • the decoder decodes the additional background residue information
  • the computed background residue information and the added background residue information is added to the second image frame.
  • the decoder determines the location of the local segment, residues.
  • the decoder receives coded local segment residue information plus flags denoting the coding method for each local segment residue location;
  • the decoder decodes the local segment residue information
  • the decoded local segment residue information is added to the second frame.
  • the encoder receives the special instructions, if any, for each segments
  • Fig 3 is an overview drawing of the encoder/smart decoder system.
  • the encoder obtains, encodes and transmits the reference frame
  • decoder 3. Identical segments in the reference frame ai'e determined by both encoder and decoder;
  • the encoder obtains a new image frame
  • the encoder determines the motion of segments from step 3 by means of motion matching frame from step 5;
  • the encoder encodes motion information
  • the encoder determines previously, hidden areas, also known as background residue, which is now exposed in the second frame.
  • the encoder attempts to mathematical predict the image at the background residue regions.
  • the encoder determines if the mathematical prediction was good based upon the difference between the guess and the prediction. The encoder computes additional background residue if necessary.
  • the encoder determines structural information for the local segment residues
  • the encoder Based on the structural information from step 12, regarding the local residues, the encoder encodes the local segment residues. 14. The encoder determines if based upon the kinetic information of the segments, if the second frame should be coded in reference to the first frame. If not, it is coded as a keyframe and the routine begins at step 1.
  • the decoder receives the segment kinetic information from the encoder in step 7.
  • the decoder determines and orders the same background residue at the encoder did in step 8.
  • the decoder determines and orders the same local segment residues as determines in step 11 and 12.
  • the decoder receives the local segment residues information from the encoder and flags denoting the coding scheme.
  • the encoder receives the special information, if any, regarding each segment.
  • the encoder receives the reference frame, in this case, a picture of an automobile moving left to right with a mountain in the background.
  • the reference frame generally refers to the frame which any other frame is described in relation to.
  • Fig. 5 is the part of the flow diagram illustrating the procedure by which the encoder initially processes the reference frame.
  • Step 110 begins the process, specifically, the encoder receives the picture described in Fig.4.
  • the encoder encodes Fig 4, into a video format, and transmits it to the receptor at step 130.
  • the encoder reconstructs the encoded frame at step 140.
  • Segmentation is the process by which a digital image is subdivided into its component parts, i.e. segments, where each segment represents an area bounded by a radical or sharp change in values within the image.
  • segmentation can be done in a plurality of ways.
  • One such way is the watershed method where each pixel is connected to every other pixel in the image frame.
  • the watershed method segments the image by disconnecting pixels based upon a variety of algorithms. The remaining connected pixels belong to the same segment.
  • the encoder segments the reconstructed reference frame to determine the inherent structural features of the image.
  • the encoder segments the original image frame for the same purpose.
  • the encoder determines that the segments of Fig. 2 are the car, the wheels, the windows, the street, the sun, and the background.
  • the encoder orders the segments based upon a predetermined criteria and marks them Segments 1 through 8, respectively, as seen in Fig 7. Segmentation permits the encoder to perform efficient motion matching, motion prediction, and efficient residue coding as explained further in this description. 4.
  • the encoder encodes the kinetic or motion information regarding the movement of each segment.
  • the kinetic information is determined through a process known as motion matching.
  • Motion matching is the procedure of matching similar regions, often segments, from the first frame to the second frame. At each pixel within a digital image frame, an image is represented by numerical value. Matching occurs when a region in the first frame has identical or near identical pixel values with a region in the second frame.
  • a segment is matched with a segment in another frame when the absolute value of the difference in pixel values between the segments is below a predetermined threshold. While the absolute value of the pixel difference is often used to because it is simple and accounts for negative numbers any number of function would suffice.
  • frame 1 we have a soccer ball, with black and white squares.
  • frame 2 we have a soccer ball, with black and white squares.
  • the kinetic info ⁇ nation transmitted to the decoder can be reduced if related segments can be considered as single groups so that the encoder only needs to transmit one main representative motion vector to the decoder along with motion vector offsets to represent the individual motion of each segment within the group. Grouping is possible if there is previous kinetic information about the segments or if there is multi-scale information about the segments. Multi-scaling will be explained in section 4.2 of the encoder discussion.
  • the encoder determines if the first frame is a keyframe, i.e. not described in relation to other frames. If the first frame is a keyframe,
  • step 320 will execute the motion grouping routine, described here as section 4.1.
  • step 310 goes to step 330, where
  • step 340 executes the Multi-scaling routine in section 4.2, otherwise at step 350, the
  • the encoder cannot group the segments and then, at step 350, encoder determines that it cannot group any segments together.
  • Motion vector grouping begins at step 510 in Fig 10, where the previous motion vector of each segment is
  • the motion vector for the group is determined by combining the
  • motion vector difference i.e. the difference between the segment's motion vector and the
  • the encoder orders the groups. However, before the motion
  • step 555 described here in section 4.1.1.
  • the encoder considers a segment.
  • the encoder considers a segment.
  • the encoder determines if there is previous motion information for the segment so that its
  • step 640 The motion vector offset is initially predicted at step 650 as a function of the
  • the encoder makes the final calculation for the motion vector offset.
  • motion vector calculation could be the difference between the initial motion vector and
  • the encoder determines if there are any more segments, if so, then at
  • step 680 the encoder considers the next segment and continues at step 620. Otherwise
  • Multi-scaling grouping is an alternative to grouping segments by previous motion.
  • multi-scaling in may used in conjunction with motion grouping.
  • the stitching on a football may
  • the encoder considers the coarsest image scale (i.e. lowest resolution) for the frame
  • step 420 determines which segments have remained visible.
  • step 430 invisible segments which are wholly contained within a given visible segment
  • step 450 the encoder considers the next segment and continues at
  • step 430 Otherwise the Multi-scaling grouping process ceases.
  • the residue is the portion of the image left over after the structural information has been moved. Residue falls under two classifications; new
  • the encoder determines where the previously obstructed image regions occur.
  • the encoder orders the region using a predetermined ordering system. Using the information surrounding the regions, the encoder makes a mathematical guess as to the structure of the regions. Yet, the encoder also knows precisely what images were
  • Step 740 the encoder considers a region
  • the encoder will encode the
  • the encoder stores a flag
  • step 750 the encoder determines if there are ' any more newly unobstructed regions. If so the next region is considered and the routine continues at step 730, else it ceases at step 799.
  • the local residue is the portion of the image in the neighborhood of a segment, left over after the segments have .been moved, i.e. the car and mountain appear smaller in the subsequent frame.
  • the structure of the residue will depend on how different the new segments are from the previous segments. It may be a well-defined region, or set of regions, or it may be patchy. Different types of coding methods are ideal for different types of local residue. Since the decoder knows the segment motion, it knows where most of the local residues will be located.
  • the encoder determines the locations of the local residues and orders the regions where the local residues occurs using a pre-determined ordering scheme at section 820.
  • the encoder considers the first local residue, and makes a decision as the most efficient method of coding it and encodes it at step 840.
  • the encoder stores a flag denoting the coding mechanism as well as the coded residue at step 810.
  • step 850 If there are more local residue locations, step 860 will consider the next local
  • step 880 the keyframe routine at Fig 15a, step 880.
  • step 880 the encoder determines if the second frame should be coded as a keyframe. If yes, then step 885, the encoder discards the kinetic information, the background residue, and the local segment residues and continues at step
  • routine transmits the kinetic information, the background residue,
  • the encoder transmits embedded commands and instructions regarding each segment into the bitstream as necessary. Examples of these commands include, but are not limited to, getting static web pages, obtaining another video bitstream, waiting for text, etc.
  • the encoder can embed these commands at any point within the bitstream subsequent to the decoder ordering the segments.
  • Fig 14a is an example of one point where the commands are be embedded within the data stream.
  • the encoder At step 1610, the encoder considers the first segment. At step 1620, it transmits a special instruction. At step 1630, the encoder determines if there are any special instructions for the segment. If yes, then at step 1640, the instructions are
  • step 1650 the encoder determines if there are any more segments. If there are no special instructions associated with the segment, the encoder proceeds directly to step 1650. If there are more segments, at step 1660, the encoder considers the next segments are continues to step 1620, otherwise the routine ends at step 1699.
  • the decoder receives the encoded reference frame of a picture of an automobile moving left to right with a mountain in the background ( See Fig. 4).
  • the reference frame generally refers to the frame which other, subsequent frames are described in relation to.
  • Fig. 16 illustrates the flow diagram of the above process.
  • Step 910 begins the process where the decoder receives an encoded image frame.
  • the decoder reconstructs the encoded image frame.
  • the decoder receives a keyframe flag. This flag denotes whether the second frame is a keyframe or can it be reconstructed from the kinetic and residue information. If the second frame is a keyframe, then the decoder returns to step 910,
  • segmentation is the process by which a digital image is subdivided into its components parts, i.e. segments, where each segment represents ah area bounded by a radical or sharp change in values within the image.
  • the decoder segments the reconstructed reference frame to determine the inherent structural features of the image.
  • the decoder' determines that the segments in Fig. 4 are the car, the wheels, the doors, the windows, the street, the mountain and the background.
  • the decoder will order the
  • the decoder receives a keyframe flag from the encoder. This flag tells the encoder if the first frame is a keyframe.
  • the decoder receives the kinetic information regarding the movement of each segment.
  • the kinetic information tells the decoder the position of the segment in the new frame relative to its position in the previous frame.
  • the kinetic information is reduced if the segments with related motion can be grouped together and represented by one motion vector.
  • the kinetic information received by the decoder depends oh several factors: to wit; 1) the reference frame is a key frame, and 2) if not, is multi-scaling information available.
  • the decoder determines if the reference frame is
  • a keyframe i.e. a frame not defined in relation to any other frame. If so, then there is no
  • the decoder determines if there is multi-scale information available. If the first frame is
  • step 1 150 the motion vectors are
  • step 11 10 if the decoder determines that the first frame is not the
  • section 4.2 it may use the multi-scale grouping described in step 4.1
  • Multi-scale grouping only occurs when the first frame is a keyframe and there is
  • the decoder considers the coarsest image scale
  • step 1220 determine which segments have remained visible.
  • step 1240 If there are more
  • step 1260 the decoder considers the next segment and continues at
  • the encoder determines if there are any more segments, if so, then
  • the encoder considers the next segment and continues at step 1320.
  • the residue is the portion of the image left over after the structural information
  • Residue falls under two classifications; background and local residues.
  • the decoder considers the background residue regions and orders the regions at step 1420.
  • it makes a mathematical prediction on the structure of the first background residue location.
  • the decoder receives a flag denoting how good the prediction was and if correction is needed.
  • Step 1450 makes a decision, if the prediction is sufficient, the routine continues at step 1470, otherwise at step 1460, receives the encoded region and the flag denoting the coding scheme and reconstructs as necessary. If there are more background residue locations, at step 1470, the decoder, at step 1480, considers the next region and continues at step 1430. Otherwise the decoder goes to step 1490 where reconstruction continues and the process ceases.
  • the local segment residue is the portion of the image, in the neighborhood of the segment, left over after the segment has been moved, i.e. the car and the mountain appear smaller in the subsequent frame. Also, as explain before, the structure of the local residue may be varied. The decoder knows that most of the local residues will appear around the segments.
  • the decoder considers the first segment.
  • the decoder receives a flag denoting the coding method and receives the encoded local residue for that segment.
  • Step 1530 determines if there are any more segments and if not end at 1590 where reconstruction concludes. Otherwise at step 1540 the decoder considers the next segment and continues at step 1520. The routine ends at step 1599. 6. Special instructions
  • the decoder is capable of receiving and executing commands embedded within the bitstream and associated with the various segments.
  • the encoder and decoder are synchronized and are working with the same reference frame, the encoder is not required to transmit the structural information associated with the commands.
  • the embedded commands are held in abeyance until a user-driven event, i.e. a mouseclick, occurs.
  • Fig 24 is an example of one potential way to embed the commands.
  • the decoder considers the first segment, at step 1720 it received a special instruction flag.
  • the decoder determines, at step 1730, if there are special instructions or commands associated with the segment. If so, the decoder receives the commands at step 1740.
  • the decoder determine if there are any more segments. If there were no special instructions or commands, the decoder goes to step 1750 directly.
  • the decoder at step 1760, considers the next segment and continues at step 1720, otherwise the routine ends at step 1799.
  • the decoder determines if the user-driven event has occurred. If it has, the decoder determines which segment the user-driven event refers to at step 1820. At step 1830, the associated command is executed. The decoder proceeds to step 1840. If the user-driven event has not occurred, the routine proceeds directly to step 1840. At step 1840, if the termination command has been sent, the routine exits at step 1899, otherwise the routine continues at step 1810. 6.
  • the second frame is reconstructed into a video format based upon the kinetic motion of the segments, and local segment residues and the background residues.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Computing Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Discrete Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)
PCT/US2000/010451 1999-04-17 2000-04-17 Method and apparatus for efficient video processing WO2000064148A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
AU44685/00A AU4468500A (en) 1999-04-17 2000-04-17 Method and apparatus for efficient video processing

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US12985499P 1999-04-17 1999-04-17
US60/129,854 1999-04-17

Publications (2)

Publication Number Publication Date
WO2000064148A1 true WO2000064148A1 (en) 2000-10-26
WO2000064148A9 WO2000064148A9 (en) 2002-05-02

Family

ID=22441924

Family Applications (2)

Application Number Title Priority Date Filing Date
PCT/US2000/010451 WO2000064148A1 (en) 1999-04-17 2000-04-17 Method and apparatus for efficient video processing
PCT/US2000/010439 WO2000064167A1 (en) 1999-04-17 2000-04-17 Method and apparatus for efficient video processing

Family Applications After (1)

Application Number Title Priority Date Filing Date
PCT/US2000/010439 WO2000064167A1 (en) 1999-04-17 2000-04-17 Method and apparatus for efficient video processing

Country Status (6)

Country Link
EP (1) EP1180308A4 (ja)
JP (2) JP4943586B2 (ja)
KR (1) KR20020047031A (ja)
AU (2) AU4468500A (ja)
IL (1) IL145956A0 (ja)
WO (2) WO2000064148A1 (ja)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2002051158A2 (en) * 2000-12-20 2002-06-27 Pulsent Corporation Method of filling exposed areas in digital images
US7305033B2 (en) 2002-10-16 2007-12-04 Telecommunications Advancement Organization Of Japan Method of encoding and decoding motion picture, motion picture encoding device and motion picture decoding device

Families Citing this family (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
TW589870B (en) 2000-12-19 2004-06-01 Pts Corp Adaptive transforms
AU2002235252A1 (en) 2000-12-19 2002-07-01 Pulsent Corporation Deblurring and re-blurring image segments
TW573433B (en) 2000-12-20 2004-01-21 Pulsent Corp Efficiently adaptive double pyramidal coding
US6690823B2 (en) 2002-01-31 2004-02-10 Pts Corporation Method and apparatus for partitioning an arbitrarily-shaped area
US6909749B2 (en) 2002-07-15 2005-06-21 Pts Corporation Hierarchical segment-based motion vector encoding and decoding
US7639741B1 (en) 2002-12-06 2009-12-29 Altera Corporation Temporal filtering using object motion estimation
JP4613176B2 (ja) * 2004-01-01 2011-01-12 ディーエスエム アイピー アセッツ ビー.ブイ. 高性能ポリエチレン・マルチフィラメント糸の製造方法
KR20060088461A (ko) * 2005-02-01 2006-08-04 엘지전자 주식회사 영상신호의 엔코딩/디코딩시에 영상블록을 위한 모션벡터를베이스 레이어 픽처의 모션벡터로부터 유도하는 방법 및장치
FR2917872A1 (fr) * 2007-06-25 2008-12-26 France Telecom Procedes et dispositifs de codage et de decodage d'une sequence d'images representee a l'aide de tubes de mouvement, produits programme d'ordinateur et signal correspondant.

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6026182A (en) * 1995-10-05 2000-02-15 Microsoft Corporation Feature segmentation
US6057884A (en) * 1997-06-05 2000-05-02 General Instrument Corporation Temporal and spatial scaleable coding for video object planes

Family Cites Families (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH06197334A (ja) * 1992-07-03 1994-07-15 Sony Corp 画像信号符号化方法、画像信号復号化方法、画像信号符号化装置、画像信号復号化装置及び画像信号記録媒体
TW301098B (ja) * 1993-03-31 1997-03-21 Sony Co Ltd
US5703646A (en) * 1993-04-09 1997-12-30 Sony Corporation Picture encoding method, picture encoding apparatus and picture recording medium
DE69416717T2 (de) * 1993-05-21 1999-10-07 Nippon Telegraph & Telephone Bewegtbildkoder und -dekoder
JPH0746595A (ja) * 1993-05-21 1995-02-14 Nippon Telegr & Teleph Corp <Ntt> 動画像符号化器および復号化器
US5654760A (en) * 1994-03-30 1997-08-05 Sony Corporation Selection of quantization step size in accordance with predicted quantization noise
EP0720373A1 (en) * 1994-12-30 1996-07-03 Daewoo Electronics Co., Ltd Method and apparatus for encoding a video signal using region-based motion vectors
JP4223571B2 (ja) * 1995-05-02 2009-02-12 ソニー株式会社 画像符号化方法及び装置
DE69625626T2 (de) * 1995-05-08 2003-10-16 Digimarc Corp Verbindungsherstellung zwischen Computern beruhend auf der Dekodierung einer steganographisch in einem Audioobjekt eingebetteten Adresse
US5812787A (en) * 1995-06-30 1998-09-22 Intel Corporation Video coding scheme with foreground/background separation
DE69615812T2 (de) * 1995-08-02 2002-06-20 Koninkl Philips Electronics Nv Verfahren und system zur kodierung einer bildsequenz
JP2000515692A (ja) * 1995-12-12 2000-11-21 ザ ボード オブ トラスティーズ オブ ザ ユニバーシティー オブ イリノイ 性質限定システム上でリアルタイムの動画及び音声情報を伝送し読み出すための方法及び装置
JPH09161071A (ja) * 1995-12-12 1997-06-20 Sony Corp 領域対応付け装置および領域対応付け方法

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6026182A (en) * 1995-10-05 2000-02-15 Microsoft Corporation Feature segmentation
US6057884A (en) * 1997-06-05 2000-05-02 General Instrument Corporation Temporal and spatial scaleable coding for video object planes

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2002051158A2 (en) * 2000-12-20 2002-06-27 Pulsent Corporation Method of filling exposed areas in digital images
WO2002051158A3 (en) * 2000-12-20 2003-04-24 Pulsent Corp Method of filling exposed areas in digital images
US7133566B2 (en) 2000-12-20 2006-11-07 Altera Corporation Method of filling exposed areas in digital images
US7305033B2 (en) 2002-10-16 2007-12-04 Telecommunications Advancement Organization Of Japan Method of encoding and decoding motion picture, motion picture encoding device and motion picture decoding device

Also Published As

Publication number Publication date
EP1180308A4 (en) 2009-12-23
WO2000064167A1 (en) 2000-10-26
JP2011142663A (ja) 2011-07-21
WO2000064148A9 (en) 2002-05-02
JP2003524314A (ja) 2003-08-12
JP4943586B2 (ja) 2012-05-30
AU4468500A (en) 2000-11-02
KR20020047031A (ko) 2002-06-21
AU4468000A (en) 2000-11-02
EP1180308A1 (en) 2002-02-20
JP5130381B2 (ja) 2013-01-30
IL145956A0 (en) 2002-07-25

Similar Documents

Publication Publication Date Title
US6600786B1 (en) Method and apparatus for efficient video processing
US5594504A (en) Predictive video coding using a motion vector updating routine
JP5130381B2 (ja) 効率的な映像処理のための方法および装置
EP0528293B1 (en) Apparatus for reducing quantization artifacts in an interframe hybrid coding system with motion compensation
US6825885B2 (en) Motion information coding and decoding method
US20090304090A1 (en) Method for Scalable Video Coding
Kauff et al. Functional coding of video using a shape-adaptive DCT algorithm and an object-based motion prediction toolbox
US20070047642A1 (en) Video data compression
US5654761A (en) Image processing system using pixel-by-pixel motion estimation and frame decimation
KR0159370B1 (ko) 물체의 경계를 고려한 영상 부호화방법 및 장치
US6909748B2 (en) Method and system for image compression using block size heuristics
JP3950211B2 (ja) 動きベクトル符号化装置
US5760845A (en) Method for determining motion vectors based on a block matching motion estimation technique
JP2000503174A (ja) 動き推定のための方法
US6606414B1 (en) Method and device for coding a digitized image
KR100240344B1 (ko) 적응적인 윤곽선 부호화 장치 및 방법
Ismaeil et al. An efficient, similarity-based error concealment method for block-based coded images
Lee et al. Three-dimensional DCT/WT compression using motion vector segmentation for low bit-rate video coding
Strat Object-based encoding: next-generation video compression
KR100203658B1 (ko) 물체의 윤곽 부호화를 위한 움직임 추정장치
KR100238890B1 (ko) 개선된 텍스쳐 움직임 벡터 추정 장치 및 그 추정 방법
Li et al. Extended signal-theoretic techniques for very low bit-rate video coding
Xie et al. Layered coding system for very low bitrate videophone
Katsaggelos et al. Exploitation of Spatio-Temporal Inter-Correlation Among Motion, Segmentation and Intensity Fields for Very Low Bit Rate Coding of Video
Lee et al. Two-layer object-oriented analysis-synthesis coding based on shape parameter decomposition

Legal Events

Date Code Title Description
AK Designated states

Kind code of ref document: A1

Designated state(s): AE AG AL AM AT AU AZ BA BB BG BR BY CA CH CN CR CU CZ DE DK DM DZ EE ES FI GB GD GE GH GM HR HU ID IL IN IS JP KE KG KP KR KZ LC LK LR LS LT LU LV MA MD MG MK MN MW MX NO NZ PL PT RO RU SD SE SG SI SK SL TJ TM TR TT TZ UA UG US UZ VN YU ZA ZW

AL Designated countries for regional patents

Kind code of ref document: A1

Designated state(s): GH GM KE LS MW SD SL SZ TZ UG ZW AM AZ BY KG KZ MD RU TJ TM AT BE CH CY DE DK ES FI FR GB GR IE IT LU MC NL PT SE BF BJ CF CG CI CM GA GN GW ML MR NE SN TD TG

121 Ep: the epo has been informed by wipo that ep was designated in this application
REG Reference to national code

Ref country code: DE

Ref legal event code: 8642

122 Ep: pct application non-entry in european phase
COP Corrected version of pamphlet

Free format text: PAGES 1/28-28/28, DRAWINGS, REPLACED BY NEW PAGES 1/29-29/29; DUE TO LATE TRANSMITTAL BY THE RECEIVING OFFICE

NENP Non-entry into the national phase

Ref country code: JP