JP2001523928A - Video information compression apparatus and method - Google Patents

Abstract

SUMMARY A method and apparatus are disclosed for efficiently encoding data representing a video image, thereby reducing the amount of data that must be transferred to a decoder. The method includes transforming the dataset using a tensor product wavelet transform that can transmit the remainder from one subband to another. A set of subbands (in the form of macroblocks) is weighted, detected, and allows for prioritization of the ordered and transformed data. Motion compensation techniques are performed on the motion vectors that are positionally encoded into bitstream packets for transmission to the decoder and the subband data that produces the prediction error. Subband macroblocks and subband blocks equal to zero are identified as being in the bitstream packet to further reduce the amount of data that must be transferred to the decoder.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001] This application claims priority to provisional application number 60 / 066,638, filed November 14, 1997, which is incorporated herein by reference.

[0002]

TECHNICAL FIELD OF THE INVENTION

The present invention generally relates to devices and methods for encoding and decoding video information. More particularly, the invention relates to a method for motion estimation and motion estimation in an apparatus and in the transform domain.

[0003]

[Prior art]

Due to the limited bandwidth available on the transmission line, only a limited number of bits are available for encoding audio and video information. Video encoding techniques seek to encode video information with as few bits as possible, while still maintaining the image quality required for a given application. Therefore, video compression techniques are needed to convey the video signal by representing the remaining information with the fewest bits that can reconstruct an approximation of the original image with minimal loss of important features, excluding redundant information. Trying to reduce bandwidth. In this way, the compressed data can be stored or transmitted in a more efficient manner than the original image data.

There are many video coding techniques that improve coding efficiency by removing statistical redundancy from video signals. Many standard image compression schemes use the discrete cosine transform (
DCT) based on the block transform of the input image. For example, a well-known MPEG video coding technique developed by a group of motion picture experts is the correlation between pixels (pels) in the spatial domain (using DCT) and the image in the time domain (using prediction and motion compensation). Significant bit rate reduction is achieved by utilizing the correlation between frames.

In the well-known orthogonal and bi-orthogonal (sub-band) transforms that are based on encoding systems (including the orthogonal transforms that are superimposed), an image can be obtained without having to first block the image. Is converted. Transform encoders based on DCT transform an image without having to first block. Transform encoders based on the DCT block images for two main reasons: 1) Experience shows that the DCT
We have found that it is a good approximation to the known optimal transform (Karhunen-Roube) for x8 regions or a series of difference images, and 2) the processing of the DCT is O (NlogN
) Is increased, so that the blocking of the image limits the computational effort.

[0006] The end result is that unless the DCT-based method is specifically enhanced, the 8
It has a base function that is compactly supported by (or just outside) the x8 region. The orthogonal and bi-orthogonal transforms under consideration have elementary members that are primarily supported within a finite interval of the image, but share the extent with adjacent spatial regions. One input image is divided into a plurality of spatial frequency bands using a subband image coding technique, for example, a set of filters, and each band or channel is quantized. For a detailed discussion of subband image coding techniques, see SPIE Vol. 241-52 (February 1992) Sea Podiruchak (
C. Podilchuk) and A. Jacquin
Subband Video Coding with Dynamic Bit Allocation and Geometric Vector Quantization "(
Subband Video Coding With Dynamic Bi
t Allocation and Geometric Vector Qu
See, for example, At each stage of the subband encoding method, the signal is divided into a low-pass approximation of the image and a high-pass term representing the details lost by making that approximation.

In addition, DCT-based transform encoders are transform-invariant in the sense that the base member has supports that extend over an 8 × 8 block. This prevents motion compensation from being performed efficiently in the transform domain. Thus, most motion compensation techniques in use temporarily utilize adjacent image frames to create an error term that is a transform encoded into an 8x8 block. As a result, these techniques require an inverse transform to be performed to provide a reference frame from the frequency domain to the time domain. An example of such a system is described in US Pat. No. 5,481,5, Suzuki et al.
No. 53 and U.S. Pat. No. 5,025,482 to Murakami et al.

FIG. 1 shows a schematic block diagram of a conventional standard video compression method using DCT. At block 10, changes in the image sequence are efficiently represented by motion detection techniques, such as one technique used in MPEG when in prediction mode. In particular, the previous frame is used as a reference frame, and in forward prediction, the next frame is compared with the previous frame to eliminate temporal redundancy and rank the differences between them according to degree. This step sets the stage of motion prediction for the next frame and reduces the size of the data for the next frame.

At block 12, a determination is made as to which portion of the image has moved. Continuing with the MPEG example using the data set provided by block 10, motion estimation between frames is performed by applying motion compensation techniques to the reference frame and the next frame. The resulting prediction is subtracted from the next frame to generate a prediction error / frame. Thereafter, at block 14, the changes are converted to features. In MPEG, this is a two-dimensional 8x8 DCT
This is done by compressing the prediction error using

[0010] Most video compression techniques based on DCT or subband encoders have focused on high precision techniques that attempt to encode video information without losing accuracy in the transform stage. Such high-precision encoding techniques rely on relatively expensive microprocessors (eg, Intel PENTIUM ™ processors). And it has dedicated hardware that helps in the operation of floating point arithmetic, thereby reducing the penalty for maintaining high accuracy.

However, for many applications, such relatively expensive hardware has not proven to be practical or valuable. Therefore, there is a need for an embodiment that maintains a cheap and acceptable image quality level. However, known limited precision transforms that can be performed on low cost hardware tend to exhibit low accuracy due to the "lossy" nature of the encoding process. As used herein, a "lossy" system refers to a system that loses precision through the various stages of the encoder, thereby lacking the ability to substantially recover the input from the transform coefficients when decoding. . The inability to compensate for the accuracy reduction exhibited by these low-precision transforms was an obstacle to using such transforms.

In view of the foregoing, a video encoder that performs motion compensation in the transform domain, thereby eliminating the need for an inverse transform in the encoder and allowing a simple control structure for software and hardware devices Is required. There is also a technical need for a video encoder having a class of transforms suitable for low-precision implementations, including control structures that allow for low-cost hardware and high-speed software devices.

[0013]

[Problems to be solved by the invention]

The present invention is directed to a new and unique apparatus and method for compressing data. More specifically, the apparatus and method are adapted and configured, for example, to more efficiently encode data representing a video image, thereby reducing the amount of data that must be transferred to a decoder.

[0014]

[Means for Solving the Problems]

The present invention relates to a method for compressing data comprising a first data set and a second data set. The method includes transforming the first and second data sets into corresponding first and second transform coefficient sets. Thereafter, data representing the difference between the first and second sets of transform coefficients is generated. The generated data is then encoded for transmission to a decoder.

Transforming the first and second data sets may be performed using a tensor product wavelet transform. Further, the remainder resulting from the transform method can be sent from one subband to another.

Data representing the difference between the first and second sets of transform coefficients is generated by evaluating the difference between the first and second sets of transform coefficients to provide a motion vector. The motion vector is applied to the first set of transform coefficients to make a prediction of the second set of transform coefficients. The prediction is subtracted from the second set of transform coefficients to produce a set of prediction errors. The first and second sets of transform coefficients can be error corrected to ensure synchronization between the encoder and the decoder.

In evaluating the difference between the first and second sets of transform coefficients, a search region is generated around a subset of the transform coefficients from one of the first and second sets of transform coefficients. Thereafter, an associated subset of the transform coefficients is added to the search area from the other of the first and second sets of transform coefficients. Next, the relevant subset of transform coefficients is incrementally traversed within the search area to a position representing the best incremental match. The relevant subset can then be moved fractionally laterally within the search area to a position that represents the best fractional match.

[0018] Another embodiment of a method of compressing data comprising a first data set and a second data set comprises converting the first and second data sets to corresponding first and second sets of subbands. Including converting to Next, data is generated that represents the difference between the first and second sets of subbands. This data can be generated, for example, by implementing a motion compensation technique. Motion compensation techniques can provide outputs such as motion vectors and prediction errors. Thereafter, the generated data is encoded for transmission to a decoder.

One embodiment may also be a second set macroblock packed to form a subband macroblock grouping. Thereafter, the generated data can be obtained by a motion compensation technique as follows. The difference between the first set of subbands and the subband macroblock grouping is
Evaluated to give the motion vector. The motion vectors are added to the first set of subbands to make a prediction of the second set of subbands. This prediction is then subtracted from the second set of subbands to produce a set of prediction errors.

[0020] These differences can be evaluated between the first set of subbands and subbands and subband groupings as follows. A search region is generated around the subset of transform coefficients from the first set of subbands. A relevant subset of transform coefficients from the sub-band macroblock grouping is applied to the search region. The relevant subset of the transform coefficients is then incrementally traversed within the search area to the position representing the best incremental match. Next, the relevant subset of transform coefficients is fractionally moved within the search area to a position that represents the best fractional match.

A subband macroblock packing method for organizing subband blocks of a set of subbands resulting from transforming an image is also disclosed. The method includes separating a set of related subband blocks from a set of subbands corresponding to image macroblocks in the image. The set of related subband blocks is packed as one with the subband macroblock. The sub-band block steps related to separation and packing are repeated for each set of related sub-band blocks in the set of sub-bands to form a sub-band macroblock grouping.

The method of macroblock packing is further refined by arranging the set of related subband blocks within a subband macroblock in the same relative position as the subband block occupies in the set of subbands. it can. The method also includes locating the sub-band macroblock within the sub-band macroblock grouping at the same spatial location where the corresponding image macroblock is located within the image macroblock grouping.

After macroblock packing, a change can be detected between a first subband macroblock grouping (reference) and the next second subband macroblock grouping. The detection step is a general expression of the form Where: ec = measured distortion with respect to reference R, Wi = applied weight, G = transform coefficient of second subband macroblock grouping, and R = reference (eg first subband macroblock) ·grouping)
Based on the distortion calculation.

A more specific form of the equation for evaluating distortion is of the form

Another embodiment of the present invention is described as a finite precision method for transforming a data set into transform coefficients, wherein the data set is transformed using a tensor product wavelet pair. , The remainder coming out of it is propagated to the opposite filter path. More specifically, this embodiment can include determining the low-pass and high-pass components of the image. The low-pass component is normalized to produce a low-pass normalized output and a first remainder (rl). Similarly, the high-pass component is normalized to produce a high-pass normalized output and a second remainder (rh). The first operation (g (rl, rh)) is performed by the first and second remainder (rl (r
h)), and the resulting result is added to the approximation. Then, a second operation (f (rl, rh)) is also performed on the first and second remainders (rl (rh)), and the resulting result is added to the detail. Remaining (error propagation)
It is important to note that the propagation of can be used in any transformation, but not just a tensor product.

According to the finite precision method described above, the representation of an image is too complete. The method may include down-sampling the high-pass and low-pass components to obtain the necessary and sufficient transform coefficients representing the image in the transform domain, eg, only two.

An embodiment of the finite precision method comprises a low-pass filter having the values -1, 2, 6, 2, -1 and a high-pass filter having the values -1, 2, -1. The first operation (g (rl, rh)) and the second operation (f (rl, rh)) have the following functions: g (rl, rh) = rh, and f (rl, rh) = floor (rh / 2), where nh = 1/2.

A specific example of a tensor product wavelet transform that includes the above has the formula as well as Where X _2i = input data, data following the data X _{2i + 1} = after preceding the _{X 2i-1} = input data X21 is (took one tenth of the high-pass filter output) dataX2, Di = detailed part section , D _{i + 1} = detail part term following D, and A _i = approximate term (takes one-tenth of low-pass filter output).

[0029] Also disclosed is an encoder apparatus for predicting a change between a series of frames in a transform domain. The apparatus is configured to receive first and second frames of a series of frames, and further generates first and second sets of corresponding subbands, each supporting a set of transform coefficients. And a conversion device having an input configured as described above. A motion compensator having an input coupled to the transform device is configured to receive the first and second sets of subbands, and further comprises the first and second subbands.
Are configured to efficiently represent the difference between the sets. Also included is a difference block having an input connected to the transform device and an input connected to the output of the motion compensator.
The input received from the motion compensator is subtracted from the second set of subbands in the difference block, thereby producing a prediction error.

[0030] The motion compensator comprises a motion estimator configured to compare the first and second sets of subbands. A set of motion vectors is generated therefrom, which approximately represents the difference between the first and second sets of subbands. The motion compensator also
An input is coupled to the motion estimator and is configured to receive the motion vector of the sub-band and the first set, and further configured to generate a prediction grouping therefrom that represents a prediction of the second set of sub-bands. A motion prediction device. The prediction of the second set of subbands is subtracted from the second set of subbands in the difference block to obtain a prediction error.

A finite precision transform device for transforming an image frame into a transform domain is also disclosed. The apparatus comprises a low-pass component and a high-pass component sharing an input arranged in parallel and configured to receive an image frame. A low-pass normalizer is provided having an input configured to receive an output of the low-pass component and further configured to produce a low-pass normalized output and a second remainder (rl). Have been. The high-pass normalizer has an input configured to receive an output of the high-pass component, and is further configured to produce a high-pass normalized output and a second remainder (rl). . The first computing device is configured to receive an input of a first residue (rl) and a second residue (rh), and further comprises a first calculation (g (rl, rh)
)), Thereby generating a first calculation result. Second
Is configured to receive an input of a first remainder (rl) and a second remainder (rh), and further configured to calculate a second calculation (f (rl, rh)). , Thereby generating a first calculation result. Note that a first adder has a low-pass normalized output and an input configured to receive a first calculation result, wherein the first adder generates a sub-band approximation. Similarly, a second adder has a low-pass normalized output and an input configured to receive the second calculation result, and the second adder generates a subband detail.

The finite precision conversion device further includes a first downsampler (a circuit for sampling with a reduced sampling number) at a low-pass output and a second downsampler at a high-pass output.
Downsampling of two (sampling with a reduced number of samples) provides enough and necessary transform coefficients to reconstruct the input image at the decoder.

[0033] These and other unique features of the devices and methods disclosed herein will become more readily apparent from the following detailed description, taken in conjunction with the drawings.

[0034]

BEST MODE FOR CARRYING OUT THE INVENTION

Embodiments of the present invention provide an apparatus and method for compressing a digital video signal using limited precision conversion techniques. The embodiment improves upon techniques based on conventional lossless or lossy transforms by motion compensation, eg, evaluating and predicting motion in the transform domain, rather than in the time domain as in the prior art. . Thus, improved image quality can be achieved with less expensive hardware.

The term “motion compensation” is intended to be defined in the broadest sense. In other words, although motion compensation is often described and illustrated herein as including motion estimation and motion estimation of a group of pixels, it is also understood that motion compensation encompasses, for example, rotation and scale. Should. In addition, for example, the term "motion compensation"
May include simply generating data representing the difference between the two sets of data.

The compression efficiency is obtained both by converting the image into features and first mapping the features. The disclosure herein is shown as it relates to a sequence of images or video frames. Such image sequences are easily aligned with each other and with spatially aligned data elements (either scalar, vector or functional) determined by time or some other parameter. I understand that there is. The image sequence can be in a rectangular coordinate system, but other coordinate systems in the prior art can be used.

It should be noted that current devices and methods are for non-video applications (eg, speech,
Voice and ECG compression). That is, even though the invention disclosed herein is illustrated in a two-dimensional system (2D) (ie, video compression), the teachings generally apply to any other dimensional system to advance the art of data compression. It is also intended to be applicable.

For example, the teachings relate to a 1-1 / 2 dimensional system (1
-1 / 2D). The teachings can also be applied to three-dimensional systems (3D) (eg, magnetic resonance imaging (MRI)).

Throughout the following description, the term “frame” is independent of the shape of a single image, ie, whether it is in the time domain, the frequency domain, or any other processing performed on it. Regardless, whether for a single image in a sequence of images sent to the encoder. In addition, the term “pel” is used for pixels in the time domain, and the terms “coefficient” and “transform coefficient” are used, for example, for the representation of a pel that occurs after the pel has passed the forward wavelet transform. These terms are used to facilitate the description of the examples and are in no way intended to limit the scope of the invention.

Reference is now made to the drawings, in which like reference numbers identify like elements of the present invention, and shown in FIG. 2 in a schematic block diagram of an embodiment for compressing a series of images or a series of frames. Have been. This diagram is one of several embodiments disclosed herein. More detailed examples are discussed in the following sections.

In FIG. 2, an image is transformed at block 20 into a set of features in the transform domain. Features determined to be significant for the image (ie, those features that have been determined to have changed significantly from the past or reference frame) are selected at block 22. Significant features are efficiently represented at block 24, and are then sent to a decoder to update features in the reference frame.

For example, the original image is transformed at block 20 and represented by a set of transform coefficients. The transform coefficients of the coefficient set are then evaluated at block 22 to determine their significance by various weighting and evaluation techniques, and then ranked according to the weights. Thereafter, at block 24, motion compensation between the current frame and the past or reference frame occurs. Motion compensation may include motion evaluating changes between frames to generate a set of motion vectors. Thereafter, the motion vector is applied to the reference frame during the motion estimation step. The results from the motion prediction are subtracted from a set of transform coefficients to determine the error in the prediction. The prediction error is then optionally scaled and finally positionally encoded with the motion vector for transmission to the decoder.

Referring to FIG. 3, a schematic block diagram illustrates a more detailed configuration of the embodiment described with respect to FIG. For example, Caltech intermediate format (Caltech
An image sequence or series of video frames 26 encoded in the Intermediate Format (CIF) is sent to a converter 28.
The CIF fume has 288x352 pels. In the converter 28, the frame is a quarter CIF (QCIF), for example, a QCIF as shown in FIG.
IF image 30. The QCIF image has 144 × 176 pels.
The CIF is converted to QCIF by low pass filtering by two in both the horizontal and vertical directions and taking one tenth. For ease of processing, 144 × 176 pels are divided into image macroblocks (IMBx, x) each having 16 × 16 pels. QCIF is used herein by way of example only and is not considered at all as a limitation on the present invention. The techniques described below can easily be adapted to other image (and non-image) formats in ways known to those skilled in the art.

Referring to FIGS. 3 and 4, a QCIF image 30 is sent to blocks 32 and 36 which form block 20 of FIG. 2 where mapping to image features occurs. More specifically, the QCIF image 30 (FIG. 4 (a)) is sent to a block 32 where the forward wavelet transform converts each frame into a set of subbands 34 (FIG. 4 (b)). This organization of the transformed image (ie, the set of subbands 34) is stored in memory for later use, for example, for motion estimation, motion estimation and determination of prediction error. Suitable forward wavelet transforms that can be used for the present invention are discussed in further detail herein below.

The set of subbands 34 is sent to a block 36 for subband macroblock packing. During subband macroblock packing, subband blocks corresponding to a particular image macroblock are organized to form a subband macroblock (SMBx, x). Thereafter, each subband macroblock is spatially resident of the picture macroblock it represents as it pertains. The set of all subband macroblocks for a particular frame is called a subband macroblock grouping 40.

FIG. 5 shows a method of subband macroblock packing. During subband macroblock packing, all relevant subband blocks in the set of subbands 34 (FIG. 5 (a)) form a subband macroblock 38 as shown in FIG. 5 (a). To be reorganized during subband macroblock packing.

For example, the shaded subband block in FIG. 5A corresponding to the image macroblocks 2 and 4 (1 MB _2,4 ) in FIG. 4A is as shown in FIG. 5B. Reconstructed during subband macroblock packing in block 36 (FIG. 3) to form subband macroblocks SMB2,4. The sub-band macroblocks 38 (SMB _{0,0 to} SMB _8,10 ) are then each sub-band macroblock whose corresponding image macroblock (I
As supported by the spatial position of MBx, x), the subband macroblocks are configured into subband macroblock groupings 40 as shown in the subband macroblocks 38 (FIG. 5C). In this example, SMBs _{2 and 4} correspond to FIGS. 4 (a) and 5 (
As shown in c), it can be seen that it is significantly supported by the spatial position of 1 MB _2,4 .

Although the embodiments described herein refer only to frame images represented in QCIF, those skilled in the art will recognize that other formats can be used without departing from the teachings of the present invention. It is important to reiterate what you will easily understand. It is also important to note that a particular grouping of sub-band blocks in each sub-band macroblock is used to adapt the particular wavelets shown. There are other groupings of subband data that are more appropriate for other wavelets.

A set 30 of image macroblocks (FIG. 4A) and a set of subbands 34 (FIG. 4A)
(B)) and the above description of the sub-band macroblock grouping 40 (FIG. 5 (c)) shows that there is some correlation between image macroblocks, sub-band blocks and sub-band macro blocks. It should be readily apparent. Examples of such correlations are as follows: (a) Image macroblocks 2, 4 (1 MB2
, 4). It is shaded and the image macroblock 106 of FIG.
4 (b) (eg, sub-band block 116 and sub-band 33 (in sub-band 00 (SB ₀₀ )) in FIG.
All of the sub-band blocks 118) in SB 33) and (c) the sub-band macro blocks 2, 4 (SMB ₂ ) shaded and identified as sub-band macro block 117 in FIG. _{, 4} ). Descriptions of this specification that include coefficients having relationships such as those exemplified above may be said to be "related."

Referring again to FIG. 3, subband macroblock grouping 40 is sent to blocks 42, 46, 48 and 52 forming block 22 in FIG. In FIG. 2, the features or subband macroblocks (SMB0,0 to SMB8
, 10) has changed. In particular, the weights are sent to a block 42 that is applied to scale up or down each subband macroblock in the subband macroblock grouping 40 by an amount that evens out the perceptual significance of the subband macroblock. The output of the weighting block 42 is a weighted grouping 44.

The importance of perception by weighting is determined, for example, by Mean Opinion Score.
e can be determined by research or by the Consultative Co.
mmittee for International Telegraph
and Telephone (CCITT), the specifications of which are incorporated herein by reference. 261 and H.E. 263 can also be determined from the weights used in other encoding systems. Mean
"Discrete Cosine Transform" by K. R. Rao, K. R. and P. Yip, incorporated herein by reference for discussion of Opinion Scoring. (Discrete C
Osine Transform), Academic Press (pp. 165-
74 (1990)). After the weights have been applied in block 42 to scale each subband macroblock, the weighted grouping 44 is sent to a change detection block 46 where it is processed to determine the relative amount of change that has occurred. Is done. This change is also referred to as "significance" or, in the case of video, distortion of the weighted grouping 44. Significance can be determined for given criteria (eg, zero or past weighted groupings). The loop extending from the change / detection block 46 includes a frame delay 48 that is returned to the change / detection block 46 to use past weighted groupings as a reference. The output of the change / detection block 46 is a grouping 50 in which a change has been detected.

A zero criterion is used in the change and detection block 46, for example, when first transmitting a frame through the encoder. In this case, the entire frame is matched to zero. This is also known as intra-frame matching. As described above, the past macroblock groupings are weighted in block 42 as described above, and then delayed in delay block 48 of change and detection block 46 for use as a reference. Can also be used. Also, this latter method, known as inter-frame matching, avoids repeatedly sending redundant and / or insignificant information to the decoder.

An alternative use of zero frame matching is made during operation of the system to reproduce and maintain a relatively accurate reference image at the decoder. One method uses periodically applying the zero criterion to all of every eighth frame at a standard rate of 30 frames / second. Alternatively, the image can be refreshed probabilistically, such as by randomly or orderly matching subband blocks to zero. To facilitate any method of matching all or a portion of a frame to zero, the zero-matched sub-band blocks prevent motion compensation operations (described below) to be performed on the affected blocks. Is identified as something like Thus, the identified subband block is optionally replayed at the decoder to refresh the entire reference or a portion of the reference within the block.

Referring again to FIG. 3, the set of subbands 34 previously stored in memory and the subband macroblocks of the grouping 50 in which the change was detected were determined to have changed in each subband block. Ordered in block 52 according to the quantities, ie, the quantities that are consistent with their significance. The ordering is based on the values previously assigned by weighting and detecting the subband macroblocks in blocks 42 and 46, respectively. Block 52
Output comprises an ordered subband grouping 53 and an ordered subband macroblock grouping 54 sent over line 55.

With continued reference to FIG. 3, ordered subband grouping 53 and ordered subband macroblock grouping 54 correspond to block 24 of FIG. 2 where the modified macroblocks are represented efficiently. Blocks 56, 60, 62, 68,
72 and 76 selectively. In particular, ordered subbands, macroblocks,
The grouping 54 ("current" frame) is sent to a block 56 for motion estimation. The ordered subband grouping 53 is sent to the delay block 62,
A delayed ordered subband grouping 57 ("match" frame) is then provided on line 64 for motion estimation and motion estimation of blocks 56 and 60. A set of motion vectors 58 is generated in the motion estimation block 56 and sent to the block 60 for motion estimation and also sent to the block 76 for positional encoding in the manner described below.

The motion vector 58 sent to the motion prediction block 60 is used to modify the delay ordered subband grouping 57 to generate a predicted grouping 66. Difference block 68 receives ordered subband grouping 53 and subtracts predicted grouping 66 therefrom to obtain grouping difference 70 (ie, prediction error). The grouping difference 70 is further enlarged / reduced in a block 72 to become an enlarged / reduced grouping difference 74. Those skilled in the art will recognize that the smaller the number of non-zero grouping differences 70, the more accurately the set of motion vectors 58 predicted changes between the current frame and the reference frame. And the smaller the difference, the fewer bits that need to be sent to the decoder to correct the deficiencies in the motion estimation.

The set of the enlarged / reduced grouping difference 74 from the enlargement / reduction block 72 and the motion vector 58 from the motion evaluation block 56 is positionally encoded as a macroblock of the block 76. There, the data is efficiently organized into bitstreams. The encoded bitstream grouping 78 is output from block 76 and transmitted over transmission line 80 to a decoder 82 for inverse processing. Transmission can be through a variety of media, for example, electronic, electromagnetic or optical media.

With respect to bitstream formatting, there are several well-known standard techniques for formatting bitstreams. H. The format used in a H.263-based encoder system is one example. A bitstream is basically a serial string of bit packets. Each packet represents a particular class of data.

For example, a bit packet may include system level data, video, control and audio data. When data is received for the positional encoding of block 76, it is organized into bit packets according to the format in use. Generally, a set of bit packets representing a video frame begins with bits that identify it as a new frame. The amount of quantization and other control codes generally follow. Thereafter, the list of macroblocks representing the scaled grouping difference 74 is encoded. For QCIF, the number of macroblocks is equal to 99. (See FIG. 5 (c).)

To facilitate more efficient data transfer, a macroblock zero bit (MBZer) indicating that each macroblock has non-zero data in the macroblock
o-bit). If a macroblock is present, control information for the macroblock containing the associated set of motion vectors 58 is sent, followed by subband data, ie, the associated scaled grouping difference 7.
4 follows. Including such information is through transmission line 80 because the absence of a macroblock is represented by a single symbol instead of all the bits required to identify the entire string of macroblock coefficients equal to zero. Significantly reduce the number of bits sent.

Another situation that can have more efficiency is when only a few of the subband blocks within a subband macroblock are zero.
Some embodiments include raising a sub-band zero flag (SBZero flag) for sub-bands having a coefficient equal to zero. The subbands with zero coefficients from the scaled grouping difference 74 are the corresponding subband blocks of the ordered subband grouping 53 and the predicted grouping 66
Indicates that there was no change between Significantly fewer bits are required to represent the SBZero flag than to represent each coefficient equal to zero separately. Of course, the decoder is programmed to recognize both the MBZero-bit and SBZero flags to interpret the symbols introduced during the positional coding in block 76.

An example of a zero run-length code for symbolizing a string of zeros is as follows.

With continued reference to FIG. 3, an encoded bitstream grouping 78
Is received by a decoder 82 via a transmission line 80 and sent to a positional decoding block 86 which reverses the effect of the positional coding block 76. The set of motion vectors 58 is extracted from the bitstream grouping 78 and sent to the prediction block 98. The decoded and scaled grouping difference 88 in the subband format (FIG. 4B) is provided to a quantization recovery block 90. In a quantization recovery block 90, the past transform coefficients and the past and current dequantization terms are used to recover the quantized transform coefficient values, ie, they are used to recover the grouping difference 70. Can be

The set of subbands 92 (coder reference frames) is sent to a delay block 94. The subband delay set 96 is sent from the delay block 94 to the prediction block 98. Similar to the process performed in the motion prediction block 60 of the encoder, the set of motion vectors 58 is applied to a delay set 96 of the subbands of the prediction block 98. There, the delay set 96 of subbands is modified to generate a predicted grouping 100 (ie, a subband representation of the updated image that does not include the grouping difference 70). The grouping difference 70 and the predicted grouping 100 are added in an adder block 102 that generates a set 92 of subbands (ie, a new reference frame). Finally, an inverse wavelet transform is performed on the set of subbands 92 at block 104. This step is essentially the reverse of the forward wavelet transform 32 described briefly above and will be described in more detail herein below. The resulting output from block 104 is the restored image 1
05.

As previously described and shown in FIGS. 3 and 4, the QCIF image 30 (FIG. 4 (
a)) converts each video frame to a set of subbands 34 (FIG. 4 (b)).
To the forward wavelet transform 32 which forms An embodiment of the transform block 32 utilizes a tensor product wavelet transform. For a detailed discussion of the tensor product wavelet transform,
Joel Lauzienne, which is incorporated herein by reference.
Rossiene and Ian Greenshield
ds), "Standard Wavelet Basic Compression of Images" (Standard Wavelet)
Basis Compression of Images, Optical Engineering, Vol. 33, No. 8, August 1994. Other finite precision transforms such as the well-known Mallat, GenLOT or Harr transforms can also be used. For a discussion of such suitable alternative wavelet transforms, G. Strang and T. Nguyy, which are incorporated herein by reference.
en), "Short Waves and Filter Banks" (Wavelets and Filter)
er Banks), Wellesley-Cambridge Press
(1997).

Referring to FIG. 4B, the subband 34 after the forward wavelet transform 32 has passed.
Is shown. As noted earlier, the forward wavelet transform process utilizes a tensor product wavelet transform or other well-known finite precision transforms modified herein to reduce the effects of finite precision production. In general, the transformation process is (m + 1) x (
n + 1) will consist of mxn steps to create subbands. In one embodiment discussed below in conjunction with FIG. 6, the conversion process consists of 3 × 3 steps to create a total of 16 subbands. Other embodiments within the scope of the invention can be made in accordance with the disclosure provided herein.

Referring to FIG. 6 (a), the forward wavelet conversion process is performed by the QCIF image frame 3
Filter 0s on a column-by-column basis using three stages. Each stage includes a low pass filter 108 and a high pass filter 110. In one embodiment, each low pass filter 108 has a value of -1, 2, 6, 2 and each high pass filter has a value of -1, 2, -1.

After filtering, the low-pass and high-pass components are scaled, respectively, taken down or reduced by a factor of 10 at each stage by decimators 112 and 114, respectively. As a result, the components of the sample values including the discrete signal are removed.
In the illustrated embodiment, the input image is halved in number to discard every other sample. Taking the twentieth will ultimately provide the necessary and sufficient transform coefficients to allow for accurate reproduction of the input. Thereafter, the reduced number of low-pass and high-pass component extractions are normalized at each stage in a manner described in greater detail herein below with respect to FIG. The output of the first stage includes a low pass filter component AOR and a high pass filter component DOR. The low pass filter AOR is decomposed a second time, and then a third time obtains additional column details D1R and D2R and a column average A2R.

The column outputs D _OR , D _1R , D _2R and A _2R in the column stages shown in FIG. 6 (a) are then applied on a row-by-row basis to the stages shown in FIG. 6 (b). . Each of the three stages shown in FIG. 6 (b) comprises a filter pair, downsampling and normalization process applied in the same manner as shown above in conjunction with FIG. 6 (a). The conversion output is
A set 34 of subbands as described above with respect to FIG. 3 and illustrated in FIG. 4 (b).

Referring now to FIG. 4 (b), for identification purposes, each sub-band is
Is identified by the symbol SBi, j. Where i = 0, 1, 2 or
J = 0, 1, 2, or 3 for 3 and each row. Shadowed subband block
(For example, SB₀₀Subband block 116 and SB₃₃Subbands within
Block 118) is a block of 1 MB of the QCIF image 30 shown in FIG.₂₄Corresponding to
Due to the 1/10 extraction process described above, each corresponding subband block is
, For example, SB₀₀Sub-band block 116 contains 8 × 8 coefficients and SB ₃₃ Sub-band block 112 is proportionally reduced to include 2 × 2 coefficients.
It is. As described above, the associated subband block (eg, subband position 2,
4 each subband (SB₀₀Or SB₃₃Subband block in))
Is used in block 36 (FIGS. 3 and 5) to facilitate some processing steps.
Collected during the sub-band macroblock packing step.

Referring now to FIG. 7, according to a feature of the disclosed embodiment, the remainder between each stage of the sub-band encoding process is based on the error introduced for the finite precision transform. Propagated to the opposite filter path to compensate. The propagated residue is used to adjust the coefficients in the opposite filter path to compensate for the loss of accuracy. This process results in a non-linear transformation. Further, the process of modifying the filters does not make them two orthogonal or orthogonal.

FIG. 7 illustrates the implementation for propagating the remainder to the opposite filter channel in the first stage of the column transformation shown in FIG. 6 (a). Similar production is included in each of the column and row phases. The coefficients of the input frame 30 are calculated using the low-pass filter 1
08 and the high-pass filter 110. The results are subtracted in samplers 112 and 114, respectively. The decomposed result of the low-pass filter 108 is normalized in a low-pass normalization process 120 that produces a low-pass normalized output 122 and a low-pass residual rl. The decomposed result of the high-pass filter 110 is the high-pass normalized output 126 The high-pass normalization process 1 that produces the high-pass residual rh
It is normalized at 24. The remaining rl and rh resulting from each normalization process 120 and 124, respectively, are passed through functions g (rl, rh) 128 and f (rl, rh) 130 as shown. The result of the function g (rl, rh) 128 is added to the low-pass normalized output 122 in an adder 132 to be AOR (first step average). The result of the function f (rl, rh) 130 is added to the high-pass normalized output 126 in adder 133 to become a DOR (first stage missing detail).

For the filters L = {− 1, 2, 6, 2, −1} and H = {− 1, 2, −1}, one embodiment of the remaining functions is as follows. f (rl, nh = 1/2)
rh) = floor (rh + ／) and g (rl, rh) = rh. The remaining above operations are repeated for each filter pair, reducing the bit allocation in the transform output.

An example of a tensor product wavelet pair is Where: X _2i = input data X _2i-1 = X data preceding input data X _2i , X _{2i + 1} = data following input data X _2i , D _i = detailed section (1/10 height pass filter _{output), D i + 1} = More unit section Di to the following detailed part section, _{a i} = the low pass filter output took 1 approximation term (10 minutes).

The above description of the tensor product wavelet transform refers to high-pass (details) and low-pass (approximate)
The two-way split into components is shown. In addition, the description may refer to the first band to the second band, the second band to the first band, or the first band to the second band, and the second band to the second band. The possibility of propagating in both directions to this band. The above-described embodiments should not be construed as limiting the scope of the present invention in any way for the purpose of illustrating the basic concept of the present invention.

For example, a tensor product wavelet can have a first stage where the three-way split includes a high pass filter, a mid pass filter and a low pass filter. The output of the low-pass filter can then be repeated, that is, the second stage with the three-way split can be applied to the output of the low-pass filter, for a total of 5 subbands. In such an embodiment, the rest could be propagated from the low-pass and high-pass filters to the mid-pass filter. This embodiment is just one example of how the tensor product wavelet transform can be varied and still maintain the scope and spirit of the disclosed invention. Those skilled in the art will recognize that there are many other ways in which the input can be split and repeated at each stage, as well as many other ways in which the rest can be propagated between subbands. Will understand easily.

In addition, the above description of the remaining propagation is not intended to limit its use to tensor product wavelet transforms. You can use it with any transformation. For example, the remaining propagation can be used with a discrete cosine transform (DCT). Also, the remaining propagation can be used in a lossless or lossy manner.

As described herein above, the output of the forward wavelet transform 32 may be a full or over-complete representation of the QCIF image 30. The complete representation of the QCIF image 30 includes a set of subbands that is just enough to display the contents of the image. The over-complete representation of the QCIF image 30 includes a complete representation and a redundant, selective or additional sub-band representation to facilitate motion compensation as described later herein. Each expression has a value in the disclosed embodiment. For example, over-complete representations can include various image changes (eg, translational motion, rotational motion, and scaling).
Can be included. These changes can be withdrawn as needed during motion compensation, reducing the problem of representing image changes to one of the indexing approaches.

With respect to the forward wavelet transform described above, the transformed image frame structure exemplified herein is for a luma component, but the structure is also a chroma (
It should be noted that it is not described separately because it can be applied to the chroma) component.

The change and detection block 4 described herein above with respect to FIG.
With respect to 6, it is noted that a zero criterion or some other criterion (eg, past weighted groupings output through delay 48) may be used to detect weighted grouping 44 change. An embodiment of the change / detection block 46 includes a change detection metric of the general formula (weighted grouping 44 is to be applied):

[0081] Where e _c = measured distortion relative to reference R, W _i = applied weight, G = current grouping of sub-band transform coefficients, and R = reference (eg, zero or sub-band coefficients obtained through delay block 48). Previous grouping) Change detection metric may take the following more specific form

The change / detection 46 removes some weighted macroblocks of the weighted grouping 44 if the bit allocation is determined to be too high for something to be output from the change / detection 46. To do so, the information provided by feedback 132 (FIG. 3) from coded bitstream grouping 78 can be utilized. Further, the change detection block 46 can replace another feature (eg, a sub-band block) with another that would appear to better represent that feature.

As described herein above and shown in FIG. 3, ordered subbands
Grouping 53 and ordered subband macroblock grouping 54
They are sent to a delay block 62 and a motion estimation block 56, respectively, via a line 55 for motion estimation. At block 56, the comparison process determines whether the subband blocks of the ordered subband macroblock grouping 54 (ie, the “current” frame) and the delayed ordered subband grouping 57 (ie, the “reference” frame).
With the associated search area. Those skilled in the art will recognize some advantages in utilizing an ordered subband macroblock grouping 54 for the current frame and a delayed ordered subband grouping 57 for the reference frame. However, it should also be appreciated that other groupings and combinations that adhere to the teachings of the present invention can be utilized. The comparison process performed in block 56 is to block 60 for motion estimation and block 76 for positionally encoding in the bitstream, as described briefly herein above.
, A set 58 of motion vectors sent to

With reference to FIGS. 8 and 9, the motion estimation of the block 56 and the generation of the set of motion vectors 58 will now be described in more detail. FIG. 8 shows a delay-ordered subband grouping 57. The delay-ordered subband grouping 57 is shown in FIG.
b) Similar to the set of subbands 34 shown in b), but having its subband blocks ordered in block 52 (FIG. 3) and being delayed by at least one frame in delay block 62 Was further processed by To facilitate determining the individual motion vectors are determined around at least one in subband blocks in the search region is a subband (SB ₀₀ to SB _33). The sub-band blocks in each sub-band selected to define a search area around them have been defined as significant in the change / detection block 46. It is often sufficient to produce work Li based motion vectors to a significant subband blocks in SB _00.

With continued reference to FIG. 8, a search area created around each subband block corresponding to image macroblocks 2 and 4 (1 MB ₂ , ₄ ) of the QCIF image 30 (FIG. 4A) It is shown. The size of the search area may be changed. However, the search area around the subband blocks will always be proportional to the fractional relationship between them and the image. For example, as shown at 136, the QCIF image 30 (
The basic search area of the PxP pel of FIG. 3) is to the search area around the subband block 137 of SB ₀₀ (FIG. 8) of P / 2 × P / 2 and of P / 4 × P / 2 as indicated at 139. moves to a search area around the subband block 140 of SB _01.

For the example of motion estimation given below in this specification, the PxP search area 107 of FIG. To be included. Therefore, the P / 2xP / 2 search area 136 (FIG. 8) includes 16 × 16 coefficients that are four times the size of the subband block 137 (8 × 8 coefficients). Then, the P / 4 × P / 2 search area 13
9 is 16 ×, which is four times the size of subband block 140 (8 × 4 coefficients)
Includes a coefficient of eight. As described further below in this specification, the sub-band search region is the motion for some or all significant sub-band blocks (0,0 to 8,10) of the sub-bands (SB00 to SB33). Used to help determine the vector.

The reference size (P × P) of the search area can be determined by empirical or statistical analysis, for example, in consideration of the amount of movement expected from a frame. Also, the computational effort required to perform a search for a given search area must be considered. It will be readily appreciated by those skilled in the art that the larger the search area, the more computational resources are required for the fixed processor, and therefore more interframe delay. Conversely, the smaller the search area, the less computational resources are required except for sacrificing image quality. This is especially true during high image-motion periods. That is, the quality of the image is degraded since a portion of the motion is located outside the search area, and thus may prevent accurate motion vector selection.

As described above, the ordered subband grouping 53 and the ordered subband macroblock grouping 54 are each composed of a delay block 62 from the block 52.
To the motion estimation block 56 via line 55. In the example below herein, the search area is disposed around the sub-band blocks 2 and 4 SB _{0 0} subband grouping 57 with the delay sequence (Figure 8). Then, in order to subband blocks SB ₀₀ in the subband macroblocks 2 and 4 in the ordered subband macro subband grouping 54 (see sub-band block 116 in FIG. 5 (c)) changes Used to move the search area sideways. But,
As mentioned above, any selection of sub-bands, ie all of the sub-bands, may be used following the method described below.

Referring now to FIGS. 3, 8 and 9, as described above, ordered subband grouping 53 is delayed in delay 62 to generate delayed ordered subband grouping 57 (“reference” frame). Is done. Delayed ordered subband grouping 57
Is sent to the motion estimation block 56, where the search area 136 is
It has been identified as having a P / 2xP / 2 regions in _{SB 00} around the block 137. In this example, the search area is equal to a 16 × 16 coefficient. The ranked subband macroblock grouping 54 (the "current" frame) is also sent to a motion estimation block 56, where subband block 11 in FIG.
The sub-band block 138 (FIG. 9 (a)), similar to the shaded region of FIG. 6, is retrieved for use in the comparison process described below.

Referring now specifically to FIGS. 9 (a) through 9 (d), there is shown the process by which the motion vector (MVx, x) is determined in the motion estimation block 56 of FIG. In the example below, the motion vector is one sub-band block (ie, SB00
Are determined for the sub-band blocks 2, 4) of the sub-band. However, it determines motion vectors for each significant subband blocks in each subband _{(SB 00} to _{SB 33).}

Referring to FIG. 9A, the sub-band block 138 of the ordered sub-band macroblock grouping 54 has a delay-ordered sub-band grouping 57.
It is in the search area 136 (FIG. 8). Subband block 138 is almost superimposed on subband block 137 of delay ordered subband grouping 57. As described above, the ordered subband macroblock grouping 54 has the same structure as the subband macroblock grouping 40 shown in FIG. The delay-ordered subband grouping 57 has the same structure as the set of subbands 34 shown in FIG.

Referring again to FIG. 9A, the coefficient 141 of the search area 136 (shown as four circles with an “x” each) and the coefficient 141 of the subband block 138
42 (shown as four circles) is used herein to facilitate illustrating how to determine the motion vector. It is assumed for this example that coefficients 141 and 142 are approximately equal in value and that the remaining coefficients (not shown) are different from coefficients 141 and 142 but of values approximately equal to each other. The difference between the positions of the coefficients 141 and 142 is indicative of (a change between, for example, a translational movement) between the two video frames.

Referring to FIG. 9B, the block 138 moves sideways, that is, searches are performed in a predetermined stepwise pattern, and the search area 136 includes the steps between the subband block 138 and the search area 136. Try to determine the total absolute difference. One skilled in the art will recognize that various lateral movement patterns can be used. Note that criteria other than the total absolute difference can be used as a basis for comparison. The first comparison seeks to find the best match using incremental or full-step movement of subband block 138. An incremental movement is a full shift or step in either the x or y direction. For example, in searching the entire search region 136, the subband block 138 shifts within the search region 136 by ± 4 increments in the x direction, ie, only the transform coefficients, and ± 4 increments in the y direction. Since subband block 138 has 8 × 8 coefficients, while search area 136 has 16 × 16 coefficients, subband block 138
Shifts ± 4 increments in the x and y directions.

With continued reference to FIG. 9 (b), after performing an incremental search, it can be seen that the best matches are 3 full incremental moves in the positive x direction and 2 full incremental moves in the positive y direction. . Thereafter, as seen in FIG. 9 (c), a fractional difference is determined to more accurately represent the difference between the subband block 138 and the search area 136. To facilitate this process, a mask representing the appropriate fractional motion for a particular subband is applied to subband block 138.

For example, SB ₀₀ is ４ of the size of the associated macroblock in the original image (see 1 MB2, 4 in FIG. 4 (a)), so subband block 1
38 can reproduce 1MB2, 4 finer movements more accurately 4
There is a fractional move. That is, the sub-band block 138 has ±
One-half and ± 1/2 increments in the y-direction can be moved. Therefore, the four fraction mask 143 is used to modify the subband block 138 in search of the best match.

With continued reference to FIG. 9C, four masks 143 are applied to subband block 138. During each mask application, the total absolute difference between the coefficients in subband block 138 and search area 136 is determined. If a better match is found, the fractional mask is added to the motion vector as compared to that determined during the above incremental search. In this example, the best match is determined to be + ／ fractional motion in the positive x direction. The resulting x and y components of the motion vector are +31/2 and +2, respectively. One skilled in the art will recognize that it is unusual to obtain as exact a match as shown in the example above. In this sense, the "best match" between the subband block coefficients and the search area coefficients is the "best match" between the two.
It may be more accurately described as "closest approximation." Motion estimation is used later to compensate for this lack of accuracy.

Referring to FIG. 9D, the x and y components of the motion vector are scaled up and down with their signs reversed. More particularly, applying a -1 in each of the x and y components, and SB ₀₀ is in this example being used for motion estimation is to place a 2 in each of the x and y components. The sign of the x and y components is such that when the motion vector is applied to the delayed ordered subband grouping 57 during motion estimation (discussed in more detail below), the appropriate coefficients are "from" the previous frame position. Inverted to be moved to the "current" frame position. Then, the x and y components are converted to the original QCIF image (
For the relevant macroblock of 1 MB ₂ , ₄ ), the movement determined above (x = 3 ·
(Y = 2)). To scaling during the motion estimation, to be able to determine the x and y components used to shift the appropriate coefficient in the subband SB ₀₀ to SB ₃₃ more easily.

In this example, the resulting motion vectors identifying the motion of the subband blocks in SMB ₂ , ₄ are x = −7 and y = −4 (MV ₂ , ₄ ). MV ₂ , ₄ stores a set 58 of motion vectors. MV ₂ , ₄ therefore predicts a small set of ordered coefficients subband grouping 53 (“current” frame) from each subband of delay ordered subband grouping 57 (“reference” frame). To represent their move to a new position. The above process, for example, is repeated for each significant subband blocks SB _00. Processing generally proceeds in ranking order, ie, from the macroblock with the largest amount of movement to the one with the smallest amount of movement. A completely meaningless subband block would not be assigned a motion vector since it is completely unthinkable. This may occur, for example, when there is no or no change in position between frames. It can also occur when a subband block is matched to zero, as described herein above.

If different subbands are to be used to calculate the motion vector, the incremental and fractional motions are determined in a manner similar to that described above using the specific subband proportionality for the QCIF image 30 Will be done. For example, if the subband block of SB ₀₁ is used to create a motion vector, applying the following criteria. Region size = 16 × 8 coefficients, x-fraction mask = 、, ± １ ／ and ± ３ increments, y-fraction mask = ±± increment, x scaling = 4, and y scaling = 2.

The advantage of using the above method is that its separable filter can be used. In other words, the filter used for incrementing and fractional movement of one subband block can be used for incrementing and fractionalizing another subband block. For example, the sub-band block of SB ₀₀ has x = ± 1/2 and y
== １ ／ 1/2 with four possible fractional movements. And the sub-band of SB ₀₁
The blocks have possible fractional movements of x = ± １ ／, ± １ ／ and ± ３, and y = ± １ ／. Due to the common fractional shift of x = ± １ ／ and y = ± １ ／ in SB ₀₀ and SB ₀₁ , a single separable filter is used for x = ± in both subbands.
It can be used for fractional moves of +1/2, x = -1 / 2, y = + 1/2 and y = -1 / 2. This method can be used for all common fractional moves in the delay ordered subband grouping 57. The same advantageous use of separable filters can be implemented in the motion estimation block 60.

Referring to FIG. 10, after all significant sub-band blocks have been processed in the motion estimation block 56, the motion block set 58 becomes the motion prediction block 6
It is output to the position coding block 76 with 0. In the motion prediction block 60,
The motion vector is a delayed ordered subband grouping 5 of a set of coefficients.
7 ("reference" frame) ordered subband grouping 5 from each subband
3 (the "current" frame) is used to calculate the shift to their new position to predict.

To determine which mask to use to generate such a shift, the x and y components are multiplied by the corresponding modulo reciprocal of each subband block. For example, the coefficient 148 determined to have moved to the 2,4 position of SB ₀₀
X of MV _2,4 to determine the x and y components for shifting the 8 × 8 set of
And the y component are each multiplied by the inverse of the corresponding modulo-2. The result of this calculation is x
= -31 / 2 and y = -2. Therefore, x = -3 incremental movement mask, x
A fractional moving mask of = -1 / 2 and an incremental moving mask of y = -2 are applied to the 8x8 coefficients 148.

As a second example, 8 × of the coefficient 149 determined to move to the position 2 and 4 of SB ₀₁
4 in order to determine the x and y components for shifting the set, x component of the _{MV 2, 4} is multiplied with inverse modulo 4, the y-component of the _{MV 2, 4} is multiplied by the reciprocal of the modulo 2. The result of this calculation is x = -1.3 / 4 and y = -2. Therefore, x
==-1 incremental movement mask, x = -3 / 4 fractional movement mask, and y = -2 incremental movement mask.

FIG. 10 shows the movement of the entire set of coefficients to subband blocks corresponding to SMB ₂ , ₄ . Applying a motion vector (MVx, x) from the set of motion vectors 58 to a delayed ordered subband grouping 57 ("reference" frame) yields a prediction of an ordered subband grouping 53 ("current" frame). And is referred to as a predicted grouping 66 (FIG. 3).

An alternative embodiment of the masking process described above for determining fractional movement between frames involves using a 3 × 3 coefficient mask. These masks take a weighted average of the coefficients surrounding the selected coefficient. In an alternative method, the set of motion vectors 58 containing only the incremental movement is divided into each sub-band (SB _{00 to} SB ₃₃ ).
Or it is determined as shown in above FIG. 9 (a) and 9 (b) for each significant subband block in a selected number of sub-bands (e.g., SB ₀₀ only). The set of motion vectors 58 is sent to the motion prediction block 60.

In the motion prediction block 60, the set of motion vectors 58 is applied in a manner similar to that shown in FIG. Is shifted incrementally. Thereafter, each coefficient of each shifted set of coefficients has a 3 × 3 mask applied to it. The applied mask determines a weighted average of the coefficients surrounding each shifted coefficient. The result of the calculation is a prediction of the shifted coefficient, ie a new value of the coefficient.

All of the motion vectors from the set of motion vectors 58 are delayed ordered subbands.
After all of the coefficients shifted by the motion vectors applied to grouping 57 and having applied a 3x3 mask to them, the result is the predicted grouping 6
6 is output from the motion prediction block 60. Of course, this process
It is repeated in the prediction block 98 of the decoder 82 to replicate the masking process performed in the motion prediction block 60.

After the prediction has been determined by either of the methods described above, the predicted grouping 66 is passed to a difference block 68 where the difference between the ordered subband grouping 53 and the predicted grouping 66 is determined. It is. As described above, the difference block 68
Generates a grouping difference 70.

Although the motion compensation method described herein is illustrated as working in conjunction with tensor product wavelets, it is important to note that the method can be used with other forms of transformation. is there. This involves using motion compensation methods with other transforms in either the time domain or the transform domain. For example, data converted by DCT can be motion corrected in the same manner as described above. That is, transform coefficients 64 in each of the 8 × 8 block of DCT is similar to that used to transform coefficients 64 in each of the 8x8 sub-band blocks SB ₀₀ tensor product wavelet to motion compensation The motion can be corrected by the following method.

Referring now to FIG. 11, another embodiment of a video encoder is shown.
As in the embodiment described above and shown in FIG.
This is done in the transform domain of blocks 150 and 152, respectively. The front part of the embodiment is the same as that described above and shown in FIG. More specifically, the CIF image 26 is converted in a converter 28 into a QCIF image 30. The QCIF image 30 is transformed and converted into a subband macroblock grouping 40 by an image characterizing the mapping component 20. Then, a set of subbands 34 and a subband macroblock grouping 40
Are transformed into an ordered subband grouping 53 and an ordered subband macroblock grouping 54, respectively, by components associated with determining the modified features of 22.

Also, the fact that the ordered subband macroblock grouping 54 is sent to the motion evaluation block 150 and the ordered subband grouping 53 is sent to the difference block 68 is the same as the embodiment shown in FIG. The same is true. However, instead of using the delay-ordered subband grouping 57 as a reference frame, an error-corrected subband grouping 171 with the cumulative error added thereto is sent to the delay block 156, whereby the delay subband Generate a grouping 172 ("reference" frame). Such a modification is necessary when the quantization (or scaling) is large enough to significantly alter the prediction error 70 generated in the difference block 68.

To create the error correction subband grouping 171, when the system is matched to zero, for example, when the system is started or when the decoder reference is to be refreshed, a partial ordered sub The band grouping 53 is passed unchanged through the difference block 68 and stored in memory. Thereafter, the prediction error 70 of each next frame passes through the quantization block 158, so that the prediction error 70 is accumulated, ie, added to the reference. The updated reference image is the delay block 1
56, thereby generating a delayed subband grouping 172. By utilizing this method, the encoder reference remains synchronized with the decoder reference. One skilled in the art will recognize that when a significant amount of scaling and / or quantization is performed between motion estimation and positional encoding, such an arrangement maintains synchronization between the encoder and the decoder. Would be effective.

After the motion estimation block 150 and the motion prediction block 152 have received the delayed subband grouping 172 from the delay block 156, motion estimation and motion prediction are determined by procedures similar to those described herein above. 8 to 10
Is shown in the figure. The forward feed 159 is provided between the change detection 46 and the quantization block 158, and adjusts the amount of quantization to be performed on a specific block according to the amount of change of the block. When a large amount of change is detected in change detection 46, a number of bits are allocated for quantization. And conversely, when a small change is detected in change detection 46, a proportionately smaller number of bits are allocated for quantization.

Referring next to FIG. 12, another embodiment of the video encoder is still shown. The front part of this embodiment is similar to the embodiment described above and illustrated in FIGS. However, unlike the embodiments described above, motion estimation is performed in the image domain. This embodiment utilizes a special hardware arrangement currently available for some processors.

In FIG. 12, CIF image 26 is converted to QCIF image 30 in converter block 28. The QCIF image 30 is transformed and mapped to mapping component 2
The image is converted into a subband macroblock grouping 40 by an image characterizing 0. Sub-band macroblock grouping 40 is a component 22 that has been modified to determine sub-band macroblock ranking.
Is processed by components associated with determining a feature having The result is an ordered subband grouping 53 applied to the set of subbands 34. Thereafter, the ranked subband grouping 53 is sent to the difference block 68. The QCIF image 30 (also referred to as the “current” frame) is also sent to a motion estimation block 160 and a delay block 166 to determine a set of motion vectors 162. More specifically, image frame 30 is a delay block 166 that generates a delayed image frame 167 (also referred to as a “reference” frame).
Is delayed. Referring to FIG. 13, the delayed image frame 167 is a PxP
A pel search area is sent to a motion estimation block 160 that is created around each significant image macroblock. For example, a PxP pel search area 107 is set around image macroblocks 2 and 4 (1 MB2 and 4). Based on empirical analysis, 3
A 2x32 pel search area 107 is used as a search area around a 16x16 pel image macroblock of a QCIF image frame.

In the motion estimation block 160, each significant image macroblock (IMBx, x) of the frame of the current QCIF image 30 is in the corresponding search area of the delayed image frame 167 for determining a motion vector. For example, 1MB2,4
Retrieved from the QCIF image 30 and placed in the retrieval area 107 of the delayed image frame 167. This process is similar to that implemented in the transform domain as described above and shown in FIGS. 8 and 9 (a).

In a manner similar to that described above and shown in FIG. 9 (b), 1 MB ₂ , ₄ is the minimum total absolute difference at each step between 1 MB ₂ , ₄ and the search area 107. Is moved sideways in the search area 107 for which is to be determined. However, unlike the subband search described above, a fractional search is unnecessary when the search is in the image domain. Thus, after determining the incremental motion of 1 MB2,4, the x and y coordinates are reversed (multiplied by -1) and the set of motion vectors 162 is stored in memory. The motion vector is sent to the motion prediction block 154 and the positional coding block 76. Thereafter, the motion vectors are applied to the delay subband grouping 172 in a manner similar to that described above with respect to FIGS. 3 and 11 and shown in FIG.

Referring now to FIG. 14, another embodiment of a video encoder is shown wherein the front portion is similar to the embodiment described above and illustrated in FIGS. is there. However, unlike the embodiments described above, motion estimation and motion estimation are performed in the image domain.

In FIG. 14, a set of motion vectors 162 is determined in a manner similar to that described above and illustrated in FIGS. The set of motion vectors 162 is sent to a block 164 for motion estimation and a block 76 for position encoding. In a manner similar to that described above and illustrated in FIGS. 11 and 12, the error-corrected sub-band grouping 171 with the cumulative error added thereto includes a delay block 156.
, Thereby generating a delayed subband grouping 172 (“reference frame”). However, unlike the embodiment described above, the delayed subband grouping 172 is then reconstructed by an inverse wavelet transform block 174 to form a reconstructed image 176. The restored image is the QCI shown in FIG.
It has a configuration similar to that of the F image 30.

Alternatively, instead of restoring the entire delay subband grouping 172, the rate of restoring a portion of the grouping can be increased. For example, 3,
Using 5 filters, a reconstructed area with 48x48 pels can be obtained. A number of regions are selected based on the importance of the image macroblocks (16 × 16) around which those regions are gathered (ie, the changes detected in those regions).

In the motion prediction block 164, the image 176 in which the set of motion vectors 162 is restored (or if only a few regions are subjected to inverse wavelet transform, the restored 48 ×
48 pel area). The set of motion vectors 162 is applied to the reconstructed reference image 176 in a manner similar to that described above and illustrated in FIG. 10 to shift the set of transform coefficients in the subband representation of the QCIF image. Thereafter, the prediction 178 is sent to the forward wavelet transform block 180, which generates the predicted grouping 66. The predicted grouping 66 is then the difference block 6
Subtracted from the ordered subband grouping 53 in FIG. Quantization is performed at block 158, and the errors are accumulated to maintain a reference (as described above) and forwarded to positional coding block 76. Positional encoding of the quantized error and motion vector 162 occurs as described above and is transferred over transmission line 80 to the decoder.

Although illustrated herein as software implementation, the principles of embodiments of the present invention could also be implemented in hardware by means of an application specific integrated circuit (ASIC). If possible, include ASI, including necessary memory requirements
C production is (i) to minimize power consumption consistent with the embodiment; and (ii)
Full color video at data rates less than 13.5 MHz (eg, all CCIR6
01) must operate at the pel rate to allow compression. It is expected that power consumption will be reduced by a factor of ten by utilizing ASICs compared to conventional software and processor implementations.

Alternatively, optical methods can be used to further reduce power. As described above, an approximation to the image is made at each stage of the wavelet transform, and the details lost by making this approximation are recorded. In optoelectronic or optical production,
The manner in which light is collected and the associated charge is detected can be adjusted to collect each of the samples of the approximate image. If these approximate images are superimposed together in parallel, the detail terms can be calculated from their intermediate values by either analog or digital means. Preferably, analog means are used to calculate the detailed terms as the output of the analog stage.

The detail terms can be quantized by using a bit-serial analog-to-digital converter that implements a quantization strategy. The resulting bit stream is compressed. In this way, the photon / light device operates at a compressed data rate rather than an image data rate (as in an ASIC) or a processor data rate (as in a conventional processor). This will consume very little current and will result in a low power requirement production. It is expected that the implementation of the optical method will further reduce power consumption by a factor of nearly ten times that of the ASIC implementation.

The embodiments and variations illustrated and described herein are merely illustrative of the principles of the invention, and various modifications thereof will occur to those skilled in the art without departing from the scope and spirit of the invention. You should understand what you can do.

[Brief description of the drawings]

 Exemplary embodiments of the present invention will be described in detail with reference to the following drawings.

FIG. 1 shows a discrete cosine transform (D) in which motion compensation is performed in the image domain.
1 is a schematic block diagram of a prior art standard video compression method using CT).

FIG. 2 is a schematic block diagram illustrating a general configuration of an embodiment of the present invention that includes features for motion compensation performed in the transform domain.

FIG. 3 is a schematic block diagram of a more detailed configuration of the embodiment illustrated in FIG. 2;

4A shows a QCIF image having image macroblocks (IMBx, x) 0, 0 to 8, 10. FIG.

FIG. 4 (b) is the QC after the image frame is transformed by the forward wavelet transform.
4 shows a subband representation of an IF image.

FIG. 5 (a) shows a subband representation of the QCIF image illustrated in FIG. 4 (b).

FIG. 5 (b) shows a set of sub-band macroblocks (SMBx, x) generated from the sub-band representation shown in FIG. 5 (a).

5 (c). The sub-band macroblocks (SMBx, x) in FIG. 5 (b) are spatially corresponding to their associated picture macroblocks (IMBx, x) in FIG. 4 (a). 4 shows the organization of band macroblocks.

FIGS. 6 (a) and 6 (b) show diagrammatically the filter banks for transforming the input image and taking a tenth and their respective vertical and horizontal sub-bands produced from each filter bank. FIG.

FIG. 7 shows an architecture for moving finite precision computations in a filter bank from wideband to lowband and vice versa.

FIG. 8 shows a case where the search band is PxP pel and the size of the input image is QCI.
When F, the search area in the conversion domain for each subband (SBij) corresponding to image macroblocks 2 and 4 (1 MB2, 4) in the image domain is described in detail in the search domain for the search band of SB00. Show.

9 (a) -9 (b) show how motion is evaluated in the transform domain.

FIG. 10 shows how motion is predicted in the transform domain.

FIG. 11 is a schematic block diagram showing another detailed configuration of the embodiment shown in FIG. 2;

FIG. 12 is a schematic block diagram illustrating another detailed embodiment of the present invention in which motion estimation is performed in the image domain and motion estimation is performed in the transform domain.

FIG. 13: When the input size is QCIF, image macro blocks 2, 4 (
5 shows a P × P pel search area when searching in an image domain around 1 MB2, 4).

FIG. 14 is a schematic block diagram illustrating another detailed embodiment of the present invention in which motion estimation and motion prediction are performed in the image domain.

[Procedure amendment]

[Submission date] March 6, 2001 (2001.3.6)

[Procedure amendment 1]

[Document name to be amended] Drawing

[Correction target item name] Fig. 1

[Correction method] Change

[Correction contents]

FIG.

[Procedure amendment 2]

[Document name to be amended] Drawing

[Correction target item name] Figure 2

[Correction method] Change

[Correction contents]

FIG. 2

[Procedure amendment 3]

[Document name to be amended] Drawing

[Correction target item name] Fig. 4

[Correction method] Change

[Correction contents]

FIG. 4

[Procedure amendment 4]

[Document name to be amended] Drawing

[Correction target item name] Fig. 5

[Correction method] Change

[Correction contents]

FIG. 5

[Procedure amendment 5]

[Document name to be amended] Drawing

[Correction target item name] Fig. 8

[Correction method] Change

[Correction contents]

FIG. 8

[Procedure amendment 6]

[Document name to be amended] Drawing

[Correction target item name] Fig. 9

[Correction method] Change

[Correction contents]

FIG. 9

[Procedure amendment 7]

[Document name to be amended] Drawing

[Correction target item name] FIG.

[Correction method] Change

[Correction contents]

FIG. 10

[Procedure amendment 8]

[Document name to be amended] Drawing

[Correction target item name] FIG.

[Correction method] Change

[Correction contents]

FIG. 11

[Procedure amendment 9]

[Document name to be amended] Drawing

[Correction target item name] FIG.

[Correction method] Change

[Correction contents]

FIG.

[Procedure amendment 10]

[Document name to be amended] Drawing

[Correction target item name] FIG.

[Correction method] Change

[Correction contents]

FIG. 13

[Procedure amendment 11]

[Document name to be amended] Drawing

[Correction target item name] FIG.

[Correction method] Change

[Correction contents]

FIG. 14

──────────────────────────────────────────────────続 き Continuation of front page (81) Designated country EP (AT, BE, CH, CY, DE, DK, ES, FI, FR, GB, GR, IE, IT, LU, MC, NL, PT, SE ), OA (BF, BJ, CF, CG, CI, CM, GA, GN, GW, ML, MR, NE, SN, TD, TG), AP (GH, GM, KE, LS, MW, SD, SZ, UG, ZW), EA (AM, AZ, BY, KG, KZ, MD, RU, TJ, TM), AL, AM, AT, AU, AZ, BA, BB, BG, BR, BY, CA, CH, CN, CU, CZ, DE, DK, EE, ES, FI, GB, GE, GH, GM, HR, HU, ID, IL, IS, JP, KE, KG, KP , KR, KZ, LC, LK, LR, LS, LT, LU, LV, MD, MG, MK, MN, MW, MX, NO, NZ, PL, PT, RO, RU, SD, SE, SG, SI, SK, SL, TJ, TM, TR, TT, UA, UG, US, UZ, VN, YU, ZW. 5C059 KK19 MA05 MA23 MA24 NN08 NN15 NN28 NN37 PP04 UA02

Claims (60)

[Claims] Converting the first and second data sets into corresponding first and second sets of transform coefficients; generating data representing a difference between the first and second sets of transform coefficients; A data compression method comprising first and second data sets, comprising encoding data generated for transmission. 2. The data compression method according to claim 1, wherein converting the first and second data sets is performed using a -tensor product wavelet transform. 3. The data compression method according to claim 2, wherein the remainder from one subband is transmitted to another subband. Generating data representing a difference between the first and second sets of transform coefficients; evaluating the difference between the first and second sets of transform coefficients to provide a motion vector; Applying a motion vector to a first set of transform coefficients to generate a prediction of the set of transform coefficients, and subtracting the prediction from a second set of transform coefficients resulting in a set of prediction errors. 2. The data compression method according to 1. 5. The data compression method according to claim 4, wherein the first and second sets of transform coefficients are corrected for errors. 6. The method of claim 4, wherein applying a motion vector to the first set of transform coefficients further comprises applying a mask around each performed transform coefficient. 7. Estimating a difference between the first and second sets of transform coefficients generates a search area from one of the first and second sets of transform coefficients around a subset of the transform coefficients. Applying a relevant subset of transform coefficients from the other of the first and second sets of transform coefficients to the search region; and a relevant subset of transform coefficients incrementally to locations representing the best incremental match in the search region. 5. The data compression method according to claim 4, further comprising moving. 8. The data compression method of claim 7, further comprising: moving the relevant subset of transform coefficients in the search area fractionally laterally to a position representing a best fractional match. 9. The method of claim 1, wherein transforming the first and second data sets generates the first set of transform coefficients as a first set of subbands and the second set of transform coefficients as a second set of subbands. Item 2. The data compression method according to Item 1. 10. The data compression method of claim 9, further comprising a macroblock packing a second set of subbands to form a subband macroblock grouping. 11. The method of claim 11, wherein the subbands in the subband macroblock grouping are:
11. The data compression method according to claim 10, further comprising applying a weight to the macroblock. 12. The data compression method of claim 10, further comprising detecting a change between subband macroblock grouping and a reference. 13. The general equation of the form: detecting a change between a subband macroblock grouping and a reference comprises: 13. The data compression method according to claim 12, wherein the method is based on a distortion evaluation according to: 14. A more specialized form of the equation for detecting changes between subband macroblock groupings and a reference. 14. A data compression method according to claim 13, wherein the method is based on a distortion evaluation according to: 15. A method for generating data representative of a difference between a first and second set of transform coefficients, wherein the difference between the first set of subbands and the subband macroblock grouping to provide a motion vector. Evaluating the motion vector, applying the motion vector to the first set of subbands to generate a prediction of the second set of transform coefficients, and from the second set of subbands to obtain a set of prediction errors. The data compression method according to claim 10, further comprising subtraction. 16. Evaluating a difference between the first set of subbands and the subband macroblock grouping generates a search region from the first set of subbands around a subset of transform coefficients. Applying the relevant subset of transform coefficients from the sub-band macroblock grouping to the search region; and incrementally laterally moving the relevant subset of transform coefficients to a position representing the best incremental match in the search region. The data compression method according to claim 15, comprising: 17. The data compression method of claim 16, further comprising: moving a relevant subset of transform coefficients in the search area fractionally laterally to a position representing a best fractional match. 18. The data compression method of claim 1, wherein encoding the generated data for transmission further comprises identifying a subset of the generated data equal to zero. 19. transforming the first data set and the second data set into a corresponding first and second set of transform coefficients; and providing a motion vector between the first and second set of transform coefficients. Estimating the difference; predicting a second set of transform coefficients by applying the motion vector to the first set of transform coefficients; transforming the predicted second set of transform coefficients to obtain a prediction error with a second transform A data compression method comprising first and second data sets comprising subtracting from a set of coefficients and encoding a prediction error and a motion vector for transfer to a decoder. 20. The data compression method according to claim 19, wherein converting the first data set and the second data set is performed using a tensor product wavelet transform. 21. Evaluating a difference between the first and second sets of transform coefficients: generating a search area from one of the first and second sets of transform coefficients around a subset of the transform coefficients; Applying an associated subset of the transform coefficients from the other of the first and second sets of transform coefficients to the search region; and incrementally laterally translating the associated subset of the transform coefficients to a position representing the best incremental match in the search region. 20. The data compression method according to claim 19, comprising moving. 22. The data compression method of claim 21, further comprising: moving a relevant subset of transform coefficients in the search area fractionally laterally to a position representing a best fractional match. 23. Transformation of a first data set and a second data set generates a first transform coefficient as a first set of subbands and a second transform coefficient as a second set of subbands. The data compression method according to claim 19. 24. The data compression method of claim 23, further comprising a macroblock packing a second set of subbands to form a subband macroblock classification. 25. The method of claim 25, wherein the subbands in the subband macroblock grouping are
The data compression method according to claim 24, further comprising applying a weight to the macroblock. 26. The data compression method of claim 24, further comprising detecting a change between subband macroblock grouping and a reference. 27. A method for detecting a change between a subband macroblock grouping and a reference comprises a general equation of the form 28. The data compression method according to claim 27, wherein the method is based on a distortion evaluation according to: 28. The data compression method of claim 19, wherein encoding the prediction error and the motion vector for transport to a decoder further comprises identifying a subset of the prediction error equal to zero. 29. A first data set and a second data set corresponding to a first
And estimating the difference between the first and second sets of transform coefficients to provide a motion vector, applying the motion vector to the first set of transform coefficients. Compressing the first and second data sets, including predicting two sets of transform coefficients, and subtracting the predicted second set of transform coefficients from the second set of transform coefficients to obtain a prediction error. Data compression method. 30. The data compression method according to claim 29, wherein the first transform coefficient set is corrected for an error. 31. transforming the first data set and the second data set to generate a corresponding first and second set of transform coefficients; first and second transform coefficients to provide a motion vector. Estimating the difference between the sets, applying a motion vector to the first data set, and then predicting a second set of transform coefficients by transforming the prediction result, and transforming to obtain a prediction error. A data compression method in an encoder for reducing the number of bits transferred to a decoder, comprising subtracting a prediction result from a second set of transform coefficients. 32. The data compression method of claim 31, further comprising: inversely transforming the first set of transform coefficients and providing the first set of transform coefficients as a reference during prediction. 33. The data compression method according to claim 32, wherein the first transform coefficient set is corrected for an error. 34. separating a set of related subband blocks from a set of subbands, packing together a set of related subband blocks as subband macroblocks, and subband macroblocks. A subband block packing method corresponding to a subset of the data set, comprising repeating the above separating and packing steps for each set of related subband blocks in the set of subbands to form a grouping . 35. The method according to claim 35, wherein the packing step comprises subbands in the set of subbands.
35. The subband macroblock packing method of claim 34, comprising arranging a set of related subband blocks within the subband macroblock to the same relative position occupied by the block. 36. The packing step includes placing the subband macroblocks in the subband macroblock grouping at the same spatial location as the corresponding data subset is located in the data set. 34. A subband macroblock packing method according to claim 34. 37. Converting a data set using a tensor product wavelet having at least two filter paths, and propagating the remainder induced during the conversion between at least two of the filter paths. How to convert a data set containing a to conversion coefficients. 38. The remainder of the at least two filter paths from the first filter path is propagated to the second filter path of the at least two filter paths, and the remainder from the second filter path is propagated to the first filter path. Item 38. The method according to Item 37. 39. The method of claim 37, wherein the tensor product wavelet transform is a tensor product wavelet pair for determining a high pass component and a low pass component. 40. The transform of the data set and the remaining propagation between the filter paths determine the low-pass and high-pass components of the data set; the low-pass normalized output and the first remainder (rl) Normalizing the low-pass component to generate; normalizing the high-pass component to generate a high-pass normalized output and a second residue (rh); first and second residues Perform a first operation (g (rl, rh)) and add the resulting results to a low-pass normalized output to generate an approximation; and a first and second remainder (rl, rh). 40. The method of claim 39, comprising performing a second operation (r (r, rh)) on (rh) and adding the resulting results to a high-pass normalized output to generate details. . 41. The method of claim 40, further comprising down-sampling the low-pass and high-pass components. 42. The low-pass component is determined using a filter having the values -1, 2, 6, 2, -1; the high-pass component is a filter having the values -1, 2, and -1 And the following function a (rl, rh) = rh and f (rl, rh) = floor (rh + ／), where nh = １ ／ , Rh)) and a second operation (f (rl, rh))
40. The method of claim 39 comprising: 43. The tensor product wavelet pair: as well as 40. The method of claim 39, wherein the method is in the form: 44. A data set comprising transforming a data set utilizing an encoding technique and propagating a remainder derived during encoding from a first filter path to a second filter path of an encoder. How to encode into transform coefficients. 45. The encoding method according to claim 44, further comprising propagating the remainder from the second filter path to the first filter path. 46. The encoding method according to claim 44, wherein the encoding technique is a tensor product wavelet transform. 47. The encoding method according to claim 44, wherein the encoding technique is a discrete cosine transform. 48. determining a first filter component of the data set in the first filter path; determining a second filter component of the data set in the second filter path; generating a normalized output and a remainder. Normalizing the first filter component to perform
And propagating to a second filter path. 49. Generating a search region around a data subset from one of the first and second data sets; and providing a search region from another of the first and second data sets to the search region. Applying a data subset, and incrementally moving an associated data subset laterally to a position representing a best incremental match in the search area. How to evaluate the changes that have occurred. 50. The method of claim 49, further comprising: moving the relevant data subset laterally and fractionally to a position in the search area that represents a best incremental match. How to evaluate the changes that occurred between them. 51. A transforming device having an input configured to receive first and second sets of data, and further configured to generate corresponding first and second sets of subbands. And having an input coupled to the converter, configured to receive the first and second sets of subbands, and further efficiently representing the difference between the first and second sets of subbands. A coding device comprising the motion compensation device configured as described above. 52. The encoding apparatus according to claim 51, wherein the motion compensation apparatus performs all operations on the first and second sets of subbands in the transform domain. 53. The prediction from the motion compensator and the second sub-band from the transformer.
52. The method of claim 51, further comprising: a difference block configured to receive the set of sub-bands and further configured to determine a difference between the prediction and a second set of subbands to generate a prediction error. Encoding device. 54. A motion compensator is coupled to the transformer, and the first and second subbands are used to generate a motion vector.
A motion estimation device configured to compare the set of sub-bands and configured to receive the motion vector of the sub-band and the first set, and a second set of sub-bands The encoding device according to claim 51, further comprising: a motion prediction device configured to generate a prediction of. 55. An input configured to receive a first data set and a second data set, and further comprising a first set of sub-bands and a second set of sub-bands therefrom, respectively. A transform device configured to generate, and an input coupled to the transform device and configured to receive the first set of sub-bands and the second set of sub-bands; A change detection coding device comprising: a macroblock packing device configured to generate a band macroblock representation and a second subband macroblock representation, respectively. 56. A system configured to communicate an input with a macroblock packing device for receiving a first subband macroblock representation and a second subband macroblock representation based on perceptual significance. 56. The encoding device of claim 55, further comprising a weighting device configured to scale. 57. An input having an input configured to communicate with a macroblock packing device, wherein the first subband macroblock representation and the second subband macroblock representation are compared and between them. The encoding device of claim 55, further comprising a change detection device configured to determine a change, wherein the change detection device is further configured to generate a change detection grouping that reflects the change. 58. The apparatus of claim 5, further comprising a macroblock ranker coupled to the input change-detector and configured to order the change detection grouping.
8. The encoding device according to 7. 59. A first subband macroblock representation and a second subband macroblock representation
Comparison of macroblock expressions is a general equation 58. The encoding device according to claim 57, wherein the encoding device is based on a distortion evaluation according to: 60. A first subband macroblock representation and a second subband macroblock representation.
A more specialized form of equation comparing macroblock expressions 60. The encoding device according to claim 59, wherein the encoding device is based on a distortion evaluation according to: