JPH05509209A - Hierarchical Entropy Coding Lattice Threshold Quantization Coding Method and Apparatus for Image and Video Compression - Google Patents

Abstract

(57) [Summary] This bulletin contains application data before electronic filing, so abstract data is not recorded.

Description

[Detailed description of the invention] for hundreds and videos entropy,.

, −゛ and and U and 11 The present invention relates to a data communication and signal processing method and apparatus, and will be described in detail. For example, if the array of image data transmitted over a telephone communication channel is The present invention relates to a method and apparatus for fast and efficient encoding and decoding. of the image arrangement, specifically, a natural system, such as is exemplified by a television signal, for example. The transmission of generated image sequences continues to be the subject of considerable research. Typically, research They cut to the high redundancy of successive images of an array and image with a correlation coefficient close to unit light. Data is often modeled as a Markov process. The 3D Markov model is , differential pulse code modulation (DPCM) and transform to account for interframe redundancy. Provide motivation for utilizing encoding techniques.

By analyzing the properties of standard videos, we can identify the major changes that occur between successive frames. It is easy to understand that is the non-homogeneous motion of the object within the frame.

Accurate devices and methods for estimating and compensating for this spatially dependent motion are Significantly better than could be achieved by simply displaying the difference between consecutive frames It is possible to configure an interframe data compression method and device that can have high performance. It will also be understood that

As a result, various motion compensated encoding methods and apparatus have been developed. these The system typically uses a receiver-based motion compensation system or a transmitter-based motion compensation system. Any motion compensation system based on Receiver-based motion compensation system When in motion, the receiver makes a prediction about the motion and records the predicted motion. Compensate the previous frame. Transmitters operating in a similar manner are expected to error that describes what must be done at the receiver to correct the frame Send only a signal. The error signal is coded circularly to reduce its bandwidth.

For transmitter-based motion compensation systems, the motion estimation process occurs only at the transmitter. . Displacement vectors are usually determined over different regions of the image and this data is It is then transmitted to the receiver along with the error information data signal. At the receiver, the transmitter A compensation program is first applied to the previously encoded image using the motion information given by process is carried out. The error signal given by the transmitter is compensated in this way. It is added to the receiver image to maintain image quality.

Multiple displacement vectors typically given λ for transmitter-based motion compensation systems vectors, and a few (in at least one preferred example each vector covers a given area of the image) or associated with a block. Blocks usually do not overlap and for example 8 strokes It has a size obtained by multiplying a pixel by 8 pixels. Different methods are available for each block. Used to encode motion compensation data associated with the lock.

There are also many ways to encode error information data signals in transmitter-based motion compensation systems. It is used to. For example, U.S. Patent No. 4.727 by HLnvan ．． No. 422, a lossy compression method and apparatus is disclosed. These methods are Compression of data information is not required separately, although it provides very advantageous and impressive results. High-quality image reproduction that improves and thereby still uses less channel bandwidth It is still hoped that this will become possible. For example, associate images with provide better control over data transmission by controlling the bit rate is separately desired.

For example, during a scene or scene change, there is considerable information to be transmitted. uses insufficient bandwidth to send all of the information during a single frame time. Can not. Therefore, various methods are used to determine the bits of information transmitted through a channel. Implemented to selectively limit the number. ”A Method and Ap Paratus for Efficiently Coding and De 198 titled coding Image 5equences+ No. 4,816,914, filed January 7, 2007, One of these methods, disclosed in Quadtree or quadtree coding is used in connection with transmission. quadtree code A more graceful deterioration of the image (graceful degregation) occurs during large movements or scene changes. gradation).

The hierarchical base disclosed in U.S. Pat. No. 4.1149.810 by Erlcsson Other methods such as vector quantization provide other benefits in the encoding process.

Other encoding methods have been used to encode the source data. In detail, Lattice has been studied in papers on channel and source coding applications. this Much of this work is an encyclopedic collection of research into lattice theory and applications. It is summarized as follows. Most of the current work in data compression is based on generalized vector quantization (VQ), but several researchers have recently used lattice quantization (VQ). I should revive my interest in (researching) LQ.

Both lattice and vector quantization are used to solve multidimensional spaces with finite or countable points. Associate with a set ++ U.S. Patent No. 4,849,810 by Ercsson In the vector quantization disclosed in , there is a code table that includes a finite set of points. Consisting of learning methods or analysis. Each source vector has a certain distortion Put the "nearest" points (or code vectors) into the codebook as measured by the standard. It is quantized by placing it. For example, if the squared error criterion (L-2 norm) is can be used, but other measures are equally valid. The encoder receives an index of the code vector. The receiver continues to approximate the source vector with the code vector during reconstruction. resemble

A lattice is defined as a finite set of points and all its possible linear transformations (standard (generally speaking), but yields an infinite set of points. If the source vector is (even if that is included in the grid as measured by some distortion criterion (such as a squared error criterion). The placement of the nearest points allows quantization according to lattice quantization. The way the grid is The indices of the grid points are transmitted and received. The transmitter approximates the source vector at the grid point.

It is therefore an object of the present invention to use lattice quantization to achieve significantly smaller bandwidths and and image arrays across communication channels, providing high reliability and fidelity. It is to perform transmission. Another object of the invention is to Controls the number of bits used for transmission and allows images to be displayed during scene changes or during periods of large motion. The purpose is to provide an elegant deterioration of the image. Another object of the invention is to scan an array of Motion compensation code that reliably transmits and receives accurate estimates of displacement errors in images Encoding and decoding methods and devices, and local displacement reconstruction in image transmission devices Improved motion estimation error encoding and A decoding method and apparatus.

The present invention provides an image transmission system, more specifically, an inter-frame error in a motion compensated image transmission system. Encodes the difference data and transmits the image frame from the transmitting station to the receiving station. The present invention relates to a method and apparatus for transmitting sequences. The method uses the current image frame reduce the interframe predicted image data representing the prediction, represent the current image prediction, and Generating a predictive pyramid data structure with multiple reduction levels and Reduce the unencoded current image data to display unencoded image frames and Create a current image pyramid data structure that represents an image and has multiple reduction levels. The stages that occur and the entropy-encoded lattice threshold quantization encoding method are Bell-based predictive pyramid data structure and current image pyramid data structure to express the difference between the predicted image data and the uncoded current image data. The characteristic lies in the step of generating encoded data shown in FIG.

In another aspect, the method includes implementing a lattice encoding method using an E-8 lattice structure. This feature exists when applied to a level data structure that is lock-by-block based. In another aspect of the invention, the method comprises determining a predicted image for a current image frame. data and unencoded current image data to display the unencoded current image frame. The distinctive feature is the step of forming a difference image that displays the difference between be. This method reduces difference images and creates a difference image pyramid with multiple reduction levels. Generate a data structure and perform lattice quantization encoding on a level-by-level basis. selectively applied to the image pyramid data structure, predicting image data and unencoded current The feature is that encoded data representing the difference between the image data and the image data is generated.

The apparatus of the present invention reduces interframe predicted image data for a current image frame. , a circuit for generating a predictive pyramid data structure with multiple reduction levels, and a code Reduce the unencoded current image data to display the unencoded current image frame and A circuit for generating a current image pyramid data structure with a reduction level of Lopi coded lattice threshold quantization coding with predictive pyramids on a level-by-level basis data structure and the current image pyramid data structure to predict the image data. and the encoded current image data. It has characteristics. This device uses a block vibe encoding method using an E-8 lattice. The circuit is unique in its application to lock-based data structures.

In yet another aspect of the invention, the image frame is configured to transmit an array of image frames. The device for encoding interframe error data in the transmission system is Displaying the predicted image data and uncoded image frames for the uncoded current image A process for forming a difference image that displays the difference between image data on a pixel-by-pixel basis. Features a road. Difference molecule that reduces the difference image and has multiple reduction levels □ Generate image pyramid data structure and entropy encoded lattice threshold quantization encoding is applied to the differential image pyramid data structure on a level-by-level basis to make predictions. Generate encoded data representing the difference between the image data and the encoded current image data. A reduction circuit is provided for this purpose.

Firefly meeting day Other objects, features and advantages of the present invention will be explained in certain preferred embodiments with reference to the drawings. It will become clear from the following description of the embodiments.

FIG. 1 is an electrical block diagram of a standard image transmission system according to the present invention as set forth in the claims. It is.

FIG. 2 shows an electrical block of a transmitter of a motion compensated image encoding device using the present invention. It is a diagram.

Figure 3 shows a motion compensated image code for receiving channel signals from the transmitter in Figure 2. FIG. 2 is an electrical block diagram of a receiver of the conversion device.

FIG. 3A is a block diagram of lossy compressor 28 according to the present invention.

FIG. 4 is a block diagram of a lossy compressor 46 according to the present invention.

FIG. 5 is a schematic diagram of a one-dimensional decimation process.

FIG. 6 is a detailed electrical block diagram of lossy compressor 46 in accordance with the present invention.

FIG. 7 encodes the output of the alternative path I6 of a lossy compressor according to an aspect of the present invention, The data encoded in this way is sent to the channel 20 for transmission to the receiver 21. FIG. 2 is a general block diagram of an embodiment in which data is sent to

Figure 8 is a schematic representation of the relative placement of adjacent blocks used in next block value prediction. be.

Figure 8A is a schematic representation of the relative placement of adjacent motion vectors used in a linear predictor. be.

FIG. 9 is a probability density function divided into step bands and illustrates the centroid placement.

FIG. 10 is an electrical block diagram of a coding device according to the present invention.

FIG. 11 shows the entropy-encoded lattice threshold quantization code according to the present invention. FIG. 2 is an electrical block diagram of the encoding device.

Of... Referring to FIG. 1, according to a preferred embodiment of the present invention, a video signal is converted into an analog - camera 1 for providing to digital converter and frame buffer device 12; 0. Analog-to-digital converters and framebars The frame buffer section of the buffer device 12 stores a raster of 256 x 240 pixels. Can store a full frame of an image sampled across, say, 8 bits .

The entire encoding and motion compensation process occurs in the digital domain. . The transmitter includes an error signal circuit 14 and a motion estimation and encoding circuit 16.

Channel encoder 18 provides error circuit 14 and motion compensation and encoding rotation criteria. Taste. Hypothetically and in the illustrated embodiment, the motion estimator has a previous input and a current Non-overlapping of current input images The motion estimation and encoding circuit 16 shown in FIG. According to the preferred embodiment, the current human frame image obtained via line 22 The illustrated embodiment uses a region matching technique between blocks. If now If motion occurs in a certain area of the image, it is estimated! is any block in the previous image is the best match for the block of the current image; The displacement value provides the current encoded motion signal. As mentioned above, loss pressure This output of the compressor, upon decoding, passes through line 32 (giving a measure of the motion displacement). ) do not reproduce the signals exactly and therefore consist of an error signal associated with them. have Nevertheless, the reduction in data requirements of a lossy compressor is The use of a lossy compressor is reasonable when compared to the exact encoding method of It is a meaningful advance in the field of technology. One preferred lossy compression circuit applies Predictive pulse coded modulation (ADPCM) is used.

Referring to FIG. 3A, in the illustrated preferred embodiment of the invention, the lossy compression circuit Path 28 provides a linear predictor 28a of the motion vector field and The prediction error of the field and the number of bits required to describe the predictor (bandwidth). Additional circuitry is included to reduce the bandwidth. Rask scan pattern shown in Figure 8A 3A, linear predictor 28a was previously encoded (0 ) the four neighboring elements (obtained earlier through line 3o) to predict the current motion vector (marked with an X). then,( Additional circuitry (embodied by switch circuit 28b) provides three possible choices: make decisions based on

a) reset the estimated motion vector to zero (signal through line 188) and Send this to the receiver.

b) Set the estimated motion vector (through line 28c) to the predicted value and Send this to the receiver.

C) The prediction error (difference between the original estimated motion vector and the predicted motion vector) is quantized. quantize at 28d and send the quantization index to the receiver.

Associated with each selection are (bits N used to send that decision ) and the difference caused by the use of the chosen motion vector There is an error E of the mean squared error of each image block. The additional circuit 28b is an entropy – Constraint error measure, i.e. a weighted combination of used bits and error ( For example, a measure is equal to (aN+bE), where a and b are This determination is made using the experimentally determined constant )). minimum entro Results that produce a peak constraint error are selected and sent. The encoding process is estimated By sending one bit of information about whether to reset the motion vector to zero or not. start. If it is reset to zero, then Nothing is sent beyond this point. Otherwise, the second bit of information is sent out. , the receiver can either use only the predicted lagging vector or add additional correction information (quantized error) will be sent. Finally, if corrective information is needed For example, a quantization index of the prediction error is sent. Different number of bits for encoding To avoid loss of data, the lossy compressor uses arithmetic encoding for the first two steps. and use Huffman encoding for the last step.

As mentioned above, the output of the lossy compression circuit through line 30 is sent to encoder 8. It will be done. In addition, these signals can be detected by the receiver without any errors in the channel. is used by the error circuit 14 to determine what the , the prediction error signal, i.e., the receiver based on the encoded motion signal through line 30. Determine the signal that represents the difference between what you would have predicted based on the image input and the true image input. Provide a mechanism for

The output of the lossy compressor through line 30 is used by a reconstruction circuit 34, which At the output, a signal representing a measure of the motion displacement, the motion vector, is generated on line 32.

The difference between the signal on line 36, the output of the reconstruction circuit and the signal on line 32. The symbol represents the encoding error introduced by the lossy compression circuit 28. Through line 36 The output of the shear reconstruction device 34 operates in a spatial domain and associates each pixel with a motion displacement vector. is directed to a motion field interpolation circuit 38 that associates the motion field. Thus, the line The input signal passing through 36 is a band consisting of elements such as 4x4 blocks of pixels. While displaying motion displacements for a region or group, the motion field is The field interpolator processes that data so that there is a motion displacement vector associated with each pixel. Disassemble. The resulting output of the motion field interpolator through line 40 is the motion reconstruction signal. It is called a number.

The motion reconstruction signal is passed to a motion compensation device 42 which forms part of an error reconstruction loop 43. applied. The error reconstruction loop includes a frame buffer 44 and a lossy compression circuit 46. and a reconfiguration circuit 48. Losses through lines 22 and 51 respectively The inputs to the compression circuit are the original input image and estimated receiver signal for the current frame, That is, if there is no additional data, the receiver will reconstruct and display This is a signal from Koro. A lossy compressor 46 includes a Separately encoded data, error reconstruction to reduce and in principle eliminate differences Impart the signal to the receiver. The difference is encoded, its bandwidth and the resulting The error reconstructed signal on line 52 is sent to channel encoder 18. Supplied. The lossy compressor 46 of the Erlcsson patent mentioned above uses a gain/shape vector. is a two-dimensional block encoder that uses vector quantization, and the output of the block transform is According to the process described above in connection with lossy compressor 28, the bandwidth is separately effectively reduced. is read and encoded. However, in the preferred illustrated embodiment of the invention, the hierarchical An entropy-encoded lattice-threshold quantization encoding method and apparatus is used in a lossy compressor 46. It is effectively used when carrying out.

The error reconstruction signal is also sent to reconstructor 48, which reconstructs the loss provides the opposite operation to that imposed by sexual compressor 46. Therefore, the reconstructor An error reconstructed image of line 54 results at the output of position 48. The error reconstructed image is motion is added to the expected output of the compensator (the two estimated receiver images in line 51), and the resulting the estimated receiver (which is the predicted receiver image for the previous frame) The image is stored in frame buffer 44.

As mentioned above, the input to frame buffer 44 is the estimated receiver image. reception This receiver image takes into account all the data received by the receiver for a given frame. corresponding to the reconstructed receiver image. Output image from frame buffer on line 64 The image is determined by motion compensation circuit 42 according to the output of motion field interpolator 38 on line 40. This is an image of the correction being made. The output of motion compensator 42 is transmitted to lossy compressor 28 represents the predicted receiver image as a result of output data reconstruction from .

At the receiver 21, referring to FIG. 3, data from the channel is channel decoded. The resulting receiver error reconstruction on line 72 is decoded by The encoded motion signal representation of the receiver in line 74 and the encoded motion signal representation in line 74 are respectively transmitted to reconstruction circuit 7. 6, is supplied to the motion compensator 99 and the reconstruction circuit 78. Reconfiguration circuit 76 and 78 respectively perform decoding of the code used by the transmitter, and as detailed below. , which leads to the operations performed by the transmitter's reconfiguration circuits 48 and 34. error reconstruction The output of the component circuit 76 is such that the motion compensation signal on line 82 is sent to the error image display on line 84. is applied to a recovery loop 80 which generates a reconstructed receiver signal on line 86. . This signal is applied to a digital-to-analog conversion circuit 90 and viewed from there. It is given to the monitor 92 for listening.

The motion reconstruction signal is a motion field corresponding to motion field interpolator 38 of FIG. generated by a code interpolator 96. As mentioned above, the motion field interpolator A λ frame interpolator provides a motion vector for each pixel of the image, and a λ frame interpolator accurately predict what the image looked like at any selected time between frames. allow measurement. The reconstructed receiver image on line 86 is sequentially stored in the frame buffer 9 8 and fed to a motion compensator 9, so that the motion compensator 99 A signal is also received from a motion field interpolator 96. Expected receiver image without error correction The output of the motion compensator representing the image corresponds to the signal on line 51 of the transmitter and is supplied to adder 100 for combination with the output of error reconstruction circuit 84. The transmitter and receiver circuits of FIGS. 2 and 3 may be used, for example, in the above-mentioned US patents. 4.727.422 and 4,816.914. Can be fixed in multiple ways. These alternative embodiments of transmitter and receiver structures can be Although applicable in communication configurations, entropy-encoded lattice-threshold quantization codes The present invention, which is first disclosed and relates to an apparatus for Therefore the standard transmitter and receiver mechanisms described above, regardless of which one is used. I will only discuss the relationship with the development.

Motion field interpolators (38, 96) in transmitter and receiver circuits and transmitter circuits The lossy compressor 28 of the It is described in detail in National Patent No. 4,849,810. However, Er1css A lossy compressor 46, also disclosed in the on patent, uses its entropy-encoded lattice threshold Modified by the use of value quantization encoding methods. So the lossy compressor is here will be explained in detail.

It should be noted that the adaptive filter and motion estimator advantageously used by the present invention are the above-mentioned E Also disclosed in US Pat. No. 4,849,810 to Rlcsson.

As mentioned above, lossy compressor 46 receives as input the original uncoded code on line 22. and a signal representing an estimated receiver image on line 51. The lossy compressor 46 is These signals are used to encode these differences, and the encoding error re-signal is input on line 52. Output configuration signal. This signal is not compensated accordingly by the motion compensation system. Correct most errors.

Referring now to FIGS. 4 and 5, the estimated receiver image ("distorted") on line 51 ” image) and the original sin encoded image on line 22 is decimated. The decimation circuits 502 and 504 reduce (decimate) the number of times ( i.e., filtering and subsampling are performed as described below) . At each reduction step, the image is subsampled by two factors both horizontally and vertically. It will be done. Thus, for the luminance image, five levels of images are available for the luminance image in the illustrated embodiment. About 256X240.128X120.64X60.32X30 and 16 Obtained with a resolution of X 15 pixels. A set of images with different image resolutions is usually referred to as “resolution”. It is called the degree pyramid. The base of the pyramid is a full resolution image and the pyramid The top of the mid is a 16×15 pixel image in the illustrated example.

Similar resolution pyramids are used for the rIJ and “Q” chrominance components of color images. It is formed by However, in the following explanation, only the luminance component of the image will be explained. same Similar equipment and processing steps are equally applicable to the chrominance components of the image.

According to the lattice threshold quantizer, the sign of the image difference between the distorted image and the original sin encoded image from the top level of the resolution pyramid to the bottom level on a level-by-level basis. This is performed by encoding circuit 506. Process the video without any additional bits End at that resolution when not available for transmission. In this way, moderation (mo (derate) During exercise, the device typically has a 256 x 240 pixel That is, when the base level is reached and during large movements, the encoding can stop at the 128X120 level. Ru. Typically, during scene changes, the device transmits bits early in the pyramid. Use up or split the available list between several pyramid levels. like this Generally, there are large changes in the image or scene, but the details are put into later frames. is typically first described at a high level.

In particular, we prefer using entropy-encoded lattice-threshold quantization (EC-LTQ). According to the preferred hierarchical encoding device, the encoding is done at the top level, i.e. 16x15 pixels. Start with the statue.

16X 15 versions of distorted or background images are used for prediction. It will be done. This is generated at the receiver in the absence of any additional information (reduced ) Recall that it corresponds to images. Referring to Figure 6, this top water The quasi-prediction is subtracted from the 16×15 reduced top level image of the original image. top level The difference image representing the error in is quantized and the quantized information is transmitted to the receiver. is directed to the encoder 18. After that, the quantized difference image is 16X 15 16X 15 is added to the predicted image at the level and the receiver also generates it. At lower levels that form the reconstructed image, the predicted version of the image is formed differently. be done. According to the present invention, prediction is performed using a high-level reconstructed image and a current level distorted image. It is derived from the image as follows.

First, interpolate the higher level distorted image and subtract it from the current level distorted image. An interpolated error image is derived. The resulting distorted interpolation error image is essentially Extracts spatially high frequencies in the distorted image, i.e., information that is not present in the high-level image. put out The high-level reconstructed image is sequentially interpolated to form an interpolated reconstructed image at the current level. Ru. After Ik, the distorted interpolation error image or background image is selectively re-interpolated. added to the constituent images to generate a predicted image.

Distortion compensation, as detailed in co-pending application no. 071521.976. An error image or a background image is used, in which case it modifies the prediction. It improves, but it doesn't improve anything else. This is determined based on block by block. and the decision is sent as "side" information to the receiver.

The step of generating a difference signal at this lower level is then the same as the step at the top level. The same is true. That is, the current level predicted image is subtracted from the current level original image. The difference is quantized and transmitted to the receiver. Then the quantized difference is level is added to the predicted image to form a new reconstructed image. This procedure is done at the bottom Iterates through the resolution pyramid until a level is reached. Bottom level reconstruction image is the output image at that level, and it is the very image displayed by the decoder. This is the image.

That image is also used as described above to form the distorted image for the next frame. Ru. Reconstruction of the distorted image at the transmitter is performed by the reconstruction circuit 48 as described above. It will be done.

If all available bits have been used before reaching the bottom level , the low-level predictions are still generated in a similar manner, but without any encoding, That is, no quantized difference information is sent to the receiver. Instead, the lowest level The prediction at is added to the output at that level or the prediction as described above, and the result is The output or reconstructed image (at line 522) at the top level.

The scalar quantizer 516 used in relation to the top level prediction error is around zero. Uniform quantization system with dead zone! It is. In the illustrated embodiment, it is indicated by the symbol is encoded with 1 bit and the magnitude with 7 bits. Regarding magnitude or absolute value The threshold values (T(i)) are arranged below.

T(i)=i*T i=1.2,...,N (Formula 1) The reconstruction level (R(i)) is defined by:

R(0)=O R(i)=(i+Delta*R)i=1.2,...,N (Formula 2) Therefore, the value of X, where X is greater than T(k) but less than T(k+1) is also small, is assigned to the quantizer index value and the value R(k) is It is reconfigured at the receiver as having The quantizer may also be symmetric around zero. and set all values when the absolute value is less than T(1) equal to zero .

For the lower levels of the resolution pyramid, the predicted image is the output image from the next higher level. generated by combining the image with a similar level of distortion. In this way, prediction errors The difference is formed by taking the difference of the original image of the current water rate, the difference image is a grid threshold The encoded and quantized difference using a value quantizer is added to the prediction, Obtain an output image that is new to current standards. Y, I and Q components as three separate images be treated as such.

Considering the lower level in more detail, the predicted image is a distorted image of the current level and the output of the next level. generated by combining the distorted image from a higher level. To elaborate, distortion The interpolation error of the image is the current level distorted image 524 and (interpolated by the circuit 526) ) is generated using an interpolated version of the distorted image from the next higher level. The supplement The interpolation error is the difference between the current level distorted image and a similar image that is reduced and interpolated. It is. As mentioned above, it is the loss in the reduction that forms the next higher level image. Contains details of the distorted image. The output image from the next higher level is the interpolation circuit 527 is interpolated to obtain the current standard image. Then, the distortion interpolation error on line 528 or The background image is conditionally added to the interpolated output image by adder 530. Form a prediction. That is, for each block of 8x8 pixels, a square or The squared error is between the original image stored in 532 and the three possible predictions, i.e. the distortion The interpolated output image from the next higher level that subsumes and does not subsume the interpolation error and further It is determined between that which subsumes the background image.

Removal of distorted interpolation errors is equivalent to low-pass filtering of distorted images for prediction. be. This effective filtering process shows that it results in a significant reduction in prediction error. All blocks that provide, i.e. those blocks where motion compensation was not successful In the example shown, the weighting, for example 165, is carried out by multiplying the factors. If the "disambiguation error" created by using the distortion interpolation error is smaller than the error using ” results in a filtering process called ``. Background The use of images allows the display of non-moving objects even if these objects are briefly occluded. This is equivalent to long-term storage that can be maintained by a decoder. this bag Round frames should only be used when significant gains are realized over other choices. It should be. Therefore, the weighting is such that the factor is 20~ About 25% larger.

Ambiguity and background information is 1 or 2 bits per each 8x8 block ( 3 state) generates a word.

This is similar to the method used in motion vector field encoding and these two Bittu will “answer” the following two questions.

a) Does the device obfuscate the current (8×8) block? b) If obfuscation is carried out If not, should distortion prediction or background images be used? and For example, 1 indicates disambiguation and zero indicates disambiguation. The information is mentioned above is encoded using arithmetic encoder 534, and each word is one bit. Therefore, once the “ambiguous place map” is encoded, non-zero values cannot be encoded. There is no need to encode it.

For ambiguity information, a given arithmetic encoder 534 uses six binary variables, 32 select one of the states with the corresponding probability. Binary variables are blocked to similar levels. The four previously encoded obfuscation words for adjacent locks and the high-level adjacent Ambiguity for the next higher level block that corresponds to the contact element, i.e. the current block. It is a popular word. Thus, the encoder ensures that the disambiguation at the − level propagates to lower levels. We do not make explicit use of the fact that reflected in the probabilities for various states with elements.

The prediction error itself is encoded by a lattice quantizer 536. In this way, each water In the standard, the Y, 1 right and Qmm acid components are treated as three separate images. Each level Each different image generated at each time is thus divided into blocks of 4x2 pixels. be divided. Each block becomes a "vector" and first determines the nearest point This point is then determined for lattice quantization, as detailed below. more encoded.

The residual level is such that the image data is encoded in the preferred embodiment as described below. except that the use of procedures applied at the (30X32) level and element 524.5 26.527 and adder 538 and elements 530.534 and 536 are equal pieces. Can be encoded by use.

The disclosed entropy-encoded lattice-threshold quantization is described in U.S. Patent No. 4.8 to Erlcsson. 49,810. Vector quantization (VQ) code Comparing the entropy-encoded lattice-threshold quantization encoding and the entropy-encoded lattice threshold quantization encoding, we From the theory of Depending on the performance of know that the target boundary is approaching. However, the complexity of the vector quantization process also limits A clever way to evolve without boundaries and obtain good performance even at moderate complexity. The lattice quantizer that must be used, which is optimal according to Shannon's idea, is the one according to the present invention. Previously, only a small number of sources and dimensions were known. For example, hexagonal The shaped grid is optimal for independent, equally distributed, uniform sources in two dimensions. So This shows that a rectangular grid works well for independent, equidistributed Labrasian unification. I also understand.

In general, a lattice with a high spherical packing density provides a high encoding rate and efficient performance. If entropy coding is used for index transmission, it is possible to It can be seen that it works well. at a low rate and disclosed in this application Achieving good performance for sources with such memory is elusive. That's the goal.

entropy one According to the present invention, the optimal This performance can only be obtained by using a multidimensional grid. The preferred encoding structure is It is called an entropy-encoded lattice threshold quantizer (EC-LTQ) and is shown in Figure 10. 9 A preferred embodiment of the present invention is an 8-dimensional lattice with the highest known sphere packing density. , using an E-8 grid.

Vector quantum as disclosed in U.S. Pat. No. 4,849,810 by Erlcsson The encoder converts the encoder to the decoder and the decoder to the decoder during the training period. Optimize sequentially for the encoder, typically using separate encoding such as entropy encoding. (possibly) by resulting in a locally optimal (though not globally) are designed for some given probability density function (pdf) (although not measured). It will be done. On the other hand, most lattice quantities only have memoryless probability density functions. The simplest one-dimensional lattice (Z-1 or uniform quantizer) works best, but ) approaches the G15h-Pierce performance boundary of the scalar quantizer. It turns out that it is even possible to achieve this, with very good performance. , the use of the centroid of each Voronoi region for the reconstructed values of the quantizer (Fig. ) and the combination of a uniform threshold quantizer and an efficient entropy code (Fig. ) is obtained. The predetermined strain-rate operating point depends on the step size of the quantizer. Fixed.

The entropy-encoded lattice threshold quantizer of the present invention is a vector-based uniform threshold quantizer. Generally applicable to cases. As mentioned above, according to the present invention, the E-8 grid and a separate optimal decoder is designated for that purpose.

The encoder output is encoded by using a neighborhood optimal entropy code. The decoder is Given the source probability density function and strain measure, vOronoi of each grid point is It consists of a countable set of centroids, one centroid per region.

The performance of the lattice threshold quantizer is similar to the G15h-Pierce boundary of the scalar quantizer. between Shannon's operating strain-rate boundary. Entropy-encoded lattice threshold quantization Although the device is incorporated for either lattice, the entropy-encoded lattice threshold The performance of value quantization is relatively independent of the given rating used and is therefore the simplest n-dimensional lattices, even n-dimensional cubic (or Z-n) lattices, are more complex and denser. It can function as well as a high E-8 or 24-dimensional Leech grating.

Even if entropy-encoded lattice-threshold quantization achieves the distortion-rate performance of vector quantization, (and prefer entropy-encoded lattice-threshold quantization to vector quantization) There are reasons for this. First, the conventional vector quantization code table is finite. So, there exists a hypersphere that encompasses all code vectors. outside the hypersphere The source vectors present typically have low probabilities and the average vector quantization They add little to distortion, yet they rarely occur. appears in the output of the decoder as an easily perceptible and obtrusive artifact. For example, in image coding, vector quantization overflow can cause instantaneous distortion. code leads to easily perceptible localized obfuscation or coarse graining, which affects speech coding. , it causes momentary but audible distortion. The lattice quantizer is Even a source vector with a particularly large energy still remains near a point on the lattice. Since it is arranged as follows, it does not have any overload area.

Lattice quantizers are particularly useful for source coding systems that require occasional high rates. improve the sensing performance of the

Second, there are fast encoding methods for lattices and other methods especially for high sphere packing densities. For gratings with degrees (e.g. E-8 and 24-dimensional Leech gratings) It is packed. These methods find a single vector in the grid closest to a given source vector. Perform optimal encoding by arranging points. In contrast, vector quantization Most fast encoding methods rely on suboptimal search of a finite codebook for vector quantization. The codebook itself is typically either require significantly more memory than is needed to store be.

Third, assuming we have a good predictor that removes a significant amount of memory from the source, The source model (though not necessarily i, i, d) peaks of the quantizer Converting to a probability density function optimally encodes the most probable source vector. only to eliminate complexity and memory and for small isolated values of frequency Very good distortion-rate performance can be achieved by leaving a simple suboptimal encoding technique obtained by a Lopi coded lattice threshold quantizer. This is a real-time view of disclosures. Important for practical implementation of entropy-encoded lattice-threshold quantization such as video compression. It is. The preferred embodiment of the present invention is a novel system that accomplishes this for the E-8 grid. show the standard method.

In a preferred embodiment of the invention, entropy-encoded lattice-threshold quantization is performed using closed-loop The entropy-encoded lattice threshold of the disclosure, although applied to the RAMID image compression method. Quantization techniques include open-loop pyramid image compression, conventional subband and transform surfaces @ and any other hypothetical compression technique using speech coding and vector quantization. It can equally be applied to techniques.

Entropy for E-8, 1 value Preferred entropy-encoded lattice-threshold quantization for hierarchical compression according to the present invention The examples use an E-8 grid and the apparatus and method of the invention is applied to this grid. A method for efficiently implementing entropy-encoded lattice threshold quantization is provided.

The E-8 lattice is composed of one D-8 lattice and the vector 0.5★ (11111111). another D that is translated and the index is an entropy code and consists of a combination with an index-8 lattice. D-8 lattice has an even total It consists of a total of eight dimensional vectors with integer elements. For example, (1-300 0000) is a point on the D-8 lattice, (11100000) is not, Since the transformation vector also has an even sum, the components of all points in the E-8 grid are The sum is an even number. Note also that the E-8 lattice contains all zero vectors. sea bream.

All points in the E-8 lattice are defined by the value of their L-2 norm (sum of squares). Alternatively, they can be grouped into shells, where the number of shells increases with the L-2 norm. Ru.

An all-zero vector is the only grid point with a norm of zero, and This constitutes shell 0. There are 240 points with L-2 norm of 2.0, which These points constitute shell 1. 2160 pieces each with L-2 norm of 4.0 The point is in shell 2 (and so on). For example, the standard pyramid bidet mentioned here For coding applications, the probability density function is very “spiky” and the source vector More than 90% of the cells are in shell 0. 10% of the remaining amount, roughly 80% to 90% or so are in shell 1 and shell 2, and the efficient encoding of shell 1 and shell 2 is the value value, but allows simpler methods to be used for shells 3 and above. make it look like

The entropy-encoded interlattice (if childization process is performed in five steps. quantize the base vector to the nearest grid point and sign the grid point to a single grid index. To identify the closest one to the encoded spectrum, we furthermore, if round If the error Rn is not negative (≧○), it is the same whether the coordinates are rounded or truncated. You can get similar values. If Rn is negative, the truncated value is less than the rounded value It is l. Combining these facts, the odd-evenness of the sum of rounded and truncated coordinates is, if R1-R2''...-R8≧0 (here (') means exclusive OR operation). taste), then only then is it the same.

If the exclusive OR operation yields a negative value, the parity of the rounded coordinates is that of the truncated coordinates. The opposite is true. Oddity in both cases of rounding and adding 1/2 to the truncated value is the truncated value This can be checked by calculating only the sum of and R1''R2-...-R8.

Finally, the largest absolute value error obtained by truncating and adding 1/2 is due to the rounding action. occurs for the coordinate with the smallest absolute value error obtained by . sign of error To move to the coordinates of the nearest grid point, rIj must be added or subtracted. Indicates whether or not the If the coordinates are changed, its corresponding error term (Rn or Tn) must also be updated as follows. If the rounding coordinates change If it becomes Rn (new) = 2★ [Rn (old)] - 1 On the other hand, if the truncated and 1/2 addition coordinates are If it changes, its corresponding effect on Rn is Rn (new) = 2★ [Rn (old)] It is.

The E-8 lattice point closest to the index and the 1st order of hash vectors Since we have found that the point must be clearly identified to the receiver, we have the 8th order Bears in the original block (where each bear is a 4X4 block consisting of pixels) ), we first create one bit for each 4x4 block. generate an array of bits, in which both grid points corresponding to the block belong to shell O. then the bit is zero and one or more lattice points belong to a shell other than shell O. If so, the bit is 1. In this array, the highest resolution level, i.e. At the 256X240 level, isolated items are set to zero and "popping" Limit the effect. Arithmetic encoding is used to encode this information.

For each remaining ``1'' in the array, we The number of legs is jointly Huffman encoded. Most source vectors are shell 1 or encodes into shell 2, so we have four cases: shell 0, shell 1, shell 2 and shell 3. or more. We have already determined that at least one grid point is outside the crotch O. side and we know there is a possibility of 15, so we have 15 hafmas. Use a table of sign codes.

We then explain how the receiver finds a given point in the identified shell. state Since there is only one point in shell O, we cannot send any lattice index. There is no need to be If the point is in shell I or shell 2, the procedure described below Identify a given point for , and use symmetry to find the associated centroid and probability distribution. Used to reduce memory required. Only 920 centroids are shell l and shell 2 for 2400 points. The remainder is an array. similar center of gravity Conveniently, grid points with columns have similar source probabilities and 920 Huffman codes (preferably only the best entropy encoding method) is stored. The receiver knows the number of shell lattice points. Therefore, the preferred implementation is effectively two sets of Huffman code tables, one table for shell 1. 120 codes and the other for shell 2 contains 800 codes, (one table from each set shows the energy usage step of the source vector) Selected by calculating ratios to dimensions. This allows us to apply the grid points ). Finally, if the point is in shell 3 or higher , we usually calculate the points by jointly Huffman encoding the bare data. clearly transmits the coordinates of

According to the illustrated embodiment, the energy of a lattice point can be calculated and its shell identified. Ru. Each 8-dimensional grid point corresponds to a 2 column by 4 row block of pixels. shell 3 or more, with sufficient inter-column correlation to adjust the jointly encoded coordinate bear. There is. If we specify the spatial relationship of the grid points as shown below, ×0×4 ×1×5 ×2×6 ×3×7 Bear points (xQ, x4), (x1, x5), (x2, x6) and (x3, x7 ) are jointly encoded. (If the bear exceeds a certain dynamic range , they are encoded separately. In addition, the range of coordinates is strictly limited and (It is also necessary to limit the dimensions of the Huffman table that can be used.) The receiver will be explained later. Similarly, the grid points are reconstructed by taking the product of the grid coordinates, step size and damping factor. configured.

Shell 1 and Shell 2 are encoded using Huffman codes based on calculated probabilities. straight The direct implementation method uses 240q 8-dimensional centroids and 2400 Huffman codes as 2 2400 points in one shell, but the symmetry between the points Attention saves a considerable memory element. To elaborate, we know that half of the points are Note that it is the opposite of half. We have a bear with opposite polarity (i.e. Note that this exhibits polar symmetry). Much of the rest is below, columns, rows. A mutual exchange formed by an exchange or an exchange of both columns and rows.

XO×4 ×4 XO×3 ×7 ×7 X3×1×5 or ×5×1 or × 2×6 or ×6×2x2 x6 x6 x2 xl x5 x5 xIX3 X7 X7 X3 XOX4 X4 XO (original) (column exchange) (row conversion) ( (both exchanged) This is the average reflective residual proposed by Baker and Budge. It is used to reduce the codebook size of the residual vector quantizer.

We multiply the lattice points by the inverse generation matrix of E-8 and then calculate the modulo 2 of each component. The 8-bit group number (GN) of the grid point, which is the number found by taking Identify these symmetries by calculating. The matrix is as follows.

, so even if you multiply any one of them by a matrix, the value will be square norm Among the 256 possible group numbers (GN), with only one nonzero component equal to ( (having a value of zero) - the number of groups all correspond to the lattice points of shell 0 of O, and 120 are Each corresponds to two of the 240 grid points in shell 1 and the remaining 135 in shell 2. Corresponds to 2160 points. − Grid points and their opposites give rise to similar numbers of groups. It is clear that Thus, each of the 120 group numbers for shell 1 This corresponds to one grid point and vice versa. Closer inspection reveals that shell 2 Each of the 35 group numbers corresponds to exactly 16 grid points, of which 8 reveals that is the opposite of the other eight. Each group of eight also The columns of the 0 matrix are mutually orthogonal and therefore the following method can be used to determine them. Then form an 8x8 matrix using each point a (i) T (i = O11, ..., 7) (here, T denotes the transformation matrix, and the vector a(i) is the i-th 8th order is the original grid point). (assuming that the points are orthogonal and have similar norm) Ru. Finally, it is related by a type of symmetry to lattice pairs with opposite polarities. yields a vector with The following quadratic form yields an index between 0 and 7. Cheating.

C★ [12345678] 1 a (4) T l a (i) where C is any is the reciprocal of the square norm for point a(i). This calculation is performed in advance with C and [23 4s　The multiplication of the 6781 row vector and the 8x8 matrix can be simplified by combining them. , yielding a single 8-dimensional row vector. The result of the dot product is divided by 0 (a unit between -8 and 8). One index. Thus, only 135 row vectors are located at the grid points of shell 2. It suffices if it is stored accordingly. Of the 135 groups in shell 2, 65 of them The lattice points simply share polar symmetry, and among the other 70 group numbers, the other 4 Are there only four points that share exactly one type of exchange symmetry in addition to polar symmetry with one? (either rows or columns or row/column exchange). number of groups Knowing which of these symmetries (if any, the receiver has a similar probability) , we have one centroid and one centroid for each polarity and symmetry. Only two probabilities are needed. We have for the 920 centroids and the total probability , 120 for shell 1 and (65*8+70*4)=800 for shell 2. I need.

The entropy of the index and ′.

We define the lattice points of shell 1 and shell 2, respectively (as known in the art) Entropy encode that group number (using Huffman encoding) and polarity of the points by sending 1 bit (with entropy 1) for If the point 3 or more if it belongs to shell 2 (known immediately to the receiver from the group number) identifies which of the eight points (of that polarity) the bit is.

These three bits can be calculated at the transmitter from the dot product operation described above. If the points are exchanged vs. With symmetry, one of these three bits has full entropy.

The decoder knows from the group number how many additional bits are needed. , hence the code is comma-exempt, i.e. the decoder uses separate information (bit The code is therefore efficient.

Vector At the receiver, the vectors are the grid coordinates or associated centroids and the step dimensions and attenuation factors. It is reconstructed by taking the product of

mark J 已 i ball Entropy-encoded lattice-threshold quantization works across different source distributions and bit rates. In this application, entropy-encoded lattice-threshold quantization is used to operate is used for hierarchical pyramid video encoding, and the centroid of shell 1 or 2 is only used at the three highest resolution levels of the pyramid. Furthermore, the statistics are different. Also, at any one level, the source probability density function is As the step size changes, the deviation tends to change. For example, if If the force energy is large compared to the step size, the probability distribution over the grid points is , much wider than when the ratio is small. To grapple with these changes, some The input energy of the level for the square of the step size of the level is determined. is determined by thresholding the ratio of In a preferred implementation, the device A set of entropy code tables is stored for each type. Furthermore, the reconstructed value of the vector The attenuation rate applied to the It changes with the shell of the place. The reconstruction is thus given by:

(Reconstruction) = (center of gravity) ★ (step dimension) ★ Attenuation - factor (step dimension) ★Attenuation - table (shell, level, type) Here, the damping factor changes with the step size, as shown in Figure 11. Ru. Attenuation - Another attenuation indicated by the table is the given shell to which the grid point belongs, the encoded take a certain value according to the level in the pyramid that is being developed and the species currently in force. Ru. For grid points of shell 3 and above, the centroid used in this formula is It is the child point itself. These attenuation values are adjusted to create "popping" in the reconstructed signal. Limit the artificial structure of. In general, the cumulative attenuation reduces the reconstruction value to the grid point Vor is not large enough to be moved outside the onoi area, as defined herein. The given function value established by the relationship is a constraint on regression analysis and some heuristics. and have to be determined in collaboration with perceptual experiments; is implemented as a linear function of the curve and each part, but generalizes to a continuous functional relationship. I can do it.

L1 stop l Since only one set of centroids is used together with entropy-encoded lattice threshold quantization, the sort The data is normalized by the step size, making its probability density function a more fixed set. According to the illustrated embodiment, the appropriate step size is The method used to estimate, the "whitening rule", is a bit allocation method. child The method is described in earlier patent Erlcsson U.S. Pat. No. 4.849.810. It can also be used for closed-loop five-level hierarchical pyramid vector quantization. This person The method calculates the number of bits used at each of the three high-resolution levels of the pyramid by Bits★★3 +d) 2. Closed-loop estimation: Open-loop estimation S, (0) is determined sequentially by regression method. is modified by the multiplication parameter C'.

5c(0)(n)=c’(n)★S,(0)(n)c’(n) =c′(n-1)★exp(h*diff-bits(n-1))where , n is the frame index and diffjits(n-1) is the frame ( n-1), and h is a constant It is a number.

In the preferred implementation, the difjbits process performs a low-pass filter operation. be exposed.

Five constants, a, b, c, d and h, must be determined experimentally. formula can be combined into one, reducing the number of power exponents.

5c(0)(n)”c′(n-1)★5qrt(Energy(n))★e xp (a★bits★★3÷ b *bits(n)★★2+c *bits[n)+d+h *difjbi ts(n-1)) In summary, according to a preferred embodiment of the invention, at the top level, the device Use childization. Next level, 32X30* quasi for Y component, chrominance 15x16 levels for the components, and the distortion interpolation error is selectively interpolated from the top level. At this level, obfuscating information is transmitted. place In a preferred embodiment of the present invention, any image information, i.e., output image error, The predicted image core, which is also not encoded in the illustrated embodiment, is the output from this level. Ru. At the three bottom levels, all of which are treated equally, adaptive disambiguation behavior is used when generating the predicted image, and the prediction error is encoded using lattice threshold quantization. It will be done. This is illustrated schematically in FIG.

In the illustrated example, a situation, such as a scene change, is This can occur when even the code cannot be encoded. In this case, the encoding is Next higher level (60X64 for luminance and 30X32 for chrominance) ) to stop. If encoding at this level is the desired number of bits per frame If you generate more bits than The output is temporarily slow due to the large number of bits generated. two bottom levels For this purpose, predictions are generated normally and disambiguation information is transmitted, but with a grid threshold amount Childhood is not carried out.

Referring to FIG. 3, at the receiver, the transmitted and encoded data is decoded. and a new frame has occurred. In detail, it represents a resolution pyramid. The data is decoded by the reconstruction circuit 76 level by level from the top of the pyramid to the bottom. be done. At the top level, the quantized difference image is decoded (selectivity (in adder 100) is added to the distorted image at that level. As a result, the top The output image is reconstructed at the unit level. The lower levels are sequentially processed, starting with the reconfiguration circuit 76. The transmission obfuscation information obtained at line 84 is used to form a prediction and sequentially calculate the difference image and selectively add it to the predicted image to create a new value for that level. is reconstructed (by adder 100) by forming a reconstructed image. Professional Seth reaches bottom level and bottom water 11! The image of Wataru is the display frame bag. The lossy compressor 4 continues until it is transferred to the Arithmetic used in motion transformation coefficients, decoding of disambiguation information and image information, from 6 The decoder operates according to what is well known in the art. A non-zero location is code by using different probabilities depending on which state the encoder was in. Now that the arithmetic decoder has decoded, the arithmetic decoder uses the map ) for each position. Combination with encoded data In each map location, the state sequentially determines whether or not zero is used for each map location. Decide whether Once the map pointing to the location of non-zero values is decoded, 8 The bit value is decoded, incremented by 1, and placed at the appropriate location on the map. Ru.

Examining in more detail the occurrence of the resolution pyramid and its decoding, the process is A reversal of the method used in lossy compressor 46 follows. Thus, receiver decoding Since the process follows the transmitter decoding process, the prediction at the top level is distorted generated from a partial level image. The quantizer index is determined by using an arithmetic decoder. is decoded and the quantized difference image is reconstructed from the quantization index and (adder 522a and transmitter line 522) (corresponding to the output) gives the top level output image. At the lower level, the prediction is Formed by selectively adding distortion interpolation errors to the interpolated output image from the level (corresponding to transmitter adder 530). That output image, at the next output level The distorted image is interpolated to give an image at the current level. The obfuscation information is stored in the arithmetic decoder. is decoded by using and sequentially where the obfuscation code is zero for it. In each 8x8 block of interpolated high level output, the current level distorted image and the interpolated high level The differences between the level-distorted images are added together (corresponding to transmitter adder 538).

According to a preferred embodiment of the invention, at a low level, the lattice threshold quantization information is be done. Each 4x4 block with non-zero map bits has two 4x 0 centroid decoded by first determining the number of shells in the 2-lattice lattice vector are sequentially applied to the shell O (and for each lattice scaled by the damping value) is found. The output image is created by adding this reconstruction difference to the prediction. It is then formed. (this corresponds to the operation of adder 540 in the transmitter), the last The output image from the bottom level in the reconstructed image is sequentially displayed as the final output image. transferred to the ray frame buffer.

The illustrated lossy compressor 46 may be implemented in hardware, software, or can be implemented by a combination of them.

The disclosed hierarchical entropy encoded lattice threshold quantization encoding method and apparatus utilizes motion compensation. It can also be used advantageously in connection with the transmission of an array of images without compensation. this For example, if λ, the estimated receiver image of line 51 in FIG. 2 is the frame of line 64. It is designated as the frame output of buffer 44. In addition, the input image of line 22 The estimated receiver images on lines 51 and 51 must be individually reduced by lossy compressor 46. It is not necessary and can be used as an input to the differential circuit 720 as shown in FIG. , its output representing the error signal at the receiver is in turn subjected to the reduction and reduction according to the invention described above. and a hierarchical lattice-threshold quantization encoding operation can be input to a lossy compressor. can. In the receiver, the motion compensator 99 and its associated circuitry are If not used, it will be removed as well. Similarly, the reconfiguration circuit 76 Error image display of line 84 when the circuit of FIG. 7 is used. Reconfigure. These modifications will be obvious to those skilled in the art.

Entropy-encoded lattice-threshold quantization is used for multidimensional data with uniform vector quantization. It can also be used in encoding other arrays of vectors. Elaborating, the above simplification can be adopted for such data, and consider the probability density function for vector data. It will be corrected.

Additions, deletions, substitutions and other modifications in certain preferred embodiments of the invention will be obvious to those skilled in the art. and these are the inventions set forth in the claims below. It should be included within Ming's technical thought.

FIG.7 Summary book In order to transmit the array of image frames from the transmitter (8) to the receiver (21), the image The transmission system, to be more specific, is a motion compensated image transmission system that encodes interframe error data. The apparatus and method uses rank-order entropy-encoded lattice-threshold quantization (46); Increase data compression of transmitted images. The method and apparatus provide interframe prediction images. reducing (502) L the image data and unencoded current image data (504); Hierarchical entropy encoded lattice threshold quantization encoding (506) resulting in a pillar Applies to mid-data structures. Lossy coding applied on a level-by-level basis encoded data representing the image difference between the predicted image data and the uncoded original image. generate data. The method and apparatus may be used with or without motion compensation. It can be applied to array transmission systems for image frames (or other pattern data such as audio). Ru.

international search report

Claims (58)

[Claims] 1. In order to transmit an array of image frames, the image transmission system uses inter-frame error data. A method of encoding interframe predicted image data for the current image frame. A stage that reduces data and generates a predictive pyramid data structure with multiple levels of reduction. floor and reducing unencoded current image data representing an unencoded current image frame; generating a current image pyramid data structure having the plurality of reduction levels; , hierarchical entropy coding lattice threshold quantization coding, based on selected levels to the predicted pyramid data structure and the current image pyramid data structure. encoding to display the difference between the predicted image data and the unencoded current image data. and generating interframe error data. 2. In order to transmit an array of image frames, the image transmission system uses inter-frame error data. A method of encoding interframe predicted image data for the current image frame. A stage that reduces data and generates a predictive pyramid data structure with multiple levels of reduction. floor and reducing unencoded current image data representing an unencoded current image frame; generating a current image pyramid data structure having the plurality of reduction levels; , a hierarchical lattice threshold quantization encoding is applied to the prediction pyramid based on the selected level. applied to the predicted image data structure and the current image pyramid data structure. generates encoded data that displays the difference between the image data and the unencoded current image data. A method for encoding inter-frame error data, comprising the steps of: 3. The application step includes: applying the hierarchical encoding to the data structure on a block-by-block basis; , Predicted image data appropriately maps block portions of the current image at the level of the pyramid structure. Claims comprising the step of obscuring the block of difference display when the block fails to display the difference. 1. The interframe error data encoding method as set forth in item 1 below. 4. The application step includes: using an arithmetic encoding and partially generating said encoded representation. The inter-frame error data encoding method according to scope 1. 5. applying said encoding to a level of said data structure on a block basis; stages and Move the block location boundaries from frame to frame in said array of image frames. , improving the coding efficiency. Difference data encoding method. 6. Claims comprising the step of applying an E-8 lattice for said lattice threshold quantization. 1. The interframe error data encoding method as set forth in item 1 below. 7. The application step includes: Subtract the predicted image data structure from the current image data structure at the top level and The step of generating a quasi-output image and combining the predicted image data at a lower level and the next higher level. By taking the difference between the interpolated predicted image data, the distortion interpolation error data is calculated at a low level. the step of forming a data structure and the interpolated output image of the next higher level and the distortion at the lower level; By combining with the song interpolation error data structure, each lower level can be predicted. and forming a measured image. data encoding method. 8. The prediction forming step interpolates the output image from the next higher level and generates the interpolated output image. 8. An interframe error data code as claimed in claim 7, comprising the step of generating an image. method. 9. The lattice quantization and encoding are performed at multiple levels at the bottom of the pyramid data structure. a step of applying only to the predicted image error in At the next highest level, only the disambiguation information is transmitted, and at the top level of processing, the scalar quantum The inter-frame error according to claim 7, further comprising the step of transmitting only digitized data. Data encoding method. 10. To transmit an array of image frames, the image transmission system uses interframe error data. In a device for encoding data, an interframe predicted image data is used for the current image frame. generate a predictive pyramid data structure with multiple reduction levels. means and reducing unencoded current image data representing an unencoded current image frame; means for generating a current image pyramid data structure having the plurality of reduction levels; , hierarchical entropy coding lattice threshold quantization coding, based on selected levels to the predicted pyramid data structure and the current image pyramid data structure. encoding to display the difference between the predicted image data and the unencoded current image data. and means for generating data. 11. The application means include: means for applying the hierarchical encoding to the data structure on a block-by-block basis; , Predicted image data appropriately maps block portions of the current image at the level of the pyramid structure. Claims comprising a means for obscuring the block of difference display when the block fails to display the difference 11. The interframe error data encoding device according to item 10. 12. The application means include: Claim 1 further comprising arithmetic encoding means for partially generating said coded representation. The interframe error data encoding device according to item 0. 13. applying said encoding to a level of said data structure on a block basis; means and Move the block location boundaries from frame to frame in said array of image frames. , and means for improving encoding efficiency. Error data encoding device. 14. Claim comprising means for using an E-8 lattice for the threshold quantization encoding. The inter-frame error data encoding device according to item 10. 15. The application means include: Subtract the predicted image data structure from the current image data structure at the top level and A means for generating a quasi-output image and a method for combining predicted image data at a lower level with the predicted image data at the next higher level. By taking the difference between the interpolated predicted image data, the distortion interpolation error data is calculated at a low level. means for forming a data structure, interpolated output image of the next higher level and distortion at the lower level; By combining with the song interpolation error data structure, each lower level can be predicted. The inter-frame error data set according to claim 10, further comprising means for forming a measured image. data encoding device. 16. To transmit an array of image frames, a motion-compensated image transmission system In a method of encoding error data, Reduce inter-frame predicted image data for the current image frame, with multiple reduction levels generating a predictive pyramid data structure having: reducing unencoded current image data representing an unencoded current image frame; generating a current image pyramid data structure having the plurality of reduction levels; , hierarchical entropy coding lattice threshold quantization coding, based on selected levels to the predicted pyramid data structure and the current image pyramid data structure. encoding to display the difference between the predicted image data and the unencoded current image data. and generating interframe error data. 17. The application step includes: applying the hierarchical encoding to the data structure on a block-by-block basis; , Predicted image data appropriately maps block portions of the current image at the level of the pyramid structure. Claims comprising the step of obscuring the block of difference display when the block fails to display the difference. 17. The interframe error data encoding method according to item 16. 18. The application step includes: using an arithmetic encoding and partially generating said encoded representation. The inter-frame error data encoding method according to scope 16. 19. applying said encoding to a level of said data structure on a block basis; stages and Move the block location boundaries from frame to frame in said array of image frames. , and improving the encoding efficiency. Error data encoding method. 20. The application step includes: Subtract the predicted image data structure from the current image data structure at the top level and The step of generating a quasi-output image and combining the predicted image data at a lower level and the next higher level. By taking the difference between the interpolated predicted image data, the distortion interpolation error data is calculated at a low level. the step of forming a data structure and the interpolated output image of the next higher level and the distortion at the lower level; By combining with the song interpolation error data structure, each lower level can be predicted. and forming a measured image. data encoding method. 21. The prediction forming step interpolates the output image from the next higher level and the interpolated output 21. Interframe error data according to claim 20, comprising the step of generating an image. Encoding method. 22. The entropy-encoded lattice threshold quantization and encoding is performed on the pyramid data applying only to predicted image errors at multiple levels at the bottom of the structure; At the next highest level, only the disambiguation information is transmitted, and at the top level of processing, the scalar quantum 21. The interframe error according to claim 20, further comprising the step of transmitting only encoded data. Difference data encoding method. 23. To transmit an array of image frames, a motion-compensated image transmission system In a device for encoding error data, Reduce inter-frame predicted image data for the current image frame, with multiple reduction levels means for generating a predictive pyramid data structure having; reducing unencoded current image data representing an unencoded current image frame; means for generating a current image pyramid data structure having the plurality of reduction levels; , hierarchical entropy coding lattice threshold quantization coding, based on selected levels to the predicted pyramid data structure and the current image pyramid data structure. encoded data that displays the difference between the predicted image data and the encoded current image data. and means for generating interframe error data. 24. The application means include: means for applying the hierarchical encoding to the data structure on a block-by-block basis; , Predicted image data appropriately maps block portions of the current image at the level of the pyramid structure. Claims comprising a means for obscuring the block of difference display when the block fails to display the difference 24. The interframe error data encoding device according to item 23. 25. The application means include: Claim 2 further comprising arithmetic encoding means for partially generating said coded representation. 3. The interframe error data encoding device according to item 3. 26. applying said encoding to a level of said data structure on a block basis; means and Move the block location boundaries from frame to frame in said array of image frames. , and means for improving encoding efficiency. Error data encoding device. 27. The application means include: Subtract the predicted image data structure from the current image data structure at the top level and A means for generating a quasi-output image and a method for combining predicted image data at a lower level with the predicted image data at the next higher level. By taking the difference between the interpolated predicted image data, the distortion interpolation error data is calculated at a low level. means for forming a data structure, interpolated output image of the next higher level and distortion at the lower level; By combining with the song interpolation error data structure, each lower level can be predicted. and a means for forming a measured image. data encoding device. 28. To transmit an array of image frames, the image transmission system uses interframe error data. In a method for encoding data, The predicted image data for the current image frame and the uncoded current image frame Displays the difference between the unencoded current image data to be displayed on a pixel-by-pixel basis. forming a difference image; a difference image pyramid data structure that reduces the difference image and has multiple reduction levels; The step of generating the field and the hierarchical entropy coding lattice threshold quantization coding are selected. The predicted image is applied to the difference image pyramid data structure based on the level generate encoded data representing the difference between the data and unencoded current image data A method for encoding inter-frame error data, comprising the steps of: 29. forming the predicted image data using interframe motion compensation; 29. The interframe error data encoding method according to claim 28. 30. The application step includes: applying the hierarchical encoding to the data structure on a block-by-block basis; , Predicted image data appropriately maps block portions of the current image at the level of the pyramid structure. Claim comprising the step of obscuring a block of predicted image display when the predicted image display fails to display. 29. The interframe error data encoding method according to item 28. 31. The application step includes: using an arithmetic encoding and partially generating said encoded representation. The inter-frame error data encoding method according to scope 28. 32. applying said encoding to a level of said data structure on a block basis; stages and Move the block location boundaries from frame to frame in said array of image frames. , and improving the encoding efficiency. Error data encoding method. 33. The application step includes: Between the difference image data at a lower level and the interpolated reconstructed difference image data at the next higher level forming an interpolation error data structure at a lower level by taking the difference between 29. The interframe error data encoding method according to claim 28. 34. To transmit an array of image frames, the image transmission system uses interframe error data. In a method for encoding data, The predicted image data for the current image frame and the uncoded current image frame Displays the difference between the unencoded current image data to be displayed on a pixel-by-pixel basis. forming a difference image; a difference image pyramid data structure that reduces the difference image and has multiple reduction levels; The step of generating the field and the hierarchical lattice threshold quantization encoding are based on the selected level. The predicted image data and the uncoded image data are applied to the differential image pyramid data structure. generating encoded data representing a difference between row image data. Inter-frame error data encoding method. 35. To transmit an array of image frames, the image transmission system uses interframe error data. In an apparatus for encoding data, The predicted image data for the current image frame and the uncoded current image frame Displays the difference between the unencoded current image data to be displayed on a pixel-by-pixel basis. means for forming a difference image; a difference image pyramid data structure that reduces the difference image and has multiple reduction levels; A hierarchical entropy-encoded lattice-threshold quantization encoding is selected. The predicted image is applied to the difference image pyramid data structure based on the level A method of generating encoded data representing the difference between data and encoded current image data. An inter-frame error data encoding device comprising: 36. means for forming the predicted image data using interframe motion compensation; The interframe error data encoding device according to claim 35. 37. The application means include: means for applying the hierarchical encoding to the data structure on a block-by-block basis; , Predicted image data appropriately maps block portions of the current image at the level of the pyramid structure. A claim comprising means for obscuring a block of predicted image display when the predicted image display fails to be displayed. 36. The interframe error data encoding device according to item 35. 38. The application means include: Claim 35 further comprising arithmetic encoding means for partially generating said encoded representation. The inter-frame error data encoding device as described in . 39. applying said encoding to a level of said data structure on a block basis; means and Move the block location boundaries from frame to frame in said array of image frames. , and means for improving encoding efficiency. Error data encoding device. 40. The application means include: Between the difference image data at a lower level and the interpolated reconstructed difference image data at the next higher level Provides a means for forming an interpolation error data structure at a lower level by taking the difference between 36. The interframe error data encoding device according to claim 35. 41. To transmit an array of image frames, the image transmission system uses interframe error data. In an apparatus for encoding data, The predicted image data for the current image frame and the uncoded current image frame Displays the difference between the unencoded current image data to be displayed on a pixel-by-pixel basis. means for forming a difference image; a difference image pyramid data structure that reduces the difference image and has multiple reduction levels; A means for generating a field and a hierarchical lattice threshold quantization encoding based on the selected level. Apply the difference image pyramid data structure to the predicted image data and encode the current and means for generating encoded data representing a difference between the image data and the image data. Inter-frame error data encoding device. 42. To transmit an array of image frames, the image transmission system transfers image data between frames. In the method of encoding data, entropy-encoded lattice threshold quantization encoding is applying the image data to the image data based on the image frame; and the array of image frames. Move block location boundaries from frame to frame to improve coding efficiency A method for encoding interframe image data, comprising steps. 43. The moving step includes: Move the block boundary pixels in each of the axes that define the image plane The interframe image data encoding method according to claim 42, comprising the step of: . 44. To transmit an array of image frames, the image transmission system transfers image data between frames. In a device that encodes data, entropy-encoded lattice threshold quantization encoding is means for applying the image data to the image data based on the image frame; and the arrangement of image frames. Move block location boundaries from frame to frame to improve coding efficiency An interframe image data encoding device comprising means. 45. The transportation means is The block boundaries in the direction of each of the plurality of axes defining the image plane are defined as a plurality of pixels. Inter-frame image data encoding according to claim 44, comprising exchanging means. Device. 46. To transmit an array of image frames, the image transmission system encodes the interframe error. In the encoding device, inter-frame predicted image data is generated for the current image frame. means for reducing and generating a predictive pyramid data structure having multiple levels of reduction; , reducing unencoded current image data representing an unencoded current image frame; means for generating a current image pyramid data structure having the plurality of reduction levels; , a hierarchical lattice threshold quantization encoding is applied to the prediction pyramid based on the selected level. applied to the predicted image data structure and the current image pyramid data structure. generates encoded data that represents the difference between the image data and the unencoded current image data. An interframe error encoding device comprising means. 47. In a method of encoding an array of multidimensional data vectors, Applying lattice threshold quantization encoding to the array of multidimensional data vectors, generating an array of grid points closest to the vector; Entropy encode grid points that identify data representing said array of nearest grid points. the stage of and providing the entropy encoded data to a communication channel. Original data vector array encoding method. 48. The applying step applies, for each data vector, the threshold quantization encoding. using an E-8 grid; Add the absolute value of the rounding error for vector coordinates over all said dimensions the stage of determining the odd-evenness of the sum of candidate grid points for the vector; and determining the nearest grid point to the data vector. Multidimensional data vector array encoding method described in Section 1. 49. determining the identity of each grid point within the selected shell; entropy encoding the index for transmission over the communication channel; Multidimensional data vector array encoding according to claim 47, comprising the step of: Method. 50. The indexing step includes: combining the lattice points to form a 4x4 block, and both lattice points are in the lattice shell 0 identifying whether the encoding shell numbers for all points not in shell 0; The index of each lattice point that is not in shell 0 using the symmetry of the lattice index identifier 49. A multidimensional data vector arrangement according to claim 49, comprising the step of identifying Column encoding method. 51. The index identification step includes: for a croup of grid points, identifying a parent centroid index; using spatial symmetry and identifying changes in the centroid index of the parent. A multidimensional data vector array encoding method according to claim 50. 52. Clear grid point coordinates for points in shell 3 or higher level shells 52. The multidimensional data vector array according to claim 51, comprising the step of transmitting Encoding method. 53. In an apparatus for encoding an array of multidimensional data vectors, Applying lattice threshold quantization encoding to the array of multidimensional data vectors, means for generating an array of grid points closest to the vector; Entropy encode grid points that identify data representing said array of nearest grid points. means to and means for providing said entropy encoded data to a communication channel. Original data vector array encoding device. 54. The applying means is configured for each data vector for the threshold quantization encoding. means for using an E-8 grid; Adding the absolute value of the rounding error for vector coordinates over all dimensions step by step, determining the odd-evenness of the sum of candidate grid points for the vector; and means for determining the nearest grid point to the data vector. A multidimensional data vector array encoding device as described in 2. 55. means for determining the identity of each grid point within the selected shell; entropy encoding the index for transmission over the communication channel; Multidimensional data vector array encoding according to claim 53, comprising means for Device. 56. The indexing means is A means of combining grid points to form a 4x4 block, and both grid points are in the shell 0 of the grid. a means for identifying whether the means for encoding shell numbers for all points not in shell 0; The index of each lattice point that is not in shell 0 using the symmetry of the lattice index identifier The multidimensional data vector array according to claim 55, further comprising means for identifying the Column encoder. 57. The index identification means includes: means for identifying a parent centroid index for a croup of grid points; and a means for identifying changes in the parent centroid index using spatial symmetry. A multidimensional data vector array encoding device according to claim 56. 58. Explicitly specify grid point coordinates for points in shell 3 or higher level shells The multidimensional data vector array code according to claim 57, comprising means for transmitting. encoding device.