JP2005218124A - Data compression system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an appropriate quantization corresponding to a property of an output unit without elongating sign streams. <P>SOLUTION: Compression bitstreams are input into a parser. The parser chooses and outputs all or a portion of the compression bitstreams according to a request. For example, a coded data for displaying a picture on a monitor is output by choosing a coefficient of compressibility of low resolution. Alternatively, a compression data is chosen so that un-losing nature expanssion of an attention region may be made into enable. In one embodiment, the parser, according to a request, outputs a bit required for a variance from a pre-view picture to a printer resolution picture or a full-sized medical monitor picture. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to the field of data compression and decompression systems, and more particularly to a data compression system including a parser.

  Data compression is a very useful tool for storing and transmitting large amounts of data. For example, the time required for image transmission such as facsimile transmission of a document is drastically shortened by reducing the number of bits required for image reproduction using compression.

  Conventionally, many different data compression methods exist. The compression methods can be roughly classified into two categories: lossy coding and lossless coding. Lossy coding is coding that results in loss of information and thus does not guarantee a complete reproduction of the original data. The goal of lossy coding is to make sure that even if it changes from the original data, the change is not uncomfortable or noticeable. In lossless compression, all information is preserved and the data is compressed in a way that allows for full decompression.

  In lossless compression, input symbols or luminance data are converted into output codewords. As input, image data, audio data, one-dimensional data (for example, data that changes spatially or temporally), two-dimensional data (for example, changes in two spatial axis directions (or one spatial dimension and one temporal dimension) Data), or multidimensional / multispectral data. If compression is successful, the codeword is represented with a number of bits less than that required for the input symbol (or luminance data) before encoding. Non-lossy coding methods include a dictionary coding method (for example, Lempel-Ziv method), a run length coding method, a counting coding method, and an entropy coding method. In lossless image compression, the compression is based on prediction or context and coding. The JBIG standard for facsimile compression and DPCM for continuous tone images (differential pulse code modulation—an option of the JPEG standard) are examples of lossless compression for images. In lossy compression, input symbols or luminance data are quantized and then converted to output codewords. Quantization aims to remove important feature quantities while preserving important feature quantities of data. Lossy compression systems often use transformations to concentrate energy prior to quantization. JPEG is an example of a lossy encoding method for image data.

  Recent developments in image signal processing have focused on the pursuit of efficient and highly accurate data compression coding schemes. Various schemes for transformation or pyramid signal processing have been proposed, including multi-resolution pyramid processing schemes and wavelet pyramid processing schemes. These two methods are also called a subband processing method and a hierarchical processing method. The wavelet pyramid processing method for image data is a special multi-resolution pyramid processing method that performs subband decomposition of an original image using an orthogonal mirror filter (QMF). There are other non-QMF wavelet schemes. For more information on wavelet processing, see Antonini, M. et al. , Et l. "Image Coding Using Wavelet Transform", IEEE Transactions on Image Processing, Vol. 1, No. 2, April 1992, and Shapiro, J. et al. "An Embedded Hierarchical Image Coder Using Zerotrees of Wavelet Coefficients", Proc. IEEE Data Compression Conference, pgs. See 214-223, 1993. To obtain information on reversible transformation, see Said, A. et al. and Pearlman, W.M. “Reversible Image Com- pression via Multiresolution Representation and Predictive Coding”, Dept. See of Electrical, Computer and System Engineering, Renssealaer Polytechnic Institute, Troy, NY 1993.

  Compression is often very time consuming and requires a large amount of memory. It is desirable to perform compression at higher speeds and / or with as little memory as possible. Some applications have not utilized compression because quality cannot be guaranteed, compression rates are insufficient, or data rates are not controllable. However, it is desirable to use compression to reduce the amount of information to be transmitted and / or stored.

  The prior art includes a compression system for handling natural continuous tone images. One example is the international standard Dis. 10918-1, “Digital Compression and Coding of Continuous-Tone Still Images”, CCITT Recommendation T. 81, which is usually called JPEG. The prior art also has compression systems for handling binary / noise-free / shallow pixel depth images. An example of such a system is the international standard ISO / IEC 11544, “Information Technology-Coded Redistribution of Picture and Audio Information-Progressive Bi-level Image Compression”, CCITT Recommendation T.30. 82, which is usually called JBIG. However, the prior art does not have a system that properly handles both. It would be desirable to have such a system.

  Parsers are well known in computer science. The parser plays a role in different parts of an object whose structure is not initially known. For example, a parser that runs as part of a compiler has a string in the program file that is an “identifier”, another string that makes up a reserved word, and another string that is a comment. Will decide. This parser does not determine what “meaning” the character string is, but only what type of part of the object it is.

  Most image storage formats are single use. That is, only a single resolution or a single quality level is available. Other image formats are versatile. Some prior art versatile image formats support two or three resolution / quality options, but some can specify only one of resolution or quality, not both. It is desirable to increase the available resolution and quality options.

  For example, the Internet's World Wide Web server currently provides the information needed from a large amount of data. Usually, the user can view many images on the screen and decide to print some. Unfortunately, with the current state of viewing tools, the quality of the printed output is considerably worse if the image is mainly for monitoring, and the viewing time becomes extremely long if the image is mainly for printing. End up. Acquisition of “lossless” images is impossible or must be done as a completely unique download.

  It is a general object of the present invention to provide a lossy and lossless data compression system that utilizes transformations that provide good energy concentration. More specifically, it is an object to provide a data compression system including a parser that performs device-dependent quantization according to device characteristics given by an image output device.

  A data compression system according to claim 1 comprises a memory for storing a code stream having a header having at least one marker, at least one output device, connected to the memory, and from the at least one output device. It consists of a parser connected to receive the characteristics, the parser being operable to perform device dependent quantization. According to the second aspect of the present invention, the code stream is composed of lossless compressed data, and according to the third aspect of the present invention, the at least one marker is a component used for each tile in the code stream. According to the invention of claim 4, the code stream includes a main header, and a local header is placed before each tile in the code stream. According to the fifth aspect of the present invention, the main header is applied to all tiles in the code stream, and each local header is applied only to the associated tile. At least one of these takes precedence over the main header. According to the invention described in claim 7, the parser uses a marker in the code stream to quantize the code stream, and according to the invention according to claim 8, at least one of the markers indicates frequency information. . According to the ninth aspect of the present invention, the data compression system further includes a compression device for generating a code stream. According to the invention described in claim 10, the parser comprises a quantization selection device. According to the invention described in claim 11, the quantization selection device performs conversion and quantization of a set of images with bit planes having various coefficients. Do by throwing away. According to the invention of claim 12, one of the tags indicates an importance level in the data in each tile, and according to the invention of claim 13, the tag indicates an importance level locator signal, the signal The parser will abort accordingly. According to the invention described in claim 14, the tag indicates the number of importance levels to be stored, and according to the invention described in claim 15, the tag indicates the number of bytes to be stored. According to the invention described in claim 16, the tag includes an instruction for associating the importance level with the number of bytes in each tile, and according to the invention according to claim 17, the at least one marker includes the importance level in each tile. Indicates the number of bytes.

  As is apparent from the above description, according to the data compression system including the parser of the present invention, the code data stream can be appropriately quantized according to the characteristics of the image output apparatus without decompressing the code data. It has effects such as.

  A method and apparatus for compression and decompression is described. In the following detailed description of the present invention, various specific examples such as the type of coder, the number of bits, the signal name, etc. are shown in order to fully understand the present invention. However, it will be apparent to those skilled in the art that the present invention may be practiced without such specific examples. On the other hand, well-known structures and devices are shown in block diagram form and are not shown in detail in order not to obscure the present invention.

  A significant portion of the detailed description below is given by algorithms and symbolic representations of operations on data bits in computer memory. Such algorithmic descriptions and representations are the means used by those skilled in the data processing arts to most effectively convey the substance of their work to others skilled in the art. Suppose that there is an algorithm that is generally considered a self-consistent sequence of steps that leads to the desired result. These steps are those requiring physical processing of physical quantities. Usually, though not necessarily, these physical quantities take the form of electrical or magnetic signals capable of being stored, transferred, combined, compared, and otherwise manipulated. It has proven convenient at times, principally for reasons of common usage, to represent these signals as bits, values, elements, symbols, characters, terms, numbers, or the like.

  However, it should be noted that such terms are to be associated with appropriate physical quantities and are merely convenient labels attached to these physical quantities. As will be apparent from the following description, unless otherwise specified, discussions using terms such as “processing”, “operation”, “calculation”, “judgment”, “display”, etc., refer to the physical ( Processes data expressed as electronic quantities and converts it to other data expressed as physical quantities in a computer system memory or register, similar information storage device, information transmission device or display device as well Refers to the operation and process of a computer system or similar electronic computing device.

  The present invention also relates to an apparatus for performing the operations described herein. This device may be made exclusively for the required purpose, or it may be a general-purpose computer selectively driven or reconfigured by a built-in program. The algorithms and displays presented herein are essentially unrelated to any particular computer or other device. Various general purpose machines may be utilized in programs according to those described herein, or it may be advantageous to create a more specialized device for performing the necessary method steps. The required structure for a variety of these machines will appear from the description below. Further, in describing the present invention, it is not associated with any particular programming language. It will be appreciated that a variety of programming languages may be used to implement the teachings of the invention, as described herein.

  The following terms are used in the description below. These terms already have meaning. However, the defined meanings should not be considered limited to the extent that the terms are known in the art. These terms are defined in order to help understanding of the present invention.

Alignment:
The degree of shift of the transform coefficient within a frequency band relative to other frequency bands.

Binary encoding method:
A coding scheme for binary, finite pixel depth, or noise-free data. In one embodiment, the binary coding scheme consists of pixel gray coding and a specific context model.

Bit-significance:
A number representation similar to the sign absolute value representation, where the head bit is followed by a sign bit, followed by a tail bit, if any. Embedding encodes this number representation in bit-plane order.

Context model:
Information that can be used as a cause for the current bit to be encoded and provides previously learned information about the current bit to enable conditional probability prediction for entropy coding.

Embedded quantization:
Quantization included in the code stream. For example, when the importance levels are arranged in order from the highest level to the lowest level, the quantization is performed by simply truncating the code stream. The same effect can be obtained with tags, markers, pointers, and other signals.

Entropy coder:
An apparatus for encoding or decoding current bits based on probability prediction.
An entropy coder will also be referred to herein as a multi-context binary coder. The context of the current bit is some chosen arrangement with respect to “neighboring” bits, allowing probability prediction for an optimal representation of the current bit (one or more bits). In one embodiment, the entropy coder includes a binary coder or a Huffman coder.

Fixed length:
A method of converting a specific block of data into a specific block of compressed data. For example, some methods of BTC (block truncation coding) and VQ (vector quantization). Fixed length codes are suitable for fixed rate and fixed size applications, but rate and distortion performance is often inferior to variable rate systems.

Fixed rate:
An application or scheme that has to maintain a certain pixel rate and has a limited bandwidth channel. To achieve this goal, it is necessary to compress locally on average rather than compress on average overall. For example, MPEG requires a fixed rate.

Fixed size:
An application or scheme with a limited size buffer. To achieve this goal, overall average compression is achieved, eg, a print buffer. (Applications can be fixed rate and fixed size, or both.)

frequency band:
Each frequency band represents a group of coefficients produced by the same filtering sequence.

Head bit:
In the bit-significance representation, the head bit is an absolute value bit including the first non-zero bit from the most significant bit to the first non-zero bit.

Horizontal context model:
Context model for embedded wavelet coefficients and binary entropy coder (in one embodiment).

Power, etc .:
An encoding that allows an image to be decompressed in a lossy format and then recompressed to the same lossy codeword.

Image tile:
A rectangular region chosen to allow the definition of a grid of consecutive non-overlapping sub-images, each with the same parameters. Image tiles affect the buffer size required for transform calculations in wavelet coding. Image tiles can be randomly addressed. The encoding operation processes the pixel and coefficient data in one image tile. Thus, image tiles can be parsed or decoded in random order. That is, image tiles can be randomly addressed or decoded according to various distortion levels of region of interest decompression. In one embodiment, the image tiles are all the same size except for the top and bottom ones. The image tile may be any size that is less than or equal to the size of the entire image.

Importance level:
By defining a specific system, input data (pixel data, coefficients, error signals, etc.) is logically classified into a plurality of groups having the same visual effect. For example, the most significant bit plane or planes are probably more visually important than the lower bit planes. Also, low frequency information is generally more important than high frequency information. Most practical definitions of “visual importance” relate to some error criterion, including the present invention, as described below. However, a better visual measure may be incorporated into the system definition of visual importance. Different data types have different levels of visual importance. For example, voice data has a voice importance level.

Overlap conversion:
A transformation in which a single source sample point contributes to multiple coefficients of the same frequency. Examples include many wavelets and Lapped Orthogonal Tansform.

progressive:
A code stream that is ordered so that a consistent decompression result can be obtained from a portion of the encoded data and the accuracy can be increased by increasing the data. A code stream in which the bit plane of data is ordered from shallow to deep; in this case, it usually refers to wavelet coefficient data.

Progressive pixel depth:
A code stream in which the bit plane of data is ordered from shallow to deep.

Progressive pyramid:
A series of resolution components whose size is reduced to half (1/4 in area) each time the resolution decreases.

Reversible conversion:
In one embodiment, an efficient transformation performed by integer arithmetic that can be restored based on the compression result.

S conversion:
A special reversible wavelet filter pair consisting of one 2-tap low-pass filter and one 2-tap high-pass filter.

Tail:
In bit-significance representation, a tail bit is a lower absolute value bit that is lower than the most significant non-zero bit.

Tail information:
In one embodiment, the four possible states for the coefficients expressed in bit-significance representation. It is a function of the coefficients and the current bitplane and is used for the horizontal context model.

Tail-on:
In one embodiment, two states depending on whether the tail information state is zero or non-zero. Used for horizontal context model.

Tile data segment:
The part of the code stream that completely describes one image tile. In one embodiment, all data from the tag defining the beginning of an image tile (SOT) to the next SOT or the end of image (EOI) tag.

Conversion factor:
The result of applying the wavelet transform. In the wavelet transform, the coefficient represents a logarithmically divided frequency scale.

TS conversion:
2. 6 (Two-Six) conversion. A special reversible wavelet filter pair consisting of one 2-tap low-pass analysis filter and one 6-tap high-pass analysis filter. The synthesis filter is an orthogonal mirror filter of the analysis filter.

TT conversion:
2.10 (Two-Ten) conversion. A special reversible wavelet filter pair consisting of one 2-tap low-pass analysis filter and one 10-tap high-pass analysis filter. The synthesis filter is an orthogonal mirror filter of the analysis filter.

Unified non-loss / loss:
The same compression system provides a code data stream that can be lossless or lossy restored.

Wavelet filter:
High-pass and low-pass synthesis and analysis filters used for wavelet transform.

Wavelet transform:
Transformation using constraints in both the “frequency” and “time (space)” domains. In the embodiment to be described, the conversion is composed of one high-pass filter and one low-pass filter. The resulting coefficients are thinned out 2: 1 (critical filtering) and then the filters are subjected to low pass coefficients.

Wavelet tree:
Coefficients associated with a single coefficient in the LL part of the highest level wavelet decomposition. The number of coefficients is a function of the number of levels. The span of the wavelet tree depends on the number of decomposition levels. For example, for 1 level decomposition, the wavelet tree span is 4 pixels, for 2 level decomposition, 16 pixels, and so on.

SUMMARY OF THE INVENTION The present invention provides a compression / decompression system having an encoder and a decoder. The encoder serves to encode the input data and generate compressed data, while the decoder serves to decode the already encoded data and generate reconstructed data of the original input data. The input data includes various types of data such as images (still images or moving images), sounds, and the like. In one embodiment, the data is digital signal data, but digitized analog data, text data formats, and other formats are possible. The source of the data is, for example, a memory or a communication channel for an encoding unit and / or a decoding unit.

  In the present invention, the components of the encoding unit and / or the decoding unit can be realized by hardware or software used on a computer system. The present invention provides a lossless compression / decompression system. The present invention may also be configured to perform lossy compression / decompression. The present invention can be configured to perform parsing of compressed data without decompression.

System Overview of the Present Invention The present invention represents very well the smooth edges and flat areas found in natural images. The present invention uses reversible embedded wavelets to compress images with deep pixel depth. However, reversible embedded wavelets, other wavelet transform methods and sinusoid transform methods are not good at expressing the sharp edges found in text and graphic images. This type of image can be compressed well by performing Gray coding followed by context-based bitplane coding such as JBIG. Furthermore, noise-free computer-generated images are well modeled by the binary method.

  The present invention provides a binary scheme for compression of binary and graphic images. This binary scheme also improves compression for certain types of images that do not use the entire dynamic range. In this binary system, the present invention encodes a bit plane of an image without using a transformation.

  FIG. 1 is a block diagram of an embodiment of a compression system of the present invention adopting a binary method. Note that the decryption unit of the system operates in the reverse order, and the data flow is the same. In FIG. 1, an input image 101 is input to a multi-component processing mechanism 111. The multi-component processing mechanism 111 provides optional color space conversion and optional processing for subsampled image components. The method selection mechanism 110 determines whether the image is a continuous tone image or a binary image, or which part of the image has such characteristics. The image data is sent to the method selection mechanism 110, and the method selection mechanism 110 sends the image data or a part thereof to the wavelet method processing (blocks 102, 103, 105) or the binary method processing (block 104). In the present invention, the determination of which mode to use depends on the data. In one embodiment, scheme selection mechanism 110 comprises a multiplexer. The scheme selection mechanism 110 is not used during the decoding operation.

  In the wavelet method, the reversible wavelet transform block 102 performs a reversible wavelet transform. The output of the block 102 is a series of coefficients. The embedding ordered quantization block 103 labels the coefficients after making them a bit-significance representation in order to generate an alignment of all the coefficients in the input image 101 (generated by the reversible wavelet transform block 102).

  Image data 101 is received and transformed (after appropriate multi-component processing) using a reversible wavelet as described later in a reversible wavelet transform block 102 to provide a coefficient representation representing the multi-resolution decomposition of the image. A series is generated. The reversible wavelet transform of the present invention is not complicated to calculate. This conversion can be performed by software or hardware without causing any systematic errors. Furthermore, the wavelet of the present invention is excellent in energy concentration and compression performance. These coefficients are received by the embedding ordered quantization block 103.

  The embedding ordered quantization block 103 performs embedding ordered quantization as will be described later. The result is an embedded data stream. This embedded data stream allows quantization of the resulting code stream at the time of encoding, transmission or decoding. In one embodiment, the embedding ordered quantization block 103 orders the coefficients and converts them to signed / absolute value format.

  The embedded data stream is received in the horizontal context model block 105. The horizontal context model block 105 models the data in the embedded data stream based on its importance (described later). In transform mode, the “bit plane” is the transform coefficient importance level plane, and the horizontal context model block 105 arranges the wavelet coefficients into a bit-significance representation.

  The result of the ordering and modeling is a decision (or symbol) to be encoded by the entropy coder 106. In one embodiment, all decisions are sent to one coder. In other embodiments, decisions are labeled by importance, and each importance level decision is processed by separate multiple (physical or virtual) coders. The bitstream is encoded by the entropy coder 106 in order of importance. In one embodiment, entropy coder 106 comprises one or more binary entropy coders. In another embodiment, Huffman coding is utilized.

In the binary method, the gray encoding block 104 performs gray encoding on the pixels of the input image 101. Gray coding is a bit operation that uses a portion of the correlation between the bit planes of a pixel. This is because, for an arbitrary value x and x + 1, gray (x) and gray (x + 1) differ only in one bit in their radix-2 representation. In one embodiment, the gray coding block 104 performs a point-by-point transformation on 8-bit pixels, i.e.
gray (x) = xXORx / 2
Execute. The present invention is not limited to using this form of gray coding, nor does it have to use 8-bit size pixels. However, the use of the above formula has an advantage that the pixels can be reconfigured with only a part of the most significant bits that can be used, as in the case of progressive transmission in bit plane units. In other words, this form of gray coding preserves the bit-significance ordering.

  In the binary method, the gray coding block 104 and the entropy coder 106 are used, and the data is coded for each bit plane. In one embodiment, the context model in Gray coding block 104 conditions the current bit using spatial and importance level information.

In the binary method, a context model such as JBIG is used for gray coded pixels. In one embodiment, each bit plane of the image tile is encoded separately, and each bit is conditioned and encoded in raster order using the surrounding 10 pixel values. FIG. 2 shows the geometric relationship of the context model for each bit of each bit plane in the binary scheme. This bit conditioning results in adaptive probability prediction for each unique pattern. Note that when a binary entropy coder is used for bit-plane entropy coding of gray coded values, several templates may be used for the context model of the binary entropy coder. Figure 3 shows a 7 pixel and bit plane information 2 bits for 2 ^nine context bin (bin).

  Using this context and the value of the current bit, the entropy coder 106 generates a bit stream. This same binary entropy coder 106 is used to encode both conversion mode and binary data. In one embodiment, entropy coder 106 comprises a finite state machine implemented with a look-up table. It should be noted that the present invention can be used with any binary entropy coder such as a Q coder, a QM coder, or a high speed parallel coder.

  Since the entropy coder 106 is the same for both systems, and the binary context model is simple, very little additional resources are required to implement the binary system and the conversion system in the same system. . Furthermore, although the configuration of the context model is different, the required resources for both modes are the same. That is, both modes use the same memory for context storage and the same binary entropy coder.

  The present invention may be performed on the entire image or, more generally, on a tiled segment of the image. Some tiles have better compression by the conversion method and some by the binary method. Various algorithms for selecting the mode to be used are possible. When tiles are used, random access in units of tiles is possible. In addition, the attention area can be decoded with high accuracy for each. Finally, it can be determined for each tile whether to select the conversion method or the binary method.

  It should also be noted that the images are progressive with respect to the bit plane using the dual mode system of the present invention and can be encoded in a hierarchical format as taught by JBIG.

  For decoding, one bit in the tile header may be used to specify the scheme used to encode the data. The system selection mechanism 110 is not used. It may be even more useful if a lossless mapping from the original dynamic range to the low dynamic range is possible, such as by histogram compression (see below). Look ahead as in JBIG may be used. This look ahead may use normal or deterministic prediction as in JBIG.

Selection of binary system or conversion system The system selection mechanism 110 selects a binary system or a conversion system. In one embodiment, the input image is encoded with both schemes, and the scheme selection mechanism 110 selects the scheme with the lower bit rate obtained (assuming lossless compression). That is, the mode that compresses better is selected. This method may seem expensive, but not so much. This is because both the binary method and the conversion method have relatively high software speed and small hardware. A method derived from this method is a method of determining a lower bit rate by bypassing the coder and using an entropy value.

  In another embodiment, the present invention generates a complete (or partial) histogram of image pixel values, or a histogram of differences between adjacent pixel value pairs. In the case of a histogram of pixel values, a statistical analysis of the data is used such that if the peak of the histogram occurs at a value much smaller than the dynamic range of the pixel depth, the binary method is selected.

  In one embodiment, the present invention generates a complete (or partial) histogram of the first order differences between adjacent pixel pairs. In a standard image, such a histogram is exactly a Laplacian distribution and the wavelet method will be used. However, when the histogram does not have a Laplacian distribution peak, the binary method is used.

  Both types of histograms may be generated and used together for method selection.

  As will be described later, the dn filter output of TS conversion or TT conversion is close to the first-order statistic. This suggests how the transformation is performed and a histogram is generated. A scheme is selected based on the histogram. When the method is a conversion method, the system continues to process already generated conversion coefficients. When the binary scheme is selected, the transform coefficients are discarded (or inverse transformed depending on whether the pixel is saved) and the system starts the binary scheme.

  In another embodiment, previous knowledge of region segmentation and / or document type may assist in deciding which method to choose.

  If longer encoding times are available, the tiling size can be chosen to maximize the benefits of the two schemes.

  It should be noted that in one embodiment, the system of the present invention does not include binary coding and therefore only uses lossless embedded wavelet compression (CREW) and decompression.

Wavelet Decomposition The present invention first uses reversible wavelets to perform image or other data signal decomposition (as image data). In the present invention, the reversible wavelet transform realizes a complete reconstruction system capable of restoring lossless signals having integer coefficients by integer arithmetic. An efficient reversible transformation is due to a transformation matrix having a determinant of 1 (or nearly 1).

  The present invention can provide lossless compression with finite precision operations by utilizing reversible wavelets. The result generated by applying the reversible wavelet transform to the image data is a sequence of coefficients.

  The reversible wavelet transform of the present invention can be realized using a set of filters. In one embodiment, the filters are one 2-tap low-pass filter and one 6-tap high-pass filter. In one embodiment, these filters are implemented with only addition and subtraction (and hardwire bit shifting).

  One embodiment of the present invention that utilizes Hadamard transforms is a complete reconstruction system.

  To obtain information about the Hadamard transform, see Anil K. et al. Jain, “Fundamentals of Image Processing”, p. I want to read 155. The inverse transformation of Hadamard transformation is referred to herein as S transformation.

  In the S conversion, the output can be defined as follows using the general index n.

  Note that the factor 2 in transform coefficient addressing is the result of implicit ½ subsampling. This transformation is reversible, and the inverse transformation is as follows.

  symbol

は、切り捨てて丸めること、つまり打ち切りを意味し、床関数と呼ばれることがある。同様に、天井関数

Means rounding down, that is, censoring, and is sometimes called the floor function. Similarly, the ceiling function

は最も近い整数へ切り上げて丸めることを意味する。

Means rounding up to the nearest whole number.

  Another example of a complete reconstruction system is a 2 · 6 (TS) transformation. The reversible TS transform is defined by the following equation for the two outputs of the low pass and high pass filters.

  TS conversion is reversible, and its inverse conversion is as follows.

ここで、次式によりｐ（ｎ）がまず計算されなければならない。

Here, p (n) must first be calculated according to the following equation.

  The result from the low pass filter can be used twice (in the first and second terms) in the high pass filter. Therefore, the result of the high-pass filter can be obtained only by performing two other additions.

  Another example of a complete reconstruction system is a 2 · 10 (TT) transformation. The reversible TT transform is defined by the following equation for the two outputs of the low pass and high pass filters.

  The expression for d (n) can be simplified using s (n) (and integer division by 64 can be rounded by adding 32 to the numerator). As a result, the following equation is obtained.

このＴＴ変換は可逆であり、その逆変換は次式である。

This TT transform is reversible, and its inverse transform is

ここで、ｐ（ｎ）は次式によりまず計算されなければならない。

Here, p (n) must first be calculated according to the following equation.

  In both the TS conversion and the TT conversion, as in the S conversion, the low-pass filter is formed so that the range of the input signal x (n) is the same as the range of the output signal s (n). That is, there is no increase in smooth output. When the input signal is b bits deep, the output signal is also b bits deep. For example, when the signal is an 8-bit image, the output of the low-pass filter is 8 bits. This is an important characteristic for pyramid systems where the smooth output is further decomposed, for example by applying a low pass filter continuously. In prior art systems, the range of the output signal is larger than the range of the input signal, which makes continuous application of the filter difficult. In addition, since there is no systematic error due to rounding when performing conversion by integer arithmetic, all errors of the lossy system can be controlled by quantization. Furthermore, the low-pass filter is a non-overlapping filter because it has only two taps. This property is important for hardware implementation.

  In one embodiment, multiplication by 3 and 22 is realized by shift and addition as shown in FIG. In FIG. 22, the s (n) input is connected to a multiplier 1501, and the multiplier 1501 multiplies the s (n) input by 2. In one embodiment, this multiplication is implemented as a one digit left shift of the bits of the s (n) signal. The output of the multiplier 1501 is added to the s (n) signal by the adder 1502. The output of the adder 1502 is a 3s (n) signal. The output of adder 1502 is also multiplied by 2 by multiplier 1503. Multiplier 1503 is implemented as a left shift by one digit. The output of the multiplier 1503 is added to the output of the multiplier 1504 by an adder 1505, and the multiplier 1504 multiplies the s (n) signal by 16 by a left shift of 4 digits. The output of the adder 1505 is a 22s (n) signal.

  Strict reversibility requirements for filters can be relaxed by noting the following: High pass coefficients are encoded and decoded in a certain order. Since the pixel value corresponding to the previously decoded high-pass coefficient is known accurately, it can be used for current high-pass filtering.

  The TS transform and TT transform have non-overlapping low-pass synthesis filters and high-pass analysis filters. Only the high-pass synthesis filter and the low-pass analysis filter are overlap filters.

  TS filters have good properties with respect to tile boundaries. Consider the case where the tile size is a multiple of the tree size. Then consider the application of a transformation to a portion of a signal, such as occurs when an image is divided into tiles. Since the low pass analysis filter is not an overlap filter, the low pass coefficients are not affected by tiling. That is, if the portion of the signal has a uniform number of signals, the low-pass coefficient is the same as when the entire signal is converted.

  During decoding, if there are no high-pass coefficients due to quantization and the image is reconstructed at the highest compression rate using only SS coefficients, a low-pass synthesis filter may be used across tile boundaries, Inverse transformation is performed on the entire signal using low-pass coefficients. The solution is exactly the same as when tiling is not used because it does not change with tiling of SS coefficients. This eliminates artifacts caused by performing forward transformation on a portion of the signal.

  During decoding, if there are high-pass coefficients (but the high-pass coefficients are quantized so that their values have some uncertainty), there is a boundary where there is an overlap 1D low-pass analysis filter operation You can do the following on a specimen that crosses and enters another part: For samples that do not cross the tile boundary, the minimum and maximum possible reconstruction values are determined based on the filters actually used. For that sample, only the low-pass coefficients (and the low-pass filter) are used, and by traversing the tile boundary, the reconstruction value that would have been (ie, the overlap prediction value) is determined. If the overlap prediction value is between the possible minimum and maximum reconstruction values (including those values), the overlap prediction value is used. Otherwise, the closest possible overlap value of the possible maximum and minimum reconstruction values is used. This reduces the artifacts caused by performing forward transforms on the signal fragments.

  Each time a 1D filter operation is performed, a reconstruction value is selected. If this is done correctly, each high-pass coefficient is given exactly one valid reconstruction value, and the selection cannot propagate the error to multiple levels of conversion.

Nonlinear Image Model One embodiment of the present invention utilizes a wavelet filter that is a reversible approximation of a linear filter such as a TS transform or a TT transform. In one embodiment, a reversible nonlinear filter may be used. One type of nonlinear filter similar to the TS transform and TT transform is as follows.

  The inverse transformation is the same as in the case of TS transformation and TT transformation, except that p (n) is as follows.

  In this example, q (n) is a prediction on x (2n) -x (2n + 1) from smooth coefficients (and previous detail coefficients if necessary). This prediction uses a nonlinear image model. In one embodiment, the nonlinear image model is a Huber-Markov random field. This nonlinear image model is exactly the same in the forward transformation and the inverse transformation s. In the case of an iterative image model, the number of iterations and the order are the same.

  An example of the nonlinear image model is as follows. For a certain number of iterations, each value (pixel or low-pass coefficient) x (k) is adjusted to a new value x ′ (k) (where k is 2n or 2n + 1). Any number of iterations may be used, but in one embodiment, 3 iterations are used. The value of q (n) is determined from the final iteration.

At each iteration, the change y (k) for each x (k) is calculated.

ここでＡは変化率であり、これは任意の正値でよい。一実施例ではＡ＝１である。Ｂiは差分検出子である。例えば、１次元の場合、Ｂiは次の通りである。

Here, A is the rate of change, which may be any positive value. In one embodiment, A = 1. Bi is a difference detector. For example, in the case of one dimension, Bi is as follows.

２次元の場合、水平、垂直、及び２つの対角線方向の差分を検出するための４つのＢi値があろう。Ｂiとして他の差分演算子を用いてもよい。

In the two-dimensional case, there will be four Bi values to detect horizontal, vertical, and two diagonal differences. Other difference operators may be used as Bi.

ここで、Ｔは閾値であり、これは任意の正値でよい。Ｔは画像のエッジを構成する差分を表す。一実施例では、Ｔ＝８である。

Here, T is a threshold value, which may be any positive value. T represents the difference constituting the edge of the image. In one embodiment, T = 8.

  Under the constraint of x ′ (2n) + x ′ (2n + 1) = x (2n) + x (2n + 1), the pair of values x ′ (2n) and x ′ (2n + 1) changes y It is adjusted by a pair of (2n) and y (2n + 1). This is accomplished by combining the changes y (2n), y (2n + 1) into a single change y ′ (2n), the maximum change supported by both changes.

When y (n) and y (2n + 1) are both positive or negative,
y '(sn) = 0
It is. Otherwise, if | y (2n) | <| y (2n + 1) |
y '(2n) = y (2n)
And if not
y ′ (sn) = − y (2n + 1)
It is. And
x ′ (2n) = x (2n) + y ′ (2n)
x ′ (2n + 1) = x (2n + 1) −y ′ (2n)
It is. To obtain more information about the Huber-Markov random field, see R. R. Schultz and R.C. L. Stevenson, “Improved definition image expansion”, Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing, vol. See III, pp.173-176, San Fransisco, March 1992.

  In some embodiments, the transformation is extended from one dimension to two, but for that extension, the transformation is first performed separately for each dimension with q (n) = 0. Next, three q (n) values for the LH, HL, and HH values are calculated by similar application of the image model.

Two-dimensional wavelet decomposition Multi-resolution decomposition is performed using the low-pass filter and high-pass filter of the present invention. The number of decomposition levels is variable and may be any number, but currently the number of decomposition levels is 2 to 5 levels. The maximum number of levels is log 2 (the maximum value of length or width).

  The most common way to perform transformations on two-dimensional data such as images is to apply the one-dimensional filter separately, ie along the rows and then along the columns. The first level decomposition yields four different coefficient bands, referred to herein as LL, HL, LH, and HH. These characters represent low (L) and high (H) which means the application of the smooth and detail filters defined above. Therefore, the LL band is composed of coefficients obtained from the smoothing filters in the row direction and the column direction. It is a common way to arrange the wavelet coefficients in the form as shown in FIGS.

  Each frequency subband of the wavelet decomposition can be further decomposed. The most common way is to further decompose only the LL sub-block, which would involve further decomposition of the LL frequency subband for each decomposition level as it is generated. Such multiple decomposition is called pyramid decomposition (FIGS. 4 to 7). Each decomposition is indicated by the symbols LL, LH, HL, HH and the decomposition level number. Note that pyramid decomposition does not increase the coefficient size with either the TS filter or the TT filter of the present invention.

  For example, when the reversible wavelet transform is applied recursively to an image, the first level decomposition works on the finest detail or resolution. At the first decomposition level, the image is decomposed into four sub-images (ie sub-bands). Each subband represents one spatial frequency band. The first level subbands are denoted LL0, LH0, HL0, HH0. Since the process of decomposing the original image includes 1/2 subsampling in both horizontal and vertical dimensions, as shown in FIG. 4, the first level subbands LL0, LH0, HL0, and HH0 are input respectively. Have one-fourth the number of pixels (or coefficients) in the image.

  The subband LL0 includes low frequency information in the horizontal direction and low frequency information in the vertical direction at the same time. In general, most of the image energy is concentrated in the subband. The subband LH0 includes low frequency information in the horizontal direction and high frequency information in the vertical direction (for example, horizontal edge information). The subband HL0 includes high frequency information in the horizontal direction and low frequency information in the vertical direction (for example, vertical edge information). The subband HH0 includes high frequency information in the horizontal direction and high frequency information in the vertical direction (for example, texture or oblique edge information).

  The second, third, and fourth sub-decomposition levels that follow are each created by resolving the previous level low frequency LL subband. By disassembling the first level subband LL0, as shown in FIG. 5, slightly finer second level subbands LL1, LH1, HL1, and HH1 are created. Similarly, by dividing subband LL1, as shown in FIG. 6, third-level subbands LL2, LH2, HL2, and HH2 with a coarse definition are generated. Further, as shown in FIG. 7, by dividing subband LL2, fourth level subbands LL3, LH3, HL3, and HH3 with coarser definition are created. With 2: 1 subsampling, each second level subband is 1 / 16th the size of the original image. Each sample (ie pixel) at this level represents a slightly fine detail at the same position in the original image. Similarly, each third level subband is 1 / 64th the size of the original image. Each pixel at the third level represents a fairly coarse detail at the same position in the original image. In addition, each sub-band of the fourth level is 1/256 the size of the original image.

  Since the decomposed image is physically smaller than the original image for subsampling, the entire decomposed subband can be stored using the memory used for storing the original image. That is, in the case of three-level division, the original image and the decomposed subbands LL0 and LL1 are discarded and are not stored.

  Although only four subband decomposition levels are shown, higher levels can be generated depending on the requirements of the individual system. Various parent-child relationships may be defined by other transforms such as DCT or one-dimensionally arranged subbands.

Pyramid decomposition Each frequency subband of the wavelet decomposition can be further decomposed. In one embodiment, only the LL frequency subband is resolved. Herein, such decomposition is referred to as pyramid decomposition. Each decomposition is indicated by the symbols LL, LH, HL, HH and the decomposition level number. Note that according to the wavelet filter of the present invention, pyramid decomposition does not increase the coefficient size.

  In other embodiments, other subbands may be resolved in addition to LL. In the following description, the term “LL” may be used interchangeably with “SS” (“L” = “S”). Similarly, the term “H” may be used interchangeably with “D”.

Wavelet Tree Structure There are natural and useful tree structures for wavelet coefficients of pyramid decomposition. Note that there is only one LL frequency sub-block corresponding to the final decomposition level. On the other hand, there are the same number of LH, HL, and HH bands as the number of levels. This tree structure reveals that the parents of the coefficients in a certain frequency band are the coefficients in the same frequency band with a lower resolution and are in the same spatial positional relationship.

  The root of each tree is a purely smooth coefficient. In the case of a two-dimensional signal such as an image, the root of the tree has three “children” and the other nodes each have four children. This hierarchical tree is not limited to two-dimensional signals. For example, in the case of a one-dimensional signal, the root has one child, and each node other than the root has two children. Higher dimensions are derived from the one-dimensional case and the two-dimensional case.

  FIG. 8 shows a parent-child relationship between two consecutive levels. In FIG. 8, the coefficient A is a direct parent to B, C, and D, but is also a parent to coefficients (E and H, F and I, G and J) having B, C, and D as parents. For example, B is the parent for 4 coefficients near E, 16 coefficients near H, and so on.

  The process of multi-resolution decomposition can be performed using a filter sequence.

  For an example of a one-dimensional two-level transformation implemented using a one-dimensional exemplary filter, see US patent application Ser. No. 08 / 498,695 (accepted Jun. 30, 1995, “Method and Apparatus For Compression Using Reversible Wavelet”. See "Transforms and an Embedded Codestream") and U.S. Patent Application No. 08 / 498,036 (Receivable Wavelet Transform and Embedded Code-stream Manipulation ", accepted June 30, 1995).

Execution of Forward Wavelet Transformation In the present invention, wavelet transformation is executed by a 1-D (dimensional) path in the horizontal direction and then a 1-D path in the vertical direction. The number of iterations is determined by the number of levels. FIG. 9 represents a four-level decomposition using a forward TT transform filter as previously defined.

  In another embodiment, another reversible wavelet transform filter, such as an S transform, can be used instead of an arbitrary time TT transform of any level horizontal or vertical wavelet transform. In one embodiment, a four-level decomposition is performed using a TT transform in both the horizontal and vertical directions. In one embodiment, in a four level decomposition, two of the four TT transforms are replaced with S transforms. This has little loss of compression, but has a significant effect on memory usage. The horizontal transformation and the vertical transformation will be applied alternately.

  Note that a combination of S conversion and TT conversion may be used to perform horizontal and vertical conversion. The order of transformations can be miscellaneous, but in order to be completely reversible, the decoder must know the order and perform the reverse operation in the reverse order. As described below, the decoder may be informed of the conversion order in the header.

Embedding Ordering In the present invention, the coefficients generated as a result of wavelet decomposition are entropy coded. In the present invention, the coefficients are first subjected to embedded coding, in which the coefficients are ordered in a visually significant order, or more generally, some error criterion. The coefficients are ordered taking into account (eg, distortion criteria). Error or distortion criteria include peak error and mean square error (MSE). In addition, ordering may be performed by direction (vertical, horizontal, diagonal, etc.) so as to prioritize the validity for database query over bit-significance spatial location.

  Data ordering is performed to generate an embedded quantized version of the code stream. In the present invention, two ordering methods are used. One is for ordering the coefficients, and the other is for ordering the binary values in the coefficients. The ordering of the present invention produces a bitstream that is then encoded by a binary entropy coder.

Tile In the present invention, an image is divided into tiles prior to transformation and encoding. A tile is a partial image of a whole image that is encoded completely independently, and is defined by a regular rectangular grid arranged on a numbered image as shown in FIG. The right and bottom tiles vary in size depending on the original image and the tile size.

  The tiles can be any height and width below the image size, but the choice of tile size affects performance. Small tiles, especially tiles with small vertical dimensions in raster-ordered images, can reduce work area memory. However, if the tile is too small, the compression efficiency decreases due to three factors: signaling overhead, loss of conversion efficiency at the tile boundary, and adaptation of the entropy coder to rise. The size of the tile is advantageously a multiple of the size of the lowest frequency component (CREW tree), ie a function of the number of levels (2 to the power of the level). Depending on the size of the original image, 128 × 128 or 256 × 256 tiles may be suitable for many applications.

  Tiles may be used for compression of a series of images. Thus, a tiled image may be a temporally different image (such as a movie) or a spatially different image (such as a 3D cross section such as MRI). There is no special way to signal this, but CMT may be used.

  Transforms, context models, and entropy coding work only on the pixels and coefficients of one image tile. Therefore, image tiles can be analyzed or decoded out of order, i.e., randomly addressed, or decoded to various distortion levels for region of interest expansion.

  All pixel data of one image tile is buffered in, for example, a memory that can be used at once by the encoder. Once the pixel data is converted, all coefficient data can be used in the horizontal context model. Since all coefficients can be accessed randomly, the order of embedding within an image tile can be arbitrary as long as the encoder and decoder know. Entropy coders are careless about this ordering, so the order has a significant effect on the compression ratio and must be chosen with care.

  An image tile is defined by the number of a tree (LL coefficient and all its descendants) arranged in a rectangle. The number of pixels in each tree is a function of the number of wavelet decomposition levels.

  The image reference lattice is the smallest lattice plane in which the size of each component is an integer multiple of the lattice point. This implies that for most images, the image reference grid is the same as the most frequent component.

  When a one-component image or all components have the same size, the image reference grid is the same size as the image (for example, grid points are image pixels). In the case of an image in which all of the plurality of components are not the same size, the size is defined as an integer multiple of the image reference grid point. For example, the CCIR601 YCrCb color component system is defined to have two Y components for each Cr and Cb component. Therefore, the Y component defines the image reference grid, and the Cr component and the Cb component each cover 2 units in the horizontal direction and 1 unit in the vertical direction.

Bit Significance Representation In one embodiment, the embedding order used for binary values in the coefficients is bit plane order. The coefficient is expressed in bit-significance expression. The bit-significance representation is a sign / absolute value representation in which the sign bit, rather than the most significant bit (MSB), is encoded with the first non-zero absolute value bit.

  There are three types of bits represented in bit-significance format: head bits, tail bits, and sign bits. The head bits are all zero bits from the MSB to the first non-zero absolute value bit and the first non-zero absolute value bit. The importance of the coefficient is determined by the bit plane in which the first non-zero absolute value bit exists. The bits after the first non-zero absolute value bit to the LSB are tail bits. The sign bit only represents a sign. The number of non-zero bits in the MSB, for example ± 2n, has only one head bit. Zero coefficients have no tail bits or sign bits. FIG. 12 shows an example of bit-significance expression.

If the value is a non-negative integer, such as occurs in relation to the luminance of the pixel, the order that can be employed is the bit plane order (eg, from the most significant bit plane to the least significant bit plane). In embodiments where negative integers with two's complement are also acceptable, the sign bit padding order is the same as the first non-zero bit of the absolute value of the integer. Thus, the sign bit is not considered until one non-zero bit is encoded. For example, according to the sign / absolute value notation, the 16-bit number of -7 is 1000000000000111.
It is. On a bit plane basis, the first 12 decisions are “meaningless” or zero. The first 1 bit is found in the 13th decision. The sign bit (“negative”) is then encoded. After the sign bit is encoded, the tail bit is processed. The 14th and 15th decisions are both "1".

  Since the coefficients are encoded from the most significant bit plane to the least significant bit plane, the number of data bit planes must be known accurately. In the present invention, the number of bit planes is determined by finding the upper limit of the absolute value of the coefficient value calculated from the data or derived from the image depth and filter coefficients. For example, when the upper limit is 149, there are significant 8 bits or 8 bit planes. Because of software speed, bitplane coding may not be used. In another embodiment, the bit plane is only encoded when the coefficients make sense as binary numbers.

Coefficient alignment (alignment)
The present invention aligns coefficients before bit-plane coding. This is because, like FFT and DCT, coefficients in different frequency subbands represent different frequencies. The present invention enables quantization by performing coefficient alignment. Coefficients with a smaller quantization weight are aligned to the faster bit plane side (for example, shifted to the left). Thus, if the stream is censored, these coefficients have more bits that define them compared to the more heavily quantized coefficients.

  In one embodiment, the coefficients are aligned for best rate and distortion performance from an SNR or MSE perspective. Many alignments are possible, including near-optimal alignments in terms of statistical error criteria such as MSE. Alternatively, the alignment may allow physchovisual quantization of the coefficient data. Alignment has a considerable effect on image quality (in other words on the rate-distortion curve), but has little effect on the final compression ratio of the lossless system. Other alignments may correspond to special region-of-interest fidelity coding or resolution progressive alignment, which is a special quantization.

  The alignment may be signaled in the header of the compressed data. The coefficients are encoded in bit-significance order, but the highest importance level is derived from the coefficients in the coding unit. The sign bit of each coefficient is not encoded to the highest importance level where the coefficient has a non-zero absolute value bit. This has the advantage of not coding the sign bit of the coefficient whose absolute value is zero. Also, the code bit is not encoded until the point in the embedded code stream where it is relevant. The alignment for coefficients of various sizes has no effect on the efficiency of the entropy coder since both the encoder and the decoder are known.

  FIG. 13 shows the bit depth of the coefficients in the two-level TS transform and TT transform decomposition of an image of b bits / pixel. FIG. 14 is an example of a frequency band multiplier used for coefficient alignment in the present invention. For coefficient alignment, the size of the 1-HH coefficient is used as a reference and a shift is given relative to this size.

  In one embodiment, the coefficients are shifted to account for the absolute value of the largest coefficient to produce an alignment of all coefficients in the image. The aligned coefficients are then processed in bit plane units, called importance levels, from the highest importance level (MSIL) to the lowest importance level (LSIL). Since the sign bit is not part of the MSIL, it is not encoded until the last head bit of each coefficient. Importantly, alignment only controls the order in which bits are sent to the entropy coder. There is no actual padding, shifting, storing, or encoding of the extra zero bits.

  Table 1 shows an example of alignment.

  Since the alignment for coefficients of various sizes is known to both the encoder and decoder, it does not affect the efficiency of the entropy coder at all.

  Note that the coding units of the same data set may have different alignments.

Code Stream Ordering FIG. 15 shows the ordering of the coded stream and the ordering within the coding unit. In FIG. 15, after the header 1001, the encoding unit 1002 continues in order from the uppermost band to the lowermost band. Inside the encoding unit, the LL coefficients 1003 are stored in raster (line) order without being encoded. After the LL coefficient, the importance level is entropy encoded in order from the most significant bit plane to the least significant bit plane, one bit plane at a time. At this time, the first bit plane of all coefficients is encoded, then the second bit plane is encoded, and so on. In one embodiment, the alignment is specified in the header 1001.

  In one embodiment, LL coefficients are stored in raster order without being encoded when they are 8-bit values. When the size of the LL coefficient is less than 8 bits, the LL coefficient is made 8 bits by padding. When the LL coefficient is longer than 8 bits, the LL coefficient is stored as follows. First, the most significant 8 bits of each coefficient are not encoded and are stored in raster order. The remaining lower bits of the coefficients are then packed and stored in raster order. For example, for a 10-bit LL coefficient, the least significant bits of the 4 LL coefficients are packed into 1 byte. In this way, since 8-bit LL data can be obtained for each coefficient regardless of the actual image depth, a simple image or a preview image can be generated quickly.

  The order in which the coefficients are processed within each bit plane period is from lower resolution to higher resolution and lower frequency to higher frequency. The order of coefficient subbands within each bit plane is from higher level (low resolution, low frequency) to lower level (high resolution, high frequency). Within each frequency subband, the encoding is done in a certain order. In one embodiment, the order is raster order, 2 × 2 block order, zigzag order, Peano scan order, and so on.

In the case of four-level decomposition using the code stream of FIG. 15, the order is as follows.
4-LL, 4-HL, 4-LH, 4-HH, 3-HL, 3-LH, 3-HH, 2-HL, 2-LH, 2-HH, 1-HL, 1-LH, 1- HH

  Dividing the code stream data according to importance is advantageous for storing the data in a medium or transmitting the data through a noisy channel. Error correction / detection codes with different redundancy can be used for different parts of the data. The code with the highest redundancy can be used for the header of the LL coefficient. For the less important entropy encoded data (on the basis of importance), an error correction / detection code with low redundancy can be used. When an uncorrectable error occurs, to determine whether the packet of data should be discarded (quantized), retransmitted from the channel, or reread from the storage device Importance levels are also available. For example, a high redundancy error correcting BCH code (such as a Reed-Solomon code) may be used for the most important quarter of header data, LL data, entropy encoded data. The remaining three quarters of the entropy encoded data will be protected by a low redundancy error detection checksum or CRC (Cyclic Redundancy Check). In one embodiment, packets with BCH codes are always retransmitted and not discarded, while packets with checksum or CRC codes are not retransmitted and are discarded after a failed attempt to transmit data. Let's go.

  In one embodiment, the header data specifies an error correction / detection code that is used for each piece of data. In other words, the information in the header indicates when to switch the error correction code. In one embodiment, the error correction / detection code is only changed between packets used for the communication path or between blocks used for the storage medium.

  FIG. 26 shows a code stream in which an LL coefficient (1902) that is not encoded and entropy encoded data 1903 follow the header 1901 in the embedding order. As shown, the header 1901 and the LL coefficient 1902 use the code with the highest redundancy, while the entropy encoded data 1903 uses the code with the lowest redundancy. The present invention can also employ a sliding scale in which many different codes are used, from the highest redundancy to the lowest redundancy.

Horizontal Context Model An embodiment of the horizontal context model used in the present invention will be described below. This model takes advantage of the bits in the coding unit based on the spatial and spectral dependencies of the coefficients. Available binary values of adjacent coefficients and parent coefficients may be used to generate the context. However, the context affects the decodability and somewhat affects the efficient adaptation.

Coefficient Modeling with Horizontal Context Model The present invention provides a context model for modeling a code stream generated by embedded bit-significance ordered coefficients for a binary entropy coder. In one embodiment, the context model consists of a run length count, a spatial model, a sign bit model, and a tail bit model. The run length count measures the run of bits in the same state. The spatial model includes information on neighborhood coefficients and parent coefficients related to head bits.

  FIG. 16 shows adjacent coefficients of all the coefficients of the coding unit. In FIG. 16, the adjacency coefficient is expressed in an easy-to-understand geographical notation (for example, N = north, NE = northeast, etc.). Given a coefficient, eg, P in FIG. 16, and the current bitplane, the context model can use any information obtained from all coding units prior to that bitplane. In the case of this context model, the parent coefficient of the attention coefficient is also used.

Horizontal headbit context model Headbits are the most compressible data. Therefore, a large amount of context or conditioning is used to increase the compression rate. Rather than using the value of the adjacent coefficient or parent coefficient to determine the context for the bit of interest of the coefficient of interest, that information is organized into two bits, referred to herein as tail information. This information may be stored in memory or calculated dynamically from neighboring coefficients or parent coefficients. The tail information indicates whether the first non-zero absolute value bit has already been found (eg, whether the first “on” bit has already been found), and if so, how many previous bit planes It was shown. Table 2 describes the tail information bits.

From the 2-bit tail information, a 1-bit “tail on” bit indicating whether the tail information is zero or not is synthesized. In one embodiment, tail information and tail on bits are updated immediately after the coefficients are encoded. In another embodiment, the update is done later to allow parallel context generation.

  Furthermore, these 2 bits are used to indicate the importance level to be encoded. The first 2-bit plane uses the value 0, the second 2-bit plane uses the value 1, the third 2-bit plane uses the value 2, and the remaining bit planes use the value 3. In addition, there is a run-length encoding of all zero head bits.

  The 10-bit context for the head bits consists of 2-bit information for each of the parent coefficient and W coefficient, 1-bit information for each of the N, E, SW, and S coefficients, and 2-bit importance level information.

  In one embodiment, tail information is not used for some or all frequency bands. In this way, the frequency band can be decoded without first decoding its parent.

  In another embodiment, one alignment is used to assign bit planes of each frequency band to importance levels. Another alignment is used to determine the tail-on information of the parent, but it uses fewer parent bitplanes than the actual encoded bitplane. This allows several bit planes in a certain frequency band to be decoded without decoding corresponding parent bit planes of the same importance level (see FIG. 51). For example, an image can be encoded by pyramid alignment regardless of parent tail-on information based on MSE alignment (see FIG. 50). This allows the decoder to simulate MSE alignment, or simulate any alignment between pyramid alignment and MSE alignment, and decode with pyramid alignment.

  FIG. 29 shows context dependencies. A child is conditioned to its parent. Therefore, a parent must be decoded before decoding its children, especially when decoding using an alignment different from the alignment used during encoding.

Horizontal sign bit context model After the last head bit, the sign bit is encoded. There are three sign contexts depending on whether the N coefficient is positive or negative, or whether the sign is not yet encoded.

Horizontal tail bit context model There are three contexts for tail bits depending on the value of the tail information of the coefficient of interest. (Note that tail information values are limited to 1, 2 or 3 when trying to encode tail bits)

Steps for the horizontal context model The system context model uses up to 11 bits to describe the context. This number need not be specified as a whole. The meaning of each bit position depends on the previous binary value. First of all, only one context is used to provide “run coding” with head bits. In the absence of a run of head bits, each bit is encoded with adjacent and parent coefficients that contribute to a context. Specific examples of steps are as follows.

1) Determine whether look-ahead should be performed.
If the tail information for the next N coefficients and their north neighbor coefficients are all zero, the system proceeds to step 2. Otherwise, go to step 3 for the next N coefficients. In one embodiment, N = 16.

2) Perform the look-ahead procedure.
If the bit of the current bitplane to be encoded of the next N coefficients is zero, one 0 is encoded and the system moves from step 1 to the next N coefficient. Otherwise, one 1 is encoded and the system proceeds to step 3 for the next N coefficients.

3) The state of the attention coefficient is determined and encoded.
If the tail information of the coefficient of interest is 0, the bits of the current bit plane of the coefficient of interest are the two bits of the tail information of the west coefficient and the parent coefficient (optional), the tail on bit of the north west, east, south west, south coefficient , And 1024 possible contexts created by 2 bits of importance level information, the system proceeds to step 4. Note that in one embodiment, the parent coefficient is not used, so the context is created only from the adjacent coefficient and importance level information. If the tail information of the coefficient of interest is not 0, the bit of the current bit plane of the coefficient of interest is a tail bit and is encoded by three contexts created from 2 bits of the tail information of the coefficient of interest.

4) Determine the state of the current head bit and, if necessary, encode the sign bit.
If the bit of the current bitplane of the coefficient of interest is 1, the sign of the coefficient of interest is encoded with three possible contexts created by the tail-on bit and the sign bit of the north coefficient.

  FIG. 17 is a flowchart of the above-described process. In FIG. 17, white blocks are not related to encoding, and black blocks are related to encoding. Although not shown in the figure, one context is defined for each entropy coding decision. The foregoing actions and flows will be understood by those skilled in the art.

  An example of a horizontal context model is US patent application Ser. No. 08 / 498,695 (June 30, 1995, “Method and Apparatus,” along with an example of a sign / absolute value unit that converts input coefficients to sign / absolute value format. For Compression Using Reversible Waveform Transforms and an Embedded Codestream ”) and U.S. Patent Application No. 08 / 498,036 (accepted June 30, 1995,“ Reversible Wavelet Transform and Embedded Codestream Manipulation ”).

Entropy coding In one embodiment, the entropy coding performed by the present invention is performed by a binary entropy coder. In one embodiment, entropy coder 106 comprises a Q coder, a QM coder, a finite state machine, a high speed parallel coder, or the like. A single coder may be used to generate a single output code stream. Alternatively, multiple (physical or virtual) coders may be used to generate multiple (physical or virtual) data streams.

  In one embodiment, the binary entropy coder of the present invention comprises a Q coder. To obtain information about the Q coder, see Pennebaker, W .; B., et al. "An Overview of the Basic Principles of the Q-coder Adaptive Bi- ary Arithmetic," IBM Journal of Research and Development, Vol. 32, pg. Please read 717-26, 1988. In another embodiment, the binary entropy coder uses a QM coder, which is a well-known efficient binary entropy coder. QM coders are particularly efficient for bits with very high probability skew. QM coders are used in both JPEG and JBIG standards.

  The binary entropy coder may be a finite state machine (FSM) coder. Such a coder provides a simple conversion from probability and outcome to a compressed bitstream. In one embodiment, the finite state machine coder is implemented by table lookup as both an encoder and a decoder. A variety of probability prediction methods can be used for such a finite state machine coder. The compression ratio for a probability close to 0.5 is very good. The compression ratio for a highly skewed probability depends on the size of the lookup table used. Like QM coders, finite state machine coders are useful for embedded bitstreams because decisions are encoded in the order in which they occur. Since the output is determined by a look-up table, there is no concern about a “carry over” problem. In practice, unlike Q and QM coders, there is a maximum delay between encoding and generating compressed output bits. In one embodiment, the finite state machine coder of the present invention comprises a B coder as described in US Pat. No. 5,272,478 issued December 21, 1993 “Method and Apparatus for Entropy Coding”.

  In one embodiment, the binary entropy coder of the present invention comprises a high speed parallel coder. Both QM and FSM coders need to be encoded or decoded bit by bit. A high speed parallel coder processes several bits in parallel. In one embodiment, the high speed parallel coder is implemented in VLSI hardware or a multiprocessor computer without sacrificing compression performance. An example of a high speed parallel coder that can be used in the present invention is described in US Pat. No. 5,381,145 “Method and Apparatus for Parallel Decoding and Encoding of Data” issued on Jan. 10, 1995.

  Most efficient binary entropy coders are speed limited by the basic feedback loop. One possible solution is to divide the input data stream into multiple streams and feed them to multiple encoders in parallel. The output of these encoders is a plurality of variable length encoded data streams. One challenge with this type of method is how to transmit data on a single channel. The high speed parallel coder described in US Pat. No. 5,381,145 solves this problem by a method of interleaving these encoded data streams.

  Many of the contexts utilized in the present invention are constant probabilities, which make finite state machine coders such as B coders particularly useful. Note that both high speed parallel coder and finite state machine coder disclosed in the above patent operate more efficiently than Q coder when the system uses a probability close to 0.5. Thus, both these coders have an inherent compression advantage over the context model of the present invention.

  In another embodiment, both a binary entropy coder and a fast m-ary coder are utilized. The high-speed m-original coder may be a Huffman coder.

Encoding and Decoding Process of the Present Invention The flowcharts of FIGS. 18-20 represent examples of the encoding and decoding processes of the present invention. The processing logic may be realized by software and / or hardware.

  FIG. 18 shows an example of the encoding process of the present invention. In FIG. 18, at the beginning of the encoding process, processing logic obtains input data for one tile (processing block 1201).

  Processing logic then determines whether binary encoding must be performed (processing block 1202). When binary encoding is to be performed, the process proceeds to processing block 1211 where processing logic performs Gray encoding on the input data and models each bit of each coefficient with a binary context model (processing block 1212). ). Processing continues at processing block 1208.

  If binary encoding does not need to be performed, the process proceeds to processing block 1203 where processing logic filters the data losslessly. After applying the reversible filter, processing logic determines whether another decomposition level is required (processing block 1204). If another decomposition level is required, processing logic applies a reversible filter to the LL coefficients (processing block 1205) and processing returns to processing block 1204 to make a determination again. If another decomposition level is not required, the process proceeds to processing block 1206 where processing logic converts the coefficients to signed / absolute value format. Processing logic then models each bit of each coefficient with a horizontal context model (processing block 1207) and the process proceeds to processing block 1208.

  At processing block 1208, processing logic encodes each bit of each coefficient. Then, the processing block transmits or stores each encoded data (processing block 1209).

  The processing block then determines whether other tiles are used in the image (processing block 1210). If there are other tiles in the image, processing logic loops back to processing block 1201 and the process is repeated, but if there are no other tiles, the process ends.

  FIG. 19 shows an example of the decoding process of the present invention. In FIG. 19, the process first obtains encoded data for one tile (processing block 1301). Next, processing logic entropy decodes the encoded data (processing block 1302). Processing logic then determines whether the data must be binary decoded (processing block 1203). When the data has to be binary decoded bit by bit, the process proceeds to processing block 1311 where processing logic models each bit of each coefficient with a binary context model and inverse Gray encoding for the data. (Processing block 1312). After this inverse gray encoding, the process proceeds to processing block 1309.

  When binary decoding does not need to be performed, the process proceeds to processing block 1304 where processing logic models each bit of each coefficient with a horizontal context model. Processing logic then converts each coefficient to a form suitable for filtering (processing block 1305) and applies a reversible filter to the coefficient (processing block 1306).

  After applying the reversible filter, processing logic determines whether there is another level of decomposition (processing block 1307). If there is another level of decomposition, the process proceeds to processing block 1308 where processing logic filters the coefficients reversibly and the process loops back to processing block 1307. If another decomposition level is not required, the process proceeds to processing block 1309 where the reconstructed data is transmitted or stored.

  Next, processing logic determines whether there are other tiles in the image (processing block 1310). If there are other tiles in the image, the process loops back to processing block 1301 and the process is repeated, but if there are no more tiles, the process ends.

FIG. 20 shows an example of a process for bit modeling according to the present invention. In FIG. 20, the bit modeling process first sets the coefficient variable C to the first coefficient (processing block 1401). Next, a determination of | c |> 2 ^S is made (processing block 1402). If the determination result is yes, processing proceeds to processing block 1403 where processing logic encodes the bit S of coefficient C using the tail bit model and processing proceeds to processing block 1408. This tail bit model may be a static (non-adaptive) model. If | c | is not greater than 2 ^S , processing proceeds to processing block 1404 where processing logic applies the template to the head bits (the leading 0 and the first “1” bit). After applying the template, processing logic encodes bit S of coefficient C (processing block 1405). A possible template is shown in FIG.

  Next, it is determined whether the bit S of the coefficient C is on (processing block 1406). If bit S of coefficient C is not on, processing proceeds to processing block 1408. On the other hand, if bit S of coefficient C is on, processing proceeds to processing block 1407 where processing logic encodes the sign bit. Processing then proceeds to processing block 1408.

  At processing block 1408, it is determined whether coefficient C is the last coefficient. If coefficient C is not the last coefficient, processing proceeds to processing block 1409 where coefficient variable C is set to the next coefficient and processing continues from processing block 1402. On the other hand, if the coefficient C is the last coefficient, the process proceeds to processing block 1410 to determine whether S is the last bit plane. If S is not the last bitplane, the bitplane variable S is decremented by 1 (processing block 1411) and processing continues from processing block 1401. If S is the last bit plane, the process ends.

TS Conversion Design The present invention, in one embodiment, calculates TS conversions in place in the buffer memory. In doing this, no extra lines of memory and no extra time are needed to relocate the calculated values. Although the TS transform has already been described, the present invention applies the overlap transform to the critically sampled one. In another embodiment, a TT transform is used.

  24 (A) to 24 (C) show the memory operation method employed by the present invention during the calculation of the conversion of the present invention. FIG. 24A shows an initial state of the memory. In FIG. 24A, the first row of the memory contains the smoothing ("S") coefficient, the detailed ("D") coefficient (calculated) of the previous value (n-1), and the current value (n). Smooth ("S") coefficient and partially completed detail coefficient ("B") are the values of four input samples ("X") (X2n + 2, X2n + 2, X2n + 4, X2n + 5) Comes with. The intermediate result of the conversion calculation is shown in the same memory row in FIG. Note that the only change in this row is that the values of X2n + 2 and X2n + 3 have been replaced by the values of Sn + 1 and Bn + 1 in the fifth and sixth storage elements. Thus, the present invention saves memory space by replacing stored values that are no longer needed with the results generated during the conversion calculation. FIG. 24C shows the same memory row after conversion is complete and the detailed output Dn is generated. The only change from FIG. 24B is that the partially completed detail coefficient Bn is replaced with the detail output Dn.

  After the detailed output is calculated for n, the transform calculation process continues down the line to calculate the detailed output Dn + 1.

  The following code example may be used to perform the conversion. Note that horizontal codes for forward and reverse conversion are included.

  In the following, the variable soo refers to the Sn-1 value, the variable oso refers to the Sn value, and the variable oos refers to Sn + 1.

A specific example of a code (C language) for an example of forward TS conversion is as follows.
/ *
TSFoword_1 ()
* /

void TSForward_1 (long * x, int width)
{

long * start = x;
long * ox = x + 2;
long soo;
long oso;
long oos;

oso = (* x + * (x + 1)) >>1;
oos = (* ox + * (ox + 1)) >>1;
soo = oos;

* (x + 1) = * x-* (x + 1);
* x = oso;

while ((ox + 2) -start <width) {

x = ox;
os + = 2;

soo = oso;
oso = oos;

oos = (* ox + * (ox + 1)) >>1;

* (x + 1) = * x-* (x + 1) + ((oos-soo + 2) >>2);
* x = oso;
}

x = ox;

soo = oso;
oso = oos;

oos-soo;

* (x + 1) = * x-* (x + 1);
* x = oso;
}

A specific example of a code (C language) for an example of inverse TS conversion is as follows.
/ *
TSReverse_1 ()
* /

void TSReverse_1 (long * x, int width)
{

long * start = x;
long * d = x + 1;
long ns = * (x + 1);
long P;

while (x + 2-start <width) {

p = * d-((* (x + 2) -ns + 2) >>2);
ns = * x;

* d = * x- (P >>1);
* x + = ((p + 1) >>1);

x + = 2;
d = x + 1;
}

p = * d;
* d = * x- (p >>1);
* x = ((p + 1) >>1);

  Although only a one-dimensional example is shown, the present invention can be used in multiple dimensions and multiple levels. Note that this approach can be used for any overlap transform that replaces values that are no longer needed for other computations, one-to-one with partial or final results.

  FIG. 25 shows a two-dimensional representation of a 3-level memory buffer. In FIG. 25, coefficient values are stored in the memory locations of the blocks 1801 to 1804. That is, each of the blocks 1801 to 1804 is an 8 × 8 block of coefficient values.

  The coefficients are arranged at intervals of 2 to the natural number. Any coefficient can be accessed given a level and an S or D offset. Therefore, access can be made by selecting specific levels and horizontal and vertical frequencies. The buffers may be accessed in raster order.

Unit Buffer Configuration In one embodiment of the present invention, a single buffer supports the transform block, context model block, and encoding block of the compression system. This buffer is a two-dimensional scrolling memory buffer that can access coefficients efficiently and does not require any other memory. Each line of the buffer is accessed via a pointer stored in the line access buffer. FIG. 23A and FIG. 23B show the configuration of this scrolling buffer. The line access buffer 1601 contains pointers pointing to the respective lines of the buffer 1602.

  Scrolling is done by rearranging the pointers stored in the line access buffer. Examples thereof are shown in FIGS. 23A and 23B. FIG. 23A shows the initial state of the buffer. Referring to FIG. 23B, after lines A, B, and C are removed from the buffer and replaced by lines G, H, and I, respectively, the line The pointer in the access buffer is changed so that the first pointer points to line D in the buffer, the second pointer points to line E, and the third pointer points to line F. Next, pointers to lines G, H, and I occupy the last three positions of the line access buffer. Note that the present invention is not limited to having a buffer of 6 lines, and this is merely used as an example. A buffer with a higher number of lines is commonly used, which will be well known to those skilled in the art. Thus, access via the line access buffer has the appearance that the unit buffer scrolls without physically moving the storage. This allows a minimum amount of memory to be used without sacrificing speed.

  In the present invention, by using such a unit buffer, only one band of the image is always stored in the memory, and the application of the overlap transform to the entire image is supported. To achieve this, the wavelet transform is applied only to the number of lines of the image necessary to fully compute the set of wavelet coefficients that make up at least one band of wavelet units. In such a case, the set of completed wavelet coefficients can be modeled, entropy encoded, and removed from the relevant portion of the wavelet unit buffer. The wavelet coefficients that have not been calculated remain as they are in order to complete the calculation in the next iteration. Then, the wavelet unit buffer can be scrolled by rearranging the line pointers, and other data can be put in the empty part of the wavelet unit buffer. Thus, the wavelet coefficient being calculated can be completely calculated.

  As an example, consider that the high-pass filter applies an overlap transform that depends on the current coefficient and the next low-pass filter coefficient. In this example, only two levels of decomposition would be applied to the image data, which implies that one wavelet unit is 4 elements long.

  In order to fully compute the set of wavelet coefficients that make up at least one band of wavelet units, the height of the wavelet unit buffer is at least 8 lines or 2 wavelet units.

  When applying wavelet transform to a two-dimensional wavelet unit buffer, first, the one-dimensional wavelet transform is applied to each row (line) of the buffer. A one-dimensional wavelet transform is then applied to each column of the buffer.

  When applying a one-dimensional wavelet transform to each wavelet unit buffer column, the high-pass filter calculation is partially performed on the last element of each column that depends on image elements not stored in the unit buffer. It can only be ended. This is illustrated in FIG.

  When performing the second level wavelet decomposition, again the high pass filter calculation can only be partially completed for the last element of each column. This is illustrated in FIG.

  In one embodiment, when many decomposition levels are used, wavelet transform is applied only to SS coefficients (1SS in FIG. 55 for the second decomposition level and 2SS in FIG. 56 for the third level). It will be. In such a case, the row and column location of the unit buffer will be skipped to ensure reading and writing the appropriate entry in the buffer.

  In this example, the upper half of the buffer contains a set of wavelet coefficients that have been calculated, and this wavelet coefficient set comprises a band of wavelet units that are modeled, entropy coded, and Can be removed.

  When the upper half of the buffer is free, the buffer can be sucrose by half the height. Here, the next four lines of the image can be read into the buffer. One-dimensional wavelet transform can be applied to each new line stored in the buffer. In the column direction of the buffer, the partially calculated coefficients can be calculated completely, but again, the last element of each column can only be calculated partially.

  The same is done for the second level wavelet decomposition. Again, the upper half of the buffer contains the calculated wavelet coefficients, at which stage the process repeats until there are no more image lines to process.

  The reordering of the line pointers in the line access buffer can be done in various ways. One way is to create a new line access buffer and copy the pointer from it to the old line access buffer. To scroll modulo the height of the wavelet unit buffer, the pointer stored in element i of the old line access buffer would be copied to the index (i + number of lines).

  Note that since the three stages of the compression system are performed on the data in the buffer before the data is extracted from the buffer, the coefficients are generally variously ordered. When raster ordered data manipulation is performed, the scroll buffer of the present invention is in time with minimal memory.

  The line access buffer is managed by software (and / or hardware) and the pointer is manipulated. This software also knows what data in the buffer can be processed and taken out of the buffer.

Alignment Method The present invention shifts the coefficient value to the left by an arbitrary amount. In one embodiment, this alignment is performed by a virtual alignment method. This virtual alignment method does not actually shift the coefficients. Instead, while processing the coefficients one bit plane at a time, the actual bit plane that requires alignment for a particular coefficient is calculated. Given the importance level and the amount of shift to be applied to a particular coefficient, the present invention accesses the desired absolute bit plane of that coefficient if it is within the range of bit planes where it is possible. That is, the desired absolute bitplane for a particular coefficient is given by the current importance level minus the amount of shift to be applied to that coefficient. This desired bit plane is considered valid when it is greater than or equal to the smallest valid absolute bit plane and less than or equal to the largest valid absolute bit plane.

  Two alignment methods are common. The first method, called the mean square error (MSE) method, is such that the MSE is reduced or minimized when a full-frame reconstructed image is compared with the original image. This is a method for aligning coefficients. FIG. 50 is an example of this alignment. See also FIG.

  The second method is a pyramid-type alignment method that provides good rate and distortion performance when the image is reconstructed to a pyramid level size. Here, the coefficients of adjacent levels do not have a common important level, that is, there is no overlap. The alignment on the left side of FIG. 51 represents a strictly pyramidal alignment for 3-level TS transformation. The right side of FIG. 51 represents a level 2 pyramid alignment. (The strictly pyramidal portion of FIG. 51 may be referred to as a level 3 and 2 pyramid alignment.) In each case, the coefficients within the level are aligned with respect to the MSE.

  FIG. 52 shows an exemplary relationship between memory storage coefficients and one alignment.

  According to the present invention, since there is no need to perform an actual shift operation, there is no memory size limitation. Furthermore, the present invention does not require extra memory and can easily implement any alignment method.

Histogram Compression The present invention may utilize histogram compression. In one embodiment, histogram compression is utilized before being subjected to conversion or binary processing. Histogram compression improves the compression rate for some images. Such an image is typically an image in which some value of the dynamic range is not used for any pixel. In other words, there is a gap in the dynamic range of the image. For example, when an image takes only values 0 and 255 out of a total of 256 values, a new image having a one-to-one correspondence with the original image but a much smaller dynamic range can be generated. This image generation is accomplished by defining an increasing function that maps integers to values taken by the image. For example, when an image uses only values 0 and 255, 0 is mapped to 0 and 1 is mapped to 255 in mapping. In another embodiment, when the image has only even (or odd) pixels, the pixel value is remapped to a value between 0 and 128.

  After performing the histogram compression, the image data may be compressed by the reversible embedded wavelet of the present invention. Thus, histogram compression is used in the preprocessing mode. In one embodiment, the histogram is based on a Boolean histogram, such as keeping a list of whether values occur. First, all occurrences are recorded in ascending order. Next, each value is mapped sequentially from 0.

  In one embodiment, guard pixel values are used to reduce the effects of errors. Since adjacent remapped pixel values may correspond to real pixel values separated by a large gap, small errors in remapped values may result in large real value errors. By adding a premium value near the remap value, the effect of such errors will be reduced.

  The decoder is informed that the mapping has been used to reconstruct the original image. This mapping may be indicated in the header. This allows a similar table to be created for post-processing at the decoder. In one embodiment, the decoder is notified of the range for each tile. In one embodiment, the present invention first informs that the mapping is performed, and then informs the number of missing values (eg, 254 in the above example). The cost for notifying the use of histogram compression is only 1 bit. This bit will be followed by a table of all remap values.

  In one embodiment, to reduce the amount of notification when performing histogram compression one tile at a time, one bit informs whether the new Boolean histogram is the same as or different from the Boolean histogram used immediately before. In such a case, the new Boolean histogram is notified to the decoder when it is (only) different from the previous histogram. Even if the new Boolean histogram is different from the previous one, there are usually similarities between them. More specifically, since the exclusive OR of the two histograms is more compressible by the entropy coder, it may be generated and notified to the decoder.

  The histogram can be notified by transmitting as many bits as the size dynamic range (for example, 256 bits for 8-bit depth). The arrangement order of the bits corresponds to the pixel value. In this case, a certain bit is 1 when a corresponding value is used in the image. This bit sequence may be entropy encoded under a first order Markov context model to reduce header cost or minimize headers.

  In another embodiment, when the missing values are a majority, the generated values will be recorded in order, otherwise the missing values are recorded in order.

  In one embodiment, the binary scheme of the present invention will be utilized for compression of palletized images. The palette will be stored in the header. However, since the paletted image will not be embedded, quantization for lossy decompression does not give reasonable results. In another embodiment, the palletized image may be converted to a continuous tone (color or grayscale) image, and each component may be compressed in a conversion or binary manner. This allows reasonable lossy compression.

  One type of image is a continuous tone image where one specified color (or a small subset of the specified color) is used for a specific purpose. The special purpose color may be for notes. For example, a grayscale medical image may have computer generated color characters to identify the image. Another special purpose color may indicate that the pixel in the overlay image is transparent and the pixel in the lower image is displayed instead. The forbidden color will be separated into separate component images. The continuous tone component and the special color component will be compressed / decompressed in a conversion method or a binary method.

  The conversion method and the binary method are often used for luminance data, but other types of two-dimensional data such as an alpha channel for alpha mixing may be used.

Parser
According to the present invention, the code stream can be parsed before transmission or decoding without decompression. This parsing is performed by a parser that can abort the bitstream and convey only the amount of information needed for a particular quantization. To support this parser, a marker and a pointer determine the position of each bit plane of the coding unit in the bitstream.

  The present invention provides device-dependent quantization performed by parsing in an image compression system. The use of markers in the compression system allows quantization that is selected depending on the device after encoding. The output device reports its characteristics to the parser, and the parser quantizes the encoded file for that particular device. This quantization is by omitting part of the file. The use of the reversible wavelet transform enables image loss-free restoration or image restoration with various distortions that are visually lossless depending on the apparatus.

  According to the present invention, quantization can be performed after encoding. 27 and 28 are block diagrams of a compression system including a parser. 27 and 28, the original uncompressed image 2101 is input to the compression apparatus 2102 of the present invention. The compression device 2102 performs lossless compression of the image 2101 into the compressed bit stream 2103 and adds a marker to the compressed bit stream 2103.

  The compressed bitstream 2103 is input to a parser 2104, which provides a portion of the compressed bitstream 2103 as an output. The part may be the entire compressed bitstream 2103 or only a part thereof. The requesting agent or device provides its device characteristics to the parser 2104 when the decompressed image is needed. In response, the parser 2104 selects and sends the appropriate portion of the compressed bitstream 2104. Parser 2104 does not perform pixel or coefficient level computations or entropy encoding / decoding. In another embodiment, parser 2104 may perform some of such work.

  The parser 2104 can provide encoded data for displaying an image on a monitor by selecting a compression coefficient for low resolution. For different requirements, parser 2104 selects the compressed data to allow lossless decompression of the region of interest (ROI). In one embodiment, the parser 2104 sends the bits necessary for the transition from the preview image to the printer resolution image or full size medical monitor image (possibly with a 16-bit pixel depth), as required.

  The data provided by the parser 2104 is output to the communication path and / or the storage device 2106. The decompression device 2107 accesses the data and decompresses the compressed data. The decompressed, that is, reconstructed data is output as a decompressed image 2108.

  In FIG. 29, a bit plane in the 2HH frequency band is encoded using information from the 3HH frequency band. FIG. 29 has been rewritten in FIGS. 50 and 51 to show the bit plane more clearly. When coefficients are stored as shown in FIG. 50 (MSE), truncation of the compressed bitstream is almost the same as MSE rate / distortion optimal quantization. This truncation is indicated by the shaded marker in FIG. Examining FIG. 30, this arrangement may be appropriate for a printer but not sufficient for a monitor. As shown in FIG. 51, truncation of the bitstream provides images of various resolutions when the coefficients are stored “in a pyramid”, that is, when all bits of a frequency band are stored first. .

  Skillful use of the markers will allow for both types of truncation and will produce low resolution and low fidelity images. The shading changes in FIGS. 50 and 51 represent the bitstream truncation that would produce the highest resolution and low fidelity bitstream. If all of the LH, HL, and HH coefficients are further truncated, the resolution of the image will be lowered.

  In many image compression applications, an image is compressed only once but may be decompressed many times. Unfortunately, for most compression systems, the amount of loss allowed and the appropriate quantization must be determined at the time of encoding. Progressive systems give a set of progressively more detailed images, but lossless reconstruction is generally not possible, i.e., sending out "difference images" encoded in a lossless manner unrelated to progressive buildup This provides a lossless reconstruction.

  In the present invention, the encoder stores enough information to decompose the various coefficients into frequency and bit plane elements. In one embodiment, a marker is inserted into the bitstream to inform what is in the next entropy encoded data unit. For example, the marker may indicate that the next entropy encoded data unit contains HH frequency information for the third bit plane from the most significant bit.

  If someone wants to examine the image on the monitor, he will request the information needed to produce a low resolution grayscale image. When the user wants to print the image, the information required to generate a high resolution binary image will be required. Finally, when the user wants to perform a compression test, or to perform statistical analysis of sensor noise or medical diagnosis, a lossless version of the image will be required.

  FIG. 31 is a block diagram regarding the exchange with the parser, the decoder, and the output device. In FIG. 31, parser 2402 is connected to receive lossless compressed data with markers, as well as device characteristics of one or more output devices, eg, display module 2405 shown. Based on this device characteristic, parser 2402 selects the appropriate portion of the compressed data and sends it to channel 2403 which forwards the data to decompressor 2404. The decompressor 2404 decrypts the data and provides the decrypted data to the display module 2405.

  The present invention provides data streams with improved support for the World Wide Web and other image servers. Some parts of the data stream can support low spatial resolution and high pixel depth images for monitoring. Another part can support high spatial resolution and low pixel depth printers. The entire data stream provides lossless transmission. Since these three uses are supported by the same compressed data, there is no need to send any extra data if the browser requests the monitor image, the print image and the lossless image in turn. The transmitted monitor image information required for the print image can be reused for the print image. The transmitted monitor image and print image information can be reused for non-loss images. The present invention reduces transmission time (transmission costs) for browsing and minimizes the amount of data that must be stored on the server.

  In the system of the present invention, the image is compressed only once, but various markers are stored to indicate what the data is. A World Wide Web (WEB) server will then receive the request for display and provide the necessary coefficients. The WEB server does not need to do any compression or decompression, it just selects the appropriate part of the bitstream to send.

  Quantization with such a parsing system results in a substantial increase in bandwidth even without the high lossless compression with reversible wavelets and context models. This parsing system can also be used to select high quality areas of interest.

  FIG. 32 shows a quantization selection device. In one embodiment, the selection device is configured by software to determine an appropriate quantization profile for various devices. The image is transformed and quantized by discarding bit planes of various frequency bands. Next, an inverse wavelet transform is performed. The reconstructed image is processed in some way consistent with the display. For high resolution images displayed on a monitor, the process may be some sort of scaling. In the case of a printer, it may be some sort of thresholding or dithering. The same processing is applied to the original image and compared with the compressed image. Although the mean square error was used as an example, any visual difference criterion may be used. By using errors due to quantization of various bit planes, bit planes are selected and quantized so that distortion due to bit rate saving is minimized. This process will continue until the desired bit rate or distortion is reached. Once the representative quantization for various image processing operations is determined, it is not necessary to simulate the quantization and representative values will be used.

  Of course, for simple image processing operations such as scaling, the effects of quantization in various frequency bands can be analytically determined. For other operations such as dithering and contrast masking, it is much easier to find a nearly optimal quantization by simulation.

  In FIG. 32, the code stream 2500 is subjected to decompression 2501 including quantization and lossless decompression 2503. Image processing or distortion models 2502 and 2504 are applied to the expansion result. The output is an image and a difference model such as MSE or HV5 difference model 2505 is applied. Based on the result of the difference determination, the alignment is adjusted (2504) and therefore the quantization is also adjusted.

  To facilitate parsing, the present invention utilizes a series of header cues. In one embodiment, the code stream structure of the present invention includes a main header having one or more tag values. A tag in the main header informs information such as the number of components used for all tiles in the codestream, subsampling and alignment. In one embodiment, each tile in the code stream is preceded by its header. The tile header information applies only to that tile and may override the main header information.

  Each header includes one or more tags. In one embodiment, there are no inline markers. The header tag indicates the amount of compressed data from a known point to where the user resets the coder. In one embodiment, every tag is a multiple of 16 bits. Thus, the main header and tile header are all multiples of 16 bits. It should be noted that any tag may have a bit number that is a multiple of a number other than 16. Each tile data segment is inserted with an appropriate number of zeros so that the number of bits is a multiple of 16.

  In one embodiment, each tile header may indicate its tile size. In another embodiment, each tile may indicate where the next tile begins. When backtracking of the code stream is possible, encoding may be simplified by inserting all such information into the main header. The parser can use information about the code stream to perform its quantization.

  In one embodiment, the tile header may indicate whether the tile was encoded in wavelet or binary format. The importance level indicator relates the tile data to the importance level. The importance level locator informs of possible censoring positions. For example, when the same distortion is desired for each tile, the parser can abort the codestream at the appropriate location if it knows which importance level is equal to the desired distortion level. In one embodiment, each tile has approximately the same distortion, rather than having the same number of bits.

  The present invention has an importance level locator tag so that it can have multiple tiles and an indication of where to end each tile.

Tags and pointers Parsing markers and other information used for decoding or parsing may be placed in the tags. In one embodiment, the header provides control information by tags that follow the following rules.

  Tags can be fixed or variable. Depending on the number of components, the number of tiles, the number of levels, or the number of reset or desired information, the tag may vary in length.

  If the image is parsed and quantized, the tags are changed to represent new image characteristics.

  0 is inserted into the reset point in the data stream so that the number of bits is a multiple of 8. The entropy coder can be reset at some point in the code stream, but that point is determined at the time of encoding (but it can only be done at the end of one importance level encoding) . This reset means that all state information (context and probability) in the entropy coder is returned to a known initial state. Next, 0 is inserted into the code stream up to the next multiple of 8 bits.

  The parser uses only the code stream tag as a clue when quantizing the image. In one embodiment, tile length, component length, reset, bit to importance level, and importance level locator tags are used for this quantization process.

  After the image is quantized by the parser, all of its tags are changed to reflect the new code stream. This usually affects the image, tile size, number of components, component span, all lengths and pointers, etc. Further included is an information tag that describes how the image was quantized.

Table 3 is a list of all tags in one embodiment of the present invention. The description and terminology are often different from JPEG, but the same markers and identifiers are used where possible. Every image has at least two headers: a main header at the beginning of the image and a tile header at the beginning of each tile. (Every code stream contains at least one tile)
Three types of tags are also used: separator tags, function tags and information tags. Separator tags are used for header and data framing. The function tag is used to describe the encoding function to be used. Information tags provide optional information about the data.

Note that “x” means that the tag is not used in the header. Either a TLM tag in the header or a TLT tag in each tile is required, but not both. A component pointer is only needed when there are two or more components.

  FIG. 33 shows the arrangement of delimiter tags in the code stream of the present invention. Each code stream has only one SOI tag, one SOC tag, one EOI tag (and at least one tile). Each tile has one SOT tag and one SOS tag. Each delimiter tag is 16 bits and does not include length information.

  The SOI tag indicates the start of a JPEG file and is a 16-bit JPEG magic number.

  The SOC tag indicates the beginning of the file and comes immediately after the SOI tag. The SOI tag and the SOC tag as a whole are 16 bits forming a unique number.

  The SOT tag indicates the beginning of a tile. There is at least one tile in the code stream. SOT serves as a check to ensure that the stream is still synchronized.

  The SOS tag indicates the start of “scan”, followed by the actual image data of the tile. The SOS indicates the end of the tile header, and there must be at least one SOS in the CREW code stream. The data between the SOS and the next SOT or EOI (end of image) is a multiple of 16 bits, and 0 is inserted into this code stream if necessary.

  The EOI tag indicates the end of the image. The EOI serves as a check to ensure that the stream is still synchronized. There is at least one EOI in the code stream.

  The function tag describes the function used to encode the entire tile or image. Some of these tags are used for the main header, but can be overridden using the same tag with different values during the encoding of individual tiles. SIZ tags are image grid width and height, tile width and height, number of components, color space conversion (when required), size of each component (pixel depth), and how the components fill the reference grid Define This tag appears only in the main header and not in the tile header. Each tile causes all of its components to provide the same characteristics. Since many of the parameters defined here are also used for other tags, the SIZ tag must immediately follow the SOC tag. The length of this tag is stored in Lsiz, the first field after SIZ, but depends on the number of components. FIG. 34 shows the image and tile size syntax for the SIZ tag.

  The following is an explanation list of the size and value of each element.

  SIZ: Marker

  Lsiz: Tag length in bytes, not including markers (must be an even number).

  Xsiz: the width of the image reference grid (the same as the image width in a one-component image or an image having color components by common sub-sampling)

  Ysiz: The height of the image reference grid (the same as the image height in the case of a one-component image or an image having color components by common subsampling)

  XTsiz: 1 tile image reference grid width. The tile must be wide enough to hold one sample of every component. The number of tiles in the image width is

に等しい。

be equivalent to.

  YTsiz: the height of the tile image reference grid. The tile must be tall enough to hold one sample of every component. The number of tiles in the image height is

に等しい。

be equivalent to.

  Csiz: the number of components in the image.

  CSsiz: Color space conversion type (when necessary). This tag is not comprehensive. (Many multi-component spatial transformations cannot be specified here. They are not within the file format of the present invention and need to be indicated elsewhere). Table 4 shows values for color space conversion.

The subsampling described in this tag is applied to images where the highest resolution is not available for each component. The system of the present invention has another way to reduce the size of less important components when the highest resolution is available.

  Ssizi: Accuracy of i-th component (pixel depth). This parameter, XRsiz and YRsiz, is repeated for all components.

  XRsizi: X-dimensional size of the i-th component. For example, the number 2 means that the component contributes to two horizontal reference grid points. This parameter, Ssiz and YRsiz are repeated for all components.

  YRsizi: Y-dimensional size of the i-th component. For example, the number 2 means that the component contributes to two vertical reference grid points. This parameter, Xsiz, and XRsiz are repeated for all components.

  res: A zero padding byte placed last when needed.

  The COD tag describes an encoding method such as a binary method or a wavelet method used for an image or tile, a conversion filter, and an entropy coder. This tag is included in the main header and can also be used for tile headers. The length of this tag depends on the number of components. FIG. 35 shows an encoding scheme syntax. Table 6 shows the sizes and values for the encoding scheme.

  COD: marker.

  Lcod: Tag length in bytes, not including markers (must be an even number).

  Ccodi: Coding method for each component.

  res: A zero padding byte placed last when needed.

  For each component, the ALG tag describes the pyramid level number and coefficient alignment. ALG is used for the main header and can also be used for tile headers. The length of this tag depends on the number of components and in some cases also on the number of levels. FIG. 36 shows an example of the component alignment syntax of the present invention. In FIG. 36, the following components are included.

  ALG: This marker indicates the size and value of the component alignment parameter.

  Lalg: Tag length in bytes, not including markers (even).

  Palgi: Number of pyramid decomposition levels of the i-th component. This parameter, Aalg, and possibly Salg are also repeated as one record for each component.

  Aslgi: Alignment of i-th component. This table entry describes the alignment of the coefficients and is repeated for every component. Table 9 shows the value of the Aalg parameter.

Palg and possibly Salg are also repeated as one record for each component.

  Talgi: Table 10 shows the tail information selection method.

Salgij: The alignment value of the j-th sub-block of the i-th component, and is used only when the value of Aalgi for the component is “custom alignment”. This number is 8 bits or 16 bits depending on which custom alignment is chosen and is repeated in turn for every frequency band of the image for that component. (For the binary method, Salgij is the alignment value of the i-th pyramid level)
When Salgij is used, Salgij, Aalg and Palg are repeated as one record for each component.

  res: A zero padding byte placed last when needed.

  The TLM tag describes the length of every tile in the image. The length of each tile is the length measured from the first byte of the SOT tag to the first byte of the next SOT tag (of the next tile), or EOI (end of image). In other words, the length is a list of pointers to tiles or a daisy chain.

  The code stream includes either a single TLM tag or a TTL tag for each tile, but not both. When a TLM tag is used in the main header, the TLT tag is not used. Conversely, when each tile ends with a TLT tag, the TLM tag is not used. The individual tile length values in the TLM header are the same values that would be used for the corresponding TLT tag if TLM was not used. The length of the TLM tag depends on the number of tiles in the image. FIG. 37 shows an example of the tile length / main header syntax.

  TLM: Table 11 shows the sizes and values of tile length / main header parameters.

  Ltlm: The length of the tag in bytes, not including a marker (even number).

  Ptlmi: Length in bytes from the SOT marker of the i-th tile to the next SOT (or EOI) marker. This is repeated for every tile in the image.

  The TLT tag describes the length of the current tile, which is the length measured from the first byte of the SOT tag to the first byte (or EOI) of the next tile's SOT tag. In other words, the TLT is a pointer to the next tile. An example of the TLT syntax is shown in FIG.

  Either a TLM tag or a TLT tag is required, not both. When used, the TLT tag is required for all tile headers and the TLM tag is not used. These tile length values are the same for both markers.

  TLT: Table 12 shows the size and value of the tile length / tile header parameter.

  Ltlt: Tag length in bytes, not including markers (even number).

  Ptlt: Length (in bytes) from the SOT marker of the tile to the next SOT marker (or EOI marker).

  The CPT tag indicates the first byte of all components other than the first component in the tile from the first byte of the SOT. The component encoded data is arranged in a non-interleaved format within each tile and starts on an 8-bit boundary. At this point, the entropy coder is reset.

  This tag is used for the tile header of every tile when the image contains more than one component. The size of the variable length tag depends on the number of components in the image. An example of the syntax of the component pointer is shown in FIG.

  CPT: Table 13 shows the size and value of the parameter of the component pointer.

  Lcpt: Tag length in bytes, not including markers (even number).

  Pcpti: The number of bytes from the SOT tag of the current tile to the beginning of the next component. Since the data of the first component starts immediately after the SOS tag, the number of Pcpt values is smaller than the number of components. New component data begins on an 8-bit boundary.

  The IRS tag indicates a reset in the data from the first byte of the SOT tag of the current tile. These resets are found on 8-bit boundaries after the end of the critical level where encoding is complete. The component of the point at which reset occurs can be determined by the relationship between the CPT tag value and the reset pointer. The length of this tag depends on the number of resets utilized by the decoder. An example of the importance level reset syntax is shown in FIG.

  IRS: Table 14 shows the size and value of the importance level reset parameter.

  Lirs: Tag length in bytes, not including markers (even).

Iirs ⁱ : Number of the current importance level at the i-th reset. This Iirs tag and the corresponding Pirs tag form a kind of record, which is repeated for each reset. These records are ordered from the highest importance level with reset to the lowest importance level with reset, followed by the importance level of the next ingredient, and so on until the last ingredient. .

Pirs ⁱ : The number of bytes from the SOT tag of the current tile to the i-th reset byte. The Pirs tag and Iirs tag form a kind of record, which is repeated for each reset. These records must be ordered from the smallest pointer to the largest pointer. That is, these records indicate each reset byte in the order in which it appears in the code stream (the smaller the number, the more physically it appears earlier).

  Specific information tags are included solely for information purposes. These information tags are not necessary for the decoder, but will help the parser.

  For example, the VER tag describes a major version number and a minor version number. This tag is used for the main header. Although this tag is defined, it does not mean a functional level required for decoding an image. In fact, its purpose is to allow any decoder and parser to decode and parse any version of the codestream of the present invention. An example of the version number syntax of the present invention is shown in FIG.

  VER: Table 15 shows the size and value of the version number parameter.

  Lver: The length of the table in bytes, not including the marker (even).

  Vver: Major version number.

  Rver: minor version number.

  The BVI tag associates the number of bits with the importance level relative to the image width. This optional tag is used in the main header. The size of this variable length tag depends on the number of importance levels counted by the encoder. An example of bit-to-importance level syntax is shown in FIG.

  BVI: Table 16 shows the size and value of the tile length main header parameter.

  Lbvi: The length of the tag expressed as the number of bits (not even) including no marker.

Cbvi ⁱ : This informs which component data is described. This Cbvi parameter together with Ibvi and Pbvi forms a record, which is repeated for all components and importance levels described. The order must be such that all importance level descriptions for the first component, all importance level descriptions for the next component, and so on are followed.

Ibvi ⁱ : The importance level number encoded for the number of bytes in Pbvii in the current component. This number (s) is selected during encoding to convey the point of interest of the rate / distortion curve. This Ibvi parameter forms one record with Cbvi and Pbvi, which is repeated for all components and importance levels described.

Pbvi ⁱ : The number of bytes in the encoded file that contains all data associated with the number of importance levels in the main and tile headers and Ibvii. This Pbvi parameter forms one record with Cbvi and Ibvi, which is repeated for all components and importance levels described.

  res: A zero padding byte placed last when needed.

  The ILL tag describes a pointer to a code stream corresponding to the end of the importance level of the encoded data. The ILL tag is similar to the IRS tag, but points to data that is neither reset nor inserted into the 8-bit boundary. This tag allows the parser to find and censor tiles with approximately the same distortion on an image width basis. This tag is optional and is used only in the tile header. The length of this tag depends on the number of importance levels enumerated. An example of the importance level locator syntax is shown in FIG.

  ILL: marker. Table 17 shows the size and value of the importance level locator parameters.

  Lill: Tag length in bytes, not including markers (even).

Ill ⁱ : The importance level number encoded for the number of bytes in Pilli. Each of these numbers is selected at the time of encoding to convey the point of interest of the rate / distortion curve. This Iill number, together with the Pill parameter, forms one record, which is repeated in the order of highest importance level to lowest importance level for the fastest component, and so on. A similar record follows up to the sex level.

Pil ⁱ : The byte from the first byte of the SOT of the current tile indicates the byte that completes the importance level of Illi in the encoded data of the tile. This Pill number forms one record with the Iill parameter, which is repeated in order from the highest importance level to the lowest importance level in the fastest component, and so on, from the highest importance level in later components to the lowest. A similar record follows up to the critical level.

  The RXY tag defines the X resolution and Y resolution of the image reference grid with respect to actual dimensions. This tag is only used for the main header. An example of the resolution (pixel / unit) syntax is shown in FIG.

  RXY: Table 18 shows the size and value of the parameter for designating the resolution (pixel / unit).

  Lrxy: Tag length in bytes, not including markers (even).

  Xrxy: the number of reference grid pixels per unit.

  Yrxy: number of reference grid lines per unit.

  RXrxy: X-dimensional unit. Thus, the horizontal resolution is Xrxy grid pixels / 10 (RXrxy−128) meters.

  RYrxy: Y-dimensional unit. Therefore, the vertical resolution is Yrxy grid lines / 10 (RYrxy-128) meters.

  The CMT tag allows unstructured data in the header. This tag can be used for either the main header or the tile header. The length of this tag depends on the length of the comment. An example of the syntax of the comment is shown in FIG.

  CMT: Table 19 shows the size and value of the comment parameter.

  Lcmt: Tag length in bytes, not including markers (even number).

  Rcmt: Tag registration value. Table 20 shows registration parameter values.

  Ccmti: bytes of unstructured data. Repeated arbitrarily.

  res: zero padding byte placed last when needed.

  The QCS tag describes how much the quantized code data has been quantized. When quantization is performed by the parser or encoder, this tag helps the encoder roughly determine how far to encode with respect to the importance level. This tag is optional and is used only for tile headers. An example of the syntax of the quantized code stream is shown in FIG.

  QCS: Table 21 shows the sizes and values of the parameters of the quantized code stream.

  Lqcs: Tag length in bytes, not including markers (even number).

Cill ⁱ : current component number. This Cill number forms a record with Iqcs, which is repeated in order from the highest importance level to the lowest importance level in the fastest component, from the highest importance level to the lowest importance level in later components. Followed by a similar record that identifies

  Iqcsi: This is the importance level where at least a portion of the encoded data remains. All data remaining from that point until the next reset has been censored (quantized).

  res: A zero padding byte placed last if necessary.

Loss factor reconstruction In one embodiment, the present invention performs lossy reconstruction by rounding values to a set of predetermined integer values. For example, all coefficients between 0 and 31 are quantized to 0, all coefficients between 32 and 63 are quantized to 32, and so on. FIG. 47 shows a typical distribution of coefficients when not quantized. Such quantization may be performed when the lowest bit of each coefficient is not known. In another embodiment, the central value of each range may provide a more accurate value that represents the coefficient group. For example, all coefficients between 64 and 127 are quantized to 95. When a value is quantized to a point, that point is called a reconstruction point.

  Due to the differences between the images, the resulting distribution is distorted. For example, compare curve 2701 and curve 2702 in FIG.

  In the present invention, the reconstruction point is selected based on its distribution. In one embodiment, a distribution is estimated and a reconstruction point is selected based on the estimated distribution. The estimated distribution is generated based on already known data. Prior to collecting data, the default reconstruction point will be used. Thus, the present invention provides an adaptive lossy reconstruction method. Furthermore, the present invention is a non-iterative method that improves coefficient reconstruction. In order to compensate for the non-uniform use of the range due to the difference in distribution, the present invention defines as follows.

ただし、２^Sは利用できるデータを基に復号化器により測定された標本分散であり、Ｑは復号化器に知らされた量子化である。次に、非ゼロ係数を０から遠ざけることによって、それを修正する。

Where 2 ^S is the sample variance measured by the decoder based on the available data, and Q is the quantization informed to the decoder. It is then corrected by moving the non-zero coefficient away from zero.

ただし、ｉは任意の整数である。

However, i is an arbitrary integer.

  In one embodiment, all non-zero coefficients are adjusted to a certain reconstruction level after all decoding is complete. To make this adjustment, it is necessary to read, possibly modify, and write each coefficient.

  In another embodiment, when each bitplane of each coefficient is processed, if that coefficient is non-zero, the appropriate reconstruction value for that coefficient is stored. When decoding stops, all coefficients are set to their appropriate reconstructed values. This eliminates the need for a separate memory for setting the reconstruction level.

Color The present invention can be applied to color images (and data). The multi-component processing mechanism 101 of FIG. 1 performs processing required for color data. For example, the YUV color space has three components, that is, a Y component, a U component, and a V component, and each component is encoded separately.

  In one embodiment, the entropy encoded data for each component is separated from the entropy encoded data for the other components. In this embodiment, there is no component interleaving. Separating data by component helps, when combined with pyramid alignment, allows a decoder or parser to easily quantize different components separately.

  In other embodiments, different components of entropy encoded data are interleaved in frequency band units or importance level units. This is beneficial when combined with MSE alignment because a common truncation can be used for quantization of all component data. For this interleaving scheme, the encoder needs to provide a relationship between frequency bands or importance levels of different components. Since the frequency band or importance level will be a fairly large amount of encoded data, the parser or decoder will be able to quantize the components independently using markers.

  In yet another embodiment, different components of entropy encoded data are interleaved pixel by pixel or coefficient. This is beneficial because when combined with MSE alignment, a common truncation acts on all components. For pixel-by-pixel interleaving, the decoder and parser must make use of the same component relationships as defined in the encoder.

  According to the present invention, subsampling can be performed in the same system.

  In one embodiment, each component is stored separately. By using a decompressor and parser, when generating a lossy output image, only a selection of decomposition levels and components will be obtained from each of the separate component memories. For example, in the YUV color space, all of the decomposition levels will be acquired for the Y color component, but all of the decomposition levels other than the first decomposition level will be acquired for the U component and the Y component. The resulting image combination is a 4: 1: 1 image. Note that different types of images can be obtained by using different portions of the data stored in the memory.

  Many types of multi-component images can be processed. In addition to YUV, the image data may be RGB (red, green, blue), CMY (cyan, magenta, yellow), CMYK (cyan, magenta, yellow, black) or CCIR601 YCrCb. Multispectral image data (eg, remote sensing data) may also be used. For visual data such as RGB and CMY, US patent application Ser. No. 08 / 436,662 (accepted May 8, 1995, “Method and Apparatus for Reversible Color Compression”) Lossy color space conversion can be used.

Bit Extraction The present invention can calculate a context model and encode bits to enhance bit extraction. Specifically, the context model for head bits is based on information provided by neighboring pixels. Often this context is zero, especially when performing lossy compression. Due to the approximate statistics of the headbit context, the present invention provides a mechanism for maintaining the context for the headbit.

  In one embodiment, the memory is cleared prior to encoding. The context remains until its parent, one of the neighboring pixels, or the pixel of interest changes. When changed, the context memory is updated for all affected contexts. When using tail information, only neighboring pixels and children are updated. When the head bit is on, the memory is updated only once per coefficient.

  In one embodiment, each coefficient is stored as a 32-bit integer consisting of 1 bit of sign, 4 bits of tail-on information, 8 bits of context, followed by 19 bits of the coefficient. An example of the coefficients is shown in FIG.

  In one embodiment, four different cases of tail-on information are used to generate five different cases.

  When the 4-bit value of the tail-on information is 0, the bit of the current bit plane of the absolute value bit of the current coefficient is encoded using the context bit. If the bit is 0, the process ends. If the bit is 1, the coefficient sign is encoded. The first bit of tail-on information is then inverted, the north, northeast, west, south, east and four child contexts are updated and the process ends.

  In the case where the 4-bit value of the tail on information is 1, the bits of the current bit plane of the absolute value bits of the current coefficient are encoded using the constant context for that case. The second bit of the tail on information is inverted. The east and child contexts of the current coefficient are updated. The process ends.

  In the case where the 4-bit value of the tail-on information is 7, the bits of the current bit plane of the absolute value bits of the current coefficient are encoded using the constant context for that case. The third bit of the tail on information is inverted. No update is required for any context. The process ends.

  In the case where the 4-bit value of the tail-on information is 3, the bits of the current bit plane of the absolute value bits of the current coefficient are encoded using a certain context for the case. The fourth bit of the tail on information is inverted. The east and child contexts of the current coefficient are updated. The process ends.

  In the case where the 4-bit value of the tail-on information is 15, the bits of the current bit plane of the absolute value bits of the current coefficient are encoded using the constant context for that case. No bits of tail-on information need be inverted. The process ends.

  FIG. 48 shows an example of the coefficient of the present invention. In FIG. 48, a coefficient 2801 is composed of a sign bit 2802, a tail neon information bit 2803 that follows, a context bit 2804 that follows, and a coefficient absolute value bit 2805 that follows. The foregoing process is illustrated in the flowchart of FIG.

  By using this technique of updating the entire context when a change occurs, context modeling works fast as long as the head bit is predominantly zero. This is especially true for lossy coding.

Huffman Coding of Lossless Wavelet Coefficients The present invention, in one embodiment, encodes wavelet coefficients using Huffman coding. The alphabet for Huffman coding consists of two parts. The first part is equal to the length of a zero coefficient run, and the second part is a hash value of a non-zero terminator coefficient. FIG. 53 shows the alphabet format, which consists of the number of 0 coefficients, in other words, 4 bits indicating the length of the run, and 4 bits indicating the hash value from 0 to 15 following the number.

  This hash value is the value N, which is the integer part of the logarithm base 2 of the absolute value of the non-zero terminator coefficient. In one embodiment, this hash value is the number of bits required to represent the value N. For example, when N = −1, 1, the hash value is 1. On the other hand, if N = −3, −2, 2 and 3, the number of bits required to represent the value N is two. Similar correspondence is used in JPEG.

  In such situations, the maximum length of a zero coefficient run allowed is 15. When a run exceeds 15, a special token will be used to indicate that 0 is a new run followed by 16 runs. One such exception token is all zeros in both the first 4 bits and the last 4 bits. In one embodiment, all 16 tokens where the second 4 bits are 0 are used for exception cases. Thus, there are 256 8-bit Huffman tokens.

  In one embodiment, a table for Huffman tokens is created. In one embodiment, the table is used for all images. In another embodiment, many tables are created and one particular table is chosen depending on the quantization. Each table is selected based on the number of bits to be quantized. That is, the table is selected depending on whether the number of bits to be quantized is 1 bit, 2 bits, 3 bits, or the like. In another embodiment, the Huffman code is for a specific image and is stored / transmitted with the image.

  One Huffman token is generated to use the table. This token is then sent to the table in which it is encoded.

  The Huffman token specifies a run length of zero and a hash value of a non-zero terminator symbol, but an extra bit is required to uniquely identify the terminator symbol. One embodiment of the present invention provides these extra bits. After the Huffman token is replaced with a Huffman codeword (eg, obtained from a table, etc.), an extra bit equal to the hash value of the terminator symbol is written. For example, in the case of -1,1, the extra 1 bit is written, but in the cases of -3, -2,2,3, the extra 2 bits are written. Thus, the present invention provides Huffman coding that is variable in size with additional bits that uniquely identify terminator symbols.

  Other m-ary coders may be used. For example, one alphabet and m-element code may be used for the 0 coefficient, and another alphabet and m-element code may be used for the hash value.

  In one embodiment, a set of Huffman tables for each quantization level is pre-calculated and used for most images. The compression will be monitored while using one table to select between the various tables. Based on the results of using that table, a switch to a table with more or less skew will occur.

  All the coefficients of the present invention are placed in a buffer. For each buffer, a determination will be made as to which table to use. Three bits and one header may be used to indicate which of the eight Huffman tables should be used. Thus, the table selection will be made by informing the header.

  The order in which the coefficients are encoded is important. Note that in prior art coefficient coding, for example in JPEG, the coefficients are compressed in zigzag order. In the present invention, since all the coefficients are in a buffer, they cannot be in zigzag order. The zigzag order can be extended to compression with an embedded wavelet (tree order) if understood as an order from low to high frequency.

  In one embodiment, the coefficients are encoded in a linear order for the entire buffer. Such an example is shown in FIG. Note that in this embodiment, the first block of smoothing coefficients is excluded.

  In another embodiment, all blocks are encoded in raster order into higher frequency blocks than lower frequency blocks. Such an example is shown in FIG. Due to memory constraints, all of one frequency path may not complete before another frequency path begins. When limited by memory, another method is to encode one tree at a time. Starting from the root, all trees are encoded horizontally. However, routes that are smooth coefficients are not included. This method is shown in FIG. FIG. 54C shows the first tree. One line is taken from the first set of subblocks, two lines are taken from the next set of subblocks, and the next set of subblocks is taken. 4 lines are taken. Such an embodiment is possible because these lines are available before other lines become available.

  An exception token may be stored to indicate that the remaining tree consists of zero coefficients. This prevents the same token representing 16 zeros from being used again and again.

  In one embodiment, all importance levels are encoded by Huffman coding. In another embodiment, one or more groups of importance levels are encoded by Huffman coding. All importance levels may be encoded with Huffman encoding for each separate group, or some importance levels may be encoded with Huffman encoding, and the remaining importance levels may be encoded with horizontal context models and binary Encoding may be performed by an entropy coder.

  Coding by Huffman coding of one group of importance levels is performed as follows. When all bits of importance level coefficients in the group are head bits, the coefficients are Huffman coded as zero coefficients (possibly as part of the run count). If the bits of the coefficient are all tail bits, they are encoded as extra bits (possibly terminating the run). Huffman codewords are not used. When the coefficient bits contain sign bits (in addition to head bits or tail bits), both the Huffman codeword and the extra bits (possibly terminating the run) are encoded.

  Executing Huffman coding for multiple importance levels reduces the execution cost. However, truncation in the middle of Huffman encoded data leads to deterioration of rate and distortion. If a group of importance levels is Huffman coded, it becomes possible to censor the group at the beginning / end of the group for better rate distortion. For some applications, a limited number of required quantization points are known at the time of encoding. Importance levels without quantization points can be Huffman coded along with subsequent levels.

Applications The present invention can be used in many applications. Some such applications are described below by way of example. Specifically, the present invention can be used for a hand-end application with a high resolution and a large pixel depth and an application that does not allow artifacts. According to the present invention, high-end applications can maintain the highest quality in a high-quality environment, and at the same time, the same compressed data can be used in applications where bandwidth, data storage or display functions are further limited. This is exactly the device-independent representation that is generally required in modern image applications such as web browsers.

  The superior lossless compression performance of the present invention for deep pixel depth images (10 bits to 16 bits / pixel) is ideal for medical images. In addition to lossless compression, the present invention is a true lossy compression device that is free of many artifacts known to block-based compression devices. Lossy artifacts resulting from the use of the present invention tend to be along sharp edges and are therefore often not visible due to the visual masking phenomenon of the human visual system.

  The present invention can be used in applications related to the pre-press industry where images are often very high resolution and have a high pixel depth. The pyramid decomposition of the present invention makes it easy for a prepress operator to perform image processing operations on a low resolution lossy version of an image (on a monitor). Once the operation is over, the same operation can be performed on the lossless version.

  The present invention can also be applied to a facsimile document that tends to take a long time if it is not compressed. According to the present invention, it is possible to output a very high-quality image from a fax apparatus having various spatial resolutions and pixel resolutions.

  The present invention can also be used in image archiving systems that require compression, particularly to increase storage capacity. The device independent output of the present invention is beneficial because it allows the image archiving system to be accessed by systems with different bandwidth resources, memory and display. The progressive transmission function of the present invention is also useful for browsing. Finally, the present invention provides a lossless compression that is desirable for output devices of image archiving systems.

  Due to the hierarchical progressive nature of the non-lossy or high quality lossy data stream of the present invention, the present invention is ideal for the World Wide Web, especially where device independence, progressive transmission and high quality are essential.

  The present invention is also applicable to satellite images, particularly satellite images that tend to have high pixel depth and high resolution. Furthermore, the use of satellite images limits the bandwidth of the communication path. Since the present invention is flexible and has progressive transmission characteristics, it will be possible to browse or preview images by humans.

  Applications where bandwidth is limited at a "fixed rate", such as ATM networks, require a means to reduce data when the data overflows available bandwidth. However, there should be no quality penalty when there is sufficient bandwidth (or when the data is highly compressible). Similarly, "fixed size" applications such as memory limited frame stores in computers and other imaging devices also require a means to reduce data when the memory is full. Again, there should be no penalty for images that can be losslessly compressed to an appropriate amount of memory.

  The embedded codestream of the present invention is suitable for both these applications. The embedding operation unconditionally allows the code stream to be truncated or truncated for transmission or storage of lossy images. If truncation or truncation is not required, the image will arrive without loss.

  In summary, the present invention provides a single continuous tone image compression system. The system of the present invention is lossless and lossy for the same code stream, and utilizes embedded quantization (included in the code stream). The system of the present invention is also pyramid-type, progressive, provides interpolation means and is easy to implement. Thus, the present invention provides a flexible “device independent” compression system.

  Integrated lossy and lossless compression systems are very useful. The latest lossy and lossless compression can be performed on the same system, as well as the same code stream. The system can determine whether the lossless code of the image is preserved or truncated into a lossy version during encoding, during storage or transmission of the code stream, or during decoding.

  The lossy compression provided by the present invention is achieved by embedded quantization. That is, the code stream includes quantization. The actual quantization (or visual importance) level may be a correlation point element with the decoder or channel, not necessarily a correlation element with the encoder. If bandwidth, storage and display resources allow, the image is restored losslessly. If not, the image is quantized as required by the most constrained resource.

  The wavelet used in the present invention is a pyramid type, and ½ decomposition of an image without a difference image is executed. This is a very special hierarchical decomposition. The pyramidity of the present invention is ideal for applications that require thumbnails for image browsing or for display by low resolution devices.

  The use of embedding in the present invention is progressive, more specifically in bit-plane order, that is, the order in which the MSB is followed by the lower bits. Specifically, although the present invention is progressive in the wavelet domain, both the spatial domain and the wavelet domain may be decomposed progressively. For applications that have spatial resolution but low pixel resolution, such as printers, the progressive ordering of bits in the present invention is ideal. These features are obtained with the same code stream.

  The present invention can be implemented relatively easily with either software or hardware. The wavelet transform can be calculated with only four addition / subtraction operations and a few shifts for each high-pass and low-pass coefficient pair. Embedding and encoding is performed by a simple “context model” and a binary or m-ary “entropy coder”. This entropy coder can be realized by a finite state machine, a parallel coder or a Huffman coder.

It is a block diagram of one Example of the compression system of this invention. It is a figure which shows an example of the possible geometric relationship of the context model with respect to each bit of each bit plane in a binary system. It is a figure which shows an example of the possible geometric relationship of the context model with respect to each bit of each bit plane in a binary system. FIG. 6 is a diagram illustrating a first level decomposition. FIG. 6 is a diagram illustrating a second level decomposition. FIG. 6 is a diagram illustrating a third level decomposition. FIG. 6 is a diagram illustrating a fourth level decomposition. The parent-child relationship between the two levels before and after is shown. It is a figure which shows an example of the wavelet decomposition | disassembly process using only TT transformation. It is a figure which shows an example of the wavelet decomposition process using TT conversion and S conversion. It is explanatory drawing of the tiling of an image. It is a figure which shows the example of bit-significance expression. It is a figure which shows the coefficient size in this invention. It is a figure which shows the example of the multiplier for frequency bands used for the coefficient alignment in this invention. It is a figure which shows an example of a structure of a code stream. It is a figure which shows the adjacent relationship between a coefficient (or pixel). It is a flowchart of a tail bit processing process. 3 is a flowchart of an example of an encoding process of the present invention. 6 is a flowchart of an example of a decoding process of the present invention. 4 is a flowchart of the modeling process of the present invention. FIG. 4 shows templates that can be used for the modeling process. It is a block diagram which shows an example of a part of TT conversion filter. It is explanatory drawing of the scroll buffer of this invention. It is explanatory drawing of memory operation employ | adopted as this invention. It is a figure which shows the two-dimensional representation of the memory buffer for 3 levels. It is a figure which shows an example of the code stream of this invention. It is a block diagram of the compression system provided with the parser. FIG. 28 is a block diagram of a decompression system corresponding to the compression system of FIG. 27. It is a figure which shows a context dependency. It is a figure which shows the use defined from the surface of pixel depth and spatial resolution. It is a block diagram which shows an example of interaction with a parser, a decoder, and those output devices. It is a block diagram which shows an example of a quantization selection apparatus. It is a figure which shows arrangement | positioning of the division | segmentation tag in a code stream. It is explanatory drawing of a SIZ tag. It is explanatory drawing of a COD tag. It is explanatory drawing of an ALG tag. It is explanatory drawing of a TLM tag. It is explanatory drawing of a TLT tag. It is explanatory drawing of a CPT tag. It is explanatory drawing of an IRS tag. It is explanatory drawing of a VER tag. It is explanatory drawing of a BVI tag. It is explanatory drawing of an ILL tag. It is explanatory drawing of a RXY tag. It is explanatory drawing of a CMT tag. It is explanatory drawing of a QCS tag. Figure 6 is a graph showing a typical distribution for lossy reconstruction. It is a figure which shows a typical coefficient. It is a flowchart of a tail information analysis process. It is a figure for demonstrating the MSE alignment method. It is a figure for demonstrating the pyramid alignment method. FIG. 5 illustrates an exemplary relationship between memory storage coefficients and alignment. It is a figure which shows an example of a code word. It is a figure for demonstrating the method of the syntax analysis of the coefficient by a Huffman encoding method. It is a figure which shows the intermediate | middle format of 2D memory in the case of performing a 2nd level wavelet decomposition using a unit buffer. It is a figure which shows the intermediate | middle type | mold of 2D memory in the case of performing a 3rd level wavelet decomposition using a unit buffer.

Explanation of symbols

101 Input Image Data 102 Lossless Wavelet Transform Block 103 Embedding Ordered Quantization Block 104 Gray Coding Block 105 Horizontal Context Model Block 106 Entropy Coder 110 Method Selection Mechanism 111 Multi-Component Processing Mechanism 1001 Header 1002 Coding Unit 1003 LL Coefficient 1004 First bit plane 1005 Second bit plane 1006 Final bit plane 1501 Multiplier 1502 Adder 1503 Multiplier 1504 Multiplier 1505 Adder 1601 Line access buffer 1602 Buffer 1901 Header 1902 LL coefficient 1903 Entropy encoded data 2101 Compressed Original image 2102 Compressor 2103 Lossless compressed bitstream with marker 2 04 parser 2106 communication path or storage device 2107 decompression device 2108 decompressed image 2401 lossless compressed data with marker 2402 parser 2403 communication path 2404 decompression device 2405 display module 2500 code stream 2501 decompression including quantization 2502 image processing or distortion model 2503 Lossless extension 2504 Image processing or distortion model 2505 MSE or HVS difference model 2506 Alignment adjustment 2801 Coefficient 2802 Sign bit 2803 Tail-on information 2804 Context bit 2805 Coefficient bit

Claims (17)

A memory for storing a code stream having a header with at least one marker;
At least one output device,
A data compression system comprising a parser connected to the memory and connected to receive device characteristics from the at least one output device, the parser being operable to perform device dependent quantization.   2. The data compression system according to claim 1, wherein the code stream is composed of lossless compressed data.   The data compression system of claim 1, wherein the at least one marker indicates the number of components used for each tile in the codestream, subsampling and alignment.   2. The data compression system of claim 1, wherein the code stream includes a main header, and a local header is placed before each tile in the code stream.   5. A data compression system according to claim 4, wherein the main header is applied to all tiles in the codestream and each local header is applied only to the associated tile.   6. The data compression system according to claim 5, wherein at least one of the local headers has priority over the main header.   2. A data compression system according to claim 1, wherein the parser uses markers in the code stream to quantize the code stream.   8. The data compression system according to claim 7, wherein at least one of the markers indicates frequency information.   The data compression system according to claim 1, further comprising a compression device for generating a code stream.   2. The data compression system according to claim 1, wherein the parser comprises a quantization selection device.   11. The data compression system according to claim 10, wherein the quantization selection device performs conversion and quantization of a set of images by discarding bit planes of various coefficients.   The data compression system of claim 1, wherein one of the tags indicates a level of importance within the data in each tile.   2. The data compression system of claim 1, wherein the tag indicates an importance level locator signal, and the parser terminates according to the signal.   2. A data compression system according to claim 1, wherein the tag indicates the number of importance levels to be stored.   2. The data compression system according to claim 1, wherein the tag indicates the number of bytes to be stored.   The data compression system according to claim 1, wherein the tag includes an instruction for associating the importance level with the number of bytes in each tile.   8. The data compression system of claim 7, wherein the at least one marker indicates the number of bytes of importance level in each tile.

Images