JP2007267384A - Compression apparatus and compression method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To perform high-speed lossy/lossless compression. <P>SOLUTION: In an integrated circuit (IC) incorporating a printer compression/decompression function, during encoding, pixel data are input from a band buffer through a pixel data I/F 2801 to a wavelet transform block 2803, and converted to reversible wavelet, and highest importance data of a transform coefficient are applied to a context model 2805 for encoding. Lowest importance data of the transform coefficient are stored through a coefficient data I/F 2804 into the band buffer, then read out, given to the context model 2805, and encoded. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to the field of data compression and decompression systems, and more particularly to lossless and lossy data encoding and decoding in compression / decompression systems.

  Data compression is a very useful tool for storing and transmitting large amounts of data. For example, the time required for image transmission such as facsimile transmission of a document is drastically shortened by reducing the number of bits required for image reproduction using compression.

  Conventionally, many different data compression methods exist. The compression method can be roughly divided into two categories: lossy coding and lossless coding. Lossy encoding is an encoding that results in loss of information and thus does not guarantee a complete reproduction of the original data. The goal of lossy coding is to keep the change from being uncomfortable or noticeable even if it changes from the original data. In lossless compression, all information is stored and the data is compressed in a fully recoverable manner.

In lossless compression, input symbols or luminance data are converted into output codewords. As input, image data, audio data, one-dimensional data (for example, data that changes spatially or temporally), two-dimensional data (for example, changes in two spatial axis directions (or one spatial dimension and one temporal dimension) Data), or multidimensional / multispectral data. If compression is successful, the codeword is represented with a number of bits less than that required for the input symbol (or luminance data) before encoding. Lossless coding methods include a dictionary coding method (for example, a Lempel-Ziv method), a run length coding method, a count coding method, and an entropy coding method. In lossless image compression, the compression is based on prediction or context and coding. JBIG standard for facsimile compression (ISO / IEC
11544) and DPCM (Differential Pulse Code Modulation-JPEG standard option for continuous tone images) are examples of lossless compression for images. In lossy compression, input symbols or luminance data are quantized and then converted to output codewords. The purpose of quantization is to preserve important feature quantities of data while removing unimportant feature quantities. Lossy compression systems often use transforms to concentrate energy prior to quantization. JPEG is an example of a lossy encoding method for image data.

  Recent developments in image signal processing have focused on the pursuit of efficient and highly accurate data compression and coding schemes. Various schemes for transformation or pyramid signal processing have been proposed, including multi-resolution pyramid processing schemes and wavelet pyramid processing schemes. These two methods are also called a subband processing method and a hierarchical processing method. The wavelet pyramid processing method for image data is a special multi-resolution pyramid processing method for subband division of an original image using a quadrature mirror filter (QMF). There are other non-QMF wavelet methods. For more information on the wavelet processing method, refer to Non-Patent Documents 1 and 2, for example. To obtain information on reversible transformation, see Non-Patent Document 3.

Antonini, M., et al., “Image Coding Using Wavelet Transform”, IEEE Transactions on Image Processing, Vol.1, No.2, April 1992 Shapiro, J., “An Embedded Hierarchical Image Coder Using Zerotrees of Wavelet Coefficients”, Proc. IEEE Data Compression Conference, pgs. 214-223, 1993 Said, A. and Pearlman, W., “Reversible Image Compression via Multiresolution Representation and Predictive Coding”, Dept. of Electrical, Computer and System Engineering, Renssealaer Polytechnic Institute, Troy, NY 1993

  Compression is often very time consuming and requires a large amount of memory. It is desirable to perform compression at higher speeds and / or with as little memory as possible. In some applications, compression has not been used at all because quality cannot be guaranteed, compression rate is insufficient, or data rate is not controllable. However, the use of compression is desirable to reduce the amount of information to be transmitted and / or stored.

  Digital copiers, printers, and multi-function devices have greatly improved performance with frame storage. If frame storage is compressed, the memory required for frame storage in these products is reduced, thus reducing its cost. However, many frame stores are implemented with random access memory (RAM). RAM is fast but generally expensive. A hard disk can also be used as memory and is generally considered cheap (generally not as expensive as RAM). Therefore, a system manufacturer would find it advantageous to use a hard disk instead of RAM for frame storage or the like to manufacture a cheaper system.

  One problem with using a hard disk for applications where speed is important is that it is difficult to call information directly from the hard disk as fast as RAM. Many hard disks also use compression to increase the amount of information that can be stored on the disk when information is stored on the disk. The time it takes to perform this compression may also be a factor in discouraging the use of hard disks in speed critical applications. The slowness inherent in the use of hard disks and the use of compression makes it difficult to use hard disks for speed critical applications.

  Therefore, the main object of the present invention is to enable high-speed lossy / lossless compression. Another object of the present invention is to provide a system that can use an inexpensive hard disk instead of an expensive RAM. Another object of the present invention is to match the speed to the hard disk and use compression to allow the hard disk to be matched to the bandwidth of other parts of the system, such as the print engine. Another object of the present invention is to allow the RAM to be used when the time for compression and decompression is not very slow compared to the RAM speed. Other objects of the present invention will become apparent from the following description.

The invention of claim 1 is a compression apparatus for compressing coefficients generated as a result of wavelet transform, the means for dividing the coefficients into highest importance data and low importance data, and the highest importance data. Means for immediately sending out to the context model for encoding in order of coefficients; a memory storing the low importance data and a plurality of instruction bits in the low importance data; and after encoding the highest importance data, And means for encoding the low importance data stored in the memory and performing ordering based on the plurality of instruction bits.

The invention of claim 2 is a compression method for compressing a coefficient generated as a result of wavelet transform, the step of dividing the coefficient into highest importance data and low importance data, and the highest importance data Immediately sending to the context model for encoding in order of coefficients; storing the low importance data and a plurality of instruction bits in the low importance data in memory; and after encoding the highest importance data And encoding the low importance data stored in the memory and performing ordering based on the plurality of instruction bits.

  According to the present invention, high-speed lossy / lossless compression is possible, an inexpensive hard disk can be used for the compression / decompression system, the speed is matched to the hard disk, and the hard disk is compressed using compression. Many effects can be obtained, such as being able to match the bandwidth of other parts of the system, such as a print engine.

  A method and apparatus for compression and decompression is described. In the following description, various specific examples such as delay type, bit rate, filter type, and the like are shown. However, it will be apparent to those skilled in the art that the present invention may be practiced without such specific examples. On the other hand, well-known structures and devices are shown in block diagram form and are not shown in detail in order not to obscure the present invention.

  A significant portion of the detailed description below is given by algorithms and symbolic representations of operations on data bits in computer memory. Such algorithmic descriptions and representations are the means used by those skilled in the data processing arts to most effectively convey the substance of their work to others skilled in the art. Suppose that there is an algorithm that is generally considered a self-consistent sequence of steps that leads to the desired result. These steps are those requiring physical processing of physical quantities. Usually, though not necessarily, these physical quantities take the form of electrical or magnetic signals capable of being stored, transferred, combined, compared, and otherwise manipulated. It has proven convenient at times, principally for reasons of common usage, to represent these signals as bits, values, elements, symbols, characters, terms, numbers, or the like.

  However, it should be noted that such and similar terms are to be associated with the appropriate physical quantities and are merely convenient labels attached to these physical quantities. As will be apparent from the following description, unless otherwise specified, discussion using terms such as “processing”, “operation”, “calculation”, “judgment”, “display”, etc. Processes data expressed as (electronic) quantities and converts them into other data expressed as physical quantities in the same way as memory or registers in computer systems, similar information storage devices, information transmission devices or display devices It refers to the operation and process of a computer system or similar electronic computing device.

  The present invention also relates to an apparatus for performing the operations described herein. This device may be made exclusively for the required purpose, or it may be a general-purpose computer selectively driven or reconfigured by a built-in program. Such programs include, but are not limited to, floppy disks, optical disks, CD-ROMs, magneto-optical disks, read-only memories (ROMs), random access memories (RAMs), EPROMs, EEPROMs connected to the computer system bus. May be stored in a computer readable storage medium, such as a magnetic card or optical card, or other medium suitable for storing electronic instructions. The algorithms and displays presented herein are essentially unrelated to any particular computer or other device. Various general purpose machines may be utilized with programs according to those described herein, but it may be more convenient to create a more specialized device for performing the required method steps. Absent. The required structure for a variety of these machines will appear from the description below. In addition, the present invention is described without being associated with any particular programming language. It will be appreciated that a variety of programming languages may be used to implement the invention, as described herein.

  The following terms are used in the description below. These terms already have meaning. However, the defined meanings should not be considered limited to the extent that the terms are known in the art. These terms are defined in order to help understanding of the present invention.

ABS encoding:
A parallel entropy encoding method that uses a simple code (for example, a run code) for bit generation and a probability prediction based on a used codeword (for example, a probability prediction using a table). In one embodiment, ABS coding also includes methods for multiplexing and demultiplexing streams from several coders.

Alignment:
The degree of shift of the transform coefficient within a frequency band relative to other frequency bands.

Arithmetic coding:
It is not limited to Shannon / Elias encoding and binary entropy coder by finite precision arithmetic.

B encoding:
A binary entropy coder that uses a finite state machine for compression. Unlike Huffman coding, the use of a finite state machine handles binary symbols well and is effective for a range of input probabilities.

Binary entropy coder:
A noiseless coder that operates on binary (yes / no) decisions, often expressed as the highest probability symbol (mps) and the lowest probability symbol (lps).

Binary method:
A coding scheme using pixel edge-fill Gray encoding and a special context model.

Binary context model:
A context model for image data with a small number of binary values and gradations.

Bit-significance:
A number representation similar to the sign absolute value representation, where the head bit is followed by a sign bit, followed by a tail bit, if any. Embedding encodes this number representation in bit-plane order.

Child-based order:
Scan order of 2D images. Similar to raster order, but operates in 2 × 2 block units. Consider scanning a “parent” frequency band in raster order. Each coefficient will have 4 children. These children are ordered from top left to top right, bottom left, bottom right, followed by the next parent and next four child pairs, and so on until the end of the line. The process then returns to the next two lines and finally ends at the lower right corner. No lines are skipped. The child base order is also called 2 × 2 block order.

coefficient:
Ingredient after conversion.

component:
Image component. A component constitutes a pixel. For example, the red, green, and blue bands are component bands. Each pixel is composed of red, green, and blue components. Components and component bands can contain any type of information that has a spatial mapping to the image.

Context model:
Information that can be causally used for the current bit to be encoded and provides previously learned information about the current bit to enable conditional probability prediction for entropy coding. In a binary image, one possible context for a pixel is the two preceding pixels on the same line and the three pixels on the previous line.

Split level:
Position in wavelet split pyramid. It is directly related to resolution.

Efficiency conversion:
A transformation that achieves the highest energy concentration in the coefficients while representing them using the least number of bits.

Embedded context model:
A context model that divides context bins and results into importance levels so that efficient lossy compression can be achieved if important values are preserved.

Embedding by ordering:
A special case of an embedded context model where there is no explicit importance labeling, but the compressed data is ordered so that the most important data comes first.

Embedded quantization:
Quantization included in the codestream. For example, when the importance levels are arranged in order from the highest level to the lowest level, the quantization is performed by simply truncating the code stream. The same function can be obtained by tags, markers, pointers, and other signals. Multiple quantization can be performed on an image at the time of decoding, but only single embedded quantization can be performed at the time of encoding.

Entropy coder:
An apparatus for encoding or decoding current bits based on probability prediction. As used herein, an entropy coder may also be referred to as a multiple context binary coder. The context of the current bit is a “neighbor” bit in some chosen arrangement, allowing probability prediction for an optimal representation of the current bit (one or more bits). In one embodiment, the entropy coder is a binary coder, a parallel run length coder, or a Huffman coder.

Entry points:
A point in the encoded data that begins with a known encoding state. From this point, the decoder can start decoding without decoding the previous data. This usually requires resetting the context and binary entropy coder to the desired state. The encoded data of each encoding unit starts from an entry point.

Fixed length: A method of converting a specific block of data into a specific block of compressed data. For example, some methods of BTC (block coding) and VQ (vector quantization). Fixed length codes are suitable for fixed rate and fixed size applications, but rate and distortion performance is often inferior to variable rate systems.

Fixed rate:
An application or system that maintains a certain pixel rate and has a limited bandwidth channel. In one embodiment, locally averaged compression is achieved to achieve this goal, rather than globally averaged compression. For example, MPEG requires a fixed rate.

Fixed size:
An application or system that has a limited size buffer. In one embodiment, overall average compression is used to achieve this goal. For example, print buffer. (Applications can be fixed rate, fixed size, or both.)
frequency band:
Each frequency band represents a group of coefficients produced by the same filtering sequence.

Head bit:
In the bit-significance representation, the head bit is an absolute value bit including the first non-zero bit from the most significant bit to the first non-zero bit.

Huffman coder:
In general, a fixed length coder that generates an integer number of bits for each symbol.

Importance level:
A unit of encoded data before compression corresponding to one entire bit plane of embedded data. The importance level includes all the appropriate bit planes from different coefficient frequency bands.

LPS (lowest probability symbol):
An event in a binary decision with a probability of less than 50%.
Which is chosen to be mps or lps when the two events are of equal probability is not important if the encoder and decoder make the same choice.

Lossless / noiseless / lossless coding:
Data compression in a way that allows complete restoration of the original data.

Lossy encoding:
Data encoding that does not guarantee complete restoration of the original data. It may change from the original data so that it is not visually uncomfortable or noticeable. In many cases, a fixed rate is possible.

MPS (highest probability symbol):
An event in a binary decision with a probability greater than 50%.

Duplicate conversion:
A transformation in which a single source sample point contributes to multiple coefficients of the same frequency. Examples include many wavelets and overlapping orthogonal transforms.

Parent coefficient: The coefficient or pixel at the next higher level that covers the same image space as the current coefficient or pixel. For example, in FIG. 1, the parent of the 1SD coefficient is the 2SD coefficient, which is the parent of the 3SD coefficient.

Probability prediction machine / module:
An element of an encoding system that tracks probabilities within a context.

Progressive pixel depth:
A code stream in which bit planes of full resolution data are ordered from shallow to deep.

Progressive pyramid:
A series of resolution components in which each dimension is halved (1/4 in area) each time the resolution is lowered.

Q coder:
A binary arithmetic coder in which multiplication is replaced by addition, the probability is limited to a discrete value, and the probability prediction value is updated when the bit is output.

Raster order:
Scan order of 2D images. Start from the upper left corner, move from left to right, then return to the left edge of the next line, and finally end at the lower right corner. No lines are skipped.

Reversible conversion:
In one embodiment, the efficiency conversion realized by integer arithmetic that can restore the compression result to the original data.

Tail bit (or tail):
In bit-significance representation, a tail bit is an absolute value bit lower than the most significant non-zero bit.

Tile data segment:
The part of the code stream that completely describes one coding unit.

TS conversion:
2. 6 (Two-Six) conversion. A special reversible wavelet filter pair consisting of a 2-tap low-pass analysis filter and a 6-tap high-pass analysis filter. The synthesis filter is a quadrature mirror filter of the analysis filter.

TT conversion:
2.10 (Two-Ten) conversion. A special reversible wavelet filter pair consisting of a 2-tap low-pass analysis filter and a 10-tap high-pass analysis filter. The synthesis filter is a quadrature mirror filter of the analysis filter.

Unified lossless / lossy:
The same compression system provides a codestream that can be lossless or lossy restored. In one embodiment of the invention, this codestream may have no configuration information or instructions for the decoder.

Wavelet filter:
High-pass and low-pass synthesis and analysis filters used for wavelet transform.

Wavelet transform:
A transformation with constraints in both the “frequency” and “time (or space)” domains. In one embodiment, the conversion is made up of a high-pass filter and a low-pass filter. The resulting coefficients are decimated by 1/2 (critically filtered) and the filters are subjected to low pass coefficients.

Wavelet wood:
Coefficients and pixels associated with a single coefficient in the SS part of the highest level wavelet split. The number of coefficients is a function of the number of levels. FIG. 1 shows the coefficients included in one wavelet tree. The span of the wavelet tree depends on the number of division levels. For example, the span of a wavelet tree is 4 pixels for 1 level division, 16 pixels for 2 level division, and so on. FIG. 59 shows the number of pixels aggregated in one wavelet tree for various levels. In the two-dimensional case, each wavelet tree consists of three subtrees called SD, DD, and DS.

<Overview>
The present invention provides a compression / decompression system having an encoding unit and a decoding unit. The encoder serves to encode the input data and generate compressed data, while the decoder serves to decode the already encoded data and generate reconstructed data of the original input data. As input data, there can be various types of data such as images (still images or moving images), voices, and the like. In one embodiment, the data is digital signal data, but digitized analog data, text data formats, and other formats are possible. The source of the data is, for example, a memory or a communication path for an encoding unit and / or a decoding unit.

  In the present invention, the components of the encoding unit and / or the decoding unit may be realized by hardware, or may be realized by software as used on a computer system. The present invention provides a lossless compression / decompression system. The present invention can also be configured to perform lossy compression / decompression.

  The system of the present invention uses high-speed lossy / lossless compression with reversible wavelets, which will be described in more detail later. The system may be a printer such as a laser printer. In one embodiment, the printer significantly reduces the amount of expensive random access memory (RAM) required by using an inexpensive hard disk to store a given page. Compression is used to match the limited bandwidth of the hard disk or other storage device to the larger bandwidth required by the print engine. The encoding technique of the present invention satisfies the high speed and real-time properties required by the print engine, and the present invention can perform either excellent lossy compression or lossless compression depending on the characteristics of the image and the burst property of the hard disk.

  In the following detailed description, an overview of reversible wavelet compression, compressed frame storage applications, color laser printers, and printer chip examples are described. The printer's rendering engine uses the hard disk for storage. Since hard disks are slower than print engines, compression is used to match rates. Display list techniques may also be used to reduce the memory required during rendering. Display list-based rendering engines allow compression systems to process image bands independently. Although the present invention will be described in connection with a printer system, the present invention is applicable to other systems including a compression and / or decompression subsystem.

  An embedded integrated lossless / lossy compression system is also described herein. The embedded nature of the system allows quality to be determined by the transfer rate of the disk. For images that are easy to compress (eg, most documents consisting of text and / or line drawings), lossless compression is performed. For images that are difficult to compress (eg, noisy natural images and / or halftone documents), high quality lossy compression is performed.

  For a description of a system that supports both lossless and high-quality lossy compression of color images, see US patent application Ser. No. 08 / 642,518 (accepted May 3, 1996, “Compression and Decompression with Wavelet Style and Binary Style Including Quantization”. by Device-Dependent Parser ”) and US patent application Ser. No. 08 / 436,662 (accepted May 8, 1995,“ Method and Apparatus for Reversible Color Conversion ”).

<Reversible wavelet>
The present invention utilizes compression by reversible wavelets.

<Wavelet division>
The present invention initially uses reversible wavelets to split an image (as image data) or other data signal. In the present invention, the reversible wavelet transform realizes a complete reconstruction system capable of lossless restoration of a signal having integer coefficients by integer arithmetic. The inverse efficiency transformation is a reversible transformation using a transformation matrix whose determinant is 1 (or almost 1).

  In the present invention, lossless compression can be performed with a finite precision operation by using a reversible wavelet. The result generated by applying the reversible wavelet transform to the image data is a sequence of coefficients.

  The reversible wavelet transform of the present invention can be realized using a set of filters. In one embodiment, the filters are a 2-tap low-pass filter and a 6-tap high-pass filter for performing a transformation referred to herein as a TS transform or a 2 · 6 transform. In another embodiment, the filters are a 2-tap low-pass filter and a 10-tap high-pass filter for performing a transformation referred to herein as a TT transform or a 2 · 10 transform. These filters can be realized only by addition and subtraction (and hard wire bit shift).

  The TT conversion has at least one advantage and at least one disadvantage compared to the TS conversion. The advantage is that the TT transform provides better compression than the TS transform. The disadvantage of TT conversion is that a long 10-tap filter requires high hardware costs.

<2D wavelet division>
Multi-resolution division is performed using the low-pass filter and high-pass filter of the present invention. The number of division levels is variable and may be any number, but currently the number of division levels is from 2 to 8 levels. The maximum number of levels is log 2 (the maximum value of the input length or width).

  The most common way of transforming two-dimensional data such as an image is to apply the one-dimensional filter separately, that is, along the lines and then along the columns. The first level of division yields four different band coefficients, referred to herein as SS, DS, SD, and DD. These letters refer to the aforementioned smooth (S) filter and detail (D) filter corresponding to the low (L) pass filter and the high (H) pass filter, respectively. Therefore, the SS band consists of coefficients obtained from smooth filters in both the line direction and the column direction.

  Each frequency subband of wavelet division can be further divided. The most common way is to further divide only the SS sub-band, which will be further divided when the SS frequency sub-band of each division level is generated. Such multiple division is called pyramid division. Each division is displayed by symbols SS, SD, DS, DD and division level number.

  It should be noted that the coefficient size does not increase by pyramid division even if either the TS transform or the TT transform of the present invention is used.

  When the reversible wavelet transform is applied recursively to the image, the first level segmentation works on the finest detail or resolution. At the first division level, the image is decomposed into four partial images (ie, partial bands). Each partial band represents one spatial frequency band. The first level partial bands are represented as 1SS, 1SD, 1DS, and 1DD. Since the process of dividing the original image includes 1/2 subsampling in both the horizontal and vertical dimensions, the first level subbands 1SS, 1SD, 1DS, and 1DD are respectively the pixels of the image that the input had (or Of the number of coefficients).

  The partial band 1SS includes information on the low frequency in the horizontal direction and information on the low frequency in the vertical direction at the same time. In general, most of the image energy is concentrated in this partial band. The partial band 1SD includes low frequency information in the horizontal direction and high frequency information in the vertical direction (for example, horizontal edge information). The partial band 1DS includes high frequency information in the horizontal direction and low frequency information in the vertical direction (for example, vertical edge information). The partial band 1DD includes high frequency information in the horizontal direction and high frequency information in the vertical direction (for example, texture or oblique edge information).

  Subsequent lower second, third, and fourth division levels are each created by dividing the previous level low frequency SS sub-band. The first level partial band 1SS is divided to create slightly finer second level partial bands 2SS, 2SD, 2DS, and 2DD. Similarly, the partial band 2SS is divided to generate third-level partial bands 3SS, 3SD, 3DS, and 3DD with a coarse definition. Further, by dividing the partial band 3SS, fourth-level partial bands 4SS, 4SD, 4DS, and 4DD having a coarser definition are created. With ½ subsampling, each second level subband is 1 / 16th the size of the original image. Each sample (ie pixel) at this level represents a slightly fine detail at the same position in the original image. Similarly, each partial band of the third level is 1 / 64th the size of the original image. Each pixel at this level represents a fairly coarse detail at the same location in the original image. Further, each partial band of the fourth level is 1/256 the size of the original image.

  Since the divided image is smaller than the original image for sub-sampling, the memory used for storing the original image can be used for storing all of the divided sub-bands. That is, in the case of three-level division, the original image and the divided partial bands 1SS and 2SS are discarded and are not stored.

  Although four sub-band splitting levels have been described, it is possible to generate more levels depending on individual system requirements. Different parent-child relationships may be defined using other transforms such as DCT or one-dimensionally arranged subbands.

According to the wavelet filter of the present invention, pyramid division does not increase the coefficient size.
In other embodiments, other sub-bands besides SS may be split.

<Wavelet tree structure>
There are natural and useful tree structures for wavelet coefficients of pyramid division. The result of the subband division is a single SS frequency subband corresponding to the final division level. On the other hand, there are as many SD, DS, and DD bands as the number of levels. This tree structure reveals that the parents of the coefficients in a certain frequency band are the coefficients in the same frequency band with a lower resolution and have the same spatial positional relationship.

  In the present invention, each tree includes an SS coefficient and three subtrees, that is, subtrees of DS, SD, and DD. The process of the present invention is normally performed on these three subtrees. The root of each tree is a purely smooth coefficient. In the case of a two-dimensional signal such as an image, there are three subtrees, and each subtree has four children. This tree hierarchical structure is not limited to a two-dimensional signal. For example, in the case of a one-dimensional signal, each subtree has one child. Higher dimensions are derived from the one-dimensional case and the two-dimensional case.

  The multi-resolution partitioning process can be performed using a filter sequence. For an example of a two-dimensional two-level transform implemented using a one-dimensional typical filter, see US patent application Ser. No. 08 / 498,695 (June 30, 1995, “Method and Apparatus For Compression Using Reversible Wavelet Transforms and an Embedded Codestream ") and U.S. Patent Application No. 08 / 498,036 (accepted June 30, 1995," Reversible Wavelet Transform and Embedded Codestream Manipulation ").

<Perform forward wavelet transform>
In the present invention, wavelet transformation is performed by two 1-D operations, that is, by first performing a horizontal 1-D operation and then performing a vertical 1-D operation. In one embodiment, one piece of hardware performs horizontal computations and another piece of hardware performs vertical computations.

  The number of iterations is determined by the number of levels. In one embodiment, a four level split is performed using TT transform for both horizontal and vertical directions. In another embodiment, 4-level splitting is performed using TS conversion instead of TT conversion.

  The conversion of the present invention is very efficient in computing efficiency. In one embodiment, the present invention orders the calculations performed in the transformation to reduce the required on-chip and off-chip memory capacity and bandwidth.

<Calculation order and data flow for conversion>
As described above, in the present invention, the basic unit for calculation of transformation is a wavelet tree. Considering the case of 4-level conversion, each wavelet tree is a 16 × 16 pixel block. Pixels of 16 × 16 blocks (all four components in the case of CMYK images) are input to the conversion unit of the present invention, and all possible calculations for coefficient generation are executed. (The inverse transformation is similar, a 16x16 coefficient block is input and all possible calculations are performed)
Since the present invention uses overlap transformation, information from the preceding neighboring tree is stored and used for calculation. The boundary between the current wavelet tree and the preceding neighbor information is referred to herein as a seam. Information stored across the seam to perform the transformation of the present invention will be described in detail later.

<Ordering wavelet trees>
For certain applications (eg printing), the coding unit of the present invention is wide and small in length, so the ordering of the wavelet tree to compute the transform is important. In one embodiment, each coding unit consists of 4096 × 256 pixels.

  In the following description, each encoding unit is composed of 4096 × 256 pixels. However, it should be noted that the ordering described below can be applied to coding units of other sizes. FIG. 2 shows an order similar to the raster order. This order is referred to herein as a long seam transformation order. In FIG. 2, the bold lines indicate the amount of data stored across the seam, and how much memory is required for the conversion calculation. This data is proportional to one wavelet tree in the case of horizontal transformation, but is proportional to the image width (4096 in this example) in the case of vertical transformation. With the amount of memory for this data, it may be necessary to use external memory. However, since it is close to the raster order, data can be output from the conversion unit (for example, to a printer in a printer application) as soon as one horizontal line of the wavelet tree is converted into pixels in the inverse conversion period.

  FIG. 3 shows another order, which is referred to herein as a short seam order. The amount of memory for the seam is proportional to the height of the coding unit (256 in this example) in the case of horizontal transformation, and is proportional to one wavelet tree in the case of vertical transformation. This order greatly reduces the amount of memory required and allows on-chip storage.

  FIG. 4 shows another short seam order. There is a burden that the amount of memory is proportional to two wavelet trees, but the number of consecutive pixels processed in raster order increases. This order or similar order allows more efficient use of the band buffer's fast page mode RAM or EDO (extended data out) RAM with little increase in the cost of seam memory. This efficiency gain is obtained because most of the memory is optimized for accessing adjacent memory locations. Therefore, the use of adjacent memory access increases according to the seam order, thereby improving the memory usage efficiency.

<Calculation to obtain one wavelet tree>
The following equations define both TS conversion and TT conversion. A smooth signal s (n) that is the output of the low-pass filter for the input x (n) and a detail signal d (n) that is the output of the high-pass filter are calculated as follows.

The inverse transformation is shown by the following equation.

However, p (n) is calculated by the following equation.

  TS conversion and TT conversion differ in the definition of t (n).

The case of TS conversion is as follows.

The case of TT conversion is as follows.

  In the following explanation,

Means rounding down, that is, censoring, and is sometimes called the floor function.

<TS conversion>
As a result of using a 6-tap filter and a 2-tap filter in even positions, three pieces of information must be stored. A 6-tap filter requires 2 delays. A 2-tap filter requires one delay to be able to center its result relative to the result of a 6-tap filter. Specifically, it is necessary to store two s (•) values and one d (•) value or partial result of d (•) calculation. The memory of these values is the same regardless of whether the filtering process crosses the seam or not.

  FIGS. 5 to 12 show the results obtained each time a TS transform filter for four-level transform is applied to the wavelet tree of the present invention. In these drawings, the output of the low-pass filter is displayed as “s” instead of smooth, and the output of the high-pass filter is displayed as “d” instead of detail. “B” means an intermediate value used for calculating “d”, that is, an x (2n) −x (2n + 1) value. The “B” value is used at the time of forward conversion, and in the case of reverse conversion, the “d” value that is not used for any calculation is stored at that location. The symbol “sd” indicates that a certain coefficient is the result of applying a low-pass filter in the horizontal direction and then applying a high-pass filter in the vertical direction. The meanings of “ds”, “dd”, “ss”, “dB”, and “sB” are the same. The bold square corresponds to 256 input pixels. The shaded “s”, “ds”, “ss” values are calculated using the previous wavelet tree and stored for use in the current wavelet tree.

  In the case of forward conversion, the input of level 2, level 3 and level 4 conversion is the “ss” coefficient obtained at the previous level. Since the “sd”, “ds”, and “dd” coefficients are complete, they can be output when calculated. Inverse transformation reverses all computations in reverse order, that is, level first to 4th level, then 3rd level, 2nd level, and finally 1st level, direction first to vertical, then horizontal Do in order. In the conversion path, the data flow of the forward conversion and the reverse conversion is the same and only the calculation is different.

<TS conversion hardware>
FIG. 13 is a block diagram of an embodiment of a forward / reverse filter unit for realizing a one-dimensional filter. Only memory and computing units are shown, no hardwired shifts are shown. In FIG. 13, the filter unit 4000 processes both forward and inverse transforms. Other embodiments use separate units for forward and inverse transformations. For forward conversion, an input of size “n” is used and an “s” output and a “d” output are generated. In the case of the inverse transformation, the “s” input and the “d” input are used and another output is generated.

  Adder 4001 is connected to receive n-bit inputs, add the inputs and provide output x (2n + 2) + x (2n + 3). The adder 4002 subtracts one n-bit input from the other n-bit input and outputs a value of x (2n + 2) −x (2n + 3). The outputs of the adders 4001 and 4002 are connected to one input of multiplexers (muxes) 4003 and 4004, respectively. The other inputs of mux 4003 and 4004 are connected to receive s and d inputs, respectively. In one embodiment, the s input is n bits and the d input is greater than n bits.

The outputs of mux 4003 and 4004 are controlled by forward / reverse control signals that indicate whether the filter is in forward mode or reverse mode. In both the forward mode and the reverse mode, the output of the mux 4003 is s (n + 1). On the other hand, the output of the mux 4004 is p (n + 1) in the forward mode and d (n + 1) in the reverse mode. The outputs of mux 4003 and 4004 are connected to the input of register file 4005 together with the feedback of s (n) output from mux 4006. The register file 4005 has an entry for each component for the length of one wavelet. Data typically passes through register file 4005. Based on the spatial position, the input to the register file 4005 is delayed and output. Address input controls the output of register file 4005. In one embodiment, the register file 4005 consists of two memory banks with one port per bank and is used by alternately accessing the two memory banks in a ping-pong manner.
The output of mux 4003 is also the s output of the filter unit.

  The output of register file 4005 is connected to the input of mux 4006 along with the external buffer data at seam buffer input 4020. The output 4006A is s (n-1) obtained by delaying the output of the mux 4003 twice. The output 4006B is s (n) obtained by delaying s (n + 1) once. The output 4006C is p (n) for forward conversion and d (n) for reverse conversion. The mux 4006 is also controlled to output externally buffered seam data to the seam buffer output 4021. The output 4006C is connected to one input of adders 4008 and 4009. The other input of the adders 4008 and 4009 is the output of the mux 4015. mux 4015 handles boundary conditions. On the boundary, mux 4015 outputs a 0 wired to one of its inputs. In some embodiments, the wire 0 may be changed to use a different value. Under non-boundary conditions, the mux 4015 outputs t (n) output by subtracting the other input s (n−1) from one input s (n + 1) by the adder 4007.

The adder 4008 adds the output 4006C of the mux 4006 and the output of the mux 4015, and generates the d output of the filter unit.
The adder 4009 subtracts the output 4006C of the mux 4006 from the output of the mux 4015. The output of the adder 4009 is added by s (n) of the output 4006B of the mux 4006 and the adder 4010 to generate an n-bit output of the filter unit. The output of adder 4009 is also subtracted from s (n), which is output 4006B of mux 4006, by adder 4011, and adder 4011 outputs the other n-bit output of the filter unit in the reverse direction.

  For seams longer than one wavelet tree, the seam data may be stored in on-chip static RAM (SRAM) or external memory, rather than in register file 4005. The mux 4006 provides access to such additional seam memory.

  Most of the hardware cost of the filter unit 4000 is due to the register file 4005. The total required memory amount is determined by the number of filter units. In one embodiment, a total of 60 locations are required to store three values (s, s, d or ss, ss, sd). When more filter units are used, less memory is required for each filter unit. Therefore, the hardware cost for using a large number of filter units is low.

  Fast inverse transformation allows for a reduction in the delay time between the end of decoding and the start of a data output operation such as printing. This reduces the work area memory required for decompression and allows for larger coding units. Fast forward transforms allow the filter to process bursts of data when more bandwidth is available, and therefore transforms process more data into the context model when the context model can process the data quickly with look ahead. Makes it possible to supply to. If forward conversion cannot follow the context model during the encoding period, the bandwidth of the disk being encoded is wasted and the printing start time is delayed. Also, by providing a large number of filters, control and data flow can be simplified.

  FIG. 14 is a block diagram of an embodiment of the first level forward conversion according to the present invention. In FIG. 14, two filter units 401 and 402 as described in FIG. 13 perform the first level of conversion. The filter unit 401 performs level 1 horizontal conversion, and the filter unit 402 performs level 1 vertical conversion. In one embodiment, the first level transform processes a 2 × 2 block of input. The four registers 403 to 406 function as delay units for delaying the output of the filter unit 401. This is called child-based order. The register 403 receives the s output of the filter unit 401, and the registers 404 and 405 receive the d output. The output of register 404 is connected to the input of register 406. The outputs of the registers 403 and 406 are connected to the input of the mux 407, and the s output of the filter unit 401 and the output of the register 405 are connected to the input of the mux 408. The two muxes 407 and 408 select the input of the filter unit 402 from the delayed coefficients output from the filter unit 401.

  Filter unit 401 sequentially processes two vertically adjacent input pairs to generate four coefficients, which are appropriately delayed by registers 403-406 for each component and input to filter unit 402. can do. Three of the four results can be output immediately, but the “ss” output is further processed.

  The first level forward transformation processes groups of 4 pixels in a 2 × 2 arrangement. For explanation, it is assumed that the pixels a and b are in the first row and the pixels c and d are in the second row. The operation of the first level conversion in FIG. 14 is as follows. In the first cycle, the filter unit 401 performs horizontal conversion on the pixels a and b. The filter unit 401 generates Sab and Dab, Sab is stored in the register 403, and Dab is stored in the registers 404 and 405. In the next cycle, the pixels c and d are horizontally processed by the filter unit 401. As a result of applying the filter unit 401, Scd and Dcd are generated, and Scd is stored in the register 403 and Dcd is stored in the registers 404 and 405. In this cycle, the filter unit 402 processes Sab from the register 403 and Scd from the register 405 to execute a vertical path of conversion, and generates SS and SD. Also during this second cycle, the value Dab moves from register 404 to register 406. In the next cycle, the value Dab from the register 406 and the value Dcd from the register 405 are processed by the filter unit 402, and the filter unit 402 generates outputs DS and DD. During the same cycle, the filter unit 401 processes the next 2 × 2 block of a and b pixels.

FIG. 15 is a block diagram of an embodiment of forward conversion according to the present invention. In FIG. 15, a level 1 conversion unit 502 performs level 1 conversion. In one embodiment, the level 1 converter 502 comprises the level 1 converter of FIG. The filter unit 505 processes the conversion levels 2, 3, and 4. Memory 503 stores “ss” coefficients until enough coefficients are obtained to perform the transformation. The number of coefficients that need to be stored is shown in FIG. (Each location stores one coefficient for each component)
The order unit 504 multiplexes the appropriate input to the filter unit 505. The input buffer 501 and the output buffer 506 may be necessary to match the transfer order required for this converter and the order required for the band buffer or context model.

In the case of reverse conversion, the data flow is reversed, and after level 4 reverse conversion is performed, level 3, level 2, and level 1 conversion are performed in order. The output of level 2 conversion is input to the level level 1 conversion unit 502 which is first level conversion hardware. Further, the vertical filter process is performed before the horizontal filter process. Since one of the horizontal filtering and the vertical filtering is the same except that one requires access to additional memory for the seam, it is possible to reverse the data flow with a small amount of multiplexing. Prior to the inverse conversion, it is necessary to convert the 2-byte coefficient from an embedded form with two signal bits to an ordinary two's complement number.
The elements shown in FIGS. 14 and 15 can be used similarly in the case of TT conversion.

<Conversion timing>
The conversion timing of the forward conversion unit in FIG. 15 is based on the timing of each filter unit. The first filter unit or filter unit 401 calculates the horizontal level 1 transform, and the second filter unit or filter unit 402 calculates the vertical level 1 transform. The third filter unit or filter unit 505 computes level 2-4 transforms or is idle.

  In one embodiment, the third filter unit (505) calculates horizontal conversion during even clock cycles and vertical conversion during odd clock cycles if not idle. The timing for the reverse conversion is the same (but reverse).

  In the example described below, 2 × 2 blocks in a wavelet tree are processed in raster order transposition order. Note that if 2 × 2 blocks in the wavelet tree are processed in raster order, the input / output (I / O) buffering required to support high speed page mode DRAM / EDO (extended data out) DRAM is required. Maybe less.

  FIG. 16 is a timing chart when the coefficients are output. The following timing is for each pixel. There are four components per pixel.

Start from time 0 do:
for
(x + 0; x <16/2; y ++)
for (y = 0; y <16; y ++)
Apply level 1 horizontal filter to (x, y) Start at time 1 do:
for (x = 0; y <16/2; y ++) /
for (y = 0; y <16/2; y ++)
for (xx = -1; xx <1; xx ++) / * 0 = smmoth, -1 = previous detail * /
Apply level 1 vertical filter to (2 * x + xx, y)
for (x = 0; x <8/2; x ++)
Start from time 18 + x * 32, do:
for (y = 0; y <8; y ++)
Apply level 2 horizontal filter to (x, y)
for (x = 0; x <8/2; x ++)
Start at time 21 + x * 32, do:
for (y = 0; y <8/2; y ++)
for (xx = -1; xx <1; xx ++) / * 0 = smooth, -1 = previous detail * /
Apply level 2 vertical filter to (2 * x + xx, y)
for (x = 0; x <4/2; x ++)
Start at time 66 + x * 64, do:
for (y = 0; y <4; y ++)
Apply level 3 horizontal filter to (x, y)
for (x = 0; x <4/2; x ++)
Start at time 69 + x * 64, do:
for (y = 0; y <4/2; y ++)
for (xx = -1; xx <1; xx ++) / * 0 = smooth, -1 = previous detail * /
Apply level 3 vertical filter to (2 * x + xx, y) at time 138
Apply level 4 horizontal filter to (0,0) at time 140
Apply level 4 horizontal filter to (0,1) at time 141
Apply level 4 vertical filter to (0,0) / * smooth * /
At time 143
Apply level 4 vertical filter to (-1,0) / * previous detail * /

<TT conversion>
FIGS. 17 to 24 show the results (outputs) of each one-dimensional filter process of TT conversion. The bold rectangle indicates the coefficient in one wavelet tree corresponding to the currently processed input pixel, and the shaded one indicates the coefficient from the previous wavelet tree stored. The value marked “B” is the stored intermediate result (and also the difference between adjacent samples). The TT transform is similar to the TS transform, but requires more memory.

  FIG. 25 is a block diagram of a 10 tap forward / reverse filter unit. Note that hardwired shfts and rounding offsets are not shown to avoid complexity. Note that mux 806 in FIG. 25 can also be used for mirroring at the transformation boundary. In order to perform the mirror processing once, it is necessary to perform "d" input zeroing and s (n + 2) input multiplexing of the overlapping unit.

  In FIG. 25, adders 801 and 802 are connected to receive a 2n bit input during the forward pass period of the filter unit. The adder 801 adds 2n-bit inputs and outputs a value, and this value is given to one input of the mux 803. Adder 802 subtracts one input from the other input and generates an output to one input of mux 804. The mux 803 and 804 are also connected to receive s and d inputs, respectively, for reverse mode operation of the filter unit. The output of mux 803 is an n-bit input, which is s (n + 2), and the output of mux 804 is an n + 1-bit input, which is p (n + 2) for the forward path and d (n + for the reverse path 2).

  The outputs of mux 803 and 804 are connected to the input of memory 805. Outputs 806A and 806D to 806F of the mux 806 are also connected to the input of the memory 805. The memory 805 delays the input based on the spatial arrangement and outputs it to the output. In one embodiment, the memory 805 is comprised of a register file or SRAM that has two banks and has one port per bank and is operated ping-pong. An address is input to the input of the memory 805 to control the output sent to the mux 806. In one embodiment, this address stores 16 or 28 locations per component.

  The output of memory 805 is connected to the input of mux 806 along with external buffer data received from seam buffer input 802. The output 806A of the mux 806 is obtained by delaying s (n + 1), that is, s (n + 2) of the output of the mux 803 once (single delay). The output 806B of the mux 806 is obtained by delaying s (n), that is, the output of the mux 803, twice (double delay). The output 806C of the mux 806 is obtained by delaying the output of p (n), that is, the mux 806 twice in the forward path, and by delaying the output of d (n), that is, the mux 804, in the reverse path. The output 806E of the mux 806 is obtained by delaying s (n-1), that is, the output of the mux 803 three times. Finally, the output 806F is p (n + 1) in the forward path, that is, the output of the mux 804 delayed once, and d (n + 1), that is, the output of the mux 804 is delayed once in the reverse path. Is.

  Duplicate unit 807 is connected to receive the output of mux 803 as well as the outputs 806A, 806D, 806E of mux 806. Duplicate unit 807 generates t (n) in response to the input. An example of the overlapping unit is shown in FIG.

  The output t (n) of the duplication unit 807 is connected to adders 808 and 809. The adder 808 adds t (n) with the output 806C of the mux 806 to generate the D output of the filter unit. The adder 809 subtracts the output 806C of the mux 806 from t (n). The output of the adder 809 is connected to each input of the adders 810 and 811. The adder 810 adds the output of the adder 809 and the output 806B of the mux 806 to generate one of the n-bit outputs of the filter unit when operating as an inverse filter unit. The adder 811 subtracts the output of the adder 809 from the output 806B of the mux 806, and generates the other output of the filter unit when operating as an inverse filter.

  FIG. 26 is a block diagram of one embodiment of an overlapping unit for the forward / reverse filter of FIG. In FIG. 26, this overlapping unit includes adders 901 to 906, multipliers 907 to 909, and a divider 901. The multiplier and divider may be hard wired shift.

  The overlapping unit in FIG. 26 calculates t (n) for the above-described TT conversion. In FIG. 26, adder 901 receives an s (n + 2) input, is connected to subtract it from the s (n-2) input, and generates an output connected to one input of adder 903. . Adder 902 is connected to receive s (n-1) and subtract it from s (n + 1). The output of the adder 902 is connected to the inputs of multipliers 907 and 908. Multiplier 907 doubles the input. In one embodiment, this multiplication is done by shifting the input bits one bit to the left. The output of the multiplier 907 is connected to the other input of the adder 903.

  Multiplier 908 multiplies the output of adder 902 by 16. In one embodiment, this multiplication is done by left shifting the bits output from adder 902 by 4 bits. The output of the multiplier 908 is connected to one input of the adder 905. The output of the adder 903 is connected to one input of the adder 904 and the input of the multiplier 909.

  Multiplier 909 doubles the output of adder 903. In one embodiment, this multiplication is done by shifting the bits output from adder 903 left by one bit. The output of the multiplier 909 is connected to the other input of the adder 904. The output of the adder 904 is connected to the input of the adder 906 and is added to the hard-wired input 32. The output of adder 906 is connected to the input of divider 910. Divider 910 divides the input by 64. In one embodiment, this division is done by right shifting the bits of the input by 6 bits. The output of the divider 910 is t (n) output. FIG. 26 shows the current value for each output.

  In both the reversible TS conversion and the reversible T conversion, the low-pass filter can be made so that the range of the input signal x (n) is the same as the range of the output signal s (n), as in the S conversion. That is, the smooth output does not increase. When the input signal is 8 bits deep, the smooth output is also b bits. For example, when the signal is an 8-bit image, the output of the low-pass filter is 8 bits. This is an important characteristic for pyramid systems where the smooth output is further compressed, for example by continuous low pass filtering. In conventional systems, the output signal range is larger than the input signal range, making it difficult to filter continuously. In addition, since there is no systematic error in the integer operation of conversion, all errors in the lossy system can be controlled by quantization. Furthermore, since the low-pass filter has only two taps, it becomes a non-overlapping filter. This property is important for hardware implementation.

<Embedding ordering>
In the present invention, the coefficients generated as a result of wavelet division are entropy coded. In the present invention, the coefficients are first subjected to embedding ordering, in which the coefficients are ordered in a visually significant order, and more generally some error distance (eg, distortion distance). The coefficients are ordered with respect to The error distance or distortion distance includes a peak error and a mean square error (MSE). In addition, the ordering may be performed in such a way that priority is given to the validity for database query over bit-significance space arrangement and for each direction (vertical, horizontal, diagonal, etc.).

  Data ordering is performed to cause embedded quantization of the codestream. In the present invention, two ordering methods are used. That is, one is for ordering the coefficients, and the other is for ordering the binary values in the coefficients. The ordering of the present invention produces a bitstream that is then encoded by a binary entropy coder.

<Bit Significance Expression>
Even when the original component is unsigned, most transform coefficients are signed numbers (any coefficient output from at least one detail filter is signed). In one embodiment, the embedding order used for binary values in the coefficients is bit-plane order. The coefficients are represented in bit-significance representation before being encoded. A bit-significance representation is a sign / absolute representation in which the sign bit is encoded with the first non-zero absolute bit, not the most significant bit (MSB). That is, the sign bit does not come before the entire absolute value bit, but comes after the first non-zero absolute value bit. Also, the sign bit is considered to be in the same bit plane as the most significant non-zero absolute value bit.

  The bit-significance format represents a numerical value by three sets of bits: a head bit, a tail bit, and a sign bit. A head bit is all zero bits (including the first non-zero absolute value bit) from the MSB to the first non-zero absolute value bit. The importance of the coefficient is determined by the bit plane in which the first non-zero absolute value bit appears. The tail bit consists of absolute value bits after the first non-zero absolute value bit up to LSB. The sign bit simply represents a sign, for example, 0 represents a positive sign and 1 represents a negative sign. A numerical value such as ± 2 to the nth power with MSB being a non-zero bit has only one head bit. Zero coefficients have no tail bits or sign bits. FIG. 61 shows all possible values of 4-bit coefficients from -7 to 8.

  In FIG. 61, the bit-significance representation represented in each stage includes 1 bit or 2 bits. In the case of 2 bits, the first bit is the first 1 bit followed by the sign bit.

If the value is a non-negative integer, such as occurs with respect to pixel brightness, the order that can be employed is the bit plane order (eg, the order from the most significant bit plane to the least significant bit plane). In embodiments where negative integers with two's complement are also acceptable, the sign bit padding order is the same as the first non-zero bit of the absolute value of the integer. Thus, the sign bit is not considered until one non-zero bit is encoded. For example, according to the sign / absolute value notation, the 16-bit number -7 is 1000000000000111.
It is. On a bit plane basis, the first 12 decisions are “insignificant” or zero. The first 1 bit is found in the 13th decision. The sign bit (“negative”) is then encoded. After the sign bit is encoded, the tail bit is processed. The 14th and 15th decisions are both "1".

  Since the coefficients are encoded from the most significant bit plane to the least significant bit plane, the number of data bit planes must be known accurately. In the present invention, the number of bit planes is determined by finding the upper boundary of the absolute value of the coefficient value calculated from the data or derived from the image depth and the filter coefficients. For example, when its upper boundary is 149, there are significant 8 bits or 8 bit planes. At the speed of software, bitplane coding may not be available. In another embodiment, the bit plane is only encoded when the coefficients make sense as binary numbers.

<Coefficient digit alignment>
The present invention aligns the coefficients with each other prior to bit-plane coding. This is because, like FFT and DCT, coefficients in different frequency sub-bands represent different frequencies. The present invention controls quantization by aligning the coefficients. The lighter quantized coefficients are aligned to the faster bit plane side (for example, shifted to the left). Thus, if the stream is censored, these coefficients will have more bits that define them than the more heavily quantized coefficients.

  In one embodiment, the coefficients are aligned for best rate and distortion performance in terms of SNR or MSE. There are many possible alignments, including a near-optimal alignment from a statistical error distance such as MSE. Alternatively, justification may allow physchovisual quantization of the coefficient data. The alignment has a considerable effect on the image quality (in other words on the rate / distortion curve), but has little effect on the final compression ratio of the lossless system. Other digit alignment, that is, resolution progressive alignment, may be suitable for attention area fidelity encoding, which is a special quantization.

  Alignment may be indicated in the header of the compressed data, or may be fixed for individual applications (ie the system has only one alignment). The alignment for various sized coefficients is known to both the encoder and the decoder and thus has no effect on the efficiency of the entropy coder.

  FIG. 28 shows bit depths of various coefficients in the two-level TS conversion and TT conversion division of an image of b bits / pixel. FIG. 29 is an example of a multiplier for the frequency band used for coefficient alignment in the present invention. For coefficient alignment, the size of the 1-DD coefficient is used as a reference, and a shift relative to this size is given. The n shift is to multiply 2 to the power of n.

  In one embodiment, alignment of all coefficients in the image is generated by shifting the coefficients relative to the absolute value of the largest coefficient. The coefficients after the alignment are processed in bit plane units called importance levels from the highest importance level to the lowest importance level. The code is encoded with the last head bit of each coefficient. The sign bit is at the importance level of the last head bit. The important thing is that justification controls the order in which bits are sent to the entropy coder. The padding, shifting, storing, and encoding of the extra 0 bits are not actually performed.

FIG. 62 shows an example of the number of digit alignment for coefficient digit alignment.
The alignment for coefficients of various sizes has no effect on the efficiency of the entropy coder since it is known to both the encoder and the decoder.
Note that encoding units of the same data set may have different alignments.

<Code stream ordering and context model>
FIG. 27 shows the ordering of the code streams and the ordering within the coding unit. In FIG. 27, after the header 1001, the coding unit 1002 is sequentially continued from the upper band to the lower band. (In an application designed for a single image type, header 1001 is optional.)
Each coding unit is composed of data 1003 having the highest importance, data 1004 having a lower importance, and data 1005 having the lowest importance.

  The context model determines both the order in which data is encoded and the conditioning for specific bits of data. Think about ordering first. The highest level ordering of the data is described above. The data is “less important data” (sometimes referred to as MIC) that is lossless encoded in the order of conversion, and “low importance data” (referred to as LIC) that is encoded using the embedded integrated lossless / lossy method. Sometimes).

  The order in which the coefficients are processed within each bit plane is the order from a lower resolution to a higher resolution (an order from a lower frequency to a higher frequency). The order of the coefficient sub-bands in each bit plane is from higher level (low resolution, low frequency) to lower level (high resolution, high frequency). Within each frequency subband, the encoding is done in a certain order. In one embodiment, the order is raster order, 2 × 2 block order, serpentine order, Peano scan order, and so on.

In the case of 4-level division using the code stream of FIG. 27, the order is as follows.
4-SS, 4-DS, 4-SD, 4-DD, 3-DS, 3-SD, 3-DD, 2-DS, 2-SD, 2-DD, 1-DS, 1-SD, 1- DD

  One embodiment of the horizontal context model used in the present invention will be described below. This model uses bits in a coding unit based on spatial and spectral coefficient dependencies. The binary values obtained for the neighborhood and parent coefficients can be used to generate the context. However, context has a causal relationship with decodeability, and more or less with efficient adaptation.

  The present invention provides a context model for modeling embedded bit-significance-order bitstreams generated by coefficients for binary entropy coders.

  FIG. 56 shows the neighborhood coefficients of each coefficient of the coding unit. In FIG. 56, the neighborhood coefficients are represented in an easy-to-understand geographical notation (for example, N = north, NE = northeast, etc.). Given a coefficient and the current bit plane, such as P in FIG. 56, the context model can use any information in all of the coding units prior to that bit plane. In the case of this context model, the parent coefficient of the attention coefficient is also used.

  The head bit is the most compressible data. Therefore, a large amount of context or conditioning is used to increase the compression rate. Rather than using the value of the neighborhood coefficient or the parent coefficient to determine the context of the bit of interest of the coefficient of interest, the information is grouped into two indicator bits described in connection with FIG. This information may be stored in memory or calculated dynamically from neighborhood coefficients or parent coefficients.

<Embedding from memory to disk>
One embodiment of the embedding scheme for the present invention is the fact that there is no extra space in the band buffer memory that can be used as work area memory because the entire band buffer memory is filled with data at the start of data encoding. Based on. In the present invention, the data to be embedded is stored in a memory and is data with low importance. Data with higher importance is directly encoded. The least important data consists of several least significant bits.

  In one embodiment, when a portion of each coefficient is rewritten to memory for later encoding, the head bits and tail bits are of course also known to ensure the correct encoding, as well as the head and tail bits. I have to know. In one embodiment, two or more indicator bits (eg, three bits, four bits, five bits, etc.) are used to indicate head, tail and sign bit information.

  In one embodiment, when 8 bit memory locations are used, 2 indication bits indicate head, tail and sign bit information. If two instruction bits are used, the least important six importance levels can be rewritten to the memory along with the two instruction bits. One indication bit indicates whether the most significant bit of the 6 importance level is a head bit or a tail bit. When indicating that the first indicator bit is a head bit, the second indicator bit is the sign of the coefficient. On the other hand, when the first indicator bit indicates that the most significant bit of the data rewritten to the memory is a tail bit, the second indicator bit is a free indicator bit and additional tail information, e.g. It can indicate whether the importance tail bit is the first tail bit or a later tail bit.

  FIG. 30 shows the coefficients divided into highest importance data 1301 (referred to as MIC) and low importance data 1302 (referred to as LIC). In one embodiment, the MIC consists of the upper 6 bits of each coefficient and the LIC consists of the lower 6 bits. The highest importance data 1301 is sent to the context model and immediately encoded in coefficient order. This data need not be buffered in external memory. The low importance data 1302 is written in a memory (for example, RAM), and then encoded and embedded in order of importance. In addition, two instruction bits in the data are written into the memory. The instruction bit 1303 indicates whether the most significant bit of the data written in the memory is a head bit. The instruction bit 1304 gives the sign of the coefficient or indicates whether or not the first tail bit is included in the data. The instruction bits may be stored in a form concatenated with the low importance data 1302 or may be stored in another memory or memory location associated with the memory storing the data 1302 and associated with each part of the coefficient. The bits may be stored so that they can be identified.

The example in FIG. 63 shows how to use 2-bit instruction bits. The columns in FIG. 63 are arranged to match the data format in FIG. The sign bit is indicated by “S”, the tail bit is indicated by “T”, the arbitrary bit is indicated by “X”, and the value of the tail-on bit is “h” or “t”. It is represented. In FIG. 63, the instruction bits are h = 0 and t = 1. This convention may be used as is in other embodiments. In one embodiment, the 0 sign bit in FIG. 63 indicates a positive sign and the 1 sign bit indicates a negative sign. The reverse assignment may be used. Note that the sign bit is always stored with the first “on” bit, so it can be encoded and embedded at the same time.
In FIG. 63, “T” indicates the corresponding bit of the coefficient and is 0 or 1.

  In one embodiment, during the decryption period, the most important data is written to memory as it is decrypted, and at the same time it is suitable to initialize the memory to store the less important data. Two instruction bits are written to the memory. (Depending on the alignment of the coefficients, part of the highest importance data may also be stored in the next byte.) This initialization allows the low importance data to be decoded one bit plane at a time. It is only necessary to read and write 1 byte per coefficient (less than 1 byte in some embodiments). When a coefficient is read and input to the inverse transform, the coefficient is converted to a normal numerical format (eg, 2's complement format).

In addition to “highest importance data” and “low importance data”, some data may be discarded or quantized during encoding. The coefficient is divided by the quantization scale factor (2 to the power of Q-1). (Coefficient quantization is described in the JPEG standard)
In the invention, since the division is performed by truncating the bit plane, the quantization is a power of two. For example, since Q = 1 means division by 1, the coefficient does not change, but since Q = 2 means division by 2, one bit plane is truncated. These divisions may be performed by a shift, for example, a shift of one bit position when Q = 2. FIG. 31 and FIG. 32 show the formats of the highest importance data and the low importance data in consideration of both quantization and coefficient alignment for various partial bands.

  FIG. 31 shows a lossless case where data is not truncated. Following the JPEG convention, this is called quantization A = 1. This is because the actual coefficient is divided by 1 (no loss). The highest importance data is not shaded. Low importance data is shaded.

  FIG. 32 shows a case where data for one bit plane is truncated. Since truncating one bit plane is equivalent to dividing by 2, Q = 2. The discarded bit plane is represented in black.

  Note that in addition to those shown in FIGS. 31 and 32, the highest importance data includes SS coefficients. The coefficient bits are shown as 8-bit data, but when using the reversible color space, 8-bit data is required, and the size of the color difference coefficient increases by 1 bit.

  In the present invention, the code bit context model includes the encoding of the code following the last head bit. There are three contexts related to the sign, depending on whether the N coefficient is positive or negative and whether the sign is not yet encoded. Alternatively, only one context may be used for the code, i.e. the code may always be encoded as 50%.

<Coding order of wavelet coefficients>
An example of the encoding order of wavelet coefficients is summarized in the following pseudo code:
Encode highest importance data Encode the location of the low importance bitplane with the first data
for each low importance data bitplane do
Encode one low-importance data bitplane

When the highest importance data is encoded, the first bit plane of low importance data that is not only zero head bits is determined for each coefficient. This allows the encoder and decoder to look-ahead over the entire bit plane of low importance data. This is useful in the case of encoding in black and white data units in which all information is in K coefficients and CMY coefficients are all 0. Not encoding the bit planes individually improves the compression rate, especially when R2 (7) is the longest run-length code that can be used. (See US Pat. Nos. 5,381,145 and 5,583,500 for “R2” code)
However, when four parallel coding cores process components simultaneously, the processing speed is determined by the component with the most bit planes to be encoded. The cores assigned to other components are idle during the period of the uncoded bitplane.

  FIG. 33 is a flowchart for explaining an example of the pseudo code processing. In FIG. 33, the context model first encodes the highest importance part (MIC) (processing block 1401). After encoding the MIC, processing logic encodes the location of the bit plane with the first data in the low importance part (LIC) (processing block 1402). This is for the entire coding unit. When there are 6 bit planes in the LIC, none of the bit planes contain data, or 1 or 2, 3, 4, 5 or 6 bit planes contain data. Next, processing logic sets the LIC bit plane having this first data in the current LIC bit plane variable (processing block 1403).

  It is determined whether all LIC bitplanes with data have been encoded (processing block 1404). If all of them have been encoded, the process ends. Otherwise, one LIC bit plane is encoded (processing block 1405), and the next LIC bit plane is set in the current LIC bit plane variable (processing block). 1406). Processing then loops back to processing block 1404.

<Encoding order of highest importance data>
An example of the encoding order of the highest importance data is as follows:
for each tree do
Encode SS coefficient
Perform MIC look-ahead (or perform tree look-ahead)
for each non-SS coefficient
for each bit (plain) with data do
Encode head bit or tail bit
if coefficient is not 0
Encode the sign bit

  For the highest importance data, one wavelet tree is processed at a time. Again, the highest importance data is not embedded. The MIC look-ahead determines a bit plane whose head bits are not all zero for all non-SS coefficients in the wavelet tree. In one embodiment, a 4-bit number is sufficient to identify the first bit plane to be encoded separately. In another embodiment shown in FIG. 34, one bit is used to indicate that all the coefficients 1501 (shaded area) except the second division SS are 0, and the other bit is Used to indicate that all coefficients 1503 other than the first division SS are zero. These 2 bits are used in addition to the 4 bits used to identify the first bit plane.

  In other embodiments, a tree look-ahead may be used in which SS coefficients are encoded and then the first bit plane with non-zero head bits for the entire tree is encoded.

  Actual encoding / decoding of SS coefficient bits (9 bits if reversible color space is used) to compensate for context repeat delay when conditioning is used for encoding SS coefficients and the first bitplane And look ahead values may be alternated. When conditioning is not used, no alternation is required.

As described above, the context model of the present invention uses look-ahead. An example of this look-ahead may be used for highest importance data or highest importance part (MIC). In one embodiment, as shown in FIG. 34, 6 bits for each tree, ie, 4 bits for the maximum bit plane, 1 bit for the level 0 all-zero bit plane, and 1 bit for the level 1 all-zero bit plane. One bit is used for this purpose. When the maximum bitplane is zero, the extra 2 bits are redundant but not a big deal. Another method uses adaptive coding decisions to determine “(isolated) zero / non-zero”. Non-zero coefficients are further described by:
-M-term operation to determine coefficient value and sign (total: 2 cycles / coefficient)
An adaptive coding decision is used to determine “± 1 / non ± 1”.
Another cycle is used to determine the sign when the absolute value is 1 and to determine the sign and value when the absolute value exceeds 1. (Total: 3 cycles / coefficient)
Similarly, “± 1 / non ± 1”, “± 2,3 / non ± 2,3”, etc. can be performed at a total of 4 cycles / coefficient.
・ The following procedure:
if MIC bitplane is not all zero then
Adaptive coding of decision "-1,0,1" or "other"
if “-1,0,1” then
Adaptive encoding of “0” or “-1, + 1”
if “-1, + 1” then
Describe the sign bit
else
Adaptive encoding of “-3, -2,2,3” or “others”
if “-3, -2,2,3” then
"-2,2" or "-3,3" is described with 1 bit
Describe the sign bit
else
Describe the value with the maximum number of bits determined for the tree
Describe the sign bit

  The 1-bit or multiple-bit “description” may be adaptive coding, coding with 50% probability, or simply copying to a coded data stream.

  When all or most of the bit planes are to be encoded individually, there are bit planes that are not used for alignment at some level of conversion, and this unused bit plane is not encoded. There are many options for handling bit-to-context delay for head bits and tail bits. One method is a method of alternately processing three coefficients, DD, SD, and DS. The sign bit of the non-zero coefficient can be encoded at the end of the coefficient. This is because the highest importance data is always lossless and does not need to follow immediately after the first “on” bit.

  An example of a flowchart describing the pseudo code for encoding the highest importance part (MIC) is shown in FIG. In FIG. 35, processing begins when the processing logic sets the first tree as the current tree (processing block 1601). Next, processing logic encodes the SS coefficient (processing block 1602). After encoding the SS coefficient, processing logic encodes the position of the first bit plane with data in the MIC of the tree (processing block 1603), i.e., performs MIC look-ahead.

  Next, processing logic determines whether the MIC of the entire tree is zero (processing block 1604). If the MIC for the entire tree is zero, processing logic proceeds to processing block 1614, otherwise, processing logic moves to processing block 1605 where processing logic makes the first non-SS coefficient of the tree the current coefficient.

  After setting the current coefficient to the first non-SS coefficient of the tree, processing logic makes the first bit plane with data the current bit plane (processing block 1606). Next, processing logic encodes one bit of the current coefficient of the current bit plane (processing block 1607). Thereafter, processing logic determines whether all of the bitplanes have been encoded (processing block 1608). If the entire bit plane has not been encoded, the processing block sets the next bit plane as the current bit plane (processing block 1609) and proceeds to processing block 1607. When all of the bit planes have been encoded, processing logic determines whether the current coefficient is zero (processing block 1610). If the current coefficient is not zero, processing logic encodes the sign bit (processing block 1611) and proceeds to processing block 1613. If the current coefficient is zero, processing logic proceeds to processing block 1613.

  At processing block 1613, processing logic determines whether all the coefficients in the tree have been encoded. If all the coefficients of the tree have not been encoded, processing logic sets the next coefficient in the tree as the current coefficient (processing block 1612) and proceeds to processing block 1606. When all the coefficients in the tree have been encoded, processing logic determines whether all trees have been encoded (processing block 1614). If all trees have been encoded, processing ends; otherwise, processing continues at processing block 1615 where processing logic sets the next tree as the current tree and transitions to processing block 1602.

  FIG. 36 is a block diagram of one embodiment of a formatting unit and context model used during the encoding pass period of the highest importance data. In FIG. 36, a barrel shifter 1701 is connected to receive the absolute value of the coefficients and the quantization level used during encoding to ensure lossless compression so that the highest importance data does not exceed the minimum disk bandwidth. . In this way, the barrel shifter 1701 is controlled at the quantization level. In one embodiment, barrel shifter 1701 supports 1, 2, 4 or 8 quantization by shifting absolute value bits by 0, 1, 2 or 3 bits. In other embodiments, lower or higher quantization numbers are supported, for example only 2 quantization is supported.

  The output of the barrel shifter 1701 is composed of lower 6 bit planes, ie, low importance data, and the remaining high order bit planes, ie, highest importance data. In other embodiments, a simple separation mechanism is used to produce these two outputs.

  Both outputs of the barrel shifter 1701 are input to the head bit plane unit 1702, and a bit plane including data is discriminated. The first bit plane unit 1702 is used to find a bit plane having the first “on” bit for all coding units used during low importance data processing. (See FIG. 27). Another leading bit plane unit 1706 is also connected to receive the highest importance data output from the barrel shifter 1701. The head bit plane unit 1706 is used for each tree during high importance data processing. An example of the first bit plane unit will be described later with reference to FIG.

  Barrel shifter 1701 is also connected to comparison units 1703 and 1704. These comparison units 1703 and 1704 perform two comparisons on the highest importance data and generate 2-bit instruction information for the low importance data. The comparison unit 1703 indicates whether the tail bit has already appeared (ie, whether the encoding is already in the tail) by determining whether the highest importance data is zero. The output of the comparison unit 1703 is a tail on bit. The comparison unit 1704 determines whether or not the highest importance data is 1. If the highest importance level data is 1, the output is 0 from FIG. The output of the comparison unit 1704 is connected to one input of a multiplexer (MUX) 1705. The other input of mux 1705 is connected to receive the sign bit. The select input of mux 1705 is controlled by the output of comparison unit 1703, and the output of mux 1705 is a “leading tail” bit 1304 when indicating that the output of comparison unit 1703 is a tail bit. However, when the output of the comparison unit 1703 indicates that it is a head bit, the mux 1705 is controlled to output a code.

  In one embodiment, the comparison units 1703 and 1704 are comprised of simple bit comparators.

  A memory 1707 is connected to receive the sign bit, the highest importance data output from the barrel shifter 1701, and the output of the first bit plane unit 1706. Memory 1707 is used to delay the coefficients so that parent and neighbor information can be used for conditioning. The configuration of the memory 1707 will be described later.

  The context model (CM) 1710-1712 provides conditioning for codes, heads, tails, and other bits. Each of these context models will be described later.

  FIG. 37 shows an embodiment of the head bit plane unit. In FIG. 37, the leading bit plane unit 1800 has an OR gate 1801 connected to receive the coefficients and feedback from the output of the register 1802. The output of the OR gate 1801 is connected to the input of the register 1802. The register 1802 is controlled by a tree start / coding unit reset instruction. The output of the register 1802 is connected to the priority encoder 1803. The output of the priority encoder 1803 is the output of the first bit plane unit 1800.

  Initially, register 1802 is cleared. Each bit of the register 1802 is ORed with each bit of the input coefficient by an OR gate 1801. For each bit of coefficient 0, the value in register 1802 remains the current value and is output to priority encoder 1803. For each bit of coefficient 1 (for example, the leading 1), the output to the register 1802 of the OR gate 1801 is 1, and this is input to the priority encoder 1803. The priority encoder 1803 finds the leading 1 and this is the first bit plane with a coefficient of 1.

<Processing order of low importance data>
Each bit plane of the lowest importance data is processed as follows:
for each tree do
for each coefficient do
if Start of look-ahead interval
Perform look ahead
if lookahead is not active
Encode head bit or tail bit
if First “on” bit
Encode the sign bit

  One embodiment of the LIC bit-plane encoding process is shown in the flowchart of FIG. The LIC bitplane encoding process begins with processing logic making the first tree the current tree (processing block 1901). Next, processing logic sets the first non-SS coefficient of the current tree as the current coefficient (processing block 1902). After making the first non-SS coefficient of the current tree the current coefficient, the processing block determines whether the encoding is for the start point of the look-ahead interval (processing block 1903). When the encoding process is at the start of the look-ahead interval, processing logic performs a look-ahead (processing block 1904) and proceeds to processing block 1905. If the encoding process is not at the start of the look-ahead interval, processing logic immediately moves to processing block 1905 to determine if the look-ahead is active.

  If the look ahead is active, processing logic proceeds to processing block 1909 to determine if all the coefficients of the current tree have been encoded. Processing continues from processing block 1913 when all coefficients of the current tree are encoded, otherwise processing logic sets the next coefficient of the current tree after the look-ahead interval as the current coefficient (processing block 1910). Proceed to process block 1903.

  When lookahead is not active, processing logic encodes the head or tail bits (processing block 1906) and then determines whether the first non-zero bit has been received (processing block 1907). If the first non-zero bit has not been received, processing logic proceeds to processing block 1911. If the first non-zero bit has been received, processing proceeds to processing block 1908 where processing logic encodes the sign bit and processing proceeds to processing block 1911.

  At processing block 1911, processing logic determines whether all the coefficients of the current tree have been encoded. If all the coefficients of the current tree have not been encoded, processing logic sets the next coefficient of the current tree as the current coefficient (processing block 1912) and moves to processing block 1903. If all the coefficients of the current tree have been encoded, processing proceeds to processing block 1913 where processing logic determines whether all trees have been encoded. If all trees have not been encoded, processing logic sets the next tree as the current tree (processing block 1914) and continues processing from processing block 1902. If the entire tree has been encoded, the process ends.

  It is not important to process one wavelet tree at a time, but it may be convenient because the transformation causes the data to be read and written in that order. When data is processed for each wavelet tree, the bit-context delay can be adjusted by alternation between DD, SD, and DS coefficients (alternation between subtrees). Alternatively, one partial band may be encoded at a time. Whatever order is chosen, head / tail bits that are not used to align different sub-bands are not encoded and therefore do not require wasted cycles.

  FIG. 39 is a block diagram of one embodiment of a look ahead and context model for low importance data. In one embodiment, the highest importance data and the lower importance data utilize the same context model (CM) that provides conditioning for the sign, head and tail bits.

  In FIG. 39, context models 2001-2003 are connected to receive input data. The code context model 2001 is connected to receive tail on bits, code / head tail bit indication and data. The head bit context model 2002 is connected to receive tail on bits and data. The tail bit context model 2003 is connected to receive tail on bits, code / head tail bit indication and data. Each context model 2001-2003 generates one context according to the input.

  The context generated by the context models 2001-2003 is connected to the input of mux 2004. The mux 2004 is controlled by the previous bit and the bit-significance representation itself. The head bit context model 2002 is used until 1 bit comes to the data input. The code context model 2001 is used when the last bit is the first one bit of the head. Thereafter, the tail bit context model 2003 is used.

  The output of mux 2004 is connected to a “= head?” unit 2005 and a first in / first out (FIFO) buffer 2006. The “= head?” Unit 2005 checks whether the current context is a head bit context with a neighbor and a parent zero head bit. When all the contexts are in the head, the FIFO buffer 2006 is cleared by a signal output from the “= head?” Unit 2005.

  Contexts and results are buffered in FIFO 2006 or other memory during the look-ahead period. At the end of the look-ahead interval, look-ahead decisions and / or individual decisions are encoded if necessary. If the coefficients are processed one wavelet tree at a time, the look-ahead FIFO may use a single FIFO for all subbands, or multiple, one for each subband. A FIFO may be used.

  If it is desired to reduce multiplexing, look-ahead may also be used for the highest importance data. However, it is somewhat redundant to use both the look-ahead and the head bit plane for each tree.

  When a core allocated to a certain component encodes a code bit, a core allocated to another component that does not encode a code bit of the same bit plane is in an idle state. Thus, up to 4 clock cycles may be used for a sign bit when each core encodes a sign bit of another bit plane. In one embodiment, there are up to 6 bits of head or tail bits per coefficient.

  One possible problem with timing is that the highest importance portion is fully compressed, which results in the disk being idle during the decoding period of that portion of data. If the band buffer has sufficient memory bandwidth, look-ahead may be used to speed up the processing of the most important data. In that case, low importance data may be the beginning of the head. It would also be ok for the disk to have a burst transfer rate higher than the maximum sustained rate. A hard disk usually has a significant buffer, and prefetching into this buffer will eliminate idle time.

<Partial conditioning of the context model>
The conditioning used in the context model is determined by a trade-off between hardware cost and compression ratio. Therefore, in the following part, many options for conditioning are presented for the designer's consideration.

<Context model for SS coefficient>
In one embodiment of the context model, SS coefficients are not encoded. Since SS coefficients are at most 256 times less than the original data, there is little benefit in encoding them. If it is desired to encode the SS coefficient, it may be handled by Gray encoding, conditioning by the previous bit in the same coefficient, and / or conditioning by the corresponding bit in the previous coefficient.

<Context model for the first bit plane>
The 4 bits of the head bit plane information for the highest importance data of each wavelet tree can be processed in the same way as the SS coefficient. This only increases the size of the original data by a factor of 521. In one embodiment, they may not be encoded because they are smaller than the original data, and may be conditioned in some way with Gray encoding.
Similarly, when 6 bits are used according to FIG. 34, these 6 bits may be processed in the same way as SS coefficients.

<Context model for head bits>
FIG. 40 is a block diagram of one embodiment of a context model that provides conditioning for head bits. In FIG. 40, the context model 2100 includes a shift register similar to that found in the bit-plane context model. The key difference is that it does not use the leading coefficient of the current bitplane, the conditioning is based on tail-on information, and uses already encoded information in all the leading bitplanes and the current bitplane. . Also, use several bits to identify the encoded bit plane or group of bit planes, the importance level and the rate of the encoded partial band or partial band group generated by partial band bucketing.

  In FIG. 40, the context model 2100 has two inputs: a current importance level 2110 and a coefficient 2111 from memory. The current importance level 2110 is connected to the inputs of the tail-on information / bit generator block 2101 and the importance level / subband bucketing block 2102. The coefficient 2111 from the memory is connected to the block 2101 and the registers 2103 to 2106.

  Block 2101 receives the coefficient and determines the presence or absence of a 1 bit. In one embodiment, block 2101 also identifies the location of one bit. The output of block 2101 is 1 bit or 2 bits based on tail-on information. In one embodiment, the tail information indicates whether or not the first non-zero absolute value bit is observed (whether or not the leading “on bit” is observed), and if so, what Indicates whether it is before the bit plane. FIG. 64 is an explanation of tail information bits.

  From this 2-bit tail information, a 1-bit “tail-on” value indicating whether the tail information is zero or not is synthesized. In one embodiment, tail information and tail on bits are updated immediately after the coefficients are encoded. In another embodiment, the update is done later to allow parallel context generation.

  In addition, these 2 bits are used to indicate the importance level to be encoded. The first 2-bit plane uses the value 0, the second 2-bit plane uses the value 1, the third 2-bit plane uses the value 2, and the remaining bit-planes use the value 3. In addition, there is an all-zero headbit run-length encoding.

  The 10 bits of the context for the head bit are 2 bits from the parent and W (west) coefficients respectively, 1 from each coefficient of N (north), E (east), SW (southwest), and S (south). It consists of bit information and 2-bit importance level information.

  In one embodiment, tail information is not used for some or all frequency bands. By doing so, the frequency band can be decoded without decoding its parent in advance.

  In another embodiment, a single alignment is used to assign bit planes of each frequency band to importance levels. Another alignment is used to determine the tail-on information of the parent, but this uses fewer parent bitplanes than those actually encoded. This allows several bit planes in a frequency band to be decoded without decoding the parent corresponding bit plane of the same importance level (see FIG. 57). For example, an image can be encoded by pyramid alignment regardless of parent tail-on information based on MSE alignment (see FIG. 58). This allows the decoder to decode with pyramid alignment and simulate MSE alignment or any alignment between pyramid alignment and MSE alignment.

  In FIG. 40, the output of the block 2101 is connected to the inputs of the registers 2103 to 2106. Registers 2103 to 2106 accumulate neighboring data. For example, the up / left shift register 2103 holds the bit immediately above the current coefficient throughout the line. The current shift register 2104 holds the coefficient bit of the current line, and the lower / right shift register 2105 holds the line from the line immediately below the shift register. Finally, the parent register 2106 holds parent data. The output of the shift register constitutes a context.

The output of the importance level / subband bucketing block 2102 is also used as a context. This is the case where multiple partial bands and different levels become part of the context when trying to encode with the same context. In such a case, the output of block 2102 is combined with the outputs of registers 2103 to 2106 to form a context. Otherwise, the context consists only of the outputs of registers 2103-2106.
One bit is output from the context model 2100.

  In consideration of a bit-context delay for using data of the current bit plane, encoding may be performed by alternation between DD, SD, and DS coefficients (alternation between subtrees).

Note that a memory is required to store the coefficients necessary for conditioning (see FIG. 36). The memory usage for one embodiment of a context model conditioned by all neighborhood coefficients and parents is shown in FIG. A short seam conversion order is assumed. (External memory can also be used to support long seam transformation order, which would require both additional memory and bandwidth.)
Conditioning by high level parents is very costly. The level 4 DD coefficients for a tree are not calculated until 16 trees later than most of the level 1 DD coefficients for that tree. In addition, storing the entire coefficient to be encoded later (coefficient not shaded in FIG. 41) only stores tail-on information (shaded in FIG. 41) for later use in conditioning. It is very expensive compared to memorizing. Conditioning based only on “west” information in the same tree and the parent generated without data from the “west” tree would significantly reduce the amount of memory required. If parent or west information is not available, it is useful to copy information obtained from north or east.

<Sign bit context model>
The context model that gives the conditioning for the sign bit is simple. When the sign of the upper pixel is known, it is used for conditioning. When the sign of the upper pixel is not known, the bit is not encoded. ((R2 (0)) is used, or uncoded (R2 (0)) can be used for all code bits).

  FIG. 42 is a block diagram of one embodiment of a context model for code bits. In FIG. 42, mux 2301 receives north sign bit 2303 and 0 bit 2304 and is controlled by north tail on bit 2302 to set north sign bit 2303 when north tail on bit 2302 is 1. Outputs 0 otherwise. Thus, the north pixel provides a north tail on bit 2302 and a north code bit 2303 to provide context for the south pixel.

<Tailbit context model>
Conditioning is not used for tail bits. In one embodiment, a fixed probability state is used and no probability update is used. FIG. 65 shows three choices of codes used for tail bits. The second option using R2 (1) and R2 (0) is a good choice.

In one embodiment, a golden ratio code suitable for the probability of M≈60% and L≈40% is as follows:
Input codeword MMM 00
MML 110
ML 01
LM 10
SS 111

<Summary of context bin (bin)>
The minimum number of context bins available in the system is as follows: The SS, the leading bit plane of each tree, the sign and the tail bit are all unencoded (R2 (0) code is used). There is no need to store the PEM state or the highest probability symbol (MPS) bit, but there must be logic to select the R2 (0) code. Therefore, depending on how this is considered, the hardware cost is 0 or 1 context bin. For head bits, adaptive coding should be used. For low importance data, the bit plane conditioning is not important because one bit plane is encoded at a time. For the highest importance data, the leading bit planes of each wavelet tree will reduce the number of bit planes sufficiently, so conditioning with those bit planes is not important. The effectiveness of subband conditioning is not very clear, but it can also be ignored in this minimal example context. Three neighborhood coefficients and one parent coefficient tail bit may be used for a total of 4 bits (16 context bins). One additional context bin may be used for look ahead. (It may be more convenient to map the two context bins together to make room for the look ahead so that the memory size is still a power of 2.)

  When using 4 cores (contexts need to be copied 4 times) and using 2 context memory banks per core, how to consider a "non-encoded" context, and 2 head context bins Depending on whether they are mapped together, the minimum number of contexts to use will be between 128 and 144.

A well-conditioned system is as follows:
• For SS (9 bits) and leading bit plane (4 bits), use 4 context bins per bit, for a total of 52 context bins. (These context bins may be divided into banks and need not be copied.)
-The tail bits are not encoded and both R2 (0) and R2 (1) are used. Depending on how this is considered, the cost is 0, 1 or 2 context bins.
Two adaptive contexts and one “no code” context are used for the sign bit.
Head bits may use 8 bits for neighborhood / parent coefficients and 2 bits (1024 context bins) for partial band / bitplane information.
One context is used for look ahead.

  Another example of a context model is US patent application Ser. No. 08 / 498,695 (June 30, 1995, “Method and Apparatus For”, with an example of a sign / absolute unit that converts input coefficients to sign / absolute value format. Compression Using Reversible Wavelet Transforms and an Embedded Codestream ”), US Patent Application No. 08 / 498,036 (accepted June 30, 1995,“ Reversible Wavelet Transform and Embedded Codestream Manipulation ”), US Patent Application No. 08 / 642,518 (1996) Accepted May 3, 1996, “Compression and Decompression with Wavelet Style amd Binary Style Including Quantization by Device-Dependent Parser”, and US Patent Application No. 08 / 643,268 (accepted May 3, 1996, “Compression / Deconpression Using Reversible Embedded Wavelets ”).

  This context model provides a context for entropy coding of data. In one embodiment, all entropy encoding performed by the present invention is performed by a binary entropy coder. A single coder may be used to generate a single output code stream. Alternatively, multiple (real or virtual) coders may be used to generate multiple (real or virtual) data streams.

<M-term coding for LIC>
FIG. 43 illustrates the use of M-term coding for LIC. When M-term coding is used for compression coding, it acts as a look-ahead (as shown). First, the state of the next 8 coefficients is examined. If there is something in the head, entropy coding is performed on the head bits, and all head bits are entropy coded one bit per cycle until all head bits of 8 coefficients are coded. In FIG. 43, 1 head bit is encoded in the first cycle and the third cycle, while 0 head bit is encoded in the second cycle and the fourth cycle. When all head bits are entropy encoded, the code bits and tail bits are encoded in the same cycle. For example, in FIG. 43, the code bits and tail bits that come after one head bit are all encoded in the fifth cycle. In this way, the total number of cycles is reduced.

<Application of the present invention to a printing system>
FIG. 44 is a block diagram of an embodiment of a printer front end. In FIG. 44, a renderer 2501 receives data in the form of a page description language or a display list. The renderer 2501 may be configured by raster image processing. For each location (eg spot), the renderer 2501 determines its color (depending on the application, eg black and white, 8-bit RGB value, 8-bit CMYK value). The output of the renderer 2501 is a set of pixels formatted in a band and is stored in a band buffer (memory) 2503.

  In another embodiment, data in a page description language (PDL) such as Adobe Postscript (registered trademark) or Microsoft Windows (registered trademark) GDI is converted into a display list. This display list is used to generate a band of pixels. In this embodiment, the pixels are assumed to represent continuous tone values, and any halftoning or dithering required by the print engine will occur after decompression.

  In the present invention, the memory used as the band buffer 2503 is also used as a compression work area (does not increase the necessary memory). This double use will be described later in detail.

  The compressor 2504 compresses each pixel band. The compressor 2504 will also operate when the input to the compressor 2504 is halftoned or dithered pixels, but the compression rate will not be good when using wavelet processing. A binary context model can be used for halftone or dithered pixels. The compressor 2504 writes the compressed data to the disk 2505. The disk 2505 may be a hard disk. In another embodiment, disk 2505 may be random access memory (RAM), flash memory, optical disk, magnetic tape, any type of storage means, and any type of communication channel.

  FIG. 45 is a block diagram of an embodiment of a printer back end. In FIG. 45, the printer back end 2500 includes a decompressor 2602, a band buffer (memory) 2603, and a print engine 2604 connected to a disk 2505. The decompressor 2602 reads compressed data from the hard disk 2505 and decompresses it. The decompressed data is stored in a band buffer (memory) 2603 in a pixel format. Band buffer 2603 may be the same memory as band buffer 2503 that serves as a work area for compressor 2504. In order to be able to send pixels to the print engine 2604 in real time, the decompressor 2602 keeps the band buffer 2603 fully filled with data.

  FIG. 46 is another embodiment including optional image quality improvement. In FIG. 46, the pixels output from the decompressor 2602 are input to the band buffer 2603 via the image quality improvement block 2705, but other information, that is, information (partial coefficients) that has not yet become a pixel, is input to the band buffer 2603. Sent directly. The image quality improvement block 2705 performs functions such as interpolation processing, smoothing processing, error diffusion processing, halftone processing, and / or dither processing.

  Because of the bandwidth required between the decompressor 2602 and the band buffer 2603, the decompressor 2602 first writes the transform coefficient into the band buffer 2603, accesses the band buffer 2603, takes in the specific coefficient, and performs the inverse transform. Then, the data is written again in the band buffer 2603. The band buffer 2603 may be small as a work area. For example, even if an image of one page is 64 Mbytes and the band buffer 2603 is 16 Mbytes, it can be considered as a small work area memory.

  For example, the data rate from the band buffer 2603 to the print engine 2604 required for 8 pages / minute for a 32-bit / pixel (8-bit 4-component CMYK), 400 dpi A4 image is about 8 Mbyte / Seconds. The transfer rate of a typical hard disk is approximately 2 Mbyte / second (for example, 1.7 to 3.5 Mbyte / second). Therefore, in order to match the bandwidth of the disk 2505 with the bandwidth of the printer, a compression ratio of about 4: 1 is required. According to one embodiment, the compressor 2504 of FIG. 44 and the expander 2602 of FIG. 45 or 46 are housed in a single integrated circuit chip.

  FIG. 47 is a block diagram of an embodiment of an integrated circuit (IC) chip incorporating a printer compression / decompression function. In FIG. 47, the pixel data interface 2801 is connected to a band buffer (not shown). The pixel data interface 2801 generates an address for reading and writing to the band buffer. An optional reversible color space converter 2802 may be included to perform reversible color space conversion. The coefficient data interface 2804 generates addresses for reading and writing, and correctly assembles 2-byte coefficients. Coefficient data interface 2804, along with pixel data interface 2801, controls line buffering or coefficient buffering that needs to be in external memory. The use of the coefficient data interface 2804 and reversible color space conversion will be described in more detail later.

  Note that the bidirectional arrows mean that data can flow in either direction. For example, when compressing data, the data flows from left to right through various components of the IC chip. On the other hand, at the time of data decompression, the data generally flows from right to left.

  When encoding data, the wavelet transform block 2803 receives the pixels from the pixel data interface 2801 or from the reversible color space converter 2802 (when included) and performs wavelet transform. In one embodiment, the transform performed by wavelet transform block 2803 is a duplicate wavelet transform. This does energy concentration for lossless and lossy image compression. In the case of lossy compression, block boundary distortion, which is a problem with JPEG, is avoided. The filter coefficients are normalized so that a good lossy compression result is obtained with scalar quantization when properly aligned. In one embodiment, wavelet transform block 2803 performs a 2 · 6 transform. In another embodiment, wavelet transform block 2803 performs 2 · 10 transform. The wavelet transform block 2803 may perform other known transforms. Various specific examples of the wavelet transform block 2803 will be described in detail later.

  The coefficients output from the wavelet transform block 2803 may be rewritten to the memory (eg, band buffer) via the coefficient data interface 2804 for later encoding. In one embodiment, the data rewritten to the memory is low importance data, details of which will be described later. Such data is then read into the IC chip and encoded.

  The coefficient output from the wavelet transform block 2803 or the coefficient received via the coefficient data interface 2804 is supplied to the context model 2805. The context model 2805 provides a context for encoding (or decoding) data by the encoder / decoder 2806. In one embodiment, the context model 2805 can pass data directly to encoding. In this way, the context model 2805 operates as the highest importance context model. The implementation configuration of various context models has been described above.

  In one embodiment, encoder / decoder 2806 comprises a high speed parallel coder. A high speed parallel coder processes several bits in parallel. In one embodiment, the high speed parallel coder is implemented with VLSI hardware or a multiprocessor computer without sacrificing compression performance. An example of a high-speed parallel coder that can be used in the present invention is described in US Pat. No. 5,381,145 “Method and Apparatus for Parallel Decoding and Encoding of Data” issued January 10, 1995.

  In other embodiments, the binary entropy coder comprises a Q coder, a QM coder, a finite state machine coder, and the like. The QM coder is a well-known efficient binary entropy coder. A finite state machine (FSM) coder provides a simple conversion from probability and outcome to a compressed bitstream. In one embodiment, the finite state machine coder is implemented as both an encoder and a decoder with table lookup. A variety of probability prediction methods can be used with such a finite state machine coder. In one embodiment, the finite state machine coder of the present invention comprises a B coder described in US Pat. No. 5,272,478 “Method and Apparatus for Entropy Coding” issued Dec. 21, 1993.

  The output of the encoder / decoder 2806 is connected to an encoded data interface 2807 that provides an interface to a disk or other storage medium or other communication path.

  The encoded data interface 2807 transmits / receives encoded data to / from the disk. In one embodiment, if a SCSI interface controller is included on the chip, it will be implemented at this point. In other embodiments, the encoded data interface 2807 communicates with an external SCSI controller. Storage or communication other than SCSI may be used.

During the decompression period, the encoded data is received from the disk (or other memory device or channel) via the encoded data interface 2807 to the encoder / decoder 2806, where the context provided by the context model 2805 is used. Is expanded. The coefficient obtained by the decompression is inversely transformed by the wavelet transform block 2803. (Note that in one embodiment, the wavelet transform block 2803 performs both forward transform and inverse transform, but in other embodiments, the forward transform and inverse transform are performed by separate blocks.)
The output of the wavelet transform block 2803 is composed of pixels, which are subjected to an optional color space transform and output to the band buffer via the pixel data interface 2801.

  The basic timing of the system during the printing period is shown in FIG. In FIG. 48, the encoded data of each encoding unit is read from the disk. As much data as possible is read and the coefficients are decoded with a short delay. When decoding is complete, the inverse wavelet transform is calculated. After this conversion is complete, the pixels can be sent to the print engine. Note that the shading in FIG. 48 clarifies when various actions occur for a specific coding unit.

<Embedding coefficients from memory to disk>
FIG. 27 shows the structure of encoded data in the present invention. In FIG. 27, the highest importance data 1003 is encoded (not embedded) in the order of coefficients immediately after conversion. Therefore, this data need not be buffered. In one embodiment, the amount of highest importance data 1003 is limited so that it can always be read from the disk.

  A certain amount of low importance data 1004 is buffered, embedded and written to disk in order of importance. The amount of data that can be buffered, embedded and written depends on the transfer time. That is, the system reads data until the transfer time from the disk ends. How much data is stored depends on the disc transfer rate. These rates are known and depend on the physical characteristics of the individual transfers.

  For images that are difficult to compress, some data is discarded during the encoding period. This data is shown as minimum importance data 1005. If there is no possibility of reading out the least important data even at the best disk transfer rate, there is no reason to store the data on the disk. For many, perhaps most images, the data will not be discarded.

  The ordering of the encoded data and the method for achieving it have already been described in detail.

  Hereinafter, management of the band buffer at the time of compression and expansion will be described, and then an embedded scheme of encoded data will be described. A hardware implementation of the conversion, context model, and encoder / decoder parallelization is also described.

<Pixel and coefficient interface>
FIG. 49 shows an example of a method for organizing pixel data. In FIG. 49, the page 3000 is divided into bands 3001 to 3004. In one embodiment, page 3000 consists of a page description in a page description language or display list and is used to generate pixels for each band. In one embodiment, each of bands 3001-3004 is rasterized separately by display list techniques. Each band 3001 to 3004 is further divided into coding units (for example, 3001A to 3001D).

  The advantage of using a large number of coding units per band is that a part of the band buffer can be used cyclically (similar to a ping-pong buffer) as a work area during the decompression period. In other words, a certain part of the pixels is decompressed and stored in the band buffer and sent to the printer, while at the same time another part of the band buffer is used as a work area for storing the coefficients while decoding the coefficients. Another part of the band buffer can be used to store the pixels corresponding to.

  FIG. 50 shows the band buffer 3101 of the page 3100. The band buffer 3101 includes encoding units 3101A to 3101D. Coding units 3101A and 3101B serve as a work area for the decompressor to store the coefficients. Coding unit 3101C stores the pixels to send to the printer, while coding unit 3101D serves as a work area for the decompressor to store the next pixel.

  When the entire page 3100 is printed, a portion of the band buffer 3101 can be used cyclically. For example, as the next encoding unit, the pixel of the encoding unit 3101D is a pixel output to the printer. When this happens, coding units 3101B and 3101C are used as a work area for the decompressor to store the coefficients. At that time, the encoding unit 3101A is used as a work area for storing the pixels that the decompressor outputs next to the printer.

  In the present invention, the coefficient is larger than the pixel. Therefore, twice as much memory is allocated to the work area memory. In another embodiment, the band is divided into many or fewer coding units. For example, in one embodiment, each band is divided into 8 coding units.

<Memory bandwidth>
The pixel data interface and the coefficient data interface cooperate to efficiently manage the band buffer memory. When high-speed page mode DRAM, EDO (extended data out) DRAM, or other memory suitable for continuous access is used, these interfaces transfer data at consecutive addresses as a sufficiently long burst, and the available bandwidth of the memory. Make effective use of width. Several small buffers may be needed to support burst access to consecutive addresses.

  FIG. 51 is a decoding timing diagram illustrating parallel memory access requests. In FIG. 51, the bandwidth required for decoding is as follows. In one embodiment, a 2 MHz pixel clock, an 8 MHz component clock, a 32 MHz decoder clock are used, the print engine requests a 1 byte / component clock, and the converter reads 2 bytes per coefficient, per component. Remember to write a byte. If the conversion is performed in half of the encoding unit time, the conversion requires 6 bytes / component clock. Conversion speed is limited by memory bandwidth, not computation time. If a bandwidth of 24 bytes / component clock is available, the transform could be calculated within 1 / 8th of the encoding unit time. When external memory is used for seams, the conversion may require extra bandwidth. In one embodiment, the highest importance portion of the encoded data requires 2 bytes to be written per component clock to decode the coefficients. For each bit plane of the low-importance portion of the encoded data, it is necessary to read and write one byte per component clock for decoding. Note that this may be smaller in some embodiments. If both operations use half of the encoding unit time, then 4 bytes / component clock bandwidth and 24 bytes / component clock bandwidth would be required respectively. If external memory is used for context seam information, extra bandwidth may be required.

  In one embodiment, the highest burst mode transfer rate is 4 memory accesses / component clock (1 access / decoder clock). Therefore, when a 32-bit data bus is used, the maximum transfer rate is slightly smaller than 16 bytes / component clock. If a 64-bit data bus is used, the maximum transfer rate is slightly smaller than 32 bytes / component clock.

<Reducing the required bandwidth of LIC memory>
Each bit of each coefficient of the LIC requires one read and one write to the external memory during decoding. (Encoding requires only one read.) These memory accesses are responsible for most of the memory bandwidth required. In one embodiment, instead of storing each LIC coefficient in units of 8 bits, the required bandwidth is reduced by storing the coefficients using less than 8 bits when possible.

  FIG. 66 shows the amount of memory required to store LIC coefficients for decoding each bit plane. In FIG. 66, 1 bit (tail on bit) is written per coefficient during the MIC processing. What is written for bit plane 5 is read again for bit plane 4: a tail on bit, what bit 5 was, and 2-3 bits including the sign bit when bit 5 was 1 is there. The ratio indicates how many coefficients are involved for each bit plane. This will be clearer from FIG. In FIG. 31, since all the coefficients from the partial bands DD1 to DS4 and SD4 have data in the bit plane 5 (as shaded), the bit plane 5 has coefficients from all the partial bands. Bitplane 0 has only coefficients from the DD1 partial band. As shown in FIG. 66, since both bit plane 4 and bit plane 5 have coefficients from the entire partial band, the ratio is 100%, but bit plane 0 has only a coefficient of 25% (DD1 portion). To the band). As decoding proceeds, some bitplanes are completed before bitplane 0 is reached.

  In FIG. 66, since decoding of the bit plane is not performed at the start of decoding, only one bit of all the coefficients (one bit per coefficient) is read to check whether it is a head or tail. As decoding progresses, the number of bits per coefficient increases.

  FIG. 52 illustrates a method of using cyclic addressing to process writing of data larger than the read data. This occurs because the processing results in more bits to be written than initially read bits. In FIG. 52, the process begins by writing one bit per coefficient (which is one eighth of the memory space). Subsequently, 1 bit is read per coefficient, and 2 to 3 bits are written simultaneously per coefficient. Next, 2-3 bits per coefficient are read, and 3-4 bits per coefficient are written simultaneously. This continues until all the data has been processed.

  There are several options for simplifying the hardware configuration. Rather than always using the minimum number of bits, possibly only 1, 2, 3, 4, 6 or 8 bits may be used. This may waste 1 bit depending on the size. Even when it is not known in the LIC that the sign bit is not encoded, the space for the sign bit will always be available.

  An option that would further reduce the memory bandwidth may be to not store the tail on bit when not needed. For example, when writing bit plane 0, there are 6 bits, head bits or tail bits. If any of these bits are non-zero, the tail on should be true and there is no need to store the tail on value and the sign bit can be stored as the seventh bit.

  The memory bandwidth for the highest importance part (MIC) may also be reduced by a variable length storage method. Instead of always using 8 bits per coefficient, using only the minimum number of bits will result in savings. If a 6-bit look-ahead value (as shown in FIG. 34) is stored instead of the zero coefficient, the memory will be used more efficiently.

<Reversible color space>
The present invention makes it possible to optionally perform a reversible color space conversion that makes the conversion between two color spaces completely reversible and can be performed with integer arithmetic. That is, by reversely transforming the converted color space data, all existing data can be obtained regardless of whether rounding or truncation is performed in the forward conversion process. Reversible color space conversion is described in commonly assigned US patent application Ser. No. 08 / 436,662 (accepted May 8, 1995, “Method and Apparatus for Reversible Color Conversion”).

  Color space conversion takes advantage of the opponent color space without compromising the ability to provide lossless results. In the lossless case, the opposite color space weakens the correlation and improves compression. In the case of lossy, the opposite color space quantizes the luminance information lighter than the color difference information, resulting in improved visual quality. When a reversible color space is used in the conversion of the present invention, embedding of luminance and color difference coefficients is superior to subsampling as lossy compression, and lossless compression is possible.

  When the reversible color space is used, it is desirable to align the coefficients so that the most significant bit of the 8-bit luminance component and the 9-bit color difference component have the same digit alignment. For lossy compression, this alignment allows the color difference data to be quantized twice as much as the luminance data, and allows lossless compression of the luminance and lossy (but very high quality) compression of the color difference. Both of these results take advantage of the characteristics of the Huffman visual system.

<Other pixel operations>
Printers often process most or all non-continuous tone documents. For example, a text image consisting only of black and white (only values 0 and 255) would be common. In one embodiment, a band histogram is created. For example, an image with only 0,255 black / white (K component) can be re-mapped to 0,1 image. Similar compaction can be applied to spot color images. Note that compression must be lossless when compaction is used. However, the lossless compression achieved is substantially improved when compaction takes place.

  Further, instead of using the overlapping wavelet transform described in this specification, a binary image and a spot color image can be processed by a context model such as JBIG based on a lossless bit plane.

  In other embodiments, the system may be designed to have a binary mode. FIG. 54 shows an example of a binary context model similar to the JBIG context model template. In FIG. 54, shift registers 3501-3503 provide a plurality of bits according to the JBIG standard. The shift registers 3501 and 3502 receive two lines above and one line above the line buffer 3500. This “top” line provides bits corresponding to the northwest (NW), north (N) and northeast (NE) position pixels of the template, as shown in FIG. The outputs of the shift registers 3501 and 3502 are directly given to the context model (memory) 3505. The output of the shift register 3503 is fed to an optional mux 3504 that can implement a JBIG standard adaptive template. The context memory (model) 3505 is connected to a probability prediction machine 3506, and the probability prediction machine 3506 is connected to a bit generator 3507. The context memory (model) 3505, the probability prediction machine 3506, and the bit generator 3507 operate in a mutually known manner.

  The output of mux 3504, together with the outputs of shift registers 3501 and 3502 and feedback from bit generator 3507, forms a context bin address that is used for the addressing of context memory 3505. In one embodiment, the context memory 3505, in one embodiment, contains 1,024 context models consisting of 6 bits for describing each probability state. This requires a 1,024 × 6 bit context memory.

  Since bit generator 3507 provides the decoded bits from the current line as part of the context address, there is a large “bit-context” delay that includes access time to the context memory.

  FIG. 55 cooperates with the same addressing block 3601 that receives the outputs of shift registers 3501 and 3502 and the output of multiplexer 3504, and another implementation that uses decoded bits from the current line to access the probability prediction machine. An example is shown. The PEM 3506 receives the previous bit and uses it to select one appropriate context from the context pair used. The selected context is updated and both contexts are rewritten to memory. The same address indication block 3601 detects an address that has already been read and data is in the probability prediction machine. The same address indication block 3801 also generates a signal to use data already in the PEM (which may be updated data) instead of stale information in the memory.

  In one embodiment, the decoder includes 1024 context bins of 6 bits that describe each probability state. This requires a 512 × 12 bit context memory. The outputs of the shift registers 3501 and 3502 along with the output of the multiplexer 3504 give a partial context bin address where only the previous bit is not used. As a result, a pair of context bins is selected from the context memory 3505. Two or more bits of the context bin can be excluded from the partial context. There are 2 n probability states in each memory location. Here, n is the number of excluded bits.

  Note that the “bit-context” delay is reduced. A context memory access may occur before the previous bit is decoded. Before the previous bit is decoded, processing of the PEM state for both paired states may begin in parallel. High speed operation can be achieved.

<Encoder rate control>
Encoder rate control requires not only the ability to quantize the data, but also measure the rate so that decisions about quantization can be made. When the rate indicates poor compression (ie not at the desired level), the quantization may be increased. On the other hand, when the rate indicates that the compression is too high, the quantization may be reduced. The rate control decision must be made in exactly the same way at the encoder and decoder.

  One way to ensure that the encoder and decoder make the same decision is to use signaling. The encoder measures the rate at a predetermined interval and stores the quantization Q in memory for use at the next interval. The decoder simply retrieves the quantization for each interval from memory. This requires extra memory. For example, a 256-location x 2-bit on-chip SRAM (for displaying +2, + 1,0, -1 changes in Q or storing 1, 2, 3, 4 Q) can generate 4096 line images. On the other hand, it would be sufficient to change the quantization Q every 16 lines.

  There are various options for rate measurement. FIG. 53 shows an encoder / decoder pair. The encoder / decoder pair shown in FIG. 53 includes a context model (CM), a probability prediction model / machine (PEM), and a bit generator (BG), and a run count reordering unit, interleaved. ) Consists of word rearrangement unit and shifter. See US Pat. Nos. 5,381,145 and 5,583,500, assigned to the assignee of the present application and incorporated herein by reference.

  Even if the decoder cannot measure the rate at the same location, the rate measurement must be clear. For example, the rate measurement is provided to the decoder as part of the compressed code stream.

  Another rate measurement option, shown as the smaller circle (position 2 in FIG. 53), is to count the beginning of the interleaved word at the encoder. This is done after the bit generation stage (position 4 in FIG. 53) in other embodiments. Since the encoder and decoder work on the codeword at the same time, rate implicit signaling may be used. This counting may be performed by computational hardware that measures the average codeword length consisting of a register and an adder that adds the codeword length. The hardware for performing this count and average bit count measurement is well known in the art and is shown as block 3401 in FIG. It will be apparent that this block may be used to make similar measurements at other locations in the system (eg, positions 1, 2, 3, 4 of both encoder and decoder). .

  Other options are to count the complete codeword size after the bit generator and before the interleaved word reordering unit (position 3 in FIG. 53), or measure the amount of data actually written to disk. It would be the method (position 1 in FIG. 53).

Rate measurement may be implicit. That is, the same rate measurement calculation is performed in both the encoder and the decoder. For example, the encoder and decoder may accumulate the average codeword size each time a new codeword starts. This is shown at position 4 in FIG. (The actual size cannot be used because the encoder does not know the size until the end of the codeword.)
When the size of the R code used for the core changes from R2 (0) to R2 (7), the average codeword size changes from 1 bit to 4.5 bits. If the probability prediction is successful, using the average will be very accurate. Otherwise, the predicted value is still valid because the difference between the minimum codeword length and the maximum codeword length is generally not very large relative to the average. The average size of the Rz (k) codeword is k / 2 + 1 bits.

  The goal is that in almost all cases, the highest importance data will be sufficiently compressed and no quantization (Q = 1) will be required. Only “pathological” images require quantization (Q> 1). However, by including a quantization function, it can be ensured that the system does not move for pathological images.

  Another benefit of encoder rate control is that encoding of low importance data can be stopped when maximum bandwidth is reached. This increases the encoding speed and reduces the total time for data output (eg, reduces the total printing time).

  It is important to store the result of the quantization change (Q value). For example, if the quantization changes, the determination of the maximum coefficient in a group of coefficients must be consistent. Also, to reconstruct the quantized coefficients (when bit planes are discarded), the number of discarded bit planes needs to be considered for best results.

<High-speed parallel coding and context model>
The entropy encoding unit of the present invention consists of two parts. One is a high-speed coding core that operates in parallel, and performs probability prediction and bit generation. The other is a context model, which provides the context used for encoding.

  The number of cores required to achieve the required speed depends on the application.

  Another part of the entropy coding system is the context model for the coefficients of the present invention. There are many possible trade-offs when building a context model. The present invention, in one embodiment, provides a context model that enables parallelism to support the use of the high speed parallel coder of the present invention at low hardware costs. Examples of this context model have been described above.

  Although only context models for wavelet coefficients have been described, the present invention is not limited to context models that support only wavelet coefficients. For example, if a bit plane coding mode is required for binary or spot color images, US patent application Ser. No. 08 / 642,518 (accepted May 3, 1996, “Compression and Decompression with Wavelet Style and Binary Style Including” Quantization by Device-Dependent Parser ”) and US patent application No. 08 / 643,268 (May 3, 1996,“ Compression / Decompression Using Reversible Embedded Wavelets ”) You can do it.

<Parallelization>
In one embodiment, four fast codes to encode / decode 8 bits per coefficient when the number of bits of the coefficient is from 8 bits to 12 bits (up to 13 bits when lossless color space is used). Cores are used. In one embodiment, each of the four components is assigned to one core, simplifying parallelization and data flow. Each coefficient can be used for up to 16 cycles for bit encoding / decoding (including decisions for look-ahead etc.).

  The present invention synchronizes the component cores even when some cores are idle due to the success of their look-ahead and other cores are processing the sign bit after the leading “on” bit. Maintain state. The total time for operating the context model varies depending on the data, and particularly varies greatly depending on the effectiveness of the look-ahead, and also varies depending on the position of the leading “on” bit.

  Many variations and modifications of this invention will become apparent to those skilled in the art after reading the foregoing description, so that the specific embodiments shown and described for purposes of illustration are in no way intended to be limiting. It should be understood as not.

It is a figure which shows a context dependency. It is a figure which shows the same order as a raster order. It is a figure which shows an example of a short seam order. It is a figure which shows the other example of a short seam order. It is a figure which shows the level 1 horizontal processing result of TS conversion. It is a figure which shows the level 1 vertical processing result of TS conversion. It is a figure which shows the level 2 horizontal processing result of TS conversion. It is a figure which shows the level 2 vertical processing result of TS conversion. It is a figure which shows the level 3 horizontal processing result of TS conversion. It is a figure which shows the level 3 vertical processing result of TS conversion. It is a figure which shows the level 4 horizontal processing result of TS conversion. It is a figure which shows the level 4 vertical processing result of TS conversion. It is a figure which is a block diagram of an example of a forward / reverse filter. It is a block diagram of an example of the first level forward conversion according to the present invention. It is a block diagram which shows an example of perfect forward conversion by this invention. It is a timing chart at the time of coefficient output. It is a figure which shows the level 1 horizontal processing result of TT conversion. It is a figure which shows the level 1 vertical processing result of TT conversion. It is a figure which shows the level 2 horizontal processing result of TT conversion. It is a figure which shows the level 2 vertical processing result of TT conversion. It is a figure which shows the level 3 horizontal processing result of TT conversion. It is a figure which shows the level 3 vertical processing result of TT conversion. It is a figure which shows the level 4 horizontal processing result of TT conversion. It is a figure which shows the level 4 vertical processing result of TT conversion. It is a block diagram of a 10 tap forward / reverse filter unit. It is a block diagram of an example of the duplication unit in the forward / reverse filter shown in FIG. It is a figure which shows the ordering of the code stream, and the ordering within a coding unit. It is a figure which shows the bit depth of various coefficients at the time of decomposing | disassembling the input image of b bit / pixel by 2 level TS conversion and TT conversion. It is a figure which shows an example of the multiplier with respect to the frequency band used for coefficient digit alignment in this invention. It is a figure which shows the coefficient divided | segmented into the highest importance data and the low importance data. It is a figure which shows the lossless case where data is not thrown away. It is a figure which shows the case where 1 bit plane of data was thrown away. It is a flowchart which shows an example of operation | movement of a compression / decompression system. It is a figure which shows the example in which 6 bits are used with respect to one tree. It is a flowchart which shows the encoding of the highest importance part. It is a block diagram of an example of the formatting unit and context model used for the encoding pass of highest importance data. It is a block diagram which shows an example of a head bit plane unit. It is a flowchart which shows an example of the encoding process of a LIC bit plane. FIG. 4 is a block diagram illustrating an example of a look ahead model and a context model for low importance data. It is a block diagram which shows an example of the context model for head bits. FIG. 6 illustrates a memory usage for an example context model with all neighbors and parental conditioning. It is a block diagram which shows an example of the context model for a code bit. It is a figure which shows an example of the parallel encoding with respect to LIC. FIG. 2 is a block diagram illustrating an example of a front end of a printer. 2 is a block diagram illustrating an example of a back end of a printer. FIG. FIG. 10 is a block diagram illustrating another example of a back end of a printer. FIG. 3 is a block diagram illustrating an example of an integrated circuit chip including a compression / decompression function of a printer. It is a figure which shows the basic timing at the time of printing. It is a figure which shows an example of the organization method of pixel data. It is a figure which shows a band buffer. FIG. 66 is a decoding timing diagram that reveals a simultaneous memory access request. It is a figure which shows the utilization method of the cyclic addressing for processing writing of data larger than the read data. It is a block diagram which shows the pair of an encoder and a decoder. It is a block diagram which shows an example of a binary context model. It is a flock figure which shows the other example of a binary context model. It is a figure which shows the neighborhood coefficient of each coefficient of an encoding unit. It is a figure which shows the pyramid digit alignment based on MSE digit alignment. It is a figure which shows the MSE digit alignment of a wavelet coefficient. It is a table | surface figure which shows the span of the wavelet tree in the case of various level numbers. FIG. 5 is a table showing the number of “ss” coefficients that need to be stored. It is a table | surface figure which shows the bit significance expression of a 4-bit coefficient. It is a table | surface figure which shows the number of alignment for coefficient alignment. It is a table | surface figure explaining the instruction | indication bit of highest importance data and low importance data. It is a table | surface figure which shows the definition of tail information. It is a table | surface figure which shows the probability state used for a tail bit. It is a table | surface figure which shows the memory amount required for the memory | storage of the LIC coefficient for decoding of each bit plane.

Explanation of symbols

401, 402 Filter unit 403-406 Register 407, 408 Multiplexer (mux)
501 Input buffer 502 Level 1 conversion unit 503 "ss" delay memory 504 Level 2 to 4 order unit 505 Filter unit 506 Output buffer 801, 802 Adder 803, 804 Multiplexer (mux)
805 Memory 806 Multiplexer (mux)
807 Duplicate unit 807 to 811 Adder 901 to 906 Adder 907 to 909 Multiplier 910 Divider 1001 Header 1002 Coding unit 1003 Highest importance data 1004 Low importance data 1005 Lowest importance data 1301 Highest importance data 1302 Low importance Degree data 1303, 1304 Instruction bit 1701 Barrel shifter 1702, 1706 First bit plane unit 1703, 1704 Comparison unit 1705 Multiplexer (mux)
1707 Memory 1710 Code context model 1711 Head context model 1712 Tail context model 1801 OR gate 1802 Register 1803 Priority encoder 2001 Code context model 2002 Head context model 2003 Tail context model 2004 Multiplexer (mux)
2005 “= head?” Unit 2006 FIFO buffer 2101 tail-on information / bit generator block 2102 importance level / subband bucketing block 2103 upper / left shift register 2104 current shift register 2105 lower / right shift register 2106 parent register 2301 multiplexer (mux) )
2501 Renderer 2503 Band buffer 2504 Compressor 2505 Disc 2602 Decompressor 2603 Band buffer 2604 Print engine 2705 Image quality improvement block 2801 Pixel data interface 2802 Reversible color space converter 2803 Wavelet transform block 2804 Coefficient data interface 2805 Context model 2806 Encoder / Decode Encoder 2807 Encoded data interface 3500 Line buffer 3501-3503 Shift register 3504 Multiplexer (mux)
3505 Context Memory 3506 Probability Prediction Machine (PEM)
3507 Bit generator 3601 Same address indication block 4001, 4002 Adder 4003, 4004 Multiplexer (mux)
4005 Register file 4007 to 4011 Adder 4015 Multiplexer (mux)

Claims (21)

buffer,
A wavelet transform unit having an input connected to the buffer for performing a reversible wavelet transform on the pixels stored in the buffer to generate coefficients at the output;
A coder connected to the wavelet transform unit for encoding the wavelet transform pixel bitplane output from the wavelet transform unit and the wavelet transform pixel bitplane received from the buffer and stored in the buffer. A compression system comprising:   2. A compression system according to claim 1, wherein the buffer comprises a band buffer for storing at least one pixel band. The coder comprises a context model and a parallel entropy coder;
The highest importance data is not embedded and is encoded in coefficient order without buffering, and a portion of the lower importance data is buffered, embedded and written to memory in order of importance. The compression system described.   2. A compression system according to claim 1, wherein said coder comprises a high-speed parallel coder.   The compression system according to claim 1, wherein the coder comprises a QM coder.   The compression system of claim 1, wherein the coder comprises a finite state machine coder.   The compression system of claim 1, further comprising an encoded data interface. Wavelet transforming the pixel information received from the buffer, wherein the wavelet transform pixel is represented by at least one bit plane of the coefficients;
A method of compression comprising: replacing a pixel in the buffer with an encoded bit plane; and encoding the bit plane in the buffer. Band buffer,
A plurality of coding units for the coefficients to be subjected to inverse transform processing;
At least one encoding unit for storing pixels to be output to the output device;
At least one coding unit for pixels corresponding to already inverse transformed coefficients output to the output device, and a decompressor connected to the band buffer, wherein the decompressor The conversion coefficient is expanded, the conversion coefficient is written into the band buffer, the conversion coefficient is read out again from the band buffer, a reverse conversion is performed on the conversion coefficient to generate a pixel value, and the pixel value is written into the band buffer. Stretching system to do. Dividing the coefficient into highest importance data and lower importance data,
Sending the highest importance data immediately to the context model for encoding in coefficient order;
Storing low importance data and a plurality of instruction bits in a memory;
And a method of encoding comprising encoding the highest importance data of all coefficients in the set of coefficients, then encoding the low importance data and embedding the plurality of instruction bits in a partially based order. An input buffer having inputs connected to receive input data and first and second outputs for transferring even and odd samples;
A first level conversion unit connected to receive the even and odd samples and generate coefficients, the horizontal low-pass coefficient and the vertical high-pass coefficient being the output of the forward converter;
A memory having a first input connected to receive ss coefficients generated by the first level conversion unit and a second input connected to receive ss coefficients from a higher level conversion filtering process ,
An ordering unit having a first input connected to the memory for ordering SS coefficients for higher filtering and a first unit connected to the ordering unit for applying a plurality of transform levels; A filter unit, which performs a higher level transformation on the ss coefficients received from the ordering unit, the ss coefficients generated by the filter unit being the second input of the memory and the ordering A forward conversion device fed back to the second input of the unit.   An apparatus for compressing an image comprising a compressor and a work area memory connected to the compressor, the work area memory being the same size as the image, wherein the compressor is in the image A compression apparatus, characterized in that the work area is used to encode an image using a coefficient larger than a pixel. A method of encoding information consisting of highest importance data and lower importance data,
Encoding the highest importance data;
Encoding the position of the first bit plane in the low importance data for each coefficient not composed of only zero head bits, and each bit plane of the low importance data not composed of only zero head bits. An encoding method comprising the step of encoding. A device that encodes information consisting of highest importance data and lower importance data,
Means for encoding the highest importance data,
Means for encoding the position of the first bit plane in the low importance data for each coefficient not consisting of only zero head bits, and each bit plane for the low importance data not consisting of zero head bits. An encoding device comprising means for encoding. A method of m-term coding of information,
Examining a predetermined number of coefficients,
Entropy encoding all head bits one by one every cycle until all head bits in the predetermined number of coefficients are encoded, and the sign bits and tail bits of the predetermined number of coefficients in the same cycle An encoding method comprising the step of encoding. A pixel data interface for transferring pixel data between the integrated circuit chip and the memory;
A reversible wavelet transform unit connected to the pixel data interface for transferring information to and from the memory via the pixel data interface;
A context model connected to the reversible wavelet transform unit for providing a context for encoding the data given by the reversible wavelet transform unit;
An integrated circuit comprising an encoder for encoding a coefficient generated by the reversible wavelet transform unit based on a context provided by the context model. A decoding device for decoding encoded data,
At least one bit generator connected to receive the encoded data and to decode the encoded data based on the probability prediction value and generate decoded bits from the current line;
A probability prediction machine connected to the at least one bit generator to provide a probability prediction value based on bits decoded from the current line; and a plurality of contexts based on a partial context address to the probability prediction machine. A decoding comprising: a context model connected to the probability prediction machine for providing the probability prediction machine selecting a context from the plurality of contexts based on the decoded bits; apparatus. Receives low and high importance data and is connected to determine which bitplane contains data and has the first on bit of the entire coding unit used when processing the low importance data The first bitplane unit that generates the bitplane indication,
A comparison mechanism connected to receive low and high importance data and generate instruction information for the low importance data;
A memory connected to receive the sign bit, the highest importance data, and an indication of the first bitplane with data, delaying the coefficients to provide conditioning information,
A first context model connected to the memory to provide a context for the sign bit;
A second context model connected to the memory and the highest importance data to provide a context for the head bit;
A context model device comprising a third context model connected to the memory and the highest importance data to provide a context for a tail bit.   A method for performing compression, comprising: measuring an average length of a codeword to confirm a coding rate; and adjusting a compression rate based on a required compression amount. Context model,
A probability prediction machine connected to the context model;
A bit generator connected to the probability prediction machine, and an encoding rate control means connected to the bit generator for controlling an encoding rate by measuring an average codeword length; system. A method for processing the least important part of a data bit plane of a set of transform coefficients,
Reading a first amount of data from the memory, and simultaneously with reading the first amount of data, to compensate for the less transformed bit planes in the lower bit planes of the set of transform coefficients A transform coefficient processing method comprising a step of writing a second amount of data larger than the amount of data into the memory.

Images