AU2005203385A1 - Managing storage of digital image data - Google Patents

Description

S&F Ref: 729890

AUSTRALIA

PATENTS ACT 1990 COMPLETE SPECIFICATION FOR A STANDARD PATENT Name and Address of Applicant: Actual Inventor(s): Address for Service: Invention Title: Canon Kabushiki Kaisha, of 30-2, Shimomaruko 3-chome, Ohta-ku, Tokyo, 146, Japan James Philip Andrew Spruson Ferguson St Martins Tower Level 31 Market Street Sydney NSW 2000 (CCN 3710000177) Managing storage of digital image data Associated Provisional Application Details: [33] Country:

AU

[31] Appl'n No(s): 2004904538 [32] Application Date: 11 Aug 2004 The following statement is a full description of this invention, including the best method of performing it known to me/us:- 5845c -1- ;MANAGING STORAGE OF DIGITAL IMAGE DATA COPYRIGHT NOTICE This patent specification contains material that is subject to copyright protection.

00oO The copyright owner has no objection to the reproduction of this patent specification or c related materials from associated patent office files for the purposes of review, but otherwise reserves all copyright whatsoever.

FIELD OF INVENTION The present invention relates generally to digital image compression and more particularly to such compression in a raster-image-processing system.

BACKGROUND

Image compression is widely used in color raster-image-processing systems, such as those found in printers. Use of compression reduces the amount of memory required to store raster images in the processing stages between pixel generation and the final output produced by a printer. Reducing the amount of memory lowers the total system cost.

In printing environments, the total uncompressed size of a generated digital image comprising pixels is often large, and avoiding storage of the entire digital image is advantageous. Such pixel images are typically generated in raster or band order. In particular, a raster image processor (RIP) processing an object graphic description of a page to be printed emits pixels, scanlines, groups of scanlines, or tiles in a stream. In some simple cases, this raster image data stream may be fed directly to a print engine. In many circumstances this is impossible, however, because the print engine requires realtime data delivery; the image must be kept in case error recovery is required, or the image must be kept for printing later in a duplexing system. Many other reasons exist.

In systems where the raster image data must be kept, the raster image data is often fed to a compressor in a pipeline fashion, such that intermediate storage between the RIP and the compressor is small compared with the total uncompressed image size. The output of the compressor is also considerably smaller than the uncompressed image size.

729890 Fig. 12A illustrates such a compression system 1200. A RIP, scanner or other source produces a source digital image 1210 that is input to a compressor 1220. The compressed output of the compressor 1220 is fed to an output buffer 1230, which for illustrative purposes is empty initially (indicated by white fill in Fig. 12A). Fig. 12B shows the same 00 5 system 1200, but in this drawing the output buffer 1230 is more than half full with compressed data (indicated by diagonal hatching, but portion of white fill indicates output buffer is not full).

Many compression schemes can be used in this environment. However, existing schemes suffer, because the available memory to receive the compression data may be over-run. Fig. 12C shows the compression system 1200 with an over-run condition in the output buffer 1230 (indicated by diagonal hatching). Further, the quality of the decompressed image may not be uniform across different image content, such as text and natural images. For example, in some cases, text material is degraded, while natural image data is compressed to a level that does not have a significant visual effect on the final output. One widely used method for color images is the baseline method of the Joint Photographic Experts Group (JPEG) standard. JPEG involves applying a discrete cosine transform (DCT) to a source image or portions of the image, quantising the coefficients, and entropy coding the quantised coefficients. The coded data can then be fed to an output buffer. However, JPEG suffers from both of the above two disadvantages.

Schemes also exist that attempt to alleviate the memory over-run problem. One simple scheme is to allocate a large buffer, but this is typically a wasteful and costly solution. Other schemes limit the total compressed image size by maintaining a sufficiently high compression factor on local regions of the image, but in these schemes trading off one part of the image that compresses well against another that does not is difficult. Other compression schemes achieve a better use of a limited memory buffer, but do not offer a uniform quality in a decompressed image containing different types of material.

The mixed raster content (MRC) imaging model is a scheme that offers a more uniform quality in a decompressed image across different types of material. For example, crisp text can be placed over a background image in a manner that facilitates good 729890 compression. However, the MRC method suffers the memory over-run problem, unless a large buffer is provided.

O These problems may also occur in many other related raster image processing systems, such as digital cameras and scanners. Thus, a need clearly exists for an improved o00 5 digital image compression technique for use in raster image processing that ameliorates or a overcomes one or more of the disadvantages of existing systems.

OSUMMARY

In accordance with an aspect of the invention, there is provided a method of storing digital image data. The method comprises the steps of: encoding in one or more scans digital image data in a band of the digital image at a first resolution; encoding in one or more scans digital image data in the same or another band of the digital image at a second resolution, the second resolution being lower than the first resolution; and managing a storage such that if the storage is full or substantially full, at least part of an encoded scan that is least perceptually significant relative to other encoded scans in the storage is discarded.

The digital image data may contain edge data in the band of the digital image at the first resolution. More particularly, the digital image data may contain significant edge data in the band of the digital image at the first resolution. The edge data may be significant if a gradient in at least one direction satisfies a predetermined condition. The digital image data may contain other data in the same or another band of the digital image at the second resolution. The other data may comprise non-edge data. The other data may comprise insignificant edge data. The edge data may be insignificant ifa gradient in at least one direction fails to satisfy a predetermined condition.

The managing step may comprise storing at least one encoded scan in the storage.

Further, the managing step may comprise overwriting in the storage at least part of the encoded scan that is least perceptually significant with at least part of the same or another encoded scan that is perceptually more significant. Further, the managing step may comprise discarding at least part of the encoded scan that is least perceptually significant without storing the discarded part of the encoded scan in the storage.

729890 The digital image data stored in scans in the storage can be decoded into a plurality of pixels. The storage may have a fixed memory size.

At least two of the encoded scans may represent edge data in a band of the digital image data. At least one encoded scan may contain data from encoded edge data and 00 5 encoded other data. The encoded scans may comprise data from only one of encoded edge and encoded other data. Encoding one or more scans containing other data may comprise N applying transform coding to at least a portion of the one or more scans. This may comprise coding according to the JPEG method. Encoding one or more scans containing edge data may comprise applying transform coding to at least a portion of the one or more scans. The transform coding may comprise a Discrete Cosine Transform (DCT). The encoding steps may comprise compressing the digital image data.

The digital image data containing edge data in a band of the digital image and digital image data containing other data in a band of the digital image may be encoded using the same encoding technique. The digital image data may be encoded in tiles. The encoding may comprise storing pointer information in a scan to the start of the encoded digital image. Encoded digital image data may be an encoded form of multiple image layers. Encoded digital image data may be in a mixed-raster content format.

In accordance with further aspects of the invention, an apparatus and a computer program product are provided, which each implement the method of the foregoing aspect.

In still another aspect of the invention, there is provided an apparatus for managing storage of digital image data. The apparatus comprises a memory of limited size, at least one encoder, and a scan output manager. The encoder(s) are able to encode in one or more scans the digital image data comprising significant edge data in a band of the digital image at a first resolution and in one or more scans the digital image data comprising other data in the same or another band of the digital image at a second resolution. The second resolution is lower than the first resolution. The scan output manager is coupled to the memory and controls storing of encoded scans such that if the memory is full or substantially full, at least part of an encoded scan that is least perceptually significant relative to other encoded scans in the memory is discarded.

The memory may have a smaller size than the size of the digital image data.

729890 b BRIEF DESCRIPTION OF THE DRAWINGS Embodiments of the invention are described hereinafter by way of example with 00 5 reference to the drawings, in which: ¢€3 Cc Fig. 1 is a flowchart illustrating an encoding method in accordance with an embodiment of the invention; SFig. 2 is a flowchart illustrating in greater detail step 120 of Fig. 1 regarding processing of a portion of a digital image; Figs. 3A and 3B are flowcharts illustrating a method of managing storage of encoded image data into an output buffer of fixed memory size; Fig. 4 is a block diagram of a computer with a raster image processor in accordance with an embodiment of the invention; SFig. 5 is a flowchart illustrating in greater detail part of step 250 of Fig. 2 for processing a high-resolution block; Fig. 6 is a flowchart illustrating in greater detail part of step 250 of Fig. 2 for determining a number of representative colors of a block; Fig. 7 is a flowchart illustrating in greater detail step 240 of Fig. 2 for processing a low-resolution block; Fig. 8 is a flowchart illustrating in greater detail step 220 of Fig. 2 regarding determining if a block has significant edges; Fig. 9 is a high-level flow diagram of a decoding process; Fig. 10 is a more detailed flow diagram of the decoding step 910 of Fig. 9; Fig 11 is a flow diagram illustrating the process of decoding a high-resolution block; Figs. 12A to 12C are block diagrams of a compression system including an output buffer that is empty, more than half filled, and in an over-run condition, respectively; Figs. 13A and 13B are block diagrams illustrating the processing of portions of an image; 729890 b -6- Fig. 14 is a block diagram of a general purpose computer system with which embodiments of the invention may be practiced; O Figs. 15A to 15C are diagrams depicting an example of an 8x8 data block, a corresponding quantized DCT block, and the corresponding decompressed block, 00 5 respectively; ¢€3 c ~Fig. 16A is a block diagram of a memory comprising a header and reserved memory used as a buffer for storingdata; Fig. 16B is a block diagram of transformed coefficients generated from input Sdigital image data; Fig. 16C lists scans and the DC coefficients for transformed coefficients of Fig.

16B; Fig. 16D is a block diagram illustrating the header of Fig. 16A in detail and shows the relationship to scan 0 blocks; and Fig 17 is a flow diagram illustrating the process of decoding a low-resolution block.

DETAILED DESCRIPTION Methods, apparatuses, and computer program products are disclosed for managing storage of digital image data. This has particular application where the storage is a buffer or memory of limited size; the storage may be of smaller size than the digital image represented by the source digital image data. The embodiments of the invention are suited to encoding image data that is output from a raster image processor in a printing system, where memory resources are limited, but is also useful for compressing other types of image data. The limited size memory is usually insufficient to store a raster image directly, so the encoding compresses the image to fit the encoded image in the limited size memory. In this way, the encoding performs image compression. Compression means encoding with the objective to reduce the size of the data representing the image and may be lossy. Encoding in a compression context normally means lossless encoding and usually but not necessarily compresses the data locally entropy encoding). Also, methods, apparatuses, and computer program products are disclosed for decoding an 729890 Sencoded digital image. Likewise here, decoding may entail decompressing the stored image. In the following description, numerous specific details, including particular compression techniques, buffer sizes, color spaces, image portions, block sizes, subblock sizes and the like are set forth. However, from this disclosure, it will be apparent to those O 5 skilled in the art that modifications and/or substitutions may be made without departing Cc from the scope and spirit of the invention. In other circumstances, specific details may be Ni omitted so as not to obscure the invention.

Where reference is made in any one or more of the accompanying drawings to CK1 steps and/or features, which have the same reference numerals, those steps and/or features have for the purposes of this description the same function(s) or operation(s), unless the contrary intention appears. In the context of this specification, the word "comprising" has an open-ended, non-exclusive meaning: "including principally, but not necessarily solely", but neither "consisting essentially or" nor "consisting only of'. Variations of the word "comprising", such as "comprise" and "comprises", have corresponding meanings.

I. Raster Image Processing System Fig. 4 provides an overview of a raster image processing system 400 in accordance with an embodiment of the invention. The system 400 comprises a computer 401, a raster image processor 402 and an associated memory 403, and an application specific integration circuit (ASIC) 404 and an associated memory 408. The computer 401 is coupled to the RIP 402, which may also be an ASIC itself, and the ASIC 404. The RIP 402 is coupled to the ASIC 404. While the RIP 402, memory 403, and ASIC 404 are depicted externally relative to the computer 401, one or more of these modules may be implemented at least partly internally in the computer 401. For example, the modules may be implemented as software carried out on the computer 401. Those skilled in the art will appreciate that in addition to a hardware implementation of the system, the embodiments of the invention may be implemented in software or a combination of hardware and software modules.

The computer 401 controls the devices within the system 400. In particular, the computer 401 loads the internal memory tables of the RIP 402 and the ASIC 404, accepts 729890 rendering jobs from an input to the computer 401, effects loading of the rendering jobs into the RIP memory 403, and handles any error condition reported by the printer engine.

O Broadly speaking, the RIP 402 generates data that is input to a color space conversion module 405 of the ASIC 404, which in turn outputs color converted RIP data 00 5 to a compression module 406. The RIP 402 is described in greater detail hereinafter, Cc following a description of the ASIC 404.

I The ASIC 404 comprises several modules coupled in a pipeline manner: a color- Sspace conversion module 405, an image encoding module 406, an image decoding module 409, and a printer interface module 411. While these modules are implemented in hardware in this embodiment, as mentioned hereinbefore, some or all of these modules 405, 406, 408 (except the printer interface module 411) may be implemented as software modules operating on a general-purpose computer, or in an embedded processor, for example, in other embodiments of the invention. The image encoding module 406 and image decoding module 409 are described in greater detail hereinafter.

The module 405 performs color-space conversion from the rendering color space to the ink values of the printer engine. This color conversion module 405 accepts color data in raster order from the raster image processor 402 and converts each pixel color into ink values, in the received order. Typically, each pixel has four ink values, one each for Cyan, Magenta, Yellow and Black inks. However, other arrangements may be practiced using different inks. Methods for performing the conversion to ink values are well known in the art. The ink values are stored into a buffer (not shown) in the image compression module 406. The buffer should have sufficient size to store a predetermined or known size portion of such ink values. For example, in this embodiment, the buffer has sufficient size to store 64 scanlines of ink values in a double-buffered arrangement two banks of 32 scanlines). While a specific buffer size and configuration are set forth, in the light of this disclosure, those skilled in the art will readily appreciate that other buffer sizes and configurations may be practiced without departing from the scope and spirit of the invention. The buffer may be in off-chip memory. In another embodiment, the buffer may be in on-chip memory.

729890 ;The image encoding module 406 receives input from the color conversion module 405, encodes the ink values, and emits encoded data to an image buffer 408 (also called a final output buffer), or storage area, coupled to the ASIC 404. The encoding produces compressed image data. The image encoding module 406 has an associated memory 407 00 5 that stores data into and out of the module 406. The memory 407 is a local on-chip a memory. Details of the encoding method are set forth hereinafter. The encoding may be n performed after the conversion into ink values, allowing separate and independent compression and decompression of the various ink value channels. Common printer engines print different ink colors in separate passes by a print head, but still other print engines print in a single pass, using several non-coincident print heads. In either of these cases, the ink values for the different channels of a single pixel are required at different times. The size of the memory 408 is substantially less than the amount required for an uncompressed image (or plurality thereof).

The image decoding module 409 decodes compressed ink values obtained from the image buffer 408 and provides the decoded ink values to an internal or intermediate buffer 410 in the order in which the ink values are required by the printer engine. The internal buffer 410 should have sufficient size to store the predetermined or known size portion of such ink values. For example, in this embodiment, the internal buffer 410 has sufficient size to store 64 scanlines of ink values in a double-buffered arrangement two banks of 32 scanlines). The double-buffered arrangement ensures that ink value data is provided to the printer interface 411 at the rate required by the printer engine. While a specific buffer size is set forth, in the light of this disclosure, again those skilled in the art will readily appreciate that other buffer sizes may be practiced without departing from the scope and spirit of the invention. This buffer may be in off-chip memory. In another embodiment, the buffer may be in on-chip memory.

The printer interface 411 is the next module in the pipeline. This module 411 interfaces with the print engine (although there may be intermediate modules between the two such as a half-toning module) and performs the handshaking required by the printer engine The module 411 reads the ink values from the buffer 410. The printer interface module 411 may also process ink-value data into a form suitable for output to the printer 729890 engine (not shown), such as dithering 8-bit values down to 1 or 2 bit values, if required.

Such processing uses methods that are well known in the art.

II. Raster Image Processor 00 5 The RIP 402 generally takes as input a high-level description of a page or C€ document that is to be printed. For example, typical examples of such high-level I descriptions are PostScript, PDF, and Windows GDI. In response to a user requesting a Sdocument to be printed, a general-purpose computer usually sends a high-level description of the document to the device control computer 401. An optional interpreter converts the high-level page description into a display list, which is in turn converted into a raster image by the RIP 402. The raster image is a raster-ordered array of image pixels, for example in the RGB color space, which is color converted and printed by a given printer.

The compression system takes the color-converted raster image output and compresses that data. Such compression reduces bandwidth and storage requirements.

The output of the RIP 402 is typically a contone raster RGB image: that is an array of red, green and blue image pixels at some given resolution. An A4 page (11 inches x 8 inches) with 600 dots per inch has approximately 32 million pixels. If each pixel is represented by a red, green and blue value, with 8 bits per color typically, the page has an uncompressed raster image size of approximately 95 Mbytes. At 1200 dpi, the size is approximately 380 Mbytes. If four colors are used CMYK), the size is almost halfa gigabyte. Thus, compression of RIP output data is desirable.

The output of the RIP 402 is not generally efficiently represented by contone image data of a fixed resolution. Particularly for text black text on a white background), the transition between the background and the text characters is desirably represented with a relatively high precision (resolution). For existing printing technology, 1200 printed dots per inch is desirable for high quality text. For natural image data on the other hand, where there is significant detail that is not specifically edge related, a lower resolution is sufficient. For example, with existing printing technology, a resolution of 300 dpi is often sufficient for natural image data.

729890 ;There. is an even clearer motivation for differing resolution requirements for printers that use half-toning. Transitions between any of the printer ink colors and the background page (or in fact between any of the printer ink colors themselves) can occur at the resolution of the printer 1200 dpi). A simple and common example is the 00 5 transition between a white page and black text. The printer can print dots so that the white 0 to black transition occurs at approximately the printer resolution 1200 dpi). On the n other hand, for portions of an image where many more color gradations natural image data) exist and in particular colors that have to be approximated by half-toning, the spatial transition between colors can only occur at a much lower resolution at the effective screen frequency of the half-toning).

Thus, the output of a RIP is desirably represented in a manner that represents sharp edges at a relatively high resolution. In particular, edges between regions where printer inks transition between full-on (meaning an area of solid ink) and full-off (meaning an area of no ink), or visa-versa, are represented at a relatively high resolution. Regions of varying or flat color (or grey values) are represented at a lower resolution. If a uniform high resolution is used across all areas in the image, extra pixels are used to represent nonedge data. Extra pixels are undesirable for compression, bandwidth, and computational reasons. On the other hand, if a uniform lower resolution is used, edge data is not represented with the best quality. The embodiments of the invention allow edge data to be represented at a relatively high resolution, while non-edge data is represented at a relatively lower resolution.

ll. Higher Resolution Edge Data A number of techniques, including mixed raster content (MRC) imaging model, exist to represent edges at a higher resolution than other image data. MRC is used as a name and also to refer to the MRC model. DjVu uses the MRC model. The MRC imaging model represents edges at a higher resolution than non-edge or other data.

The MRC imaging model has three image layers: a foreground, a background, and a mask layer. The foreground and background layers typically are multi-bit-per-pixel images and have a lower resolution compared to the mask layer. The mask layer is a 729890 -12- 1binary, single-bit-per-pixel image that determines for each pixel in an output image whether the pixel has a color value taken from either the background image or foreground O image.

The mask layer effectively provides a segmentation of the foreground and 00 5 background images and in particular segments foreground objects from the background.

m The mask layer is able to represent the transition between the foreground objects and N background edges) at the resolution of the mask layer. On the other hand, the pixel data within an object non-edge or other data) where the mask layer pixels all have the (same value is represented at a lower resolution, namely the resolution of the foreground or background image.

IV. Overview of Encoding Process Fig. 1 provides a high-level overview of an encoding process 100 implemented by the image encoding module 406 of Fig 4. A loop structure is implemented in Fig. 1 using decision block 130 with a path to step 110 to illustrate that a known portion of the image is processed and encoded at a time. The portion of the image may be 32 lines of the image, but other portions and sizes may be practiced without departing from the scope and spirit of the invention.

Processing commences in step 105. In step 110, the next portion of the image is buffered. The next set of 32 RIP image color-converted lines is buffered in the memory 407. For the first time that step 110 is performed, this is simply the first set of 32 image lines. As discussed hereinbefore, a double-buffered arrangement may be practiced, so that the color-conversion module 405 can write image data into this memory 407, while the image encoder 406 is encoding a previous set of 32 lines. In step 120, that portion of the image is encoded. That is, the current set of 32 image lines is processed. Step 120 is described below in more detail with reference to Fig. 2. For the first set of 32 image lines, the image encoder 406 waits until the color-conversion module 405 has written these first lines. In decision step 130, a check is made to determine if there are any portions of the digital image remaining to be processed more sets of image lines). If step 130 returns true (Yes) indicating there are more unprocessed image lines, processing continues at step 729890 -13- 1" 110, where a next set of 32 image lines is buffered and processed. Otherwise, if step 130 returns false processing terminates at step 135.

In this way, image lines are processed and encoded in essentially raster order, a set of 32 image lines at a time. This processing is referred to as a one-pass processing method.

00 5 Once a set of input image lines have been buffered and processed, the set is not processed again; the image lines do not need to be stored or regenerated. Hence, the input image r lines can be generated once only, in raster order for example, and buffered in a relatively small band of lines.

V. Processing of a Portion of an Image Fig. 2 illustrates step 120 of Fig. 1 in more detail. The processing of step 120 involves a loop structure to encode the set of 32 image lines, a 32x32 pixel block at a time, from the left to the right. While specific portions of the image and sub-portions are stated in this embodiment, those skilled in the art will readily appreciate in the light of this disclosure that variations may be practiced without departing from the scope and spirit of the invention.

Processing commences in step 205. In step 210, the next 32x32 pixel block of data color values) from the set of image lines is buffered. Fig. 13A illustrates a 64x64 block 1300, which is processed as 32x32 pixel blocks 1302, 1304, 1306, and 1308. The 32x32 pixel block may be stored in internal memory. Again, a double-buffered arrangement may be used, so that the next block of data can be read while a current block of data is processed. In step 220, processing is carried out to determine if the block being processed contains any significant edges. Step 220 is described in more detail hereinafter with reference to Fig. 8.

In decision step 230, a check is made to determine if there are any significant edges in the block. If step 230 returns false processing continues at step 240. In step 240, processing of a low resolution (non-edge) block is carried out. Processing continues at step 260. Otherwise, if decision step 230 returns true (Yes), processing continues at step 250. In step 250, processing of a high-resolution (edge) block is carried out. Processing 729890 ;continues at step 260. Steps 240 and 250 are described hereinafter in greater detail with reference to Figs. 7 and 5, respectively.

O In decision step 260, a check is made to determine if there are any more blocks of data in the current set of 32 image lines. If step 260 returns true (Yes), processing 00 5 continues at step 210 to process the next block of data. Otherwise, if step 260 returns false Cc processing terminates in step 265.

VI. Determining if Block Has Significant Edges Fig. 8 illustrates the process of step 220 of Fig. 2 for determining if there are any significant edges in a block of data in accordance with an embodiment of the invention.

Processing commences in step 805. In step 810, the horizontal gradient for the block is computed. The horizontal gradient may be calculated as follows:, hmn bn,n+l bin, (1) where is a pixel in row m and column n of the block, m 0, 1, 31, and n 0, In step 820, the vertical gradient for the block may be calculated similarly as follows: Vmn bin+l,n (2) where bm,n is a pixel in row m and column n of the block, m 0, 30, and n 0, 1, 31.

In step 830, a determination is made as to whether any significant horizontal or vertical edges exist in the block. This determination may be made based on the calculated gradients. A determination may be made that there is a significant horizontal or vertical edge if: h, I>T or (3) where the threshold T may be 16 for 8-bit-per-color-component data. The threshold T is an empirically derived threshold. In other embodiments of the invention, different methods may be used to calculate the gradient and to determine if there are any significant edges in the blocks. For example, an edge may be determined only if there are a number of significant horizontal or vertical edges, according to the above equation, that are 8- 729890 1connected adjacent to each other in the block). Processing terminates in step 835.

Significant edges may also be determined from attributes for each pixel, which are 0 generated by the RIP 402.

00 5 VII. Processing a High-Resolution Block C€3 ~Fig. 5 illustrates in detail step 250 of Fig. 2 for processing a high-resolution block.

Ni An input 32x32 block is processed in 8x8 sub-blocks in sub-block raster order. Fig. 13B Sillustrates how a 32x32 block 1310 is processed in 8x8 blocks 1312, 1314, 1316, 1318.

For the remainder of the description of Fig. 5, the term "block" refers to an 8x8 sub-block, unless there is a specific prefix of 32x32 32x32 block). The process of Fig. employs a loop, connecting steps 560 and 510, to process each 8x8 block in turn. Each block is encoded using the same methods as those of the JPEG standard.

Processing commences in step 502. In step 505, a high-resolution bit block type information) is encoded, and one or more representative colors for the block are determined and encoded. A single 1-bit is encoded into a first bit-stream, referred to as a first scan, to indicate to a decoder that the current block is a high-resolution block. The representative colors are determined for the 32x32 block. "Color" refers to color value, or grey-value, since only a single color component is processed. The process of setting the representative color(s) in step 505 is described in more detail hereinafter with reference to Fig. 6. Following encoding of the high-resolution bit, the number of colors and the actual colors (grey-levels) determined are encoded into this first scan. The number of colors may be a unary code using a number of 0 bits terminated by a 1 bit. The number of zero bits plus 2 represents the number of colors. Thus, the code "001" represents 4 colors, while represents 2 colors. The colors themselves follow this code for the number of colors.

The color may be represented using 8 bits. In another embodiment, for example, the 8-bit color values can be quantized to 4 bits and encoded using a Huffmnan code.

In step 510, the next 8x8 block is obtained. The first 8x8 block is used for the first time this step is performed. In step 520, an 8x8 DCT is applied to the 8x8 block. In step 530, the transformed block is quantized with a high-resolution quantization matrix.

729890 -16- 1This may be done according to the JPEG method of quantization. The entries of this matrix may be given by: O Qij 64, (4) where i 0,1, 7. Thus coefficient ij is quantized with a uniform quantization-step- 00 5 size of 64.

Cc In step 540, the DC coefficient of the 8x8 block is encoded. In step 550, the AC N coefficients are encoded. Entropy encoding is carried out according to a progressive mode Sof the JPEG standard, using a number of entropy encoders. In particular, JPEG Huffman Sencoding may be performed. Further, in accordance with the one-pass nature of the compression method, fixed Huffman tables may be used. That is, the tables may be known prior to compressing the image. The DC coefficient is entropy encoded into the first scan. The AC coefficients are entropy encoded in a number of scans. There may be four scans for the AC coefficients. AC coefficients 1-3 are encoded in a first AC scan, coefficients 4-17 in a second AC scan, coefficients 18-31 in a third AC scan, and coefficients 32-63 in a fourth AC scan. These scans are encoded according to the spectral selection method of the JPEG standard. The Huffman codes used are designed in accordance with these scan partitions to avoid the use of redundant Huffmnan codewords.

In decision step 560, a check is made to determine if there are any more 8x8 blocks to process. If step 560 returns true (Yes), processing continues at step 510 to process the next block. Otherwise, if decision step 560 returns false processing terminates at step 565.

The foregoing processing is illustrated by way of example with reference to Fig.

In particular, Fig. 15A illustrates an 8x8 block of high-resolution data 1500 at step 510 of Fig. Fig. 15B shows the quantized DCT coefficients 1520 obtained by level shifting, DCT and quantization following steps 520 and 530 of Fig. The DC coefficient (quantized coefficient in Fig. 15B is coded in the first scan (per step 540 of Fig. That is, the difference with respect to the DC coefficient in the previous block is entropy coded using a pair of Huffmnan codes as in the JPEG standard. The AC coefficients 1-3 are coded according to the JPEG standard: that is, 37, 7, and 1. In a next scan, the AC coefficients are coded: that is, quantised coefficients 8, 7, 1, 1, 1, 0, 0, 1, 729890 -17- 3, 1, 0, 0, In the last scan, the AC coefficients 18-63 are encoded (per step 550 of Fig.

that is, 0, 0, 0, 0, 0, 0, 0, 0. Following decoding (described hereinafter), the resulting decompressed block 1520 is shown in Fig. VIII. Determining Representative Number of Colors Fig. 6 shows in detail a portion of the processing of step 505 in Fig. 5 for determining a number of representative colors for a 32x32 block. Processing commences in step 605. In step 610, a histogram of the block is calculated. For 8-bits-per-color component, 16 bins evenly spaced between 0 and 255 may be used. In step 620, the histogram peaks are determined. By way of example only, this may be done in the following manner. Let ci be the count for histogram bin j, where j 0, 1, 15. Bin j is a peak if: Cj_ 1 Cj Cj+ 1 or cj_ 1 Cj cj+ 1 (Cl C16 and ci >10. (6) The latter condition attempts to avoid detecting small peaks.

In step 630, the block's pixels are vector quantized to the peak centres. In step 640, representative colors are set. The number of peaks in the histogram may give the number of colors. Further, the representative colors are the mean values of the pixels that have been vector quantized to the peaks the centroid of the vector quantization cells).

In other embodiments of the invention, different methods may be used to determine the representative colors of the block. Alternatively, knowledge of the RIP process can be used to determine the number of colors in a block. For example, for a block that contains black text rendered onto a white background, the RIP system can easily determine that there are only two colors in the block and that these colors have values of 0 and 255, respectively. The RIP system knowledge can also be used to determine if a block contains significant edges or not. Processing terminates in step 645.

729890 1" IX. Alternative Processing of a High-Resolution Block SAn alternate embodiment involves encoding a high-resolution block where only 0 two representative colors exist. The pixels for such a block are quantized to a color value of 0 or 1, depending on whether the pixels are closest to a first or a second representative oO 5 color. An 8x8 sub-block of all zeros a "zero block") is encoded with the code c A sub-block of all ones a "one block") is encoded with the code "1 For such N sub-blocks, further coding is not required. A sub-block that is not all zeros or ones San "edge block") is encoded with the code These codes and "0" Sdescribing a zero block, a one block, and an edge block, respectively, are each output to the most significant (or DC) scan.

An edge block is encoded according to the method described in steps 520 to 550 of Fig. 5. However, a quantization-step size of 1 is used in step 530. Further variations on this alternate embodiment for high-resolution blocks are possible. For example, codes for runs of zero blocks and one blocks may be used. Further, the coding of an edge block may incorporate special cases. For example, five bits are needed to describe the 32 different possible edge blocks that have an (exact) horizontal or vertical edge within those blocks.

A single bit may be used to describe whether an edge block is such a special horizontal or vertical edge block. If a block is a special horizontal or vertical edge block, the following five bits describe exactly which one of these 32 different types the edge block is. These bits are all output to the most significant scan. If the edge-block is not such a special horizontal or vertical edge block, coding proceeds as above.

This alternate embodiment for high-resolution block coding is also suitable for the encoding of binary mask layers in a number of scans of decreasing significance. For binary images, there is no initial quantizing step to binary values, as the data is already binary. For example, the mask layer may be encoded in a mixed raster content (MRC) coder.

X. Processing a Low-Resolution Block Fig. 7 illustrates in detail step 240 of Fig. 2 for processing a low-resolution block.

Processing commences in step 705. In step 710, the 32x32 block is sub-sampled to an 8x8 729890 block. The sub-sampling may be implemented by averaging each 4x4 sub-block ofpixels.

In other steps, the 8x8 block may be encoded using the methods of the JPEG standard. In step 720, an 8x8 DCT is applied to the subsampled 8x8 blocks. In step 730, each transformed block is quantized with a low-resolution quantization matrix. This may be a matrix having entries given by the entries of the JPEG recommended quantization matrix divided by 2 (and rounded down).

In step 740, the low-resolution bit block type information) and the DC coefficient are each encoded. A single 0-bit is first encoded into the first scan to indicate to a decoder that the current block is a low-resolution block block type information).

The DC coefficient is encoded according to the JPEG standard. In step 750, the AC coefficients are encoded. The entropy encoding may be implemented according to a spectral selection mode of the JPEG standard. This entropy encoding may be the same as that used in steps 540 and 550 of Fig. 5. The AC coefficients may be encoded in four scans, as above, giving a total of five scans (one DC scan and 4 AC scans). The same scans are used and the entropy-encoded data are written out to the same scans, as noted above. Processing terminates in step 755.

An entropy encoder may be provided for each scan there may be five entropy encoders). Each encoder may be implemented as part of the compression ASIC 406 of Fig. 4, and the encoders may work substantially in parallel. In another embodiment, there may be one entropy encoder that encodes each scan.

XI. Storing Encoded Digital Image Data Figs. 3A and 3B illustrate a method 300 of managing storage of encoded (compressed) image data in the output buffer 408 (labelled "memory" in Fig. which has a fixed memory size. This method 300 is performed by a scan output manager, which may be implemented as a component of the application specific integrated circuit (ASIC) 404.

The scan output manager is part of the image encoding module 406. The scan output manager may also be practiced as a software program operating as a sub-component of a larger software system implementation.

729890 SThe method 300 performed by the scan output manager commences in step 302 with the image compression module 406 commencing encoding of the color raster data.

Fig. 16A is a block diagram graphically depicting the organisation of storage 1600 in r accordance with the embodiments of the invention. The storage 1600 corresponds to 00 memory 408 in Fig. 4. Hereinafter, the storage 1600 is referred to as "final output buffer".

The final output buffer 1600 is organised to have a header 1610 and reserved memory S1620 used as a buffer for storing data (in blocks). The header 1610 includes a memory Smanagement table described in greater detail hereinafter. The memory management table may in fact be implemented partly or entirely in another memory.

In step 304, the scan output manager undertakes an initialisation process, in which: The final output buffer 1600 is divided into blocks (in the portion 1620 indicated by hashing, but not shown) that form a free block list; each block may have a pointer for addressing/linking to the next block in the linked list. Alternatively, this may be done implicitly as blocks are first needed. Memory for the header 1610 is set-aside at the head of the final output buffer 1600 for subsequent storage of information concerning each scan stream. This information forms part of the memory management table. This information includes, for each scan stream, the scan stream number and the corresponding linked-list root address (if any).

In the memory management table, an active flag for each scan is set to indicate that the corresponding entropy-encoded scan is active.

Each scan's linked-list root address and current block address in the linked list stored in the memory management table is set to zero.

Each scan's fill index stored in the memory management table is set to the value that indicates the block is full a maximum value). The fill index describes how many bytes are currently filled being 729890 used) for the currently active block (one fill index per scan). For example, for a block size of 4 kbytes after 2 kbytes of data have been written to scan 0, the fill index is 2k. After 4k, the fill index is 4k.

00 After 4k plus 2 bytes, the fill index will be 2 bytes because there is a oO Mc 5 new block with only two bytes filled.

N Fig. 16B is a block diagram showing a group 1630 of four blocks 1632, 1634, S1636, and 1638 of transformed coefficients. The scans' for the respective blocks 1632, 1634, 1636, and 1638 are: DCoo 00 and ACoo; DC 01 and ACol; DClo and AC 1 o; and DC 11 and AC 11 For ease of explanation, only four blocks 1632, 1634, 1636, and 1638 are depicted and each block is shown to have only two scans DC,, and Thus, as shown in Fig.

16C there are only two scans 1670: scan 0 for DC coefficients and scan 1 for AC coefficients. The resulting scan stream 1670 for scan 0 is also shown, comprised of the four sets of DC coefficients DCoo 00

DC

01 DCl 0 and DC 11 for the blocks 1632, 1634, 1636, and 1638. This scan stream has the value 0011100110100.

Fig. 16D shows an example of the memory management table 1610 and related blocks 1650 and 1660 of reserved memory 1620 for storing encoded data. In the memory management table 1610, there are at least three components of information for each scan stream. The first table component 1612 contains information about the scan stream number scan The second component 1614 is other information about the scan stream, such as an active flag and a fill index for each scan. "Scan" is a conceptual name, while "scan stream" is the name for physical data of a scan. The third component 1616 contains information about the linked-list root address (pointing to the first block storing data for the scan stream. Information about the current block address may be stored as well. An arrow extending from the third component 1616 points to the first block 1650 in the linked list for scan 0. This block 1650 contains part of the encoded data 1652 (coefficients DCoo 00 and DC 01 "1 0 1 1 10 0 and a pointer 1654 to the next block in the linked list. In this example the pointer 1654 points to block 1660. The block 1660 729890 -22contains encoded data 1662 (coefficients DC 1 0 and DCI "1 1 0 and a pointer 1664, which currently has a NULL value since the pointer 1664 does not point to another block.

Block 1660 is currently the last element in the linked list. The asterisk in block 1660 tn indicates the remainder of the block is empty and the free block register would point to 00 MC 5 this block in the linked list.

SAfter step 304, the scan output manager proceeds to an idle state 306, in which the scan output manager waits for a signal from an entropy encoder(s). In step 306, each time Sthe scan output manager is signalled that a fragment of entropy-encoded data is available on a particular scan stream (hereinafter called the current scan stream), the scan output manager proceeds to step 310. In decision step 310, the active flag entry of the current scan stream in the memory-management table is checked to determine if the current scan stream is active. If decision step 310 returns false (No) indicating that the current scan stream is stopped, processing continues at step 322. In step 322, the scan output manager discards the fragment of entropy-encoded data and, from step 322, processing of the manager returns to the idle state in step 306. Otherwise, if step 310 returns true (Yes), processing continues at step 312.

In decision step 312, a check is made to determine if the fill index of the current scan stream equals a maximum value. The scan output manager checks the fill-index entry of the current scan stream in the memory management table 1610. If step 312 returns false (No) indicating that the fill-index entry is not equal to the maximum allowed value, processing continues at step 326. That is, space is available in the current block to store entropy encoded data. Otherwise, if decision step 312 returns true (Yes), processing continues at step 314.

In step 314, the current scan stream is processed by either obtaining the address of a free block to store the entropy encoded data or otherwise discarding the entropy encoded data of the current scan stream (the processing may be dependent upon perceptual significance). If free space is available in the storage irrespective of perceptual 729890 significance, the current entropy encoded data will be stored in that space without discarding any least perceptually significant data. If storage space is not available for the

O

least perceptually significant entropy-encoded data, such data is discarded. This may be done by overwriting stored encoded data in the storage with the current entropy encoded 00 M 5 data if the former is the least perceptually significant, or by discarding the current entropy Sencoded data without storing the same if this data is the least perceptually significant.

OThe scan output manager performs this processing using a sub-process described Shereinafter with reference to Fig. 3B. Thus, if the address of a free block is obtained, processing continues at step 316, but otherwise if the entropy encoded data of the current scan stream is discarded, processing continues at step 306.

In decision step 316, the scan output manager considers whether the current-blockaddress entry of the current scan stream in the memory management table is zero. If step 316 returns true (Yes), processing continues at step 318. In step 318, the linked-list rootaddress entry in the memory management table for the current scan stream is set to this free-block address. Processing continues at step 320. Otherwise, if decision step 316 returns false processinlg continues at step 320.

In step 320, the current-block-address entry in the memory-management table for the current-scan stream is set to this free-block address and the free-block register is updated to the address of the next free block in the linked list of free blocks. If there are no more free blocks available in the linked list of free blocks, the scan output manager sets the free block list to zero. In step 324, the scan output manager sets the fill-index entry of the memory management table to zero. Processing continues at step 326.

In step 326, the scan output manager writes the fragment of entropy-encoded data to the current block at the fill index. The fragment is written into the final output buffer 408. The data is written to that location in the current block indicated by the currentblock-address entry and the fill-index entry in the memory-management table corresponding to the current scan stream. Once the data is written, the fill-index entry 729890 -24corresponding to the current-scan stream is incremented to account for the new data.

Processing continues at step 306 and the scan output manager returns to its idle state.

In step 308, the operations of the scan output manager are terminated in response t to instructions from a higher-level software application (not shown) in the case of a 00 S 5 software implementation or a controller (not shown) in the case of a hardware N implementation. Prior to termination in step 308, the scan output manager stores in the (ri final output buffer 408 the information for each scan stream including the scan stream number and the corresponding linked-list root address (if any) of the scan stream.

Fig. 3B illustrates the sub-processes of step 314 of Fig. 3A. Processing commences in step 340. In step 350, a check is made to determine if the free block list is zero. By determining that the free block list is zero, the scan output manager can detect that the final output buffer 408 is full. If step 350 returns false (No) indicating that the free block list is not zero, processing continues at step 360. That is, space in the storage is available for storing the current entropy encoded data. In step 360, the scan output manager obtains the address of this free block in the free block list. The scan output manager also updates the free block register by assigning it the address of the next free block in the linked list of free blocks. The free block register stores the address of the next (first) free block on the free block list. If there is no next free block in the linked list of free blocks, the scan output manger sets the free block register to zero. In step 362, processing returns to the calling process processing continues at step 316 of Fig. 3A).

Otherwise, if decision step 350 returns true (Yes) indicating that the free block list is zero, processing continues at step 352. In step 352, the scan output manager sets the active flag entry (in the memory-management table) of the least perceptually significant of the active scan streams to inactive. By assigning the linked root address of the leastperceptually-significant scan stream to the free block list, the scan output manager allows data of a more-perceptually-significant scan stream to simply overwrite the leastperceptually-significant scan stream. In this way, the embodiment performs a simple one- 729890 pass processing for fitting the encoded (compressed) data into the final output buffer 408 of.fixed memory size. The free block list then comprises the blocks of memory used by the newly inactive scan.

tr In step 354, the scan output manager assigns the linked-root address of the just 00 M 5 previously flagged inactive scan stream to the free block list. Once the free block list I becomes non-zero, that address is used as the address of the next free block. Thus, the Sscan output manager allocates the same scan, for example scan 4, to the same linked list.

If the scan output manager assigns the linked root address of the least-perceptuallysignificant scan stream to the free-block register, the scan output manager is effectively discarding from storage in the final output buffer 408 the same most-perceptuallyinsignificant scan. Thus, the perceptual degradation introduced to achieve compression of a fixed size is applied consistently to the whole image.

In decision step 356, a check is made to determine if the just previously flagged inactive stream is the current scan stream. If step 356 returns true (Yes) indicating that the entropy-encoded stream that just became inactive is the current scan stream, processing continues in step 358. In step 358, the fragment of entropy-encoded data of the current stream is discarded. In step 364, processing returns to the calling process the scan output manager returns to its idle state 306). Otherwise, if decision step 356 returns false processing continues at step 350, in which again the scan output manager determines if the free block list is zero.

The scan output manager's operation in Figs. 3A and 3B is illustrated in detail by the following code in Table 1.

TABLE 1 struct block struct block *next; word data[1000]; 729890 -26- 1; struct stream {tut lc str-uct block int bool head; *curr~ent; fill-index; active; streams[18]; free-root; n-active; struct stream str-uct block int void emit entropy encoded-data(int s, word data) if (!streams active) return; if (streams[s]. fil Uindex 1000) while (free-root NULL) n active; free-root streams [nactive]. head; streams [nactive]. active false; if (n active s) return; if (streams[ current NULL) 729890 -27- Sstreams[s].head freeroot; else streams[s].current->next free_root; streams[s].current freeroot; 00 5 freeroot freeroot->next; M€3 streams[s].current->next NULL; C streams[s].fill_index 0; 8 N streams[s].current->data[streams[s].fill_index] data; ++streams[s].fill_index; As an image is being compressed in substantially raster (or 32-line raster) order, the scan output manager ensures that the compressed image bit stream does not overflow the output buffer 408. If the buffer 408 becomes full at any stage before encoding is complete, the scan output manager discards the least-perceptually-significant scan that remains in the buffer 408. Doing so reclaims buffer space for remaining portions of the compressed image bit stream. In this way, several scans may be discarded during encoding. Any scans discarded during encoding are not part of the final compressed image bit stream. In one embodiment, the encoded scans for all four image components M, Y and K) are coded into the same buffer memory area. A more significant scan from a first component may overwrite lesser significant scans from a different second component, if there are no lesser significant scans active for the first component. In this way, the components share the memory in a dependent fashion. In alternative embodiments, a fixed amount of buffer memory area may be set aside for each component, and if required each component's scans can only overwrite less perceptually significant scans from the same component.

Each scan may be collected into a temporary buffer before being output to the main linked-list memory. Each buffer may be of size 1 kbytes, for example. However, even 729890 with this temporary buffering arrangement, the bandwidth required to output each buffer to main memory (or even into the temporary buffers) may become large during the encoding of an image where regions of the image are particularly complicated). In another t embodiment of the invention, the scan output manager may be used to manage this 00 M 5 bandwidth. -If the bandwidth required to output the compressed data the encoded N, scans) becomes larger than a given threshold, a least significant scan (or number thereof) (Ni may be discarded. Alternatively, a least significant scan's encoded data may be replaced with a smaller encoding non-zero coefficient data may be replaced with an end of block (EOB) or EOB run data). In this way, the scan output manager may be used to ensure that a sustained bandwidth is kept below a given threshold during the encoding of the image.

Different transforms may be employed discrete wavelet transform, discrete Hadamard transform, etc), and similarly different transform coders may be used.

Regardless of which transform is employed, if a compressed image bit stream comprising all scans is too large for the output buffer 408, the scan manager ensures that the leastperceptually-significant scans are discarded. In this manner, the final encoded image bit stream comprises the perceptually most important scans, fits into the output buffer 408, and is formed in one pass in substantially raster order.

Corresponding to each scan is a compressed image. For a given scan, the corresponding compressed image is the image represented by the given scan and all previous scans. Thus, corresponding to the sequence of scans is a sequence of compressed images. For increasing sequence number, the compressed image quality increases, while the compressed image size increases. The compression method selects the largest compressed image (or highest quality compressed image) that fits into the buffer 408.

Thus, for the next larger compressed image that follows in a given sequence, the final compressed image does not generally fit in the buffer 408. Some redundancy exists in that the final compressed image does not in general totally fill the buffer 408. To minimise 729890 -29this redundancy, a sequence with fine granularity is desirable. That is, the increase in size from one scan to the next is small, suggesting a lot of scans. However, the more scans that are used the more complicated the encoding process is, so there is a trade off.

There is another consideration in the design of the scan partition that can be 00 M 5 phrased in several different ways. One-way is that each scan should maximise the increase N in the compressed image quality for a given size increase (of compressed image). A related Sway is that each scan should minimise the size increase for a given increase in compressed image quality. The first way is used here for the purposes of discussion, but other ways are equally applicable.

The scans maximise the increase in quality for a given size increase, and the size increase is not too large.

In alternate embodiments, other entropy encoding techniques can be employed. For example, a binary arithmetic coder could be used to encode the bit-planes of the quantized coefficients using the sub-bit plane or multi-pass approach of the draft JPEG2000 standard published in document "Information Technology JPEG 2000 Image Coding System JPEG 2000 Committee Draft Version 1.0, 9 December 1999" (hereinafter referred to as JPEG2000).

XII. Overview of Decodin2 Returning now to Fig. 4, the image decoding module 409 (or simply the decoder) takes the stored, encoded (compressed) image from the final output buffer 408 and reconstructs the image. The image decoder 409 comprises a scan input manager for retrieving each compressed scan from the output buffer, entropy decoders for decoding the encoded scans, and an inverse DCT unit for combining the scans into a block of transform coefficients and for performing an inverse DCT on the decoded block.

Fig. 9 illustrates a high-level overview of operation of the decoder 409. A loop structure is used in Fig. 9 to illustrate that a portion of the image 32 lines) are 729890 b ;Z decoded at a time. Processing commences in step 905. In step 910, the next portion of the stored, encoded image a set of 32 image lines) is decoded. In decision block 920, a check is made to determine if there are more portions of that image sets of image 00 lines) to decode. If step 920 returns true (Yes), processing continues at step 910, in which oO M 5 a next portion of the stored, encoded image a set of 32 image lines) is decoded.

Otherwise, if decision step 920 returns false processing terminates in step 925. In this way, the stored, encoded image is decoded in essentially raster order, a portion (i.e.

32-raster-image lines) at a time.

XI. Decoding a Set of Image Lines Step 910 of Fig. 9 is illustrated in more detail with reference to Fig. 10. employs a loop structure to decode the next portion of the stored, encoded image block by block a set of 32 image lines a 32x32 block) at a time, from left to right. Processing commences in step 1005. In step 1010, the block type information is decoded for the current block in the set of 32 image lines. The block type information indicates if the block is high or low resolution. In one embodiment of the image decoder 409, this type is determined by examining the next bit in the most significant scan. That is, the resolution bit in the first scan (also containing the DC coefficient) is decoded.

In decision step 1020, a check is made using the decoded bit to determine if the block is a low-resolution block. If decision step 1020 returns false (No) indicating the block is a high-resolution block the bit is a one), processing continues at step 1040.

In step 1040, the high-resolution block is decoded. Step 1040 is described hereinafter in more detail with reference to Fig. 11. Processing continues at step 1050. Otherwise, if decision step 1020 returns true (Yes) indicating a low-resolution block, processing continues at step 1030. In step 1030, a low-resolution block is decoded. That is, all the scans corresponding to a next (low-resolution) 8 x 8 block are entropy decoded according to the JPEG method. The block is then up-sampled to a 32x32 block using interpolation.

729890 -31- The details of this are described hereinafter with reference to Fig. 17. Processing continues at decision step 1050.

In decision step 1050, a check is made to determine if there are any more blocks to 'n be processed. If step 1050 returns true (Yes), processing continues at step 1052, in which 00 M 5 the next block is obtained. Processing then continues at step 1010 to decode the next Sblock (if there is such a block). Otherwise, if step 1050 returns false processing terminates in step 1055 if all blocks in the current set of 32 image lines have been Sdecoded.

XIV. Decoding a High-Resolution Block Fig. 11 is a flow diagram illustrating steps used to decode a high-resolution block according to step 1040 of Fig. 10. Processing commences in step 1105. In step 1110, the number of representative colors and the actual colors (grey-levels) are decoded. This may be done from the most significant scan. In step 1120, each 8x8 block is decompressed.

This may be done by entropy decoding each scan for each block according to the JPEG method, inverse quantizing the decoded block, and inverse transforming the inversequantized block.

In step 1130, the decoded block is saturated to the decoded representative color(s) in a selective manner. That is, the decompressed pixels are set to the nearest color in the set of decoded representative colors. For example, if the representative colors are 0 and 255 (for 8 bit data), each decompressed pixel in the 32x32 block is set to either 0 or 255.

In particular, if a decompressed pixel value is less than 128, the pixel is set to 0; otherwise, the decompressed pixel is set to 255. In another embodiment of the invention, only pixels in decoded blocks that have non-zero quantized AC coefficients are saturated. Pixels in blocks with all AC coefficients quantized to zero are interpolated from pixels in surrounding blocks: in particular from saturated pixels or fellow unsaturated pixels that are 729890 of a similar value (compared to the set of representative colors), in surrounding blocks.

Processing terminates in step 1135.

XV. DecodinR a Low-Resolution Block 00 M 5 Fig. 17 is a flow diagram illustrating steps used to decode a low-resolution block Saccording to step 1030 of Fig. 10. Processing commences in step 1710. In step 1720, each 8x8 block of encoded low-resolution data is decoded. This may be done by entropy C decoding each scan for each block according to the JPEG method, inverse quantizing the decoded block, and inverse transforming the inverse-quantized block. In step 1720, the decoded 8x8 block is upsampled to provide a 32x32 block. Processing terminates in step 1740.

XVI. Various Embodiments The scan partition used in the first embodiment uses coding as done in a spectral selection mode of JPEG. Nevertheless, different embodiments may use different scan partitions. For example, scan partitions that use coding as done in mixed spectral selection and successive approximation modes of JPEG can be used. A quantized DCT block can be encoded in scans as described in the Table 2 (hereinafter). Each row in the table specifies one (encoding) scan. The scans are encoded in row order. The Ss Se column specifies the start (Ss) and end (Se) index, following a JPEG zigzag sequence of DCT coefficients, of a contiguous sub-sequence of DCT coefficients that are encoded in the scan. Thus, for example, the second row of the table specifies that in the second scan DCT coefficients 1 to 5 are encoded. The Ah Al column specifies the bit precision high and bit precision low parameters respectively used in the successive approximation mode of progressive JPEG. In the first scan for a coefficient, Ah is set to 0 as in the first 3 rows in the scan table.

The successive approximation mode of JPEG defines two slightly different point transforms. The use or purpose of the point transform is to essentially partition the 729890 -33- ;Zcoefficients by bit-plane. However, depending upon the point transform the result may or may not correspond to a bit-plane. Nevertheless, the term bit-plane is used herein to refer to a partition resulting from a point transform.

The scans of the Table 2 are explained more detail. In the first three scans, the data 00oO M 5 from bit-planes 1 and above are included only. The bit-planes are numbered consecutively -q from the least significant bit-plane (bit-plane zero). This is specified by the Ah:A1 0:1 Spair. The parameter Al=I specifies that bit-plane 1 is the lowest bit-plane included, while the parameter Ah=0 specifies that this is the first partition of the bit-planes, and hence to include all bit-planes above Al inclusive. Scan 1 comprises the DC coefficient (scan index That is, bit-plane 1 and above for each DC coefficient are included in this scan.

Scan 2 comprises the bit-plane data for all bit-planes above and including bit-plane 1 for AC coefficients 1-5. Similarly, Scan 3 comprises the same bit-planes for AC coefficients 6-63. In Scan 4, a new Ah:A 1:0 pair is used. This pair specifies that bit-plane A1=0 is transmitted in this scan. Thus, in Scan 4, bit-plane 0 of the DC coefficient is included. In Scan 5, bit-plane 0 for AC coefficients 1 to 5 is included. Similarly, in Scan 6, bit-plane 0 for AC coefficients 6 to 63 is included.

TABLE 2 Ss Se Ah: Al 0-0 0,1 0,1 6-63 0,1 0-0 6-63 729890 -34- The use of the DCT and a JPEG (like) encoder to compress both low-resolution and high-resolution blocks is advantageous. Low-resolution blocks, often containing

O

natural image data or flat colors, compress well with a JPEG scheme and allow a sequence of scans that facilitate a good trade-off between decompressed quality and compressed MO 5 block size (by deleting a number of least significant scans). High-resolution blocks ,I compressed and decompressed as described above also have these advantageous features.

By using a quantization matrix with relatively large values, reasonable compression is achieved even for edge blocks. On the other hand, the saturation by the decoder preserves the sharp edges of decoded blocks that contain edges. For blocks with two colors 0 and 255), lossless compression and decompression is usually achieved. Further, deleting high-frequency scans does not usually significantly degrade the quality of such decoded blocks. An edge in a block may shift slightly in such a decoded block relative to its position in the original image block, but a sharp edge is maintained. Thus, the facility to delete (most insignificant) scans allows a gradual trade-off between compression and the quality of the decoded blocks.

The JPEG-style compression and saturated decoding are used to encode the location of the edges in blocks that contain edges. Deleting scans may mean the edge is no longer exactly represented but the decoded edge remains sharp, with typically the edge itself only slightly distorted. This provides a good trade-off between compressed rate and visual quality. If scan data for an edge block is deleted, the inverse DCT is used as an interpolator. An edge is typically smooth in that the path of an edge varies smoothly in the two-dimensional image space. There are of course occasional discontinuous points where say two straight edges of different angles meet. Even if interpolating from deleted scans the inverse DCT preserves this smooth nature and the (approximate) location of any discontinuous points. In fact, a larger-block-size inverse DCT can be used to scale up edge blocks to larger resolutions. For example, a 16x16 inverse DCT could be used (on 16 x 16 blocks whose lowest 8 x 8 frequency components are the usual 8 x 8 decompressed 729890 b ;blocks) to decode a 64x64 block that is at twice the "resolution" of the usual 32x32 decoded block.

The encoding of both edge data and non-edge data in a sequence of scans of t decreasing significance is advantageous in that the encoding facilitates a memory 00 Mc 5 management technique, in which an image may be compressed in one pass into a storage ,IC of fixed size without over-flowing the storage at any stage. By'separating out data that Srepresents edges (and possibly including other information), and (mostly) non-edge data, edges can be represented at a higher resolution than (most) non-edge data, which facilitates an efficient trade-off between overall output image quality and compression.

Multiple scans are decoded in the decoding of each block. Preferably these scans are decoded in parallel to facilitate decoding speed; each decoder must be suitably offset in a pipeline implementation to allow for causal decoding.

In the first embodiment, each component of image data is compressed independently. That is, the C, M, Y and K channels are each compressed with edge data and other data independently. In alternate embodiments, the components may be compressed in a dependant fashion. For example, a block is encoded as a high-resolution block if there are significant edges in any of the components. The block is encoded by quantizing the pixels to indices representing the representative colors (and now each representative color consists of a color value for each of the color components), where there is a different index for each representative color. This array of 32x32 indices is encoded?] as described to encode the location of edge blocks. The other data may be encoded independently or interleaved. In alternate embodiments other scan partitions are possible.

In respect of an alternate embodiment, the compression technique described hereinbefore is suitable for compressing scanned data to ensure that incrementally compressed scanned data does not overflow an output buffer, or possibly exceed a compressed data output bandwidth. The classification of blocks as edge or non-edge 729890 -36blocks is generally a harder problem for scanned data as opposed to RIP or rendered data.

Scanned data is typically noisy, and it is more difficult to determine whether the original

O

image data contains sharp edges or not text edges or natural image data). Thus, an alternate method described hereinbefore with reference to Fig. 8 may be used for 00 S 5 compressing scanned data. There has to be a number of eight-connected horizontal or vertical edges within a block before the block is classified as an edge block.

OIn yet another alternate embodiment, a compressed format that allows for rotated decoding can be used. Rotated decoding is made possible through the use of blocks and pointers into the encoded scan data. In this alternate embodiment, 64 image lines are processed as a band of lines, with blocks or tiles of 64x64 pixels classified as low- or high-resolution non-edge or edge) blocks and encoded accordingly. Blocks are compressed as described hereinbefore in the first embodiment, except there are four 8x8 sub-blocks compressed for a low-resolution block and 64 8x8 sub-blocks compressed for a high-resolution block. Further, a table of pointers into the encoded scans may be maintained for each scan for each block row. That is, for each band of 64 image lines, there is a pointer to the start of the encoded scan data for each scan. A decoder can use this pointer information to decode blocks or tiles in the usual row-order or column order and by buffering a row or column of blocks and reversing the order of blocks, provide various rotated 270 degrees) and reflected decodings of the image. In the various embodiments, an enhancement is useful for blocks for which only the most significant scan contains significant data non-end of block data). For such blocks, the most significant scan for the block is completed with a special code indicating that all remaining scans contain insignificant data.

XVII. Computer Implementation The methods according to the embodiments of the invention may be practiced using one or more general-purpose computer systems, printing devices, and other suitable computing devices. The processes and modules described with reference to any one or 729890 -37more of Figs. 1-11, 13, and 15-17 may be implemented as software, such as an application program executing within the computer system or embedded in a printing device.

O Software may include one or more computer programs, including application programs, an operating system, procedures, rules, data structures, and data. The instructions may be 00 5 formed as one or more code modules, each for performing one or more particular tasks.

M€3 The software may be stored in a computer readable medium, comprising one or more of N the storage devices described below, for example. The computer system loads the Ssoftware from the computer readable medium and executes the software. Fig. 14 depicts Nan example of a computer system 1400 with which the embodiments of the invention may be practiced. A computer readable medium having such software recorded on the medium is a computer program product. The use of the computer program product in the computer system may effect an advantageous apparatus for implementing one or more of the above methods.

Fig. 14 illustrates the computer system 1400 in block diagram form, coupled to a network. An operator may use the keyboard 1430 and/or a pointing device such as the mouse 1432 (or touchpad, for example) to provide input to the computer 1450. The computer system 1400 may have any of a number of output devices, including line printers, laser printers, plotters, and other reproduction devices connected to the computer.

Input devices to the computer system 1400 may include devices such as scanners (not shown). The computer system 1400 can be connected to one or more other computers via a communication interface 1464 using an appropriate communication channel 1440 such as a modem communications path, router, or the like. The computer network 1420 may comprise a local area network (LAN), a wide area network (WAN), an Intranet, and/or the Internet, for example.

The computer 1450 may comprise a processing unit 1466 one or more central processing units), memory 1470 which may comprise random access memory (RAM), read-only memory (ROM), or a combination of the two, input/output (IO) interfaces 1472, a graphics interface 1460, and one or more storage devices 1462. The storage device(s) 1462 may comprise one or more of the following: a floppy disc, a hard disc drive, a magneto-optical disc drive, CD-ROM, DVD, a data card or memory stick, flash RAM 729890 -38- 1device, magnetic tape or any other of a number of non-volatile storage devices well known to those skilled in the art. While the storage device is shown directly connected to the bus O in Fig. 14, such a storage device may be connected through any suitable interface, such as a parallel port, serial port, USB interface, a Firewire interface, a wireless interface 00 5 IEEE 802.11 a-g, BlueTooth, etc.), a PCMCIA slot, or the like. For the purposes of this Cn description, a storage unit may comprise one or more of the memory 1470 and the storage Sdevices 1462 (as indicated by a dashed box surrounding these elements in Fig. 14).

N devices 1462 (as indicated by a dashed box surrounding these elements in Fig. 14).

Each of the components of the computer 1450 is typically connected to one or more of the other devices via one or more buses 1480, depicted generally in Fig. 14, that in turn comprise data, address, and control buses. While a single bus 1480 is depicted in Fig.

14, it will be well understood by those skilled in the art that a computer, a printing device, or other electronic computing device, may have several buses including one or more of a processor bus, a memory bus, a graphics card bus, and a peripheral bus. Suitable bridges may be utilized to interface communications between such buses. While a system using a CPU has been described, it will be appreciated by those skilled in the art that other processing units capable of processing data and carrying out operations may be used instead without departing from the scope and spirit of the invention.

The computer system 1400 is simply provided for illustrative purposes, and other configurations can be employed without departing from the scope and spirit of the invention. Computers with which the embodiment can be practiced comprise IBM- PC/ATs or compatibles, laptop/notebook computers, one of the Macintosh (TM) family of PCs, Sun Sparcstation a PDA, a workstation or the like. The foregoing are merely examples of the types of devices with which the embodiments of the invention may be practiced. Typically, the processes of the embodiments, described hereinafter, are resident as software or a program recorded on a hard disk drive as the computer readable medium, and read and controlled using the processor. Intermediate storage of the program and intermediate data and any data fetched from the network may be accomplished using the semiconductor memory.

In some instances, the program may be supplied encoded on a CD-ROM or a floppy disk, or alternatively could be read from a network via a modem device connected 729890 to the computer, for example. Still further, the software can also be loaded into the computer system from other computer readable medium comprising magnetic tape, a O ROM or integrated circuit, a magneto-optical disk, a radio or infra-red transmission channel between the computer and another device, a computer readable card such as a 00 5 PCMCIA card, and the Internet and Intranets comprising email transmissions and information recorded on websites and the like. The foregoing is merely an example of Srelevant computer readable mediums. Other computer readable mediums may be Spracticed without departing from the scope and spirit of the invention.

Industrial Applicability The embodiments of the invention are applicable to the computer and data processing industries. The foregoing describes only some embodiments of the present invention, and modifications and/or changes can be made thereto without departing from the scope and spirit of the invention, the embodiments being illustrative and not restrictive.

729890

Claims (31)

1. A method of storing digital image data, said method comprising the steps of: 00 M encoding in one or more scans digital image data in a band of the digital image at a first resolution; encoding in one or more scans digital image data in the same or another band of N the digital image at a second resolution, said second resolution being lower than said first resolution; and managing a storage such that if said storage is full or substantially full, at least part of an encoded scan that is least perceptually significant relative to other encoded scans in said storage is discarded.

2. The method according to claim 1, wherein said digital image data contains edge data in said band of the digital image at said first resolution.

3. The method according to claim 1, wherein said digital image data contains significant edge data in said band of the digital image at said first resolution, said edge data being significant if a gradient in at least one direction satisfies a predetermined condition.

4. The method according to claim 2 or 3, wherein said digital image data contains other data in the same or another band of the digital image at said second resolution. The method according to claim 4, wherein said other data comprises non- edge data.

6. The method according to claim 4 or 5, wherein said other data comprises insignificant edge data, said edge data being insignificant if a gradient in at least one direction fails to satisfy a predetermined condition.

7. The method according to claim 1, wherein said managing step comprises storing at least one encoded scan in said storage. 729890

8. The method according to claim 7, wherein said managing step comprises overwriting in said storage at least part of said encoded scan that is least perceptually O significant with at least part of the same or another encoded scan that is perceptually more significant. 00 5 9. The method according to claim 7, wherein said managing step comprises discarding at least part of said encoded scan that is least perceptually significant without N storing said discarded part of said encoded scan in said storage. The method according to claim 1, wherein said digital image data stored in scans in said storage can be decoded into a plurality ofpixels.

11. The method according to claim 1, wherein said storage has a fixed memory size.

12. The method according to claim 2 or 3, wherein at least two of said encoded scans represent edge data in a band of said digital image data.

13. The method according to claim 4, wherein at least one encoded scan contains data from encoded edge data and encoded other data.

14. The method according to claim 4, wherein encoded scans comprise data from only one of encoded edge and encoded other data. The method according to claim 4, wherein encoding one or more scans containing other data comprises applying transform coding to at least a portion of said one or more scans.

16. The method according to claim 15, wherein encoding one or more scans containing other data comprises coding according to the JPEG method.

17. The method according to claim 2 or 3, wherein encoding one or more scans containing edge data comprises applying transform coding to at least a portion of said one or more scans.

18. The method according to claim 17, wherein said transform coding comprises a Discrete Cosine Transform (DCT).

19. The method according to claim 1, wherein said encoding steps comprise compressing said digital image data. 729890 The method according to claim 4, wherein said digital image data containing edge data in a band of the digital image and digital image data containing other O data in a band of the digital image are encoded using the same encoding technique.

21. The method according to claim 1, wherein said digital image data is 00 5 encoded in tiles. c 22. The method according to claim 1, wherein said encoding comprises storing pointer information in a scan to the start of said encoded digital image.

23. The method according to claim 1, wherein encoded digital image data is an N encoded form of multiple image layers.

24. The method according to claim 1, wherein encoded digital image data is in a mixed-raster content format. An apparatus for storing digital image data, said apparatus comprising: means for encoding in one or more scans digital image data in a band of the digital image at a first resolution; means for encoding in one or more scans digital image data in the same or another band of the digital image at a second resolution, said second resolution being lower than said first resolution; and means for managing a storage such that if said storage is full or substantially full, at least part of an encoded scan that is least perceptually significant relative to other encoded scans in said storage is discarded.

26. The apparatus according to claim 25, wherein said digital image data contains edge data in said band of the digital image at said first resolution.

27. The apparatus according to claim 25, wherein said digital image data contains significant edge data in said band of the digital image at said first resolution, said edge data being significant if a gradient in at least one direction satisfies a predetermined condition.

28. The apparatus according to claim 26 or 27, wherein said digital image data contains other data in the same or another band of the digital image at said second resolution. 729890 -43-

29. The apparatus according to claim 28, wherein said other data comprises non-edge data. O 30. The apparatus according to claim 28 or 29, wherein said other data comprises insignificant edge data, said edge data being insignificant if a gradient in at least 00 5 one direction fails to satisfy a predetermined condition.

31. The apparatus according to claim 25, wherein said managing means N comprises means for storing at least one encoded scan in said storage.

32. The apparatus according to claim 31, wherein said managing means comprises means for overwriting in said storage at least part of said encoded scan that is least perceptually significant with at least part of the same or another encoded scan that is perceptually more significant.

33. The apparatus according to claim 31, wherein said managing means comprises means for discarding at least part of said encoded scan that is least perceptually significant without storing said discarded part of said encoded scan in said storage.

34. The apparatus according to claim 25, wherein said digital image data stored in scans in said storage can be decoded into a plurality of pixels. The apparatus according to claim 25, wherein said storage has a fixed memory size.

36. The apparatus according to claim 26 or 27, wherein at least two of said encoded scans represent edge data in a band of said digital image data.

37. The apparatus according to claim 28, wherein at least one encoded scan contains data from encoded edge data and encoded other data.

38. The apparatus according to claim 28, wherein encoded scans comprise data from only one of encoded edge and encoded other data.

39. The apparatus according to claim 28, wherein said encoding means for encoding one or more scans containing other data comprises means for applying transform coding to at least a portion of said one or more scans. The apparatus according to claim 39, wherein said encoding means for encoding one or more scans containing other data comprises means for coding according to the JPEG apparatus. 729890 -44-

141. The apparatus according to claim 26 or 27, wherein said encoding means for encoding one or more scans containing edge data comprises means for applying O transform coding to at least a portion of said one or more scans. 42. The apparatus according to claim 41, wherein said transform coding OO 5 comprises a Discrete Cosine Transform (DCT). 43. The apparatus according to claim 25, wherein said encoding means each N comprise means for compressing said digital image data. 44. The apparatus according to claim 28, wherein said digital image data containing edge data in a band of the digital image and digital image data containing other data in a band of the digital image are encoded using the same encoding technique. The apparatus according to claim 25, wherein said digital image data is encoded in tiles. 46. The apparatus according to claim 25, wherein said encoding means each comprise means for storing pointer information in a scan to the start of said encoded digital image. 47. The apparatus according to claim 25, wherein encoded digital image data is an encoded form of multiple image layers. 48. The apparatus according to claim 25, wherein encoded digital image data is in a mixed-raster content format. 49. The apparatus according to claim 25, further comprising a storage coupled to said managing means. An apparatus for managing storage of digital image data, said apparatus comprising: a memory of limited size; at least one encoder able to encode in one or more scans said digital image data comprising significant edge data in a band of the digital image at a first resolution and in one or more scans said digital image data comprising other data in the same or another band of the digital image at a second resolution, said second resolution being lower than said first resolution; and 729890 11"a scan output manager coupled to said memory that controls storing of encoded scans such that if said memory is full or substantially full, at least part of an encoded scan O that is least perceptually significant relative to other encoded scans in said memory is discarded. 00 5 51. The apparatus according to claim 50, wherein said memory has a smaller Cc size than the size of said digital image data. C 52. A computer program product comprising a computer readable medium Shaving a computer program recorded therein for storing digital image data, said computer N program product comprising: computer program code means for encoding in one or more scans digital image data in a band of the digital image at a first resolution; computer program code means for encoding in one or more scans digital image data in the same or another band of the digital image at a second resolution, said second' resolution being lower than said first resolution; and computer program code means for managing a storage such that if said storage is full or substantially full, at least part of an encoded scan that is least perceptually significant relative to other encoded scans in said storage is discarded. 53. The computer program product according to claim 52, wherein said digital image data contains edge data in said band of the digital image at said first resolution. 54. The computer program product according to claim 52, wherein said digital image data contains significant edge data in said band of the digital image at said first resolution, said edge data being significant if a gradient in at least one direction satisfies a predetermined condition. The computer program product according to claim 53 or 54, wherein said digital image data contains other data in the same or another band of the digital image at said second resolution. 56. The computer program product according to claim 55, wherein said other data comprises non-edge data. 729890 57. The computer program product according to claim 53 or 54, wherein said other data comprises insignificant edge data, said edge data being insignificant if a O gradient in at least one direction fails to satisfy a predetermined condition. 58. The computer program product according to claim 52, wherein said 00 5 computer program code means for managing comprises computer program code means for M storing at least one encoded scan in said storage. 59. The computer program product according to claim 58, wherein said N 59. The computer program product according to claim 58, wherein said computer program code means for managing comprises computer program code means for overwriting in said storage at least part of said encoded scan that is least perceptually significant with at least part of the same or another encoded scan that is perceptually more significant. The computer program product according to claim 58, wherein said computer program code means for managing comprises computer program code means for discarding at least part of said encoded scan that is least perceptually significant without storing said discarded part of said encoded scan in said storage. 61. The computer program product according to claim 52, wherein said digital image data stored in scans in said storage can be decoded into a plurality of pixels. 62. The computer program product according to claim 52, wherein said storage has a fixed memory size. 63. The computer program product according to claim 53 or 54, wherein at least two of said encoded scans represent edge data in a band of said digital image data. 64. The computer program product according to claim 55, wherein at least one encoded scan contains data from encoded edge data and encoded other data. The computer program product according to claim 55, wherein encoded scans comprise data from only one of encoded edge and encoded other data. 66. The computer program product according to claim 55, wherein said computer program code means for encoding one or more scans containing other data comprises computer program code means for applying transform coding to at least a portion of said one or more scans. 729890 0 -47- 67. The computer program product according to claim 66, wherein said computer program code means for encoding one or more scans containing other data comprises computer program code means for coding according to the JPEG computer program product. 00 5 68. The computer program product according to claim 53 or 54, wherein said Mc, computer program code means for encoding one or more scans containing edge data N comprises computer program code means for applying transform coding to at least a Sportion of said one or more scans. 69. The computer program product according to claim 68, wherein said transform coding comprises a Discrete Cosine Transform (DCT). The computer program product according to claim 52, wherein said computer program code means for encoding each comprise computer program code means for compressing said digital image data. 71. The computer program product according to claim 55, wherein said digital image data containing edge data in a band of the digital image and digital image data containing other data in a band of the digital image are encoded using the same encoding technique. 72. The computer program product according to claim 52, wherein said digital image data is encoded in tiles. 73. The computer program product according to claim 52, wherein said computer program code means for encoding each comprise computer program code means for storing pointer information in a scan to the start of said encoded digital image. 74. The computer program product according to claim 52, wherein encoded digital image data is an encoded form of multiple image layers. 75. The computer program product according to claim 52, wherein encoded digital image data is in a mixed-raster content format. DATED this FIRST Day of August, 2005 Canon Kabushiki Kaisha Patent Attorneys for the Applicant SPRUSON FERGUSON 729890