JP2003046789A - Image coding apparatus and image decoding apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To efficiently compress a composite image formed by composing a natural image and a non-natural image. SOLUTION: A line buffer 101 separates image data 131 read by an image scanner section 130 for reading image data into a plurality of regions. A determination section 120 determines the proportion of part occupied by regions having prescribed characteristics in the image 131. A quantization matrix selecting section 121 selects a quantization matrix used for compression of the regions on the basis of this proportion. An image data coder 103 compresses the regions by the selected quantization matrix. An attribute flag coder 102 compresses information which helps a decoder side to estimate that the region is coded by which quantization matrix.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to image data encoding and decoding.

[0002]

2. Description of the Related Art Conventionally, color still image compression methods are
JPEG method using discrete cosine transform, Wavelet
A method using conversion is often used. In this type of encoding method, the entire image is uniformly encoded.

By the way, the deterioration of the image quality of natural images such as photographs is relatively inconspicuous even after encoding. On the other hand, it can be said that non-natural images such as characters and computer graphics are prominent in deterioration of image quality when encoded.

[0004]

Here, let us consider efficient encoding of a composite image in which a photographic image and a character image are composited.

Considering the deterioration characteristic of the image quality when the image is encoded depending on the type of the image, it cannot be said that the compression efficiency is good if the entire image is uniformly encoded as in the conventional case.

Therefore, it is devised to change the quantization step at the time of encoding between the character area and the photograph area.
For example, the quantization step is roughened for a photo region where image quality deterioration is relatively unnoticeable, and finely quantized for a character or computer graphics region where image quality deterioration is relatively noticeable.

However, in order to do this, it is necessary to distinguish where in the image the region of the natural image and the region of the non-natural image exist.

To specify the area for switching the quantization step, for example, a method in which the user drags the mouse so that the entire photograph area is included in the rectangular area can be considered. In this method, the entire photo area is specified at one time, and switching can be performed only in a considerably large area unit. When the area is designated in such a large unit, a designation error may occur. In addition, the operation is complicated because the user needs to specify the area every time.

Therefore, it is an object of the present invention to efficiently compress a composite image formed by combining a natural image and a non-natural image, and to improve the quality of the compressed image as compared with the prior art.

Another object of the present invention is to improve the quality of the image after compression as compared with the conventional art by making the cutout areas of the natural image and the non-natural image smaller than before.

Another object of the present invention is to reduce the complexity of the clipping operation for natural images and non-natural images.

It is another object of the present invention to provide an image processing system, an image processing method, and a computer program for realizing these, which solve the above problems.

[0013]

In order to solve the above problems, the present invention relates to an input means for inputting image data and a plurality of areas obtained by dividing the image data input from the input means. Determining means for determining the number of pixels having a predetermined property contained in each area for each area, and selecting an encoding method used for compression of the area based on the number of pixels obtained by the determining means. Compressing means, first compression means for compressing the area by the encoding method selected by the selecting means, and information useful for specifying the encoding method used by the first compressing means And a second compression means for performing the same.

The predetermined property mentioned here is, for example, a property found in a natural image or a non-natural image.

Note that a natural image is image data such as a photograph in a narrow sense, but in a broad sense, a deterioration in image quality is relatively unnoticeable even if the compression rate is increased. On the other hand, the non-natural image is an artificial image represented by characters, fine lines, printed dots, or computer graphics in a narrow sense, but in a broad sense, the compression ratio is increased. And an image in which deterioration of image quality is relatively conspicuous. Therefore, even computer graphics may be classified as a natural image of the present application if the deterioration of image quality when the compression rate is increased is not noticeable. Further, even a photographic image may be classified as a non-natural image of the present application if the deterioration of the image quality when the compression rate is increased is noticeable.

The above-mentioned judging means may have any structure as long as it can judge the state of the area. For example, when the determining unit compresses each of a plurality of regions forming the image data input from the input unit at a predetermined compression rate, the predetermined quality becomes the following.
It may be configured to determine whether the area is the area of the second type or the area of the second type that exceeds the predetermined quality when the area is compressed at the predetermined compression rate.

The term "type" as used herein means information representing the state of the area, and may be any information as long as it is useful for the selection means to select an appropriate coding method. Further, it may be information in units of regions or information in units of pixels.

Further, the selecting means may have any structure as long as it can select an appropriate coding method according to the state of the area. For example, the encoding method used for compression of the area may be selected based on the type of each area determined by the determination means.

By the way, since the encoding method of the present application is new, a new structure is required on the decoding side.

Therefore, the present invention is an image decoding apparatus for decoding image data from coded data that is coded in units of a plurality of regions that make up the image data, and stores the coded data. Storage means, and first decoding means for decoding information regarding the type of area to be decoded from the encoded data stored in the storage means,
Judgment means for judging the encoding method used on the encoding side based on the information on the type of area decoded by the first decoding means, and information on the kind of each area judged by the determination means Selection means for selecting a decoding method to be used for decoding the area based on the above, and second decoding means for decoding the area with the decoding method selected by the selection means. And

Here, the determination means may have any configuration as long as it can estimate the encoding method used on the encoding side. In that sense, the determining means can be expressed as an estimating means. The determination means may be, for example, a first type area in which a predetermined quality is obtained when the area is compressed at a predetermined compression rate, or the predetermined quality is obtained when the area is compressed at the predetermined compression rate. It may be configured to determine whether the area is a second type area that is exceeded based on information about the type of the area decoded by the first decoding unit.

If the information on the type of area is information on a pixel-by-pixel basis, statistical processing can be performed to estimate the state of the area. Further, the information about the type of area may be information that directly specifies the encoding method. For example, it is like code information that can classify the encoding methods. The decoding method may be directly specified on the encoding side.

The selecting means may select the coding method using a function f (x) representing the coding coefficient when the ratio is x. In this case, the property of the function f (x) is
When the ratio of the natural image is high, the compression rate of the encoding is increased, and when the ratio of the non-natural image is high, the compression rate of the encoding is decreased.

[0024]

BEST MODE FOR CARRYING OUT THE INVENTION [First Embodiment] The outline of the operation of the present embodiment is as follows. The image data input from the scanner is divided into a plurality of tiles (areas) by the line buffer. Next, it is calculated how much the tile having the character attribute is included in the image data. Based on this ratio, the first encoding method used for the tile having the character attribute and the second encoding method used for the tile having the photo attribute are selected. The image data encoding unit sequentially compresses tiles using the selected encoding method. On the other hand, the information indicating what coding method is used on the coding side is also sequentially compressed for each tile. The structure of the encoded image data is
Information indicating the encoding method is arranged in the header part, and compressed data obtained by compressing tiles is arranged behind it. Hereinafter, the present embodiment will be described with reference to the drawings.

[Image Input / Output Device] FIG. 11 is a side sectional view of a multifunction digital image input / output device (hereinafter referred to as an image processing device) according to the first embodiment.

The main components of the image processing apparatus 800 are
An image reader unit 801 that reads a document image and a printer unit 802 that reproduces the image data read by the image reader unit 801.

The image reader unit 801 is 40 in this case.
This is a part that reads a document at a resolution of 0 dpi (dots / inch) and performs digital signal processing. The printer unit 802 is
This is a part that outputs an image corresponding to the original image read by the image reader unit 801 to a designated sheet in full color print.

In the image reader unit 801, the original document 80 placed on the original platen glass (hereinafter, platen) 803 is placed.
4 is illuminated by the lamp 805. The light reflected from the document is guided to mirrors 806, 807, 808, and lens 809
Collected by 3 line sensor (CCD) 8
Form an image on 10. The full-color information red (R), green (G), and blue (B) components obtained by being converted into an electric signal here are sent to the signal processing unit 811. The speed of the carriage fixing the lamp 805 and the mirror 806 is v, and the speed of the mirrors 807 and 808 is 1 / 2v.
Moves in the direction perpendicular to the electrical scanning (main scanning) direction of the line sensor to scan (sub-scan) the entire surface of the document.

In the signal processing unit 811, the read image signal is electrically processed and subjected to the image processing described later in FIG. 1, and then magenta (M), cyan (C),
It is decomposed into yellow (Y) and black (Bk) components.

The image signals of M, C, Y and Bk are sent to the laser driver 812. The laser driver 812 is
The semiconductor laser 813 is modulated and driven according to the sent image signal. The laser light is polygon mirror 814, f-
The photosensitive drum 81 via the θ lens 815 and the mirror 816.
Scan over 7. A rotary developing device 818 includes a magenta developing unit 819, a cyan developing unit 820, and a yellow developing unit 8.
21 and a black developing unit 822, four developing units alternately contact the photosensitive drum 817, and develop the latent image developed on the photosensitive drum with toner. Reference numeral 823 denotes a transfer drum, which winds the paper supplied from the paper cassette 824 or 825 around the transfer drum 823, and transfers the image developed on the photosensitive drum to the paper.

In this way, after the four colors M, C, Y and Bk are sequentially transferred, the paper passes through the fixing unit 826 and the toner is fixed on the paper and then discharged.

FIG. 1 shows image input, storage, and
It is a block diagram of a digital image device having an output means.
FIG. 6 shows a flowchart of this embodiment.

In FIG. 1, 100 is a selector for switching the image input system, 101 is a block line buffer for compression, 102 is an attribute flag data encoder, 103 is a color image encoder, 104 is a compression memory, and 105 is a large capacity. External storage device, 106 is a compression memory, 107 is an attribute flag data decoder, 108 is a color image decoder, 109
Is a block buffer for expansion. Reference numeral 110 is an I / F with the printer unit 802 described above.

When an image data input instruction is given from the control means (not shown) (S100), the image input means executes the input of image data (S101). As the image input means, for example, the image scanner 130 corresponding to the image reader unit 801 or a page description language (PD
L) rendering unit 140 and the like. Selector 10
0 switches the image input means according to the purpose.

The image scanner 130 includes a known image area separation processing unit (not shown). The image area separation processing unit determines whether or not the read pixel has a photograph attribute which is a character attribute. For example, if there is a pixel forming a black character portion in the original image, the attribute flag of this pixel is output as "1", and if it is not a black character portion, it is output as "0" (S102). In this way, the selector 100
Receives the image data 131 and the attribute flag 132.

On the other hand, in the PDL rendering unit 140, P
The DL command is interpreted to generate a print image. At this time, with reference to the information indicating the command type, the black character pixel is identified in the same manner as in the scanner unit, and the attribute flag data is generated. In this case, the image data 141 and the attribute flag data 142 are input to the selector 100.

Here, the attribute flag data 132 and 142 are not limited to those indicating black characters. For example, multi-bit information indicating various attributes such as a color character area, a halftone dot area, and a vector graphics area can be used as the attribute flag data.

The line buffer 101 of the compressed block is
Divide the image data into multiple tiles. After that, the image data for each tile is input to the image data encoder 103. The attribute flag data for each pixel is determined by the determination unit 1
20 and the attribute flag encoder 102.

For convenience of explanation, the tile size is M pixels × N pixels (M and N are natural numbers). Encoding of color information using discrete cosine transform encoding (JPEG) and encoding of attribute flag data using run length encoding are performed for each tile, that is, for each M × N pixel.

However, when discrete cosine transform coding is applied as the coding method, M and N must be multiples of the window size. In this embodiment,
The window size for compression is 8 × 8 pixels, and M =
If N = 32, the tile consisting of 32 × 32 pixels is further divided into 16 8 × 8 pixels, and JPEG compression is performed in units of 8 × 8 pixels. After that, M = N = 32
Of course, the present invention is not limited to that value.

Here, discrete cosine transform coding and run-length coding are shown as examples of the compression method.
The compression method is not limited to this, and another compression method may be used.

The code attribute determination unit 120 determines whether the attribute flag data of 32.times.32 pixels input from the line buffer 101 has a character attribute (S10).
3). The attribute flag data is attached to each pixel. In this embodiment, the attribute flag data for each tile is further determined. This is because, in the present embodiment, encoding is performed in tile units. Therefore, the number of pixels included in one tile that are determined to have the character attribute is counted (S104, S1).
05). If the number i of pixels with a character attribute included in one tile exceeds a predetermined threshold value j, the tile is determined as a tile with a character attribute (S105). This is repeated for the entire image and the number of tiles having the character attribute is counted (S107, S108). Then, the ratio of tiles having the character attribute included in one image is obtained (S109). The quantization matrix selection unit 121 selects a quantization matrix to be encoded based on this ratio, and sends a quantization matrix selection signal to the image data encoder 103.
A specific method of selecting the quantization coefficient (quantization matrix) will be described later. In the encoding unit 103, 16 8x included in one tile are selected according to the selected quantization matrix.
Quantize the 8-pixel window using the well-known DCT transform. The attribute flag encoder 102 encodes and compresses attribute flag data of pixels included in one tile.

In this way, the encoders 102 and 103
The encoded data generated in step 1 is stored in the hard disk 105 as compressed image data via the compression memory 104.

When the stored image is output from the printer section 110, the compressed image data and the compression attribute flag data stored in the hard disk 105 are read out, decoded and output in the following procedure.

First, the attribute flag decoder 107 reads and decodes M × N pixel data of the compressed and stored attribute flag data. The decryption attribute determination unit 122 performs attribute determination processing based on the decryption result of the attribute flag data. The determination process at this time is from S103 to S10 of FIG.
Since the processing is the same as that of 9, the description is omitted. Based on this determination result, the decoding quantization matrix selecting unit 123
A decoding coefficient (inverse quantization matrix in this embodiment) used for decoding is selected. The color image decoder 108 decodes the image data for each tile using the decoding coefficient selected in this way, and outputs the result to the line buffer.

The decoding attribute determining unit 122 may perform the same processing as the code attribute determining unit 120. This is because the attribute flag data is compressed by a lossless compression method such as run-length coding that does not deteriorate the data, and the determination results at the time of encoding and at the time of decoding the same tile are completely the same. Therefore, even if each tile is quantized with a different quantized coefficient, an appropriate inverse quantized coefficient is set at the time of decoding, so that correct decoded image data can be obtained.

By the way, when the attribute flag data for each tile is encoded and sent to the decoding side in place of the attribute flag data for each pixel on the encoding side, the determination unit 122 is used.
Is unnecessary. This is because the attribute flag data for each tile is the data itself that the determination unit 122 seeks. The detailed description at this time will be given in the second embodiment.

FIG. 2 is a block diagram of the color image encoder 103 and the color image decoder 108.

In the figure, the image signal 200 is input from the image input means, and in the case of a color signal,
The signal has three colors of red (R), green (G), and blue (B) and 256 gradations. The color converter 201 converts the RGB signal into a luminance color difference signal (YCbCr). Discrete cosine transform (DC
T) 202 performs spatial frequency conversion (DCT conversion) on each of the luminance and color difference signals in units of 8 × 8 pixels. Quantizer 2
03 reduces the data amount by quantizing the DCT coefficient using the set quantization matrix. A variable length coder (VLC) unit 204 further reduces the data by Huffman coding the quantized value.

The above is the color image encoder section, and the compressed image data is stored in the compression memory 205 or the mass storage device. The compression memory 205 corresponds to the compression memories 104 and 106 and the HDD 105 shown in FIG.

The stored data is decoded by the following procedure. First, the variable length decoder (VLD) 206 performs Huffman decoding. The inverse quantizer 207 restores the DCT coefficient value by the set inverse quantization matrix. IDCT20
In step 8, DCT inverse transformation is performed to restore the luminance color difference signal. The color converter 209 converts the luminance color difference signal back into an RGB signal. The color image signal 210 is output to the outside as a result of the above compression and decoding processing.

FIG. 3 is a block diagram showing an attribute flag data encoder 102 which is a run length encoder for attribute flag data.

In the figure, the determination unit 300 determines whether the value of the previous pixel and the value of the current pixel of the input attribute flag data are the same. If they are the same, the RL code generation unit 301 is used. If they are different, the LT code is generated. Switch to send data to the generation unit 302. The RL code generation unit 301 counts the number of times the same as the previous pixel data until different data comes out, and finally outputs the repeated data. The LT code generation unit 302 counts the number of cases where the data is different from the previous pixel, and outputs the code word corresponding to the count number and the minimum number of bits constituting the actual data for the count number.
The synthesizing unit 303 synthesizes the output data of the RL code generating unit 301 and the output data of the LT code generating unit 302, and outputs it as a code 304.

Here, the code attribute determining unit 120, the decoding attribute determining unit 122, the code quantization matrix selecting unit 121, and the decoding quantization matrix selecting unit 12 based on the attribute flag data.
A method of switching the quantization matrix of No. 3 will be described.

FIG. 4A shows an example of a document image in which a photograph area and a character area are mixed, and shows the structure of the entire page. FIG. 4b shows the attribute flag data generated for this document image, and here shows that only the character area is extracted.

FIG. 4c is a diagram showing a state where the entire original image is divided into tiles of 32 × 32 pixels (in this embodiment). A quantization matrix can be set for each 32 × 32 pixel area.

FIG. 4d shows an example in which the tile is divided into tiles having a character attribute and tiles having no character attribute by referring to the attribute flag data of b. The tiles shaded in the figure are identified as tiles that do not include characters.

Here, it is determined whether or not a character is included in the tile. However, it is possible to determine that a character is included if a character attribute flag is included in even one pixel in a tile of 32 × 32 pixels. Alternatively, the number of pixels of the attribute flag data having a character attribute is counted in the tile, and it can be determined that the tile includes a character only when the number of pixels exceeds a predetermined threshold value.

Once identified as shown in FIG. 4d, the quantization matrix selectors 121 and 123 count the number of tiles containing characters and the number of tiles not containing characters.
Calculate the ratio of tiles that include characters and tiles that do not.

In the example of FIG. 4D, the tiles containing characters are 42 tiles out of 130 tiles, which is 32%. On the contrary, the tiles that do not include characters are 68%. In accordance with this ratio, the quantization matrix adapted to the tile containing the character and the quantization matrix adapted to the tile not containing the character are shown in FIG.
The image data is encoded and decoded by selecting from the quantization matrix stored in. In other words, the quantization matrix is automatically changed each time in accordance with the input image data and further in accordance with the proportion occupied by each of the character area and the photograph area in the image to perform encoding and decoding.

FIG. 5 is a diagram showing the ratio of the character area and the non-character area (photo area in this example). The description of a, b, and c in the figure occupies the character area and photo area of a certain input image. The percentage is shown. For example, the input image shown at the point a indicates that the input image has an image ratio of 10% in the character area and 90% in the photograph area. That is, in FIG. 5, it is shown that the percentage occupied in the photograph region in the entire image is large, and that as the percentage occupied by the photograph region decreases toward the right, the percentage occupied by the character region also increases. There is. The ratios a, b, and c are output from the determination unit 120 and input to the quantization matrix selection unit 121. Note that the quantization matrix selection unit 121 can also be expressed as a function f (x) that returns a code that specifies the quantization matrix when the ratio (x) is input. Further, instead of the code, a function that directly outputs the quantization matrix may be used. In these cases, the quantization matrix is
May be substituted and calculated. Further, a plurality of quantization matrices may be stored in advance in a memory (not shown) or the like, an appropriate quantization matrix corresponding to the ratio x may be read out, and passed to the encoding unit. The function here may be a mathematical function or a function in a programming language.

Now, comparing the photo area with the character area,
Since the image quality deterioration due to compression is not noticeable, it can be taken roughly in the quantization step for the photo area,
The compression rate is high, that is, the storage capacity can be greatly reduced with respect to the original image. To simplify the discussion, let us consider a fixed quantization matrix for the photo area. An example of this quantization matrix for photographs is 8x8 DC.
The quantization matrix for the T coefficient is shown in T1.

[0063]

[Equation 1]

In the case of point a in FIG. 5, it is shown that the image is a high percentage of the photographic area. In such an image, the quantization step of the character area can be finely taken. Because the text area is relatively small,
This is because even if the compression rate is low, the compression rate in the photographic region greatly contributes to the entire image, and the compression rate of the image becomes high. Therefore, the quantization matrix for the character area of the image a is the quantization matrix shown by T2.

[0065]

[Equation 2]

The DCT coefficient is normally DC at the upper left of the matrix.
The quantization step is for the component, and the quantization step for the high-frequency component is expressed as it goes to the right or the bottom. The smaller the value is, the smaller the quantization step is, that is, the information of the original image is stored. T
2 is set so that the value becomes smaller as it goes to the right or lower than T1. That is, although the compression rate of the character quantization matrix is lower than that of the picture quantization matrix, deterioration of the character image due to compression can be reduced by storing the information of the high frequency component.

Next, FIG. 5B shows that the images have the same ratio of the character / photo area. Compared to an image in which the ratio is a, the contribution of the photographic region to the image compression is small because the area of the photographic region in the entire image is small. Therefore, the quantization matrix for the character region of the image whose ratio is b is as shown by T3.

[0068]

[Equation 3] Compared with T2, the quantization matrix having a high compression rate is obtained by slightly dropping the information of the high frequency component.

Similarly, when the ratio is c, the ratio of the character area is high. In other words, it has the opposite relationship to the character / photo area at point a. Here, the area of the photographic area in the entire image is very small, and therefore no contribution to the compression rate can be expected. Therefore, the quantization matrix for the character area of the image c is as shown by T4.

[0070]

[Equation 4] Compared to T3, the quantization matrix having a high compression rate is obtained by slightly dropping the information of the high frequency component. Thereby, the compression rate of the entire image can be set to a desired compression rate.

The quantization matrix of T1, T2, T3 and T4 shown here is only an example. Further, at T3 and T4, the high frequency component is dropped in order to increase the compression rate, but it can be arbitrarily changed in consideration of the balance between the high frequency component and the low frequency component.

Although four quantization matrices T1, T2, T3 and T4 have been described here, more quantization matrices may be prepared. Further, although the description has been made by fixing the quantization matrix for the photo area, it is needless to say that the quantization matrix for the photo area can be selected according to the ratio of the character / photo area.

When decoding the compressed data, first, the compressed attribute flag data is decoded, and similarly to the above, the number of pixels of the character attribute in the tile is counted to determine whether or not it is a character area. The ratio is calculated, and based on the determination result and the calculation result, the inverse quantization matrix corresponding to T1 and T2, T3, and T4 is set and decoding is performed.

In the above, the quantization matrices T1, T
2, T3, T4 and their corresponding inverse quantization matrices are the quantization unit 203 and the appropriate quantization unit 2 shown in FIG.
It is stored in advance in 07 and is selectively used according to the attribute of the image data and the ratio of that attribute in the image, and is switched in tile units.

[Second Embodiment] In the first embodiment,
The configuration is such that the setting of the quantization matrix and the setting of the inverse quantization matrix are switched while referring to the attribute flag data for each pixel. That is, in the first embodiment, when decoding the compressed image data, it was necessary to first decode the attribute flag data data for each pixel and statistically process this over one tile. In other words,
It was necessary to provide both the encoding side and the decoding side with count blocks for statistical processing.

Therefore, in the second embodiment, the quantizer T
The code information tn indicating which of the quantization matrices 1, T2, T3, T4 is used is stored in the header portion of each tile of the compressed image data.

FIG. 7 shows the configuration of the image processing apparatus of this embodiment. Until the quantization matrix is selected, the same processing as in the first embodiment is performed. Quantization matrix selection unit 1
21 is a quantization matrix Tn (n = 1, 2, 3, ...)
Is selected, the code information tn (n = 1, 2, 3, ...) Of the selected quantization matrix Tn is output to the compression memory. The code information tn is identification information of the quantization matrix Tn.

FIG. 9 is a conceptual diagram showing a data structure when the compressed image data and the attribute data are stored in the image storage units (104 to 106). Header information of the first tile, DCT-coded image information, and run-length-coded attribute flag information are stored in order from the beginning of the data, and the second tile, the third tile, ... Then, the data is written, and information up to the last tile included in one page is written. The run-length encoded attribute flag information is arranged at the position of the footer, but may be arranged at the position of the header.
The header information of each tile records the data size of the compressed data, the tile number, etc. At the same time, did T1 be used as the quantization matrix of the image data?
Code information tn indicating whether 3 or T4 is used is also recorded.
By the way, the attribute data of FIG. 9 is not necessary for the second embodiment.

When decoding the encoded and recorded data, T1, T2, T3 are referred to by referring to this header information.
It means that the information of T4 is read and the inverse quantization matrix suitable for each is selected and the inverse quantization is executed.

FIG. 8 shows the quantization matrix selection unit 1 on the decoding side.
23 shows an example of the configuration. First, when the decoding process is activated on the decoding side, the code information extraction unit 1231 reads the code information tn from the header stored in the compression memory,
It is output to the quantization matrix reading unit 1232. The quantization matrix reading unit 1232 reads the inverse quantization matrix Tn−1 corresponding to the code information tn from the storage device 1233 and outputs it to the image data decoding unit 108.

A code other than the quantization matrix code may be used. For example, the number or ratio of natural image pixels included in one tile, the number or ratio of non-natural image pixels included in one tile, or the ratio of natural image pixels to non-natural image pixels included in one tile. It may be. Further, it may be attribute information for each area.

In this embodiment, the encoded tile data and the header are alternately arranged, but other data structures may be used. For example, the headers of all tiles may be arranged in one place. Also, 2
Two corresponding headers may be placed together in front of one compressed tile. In short, any data structure may be used as long as it can be estimated how each tile is encoded by the encoding method.

In the present embodiment, when decoding the compressed image data, it is possible to switch the inverse quantization matrix only by referring to this header part, so that the attribute flag data for each pixel at the time of decoding is decoded. There is an advantage that you do not need to refer to.

Therefore, the attribute flag data encoder 102 is unnecessary on the encoding side. On the other hand, on the decryption side,
The attribute flag data decoder 107 and the attribute flag data determination unit 122 are unnecessary. Therefore, the present invention can be implemented with a simpler configuration than that of the first embodiment.

[Third Embodiment] In the third embodiment,
A configuration when the image processing apparatus according to the present application is connected to a predetermined network in FIG. 1 described in the first embodiment will be described with reference to FIG. 10.

In FIG. 10, reference numeral 701 is a LAN interface, which is an interface section that enables connection with the network 702.

The compressed image data stored in the external storage device 105 for storing the compressed image is sent to the host computer 703, the image server 704, the image input / output device 705, etc. connected to the network 702 via the LAN interface. Network transmission is possible.

Further, it is also possible to receive compressed image data from the host computer 703, the image server 704, the image input / output device 705, etc. connected to the network 702 via the network and store it in the external storage device 105.

The LAN interface can be connected to a portion other than the external storage device 105 such as the line buffers 101 and 109 or the compression memories 104 and 106. However, in consideration of the load on the network, it is considered that it is desirable to connect to the external storage device 105 because the transmission and reception of the compressed image is more efficient.

Although not shown here, for example, a JPEG encoding unit and a JPEG decoding unit are provided for performing encoding and decoding that do not require attribute flag data, and a device that does not use attribute flag data is used. It may be configured to be used during transmission and reception.

Further, the image transfer may be performed by connecting to an image transmitting / receiving device such as a facsimile using a telephone line, or by using wireless or the like, another image input / output device, an image server, a host computer. It may be configured to be able to transmit and receive an image by connecting to the above.

[0092]

Other Embodiments In the above embodiment, the integrated image processing apparatus as shown in FIG. 11 has been described. However, the present invention is applicable to a plurality of devices (for example, host computer, interface device, reader, printer, etc.). It may be applied to a system composed of a single device or a device composed of a single device (for example, a copying machine or a facsimile device).

Further, the encoding method and the decoding method of the present invention can be distributed as encoding software or decoding software. Therefore, an object of the present invention is to supply a storage medium (or recording medium) recording a program code of software for realizing the functions of the above-described embodiments to a system or apparatus, and to supply a computer (or CPU or CPU of the system or apparatus). It is needless to say that it is also achieved by the MPU reading and executing the program code stored in the storage medium. In this case, the program code itself read from the storage medium realizes the functions of the above-described embodiments, and the storage medium storing the program code constitutes the present invention. Further, by executing the program code read by the computer, not only the functions of the above-described embodiment are realized, but also an operating system (OS) running on the computer is executed based on the instruction of the program code. It goes without saying that a case where some or all of the actual processing is performed and the functions of the above-described embodiments are realized by the processing is also included.

Further, after the program code read from the storage medium is written in the memory provided in the function expansion card inserted into the computer or the function expansion unit connected to the computer, based on the instruction of the program code, It goes without saying that a case where the CPU included in the function expansion card or the function expansion unit performs a part or all of the actual processing and the processing realizes the functions of the above-described embodiments is also included.

When the present invention is applied to the above storage medium, the storage medium stores the program code corresponding to the flowchart shown in FIG. 6 described above.

[0096]

As described above, according to the present invention, a synthesized image obtained by synthesizing a natural image and a non-natural image can be efficiently compressed, and the quality of the compressed image can be improved more than before.

Further, in the present invention, the quality of the image after compression can be improved as compared with the conventional art by making the cutout areas of the natural image and the non-natural image smaller than before.

Further, it is possible to reduce the complexity of the clipping operation for the natural image and the non-natural image.

[Brief description of drawings]

FIG. 1 is a block diagram of an image processing apparatus according to a first embodiment.

FIG. 2 is a diagram illustrating an image encoding unit using DCT.

FIG. 3 is a diagram illustrating a run length attribute flag data encoding unit.

FIG. 4a is a diagram illustrating switching of a quantization matrix for each tile.

FIG. 4b is a diagram illustrating switching of a quantization matrix for each tile.

FIG. 4c is a diagram illustrating switching of a quantization matrix for each tile.

FIG. 4d is a diagram illustrating switching of a quantization matrix for each tile.

FIG. 5 is a graph showing the ratio of character attributes and photo attributes.

FIG. 6 is a flowchart of attribute determination in the first embodiment.

FIG. 7 is a block diagram of an image processing apparatus according to a second embodiment.

FIG. 8 is a block diagram of an inverse quantization matrix selection unit according to the second embodiment.

FIG. 9 is a diagram illustrating a data structure according to the second embodiment.

FIG. 10 is a diagram showing a network system configuration according to a third embodiment.

FIG. 11 is a side sectional view of the image processing apparatus.

Continued front page    F term (reference) 5C059 KK01 KK37 MA00 MA23 MA45                       MC14 MC38 ME01 ME05 PP01                       PP11 PP12 PP15 PP16 PP19                       RC14 RC22 SS28 TA16 TA47                       TB06 TC24 UA02 UA05 UA38                 5C077 LL19 MP06 PP27 PP28 PP61                       PP65 PP68 PQ08 PQ17 RR11                       RR21 TT06                 5C078 AA04 BA21 BA22 BA57 CA02                       DA00 DA01 DA02 DA05 DA06                       EA01                 5J064 AA02 BA09 BA16 BC16 BC25                       BC27 BD04

Claims (87)

[Claims] 1. An input unit for inputting image data, and a plurality of regions constituting the image data input from the input unit, each of which has a predetermined quality when the region is compressed at a predetermined compression rate. A determination unit that determines whether the region is a first type region or a second type region that exceeds the predetermined quality when the region is compressed at the predetermined compression rate; Selecting means for selecting an encoding method to be used for compressing the area based on the type of each area; first compressing means for compressing the area by the encoding method selected by the selecting means; An image coding apparatus, comprising: a second compression unit that compresses information regarding the type of area. 2. The area of the first type is an area for a non-natural image, and the area of the second type is an area of a natural image. Image coding device. 3. The non-natural image is an artificial image represented by characters, fine lines, printed dots, or computer graphics, and the natural image is a region of a photographic image. The image coding apparatus according to claim 2. 4. The region is a coding unit at the time of coding, which is composed of a plurality of pixels, and the determination unit is a pixel belonging to the first type among a plurality of pixels forming the region. It is determined whether or not the pixel belongs to the second type, statistical processing is performed for each area, and the area is classified as the first type based on the statistical amount obtained by the statistical processing. The image coding apparatus according to claim 2, wherein it is determined whether the area is a belonging area or an area belonging to the second type. 5. The information regarding the type of each area is attribute information for each area indicating which of the first and second types the area belongs to.
The image encoding device according to 1. 6. The information regarding the type of each area is attribute information for each pixel indicating which of the first and second types each pixel forming the area belongs to. The image encoding device according to claim 4. 7. The image coding apparatus according to claim 4, wherein the information on the type of each area is code information representing the coding method selected by the selecting means. 8. The statistical process is a process of calculating the number of pixels belonging to the first type among a plurality of pixels forming the region, and the determining means belongs to the first type. The image coding apparatus according to claim 4, wherein when the number of pixels exceeds a predetermined threshold value, the area is determined to be a first type area. 9. The image coding apparatus according to claim 1, wherein the coding method is a coding coefficient when compressing image data. 10. The image coding apparatus according to claim 9, wherein the coding coefficient is a quantization matrix when the image data is compressed. 11. A first obtained by the first compressing means.
The image coding apparatus according to claim 1, further comprising: a forming unit that forms a data structure including at least the compressed data of 1. and the second compressed data obtained by the second compressing unit. 12. The forming means includes the second compressed data obtained by the second compressing means in a header portion of the first compressed data obtained by the first compressing means. The image coding apparatus according to claim 11. 13. The input means includes a PDL rendering unit that interprets a PDL command described in a page description language to generate a print image, and the determination unit includes the PDL command interpreted by the PDL rendering unit. 3. It is determined whether the area is a non-natural image area based on the above.
The image encoding device according to 1. 14. The determination unit according to claim 13, wherein the determination unit determines, based on the type of the PDL command, an area including character pixels, vector pixels, or graphic pixels as the non-natural image area. Image coding device. 15. The first compression means compresses the area using lossy compression or lossless compression, and the second compression means compresses information regarding the type of area using lossless compression. The image coding apparatus according to claim 1, wherein: 16. The image coding apparatus according to claim 11, further comprising a transmitting unit for transmitting the data structure formed by the forming unit. 17. The image coding apparatus according to claim 16, wherein the transmitting means is an interface for connecting to a wireless line or a wired line. 18. The image coding apparatus according to claim 17, further comprising a transmission control unit that controls transmission of the first compressed data according to a transmission destination of the transmission unit. 19. A destination determination means for determining whether the transmission destination of the transmission means is compatible with the encoding method of the image processing apparatus, and a correspondence situation of the transmission destination is affirmed by the destination determination means. 17. The image encoding apparatus according to claim 16, further comprising: an instruction unit that instructs the first and second compression units to execute compression processing only when the above-mentioned case is performed. 20. A partner determination means for determining whether the transmission destination of the transmission means is compatible with the encoding method of the image processing apparatus, and a correspondence status of the transmission destination is denied by the destination determination means. The image coding apparatus according to claim 16, further comprising: a third compression unit that compresses the image data by a coding method that allows the other party to decode the image data. 21. An image decoding apparatus for decoding the image data from the encoded data encoded in units of a plurality of regions forming the image data, the storage device storing the encoded data. A first decoding means for decoding information on the type of area to be decoded from the encoded data stored in the storage means, and information on the type of area decoded by the first decoding means Based on the above, if the area is a first type area having a predetermined quality when the area is compressed at a predetermined compression rate, or if the area is compressed at the predetermined compression rate, the area exceeds the predetermined quality Determination means for determining whether the area is of two types, selection means for selecting a decoding method to be used for decoding the area based on information about the type of each area determined by the determination means, A second decoding unit that decodes the area by the decoding method selected by the selecting unit, and the image decoding apparatus. 22. The area of the first type is an area for a non-natural image, and the area of the second type is an area of a natural image. Image decoding device. 23. The non-natural image is an artificial image represented by characters, fine lines, printed dots, or computer graphics, and the natural image is a region of a photographic image. The image decoding device according to claim 22. 24. The determination means determines, for a plurality of pixels forming the region, whether the pixels belong to the first type or the pixels belonging to the second type. Statistical processing is performed for each area, and it is determined whether the area belongs to the first type or the second type based on a statistical amount obtained by the statistical processing. 3. The method according to claim 2, wherein
2. The image decoding device according to 2. 25. The information about the type of each area is attribute information for each area indicating which of the first and second types the area belongs to. The image decoding device according to claim 24. 26. The information regarding the type of each area is attribute information for each pixel indicating which of the first and second types each pixel forming the area belongs to. The image decoding device according to claim 24. 27. The image decoding apparatus according to claim 21, wherein the information on the type of each area is code information representing a coding method of the area. 28. The information regarding the type of each area is P
The image decoding device according to claim 21, wherein the image decoding device is information regarding a DL command. 29. The statistical process is a process of calculating the number of pixels belonging to the first type out of a plurality of pixels forming the region, and the determining means belongs to the first type. The image decoding apparatus according to claim 24, wherein when the number of pixels exceeds a predetermined threshold value, the area is determined to be a first type area. 30. The selecting unit according to claim 21, wherein the selecting unit specifies a coding coefficient used when compressing image data, and selects a decoding coefficient corresponding to the coding coefficient. Image decoding device. 31. The image decoding apparatus according to claim 30, wherein the coding coefficient is a quantization matrix when the image data is compressed. 32. The encoded data has a data structure including at least first compressed data obtained by compressing the area and second compressed data obtained by compressing information regarding a type of the area. The image decoding device according to claim 21, characterized in that. 33. The image decoding apparatus according to claim 32, wherein the encoded data includes the second compressed data as a header of the first compressed data. 34. The first decoding means decodes information on the type of area by using encoding corresponding to lossy compression or encoding corresponding to lossless compression, and the second decoding means is lossless. The image decoding apparatus according to claim 21, wherein the area is decoded by using encoding corresponding to compression. 35. The image decoding apparatus according to claim 21, further comprising receiving means for receiving the data structure. 36. The image decoding apparatus according to claim 35, wherein said receiving means is an interface for connecting to a wireless line or a wired line. 37. For each of an input step of inputting image data and a plurality of areas forming the image data input in the input step, when the area is compressed at a predetermined compression rate, a predetermined quality becomes as follows. A determination step of determining whether the area is a first type area or a second type area that exceeds the predetermined quality when the area is compressed at the predetermined compression rate; A selection step of selecting an encoding method used for compression of the area based on the type of each area, a first compression step of compressing the area with the encoding method selected in the selection step, A second compressing information about the type of compressed area
An image encoding method comprising the following compression step :. 38. The area according to claim 37, wherein the first type area is an area for a non-natural image, and the second type area is an area for a natural image. Image coding method. 39. The non-natural image is an artificial image represented by characters, fine lines, printed dots, or computer graphics, and the natural image is an area of a photographic image. 39. The image coding method according to claim 38. 40. The region is a coding unit at the time of coding composed of a plurality of pixels, and in the determining step, among the plurality of pixels forming the region, a pixel belonging to the first type is selected. It is determined whether or not the pixel belongs to the second type, statistical processing is performed for each area, and the area is classified as the first type based on the statistical amount obtained by the statistical processing. 39. The image encoding method according to claim 38, wherein it is determined whether the area belongs or the area belongs to the second type. 41. The information regarding the type of each area is attribute information for each area indicating which of the first and second types the area belongs to. Image coding method. 42. The information regarding the type of each area is attribute information for each pixel indicating which of the first and second types each pixel forming the area belongs to. The image coding method according to claim 40. 43. The image coding method according to claim 40, wherein the information regarding the type of each area is code information representing the coding method selected in the selecting step. 44. The statistical process is a process of calculating the number of pixels belonging to the first type out of a plurality of pixels forming the region, and the determining step belongs to the first type. If the number of pixels exceeds a predetermined threshold, the area is first
The image encoding method according to claim 40, wherein the image encoding method determines that the area is of the type. 45. The image coding method according to claim 37, wherein the coding method is a coding coefficient when compressing image data. 46. The image coding method according to claim 45, wherein the coding coefficient is a quantization matrix when the image data is compressed. 47. A forming step of forming a data structure including at least first compressed data obtained in the first compressing step and second compressed data obtained in the second compressing step. 38. The image coding method according to claim 37. 48. In the forming step, the second compressed data obtained in the second compression step is included in the header portion of the first compressed data obtained in the first compression step. The image encoding method according to claim 47. 49. The input step includes an interpreting step for interpreting a PDL command described in a page description language, and the determining step includes unnaturalness of the area based on the PDL command interpreted in the interpreting step. 4. It is determined whether it is an image area or not.
8. The image coding method according to item 8. 50. The method according to claim 49, wherein in the determining step, an area including character pixels, vector pixels, or graphic pixels is determined to be the non-natural image area based on the type of the PDL command. Image coding method. 51. In the first compressing step, the area is compressed by using a discrete cosine transform, and in the first compressing step, information about the type of the area is compressed by using run length coding. 38. The image coding method according to claim 37. 52. The image coding method according to claim 47, further comprising a transmitting step for transmitting the data structure formed in the forming step. 53. The image coding method according to claim 52, wherein said transmitting step is carried out via an interface for connecting to a wireless line or a wired line. 54. The image encoding method according to claim 53, further comprising a transmission control step of controlling transmission of the first compressed data according to a transmission destination. 55. A destination determination step of determining whether or not the destination of transmission corresponds to the encoding method of the image processing method, and a case where the correspondence status of the destination of transmission is affirmed in the destination determination step The image encoding method according to claim 53, further comprising: an instruction step for instructing execution of compression processing in the first and the first compression steps. 56. A destination determination step of determining whether the transmission destination is compatible with the encoding method of the image processing method; and a case where the correspondence status of the transmission destination is denied in the destination determination step. 54. The image encoding method according to claim 53, further comprising: a third compression step of compressing the image data by an encoding method that can be decoded by the other party. 57. An image decoding method for decoding the image data from the encoded data encoded in units of a plurality of regions forming the image data, the storage step storing the encoded data. A first decoding step of decoding information on the type of area to be decoded from the encoded data stored in the storing step, and information on the type of area decoded in the first decoding step Based on the above, the area is a first type area having a predetermined quality when the area is compressed at a predetermined compression rate, or when the area is compressed at the predetermined compression rate, the area exceeds the predetermined quality. A determination step of determining whether the area is of two types, and a decoding step of the area based on information about the type of each area determined in the determination step And a second decoding step of decoding the region by the decoding method selected in the selecting step, the image decoding method comprising: 58. The area according to claim 57, wherein the first type area is an area for a non-natural image, and the second type area is an area for a natural image. Image decoding method. 59. The non-natural image is an artificial image represented by characters, fine lines, printed dots, or computer graphics, and the natural image is an area of a photographic image. The image decoding method according to claim 58. 60. In the determining step, it is determined whether the plurality of pixels forming the area are pixels belonging to the first type or pixels belonging to the second type, and Statistical processing is performed for each area, and it is determined whether the area belongs to the first type or the second type based on a statistical amount obtained by the statistical processing. The image decoding method according to claim 58, wherein: 61. The information regarding the type of each area is attribute information for each area indicating which of the first and second types the area belongs to. The image decoding method according to claim 60. 62. The information on the type of each area is attribute information for each pixel indicating which of the first and second types each pixel forming the area belongs to. The image decoding method according to claim 60. 63. The image decoding method according to claim 60, wherein the information on the type of each area is code information representing a coding method of the area. 64. The information on the type of each area is P
The image decoding method according to claim 60, wherein the information is information regarding a DL command. 65. The statistical process is a process of calculating the number of pixels belonging to the first type among a plurality of pixels forming the region, and the determining step belongs to the first type. If the number of pixels exceeds a predetermined threshold, the area is first
The image decoding method according to claim 60, wherein the image decoding method determines that the area is of the type. 66. The coding method according to claim 57, wherein in the selecting step, a coding coefficient used in compressing image data is specified and a decoding coefficient corresponding to the coding coefficient is selected. Image decoding method. 67. The image decoding method according to claim 66, wherein the coding coefficient is a quantization matrix when the image data is compressed. 68. The encoded data has a data structure including at least first compressed data obtained by compressing the area and first compressed data obtained by compressing information regarding a type of the area. The image decoding method according to claim 57, wherein: 69. The image decoding method according to claim 68, wherein the encoded data includes the second compressed data as a header of the first compressed data. 70. In the first decoding step, run length coding is used to decode information on the type of region, and in the second decoding step, the region is decoded using discrete cosine transform. 58. The image decoding method according to claim 57. 71. An image decoding method according to claim 57, further comprising a receiving step for receiving the data structure. 72. The image decoding method according to claim 71, wherein in the receiving step, the data structure is received via an interface for connecting to a wireless line or a wired line. 73. A program for causing a computer to execute the image coding method according to any one of claims 37 to 56. 74. A program for causing a computer to execute the image decoding method according to any one of claims 57 to 72. 75. The image coding apparatus according to claim 4, wherein the information on the type of each area is the number of pixels belonging to the first type in the area. 76. The image coding apparatus according to claim 4, wherein the information on the type of each area is the number of pixels belonging to the second type in the area. 77. The information on the type of each area is a ratio of the number of pixels belonging to the first type to the number of pixels belonging to the second type in the area. The image encoding device according to claim 4. 78. The encoding method according to claim 75, wherein the selecting means selects an encoding method based on f (x) which is a function of information x on the type of each area. The image encoding device described. 79. Input means for inputting image data; determining means for dividing the image data input from the input means into a plurality of areas and determining the proportion of the areas occupied by the image data having a predetermined property. A selecting means for selecting an encoding method used for compressing the area based on a result of the determination obtained by the determining means; and a first method for compressing the area by the encoding method selected by the selecting means. And a second compression unit for compressing information useful for specifying the encoding method used by the first compression unit. 80. The judging means calculates a number of pixels affirmed by the first judging means and a first judging means for judging whether or not the pixels forming the area have a predetermined property. First calculating means for controlling the area, second determining means for determining whether or not the number of pixels calculated by the first calculating means exceeds a predetermined threshold value for each area, and the second determining means. Second calculating means for calculating the number of areas affirmed by the judging means, and the selecting means selects an encoding method based on the number of areas calculated by the second calculating means. The image coding device according to claim 79, wherein: 81. The image coding apparatus according to claim 80, wherein the predetermined property is a property peculiar to a non-natural image. 82. The determining means includes a third calculating means for calculating a ratio included in the entire image data of the area calculated by the second calculating means, and the selecting means includes the third selecting means. The image coding apparatus according to claim 81, wherein a coding method for each area is selected based on the ratio calculated by the means. 83. A first encoding method to be applied to the region affirmed by the second judging means, based on the ratio calculated by the third selecting means,
The image encoding device according to claim 82, wherein the second encoding method to be applied to the area denied by the second determination means is selected. 84. The image code according to claim 83, wherein said selecting means selects a coding method using a function f (x) representing a coding coefficient when said ratio is x. Device. 85. The property of the function f (x) is that the compression rate of encoding is increased when the ratio of the natural image is high, and the compression rate of the encoding is increased when the ratio of the non-natural image is high. The image coding apparatus according to claim 84, wherein a compression rate is reduced. 86. The first compressing means compresses the area using a discrete cosine transform, and the second compressing means compresses information about the type of the area using run length coding. The image encoding device according to claim 15, wherein the image encoding device is a device. 87. The first decoding means decodes information about a type of area using run-length coding, and the second decoding means decodes the area using discrete cosine transform. The image decoding device according to claim 34, which is characterized in that: