JPH10191338A - Image encoder and image decoder - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide encoding and decoding methods which efficiently compress without any distinction between a natural image and an artificial image. SOLUTION: An input image is converted by a DCT transforming part 20. A coefficient analyzing part 30 compares a constant that is preliminarily defined with coefficient data, 120 and a high pass coefficient masking part 50 and an inverse transforming part 60 acquire low pass image data 150 of the input data based on the comparison result. A pixel thinning part 70 receives the data 150 and creates a thinning image based on the comparison result. The thinning image and coefficient information are sent to a decoding side. A decoder decodes an image from the thinning image and the coefficient information.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image encoding / decoding apparatus, and more particularly to irreversible encoding of a multi-valued input image.

[0002]

2. Description of the Related Art Since an image generally contains a very large amount of data, it is general to compress the image by encoding when storing or transmitting the image. At this time, if image data to be subjected to image encoding is roughly classified into two, for example, it is divided into a natural image and an artificial image.

[0003] The former is a method in which an existing image is converted into digital data by some means. For example, a photograph is read by a scanner or a landscape is captured by a digital camera. The latter is an image in which a non-existent image is created as digital data by some means. For example, an original created by computer graphics or a word processor corresponds to this. Hereinafter, the terms natural image and artificial image are used in this definition.

In general, noise is superimposed on a natural image at the time of digital conversion, and the high frequency component tends to deteriorate. As a result, the obtained digital data has a large amount of information of lower bits and a large number of colors used. Also, when frequency analysis is performed, the components tend to be concentrated in the low band, and the high band is attenuated.

On the other hand, the artificial image does not have a large amount of information of lower bits except for a case where noise is intentionally added, and the colors used are likely to concentrate on a specific color. Also, since edges and fine lines appear sharply, important information is also included in high frequencies.

FIG. 3 shows two experimental examples for confirming the above facts.
0 to FIG. As a first experiment, DCT (Discrete Cosine Transform: Discrete Cosine Trans)
The values obtained by individually averaging the squares of the coefficients obtained by the processing were examined for some images. The result of adding this square root for each of the eight areas shown in FIG. 30 is shown in FIG. Since the DCT coefficient is described such that the frequency increases from the upper left to the lower right, in FIG. 30, the right side on the x-axis corresponds to the higher frequency. As can be seen from the figure, the components of the natural image decrease as the frequency becomes higher, whereas the components of the artificial image are distributed regardless of the frequency.

In the second experiment, adjacent pixel values were extracted from the image, and statistics of the result obtained by subtracting the left pixel value from the right pixel value were obtained. This is a value generally called a previous value difference or the like. FIG. 32 shows the results of the second experiment. As is clear from the figure, the previous value difference concentrates on 0 in the artificial image as compared to the natural image. This indicates that the prediction accuracy of the previous value prediction for predicting the right pixel value from the left pixel value increases.

Hereinafter, image coding methods effective for natural images and artificial images will be described as first and second conventional examples, respectively.

First, as a first conventional example, a conventional encoding technique for a natural image will be described. Since natural images originally have a very large amount of information, it is necessary to quantize the information by some method. Therefore, when considering the efficiency of quantization, natural images concentrate frequency components in the low band, so by quantizing the low band finely and the high band coarsely,
Quantization with a reduced average error can be realized. That is,
The effect on image quality can be minimized and the amount of information can be efficiently reduced.

Frequency conversion coding, which is one method of image coding, utilizes this characteristic to perform frequency conversion on an input image and quantize high-frequency component information particularly coarsely. As a typical example of frequency conversion coding, for example, JPEG (Joi) which is an international standard
nt Photographic Experts G
loop) DCT method. Hereinafter, the JPEG-DCT method will be described as a first conventional example.

Prior to the description of the first conventional example, DCT will be described. The DCT used in image coding is exactly what is called a two-dimensional DCT.
It is determined by processing one one-dimensional DCT independently. "International Standard Coding for Color Still Images-JPE
G algorithm-"(Endo, Interface, 199
According to 1.12, pp. 160-182), when an image block to be transformed is represented by x (m, n) and a transformed coefficient block is represented by y (u, v), 8 ×
The DCT transform equation and the inverse transform equation are applied as follows.

[0012]

(Equation 1) FIGS. 33 and 34 are configuration examples of a first conventional image lossy encoding device and a decoding device, respectively. The figure is the same as the above-mentioned "International standard encoding method for color still images-JPEG algorithm-" p. 163 is partially extracted from FIG.
The term has been modified. 33 and 34, 10 is an image input unit, 20 is a DCT unit, 35 is a coefficient quantization unit, 45 is a coefficient output unit, 110 is input image data,
120 is coefficient data, 170 is quantized coefficient data, 22
5 is a coefficient input unit, 240 is an inverse DCT unit, 250 is a decoded image output unit, 260 is a coefficient inverse quantization unit, 320 is decoded image data, and 330 is inverse quantized coefficient data.

[0014] Each part of FIGS. 33 and 34 will be described. The encoding device of FIG. 33 has the following configuration. The image input unit 10 inputs an image from the outside, and input image data 11
It is transmitted to the DCT unit 20 as 0. The DCT unit 20 performs a DCT process on the input image data 110, and sends the result to the coefficient quantization unit 30 as coefficient data 120. The coefficient quantization unit 30 performs a quantization process on the coefficient data 120 by a predetermined method, and outputs the result to the coefficient output unit 90 as quantized coefficient data 170. The coefficient output unit 90 outputs the quantized coefficient data 170 to the outside.

Next, the decoding apparatus shown in FIG. 34 has the following configuration. The coefficient input unit 220 inputs a coefficient from the outside, and sends the coefficient to the coefficient inverse quantization unit 260 as quantized coefficient data 170. The coefficient inverse quantization unit 260 performs inverse quantization on the quantized coefficient data 170 so as to be an inverse transform of the quantization performed by the coefficient quantization unit 30, and generates an inverse DCT unit 240 as inverse quantized coefficient data 330. Send to The inverse DCT unit 240 performs an inverse DCT process, which is an inverse transform of the DCT process performed by the DCT unit 20, on the inverse quantized coefficient data 330, and outputs the result as decoded image data 320 to the decoded image output unit 2.
Send to 50. The decoded image output unit 250 outputs the decoded image data 320 to the outside.

The above configuration is a part of the first conventional example,
Normally, the encoding device uses Huf as the quantization coefficient data 170.
Performs variable length encoding processing such as fman code and QM code,
In a general configuration, a decoding device performs decoding corresponding to the variable-length encoding process to obtain quantized coefficient data 170. Since these parts have nothing to do with the essence of the present invention, and omitting these parts does not impair the essence of the first conventional example, the explanation is omitted here.

The operation of the first conventional example based on the above configuration will be described. 35 and 36 are flowcharts showing the operation of the conventional example.

First, the encoding procedure of the first conventional example will be described with reference to FIG. In S10, an image is input from the outside in the image input unit 10, and input image data 110 is input.
Get. In S20, the DCT processing is performed in the DCT section 20 to obtain coefficient data 120. In S35, the coefficient quantization section 30 performs a quantization process on the coefficient data 120 by a predetermined method to obtain quantized coefficient data 170. In S75, the coefficient output section 90 outputs the quantized coefficient data 170 to the outside. In S80, it is determined whether or not all the processing of the input image data 110 has been completed. If not completed, the process returns to S10, and if completed, the encoding procedure ends.

Next, the decoding procedure of the first conventional example will be described with reference to FIG. In S115, the coefficient input unit 220 inputs a coefficient from the outside, and sets the quantized coefficient data 1
Get 70. In S125, the coefficient inverse quantization unit 260 performs an inverse quantization process to obtain inverse quantized coefficient data 330. In S130, the inverse DCT section 240 performs inverse DCT processing on the inverse quantized coefficient data to obtain decoded image data 320. At S140, the decoded image output unit 250 outputs the decoded image data 320 to the outside. S150
In step S115, it is determined whether or not the processing of the input quantized coefficient data 170 has been completed.
Then, if the decoding has been completed, the decoding procedure ends.

The quantization processing performed by the coefficient quantization unit 35 in the above operation will be described. As described above, in general frequency conversion coding, a high frequency component is coarsely quantized compared to a low frequency component. In the JPEG-DCT method, the following equation is used for linear quantization. Here, round is a function that returns the integer closest to the argument.

[0020]

(Equation 2) FIG. 37 shows a recommended quantization table of the JPEG-DCT method (see “International Standard Coding Method for Color Still Images-J.
PEG algorithm— "p.167" (refer to FIG. 9). The numbers in the figure represent quantization steps, and the larger the numerical value, the coarser the quantization. The quantization table is (1).
Similarly to the DCT coefficient in the equation, the frequency is described from the upper left to the lower right, so that the high-frequency component is quantized particularly coarsely.

Next, as a second conventional example, a conventional encoding technique for an artificial image will be described. In an artificial image, the same color often appears spatially and locally as shown in FIG. 32, so that predictive coding that combines pixel value prediction with surrounding pixels and prediction error coding is effective. Less than,
As a typical example of predictive coding, a Spatial method, which is a lossless coding method defined by the above-mentioned international standard JPEG, will be described as a second conventional example.

Prior to a specific description of the second conventional example, predictive coding will be described. The predictive coding is a method of predicting a pixel value of a pixel to be coded next, and coding a prediction error obtained by the following equation.

[0023]

(Prediction error) = (actual pixel value) − (prediction value) (4) As shown in FIG. 31, since the prediction error is concentrated on 0 in the artificial image, the code amount is generally larger than that of the natural image. Can be reduced. In particular, lossless predictive coding cannot control the code amount, but does not degrade image quality.

Hereinafter, the second conventional example will be specifically described. FIGS. 38 and 39 are configuration diagrams of a second conventional image lossless encoding device and decoding device, respectively. The figure is the same as the above-mentioned "International standard encoding method for color still images-JPEG algorithm-" p. 17 is partially extracted from FIG. 17, a decoding device is added, and terms are modified. In the figure, the same parts as those in FIGS. 33 and 34 are denoted by the same reference numerals, and description thereof will be omitted. 25 is a prediction unit, 46 is a prediction error output unit, 226 is a prediction error input unit, and 171 is prediction error data.

Each part of FIGS. 38 and 39 will be described. 38 has the following configuration. The prediction unit 25 predicts a pixel value to be encoded next using the input image data 110, and sends a difference from an actual pixel value to the prediction error output unit 46 as prediction error data 171.

The decoding device shown in FIG. 39 has the following configuration.
The prediction error input unit 226 inputs a prediction error from outside and sends it to the prediction unit 25 as prediction error data 171. The prediction unit 25 is the same as the prediction unit 25 of the encoding device, except that the prediction unit 25 refers to an image obtained by decoding the next pixel for prediction.

The operation of the second conventional example will be described based on the above configuration. 40 and 41 are flowcharts showing the operation of the conventional example.

First, the encoding procedure of the first conventional example will be described with reference to FIG. 35 are denoted by the same reference numerals, and description thereof will be omitted. In S25, the prediction unit 25 calculates a prediction error from equation (4). In S76, the prediction error output unit 46 outputs the prediction error data 171 calculated in S25 to the outside.

Next, the decoding procedure of the first conventional example will be described with reference to FIG. 36 are given the same reference numerals, and description thereof is omitted. In S116, a prediction error is input from outside to the prediction error input unit 226. S135
Then, the prediction unit 25 calculates a pixel value by adding the prediction value and the prediction error.

In the description of the operation, the prediction error calculation processing will be described. In the JPEG-Spatial system, one of the seven predictors shown in FIG. 42 is defined to be used. For example, when the prediction formula is a, the pixel value on the left of the pixel x to be encoded may be set as the prediction value.

Although the first and second conventional examples have been described above, it will be described below that it is difficult to efficiently encode a natural image and an artificial image without any distinction.

In the artificial image, important information is also included in the high-frequency component. Therefore, if the high-frequency quantization is performed as shown in FIG. 37, the image quality deteriorates, for example, mosquito noise occurs. FIG.
Examples of mosquito noise generated by the quantization table shown in 7a) are shown in FIGS. 43a) and b). FIG. 7A shows an input image, and FIG. 7B shows a decoded image. Due to such noise, it is difficult to reduce the code amount in the frequency conversion coding such as the JPEG-DCT method while maintaining the image quality of the artificial image. This is shown in FIG.

On the other hand, a natural image has different pixel values even between neighboring pixels due to the influence of noise, so that JPEG-Spatia
The code amount does not decrease in lossless predictive coding such as the 1-method. This is shown in FIG. Further, in lossless encoding, since the image quality and the code amount cannot be traded off, the code amount cannot be controlled. This directly affects the capacity of the storage medium, the communication band, and the like, and makes it difficult to construct a system.

As described above, there are images that cannot be effectively encoded in the first and second conventional examples. In order to solve this problem, a method of selectively using irreversible encoding and lossless encoding for each part can be considered. As such an example, Japanese Patent Laid-Open Publication No.
There is 3145 gazette. Hereinafter, the invention described in the publication will be described as a third conventional example.

FIG. 46 is a block diagram of a third conventional image processing apparatus. In this figure, a part of FIG. 1 of the publication is omitted and terms are modified so as not to impair the gist of JP-A-6-113145. In the figure, 15 is an artificial image input unit, 16 is a natural image input unit, 90 is an artificial image encoding unit,
91 is a natural image coding unit, 92 is an artificial image storage unit, 93
Is a natural image storage unit, 94 is an artificial image decoding unit, 95 is a natural image decoding unit, 96 is an image synthesis unit, 112 is input artificial image data, 113 is input natural image data, 114 is artificial image code data, and 115 is natural image data. Image code data, 116 is decoded artificial image data, and 117 is decoded natural image data.

Each part of FIG. 46 will be described. The artificial image input unit 15 and the natural image input unit 16 input an artificial image and a natural image from the outside, respectively.
12. The input natural image data 113 is sent to the artificial image encoding unit 90 and the natural image encoding unit 91. The artificial image encoding unit 90 and the natural image encoding unit 91 encode the input artificial image data 112 and the input natural image data 113, respectively, by a predetermined method. As 115,
The information is sent to the artificial image storage unit 92 and the natural image storage unit 93.
The artificial image storage unit 92 and the natural image storage unit 93 store the artificial image code data 114 and the natural image code data 11 respectively.
5 is temporarily stored and sent to the artificial image decoding unit 94 and the natural image decoding unit 95, respectively. The artificial image decoding unit 94 and the natural image decoding unit 95 each include an artificial image encoding unit 9.
0, a decoding process corresponding to the encoding performed by the natural image encoding unit 91 is performed on the artificial image code data 114 and the natural image code data 115, and the decoded artificial image data 116
The decoded natural image data 117 is sent to the image synthesizing unit 96. The image combining unit 96 combines the decoded artificial image data 116 and the decoded natural image data 117.

In the above description, the coding performed by the artificial image coding unit 90 is described in the first embodiment of the patent as "having a function of a reversible method such as a run-length coding method". I have. The encoding performed by the natural image encoding unit 91 is also described in the first embodiment of the patent as "JP
"Image compression method such as EG". The JPEG referred to in the patent is JPEG-DC referred to in the present description.
Refers to the T method.

[0038]

Since the first and second prior arts are designed specifically for natural images and artificial images, respectively, both of the images can be efficiently handled by a single method. It has already been pointed out that it is difficult.

In the third conventional example, since the natural image and the artificial image are encoded and decoded in a completely different manner in parallel, the processing times of both processes generally do not match. For this reason, all the code data cannot be output to the outside at the time of encoding until all the image data are available at the time of decoding, and the encoding device has a code buffer for at least one image, and the decoding device also has An image buffer for at least one image is required. These are unnecessary configurations if the image encoding / decoding apparatus has only one system.

Since both the encoding device and the decoding device have two or more systems, the size of the device is increased. Further, since an image is represented by a plurality of completely different codes, handling of the codes becomes complicated at the time of transmission and storage. Further, with respect to the image quality of the decoded image, noise may be generated at the switching part of the encoding method.

The present invention has been made in view of the above circumstances, and has as its object to provide a single encoding device and a single decoding device capable of efficiently compressing a natural image and an artificial image without distinction. .

[0042]

The present invention employs the following structure to achieve the above object. First, the invention of the image encoding device will be described.

According to the first aspect of the present invention, in the image encoding apparatus, image input means for inputting an image, frequency conversion means for performing frequency conversion for obtaining a frequency component of the image input by the image input means, A threshold processing unit that performs threshold processing on the frequency component obtained by the frequency conversion unit; and, based on a result of the threshold processing performed by the threshold processing unit, an image of a low-frequency component of the image input by the image input unit. Low-frequency image output means for outputting, and pixel thinning means for performing a predetermined thinning process on an image output by the low-frequency image output means, according to a result of threshold processing by the threshold processing means, Coefficient information output means for outputting a result of the threshold processing by the threshold processing means; and an image which has been subjected to the thinning processing by the pixel thinning means. Characterized by comprising a pull image output unit.

In this configuration, the image is represented at the optimum resolution, thereby suppressing the redundant component and reducing the code amount. Perform frequency analysis to find the optimal resolution,
Pixel thinning processing is performed based on the analysis result.

According to a second aspect of the present invention, in the image decoding apparatus, there are provided image input means for inputting an image, and frequency conversion means for performing frequency conversion for obtaining a frequency component of the image input by the image input means. A threshold processing unit that performs threshold processing on the frequency component obtained by the frequency conversion unit; and a high-frequency component among the frequency components obtained by the frequency conversion unit in accordance with a result of the threshold processing performed by the threshold processing unit. High frequency coefficient masking means for replacing with 0, inverse transforming means for performing inverse frequency transformation for converting a frequency component in which high frequency components have been replaced with 0 by the high frequency coefficient masking means into an image, and threshold processing by the threshold processing means A pixel thinning unit that performs a predetermined thinning process on the image converted by the inverse conversion unit in accordance with the result obtained by the inverse conversion unit; And coefficient information output means for outputting a result of the threshold processing by means, characterized by comprising a thinned image output means for outputting the thinned processed image by the pixel thinning unit.

Also in this configuration, by expressing an image with an optimum resolution, it is possible to suppress a redundant component and reduce the code amount.

According to a third aspect of the present invention, in the image encoding apparatus, image input means for inputting an image, and frequency conversion means for performing frequency conversion for obtaining a frequency component of the image input by the image input means. A threshold processing unit for performing threshold processing on the frequency component obtained by the frequency conversion unit; and a predetermined thinning-out operation on the image input by the image input unit according to a result of the threshold processing performed by the threshold processing unit. Pixel thinning means for performing processing, coefficient information output means for outputting a result of threshold processing by the threshold processing means, and thinned image output means for outputting an image thinned by the pixel thinning means. Features.

Also in this configuration, by expressing an image at an optimum resolution, redundant components can be suppressed and the code amount can be reduced.

According to a fourth aspect of the present invention, in the image coding apparatus, the image input means for inputting an image and the image input by the image input means are subjected to predetermined thinning processing and predetermined interpolation processing. A pseudo-decoded image generating means for generating a pseudo-decoded image; a coefficient analyzing means for determining a thinning rate based on an error between the pseudo-decoded image generated by the pseudo-decoded image generating means and the image input by the image input means; Pixel thinning means for performing a predetermined thinning process on the image input by the image input means according to the thinning rate obtained by the coefficient analyzing means, and a coefficient for outputting the thinning rate obtained by the coefficient analyzing means It is characterized by comprising an information output unit and a thinned-out image output unit that outputs an image thinned out by the pixel thinning-out unit.

Also in this configuration, by expressing an image with an optimum resolution, it is possible to suppress a redundant component and reduce the code amount.

According to the invention of claim 5, according to claim 4,
In the image coding apparatus of the above, the error used in the coefficient analysis means is a pixel value error, the absolute value of the error, the maximum value of any one of the squares of the error, or the dynamic range,
It is one of variance and SN ratio.

According to the invention of claim 6, according to claim 4,
In the image coding apparatus of the above, the predetermined interpolation processing of the pseudo-decoded image generation means includes nearest neighbor interpolation, four-point linear interpolation,
It is characterized by being either point quadratic interpolation or low-pass filter processing.

According to the seventh aspect of the present invention, in the image coding apparatus, code input means for inputting a code obtained by performing frequency conversion and entropy coding on an image, and input by the code input means. Entropy decoding means for obtaining a frequency component by performing decoding corresponding to the inverse transform of the entropy encoding performed on the code,
A threshold processing unit that performs threshold processing on the frequency component obtained by the entropy decoding unit; and a high-frequency component among the frequency components obtained by the entropy decoding unit according to a result of the threshold processing performed by the threshold processing unit. High frequency coefficient masking means for replacing with 0, inverse transforming means for performing inverse frequency transformation for converting a frequency component in which high frequency components have been replaced with 0 by the high frequency coefficient masking means into an image, and threshold processing by the threshold processing means A pixel thinning unit that performs a predetermined thinning process on the image converted by the inverse conversion unit in accordance with the result, a coefficient information output unit that outputs a result of the threshold processing by the threshold processing unit, And a thinning-out image output unit for outputting an image thinned out by the pixel thinning-out unit.

According to the invention of claim 8, according to claim 7,
Wherein the decoding by the entropy decoding means is one of Huffman coding, arithmetic coding and QM coding.

According to the ninth aspect of the present invention, in the image encoding apparatus, image input means for inputting an image, and frequency conversion means for performing frequency conversion for obtaining a frequency component of the image input by the image input means. A threshold processing unit that performs threshold processing on the frequency component obtained by the frequency conversion unit; and a high-frequency component among the frequency components obtained by the frequency conversion unit in accordance with a result of the threshold processing performed by the threshold processing unit. A high-frequency coefficient masking means for replacing the component with 0, an inverse transforming means for performing an inverse frequency transform for converting a frequency component in which the high-frequency component has been replaced with 0 by the high-frequency coefficient masking means into an image, and the threshold processing means A pixel thinning unit that performs a predetermined thinning process on the image converted by the inverse conversion unit in accordance with a result of the threshold processing; Data synthesis means for synthesizing the thinned image obtained by the means and the result of threshold processing by the threshold processing means, and synthesized data output means for outputting synthesized data synthesized by the data synthesis means. Features.

Also in this configuration, by expressing an image at an optimum resolution, redundant components can be suppressed and the code amount can be reduced.

According to a tenth aspect of the present invention, in the image encoding apparatus, an image inputting means for inputting an image, coefficient information inputting means for inputting coefficient information, and an image inputting means for inputting the image by the image inputting means. Frequency conversion means for performing frequency conversion for obtaining a frequency component; and a high-frequency component for replacing high-frequency components with zero among the frequency components obtained by the frequency conversion means in accordance with the coefficient information input by the coefficient information input means. Coefficient masking means, inverse transforming means for performing an inverse frequency transformation for converting a frequency component obtained by replacing the high frequency component with 0 by the high frequency coefficient masking means to an image, and coefficient information input by the coefficient information inputting means. Accordingly, a pixel decimation unit that performs a predetermined decimation process on the image converted by the inverse conversion unit, and an image that is input by the coefficient information input unit. And coefficient information output means for outputting the coefficient information, characterized by comprising a thinned image output means for outputting the thinned processed image by the pixel thinning unit.

Also in this configuration, by expressing an image with an optimal resolution, redundant components can be suppressed and the code amount can be reduced.

According to the eleventh aspect of the present invention, in the image encoding apparatus according to any one of the first to eighth or tenth aspects, further, image encoding is performed on the thinned image output by the thinned image output means. It is characterized by having an image coding means for performing.

According to a twelfth aspect of the present invention, in the image encoding apparatus according to the eleventh aspect, the image encoding performed by the image encoding means is either lossless encoding or predictive encoding or It is characterized by both.

According to a thirteenth aspect of the present invention, in the image encoding apparatus according to any one of the first to eighth or tenth aspects, furthermore, the source information encoding is performed on the coefficient information output by the coefficient information output means. And a coefficient information encoding unit for performing

According to a fourteenth aspect of the present invention, in the image coding apparatus according to the second, ninth, or tenth aspect, the frequency transform performed by the frequency transforming means and the inverse transforming means is a discrete cosine transform, It is a Fourier transform, a discrete sine transform, a sub-band transform or a wavelet transform.

According to a fifteenth aspect of the present invention, in the image encoding apparatus according to any one of the first to third and seventh to ninth aspects, the threshold value processing of the threshold value processing means uses a predetermined quantization table as a threshold value. It is characterized by threshold processing.

According to a sixteenth aspect of the present invention, in the image coding apparatus according to the fifteenth aspect, the quantization table used in the threshold value processing means can be set externally.

According to a seventeenth aspect of the present invention, in the image encoding apparatus according to any one of the second to seventh to tenth aspects, the high-frequency coefficient masking means uses the threshold processing means to remove a component larger than the highest frequency component. It is characterized by being replaced with 0.

According to the eighteenth aspect of the present invention, in the image encoding apparatus according to any one of the first to seventeenth aspects, the thinning-out processing of the pixel thinning-out means includes a distribution of a highest frequency component or a non-zero frequency component in the block. Is performed in accordance with the ratio derived from.

According to the nineteenth aspect of the present invention, in the image encoding apparatus according to any one of the first to eighteenth aspects, the ratio of the thinning-out processing performed by the pixel thinning-out means is as follows:
It is characterized in that it is quantized to a predetermined value.

According to a twentieth aspect of the present invention, in the image encoding apparatus according to any one of the first to eighteenth aspects, the predetermined thinning-out processing of the pixel thinning-out means includes: leaving pixels in a grid pattern; It is characterized in that it is performed at the same ratio in the horizontal direction, that the remaining pixels are arranged at substantially equal intervals, and that peak values in peripheral pixels are preferentially left.

According to a twenty-first aspect of the present invention, in the image encoding apparatus according to any one of the first to twentieth aspects, the thinning-out processing of the pixel thinning-out means includes the step of: If the image has been encoded by the encoding device, the same pixel as the previously thinned pixel is thinned out.

According to a twenty-second aspect of the present invention, in the image coding apparatus according to any one of the first to twenty-first aspects, further, a pixel value quantization unit quantizes a pixel value of an image thinned by the pixel thinning unit. It is characterized by having a conversion means.

According to a twenty-third aspect of the present invention, in the image encoding apparatus according to the twenty-second aspect, the pixel value quantizing means performs a quantization step according to a result of threshold processing by the threshold processing means. Or changing the quantization step in accordance with the size of the threshold used in the threshold processing means.

According to a twenty-fourth aspect of the present invention, in the image coding apparatus according to any one of the first to twenty-third aspects, the threshold value processing is further performed by performing a predetermined analysis on the image input by the image input means. The image processing apparatus further includes an image determination unit that determines a threshold used by the unit.

According to a twenty-fifth aspect of the present invention, in the image coding apparatus according to the twenty-fourth aspect, the image determining means determines a difference between a natural image and an artificial image, and sets a threshold value for an artificial image. It is characterized in that control is performed such that a frequency component which is set to 0 in the threshold processing by the threshold processing means is not generated.

According to a twenty-sixth aspect of the present invention, in the image coding apparatus according to the twenty-fourth aspect, the predetermined analysis processing of the image judging means includes measuring a dynamic range of pixel values, measuring a histogram of pixel values, Measurement of lower bits entropy of pixel value, measurement of edge steepness, measurement of line thickness, measurement of frequency component, externally specified, or at least one of edge, pattern, gradation, and line Is detected.

Next, the image decoding apparatus will be described.

According to a twenty-seventh aspect of the present invention, in the image decoding apparatus, coefficient information input means for inputting coefficient information, thinned image input means for inputting a thinned image, and a thinned image input by the thinned image input means And a coefficient interpolating means for calculating a frequency component by a predetermined method in accordance with the coefficient information input by the coefficient information input means, and an inverse frequency conversion for converting the frequency component calculated by the coefficient interpolating means into an image. It is characterized by comprising: an inverse conversion unit; and a decoded image output unit that outputs an image converted by the inverse conversion unit.

With this configuration, it is possible to adaptively thin out the image data in accordance with the frequency analysis and decode the compressed image data.

According to the twenty-eighth aspect of the present invention, in the image decoding apparatus, coefficient information input means for inputting coefficient information for each block, which is a fixed area of the image, and thinned image input for inputting a thinned image for each block Means, pixel value interpolating means for interpolating pixel values by a predetermined method according to the thinned image input by the thinned image input means and the coefficient information input by the coefficient information input means, and the pixel value interpolating means And a decoded image output means for outputting an image interpolated by the above.

Also in this configuration, it is possible to adaptively thin out in accordance with the frequency analysis and decode the compressed image data.

According to a twenty-ninth aspect of the present invention, in the image decoding apparatus according to the twenty-eighth aspect, the predetermined method of the pixel value interpolating means is nearest neighbor interpolation, four-point linear interpolation, nine-point two-point interpolation,
It is characterized by the following interpolation and low-pass filter processing.

According to a thirty-fourth aspect of the present invention, in the image decoding apparatus, synthesized data input means for inputting synthesized data which is data obtained by synthesizing coefficient information and a thinned image, and input by the synthesized data input means Data decomposing means for decomposing the synthesized data into a thinned image and coefficient information, and coefficient interpolating means for calculating a frequency component by a predetermined method according to the thinned image and coefficient information decomposed by the data decomposing means, It is characterized by comprising inverse transform means for transforming the frequency component calculated by the coefficient interpolating means into an image for inverse frequency transform, and decoded image output means for outputting the image transformed by the inverse transform means.

Also in this configuration, it is possible to decode image data compressed by adaptively thinning out according to frequency analysis.

According to a thirty-first aspect of the present invention, in the image decoding apparatus according to the twenty-seventh to thirty-seventh aspects, there is further provided an image decoding means for decoding a code obtained by image-encoding a thinned image into an image. The thinned image input unit inputs the image decoded by the image decoding unit as a thinned image.

According to a thirty-second aspect of the present invention, in the image decoding apparatus according to the thirty-first aspect, the decoding performed by the image decoding means is a reverse process of lossless coding and a reverse process of predictive coding. It is characterized by the following.

According to a thirty-third aspect of the present invention, in the image decoding device according to the thirty-seventh to thirty-second aspects, further, among the images converted by the inverse conversion means, the decimation input by the decimation image input means. Pixels included in an image are provided with pixel value correction means for replacing the pixel values of the thinned image with pixel values, and the decoded image output means outputs an image corrected by the pixel value correction means.

According to a thirty-fourth aspect of the present invention, in the image decoding device according to the twenty-seventh or thirty-fourth aspect, the frequency transform of the inverse transform means is a discrete cosine transform, a Fourier transform, a discrete sine transform, a subband transform. Alternatively, it is a wavelet transform.

According to a thirty-fifth aspect of the present invention, in the image decoding apparatus according to the twenty-seventh or thirty-seventh aspect, the coefficient interpolation performed by the coefficient interpolation means solves a linear simultaneous equation relating to a frequency coefficient and a pixel value. In addition, it is an operation of an inverse matrix obtained in advance for a linear simultaneous equation relating to a frequency coefficient and a pixel value, or a low-pass filter process on a thinned image or an approximation thereof.

According to a thirty-sixth aspect of the present invention, in the image encoding / decoding device, an image input means for inputting an image,
A frequency conversion unit for performing frequency conversion for obtaining a frequency component of the image input by the image input unit; a threshold processing unit for performing threshold processing on the frequency component obtained by the frequency conversion unit; High frequency coefficient masking means for replacing high frequency components with 0 among the frequency components obtained by the frequency converting means according to the result, and frequency components for replacing high frequency components with 0 by the high frequency coefficient masking means. A first inverse transform unit for performing an inverse frequency transform for converting the image into an image, and a predetermined thinning-out process for the image transformed by the first inverse transform unit in accordance with a result of threshold processing by the threshold process unit Pixel thinning means for performing processing, coefficient information output means for outputting a result of threshold processing by the threshold processing means, and pixel thinning means A thinned image output means for outputting a thinned image; coefficient information input means for inputting coefficient information which is a result of threshold processing output by the coefficient information output means; and a thinned image output by the thinned image output means. A thinned image inputting means for inputting, a coefficient interpolating means for calculating a frequency component by a predetermined method according to a thinned image input by the thinned image inputting means and coefficient information input by the coefficient information inputting means, A second inverse transform unit for performing an inverse frequency transform for transforming the frequency component calculated by the coefficient interpolation unit into an image;
And a decoded image output means for outputting an image converted by the inverse conversion means.

In this configuration, the image is represented at the optimum resolution, thereby suppressing the redundant component and reducing the code amount. Perform frequency analysis to find the optimal resolution,
Pixel thinning processing is performed based on the analysis result. Then, it is possible to decode the image data compressed by adaptively thinning out according to the frequency analysis.

According to a thirty-seventh aspect of the present invention, in the image coding method, step 1 for inputting an image, step 2 for performing frequency conversion for obtaining a frequency component of the image input in step 1; Step 3 of performing threshold processing on the frequency component obtained in Step 2; and Step 4 of replacing a high-frequency component in the frequency component obtained in Step 2 with 0 according to the result of the threshold processing in Step 3; A step 5 of performing an inverse frequency conversion for converting a frequency component obtained by replacing the high frequency component with 0 in the step 4 into an image, and an image converted in the step 5 according to the result of the threshold processing in the step 3 Step 6 for performing a predetermined thinning process on the image data, and a step for outputting the result of the threshold processing in Step 3 above. And-up 7, characterized in that it comprises a step 8 which outputs the image thinning process in the step 6.

Also in this configuration, by expressing an image at an optimum resolution, it is possible to suppress a redundant component and reduce the code amount.

According to a thirty-eighth aspect of the present invention, in the image decoding method, a step 1 for inputting coefficient information, a step 2 for inputting a thinned image, the thinned image input in step 2 and the step 1 Step 3 of calculating a frequency component by a predetermined method in accordance with the coefficient information input by Step 3, Step 4 of performing an inverse frequency conversion of converting the frequency component lent by Step 3 into an image, and Step 4 The method comprises the step of outputting a converted image.

Also in this configuration, it is possible to adaptively thin out in accordance with the frequency analysis and decode the compressed image data.

The present invention can be applied to encoding of a multi-plane image.

[0095]

Embodiments of the present invention will be described below. First, a first embodiment using DCT will be described.
Next, a second embodiment using a frequency conversion method other than DCT will be described. Finally, an example in which the present invention is applied to image processing will be described as a third embodiment.

[Embodiment 1] Prior to a specific description of the first embodiment of the present invention, a basic concept of the present invention will be described. The data amount of a digital image is determined by the resolution and the number of bits per pixel. In general, both the resolution and the number of bits of the image format are fixed by constants.

However, the information amount of the image locally changes.
For example, where there is no change in pixel value, the highest resolution is not required, and the number of bits can be limited. This means that the fixed image format contains redundant information.

In particular, the maximum resolution of a natural image is limited by the frequency characteristics and resolution of the device for digital conversion. For example, when a digital image input by a scanner having a resolution of s is managed in an image format having a resolution of 2 s, originally significant pixels are s ² / (2s) ² = 1 /
Only four. This phenomenon becomes conspicuous when the resolution of an output device is increased or when an image is enlarged.

Therefore, the expression of an image at an optimum resolution will be considered. The resolution required for a digital image depends on the highest frequency of the image. For example, the resolution pitch p is
Half the length T / 2 of the cycle T of the highest frequency of the image
Can't be shorter. Considering this conversely, it can be said that an image that is not used up to the highest frequency allowed by the resolution contains redundant pixels. Even if such redundant pixels are decimated, it is possible to interpolate later from peripheral pixels as long as the highest frequency is known.

Based on the above principle, the present invention suppresses a redundant component and reduces a code amount by expressing an image at an optimum resolution. The resolution conversion to the above-described optimum resolution is realized by a pixel thinning process. The analysis for obtaining the optimum resolution is performed by frequency analysis. The encoding / decoding process is performed on the thinned image.

The above-described principle of the present invention expressed by the following formula is applied as follows. And now all it becomes zero above the frequency f _s which is the frequency component v (f) of the image to be coded.

[0102]

(Equation 4) The frequency fs can be obtained by frequency analysis.
On the other hand, assuming that the pitch of the image format is p, the maximum frequency f _max that can be expressed is as follows as described above.

[0103]

(Equation 5) Of course, f _s ≦ f _max . Now, the pitch p _s of the resolution required to express f _s is obtained by the following equation, as in the equation (6).

[0104]

(Equation 6) In this case consists of a f _s ≦ f _max and p ≧ p _s. This pitch p _s indicates the optimum resolution in the present invention.

The effect of the present invention is qualitatively described as follows. When the input is an artificial image, as can be seen from FIG. 31, since the high frequency component contains a lot of information, most pixels cannot be thinned. However, as described in the description of the second conventional example, since lossless compression can be sufficiently performed by predictive coding or the like, there is no problem even if thinning is not performed. When the input is a natural image, the high frequency component may be quantized to some extent as described in the first conventional example. Therefore, the high-frequency component which is small to some extent can be ignored, and the maximum frequency can be lowered, so that the required resolution, that is, the number of pixels can be reduced.

The problems of the conventional example can be expressed as follows from the viewpoint of the present invention. In the first conventional example, the quantization of the frequency component is performed irrespective of the original resolution that the image should have.
As described above, the quantization of frequency components is an act of ignoring small components in the high frequency range, and is equivalent to forcibly reducing the resolution. Therefore, the image quality is degraded for an artificial image that partially requires the maximum resolution, or the amount of code is increased by reducing the quantization.

On the other hand, in the second conventional example, encoding of a natural image is performed at a high resolution which is not originally required, so that the code amount cannot be reduced.

In the third conventional example, since the lossy encoding and the lossless encoding are performed separately in the frequency space and the pixel value space, which are completely different spaces, distortion as mentioned in the problem occurs. In this regard, since the present invention processes all images from a common viewpoint of resolution, such distortion does not occur.

The schematic configuration of the present invention is as follows. The present invention is based on lossless predictive encoding, and realizes irreversible loss of a natural image that requires quantization by thinning out pixels in the preceding stage. The pixel thinning process is performed while judging whether or not a given image has an optimum resolution by frequency analysis and quantization. On the other hand, since the thinning-out processing is invalid for the artificial image, the quantization is strictly performed and only the pixels unnecessary for the lossless encoding are thinned out.

Next, the operation principle of this embodiment will be specifically described. In this embodiment, DCT is used for simple frequency analysis. The DCT and its inverse transform are represented by the above-mentioned equations (1) and (2). That is, the DCT coefficient y (u, v) is calculated based on the pixel value x
It is a linear sum of (m, n), and 6 for an 8 × 8 DCT.
One DCT process is expressed by writing and arranging four expressions.

Here, the DCT coefficient corresponds to a frequency component in the block. Therefore, if the expression that a certain block does not have a high-frequency component is expressed by an expression, if the highest frequencies in the u and v directions are f _u and f _v , respectively, the expression (8) is obtained (where 0 ≦ _fu , f _v ≦ 7).

[0112]

(Equation 7) Now, the DCT coefficient y (u, v) satisfying the expression (8) is (64)
_{- (f u +1) × (} f v +1)) because the pieces there, 8 × 8DC
Of the 64 formula of _{T (64- (f u +1)} × (f v +
1)) expressions have 0 on the left side. This is a DCT process,
Considering a linear system of equations in which the arguments are pixel values and the unknowns are DCT coefficients, the number of unknowns is 64 ( _fu + 1) × (f _v
+1). That is, (64− (f
_{Since u} + 1) × ( _fv + 1)) expressions become redundant, (64− ( _fu + 1) × ( _fv +
1)) can be removed by the operation of the equation. Eventually ( _fu +
1) If only the pixel values of × (f _v +1) are known, D
Solving a simultaneous equation called CT processing gives 64
It can be seen that it is possible to reconstruct the number of DCT coefficients and thus the pixel values.

However, the calculation accuracy is not considered here. Also, for simplicity, the explanation was made as 64 simultaneous expressions,
The processing may be considered as a combination of eight simultaneous two-dimensional independent formulas from the property of the two-dimensional DCT. The above fact is 8 × 8
There is no difference between DCTs other than the constant except for the constant.

According to the above theory, n out of 8 × 8 blocks
Although y pixels can be thinned out, there is a restriction on the way of thinning out. Since two-dimensional DCT is performed by a combination of one-dimensional DCT, it must be thinned out so that finally ( _fu + 1) * ( _fv + 1) pixels remain. However, if the configuration is such that interpolation in the u-direction is performed first two-dimensionally independently, u
What is necessary is that (f _v +1) pixels remain in the v direction when the interpolation of the direction is completed. At this time, the pixel interval is not limited, but since the pixel value can have only integer precision,
If a spatially close pixel is left, the accuracy of the interpolated pixel value may decrease.

For simplicity, the above logic is expressed by an equation using an example of a one-dimensional DCT of eight pixels. First, the equation (9) can be easily derived from the equation (1) for the transform equation of the one-dimensional DCT.

[0116]

(Equation 8) Equation (9) can be expressed as a matrix because it is simply a sum of products. If the term of cos is expressed as d (u, m), (10)
Get the expression.

[0117]

(Equation 9) Here, if f _u = 2, y (u) = 0 (u> 2).

[0118]

(Equation 10) Becomes Since the left side of the lower five equations of the equation (11) is fixed to 0, the variables on the right side can be deleted by substituting the lower three equations into the upper three equations. For example, if x (7) is deleted from x (3), Equation (12) is eventually obtained.

[0119]

[Equation 11] If three pixel values x (0), x (1), and x (2) are known from equation (12), y (0), y (1), and y (2) can be obtained. Since it is known that y (3) to y (7) are 0, x (3) to x (7) can be interpolated by the inverse transformation of equation (9). There is no restriction on how to select the variable to be deleted in the equation (11), so that the pixel values selected on the right side of the equation (12) may be any combination as long as the number is the same. However, as described above, there is a property that the wider the interval, the better the accuracy of interpolation.

FIG. 5 shows an example of a thinning method. FIG. 4a clearly satisfies the above-mentioned condition. In FIG. 3B, decoding can be performed by first performing interpolation in the u direction and then performing interpolation in the v direction. FIGS. C) and d) do not satisfy the restrictions.

An extension of the thinning method will be described. In the above, the thinning-out based on the expression (8) has been described for simplicity. In fact, equation (8) expresses well the concept of the present invention described at the beginning of this embodiment. However, in the present embodiment, since the interpolation processing can be reduced to simultaneous equations, the equation (8) can be extended. That is, when the frequency fs _u even less f _u is 0 regardless of the component v component is present, eight equations for frequency fs _u can remove from the simultaneous equations. Therefore, the number of pixels left in the u direction can be reduced to f _u . fs _u may be a plurality. The same applies to the v direction.

In the above _description , the thinning process is performed independently based on f _u and f _v , but both axes may be set to the higher frequency of f _u or f _v . This reduces the number of pixels that can be thinned out, but also reduces the number of patterns to be thinned out, so that processes such as coefficient analysis processing and interpolation processing can be simplified. Of course, values such as the average value and the minimum value of f _u and f _v may be used as long as image quality degradation is allowed. Alternatively, the values of f _u and f _v are, for example, 0, 1, 3, 7
The same effect can be obtained even if quantization is performed appropriately, such as

Incidentally, since the expression (8) is written depending on the highest frequency in the u and v directions, the area of the effective frequency component forms a rectangle on the DCT coefficient block. This is because the two-dimensional DCT is realized by a combination of the one-dimensional DCT. If the two-dimensional transform bases are completely independent of each other, they can be extended to free shapes other than rectangles. For example, the upper left triangular component may be limited to remain on the frequency component. In this case, there is no restriction on the method of thinning.

The quantization of DCT coefficients will be described. JP
As described in the description of the EG-DCT method, in the frequency transform coding, by coarsely quantizing the high frequency component, the code amount can be reduced while suppressing the deterioration of the image quality. Also in the present embodiment, for example, a quantization process using the quantization table of FIG. 37 can be applied. Although the frequency components that become 0 by quantization increase, the principle described above can be applied almost as it is.

A specific procedure for applying quantization will now be described. In the present invention, the frequency conversion is used only for analyzing the image, and the actual quantization is realized by thinning out the pixels. Therefore, the quantization of the frequency component is strictly realized by threshold processing for the absolute value. That is, each frequency component is compared with the corresponding quantization step, and if it is smaller, the frequency component is set to 0.
Is performed. If the quantization table is appropriately set, by performing inverse DCT processing on the coefficient data subjected to the threshold processing, it is possible to obtain an image with no image quality degradation and with limited high-frequency components. From then on, the principle described above may be applied as it is. The next step is to summarize. [Algorithm when Quantizing Coefficient Data] Step 1: DCT processing is performed. Step 2: The coefficient data is subjected to threshold processing, and components smaller than the quantization step are set to 0. The highest frequency component of this time is f _u, f _v. Step 3: Perform inverse DCT processing. Step 4: A thinning process is performed based on f _u and f _v obtained in step 2. If any images remain, go to step 1.

In step 2, a component which happens to be thresholded to zero occurs at a frequency lower than f _u and f _v . In this algorithm, the thinning process in step 4 is performed in step 2
Since these components are based on f _u and f _v obtained by the above equation, the processing amount does not change in both the thinning / interpolation processing even if such components are not forcibly set to 0. Therefore, the following processing may be inserted between step 2 and step 3.

Step 2.5: If any of the frequency components equal to or less than f _u and f _v have been thresholded to 0, the data is restored to the data before the threshold processing.

In the present invention, compression means such as predictive coding is assumed at the subsequent stage. Since the number of pixels to be sent to the subsequent stage can be reduced according to the principle described above, the processing at the subsequent stage can be reduced as a secondary effect of the present invention. This is effective even when image processing such as color conversion, enlargement / reduction, rotation, and clipping is performed in the subsequent stage, so that the present invention can also be applied as an accelerator for image processing.

Now that the principle has been described, a specific description of the present embodiment will be given. Hereinafter, a portion that performs a pixel thinning process on a natural image, excluding the above-described subsequent portion, will be described.

FIGS. 1 and 2 are block diagrams showing a first embodiment of the present invention. In the figure, the same parts as those in FIGS. 33 and 34 are denoted by the same reference numerals, and description thereof will be omitted. FIG.
2 and FIG. 2, reference numeral 30 denotes a coefficient analysis unit, 40 denotes a coefficient analysis output unit, 50 denotes a high-frequency coefficient mask unit, and 60 denotes an inverse DCT.
Unit, 70 is a pixel thinning unit, 80 is a thinned image output unit, 1
30 is coefficient analysis data, 140 is low frequency coefficient data, 15
0 is low-frequency image data, 160 is thinned image data, 21
0 is a thinned image input unit, 220 is a coefficient analysis input unit, 23
0 is a coefficient interpolation unit, and 310 is interpolation coefficient data.

Each part in FIGS. 1 and 2 will be described.
The encoding device of FIG. 1 has the following configuration. Coefficient analysis unit 3
0 compares the coefficient data 120 with a predetermined constant, and sends the result of the comparison to the coefficient analysis output unit 40, the high-frequency coefficient mask unit 50, and the pixel thinning unit 70 as coefficient analysis data 130. The coefficient analysis output unit 40 outputs the coefficient analysis data 130 to the outside. The high-frequency coefficient masking section 50 stores coefficient data 12 based on the coefficient analysis data 130.
Part of the high-frequency coefficient of 0 is replaced with 0, and the low-frequency coefficient data 1
As 40, it is transmitted to the inverse DCT unit 60. Inverse DCT unit 60
Performs inverse DCT processing corresponding to the inverse transform of the DCT processing performed by the DCT section 20 on the low-frequency coefficient data 140, and sends the result to the pixel thinning section 70 as low-frequency image data 150. The pixel thinning section 70 performs thinning processing on the low-frequency image data 150 based on a preset thinning method and coefficient analysis data 130, and outputs the thinned image data 160 to the thinned image output section 80. Thinned image output unit 8
0 sends out the thinned image data 160 to the outside.

Next, the decoding apparatus of FIG. 2 has the following configuration. The thinned image input unit 210 inputs a thinned image from the outside and outputs the thinned image data 160 to the coefficient interpolation unit 2.
30. The coefficient analysis input unit 220 inputs coefficient analysis data from the outside, and sends it to the coefficient interpolation unit 230 as coefficient analysis data 130. The coefficient interpolation unit 230 performs interpolation processing of DCT coefficients on the thinned image data 160 based on the coefficient analysis data 130, and sends the result to the inverse DCT unit 240 as interpolation coefficient data 310. The inverse DCT unit 240 performs an inverse DCT process on the interpolation coefficient data 310 and outputs the decoded image data 320 as decoded image data 320.
Send to 50.

The operation of the first embodiment will be described based on the above configuration. FIGS. 3 and 4 are flowcharts showing the operation of the first embodiment of the present invention.

First, the encoding procedure of this embodiment will be described with reference to FIG. In S10, an image is input from the outside in the image input unit 10, and input image data 110 is obtained. In S20, the DCT processing is performed in the DCT section 20 to obtain coefficient data 120. In S30, the coefficient analysis unit 3
At 0, the coefficient data 120 is compared with a predetermined constant, and the result is obtained as coefficient analysis data 130. In S40, a part of the high-frequency component of the coefficient data 120 is replaced with 0 based on the coefficient analysis data 130 in the high-frequency coefficient mask unit 50 to obtain low-frequency coefficient data 140. S5
If the value is 0, the inverse DCT unit 60 performs inverse DCT processing on the low-frequency coefficient data 140 to obtain low-frequency image data 150. S
At 60, the coefficient analysis data 13 at the pixel thinning unit 70
Pixel thinning processing is performed based on 0, and thinned image data 160 is obtained. In S70, the coefficient analysis output unit 40 and the thinned image output unit 80 output the coefficient analysis data 130 and the thinned image data 160, respectively, to the outside. In S80, the input image data 11 input in S10
It is determined whether all 0s have been processed. If there is unprocessed data, the process returns to S10, and if all 0s have been processed, the encoding procedure ends.

Next, the decoding procedure of this embodiment will be described with reference to FIG. In S110, the thinned image data 160 and the coefficient analysis data 130 are externally input to the thinned image input unit 210 and the coefficient analysis input unit 220, respectively. In S120, the coefficient interpolation unit 230 obtains interpolation coefficient data 310 based on the thinned image data 160 and the coefficient analysis data 130. In S130, the inverse DCT section 240 performs an inverse DCT process on the interpolation coefficient data 310 to obtain decoded image data 320. At S140, the decoded image output unit 250 outputs the decoded image data 320 to the outside. In S150, it is determined whether or not all of the thinned image data 160 and coefficient analysis data 130 input in S110 have been processed.
Returning to step 10, if all processes have been completed, the decoding procedure ends.

The coefficient analysis processing during the above operation will be described. In the coefficient analysis processing, a coarse constant is used for a high-frequency coefficient like a quantization table used in the JPEG-DCT method. However, as described above, the validity of the DCT coefficient is determined not by quantization but by simple threshold processing.

The flow of the coefficient analysis process in the case where the recommended table of the JPEG-DCT system is used will be described with reference to FIG. FIG. 7A shows coefficient data 120 obtained by DCT processing.
This is an example. On the other hand, when the quantization of the JPEG-DCT method is performed, the quantized coefficient data of FIG. In this embodiment, threshold processing is performed. For example, if the effective coefficient is represented by 1 and the invalid coefficient is represented by 0, the coefficient analysis data 1 as shown in FIG.
Obtain 30.

As described at the beginning of the description of the present embodiment, the number of pixels to be decimated is determined according to the maximum frequency in each of u and v directions. Absent. The format may be abbreviated as (4, 4). The low-frequency coefficient data 140 created by the high-frequency coefficient mask unit 50 based on the coefficient analysis data 130 is as shown in FIG.

During operation, the coefficient interpolation unit 230 is executed at S120.
Is performed by solving simultaneous equations as described at the beginning of the description of this embodiment. Note that the simultaneous equations selected are, for example, at most 64 combinations of 8 × 8 blocks, and therefore, if the inverse matrix is obtained in advance, the processing can be easily performed.

In operation, it has already been described that it is assumed that the thinned image data 160 output in S70 is encoded by the encoding device at the subsequent stage, but similarly, the coefficient analysis data 130 contains some information. It may be encoded by source encoding.

As described above, according to this embodiment, it is possible to effectively perform irreversible encoding on a natural image by using reversible encoding in the subsequent stage. When encoding an artificial image, all values of the quantization table used for the threshold processing performed by the coefficient analysis unit 30 of the encoding device in FIG.
As a result, the quantization of the coefficient data is not performed.
If there is no calculation error, the encoding device of FIG. 1 operates as a lossless encoding device. Of course, when an artificial image is input, a separate data path that bypasses the configuration of FIG. 1 may be provided.

The processing performed by the wide area coefficient masking section 50 and the inverse DCT section 60 of the encoding apparatus shown in FIG. 1 is to cut off the high frequency region of the image input by the image input section 10, ie, to perform low-pass filter processing. Is the same processing as. Therefore, the wide area coefficient mask section 50 and the inverse DCT section 60 can be a low-pass filter.

[First Extension Example] An extension of this embodiment will be described. The difference between the JPEG-DCT method and the present embodiment in the decoded image is that, in the JPEG-DCT method, all the pixel values may be slightly different from those of the input image. Pixels not subtracted are sent to the decoding side as they are. As an example, the average of the absolute values of the differences between the pixel values of the decoded image according to the JPEG-DCT method and the input image was obtained for some natural images. FIG. 7 shows the result of stratifying the result by the analysis result of the coefficient data. The result of coefficient analysis on the horizontal axis was calculated by the following equation.

[0144]

(Equation 12) In the above-described example, since the image subjected to the inverse DCT processing is output as a decoded image as it is, even if the pixels are not thinned out, the pixel value may be shifted due to a calculation error or the like. In this embodiment, it is possible to return the pixels that have not been thinned back to the original pixel values before outputting the decoded image. When such processing is performed, the same thinned-out image can be obtained by thinning out in the same way when re-encoding. Therefore, irreversible encoding can be realized in which image quality degradation does not overlap even if encoding and decoding are repeated. This is the first of this embodiment.
This is an extension example of.

In order to thin out the same pixels as in the first encoding at the time of re-encoding, the coefficient information data 130 and the thinned-out image data 160 may include information on how to thin out. FIG.
Is an example of such a data format. In this example, data relating to the thinning method is included as a header. The decimation method data may be a list of decimation methods according to the coefficient information data 130 as shown in FIG. 9, or may simply indicate an ID given in advance. Of course, such a mechanism is not necessary when the encoding and decoding are repeated by an encoding device in which the decimation method does not depend on conditions other than the coefficient information data 130.

FIG. 10 is a configuration diagram of a decoding device in a first extended example of the present embodiment. In the figure, the same parts as those in FIG. 1 are denoted by the same reference numerals, and description thereof will be omitted. Reference numeral 241 denotes a pixel value correction unit, and 321 denotes corrected decoded image data.

Each part of FIG. 10 will be described. The pixel value correction unit 241 replaces the pixels provided by the thinned image data 160 in the decoded image data 320 with the pixel values of the thinned image data 160. The result is output to the decoded image output unit 250 as corrected decoded image data 321.
Send to The description of the other parts and the description of the operation are not greatly different from those described above, and will not be repeated.

[Second Extension Example] Returning to the extension of this embodiment.
Considering the JPEG-DCT method as seen in FIG. 7 described above, it is expected that the image quality will not be significantly affected even if the pixel value is quantized at the time of predictive encoding in the latter stage of the present embodiment. You. Since the quantization allowed here is considered to depend on the threshold processing performed on the frequency component, efficient quantization is possible by controlling both in cooperation. The example of FIG. 7 described above is a result of fixed threshold processing in the quantization table of FIG. 37 a), but the tendency of the absolute value of the decoding error can be grasped from the analysis result of the coefficient data. The pixel value quantization processing may be performed with reference to this. For example, assuming that the error occurs in a uniform distribution, if the quantization step is estimated to be twice the average of the error value, equation (14) holds for each analysis type.

[0149]

(Quantization step) = (average of decoding error absolute value) × 2 (14) If the error distribution has a bias centered on, for example, 0, 2 in equation (14) may be a slightly smaller value. . In any case, this value can be calculated experimentally by statistical processing. Of course, the equation (14) may be calculated by a non-linear operation under a more complicated assumption. The above is the second extended example of the present embodiment.

FIG. 11 is a configuration diagram of an encoding device in a second extended example of the present embodiment. In the figure, the same parts as those in FIG. 1 are denoted by the same reference numerals, and description thereof will be omitted. 71 is a pixel value quantization unit, and 161 is quantization thinned image data.

Each part of FIG. 11 will be described. The pixel value quantization unit 71 quantizes the pixel value of the decoded image data 160 by a predetermined method based on the threshold value used in the coefficient analysis unit 30 and the coefficient analysis data 130, and generates the quantized decoded image data 161. The data is sent to the thinned image output unit 80. The description of the other parts and the description of the operation are not greatly different from those described above, and will not be repeated.

In the above example, of course, the pixel value quantization unit 71
May be fixed and not related to the threshold value or the coefficient analysis data 130.

[Third Extended Example] A third extended example of this embodiment will be described. As described above, both lossy coding and lossless coding can be performed by controlling the threshold used in the coefficient analysis processing. Hereinafter, a modified example in which such threshold control is dynamically performed will be described as a third extended example.

FIG. 12 is a configuration diagram of a third extended example of the present embodiment. In the figure, 31 is an image determination unit, and 131 is threshold control data.

Each part in FIG. 12 will be described. The image determination unit 31 performs image determination on the input image data 110 by a predetermined method, and compares the result with the threshold control data 1.
It is sent to the coefficient analysis unit 30 as 31. The description of the other parts and the description of the operation are not greatly different from those described above, and will not be repeated.

The image judging section 31 judges a natural image and an artificial image. Specifically, since the presence or absence of noise is estimated from the spread of the pixel value distribution, the entropy of the lower bits, the sharpness of the edge, and the like, the determination is made based on this. Although illustration is omitted, a configuration in which a distinction between a natural image and an artificial image is sent from the outside as side information may be employed. In this case, the image determination unit 31 converts the side information into the threshold control data 131.

Such threshold control may be performed for each image or may be performed for each local portion of the image. In addition, the control may be performed based on the local properties of the image, regardless of the distinction between the natural image and the artificial image. For example, the distribution of pixel values and frequency components, the sharpness of edges, the presence or absence of patterns, the presence or absence of thin lines, the presence or absence of gradation, and the like can be used as indices.

[First Simplification] Next, the simplification of this embodiment will be described. In the decoding apparatus of FIG.
A means for directly interpolating pixel values may be provided instead of the 0 and inverse DCT section 240. The means for interpolating pixel values here is the nearest neighbor interpolation, such as performed in image processing,
Any method may be used as long as it interpolates pixel values, such as point linear interpolation and 9-point secondary interpolation. In this case, since the principle of the present embodiment does not hold, image quality degradation is inevitable. However, in principle, the coefficient interpolation processing performed in this embodiment has a low-pass filter effect on DCT coefficients, and the pixel value interpolation processing enumerated above also has a low-pass filter effect. It can be regarded as an approximate approximation. The above is the first simplified example of the present embodiment.

FIG. 13 is a configuration diagram of a first simplified example of the present embodiment. In the figure, the same parts as those in FIG. 2 are denoted by the same reference numerals, and description thereof will be omitted. 231 is a pixel value interpolation unit.

The components in FIG. 13 will be described. The pixel value interpolating unit 231 interpolates the thinned-out image data 160 with the thinned-out pixels by a predetermined method, and outputs the thinned-out image data 160 to the decoded image output unit 250 as decoded image data 320. The description of the other parts and the description of the operation are not greatly different from those described above, and will not be repeated.

As described above, since the pixel value interpolation in the first simplification example has the effect of a low-pass filter, even if the same number of pixels are thinned out, a difference in image quality may occur depending on the selected pixel. Also, as described above, the pixel selection can be freely performed to some extent if the restrictions are observed. Therefore, for example, when selecting a peak value in a block when selecting a pixel to be left, the dynamic range of the block is not impaired.

When pixel value interpolation is performed, equation (8) does not need to be satisfied. Therefore, except for the problem of aliasing distortion, the image to be subjected to the thinning processing need not be limited to a high frequency band. Therefore, for example, instead of providing the high-frequency coefficient mask unit 50 and the inverse DCT unit 60 in the encoding device of FIG. 1, the input image data 110 may be directly sent to the pixel thinning unit 70. In this case, the processing can be greatly simplified. This configuration is shown in FIG. Description of each part and operation is omitted. When no quantization is performed on the high-frequency component in the coefficient analysis unit 30 in FIG. 1, the input image already satisfies Expression (8). Therefore, the encoding apparatus can be realized with the configuration shown in FIG. 14 without affecting the image quality or the code amount.

Further, even when the pixel value interpolation is performed by the decoding device, it is possible to simulate the decoded image by the encoding device. Therefore, instead of the coefficient analysis unit 30 of FIG. 1, the decoded image data 320 in which the pixel value is interpolated is simulated,
Means may be provided for determining the coefficient information data 130 while evaluating an error with respect to the input image data 110. The evaluation of the error may be based on the SN ratio, the maximum value of the error, the variance, the dynamic range, or the like. In this case, the coefficient information data 130 simply means the pixel thinning rate. In this case, since the frequency analysis is not performed, the DCT section 20 can obviously be omitted. Since the configuration can be analogized, the description below the configuration diagram is omitted.

[Second Simplification] Next, consider a case where a DCT coefficient is received as an input instead of an image. For example, JP
When receiving an image encoded by the EG-DCT method,
The data obtained by the corresponding entropy decoding is not a pixel value but a DCT coefficient. In such a case, the DCT coefficients may be directly input to the coefficient analysis unit 30 and the high-frequency coefficient mask unit 50 of the encoding device in FIG. By doing so, the DCT unit 20 and its processing can be omitted. This is a second simplified example of the present embodiment.

FIG. 15 is a configuration diagram of a second simplified example of the present embodiment. In the figure, the same parts as those in FIG. 1 are denoted by the same reference numerals, and description thereof will be omitted. 11 is a code input unit, 21 is an entropy decoding unit, and 111 is code data.

Each part of FIG. 15 will be described. The code input unit 11 inputs a code from outside and sends it to the entropy decoding unit 21 as code data 111. The entropy decoding unit 21 decodes the code data 111 and outputs the coefficient data 12
The value is sent to the coefficient analysis unit 30 and the high-frequency coefficient mask unit 50 as 0. The description of the other parts and the description of the operation are not greatly different from those described above, and will not be repeated.

When a DCT coefficient is input as described above, it is assumed that the DCT coefficient has already been quantized. In this case, it is necessary for the entropy decoding unit 21 to perform an inverse quantization process.

Further, when the quantization step of the input code is coarser than the quantization step predetermined in the coefficient analysis section 30, the processing of each section can be simplified. One is related to the coefficient analysis processing in the coefficient analysis unit 30. Since a coefficient other than 0 does not become 0 as a result of the threshold processing, it is only necessary to judge whether the frequency component is 0 or other than 0 instead of the threshold processing. For the same reason, the high-frequency coefficient mask unit 50 and the inverse DCT unit 60
Can be omitted without image degradation. In such a case, the decoding device in FIG. 15 may have a configuration capable of bypassing the above-described portion.

[Third Simplification] Next, a third simplification example will be described. According to the configurations of FIGS. 1 and 2, the coefficient analysis data 130 and the thinned image data 160 are input and output independently, but both data may be input and output collectively. This is a third simplified example.

FIG. 16 is a configuration diagram of a third simplified example of the present embodiment. In the figure, the same parts as those in FIG. 1 are denoted by the same reference numerals, and description thereof will be omitted. 72 is a data synthesizing unit, 81 is a synthesized data output unit, and 162 is synthesized data.

The components in FIG. 16 will be described. The data synthesizing unit 72 includes the coefficient analysis data 130 and the thinned image data 1
60 are combined and sent to the combined data output unit 81 as combined data 162. The composite data output unit 81 outputs the composite data 162 to the outside. The description of the other parts and the description of the operation are not greatly different from those described above, and will not be repeated. Also, a description of the decoding device corresponding to the coding device of the third simplified example is omitted because it can be easily analogized.

The data synthesizing process performed by the data synthesizing section 72 will be described. The combined data 162 needs to be combined so that the decoding device can decompose it into the coefficient analysis data 130 and the thinned image data 160. As such an example, there are several examples such as a method of simply combining two as shown in FIG. 17 and a method of combining each block as shown in FIG. Of course, other units may be used.

[Fourth Simplification] Next, a fourth simplification example will be described. If the effective frequency component of the input image is known in advance, coefficient information may be designated from outside. This is the fourth simplified example.

FIG. 19 is a configuration diagram of a fourth simplified example of the present embodiment. In the figure, the same parts as those in FIG. 1 are denoted by the same reference numerals, and description of each part and operation is omitted.

With such a configuration, for example, when it is known that the high frequency component is clearly noise, the component to be set to 0 can be directly specified, so that the code amount is reduced. An example of such an image is, for example, a natural image that has been input by a low-resolution scanner and then enlarged. Such an enlarged image may be subjected to processing such as edge enhancement in order to suppress blurring due to enlargement. The high-frequency components generated by such processing cannot be said to be noise. However, since reproduction can be performed after decoding, it is not necessary to reproduce at the optimum resolution according to the present invention. On the other hand, in the first conventional example, even such an image is reproduced up to a high frequency, so that the code amount cannot be reduced. This phenomenon becomes more pronounced as the resolution of the output device increases. FIG. 27 shows this state.

Finally, FIG. 20 shows an example of an experimental result obtained by encoding a natural image and an artificial image based on one embodiment of the present embodiment.
FIGS. 28 and 29 show a difference image between a decoded image and an input image created based on one embodiment of the present embodiment and the first conventional example for a certain natural image.

[Embodiment 2] The basic idea of the present invention is to thin out the pixels which can be regarded as being oversampled in a natural image so as to realize irreversible encoding processing in a pixel space. This concept can be extended to frequency conversion methods other than DCT. Hereinafter, an embodiment using a general frequency conversion method will be described as a second embodiment of the present invention.

FIGS. 21 and 22 are block diagrams of an image coding apparatus and a decoding apparatus according to the second embodiment of the present invention, respectively. In the figure, the same parts as those in FIG. 22 is a frequency converter, 61 is an inverse converter, 2
Reference numeral 42 denotes an inverse conversion unit.

The components shown in FIGS. 21 and 22 will be described. The frequency conversion unit 22 converts the frequency of the input image data 110 by some method, and sends out the obtained frequency components as the coefficient data 120 to the coefficient analysis unit 30 and the high-frequency coefficient mask unit 50. The inverse transform unit 61 and the inverse transform unit 242 perform inverse transform of the frequency transform performed by the frequency transform unit 22 on the coefficient data 140 and 310, respectively, and output the results to the low-frequency image data 150 and the decoded image data 32, respectively.
Output as 0. The description of the other parts and the operation is not substantially different from the description of the first embodiment of the present invention, and will not be repeated.

In the above description, the frequency conversion processing may be anything that can perform frequency conversion. For example, fast Fourier transform, discrete sine transform, subband division, and the like correspond to this.

The interpolation processing in the coefficient interpolation section 230 depends on the frequency conversion processing to be used. In the first embodiment, DC
It has been described that interpolation can be realized by solving a linear simultaneous equation for T. A similar method can be applied to a case where an image is divided into blocks by a fast Fourier transform or to a discrete sine transform.

In the case of sub-band division, for example, the validity is determined for each band, and the high-frequency coefficient can be masked by setting the components of the band regarded as invalid to 0. At the time of interpolation, coefficient interpolation can be realized by reconstructing low band components from the thinned image and supplementing the high band with zero.

As can be said for all the frequency conversion methods, if the interpolation performed by the coefficient interpolation unit 230 is replaced by pixel value interpolation as in the configuration of FIG.

[Embodiment 3] Although the present invention has been described as an image coding apparatus and a decoding apparatus, the present invention can be applied to some image processing apparatuses by changing the viewpoint. Hereinafter, as a third embodiment of the present invention, an embodiment will be described in which the present invention is applied to an image coding apparatus that handles multi-plane images. Before going into the details, a multi-plane image is defined. Here, the multi-plane image is an image created by overlapping different material images, and refers to an image in which each material image is handled separately and is overlapped on one sheet at the time of output. However, they are not separately handled for each material image, but may be classified according to image types such as natural images and artificial images as described in the third conventional example. The number of material images to be treated separately, that is, the number of planes may be any. However, in the following description, for simplicity, it is assumed that the image is composed of three planes of a natural image, an artificial image, and switching information. FIG. 23 is an explanatory diagram of such an image.

In FIG. 23, an output image is obtained by overwriting a natural image with an artificial image. Considering the encoding of a natural image, the overwritten portion is not necessary for generating an output image, so if this portion is replaced with a pixel value that is convenient for encoding, the compression ratio can be improved. Can be. FIG.
FIG. 4 is an explanatory diagram of such pixel value replacement. (FIG. 24d)
The portions hatched with diagonal lines are the portions where the pixel values can be replaced.

However, actually, it is not easy to obtain a pixel value convenient for encoding using frequency. The simplest method of pixel value replacement is a method of filling with a fixed value such as white or black. With this, the processing is simple, but an edge may occur at the boundary between the pixel value replacement portion and the original image portion, and a high compression ratio cannot be expected in frequency conversion coding. On the other hand, if the fixed value used for the replacement is replaced with the average value of the remaining pixel values, an edge is less likely to occur, but there is no guarantee that the pixel value is the optimum. In addition, an average value must be obtained, and the processing load is large.

By carefully examining FIG. 24d), it can be seen that the conditions for the thinning method described in the first embodiment and FIG. 5 are satisfied. So we consider this a thinned image,
Consider frequency conversion by applying the coefficient interpolation processing of the present invention. According to the present invention, of the combinations of frequency components representing a thinned image, it is possible to obtain a combination that sets as high a frequency as possible to zero. For example, the aforementioned JPEG-DC
In the case of the T system, encoding is performed from the low band,
Is equivalent to reducing the code amount. In addition, in the case of the present invention, since it is not necessary to fill the pixels of the replacement part, no additional processing occurs. This is the basic idea of the present embodiment.

FIG. 25 is a block diagram of an image processing apparatus according to the third embodiment of the present invention. In the figure, the same parts as those in FIGS. 1, 29, and 42 are denoted by the same reference numerals, and description thereof will be omitted. Reference numeral 17 denotes a switching information input unit, and 118 denotes input switching data.

Each part in FIG. 25 will be described. The switching information input unit 17 inputs input switching data 118 from outside,
The data is sent to the pixel thinning unit 70. The description of the pixel thinning unit 70 will be supplemented. The pixel thinning section 70 performs thinning processing on the input natural image data 113 at the portion where the input switching data 118 selects the artificial image, and outputs the thinned image data 160.

Since the operation can be inferred from the description of the other embodiments and the like, the description is omitted.

The thinning-out process performed by the pixel thinning-out section 70 is performed so as to satisfy the thinning-out condition described in the first embodiment of the present invention. This example is shown in FIG. (FIG. 26a)
When such switching information is provided, the coefficient interpolation processing of the present invention can be applied by performing a thinning method as shown in FIG. Despite the selection of the artificial image, a pixel for storing the pixel value remains, but this part may be replaced with a fixed value or an average value.

This embodiment is similar to the second embodiment in that D
It can be easily extended to frequency conversion other than CT. Although it is common to perform entropy encoding after the coefficient output unit 45, the description is omitted here because it does not affect the essence of the present invention. Further, the code created by the present embodiment can be decoded by, for example, the decoder described in the first conventional example, so that the explanation on the decoding is omitted.

[0193]

As is apparent from the above description, according to the present invention, efficient encoding / decoding processing can be realized with a single device without distinction between natural images and artificial images. Therefore, there is an effect that the page memory can be removed as compared with the case where two encoding methods are combined. Further, by performing the image processing in a later stage of the present invention, the speed of the image processing can be increased. Further, it is possible to realize an encoding / decoding process in which image quality is not deteriorated by repeating the encoding / decoding process. Further, in many cases, an actual high-resolution input image is generally obtained by enlarging a low-resolution image. In such a case, encoding can be performed with the same code amount as for the effective resolution before enlargement. Further, by applying the present invention to the encoding processing of a multi-plane image, an image including pixels to be overwritten can be efficiently encoded.

[Brief description of the drawings]

FIG. 1 is a configuration diagram illustrating an image encoding device according to a first embodiment of the present invention.

FIG. 2 is a configuration diagram illustrating an image decoding device according to a first embodiment of the present invention.

FIG. 3 is a flowchart illustrating an example of an operation of an encoding process of the image encoding device according to the first embodiment of the present invention.

FIG. 4 is a flowchart illustrating an example of an operation of a decoding process of the image decoding device according to the first embodiment of the present invention.

FIG. 5 is an explanatory diagram of a thinning process according to the first embodiment of the present invention.

FIG. 6 is an explanatory diagram of processing of coefficient data in the first embodiment of the present invention.

FIG. 7 is an explanatory diagram regarding a pixel value error in a JPEG-DCT decoded image.

FIG. 8 is a diagram illustrating information for thinning out re-encoding.

FIG. 9 is a diagram illustrating information for thinning out re-encoding.

FIG. 10 is a configuration diagram showing an extended example in the first embodiment of the present invention.

FIG. 11 is a configuration diagram showing an extended example in the first embodiment of the present invention.

FIG. 12 is a configuration diagram showing an extended example in the first embodiment of the present invention.

FIG. 13 is a configuration diagram showing a simplified example in the first embodiment of the present invention.

FIG. 14 is a configuration diagram showing a simplified example in the first embodiment of the present invention.

FIG. 15 is a configuration diagram showing a simplified example in the first embodiment of the present invention.

FIG. 16 is a configuration diagram showing a simplified example in the first embodiment of the present invention.

FIG. 17 is a diagram showing a simplified example in the first embodiment of the present invention.

FIG. 18 is a diagram showing a simplified example in the first embodiment of the present invention.

FIG. 19 is a configuration diagram showing a simplified example according to the first embodiment of the present invention.

FIG. 20 is an explanatory diagram showing an example of an experimental result of the first example of the present invention.

FIG. 21 is a configuration diagram illustrating an image encoding device according to a second embodiment of the present invention.

FIG. 22 is a configuration diagram illustrating an image decoding device according to a second embodiment of the present invention.

FIG. 23 is a diagram illustrating a multi-plane image according to the third embodiment of the present invention.

FIG. 24 is a diagram illustrating replacement of pixel values according to the second embodiment of the present invention.

FIG. 25 is a configuration diagram illustrating an image processing apparatus according to a third embodiment of the present invention.

FIG. 26 is a diagram for explaining the operation of the third embodiment of the present invention.

FIG. 27 is an explanatory diagram schematically showing a comparison of a code amount between the first embodiment and the first conventional example.

FIG. 28 is an explanatory diagram showing a comparison of image quality deterioration between the first embodiment and the first conventional example.

FIG. 29 is an explanatory diagram showing a comparison of image quality degradation between the first embodiment and the first conventional example.

FIG. 30 is an explanatory diagram of an experimental example showing characteristics of an image.

FIG. 31 is an explanatory diagram of an experimental example showing characteristics of an image.

FIG. 32 is an explanatory diagram of an experimental example showing characteristics of an image.

FIG. 33 is a configuration diagram illustrating a first conventional example of an image encoding device.

FIG. 34 is a configuration diagram illustrating a first conventional image decoding apparatus.

FIG. 35 is a flowchart showing an example of an operation of an encoding process in the first conventional example.

FIG. 36 is a flowchart showing an example of the operation of the decoding process in the first conventional example.

FIG. 37 is an explanatory diagram of an example of a quantization table used in the first conventional example.

FIG. 38 is a configuration diagram illustrating a second conventional image coding apparatus.

FIG. 39 is a configuration diagram illustrating a second conventional image decoding apparatus.

FIG. 40 is a flowchart showing an example of the operation of the encoding process in the second conventional example.

FIG. 41 is a flowchart showing an example of the operation of the decoding process in the second conventional example.

FIG. 42 is an explanatory diagram of a predictor used in the second conventional example.

FIG. 43 is an explanatory diagram of mosquito noise.

FIG. 44 is an explanatory diagram of an experimental example according to the first conventional example.

FIG. 45 is an explanatory diagram of an experimental example according to a second conventional example.

FIG. 46 is a configuration diagram showing a third conventional example.

[Explanation of symbols]

 Reference Signs List 10 image input unit 11 code input unit 15 artificial image input unit 16 natural image input unit 20 DCT unit 21 entropy decoding unit 22 frequency conversion unit 25 prediction unit 30 coefficient analysis unit 31 image determination unit 35 coefficient quantization unit 40 coefficient information output unit 45 coefficient output unit 46 prediction error output unit 50 high frequency coefficient mask unit 60 inverse DCT unit 61 inverse transform unit 70 pixel thinning unit 71 pixel value quantization unit 72 data synthesis unit 80 thinned image output unit 81 synthesized data output unit 90 artificial image Encoding unit 91 Natural image encoding unit 92 Artificial image storage unit 93 Natural image storage unit 94 Artificial image decoding unit 95 Natural image decoding unit 96 Image synthesis unit 110 Input image data 111 Code data 112 Input artificial image data 113 Input natural image data 114 Artificial image code data 115 Natural image code data 116 No. artificial image data 117 decoded natural image data 120 coefficient data 130 coefficient analysis data 131 threshold control data 140 low band coefficient data 150 low band image data 160 thinned image data 161 quantized thinned image data 162 synthesized data 170 quantized coefficient data 171 prediction Error data 210 Thinned image input unit 220 Coefficient information input unit 225 Coefficient input unit 226 Prediction error input unit 230 Coefficient interpolation unit 231 Pixel value interpolation unit 240 Inverse DCT unit 241 Pixel value correction unit 242 Inverse transformation unit 250 Decoded image output unit 310 Interpolation Coefficient data 320 Decoded image data 321 Corrected decoded image data 330 Inverse quantization coefficient data

 ──────────────────────────────────────────────────続 き Continuing on the front page (72) Inventor Hiroshi Koshi 430, Nakai-cho, Ashigarakami-gun, Kanagawa Prefecture Green Tech Nakai Inside Fuji Xerox Co., Ltd.

Claims (44)

[Claims] 1. An image input unit for inputting an image, a frequency conversion unit for performing frequency conversion for obtaining a frequency component of an image input by the image input unit, and a threshold processing for the frequency component obtained by the frequency conversion unit Threshold processing means, and according to a result of threshold processing by the threshold processing means,
A low-frequency image output means for outputting an image of a low-frequency component of the image input by the image input means; and an output by the low-frequency image output means in accordance with a result of threshold processing by the threshold processing means Pixel thinning means for performing a predetermined thinning process on the processed image, coefficient information output means for outputting a result of threshold processing by the threshold processing means, and thinning for outputting an image thinned by the pixel thinning means An image encoding apparatus comprising: an image output unit. 2. An image input unit for inputting an image, a frequency conversion unit for performing frequency conversion for obtaining a frequency component of the image input by the image input unit, and a threshold process for the frequency component obtained by the frequency conversion unit Threshold processing means for performing, threshold processing by the threshold processing means, high-frequency coefficient masking means for replacing high-frequency components among the frequency components obtained by the frequency conversion means with 0, the high-frequency coefficients An inverse transform unit that performs an inverse frequency transform for converting a frequency component obtained by replacing a high frequency component with 0 by a mask unit into an image, and according to a result of threshold processing performed by the threshold processing unit,
A pixel thinning unit that performs a predetermined thinning process on the image converted by the inverse conversion unit; a coefficient information output unit that outputs a result of the threshold processing by the threshold processing unit; and a thinning process performed by the pixel thinning unit. An image encoding apparatus comprising: a thinned image output unit that outputs a compressed image. 3. An image input unit for inputting an image, a frequency conversion unit for performing frequency conversion for obtaining a frequency component of the image input by the image input unit, and a threshold process for the frequency component obtained by the frequency conversion unit Threshold processing means, and according to a result of threshold processing by the threshold processing means,
A pixel thinning unit that performs a predetermined thinning process on the image input by the image input unit; a coefficient information output unit that outputs a result of the threshold processing by the threshold processing unit; and a thinning process that is performed by the pixel thinning unit. An image encoding apparatus comprising: a thinned image output unit that outputs a compressed image. 4. An image input unit for inputting an image, a pseudo-decoded image generation unit for generating a pseudo-decoded image of the image input by the image input unit by a predetermined thinning process and a predetermined interpolation process, Coefficient analysis means for obtaining a thinning rate based on an error between the pseudo decoded image generated by the decoded image generation means and the image input by the image input means, and according to the thinning rate obtained by the coefficient analysis means,
A pixel thinning unit that performs a predetermined thinning process on the image input by the image input unit; a coefficient information output unit that outputs a thinning rate obtained by the coefficient analyzing unit; and a thinning process that is performed by the pixel thinning unit. An image encoding apparatus comprising: a thinned image output unit that outputs a compressed image. 5. An error used in the coefficient analysis means,
5. The pixel value error, the maximum value of any one of the absolute value of the error and the square value of the error, or one of the dynamic range, the variance, and the S / N ratio.
An image encoding device according to claim 1. 6. The method according to claim 4, wherein the predetermined interpolation processing of the pseudo-decoded image generation means is one of nearest neighbor interpolation, four-point linear interpolation, nine-point secondary interpolation, and low-pass filter processing. An image encoding device according to claim 1. 7. A code input means for inputting a code obtained by performing frequency conversion and entropy coding on an image, and an inverse of entropy coding performed on the code input by the code input means. Entropy decoding means for obtaining a frequency component by performing decoding corresponding to the conversion, threshold processing means for performing threshold processing on the frequency component obtained by the entropy decoding means, according to the result of threshold processing by the threshold processing means,
High-frequency coefficient masking means for replacing high-frequency components of the frequency components obtained by the entropy decoding means with 0, and converting the frequency components in which the high-frequency components have been replaced with 0 by the high-frequency coefficient masking means into an image. Inverting means for performing an inverse frequency conversion, and according to the result of threshold processing by the threshold processing means,
A pixel thinning unit that performs a predetermined thinning process on the image converted by the inverse conversion unit; a coefficient information output unit that outputs a result of the threshold processing by the threshold processing unit; and a thinning process that is performed by the pixel thinning unit. An image encoding apparatus comprising: a thinned image output unit that outputs a compressed image. 8. The decoding of the entropy decoding means includes:
The image encoding device according to claim 7, wherein the image encoding device is one of Huffman encoding, arithmetic encoding, and QM encoding. 9. An image input unit for inputting an image, a frequency conversion unit for performing frequency conversion for obtaining a frequency component of the image input by the image input unit, and a threshold process for the frequency component obtained by the frequency conversion unit Threshold processing means, and according to a result of threshold processing by the threshold processing means,
High-frequency coefficient masking means for replacing high-frequency components with 0 among the frequency components obtained by the frequency converting means, and converting the frequency components in which the high-frequency components have been replaced with 0 by the high-frequency coefficient masking means into an image. Inverting means for performing an inverse frequency conversion, and according to the result of threshold processing by the threshold processing means,
Pixel thinning means for performing a predetermined thinning process on the image converted by the inverse converting means; and data synthesis for synthesizing the thinned image obtained by the pixel thinning means and the result of threshold processing by the threshold processing means Means, and combined data output means for outputting combined data combined by the data combining means. 10. An image input unit for inputting an image, coefficient information input unit for inputting coefficient information, frequency conversion unit for performing frequency conversion for obtaining a frequency component of the image input by the image input unit, and the coefficient A high-frequency coefficient masking means for replacing high-frequency components among the frequency components obtained by the frequency converting means with 0 according to the coefficient information input by the information inputting means; and a high-frequency coefficient by the high-frequency coefficient masking means. An inverse transform unit for performing an inverse frequency transform for converting a frequency component in which the component is replaced with 0 into an image, and an image transformed by the inverse transform unit according to the coefficient information input by the coefficient information input unit. Pixel thinning means for performing a predetermined thinning process; coefficient information output means for outputting coefficient information input by the coefficient information input means; Image encoding device characterized by comprising a thinned image output means for outputting the thinned processed image by thinning means. 11. The image processing apparatus according to claim 1, further comprising image encoding means for performing image encoding on the thinned image output by said thinned image output means.
Or the image encoding device according to 10. 12. The image coding apparatus according to claim 11, wherein the image coding performed by said image coding means is one or both of lossless coding and predictive coding. 13. The image according to claim 1, further comprising a coefficient information encoding unit that performs information source encoding on the coefficient information output by the coefficient information output unit. Encoding device. 14. The frequency transform performed by the frequency transforming means and the inverse transforming means is a discrete cosine transform, a Fourier transform, a discrete sine transform, a sub-band transform, or a wavelet transform. ,
The image encoding device according to claim 10. 15. The image coding apparatus according to claim 1, wherein the threshold processing of the threshold processing means is a threshold processing using a predetermined quantization table as a threshold. 16. The image coding apparatus according to claim 15, wherein the quantization table used by said threshold value processing means can be set externally. 17. The image coding apparatus according to claim 2, wherein said high-frequency coefficient masking means replaces a component larger than a highest frequency component with 0 by said threshold value processing means. 18. The thinning-out processing of the pixel thinning-out means,
18. The image coding apparatus according to claim 1, wherein the coding is performed in accordance with a ratio derived from a distribution of a highest frequency component or a non-zero frequency component in the block. 19. The method according to claim 1, wherein the rate of thinning-out processing performed by said pixel thinning-out means is quantized to a predetermined value.
9. The image encoding device according to 8. 20. The predetermined thinning-out processing of the pixel thinning-out means includes: leaving pixels in a grid, performing the same ratio in a vertical direction and a horizontal direction, performing the remaining pixels at substantially equal intervals, 19. The image encoding apparatus according to claim 1, wherein a peak value in a pixel is preferentially left. 21. The thinning-out processing of the pixel thinning means,
21. The method according to claim 1, wherein when an image input by said image input means has already been encoded by an image encoding device, the same pixel as the previously thinned pixel is eliminated. An image encoding device according to claim 1. 22. The image coding apparatus according to claim 1, further comprising a pixel value quantizing means for quantizing a pixel value of the image thinned out by said pixel thinning means. 23. The pixel value quantization unit changes a quantization step according to a result of threshold processing by the threshold processing unit, or changes a quantization step according to a threshold value used by the threshold processing unit. 23. The image encoding apparatus according to claim 22, wherein a quantization step is changed. 24. An image processing apparatus according to claim 1, further comprising image determination means for performing a predetermined analysis on the image input by said image input means and determining a threshold value used by said threshold value processing means. An image encoding device according to claim 1. 25. The image determining means determines a difference between a natural image and an artificial image, and sets a threshold value to 0 in the case of an artificial image so that a frequency component which is set to 0 in the threshold processing by the threshold processing means does not occur. The image encoding apparatus according to claim 24, wherein the apparatus performs control. 26. The predetermined analysis processing of the image determination means includes a measurement of a dynamic range of a pixel value, a measurement of a histogram of a pixel value, a measurement of entropy of a lower bit of a pixel value, a measurement of a steepness of an edge, and a measurement of a line. Measurement of thickness, measurement of frequency components, specified from outside, or edge,
The image encoding apparatus according to claim 24, wherein at least one component among a pattern, a gradation, and a line is detected. 27. A coefficient information input means for inputting coefficient information, a thinned image input means for inputting a thinned image, a thinned image input by the thinned image input means, and coefficient information input by the coefficient information input means. According to the coefficient interpolation means for calculating a frequency component by a predetermined method, an inverse conversion means for performing an inverse frequency conversion for converting the frequency component calculated by the coefficient interpolation means to an image, And a decoded image output means for outputting a decoded image. 28. A coefficient information input means for inputting coefficient information for each block which is a fixed area of an image, a thinned image input means for inputting a thinned image for each block, and a thinned image input by the thinned image input means Pixel value interpolation means for interpolating pixel values by a predetermined method according to the coefficient information input by the coefficient information input means, and decoded image output means for outputting an image interpolated by the pixel value interpolation means. An image decoding device, comprising: 29. The predetermined method of the pixel value interpolation means is as follows:
29. The image decoding apparatus according to claim 28, wherein the image decoding is one of nearest-neighbor interpolation, four-point linear interpolation, nine-point secondary interpolation, and low-pass filter processing. 30. Synthesized data input means for inputting synthesized data which is data obtained by synthesizing coefficient information and a thinned image, and data decomposition for decomposing the synthesized data input by the synthesized data input means into a thinned image and coefficient information. Means, coefficient interpolating means for calculating a frequency component by a predetermined method in accordance with the thinned image decomposed by the data decomposing means and the coefficient information, and converting the frequency component calculated by the coefficient interpolating means into an image. An image decoding apparatus, comprising: an inverse transform unit for performing inverse frequency conversion; and a decoded image output unit for outputting an image converted by the inverse transform unit. 31. An image decoding unit for decoding a code obtained by image-encoding a thinned image into an image, wherein the thinned image input unit uses the image decoded by the image decoding unit as a thinned image. 31. The image decoding device according to claim 27, wherein the image is inputted. 32. The decoding performed by the image decoding means,
32. The image decoding apparatus according to claim 31, wherein the image decoding apparatus performs reverse processing of reversible coding or reverse processing of predictive coding. 33. A pixel value correcting unit for replacing pixels included in the thinned image input by the thinned image input unit in the image converted by the inverse conversion unit with pixel values of the thinned image. The decoded image output means outputs an image corrected by the pixel value correction means.
3. The image decoding device according to 2. 34. The image decoding apparatus according to claim 27, wherein the frequency transform of the inverse transform means is a discrete cosine transform, a Fourier transform, a discrete sine transform, a subband transform, or a wavelet transform. 35. The coefficient interpolation performed by the coefficient interpolation means includes solving a linear simultaneous equation relating to a frequency coefficient and a pixel value, and an inverse matrix previously obtained for the linear simultaneous equation relating to a frequency coefficient and a pixel value. 31. The image decoding apparatus according to claim 27, wherein the calculation is a low-pass filter process on a thinned image or an approximation thereof. 36. Image input means for inputting an image, frequency conversion means for performing frequency conversion for obtaining a frequency component of the image input by the image input means, and threshold processing of the frequency component obtained by the frequency conversion means Threshold processing means for performing, threshold processing by the threshold processing means, high-frequency coefficient masking means for replacing high-frequency components among the frequency components obtained by the frequency conversion means with 0, the high-frequency coefficients A first inverse transform unit that performs inverse frequency transform for converting a frequency component obtained by replacing a high frequency component with 0 by a mask unit into an image, and according to a result of the threshold process performed by the threshold process unit,
A pixel thinning unit that performs a predetermined thinning process on the image converted by the first inverse conversion unit; a coefficient information output unit that outputs a result of the threshold processing by the threshold processing unit; A thinned image output unit that outputs a thinned image; a coefficient information input unit that inputs coefficient information that is a result of threshold processing output by the coefficient information output unit; and a thinned image that is output by the thinned image output unit. A thinned image input means for inputting, a coefficient interpolation means for calculating a frequency component by a predetermined method according to a thinned image input by the thinned image input means and coefficient information input by the coefficient information input means, A second inverse transform unit that performs inverse frequency transform for transforming the frequency component calculated by the coefficient interpolation unit into an image; Picture coding and decoding apparatus, characterized by comprising a decoding image output means for outputting the converted image by the inverse transformation means. 37. A step 1 for inputting an image, a step 2 for performing frequency conversion for obtaining a frequency component of the image input in the step 1, and a step 3 for performing threshold processing on the frequency component obtained in the step 2. A step 4 in which high-frequency components of the frequency components obtained in step 2 are replaced with 0 according to the result of the threshold processing in step 3, and a frequency in which the high-frequency components are replaced with 0 in step 4 Step 5 of performing inverse frequency conversion for converting the component into an image, and Step 6 of performing predetermined thinning processing on the image converted in Step 5 in accordance with the result of the threshold processing in Step 3; Step 7 of outputting the result of the threshold processing in Step 3; Image encoding method characterized by comprising the steps 8 to output an image. 38. A step 1 for inputting coefficient information, a step 2 for inputting a thinned image, and a predetermined method according to the thinned image input in step 2 and the coefficient information input in step 1. A step of calculating a frequency component; a step of performing an inverse frequency conversion of converting the frequency component calculated in the step into an image; and a step of outputting the image converted in the step. An image decoding method characterized by the following. 39. An image input unit for inputting an image, a switching information input unit for inputting switching information, a pixel thinning unit for performing a predetermined thinning process on the input image according to the switching information, Coefficient interpolating means for calculating a frequency component by a predetermined method according to the obtained thinned image and the switching information; coefficient quantizing means for quantizing the frequency component; and outputting the quantized frequency component to the outside. An image encoding device comprising: coefficient output means. 40. The apparatus according to claim 26, wherein said coefficient output means further comprises entropy coding means, and performs variable length coding such as Huffman coding, arithmetic coding, QM coding, universal coding, etc. on said quantized frequency components. The image encoding device according to claim 39, wherein: 41. The method according to claim 39, wherein the frequency component calculated by the coefficient interpolation means is calculated by a discrete cosine transform, a Fourier transform, a discrete sine transform, a subband transform, or a wavelet transform. Image encoding device. 42. The image according to claim 39, wherein the predetermined thinning processing of the pixel thinning means is performed according to the switching information so as to leave a pixel which is not overwritten in the input image. Encoding device. 43. The predetermined thinning-out processing of the pixel thinning-out means includes: leaving pixels in a grid, performing the same ratio in the vertical direction and the horizontal direction, performing the remaining pixels at substantially equal intervals, 43. The method according to claim 39, wherein the peak value in the pixel is preferentially left.
An image encoding device according to claim 1. 44. Coefficient interpolation performed by the coefficient interpolation means includes solving a linear simultaneous equation relating to a frequency coefficient and a pixel value, and an inverse matrix obtained in advance for a linear simultaneous equation relating to a frequency coefficient and a pixel value. 41. A low-pass filter process on a thinned image or an approximation thereof.
44. The image encoding device according to 2 or 43.