JP4085647B2 - Block noise removal method - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention divides image data into blocks, and then separates the second encoded data from the first encoded data obtained by the lossy compression encoding method that performs orthogonal transform, quantization, and variable length encoding. The present invention relates to an image processing apparatus for converting to a first encoded data, and an image processing apparatus and an image processing method for reducing noise generated during encoding of first encoded data when decoding second encoded data.
[0002]
[Prior art]
2. Description of the Related Art In recent years, image data compression / encoding technology has been remarkably advanced and widely used for the purpose of efficient use of storage media and speeding up of image data transmission / reception via a network. Usually, when it is necessary to highly compress image data, a lossy compression encoding method is used, but most of these use the encoding process shown in FIG.
[0003]
FIG. 1 is a block diagram of encoding processing in many lossy compression encoding systems.
[0004]
First, the encoding orthogonal transform unit 11 performs orthogonal transform on the image data for each block divided in units of M × N pixels to calculate orthogonal transform coefficients. As the orthogonal transformation, various transformations such as two-dimensional Fourier transformation, Karhunen-Loeve transformation, and discrete cosine transformation (DCT) can be used, but an 8 × 8 pixel unit widely used in JPEG (Joint Photographic Experts Group) and the like. In the case of DCT, if the orthogonal transform coefficient (DCT coefficient) after conversion is DCT [v] [u] and the image data before conversion is F [y] [x], the conversion formula is expressed as (Equation 1). Is done. Note that (x, y) represents the coordinate value in each block in the image data before conversion, and (u, v) represents the order of the frequency component in the DCT coefficient.
[0005]
DCT [v] [u] = 1/4 × Cu · Cv · ΣΣF [y] [x] · cos {(2x + 1) uπ / 16} · cos {(2y + 1) vπ / 16}
Cu, Cv = 1 / √2 (u, v = 0), 1 (otherwise) (Equation 1)
Next, the quantization unit 12 quantizes the orthogonal transform coefficient calculated in the encoding orthogonal transform unit 11. In general, the M × N size orthogonal transform coefficient calculated in each block is quantized using an M × N size quantization table. For reference, an example of an 8 × 8 size quantization table in JPEG is shown in FIG. Generally, since the set value of the quantization table is the largest factor for determining the compression level, the quantization table setting unit 13 sets a quantization table corresponding to a desired compression level. Here, if the orthogonal transform coefficient after quantization is QDCT [v] [u] and the value of the quantization table is Qtable [v] [u], the quantization is expressed as (Equation 2). Here, INT {a} represents the maximum integer value that does not exceed the value a.
[0006]
QDCT [v] [u] = INT {DCT [v] [u] / Qtable [v] [u] +0.5} (Expression 2)
Finally, the variable length encoding unit 14 performs variable length encoding on the orthogonal transform coefficient quantized by the quantization unit 12 to generate encoded data. As the variable length coding, Huffman coding, arithmetic coding, or the like is used.
[0007]
The above is the outline of the encoding process until the encoded data is obtained from the image data. On the contrary, the decoding process from the encoded data to the image data can be realized by basically performing the encoding process in the reverse order as shown in FIG.
[0008]
FIG. 2 is a block diagram of decoding processing in many irreversible compression encoding systems.
[0009]
First, the variable length decoding unit 21 performs variable length decoding on the encoded data.
[0010]
Next, the quantization table extraction unit 22 extracts information on the quantization table used at the time of encoding from the encoded data.
[0011]
Next, the inverse quantization unit 23 performs inverse quantization of the orthogonal transform coefficient decoded by the variable length decoding unit 21 using the information of the quantization table extracted by the quantization table extraction unit 22. Here, assuming that the inversely quantized orthogonal transform coefficient is RDCT [v] [u], the inverse quantization is expressed as (Equation 3).
[0012]
RDCT [v] [u] = QDCT [v] [u] × Qtable [v] [u] (Equation 3)
Finally, in the decoding inverse orthogonal transform unit 24, the orthogonal transform coefficient inversely quantized in the inverse quantization unit 23 is subjected to inverse orthogonal transform and decoded into image data. In the case of DCT in units of 8 × 8 pixels, if the image data to be decoded is G [v] [u], the conversion formula is expressed as (Equation 4).
[0013]
G [y] [x] = 1/4 · ΣΣCu · Cv · RDCT [v] [u] · cos {(2x + 1) uπ / 16} · cos {(2y + 1) vπ / 16}
Cu, Cv = 1 / √2 (u, v = 0), 1 (otherwise) (Equation 4)
The above is an overview of the encoding process and decoding process used in many lossy compression encoding systems. During this process, orthogonal transform coefficients are quantized, so signal degradation Occurs. This signal degradation appears as image quality degradation in the decoded image data, and noise that did not exist in the original image occurs. Among the noises generated here, there are block distortion and mosquito noise as noises that have a particularly bad visual effect. Block distortion is noise caused by encoding processing performed in units of M × N pixel units, and refers to a phenomenon in which pixel values become stepped at block boundaries of decoded image data. . Mosquito noise is noise caused by the loss of high-frequency components due to quantization, and the strong edges that existed in the original image are not accurately reproduced, and mosquitoes appear to fly around the edges. Say noise.
[0014]
The process of removing these noises and improving the image quality of the decoded image data is generally called an image restoration process, and this prominent conventional method is applied to the convex projection method (A. Zakhor, “Iterative procedures for reduction of reproduction”). Blocking effects in transform image coding, "IEEE Trans. Circuits Syst. Video Technol., vol. 2, pp. 91-95, Mar. 1992). Below, the outline | summary of this convex projection method is demonstrated.
[0015]
In general, smoothing is effective in reducing noise in the image, but if the entire image is smoothed by simple filter processing, the original edges that existed in the original image were used instead of reducing noise. There was a problem that was greatly blurred. In contrast, the convex projection method has been improved to minimize edge blurring by repeatedly performing smoothing processing and projection processing based on constraint conditions alternately. Here, the projecting process based on the constraint condition refers to a process of limiting the influence of the smoothing process so that the orthogonal transform coefficient after the image restoration process becomes a value within the same quantization step as that of the original image. This is because the orthogonal transform coefficient is quantized and changed to a value different from the orthogonal transform coefficient of the original image in the encoding process of the lossy compression coding method, but the error is at most within the range of the quantization step. The orthogonal transform coefficient of the original image is at least within the same quantization step as the orthogonal transform coefficient of the decoded image data, that is, not less than the lower limit value dDCT [v] [u] represented by (Equation 5) And the property that it is guaranteed that it is less than the upper limit value pDCT [v] [u].
[0016]
dDCT [v] [u] = RDCT [v] [u] −0.5 × Qtable [v] [u]
pDCT [v] [u] = RDCT [v] [u] + 0.5 × Qtable [v] [u] (Equation 5)
That is, when the orthogonal transform coefficient after the smoothing process changes outside this range, it can be determined that the original image has a very large change and excessive blurring has occurred. Here, if the orthogonal transform coefficient after the smoothing process is less than the lower limit value dDCT [v] [u], a projection process to the lower limit value dDCT [v] [u] is performed, while the upper limit value pDCT [v] [u] If u] or more, the projection processing to the upper limit value pDCT [v] [u] is performed. Thereby, noise can be reduced while preventing excessive blurring.
[0017]
FIG. 3 shows a specific processing flow when the convex projection method is applied. FIG. 3 is a block diagram of image restoration processing by the convex projection method. However, the processes of the variable length decoding unit 21 ′, the quantization table extraction unit 22 ′, the inverse quantization unit 23 ′, and the decoding inverse orthogonal transform unit 24 ′ in FIG. 3 are respectively performed by the variable length decoding unit 21 in FIG. Since the processing is the same as that of the quantization table extraction unit 22, the inverse quantization unit 23, and the decoding inverse orthogonal transform unit 24, detailed description thereof is omitted here.
[0018]
First, the constraint condition setting unit 31 uses the information of the quantization table obtained by the quantization table extraction unit 22 ′ and the orthogonal transform coefficient after inverse quantization obtained by the inverse quantization unit 23 ′ to calculate the original image. A lower limit value dDCT [v] [u] and an upper limit value pDCT [v] [u] that are in the same quantization step range as the orthogonal transform coefficient are calculated.
[0019]
Next, the smoothing processing unit 32 smoothes the image data decoded by the decoding inverse orthogonal transform unit 24 ′ by a filter process.
[0020]
The orthogonal transform unit 33 performs the same orthogonal transform as the encoded orthogonal transform unit 11 and calculates orthogonal transform coefficients from the image data smoothed by the smoothing processing unit 32.
[0021]
In the projection processing unit 34, the lower limit value dDCT [v] [u] of the orthogonal transform coefficient calculated by the constraint condition setting unit 31 and the upper limit value pDCT [v] for the orthogonal transform coefficient calculated by the orthogonal transform unit 33. ] Projection processing is performed based on [u].
[0022]
The inverse orthogonal transform unit 35 performs the same inverse orthogonal transform as that of the decoding inverse orthogonal transform unit 24, and decodes the orthogonal transform coefficients subjected to the projection processing into new image data.
[0023]
The end determination unit 36 determines whether to end or further continue the image restoration process. When continuing the image restoration processing, the processing from the smoothing processing unit 32 to the inverse orthogonal transform unit 35 is repeated again. By repeating this process, the noise of the decoded image data is further reduced, but the blur of the original edge gradually increases. For this reason, as an end determination condition setting method, a method for setting to end when the image restoration process is repeated a predetermined number of times set in advance, or an evaluation value for image quality is calculated, and this evaluation value is calculated. For example, a method for setting so that it ends when a specific condition is satisfied.
[0024]
[Problems to be solved by the invention]
By applying such an image restoration process, it is possible to reduce block distortion and mosquito noise caused by the lossy compression encoding method and improve image quality. However, image restoration processing such as the convex projection method requires information such as a quantization table used at the time of encoding with the lossy compression encoding method. For this reason, once encoded data of the lossy compression encoding method is decoded and then converted to another encoded data, the quantization table used at the time of encoding of the lossy compression method is used. Since information is lost, there is a problem that image restoration processing such as a convex projection method cannot be applied.
[0025]
[Means for Solving the Problems]
In order to solve the above-described problems, in the present invention, image processing for converting encoded data of an irreversible compression encoding method (hereinafter referred to as encoded data A) into another encoded data (hereinafter referred to as encoded data B). In the method , an encoding information extraction step for extracting encoding information such as a quantization table used for encoding the encoded data A at the time of decoding the encoded data A, and a code extracted in the encoding information extraction step Reducing noise generated during encoding of the encoded data A by including a step of storing the encoded information together with the encoded data B and an image recovery step of performing an image recovery process using the encoded information Provided is a block noise removal method capable of
[0028]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, embodiments of the present invention will be described with reference to the drawings. 4 is a block diagram showing a process of converting into another encoded data while retaining the encoding information related to the encoded data of the lossy compression encoding system.
[0029]
First, the encoding scheme determination unit 101 determines the type of encoding scheme used for the encoded data A.
[0030]
Next, the encoding information extraction unit 102 extracts encoding information such as a quantization table used when encoding the encoded data A. The encoded information extracted here is held in the encoded data B by the processing described later. The type of encoded information to be stored is the encoding used for the encoded data A. It depends on the type of method. For this reason, the type of encoding information extracted by the encoding information extraction unit 102 is determined according to the type of encoding method of the encoded data A determined by the encoding method determination unit 101.
[0031]
Next, the decoding unit 103 performs a decoding process from the encoded data A to the image data. The decoding process performed by the decoding unit 103 is the same as the processes of the variable length decoding unit 21, the inverse quantization unit 23, and the decoding inverse orthogonal transform unit 24 in FIG.
[0032]
Finally, the encoding unit 104 converts the image data decoded by the decoding unit 103 into another encoded data B. At this time, the encoded data B also stores encoded information related to the encoded data A extracted by the encoded information extraction unit 102. Here, as a method for storing the encoded information in the encoded data B, for example, an arbitrary text character string together with encoded image data such as PNG (Portable Network Graphics) and TIFF (Tag Image File Format) is used. When a specification that can be stored is determined, encoded information can be stored using this specification. In addition, in the case of a specification in which a text character string cannot be stored as described above, encoded information can be stored by a method using another file with a specific name.
[0033]
The above process is an embodiment of the process of converting to another encoded data B while retaining the encoded information related to the encoded data A of the lossy compression encoding system.
[0034]
Next, when decoding encoded data B, the flow of a process of extracting the encoding information related to the encoded data A and applying the image restoration process is shown in FIG. 5 and will be described in detail.
[0035]
FIG. 5 is a block diagram of a process for extracting the encoded information related to the previous encoded data from the encoded data and applying the image restoration process by the convex projection method.
[0036]
First, the second decoding unit 201 performs a decoding process from the encoded data B to the image data.
[0037]
Next, the pre-encoding information extraction unit 202 extracts encoding information regarding the encoded data A from the encoded data B.
[0038]
The second orthogonal transform unit 203 performs orthogonal transform on the image data decoded by the second decoding unit 201 and calculates orthogonal transform coefficients.
[0039]
Next, the constraint condition setting unit 31 ′ performs projection processing from the orthogonal transform coefficient calculated by the second orthogonal transform unit 203 and the encoded information related to the encoded data A extracted by the pre-encoded information extracting unit 202. The lower limit value dDCT [v] [u] and the upper limit value pDCT [v] [u] of the range in which the orthogonal transformation coefficient is projected by the unit 34 ′ are calculated.
[0040]
Hereinafter, the process of performing image restoration by the convex projection method on the image data decoded by the second decoding unit 201 is the same as the process of the normal convex projection method in FIG. 3 described above. Detailed description is omitted.
[0041]
The image restoration processing is performed by using the encoding information such as the quantization table used when encoding the previous lossy compression encoding method even after being converted into another encoded data by the above processing. Is possible.
[0042]
It should be noted that, as encoded information related to the encoded data A, color space information subjected to orthogonal transformation can be stored in the encoded data B. When decoding encoded data generated by encoding a color image, information on a color space when orthogonal transformation is performed is necessary. For this reason, regarding the encoding process of the encoded data A, the information of the color space at the time of orthogonal conversion is held in the encoded data B, thereby encoding the color image orthogonally converted in an arbitrary color space. Also for the data A, it is possible to apply the image restoration process when the encoded data B is decoded.
[0043]
In addition, as the encoded information related to the encoded data A, sub-sampling information indicating decimation of information can be stored in the encoded data B. In general, encoding is often performed by thinning out the data of the color difference component with respect to the luminance component. For this reason, with respect to the encoding process of the encoded data A, by holding the sub-sampling information at the time of the orthogonal transformation, the encoded data A which is arbitrarily sub-sampled and encoded can be It is possible to apply image restoration processing when decoding the encoded data B.
[0044]
Note that when the encoded data A is decoded, the image data can be viewed on a display device (not shown), and only when the encoding process from the input device (not shown) to the encoded data B is supported. It can also be realized in the form of performing the subsequent processing.
[0045]
In addition, after converting from the encoded data A to the encoded data B, when further converting from the encoded data B to another encoded data (hereinafter, encoded data C), it is stored in the encoded data B. The encoded information related to the encoded data A that has been encoded can be stored in the encoded data C together with the encoded information related to the encoded data B, so that the history of the encoded information previously encoded is maintained. It becomes possible.
[0046]
【The invention's effect】
As described above, by applying the image processing apparatus and the image processing method proposed in the present invention, the first encoded data by the lossy compression encoding method is converted into another second encoded data. In addition, it is possible to perform image restoration processing using the encoding information related to the first encoded data.
[Brief description of the drawings]
FIG. 1 is a block diagram of an encoding process in many irreversible compression encoding systems. FIG. 2 is a block diagram of a decoding process in many irreversible compression encoding systems. FIG. 3 is an image restoration process using a convex projection method. Block diagram [FIG. 4] A block diagram of a process of converting to another encoded data while retaining the encoding information related to the encoded data of the lossy compression encoding method. [FIG. 5] related to the previous encoded data from the encoded data. Block diagram of processing to extract encoding information and apply image restoration processing by convex projection method [FIG. 6] Example of quantization table [Description of reference symbols]
101 Coding method determination unit 102 Encoding information extraction unit 103 Decoding unit 104 Encoding unit

Claims (2)

A block noise removal method for image data,
Encoded data that is encoded using an irreversible compression encoding method in which image data is orthogonally transformed for each block in units of M × N pixels, orthogonal transformation data is quantized using a quantization table, and variable length coding is performed. Entering A;
From the encoded data A, while extracting the quantization table used for the encoding, a first decoding step of obtaining the first image data by decoding the encoded data A,
A second encoding step of encoding the first image data by PNG and storing the extracted quantization table and outputting encoded data B;
A second decoding step of decoding the encoded data B to obtain second image data;
An encoded information extraction step of extracting the stored quantization table from the encoded data B;
An orthogonal transformation step of performing orthogonal transformation on the second image data and calculating an orthogonal transformation coefficient;
Removing block distortion of the second image data by applying a convex projection method using said encoding-information extracting orthogonal transform coefficient calculated in the extracted quantization table said orthogonal conversion step in step Image restoration step,
A method for removing block noise.   The block noise removing method according to claim 1, wherein the second encoding step stores the quantization table in a text area of the encoded data B.

Images