JP2000278681A - Image coder, its method and served medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To generate a picture in a high-order layer by which a picture closer to an original picture can be generated. SOLUTION: An area optimization circuit 91 simultaneously calculates values of a plurality of pixels included in a target area decided by a target area decision circuit 92 and outputs the result to a high-order layer picture memory 21. The high-order layer picture memory 21 uses a received pixel value to update the pixel value having so far been stored. Since the pixel value of the original picture is compared with the pixel value of a 2nd picture and the stored pixel value is updated in the unit of areas corresponding to the result of comparison, the high-order layer picture by which a picture closer to the original picture can be generated is produced.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image encoding apparatus and method, and a providing medium, and more particularly to an image encoding apparatus and method for generating an image that can be restored to an original image, and an providing medium.

[0002]

2. Description of the Related Art An upper-layer image composed of a smaller number of pixels than the number of pixels constituting an original image is generated, and a lower-layer image substantially identical to the original image is generated from the upper-layer image (the original image is generated by Restoring) technology has been proposed by the present applicant.

[0003] In this proposal, the pixel value of the lower hierarchical image is calculated by calculating the pixel value of the prediction tap centered on the pixel of interest in the upper hierarchical image and the linear first order of the prediction coefficient corresponding to the class code into which the pixel of interest is classified. It is obtained by calculating the combination. The class code of the pixel of interest is determined from the pixel value of the class tap composed of the pixel of interest and its neighboring pixels.

Therefore, in order to generate an upper layer image capable of generating a lower layer image substantially equal to the original image, it is ideal to simultaneously optimize the pixel value and the class code (prediction coefficient).

[0005]

However, in order to simultaneously optimize the pixel value and the class code (prediction coefficient), it is necessary to simultaneously change the pixel values of all the pixels of the upper hierarchical image. Therefore, there is a problem that it is practically impossible to optimize the pixel value and the class code (prediction coefficient) at the same time because the calculation amount becomes enormous.

Therefore, it is conceivable to focus on a predetermined pixel of the upper hierarchical image and optimize the pixel value of the pixel of interest (pixel of interest) on a pixel-by-pixel basis. In this case, the pixel value of the pixel of interest is optimized. However, for pixels other than the target pixel (pixels whose pixel values have already been optimized), there is a problem that the optimized pixel value of the target pixel is not always optimal.

The present invention has been made in view of such a situation, and by optimizing pixel values for each area of an upper layer image, an upper layer image capable of generating an image closer to the original image can be generated. Is to do so.

[0008]

An image encoding apparatus according to claim 1, wherein n pixel values of a first image are generated using m pixel values of an original image. And a prediction coefficient generation means for generating a prediction coefficient using the pixel value of the original image and the pixel value of the first image; and storing a predetermined portion of the pixel data of the first image as a pixel value; A pixel data storage unit for storing a prediction coefficient in association with a class code, a pixel value which is a predetermined portion of pixel data of the first image, Calculating means for calculating a pixel value of the second image by using a prediction coefficient corresponding to a class code which is another part of the pixel data of the original image;
A comparison means for comparing the pixel values of the image of the image, a prediction coefficient update means for updating the prediction coefficient stored in the prediction coefficient storage means in accordance with the comparison result of the comparison means, and a comparison result of the comparison means. And updating the class code stored in the pixel data storage means in accordance with the comparison result of the area pixel value updating means for updating the pixel value stored in the pixel data storage means for each area and the comparison means. And a class code updating means.

According to a second aspect of the present invention, there is provided an image encoding method, comprising: a pixel value generating step of generating n pixel values of a first image using m pixel values of an original image; A prediction coefficient generation step of generating a prediction coefficient using the value and the pixel value of the first image; storing a predetermined portion of the pixel data of the first image as a pixel value; and storing the other portion as a class code. A pixel data storage step, a prediction coefficient storage step of storing a prediction coefficient in association with a class code, a pixel value which is a predetermined portion of the pixel data of the first image, and other pixel values of the pixel data of the first image. A calculating step of calculating a pixel value of a second image using a prediction coefficient corresponding to a class code which is a part, a comparing step of comparing a pixel value of an original image with a pixel value of a second image, and a comparing step Comparison result Correspondingly, a prediction coefficient update step of updating the prediction coefficient stored in the prediction coefficient storage step, and an area for updating the pixel value stored in the pixel data storage step in area units corresponding to the comparison result of the comparison step It is characterized by including a pixel value updating step and a class code updating step of updating the class code stored in the pixel data storage step in accordance with the comparison result of the comparison step.

[0010] According to a third aspect of the present invention, there is provided a recording medium, comprising:
A pixel value generating step of generating n pixel values of a first image using the pixel values of the first image;
A predictive coefficient generating step of generating a predictive coefficient using the pixel values of the image, and a pixel data storing step of storing a predetermined part of the pixel data of the first image as a pixel value and storing the other part as a class code A prediction coefficient storing step of storing a prediction coefficient in association with a class code, a pixel value which is a predetermined portion of the pixel data of the first image, and a class which is another portion of the pixel data of the first image. Calculating a pixel value of the second image using a prediction coefficient corresponding to the code;
A comparison step of comparing the pixel values of the images of the images, a prediction coefficient update step of updating the prediction coefficient stored in the prediction coefficient storage step, and a comparison result of the comparison step. An area pixel value updating step of updating the pixel value stored in the pixel data storage step in units of area, and a class code updating step of updating the class code stored in the pixel data storage step in accordance with the comparison result of the comparison step And a computer-readable program for causing the image encoding apparatus to execute the process including the steps.

[0011] In the image encoding device according to the first aspect, the image encoding method according to the second aspect, and the providing medium according to the third aspect, the pixel value of the original image is calculated using m pixel values. N pixel values of the first image are generated, a prediction coefficient is generated using the pixel values of the original image and the pixel values of the first image, and a predetermined portion of the pixel data of the first image is defined as a pixel value. And the other parts are stored as class codes. Further, the prediction coefficient is stored in association with the class code, and the prediction value corresponding to the pixel value, which is a predetermined portion of the pixel data of the first image, and the class code, which is another portion of the pixel data of the first image, is stored. The pixel value of the second image is calculated using the coefficient, the pixel value of the original image is compared with the pixel value of the second image, and the prediction coefficient stored is stored in correspondence with the comparison result. The class code is updated, and the stored pixel values are updated for each area.

[0012]

DESCRIPTION OF THE PREFERRED EMBODIMENTS A configuration example of an encoder to which the present invention is applied will be described with reference to FIG. Note that the original image input to the encoder 1 has an 8-bit pixel value per pixel, and the upper layer image generated by the encoder 1 also has an 8-bit information amount (pixel data) per pixel. And

In the following, a pixel of interest is a pixel specified to specify a position without updating a pixel value, and a pixel of interest is specified to specify a position.
In addition, it is assumed that the pixel value is updated.

In the encoder 1, an original image is processed by a pre-processing circuit 2, a pixel value updating circuit 6, a prediction coefficient updating circuit 7,
It is supplied to the class code selection circuit 8 and the convergence determination circuit 10. The preprocessing circuit 2 uses the supplied original image,
An initial upper layer image is generated and stored in the upper layer image memory 3, and an initial prediction coefficient table is generated and stored in the prediction coefficient memory 4.

The upper layer image memory 3 outputs the stored upper layer image to the selector 5. The upper layer image memory 3 updates the 4 bits on the MSB (Most Significant Bit) side of the upper layer image stored up to that point using the pixel value (4 bits) input from the pixel value updating circuit 6. Using the class code (4 bits) input from the class code selection circuit 8, the LSB (Least Significant Bit) side 4 bits of the upper layer image stored so far are updated.

The prediction coefficient memory 4 supplies the stored prediction coefficient table to the pixel value updating circuit 6, the class code selection circuit 8, the decoding circuit 9, and the convergence determination circuit 10. Further, the prediction coefficient memory 4 updates the stored prediction coefficient table using the prediction coefficients input from the prediction coefficient update circuit 7.

The selector 5 operates in response to a control signal input from the update counter 11 to control the upper-layer image memory 3.
Are sequentially output to a pixel value update circuit 6, a prediction coefficient update circuit 7, and a class code selection circuit 8 in addition to the decode circuit 9 and the convergence determination circuit 10.

The pixel value updating circuit 6 updates the pixel value (4 bits on the MSB side of the pixel data) of the upper layer image input from the selector 5 using the original image and the prediction coefficient, and outputs the updated pixel value to the upper layer image. I do. The prediction coefficient update circuit 7 generates a prediction coefficient using the upper layer image and the original image input from the selector 5, and outputs the prediction coefficient to the prediction coefficient memory 4.

The class code selection circuit 8 selects an optimum prediction coefficient in the prediction coefficient table stored in the prediction coefficient memory 4 for each pixel of the upper hierarchical image input from the selector 5, and selects the prediction coefficient. Is output to the upper layer image memory 3.

The decoding circuit 9 generates a lower hierarchical image using the upper hierarchical image input from the selector 5 and the prediction coefficients stored in the prediction coefficient memory 4, and outputs it to the convergence determining circuit 10.

The convergence judging circuit 10 calculates the S / N of the lower hierarchical image and the original image input from the decoding circuit 9, further calculates the increase, and judges that the increase of the S / N has converged. In this case, the upper layer image input from the selector 5 and the prediction coefficient table input from the prediction coefficient memory 4 are output to the subsequent stage. Also, when a control signal is input from the update counter 11, the convergence determination circuit 10 determines whether the upper-layer image and prediction coefficient memory 4
And outputs the prediction coefficient table input from to the subsequent stage.

The update number counter 11 is provided for the convergence judgment circuit 1.
0, the pixel value update circuit 6, the prediction coefficient update circuit 7, or the class code selection circuit 8 outputs a control signal to the selector 5 in response to the end of its processing, and determines the number of times the control signal has been output. It counts and outputs a control signal to the convergence determination circuit 10 when the counted value reaches a predetermined number.

Next, the operation of the encoder 1 will be described with reference to FIG.
This will be described with reference to the flowchart of FIG. This encoding process is started when the original image is input to the encoder 1. In step S1, the preprocessing circuit 2 performs preprocessing using the input original image.

This pre-processing includes steps S11 and S12 as shown in FIG. The initial upper layer image generation process in step S11 will be described with reference to the flowchart in FIG. In step S21, the preprocessing circuit 2 converts the input original image into a predetermined size (for example, FIG.
(3 × 3 pixels) as shown in FIG.

In step S22, the preprocessing circuit 2
Is the average of the pixel values of a plurality of (9 in this case) pixels included in the block divided in step S21, and the 4 bits on the MSB side of the average value (8 bits) are set as one of the initial upper layer images. As shown in FIG. 6, the pixel value of the pixel is recorded in the 4 bits on the MSB side of the upper layer image data (8 bits).

However, in all circuits that use the 4 bits recorded on the MSB side of the upper layer image data as a pixel value, the 4-bit value is converted to 8 bits and used.
If a value obtained by simply multiplying the 4-bit value on the MSB side by 16 is used, the luminance value of the entire image is reduced. To suppress this, a predetermined offset value (for example,
The value obtained by adding 7 or 8) is used as a pixel value.

The method of determining the pixel value of the initial upper hierarchical image is not the above-described method based on averaging, but
A method of directly thinning out the original image or a method of thinning out after using a low-pass filter (such as a Gaussian filter) may be used.

Referring back to FIG. After the processing in step S11, an initial prediction coefficient generation processing is executed in step S12. This processing will be described with reference to the flowchart in FIG. In step S31, the preprocessing circuit 2 determines the pixel of the initial upper layer image for which only the pixel value (4 bits on the MSB side of the pixel data (8 bits)) in step S11 of FIG. I do.

In step S32, the preprocessing circuit 2
Extracts the pixel value (4 bits each) of the class tap corresponding to the target pixel (for example, a target pixel and a total of five pixels located above, below, left, and right thereof).

In step S33, the preprocessing circuit 2
Is obtained by adding 1 bit ADRC (Adaptive Dynamic Range Coding) to the five pixel values (4 bits on the MSB side) extracted in step S32.
ding) processing to convert each to 1 bit, and arrange them in a predetermined order corresponding to the position of the pixel, for example, to obtain a 5-bit class code.
Further, the preprocessing circuit 2 assigns an arbitrary value (for example, the 4 bits on the MSB side of the 5-bit class code) to the 4 bits on the LSB side of the pixel data (8 bits) of the target pixel as shown in FIG. Record.

In step S34, the preprocessing circuit 2
Is a predetermined size (for example, 5 ×
The pixel value of the prediction tap (5 pixels) is extracted. Step S
At 35, the preprocessing circuit 2 generates a normal equation consisting of the known original image and the pixel values of the prediction taps, and the unknown prediction coefficients.

In step S36, the pre-processing circuit 36
Determines whether all the pixels of the initial upper layer image have been set as the target pixels, and repeats the processing of steps S31 to S36 until it determines that all the pixels have been set as the target pixels. If it is determined in step S36 that all the pixels have been set as the target pixel, the process proceeds to step S37.

In step S37, the pre-processing circuit 37
Generates the normal equation generated in step S35 for each 5-bit (32 types) class, and applies the least squares method to the equation to obtain prediction coefficients corresponding to the 32 types of class codes. In step S38, the preprocessing circuit 2 generates a histogram indicating the number of target pixels classified into each class code in step S33.

In step S39, the preprocessing circuit 2
Refers to the histogram generated in step S38,
The upper 16 prediction coefficients (the prediction coefficients of the 16 class codes having a large number) are arbitrarily associated with the class codes represented by 4 bits (0000 to 1111).

In step S40, the preprocessing circuit 2
Outputs a prediction coefficient associated with a 4-bit (16 types) class code to the prediction coefficient memory 4. Further, the preprocessing circuit 2 outputs to the upper layer image memory 3 an initial upper layer image including pixel data in which the 4 bits on the MSB side are a pixel value and the 4 bits on the LSB side are a class code.

As a method of generating an initial prediction coefficient other than the above-described processing, a random number may be associated with a class code (0000 to 1111) represented by 4 bits. Further, the class code may be a random number, the prediction coefficient may be a random number, or both the class code and the prediction coefficient may be random numbers.

Returning to FIG. After the pre-processing of step S1 is performed as described above, in step S2, the class code selection circuit 8 stores in the prediction coefficient memory 4 for each pixel of the input upper layer image. 16
Select the best of the types of prediction coefficients. In addition,
A configuration example and operation of the class code selection circuit 8 will be described later with reference to FIGS.

In step S3, a decoding process is performed. That is, the upper layer image input to the selector 5 from the upper layer image memory 3
Is supplied to the decoding circuit 9 in response to the control signal from The decoding circuit 9 uses the upper layer image (in this case, the initial upper layer image) input from the selector 5 and the prediction coefficient table (in this case, the initial prediction coefficient table) supplied from the prediction coefficient memory 4. A lower layer image is generated and output to the convergence determination circuit 10. As shown in FIG. 5, as a pixel of the lower hierarchical image, 3 × 3 pixels (pixels) centering on the pixel i of the lower hierarchical image at the corresponding position with respect to one target pixel of the upper hierarchical image a to i) are generated. The configuration and operation of the decoding circuit 9 will be described later with reference to FIGS.

In step S4, the convergence determination circuit 10
Calculates the S / N of the lower hierarchical image and the original image input from the decoding circuit 9, further calculates the increase, and determines whether or not the increase in the S / N has converged. If it is determined that the S / N increase has converged, or if a control signal from the update counter 11 has been received, the process proceeds to step S7. If it is determined that the S / N increase has not converged, and if the control signal from the update counter 11 has not been received, the process proceeds to step S5.

In this case, since the convergence determination process is performed on the lower-layer image decoded for the first time, the amount of increase in S / N is not calculated, and the control signal from the update number counter 11 is not received. Proceed to step S5.

In response to the end of the determination processing of the convergence determination circuit 10, the update number counter 11 outputs a control signal to the selector 5.

In step S 5, the upper layer image input to the selector 5 from the upper layer image memory 3 is supplied to the pixel value updating circuit 6 in accordance with the control signal from the update counter 11. The pixel value updating circuit 6 updates the pixel value (4 bits on the MSB side of the pixel data) of the input upper-layer image.

Before describing the pixel value updating process, a first configuration example of the pixel value updating circuit 6 will be described with reference to FIGS. The upper layer image input from the selector 5 is stored in the upper layer image memory 21 in the pixel value updating circuit 6. Upper layer image memory 2
1 supplies the stored upper layer image to the optimum pixel value determination circuit 22. The upper layer image memory 21 uses the optimized pixel value (4 bits) from the optimum pixel value determination circuit 22 to store the pixel value of the upper layer image stored up to that point (the pixel data on the MBS side). 4 bits).
The upper layer image in which the pixel values of all the pixels are optimized is output to the upper layer image memory 3 via the switch 24.

The original image and the prediction coefficient table from the prediction coefficient memory 4 are also supplied to the optimum pixel value determination circuit 22. The optimum pixel value determination circuit 22 includes a target pixel determination circuit 23
Optimizes the pixel value of the target pixel designated by, and outputs it to the upper layer image memory 21. The attention pixel determination circuit 23
The pixels of the upper layer image are sequentially determined as the pixel of interest, and the information is output to the optimum pixel value determination circuit 22. After determining all the pixels of the upper layer image as the target pixel, the target pixel determination circuit 23 outputs a control signal for turning on the switch 24.

FIG. 9 shows a detailed configuration example of the optimum pixel value determination circuit 22. In the optimum pixel value determination circuit 22, the pixel values of the pixels other than the pixel of interest are fixed,
The pixel value of the target pixel is optimized.

The pixel-of-interest determination circuit 31 determines the range affected by the change of the pixel value of the pixel of interest determined by the pixel-of-interest determination circuit 23 (the pixel at the center of the prediction tap including the pixel of interest). (Hereinafter referred to as a range of influence), and the pixels of the upper layer image existing within the range of influence are sequentially determined as the pixel of interest, and the position information is determined by the class code reading circuit 32 and the prediction tap. Output to the extraction circuit 33. In addition, the target pixel determination circuit 31
After all the pixels within the influence range are determined as the target pixels, a control signal for turning on the switch 36 is output.

The class code reading circuit 32 reads the class code of the pixel of interest (4 bits on the LSB side of the pixel data) and outputs it to the error function generating circuit 34. The prediction tap extraction circuit 33 extracts the pixel value (4 bits on the MSB side of the pixel data) of the prediction tap of 5 × 5 pixels centering on the pixel of interest and outputs it to the error function generation circuit 34.

The error function generation circuit 34 generates an error function (details of which will be described later) corresponding to the pixel of interest and outputs it to the influence error function register 35. The influence error function register 35 adds the error functions corresponding to all the pixels of interest to generate an influence error function, and outputs the result to the pixel-of-interest calculation circuit 37 via the switch 36.

The target pixel value calculation circuit 37 includes a switch 36
The pixel value of the pixel of interest is calculated by solving the influence error function input through the function (the details will be described later).

Next, the operation of the first configuration example of the pixel value updating circuit 6 will be described with reference to the flowchart of FIG. This pixel value updating process is started when the upper layer image input from the selector 5 is stored in the upper layer image memory 21 of the pixel value updating circuit 6.

In step S51, the pixel-of-interest determination circuit 23 determines the pixel of interest in the upper hierarchical image, as shown in FIG.
Output to 2. In step S52, the target pixel determination circuit 31 of the optimum pixel value determination circuit 22 determines a range (influence range) affected when the pixel value of the target pixel is changed. For example, when the size of the prediction tap is 5 × 5 pixels, a range including 5 × 5 pixels centered on the pixel of interest as shown in FIG. 11B (a lower layer image generated from the upper layer image) , The affected area is 15 × 15 pixels as shown in FIG.

In step S53, the pixel-of-interest determination circuit 31 determines one pixel within the affected area as the pixel of interest, and outputs the position information to the class code reading circuit 32 and the prediction tap extracting circuit 33.

In step S54, the class code reading circuit 32 outputs the class code (pixel data
LSB side 4 bits), and an error function generation circuit 34
Output to The prediction tap extraction circuit 33 extracts a prediction tap of 5 × 5 pixels centering on the pixel of interest and outputs the prediction tap to the error function generation circuit 34. The prediction tap includes the pixel of interest.

In step S 55, the error function generation circuit 34 generates an error function corresponding to the pixel of interest and outputs it to the influence error function register 35.

Here, the error function will be described. Nine 3 × 3 pixels of the lower hierarchical image (for example, pixels of the lower hierarchical image of FIG. 5) corresponding to one target pixel of the upper hierarchical image (for example, the target pixel of the upper hierarchical image of FIG. 5) a to i) pixel values (predicted values) yi ′ (i = 1, 2,...)
, 9, where i is different from i of the pixel i in FIG. 5) can be represented by a linear linear combination of the pixel value x of the upper hierarchical image and the prediction coefficient w, as shown in the following equation (1). .

[0056]

(Equation 1)

Where wi1 through wi25 are prediction coefficients corresponding to the class code of the pixel of interest, and x1 through x25
Reference numeral 25 denotes a pixel value of a pixel included in a prediction tap centered on the target pixel. In particular, the pixel value number xk and the prediction coefficient wik are the pixel value of the target pixel and the corresponding prediction coefficient.

Assuming that the pixel value (true value) of the original image corresponding to the pixel value (prediction value) yi 'of the lower hierarchical image is yi, the sum of square errors of 9 pixels of the lower hierarchical image corresponding to the pixel of interest is obtained. Ek can be expressed as in the following equation (2).

[0059]

(Equation 2)

Incidentally, in equation (2), the pixel value xk of the target pixel is a value to be optimized, that is, a variable. Also, the true value yi, the prediction coefficients wij, wik, and the pixel value xj are constants. Therefore, Expression (2) can be expressed as a quadratic expression of the target pixel value xk, as shown in Expression (3).

Ek = ak · xk2 + bk · xk + ck (3) where

[0062]

(Equation 3)

Is as follows. Here, Ek is called an error function.

Returning to FIG. In step S56,
The pixel-of-interest determination circuit 31 determines whether or not all the pixels within the affected area have been determined as the pixel of interest. If it is determined that all the pixels within the affected area have not been determined as the pixel of interest, the process proceeds to step S53. Return, and the subsequent processing is repeated.

Thereafter, when it is determined in step S56 that all the pixels within the influence range have been determined as the target pixel, the process proceeds to step S57. In step S57,
The target pixel determination circuit 31 outputs a control signal for turning on the switch 36. The influence error function register 35 adds the error function Ek (Equation (3)) when all the pixels within the influence range are sequentially set as a target pixel, and calculates the influence error function Echeck.

[0066]

(Equation 4)

Is generated and output to the target pixel value calculation circuit 37 via the switch 36.

Since the influence error function Echeck is the sum of the error function Ek which is a quadratic expression of the pixel value xk of the target pixel, as shown in the following equation (4), It becomes a quadratic function.

Influence error function Echeck = a ′ · xk2 + b ′ · xk + c ′ (4) where

[0070]

(Equation 5)

Is as follows.

In step S58, the target pixel value calculation circuit 37 calculates a pixel value xk = -b '/ 2a' that minimizes the quadratic expression of the influence error function Echeck as the optimum pixel value of the target pixel. Output to the upper layer image memory 21. The upper layer image memory 21 updates the stored pixel value of the target pixel using the input optimal pixel value.

In step S59, the pixel-of-interest determination circuit 23 determines whether or not all pixels of the upper hierarchical image have been determined as pixels of interest. If it is determined that all pixels have not been determined as pixels of interest, Returning to step S51, the subsequent processing is repeated.

Thereafter, when it is determined in step S59 that all the pixels of the upper hierarchical image have been determined as the target pixel, the target pixel determination circuit 23 outputs a control signal for turning on the switch 24. When the switch 24 is turned on, the information stored in the upper layer image memory 21 is displayed.
The upper layer image with the optimized pixel value is output to the subsequent upper layer image memory 3. The update number counter 11 outputs a control signal to the selector 5 in response to the end of the processing of the pixel value updating circuit 6.

Returning to FIG. After the pixel value update processing is performed in step S5 as described above, in step S6, the upper layer image input to the selector 5 from the upper layer image memory 3 corresponds to the control signal from the update number counter 11. Then, it is supplied to the prediction coefficient updating circuit 7. The prediction coefficient update circuit 7 updates the prediction coefficient table stored in the prediction coefficient memory 4 using the input upper layer image and original image.

Before describing the prediction coefficient update processing, a detailed configuration example of the prediction coefficient update circuit 7 will be described with reference to FIG.
This will be described with reference to FIG. The higher hierarchical image input from the selector 5 is supplied to the prediction tap extracting circuit 42 and the class code reading circuit 43 in the prediction coefficient updating circuit 7. The pixel-of-interest determination circuit 41 sequentially determines the pixels of the upper layer image as the pixel of interest, and outputs the position information to the prediction tap extraction circuit 42 and the class code reading circuit 43.

The prediction tap extraction circuit 42 extracts the pixel value (4 bits on the MSB side of the pixel data) of the prediction tap of 5 × 5 pixels centering on the pixel of interest, and
5 is output. The class code reading circuit 43 reads the class code of the pixel of interest (4 bits on the LSB side of the pixel data) and outputs it to the normal equation generation circuit 45.

The teacher data extraction circuit 44 extracts teacher data (true values corresponding to the pixel values of the lower hierarchical image to be generated) from the original image, and outputs the extracted teacher data to the normal equation generation circuit 45.
The normal equation generation circuit 45 outputs, for each class code of the pixel of interest, the known teacher data and the pixel value of the prediction tap,
In addition, a normal equation including a prediction coefficient as a variable is generated and output to the prediction coefficient calculation circuit 46.

The prediction coefficient calculation circuit 46 calculates a prediction coefficient (prediction coefficient table) corresponding to 16 types of class codes by applying the least square method to the input normal equation, and outputs the calculated prediction coefficient to the prediction coefficient memory 4.

Next, the operation of the prediction coefficient updating circuit 7 will be described with reference to the flowchart of FIG. The prediction coefficient update processing is started when an upper layer image is input from the selector 5 to the prediction coefficient update circuit 7.

In step S61, the pixel-of-interest determination circuit 41 determines one pixel of the upper hierarchical image as the pixel of interest, and outputs the position information to the prediction tap extraction circuit 42 and the class code reading circuit 43.

In step S62, the class code reading circuit 43 outputs the class code (pixel data
LSB side 4 bits) and reads out the normal equation generation circuit 4
5 is output. In step S63, the prediction tap extraction circuit 42 extracts the pixel value (4 bits on the MSB side of the pixel data) of the prediction tap of 5 × 5 pixels centering on the pixel of interest and outputs the pixel value to the normal equation generation circuit 45.

In step S64, the normal equation generation circuit 45 generates a normal equation for each class code of the pixel of interest using the known teacher data, the pixel value of the prediction tap, and the prediction coefficient to be updated. Output to the coefficient operation circuit 46.

In step S65, the pixel-of-interest determination circuit 41 determines whether or not all the pixels of the upper hierarchical image have been set as the pixel of interest. If it is determined that not all the pixels have been set as the pixel of interest, the process returns to step S61. , And the subsequent processes are repeated. If it is determined in step S65 that all the pixels have been set as the target pixel, the process proceeds to step S66.

In step S66, the prediction coefficient calculation circuit 46 calculates the prediction coefficients corresponding to 16 kinds of class codes by applying the least square method to the normal equation generated by the normal equation generation circuit 45 in step S64. In step S67, the prediction coefficient calculation circuit 46 outputs the obtained prediction coefficient (prediction coefficient table) to the prediction coefficient memory 4. The prediction coefficient memory 4 updates the stored prediction coefficient table by using the input prediction coefficient table. The update number counter 11 outputs a control signal to the selector 5 in response to the end of the processing of the prediction coefficient update circuit 6.

Returning to FIG. As described above, after the prediction coefficient updating process is performed in step S6, the process proceeds to step S2.
Return to In step S2, the upper layer image memory 3
Are supplied to the class code selection circuit 8 in accordance with the control signal from the update counter 11. The class code selection circuit 8
For each pixel of the input upper layer image, an optimal one of the 16 types of prediction coefficients stored in the prediction coefficient memory 4 is selected.

Before describing the class code selection processing, a detailed configuration example of the class code selection circuit 8 will be described with reference to FIG. Target pixel determination circuit 51
Sequentially determines the pixels of the upper layer image as the pixel of interest, and outputs the information to the prediction tap extraction circuit 52. The prediction tap extraction circuit 52 extracts the pixel value (4 bits on the MSB side of the pixel data) included in the 5 × 5 pixel prediction tap centered on the target pixel from the upper layer image input from the selector 5. And outputs it to the mapping circuit 53.

The mapping circuit 53 reads out a prediction coefficient corresponding to the class code input from the class code counter 58 from the prediction coefficient memory 4 and performs a linear primary operation on the read prediction coefficient and the pixel value of each pixel of the prediction tap. The combination is calculated, and the calculation result is output to the error calculation circuit 54 as the pixel value (predicted value) of the lower hierarchical image.

The error calculation circuit 54 includes a mapping circuit 53
, And an error (S / N) between the predicted value input from and the corresponding pixel value (true value) of the original image, and outputs it to the comparator 55 and the switch 57. The comparator 55 compares the error input from the error calculation circuit 54 with the error input from the minimum error register 56, and determines that the error input from the error calculation circuit 54 is smaller (S / N is larger). And a control signal for turning on the switches 57 and 59. Further, the comparator 55 outputs a control signal for incrementing the count value of the class code counter 58 after comparing the error regardless of the result of the comparison.

The minimum error register 56 supplies the stored error value to the comparator 55. Further, the minimum error register 56 updates the value stored so far by using the value input via the switch 57.

The class code counter 58 has a 4-bit counter, and its value is stored in the class code (0000 to 111).
Output to the mapping circuit and the switch 59 as 1). The counter value (class code) is stored in the comparator 5
5 is incremented by one in accordance with the control signal input from. When the value of the class code counter reaches 1111, the class code counter 58 outputs a control signal for turning on the switch 61 and sets its own counter to 0000.
Reset to.

The optimum class code register 60 updates the class code stored so far by using the class code input via the switch 59. Therefore, the optimal class code register 60 holds the optimal class code corresponding to the prediction coefficient that minimizes the error. Further, the optimal class code register 60 outputs the optimal class code of the pixel of interest to the subsequent upper layer image memory 3 via the switch 61.

Next, the operation of the class code selection circuit 8 will be described with reference to the flowchart of FIG. This class code selection processing is started when an upper layer image is input from the selector 5 to the class code selection circuit 8.

In step S 71, the pixel-of-interest determination circuit 51 determines one pixel of the upper hierarchical image as the pixel of interest, and outputs the information to the prediction tap extraction circuit 52. In step S72, the prediction tap extraction circuit 52 extracts a pixel value of a pixel included in a 5 × 5 pixel prediction tap centered on the pixel of interest from the upper layer image input from the selector 5, and outputs the pixel value to the mapping circuit 53. Output.

In step S73, the class code counter 58 outputs the value 0000 of the counter to the mapping circuit 53 as a class code. In step S74, the mapping circuit 53 reads a prediction coefficient corresponding to the class code input from the class code counter 58 from the prediction coefficient memory 4, and calculates a linear 1 between the read prediction coefficient and the pixel value of each pixel of the prediction tap. The next combination is calculated, and the calculation result is output to the error calculation circuit 54 as the pixel value (predicted value) of the lower hierarchical image.

In step S75, the error calculation circuit 5
4 calculates an error (S / N) between the predicted value input from the mapping circuit 53 and the corresponding pixel value (true value) of the original image, and outputs it to the comparator 55 and the switch 57.
The comparator 55 compares the error input from the error calculation circuit 54 with the error input from the minimum error register 56,
The error input from the error calculation circuit 54 is smaller (S /
If N is large), a control signal for turning on the switches 57 and 59 is output. Thus, the count value of the class counter 58 at that time is transferred to the optimum class code register 60 via the switch 59 and stored. Also,
The output of the error calculation circuit 54 at that time is transferred to the minimum error register 56 and stored. The comparator 55 also outputs a control signal to the class code counter 58.

In step S76, the class code counter 58 determines whether or not the value of the counter (class code) is smaller than 1111. If it is determined that the value of the counter is smaller than 1111, in step S77 the counter Is incremented by 1 and the value is output to the mapping circuit and the switch 59 as a class code.

Thereafter, the processing of steps S74 to S77 is repeated until it is determined in step S76 that the value of the counter is not smaller than 1111. If it is determined in step S76 that the value of the counter is not smaller than 1111 (the value of the counter is 1111), the process proceeds to step S78.

In step S78, the class code counter 58 outputs a control signal for turning on the switch 61 and resets its own counter to 0000. The switch 61 is turned on in response to this control signal, and the optimal class code of the pixel of interest held in the optimal class code register 60 is output to the subsequent upper-layer image memory 3. The upper layer image memory 3 rewrites the LSB side 4 bits of the pixel data of the corresponding pixel using the optimal class code input.

In step S79, the pixel-of-interest determination circuit 51 determines whether or not all the pixels of the upper layer image have been set as the pixel of interest. Is repeated. If it is determined in step S79 that all the pixels have been set as the target pixel, the process returns to step S3 in FIG.

Before describing the decoding processing in step S3, a detailed configuration example of the decoding circuit 9 will be described with reference to FIG. The pixel-of-interest determination circuit 71 sequentially determines the pixels of the upper layer image as pixels of interest, and outputs the information to the class code reading circuit 72 and the prediction tap extracting circuit 73. The class code reading circuit 72 reads the class code (4 bits on the LSB side of the pixel data) of the pixel of interest from the upper layer image input from the selector 5 and outputs it to the mapping circuit 74. The prediction tap extraction circuit 73 calculates the pixel value (4 bits on the MSB side of the pixel data) included in the 5 × 5 pixel prediction tap centered on the pixel of interest from the upper layer image input from the selector 5
Is extracted and output to the mapping circuit 74.

The mapping circuit 74 reads a prediction coefficient corresponding to the class code input from the class code reading circuit 72 from the prediction coefficient memory 4, and reads the read prediction coefficient and the prediction tap supplied from the prediction tap extraction circuit 73. , And outputs the calculation result to the lower layer image memory 75 as the pixel value of the lower layer image.

The lower layer image memory 75 stores the pixel values of the lower layer image input from the mapping circuit 74,
The convergence determination circuit 1 at the subsequent stage stores the stored pixel values in frame units.
Output to 0.

Next, the decoding process of the decoding circuit 9 will be described with reference to the flowchart of FIG. This decoding process is started when an upper layer image is input from the selector 5 to the decoding circuit 9.

In step S81, the pixel-of-interest determination circuit 71 determines one pixel of the upper layer image as a pixel of interest, and outputs the information to the class code reading circuit 72 and the prediction tap extracting circuit 73. In step S <b> 82, the class code reading circuit 72 reads the class code of the pixel of interest (LSB side 4 bits of pixel data) from the upper layer image input from the selector 5, and outputs it to the mapping circuit 74. In step S83, the prediction tap extraction circuit 73 calculates, from the upper hierarchical image input from the selector 5, the pixel value of the pixel included in the prediction tap of 5 × 5 pixels centering on the pixel of interest (4. ) Is extracted and output to the mapping circuit 74.

In step S 84, the mapping circuit 74 reads a prediction coefficient corresponding to the class code input from the class code reading circuit 72 from the prediction coefficient memory 4, and supplies the read prediction coefficient and the prediction tap extraction circuit 73. A linear linear combination with the pixel value of each pixel of the predicted tap thus calculated is calculated, and the calculation result is output to the lower layer image memory 75 as the pixel value of the lower layer image.

In step S85, the pixel-of-interest determination circuit 71 determines whether or not all the pixels of the upper layer image have been set as the pixel of interest. If it is determined that not all the pixels have been set as the pixel of interest, the process returns to step S81. , And the subsequent processes are repeated. If it is determined in step S85 that all the pixels have been set as the target pixel, in step S86, the pixel values of the lower hierarchical image are output from the lower hierarchical image memory 75 to the subsequent convergence determining circuit 10 in frame units.

Returning to FIG. Again, in step S4, the convergence determination circuit 10 calculates the S / N of the lower hierarchical image and the original image input from the decoding circuit 9 and the increase thereof, and determines whether the increase of the S / N has converged. Is determined, and S / N
Steps S2 to S6 are repeated until it is determined that the increase amount has converged or the control signal from update number counter 11 is received.

Thereafter, if it is determined in step S4 that the S / N increase has converged, or if a control signal from the update counter 11 has been received, the process proceeds to step S7. In step S7, the convergence determination circuit 10 outputs the upper layer image and the prediction coefficient table to the decoder 81 (FIG. 18) via a medium (not shown).

As described above, the encoder 1 uses the 4 bits on the MSB side as the pixel value and the 4 bits on the LSB side of the pixel data (8 bits) of the generated upper layer image as the class code. Are not changed when optimizing, and it is possible to optimize each independently.

In the encoding process shown in FIG. 2, the first step S2, that is, step S1
May be omitted from the class code selection processing immediately after the pre-processing of the above. In this case, the convergence determination result in the first step S4, that is, the S / N for the original image of the upper layer image that has been subjected to only the preprocessing in step S1 is the S / N for the original image of the initial upper layer image by the conventional encoder. /
It is better than N.

Further, the pixel value updating circuit 6 may be omitted from the encoder 1 shown in FIG. 1, and the pixel value updating process in step S5 in FIG. 2 may be omitted. That is, the pixel value of the upper layer image (4 bits on the MSB side of the image data) is not updated,
Even if only the prediction coefficient update process and the class code selection process are executed, it is possible to generate a higher-layer image better than the conventional encoder.

FIG. 18 shows an example of the configuration of a decoder for restoring an original image (generating a lower hierarchical image) from an upper hierarchical image generated by the encoder 1. This decoder 81
, The upper layer image from the encoder 1 is supplied to the class code reading circuit 83 and the prediction tap extracting circuit 84, and the prediction coefficient table is supplied to the mapping circuit 85.

The pixel-of-interest determination circuit 82 sequentially determines the pixels of the upper hierarchical image as pixels of interest, and outputs the position information to the class code reading circuit 83 and the prediction tap extracting circuit 84. The class code reading circuit 83 reads the class code (4 bits on the LSB side of the pixel data) of the pixel of interest from the upper layer image, and outputs it to the mapping circuit 85.
The prediction tap extracting circuit 84 extracts the pixel value (4 bits on the MSB side of the pixel data) of the pixel included in the prediction tap of 5 × 5 pixels centering on the pixel of interest from the upper layer image and outputs the extracted value to the mapping circuit 85. Output.

The mapping circuit 85 reads a prediction coefficient corresponding to the class code input from the class code reading circuit 83 from the prediction coefficient table, and reads the read prediction coefficient and the prediction tap supplied from the prediction tap extraction circuit 84. The linear primary combination with the pixel value of each pixel is calculated, and the calculation result is output to the lower layer image memory 86 as the pixel value of the lower layer image.

The lower layer image memory 86 stores the pixel value of the lower layer image input from the mapping circuit 85,
The stored pixel value is output to, for example, a monitor (not shown).

Next, the operation of the decoder 81 will be described with reference to FIG.
This will be described with reference to the flowchart of FIG. The decoding process is performed on the sequentially input upper layer image after the prediction coefficient table from the encoder 1 is supplied to the mapping circuit 85.

In step S91, the pixel-of-interest determination circuit 82 determines one pixel of the upper hierarchical image as the pixel of interest, and outputs the position information to the class code reading circuit 83 and the prediction tap extracting circuit 84. In step S92, the class code reading circuit 83 reads the class code of the pixel of interest (the 4 bits on the LSB side of the pixel data) from the upper layer image and outputs it to the mapping circuit 85.
In step S93, the prediction tap extraction circuit 84
5x centered on the pixel of interest from the input upper layer image
The pixel value (4 bits on the MSB side of the pixel data) of the pixel included in the five pixel prediction taps is extracted and mapped.
Output to

In step S94, the mapping circuit 85 reads a prediction coefficient corresponding to the class code input from the class code reading circuit 83 from the prediction coefficient table, and supplies the read prediction coefficient and the prediction tap extraction circuit 84. A linear primary combination with the pixel value of each pixel of the predicted tap is calculated, and the calculation result is output to the lower layer image memory 86 as the pixel value of the lower layer image.

In step S95, the pixel-of-interest determination circuit 82 determines whether or not all the pixels of the upper layer image have been set as the pixel of interest. If it is determined that not all the pixels have been set as the pixel of interest, the process returns to step S91. , And the subsequent processes are repeated. If it is determined in step S95 that all pixels have been set as the pixel of interest, step S96
Then, the pixel value of the lower hierarchical image is output from the lower hierarchical image memory 86 to a monitor (not shown).

FIG. 20 is a flowchart for explaining the operation of the encoder 1, which is executed in a different order from the above-described encoding process (FIG. 2). The encoding process includes the class code selection process executed as step S6 of the encoding process shown in FIG. 2 and the prediction coefficient updating process executed as step S5 in FIG.
Is executed immediately after the pre-processing of step S1. That is, after the pre-processing is performed, the class code selection processing is performed, and then the prediction coefficient updating processing is performed.

Note that steps S101 to S1 in FIG.
07 are performed in steps S1, S6, and S in FIG.
5, S2, S3, S4, and S7 are the same as the respective processes, and thus description thereof is omitted.

In order to execute each processing in the order shown in the flowchart of FIG. 20, the selector 5 (FIG. 1) transmits the data from the upper-layer image memory 3 in accordance with the control signal input from the update counter 11. The input upper-layer image may be sequentially output to the class code selection circuit 8, the prediction coefficient update circuit 7, the decode circuit 9, the convergence determination circuit 10, and the pixel value update circuit 6.

FIG. 21 shows the result of a simulation of calculating the S / N for the original image of the lower hierarchical image restored from a plurality of upper hierarchical images generated using the same original image. Note that the vertical axis represents S / N, and the horizontal axis represents the number of times a series of encoding processes have been executed (the number of updates).

The curve A in FIG. 21 shows the S / N of the lower hierarchical image restored from the upper hierarchical image generated according to the flowchart shown in FIG. 20, and the curve B is generated according to the flowchart shown in FIG. The curve C indicates the S / N of the lower hierarchical image restored from the upper hierarchical image, and the curve C is restored from the upper hierarchical image generated by omitting the pixel value updating process in step S4 from the flowchart shown in FIG. The curve D indicates the S / N of the lower hierarchical image restored from the upper hierarchical image generated according to the conventional method.

As is apparent from the curve A in FIG.
The lower-layer image restored from the upper-layer image generated according to the flowchart shown in (1) shows a higher S / N than the others, especially at a stage where the number of updates is small. According to the flowchart of FIG.
This indicates that an upper layer image capable of restoring a lower layer image indicating S / N can be generated in a short processing time.

Next, a second configuration example of the pixel value updating circuit 6 will be described with reference to FIG. In this configuration example, the optimum pixel value determination circuit 22 and the target pixel determination circuit 23 of the first configuration example of the pixel value update circuit 6 shown in FIG. 8 are replaced with a region optimization circuit 91 and a target region determination circuit 92, respectively. It was done. The optimum pixel value determination circuit 22 in FIG.
While optimizing the pixel value of only the target pixel, FIG.
The region optimization circuit 91 optimizes the pixel values of a plurality of pixels included in the region of interest at the same time.

The region-of-interest determination circuit 92 sequentially arranges regions (for example, 7 × 7 pixels) of a predetermined size in the upper hierarchical image.
The region of interest is determined, and the position information is determined by the region
Output to Further, the attention area determination circuit 92 outputs a control signal for turning on the switch 24 after setting all the pixels of the upper layer image as the attention area.

FIG. 23 shows a detailed configuration example of the area optimizing circuit 91. The pixel-of-interest determination circuit 101 sequentially determines the pixels of the region of interest as pixels of interest, and outputs the position information to the class code reading circuit 102 and the prediction tap extracting circuit 103. Further, the pixel of interest determination circuit 101
Determines all pixels in the region of interest as pixels of interest,
A control signal for turning on the switch 106 is output.

The class code reading circuit 102 reads the class code (the four LSB bits of pixel data) of the pixel of interest from the upper layer image memory 3 and
04. The prediction tap extraction circuit 103 extracts a 5 × 5 pixel prediction tap centered on the pixel of interest from the upper layer image memory 3 and outputs the extracted tap to the error function generation circuit 104.

The error function generation circuit 104 generates an error function (the details of which will be described later) corresponding to the pixel of interest and outputs it to the influence error function matrix register 105. The influence error function matrix register 105 generates an influence error function matrix using error functions corresponding to all the pixels of interest in the area of interest, and outputs the matrix to the pixel value calculation circuit 107 of area of interest via the switch 106.

The attention area pixel value calculation circuit 107 calculates the pixel value of the pixel in the attention area by solving the influence error function matrix input via the switch 106 (the details will be described later).

Next, the operation of the second configuration example of the pixel value updating circuit 6 will be described with reference to the flowchart of FIG. In the pixel value updating process, the pixel values of the pixels other than the attention area are fixed, and the pixel values of the pixels in the attention area are optimized. The pixel value updating process is started when the upper layer image input from the selector 5 is stored in the upper layer image memory 21 of the pixel value updating circuit 6.

In step S111, the attention area determination circuit 92 determines 7 × 7 pixels of the upper hierarchical image as the attention area, and outputs the position information to the area optimization circuit 91.
In step S112, the pixel-of-interest determination circuit 101 of the area optimization circuit 91 determines one pixel in the area of interest as the pixel of interest, and stores the position information thereof in the class code reading circuit 1.
02 and the prediction tap extraction circuit 103.

In step S113, the class code reading circuit 102 reads the class code of the pixel of interest (4 bits on the LSB side of the pixel data) and outputs it to the error function generating circuit 104. The prediction tap extraction circuit 103 extracts a 5 × 5 pixel prediction tap centered on the pixel of interest and outputs it to the error function generation circuit 104.

In step S114, the error function generation circuit 104 generates an error function corresponding to the pixel of interest and outputs it to the influence error function matrix register 105.

In step S115, the pixel-of-interest determination circuit 101 determines whether or not all pixels in the region of interest have been determined as pixels of interest, and if it is determined that not all pixels have been determined as pixels of interest. , The process returns to step S112, and the subsequent processes are repeated. Step S115
In, when it is determined that all the pixels in the region of interest have been determined as the pixel of interest, the process proceeds to step S116.

In step S116, the pixel-of-interest determination circuit 101 outputs a control signal for turning on the switch 106. The influence error function matrix register 105
An influence error function matrix is generated from the error function corresponding to the input target pixel, and is output to the target area pixel value calculation circuit 107 via the switch 106.

Here, the attention area, the prediction tap, the error function, and the influence error function matrix will be described.
As shown in FIG. 25 (A), the attention area is 4 × 7 × 7 pixels.
If the pixel is composed of nine pixels and the upper left corner is the first, and the j-th pixel j is set as the target pixel, the prediction tap (tapsj) corresponding to the target pixel j is set as shown in FIG.
As shown in (5), 25 pixels of 5 × 5 pixels centered on the pixel j are extracted. Therefore, s = 49 in FIG. 25 (A) and t = 25 in FIG. 25 (B).

In addition, three types of ranges (A) for the area of interest (area) and the prediction tap (tapsj) for the pixel of interest j
1j to A3j) are set. However, the range A1j
Is a range that belongs to the prediction tap (tapsj) and does not belong to the area of interest (area), and the range A2j is the range of the prediction tap (tapsj).
The range A3j does not belong to j) and belongs to the area of interest (area), and the range A3j belongs to the prediction tap (tapsj) and belongs to the area of interest (area).

The pixel value (predicted value) y′j of the lower hierarchical image corresponding to the pixel of interest j of the upper hierarchical image can be expressed by the following equation (5). In the following, a superscript (for example, j of y'j) indicates a number in the attention area, and a subscript (for example, t of wt) indicates a number in a prediction tap.

[0142]

(Equation 6)

Further, xpj is the p-th pixel value of the prediction tap (tapsj) of the target pixel j, and wpj is xpj of the coefficient vector corresponding to the class code of the target pixel j.
Is a coefficient related to However, actually, since one pixel of interest in the upper hierarchical image corresponds to nine pixels in the lower hierarchical image, another eight expressions similar to Expression (5) are generated.

Here, assuming that a true value (pixel value of the original image) corresponding to the predicted value y'j is yj, the error ej is represented by the following equation (6).

[0145]

(Equation 7)

However,

[0147]

(Equation 8)

Is as follows.

Note that n ′ is a value obtained by converting the number n in the area of interest into the number in the prediction tap (tapsj). Since the pixel values of the pixels located in the range A1j are not updated, y ″ j is a fixed value. Therefore, in order to find the square error corresponding to the pixel of interest, eight other equations similar to equation (6) are generated, and each of them is squared and added. The result is defined as an error function Ek.

The sum of the error functions Ek corresponding to all the pixels in the area of interest is referred to as an influence error function Ear.
ea.

[0151]

(Equation 9)

Here, as described above, Ek is (e
k) is obtained by adding 9 to 2, but for simplification, Ek = (ek) 2.

Next, let xi be the pixel value of the pixel corresponding to the number n in the area of interest, and use the influence error function Ear.
Pixel values x1 to xs that minimize the value of ea are obtained by the least square method.

First, a partial differential coefficient (the following equation (9)) based on the pixel value xi of the equation (8) is obtained, and the pixel value xi is determined so that the value becomes zero.

[0155]

(Equation 10)

Here, based on equation (7), Wji and Yi are determined as in the following equation (10).

[0157]

[Equation 11]

Assuming that the value of equation (9) is 0, a determinant such as equation (11) is obtained.

(Equation 12) Here, since Wji and Yi in Expression (10) have values corresponding to nine pixels, the determinant (1) obtained by adding these values is defined as an influence error function matrix.

Returning to the description of step S117 in FIG.
In step S117, the attention area pixel value calculation circuit 1
07 is a pixel value x1 obtained by applying a general matrix solution such as a sweeping-out method to the input influence error function matrix.
To xs, and outputs the result to the upper layer image memory 21. The upper layer image memory 21 stores the input pixel value x1
To xs to update the value stored so far.

In step S118, the attention area determination circuit 92 determines whether or not all the pixels of the upper hierarchical image have been determined as the attention area. If it is determined that all the pixels have not been determined as the attention area, Returning to step S111, the subsequent processing is repeated.

Thereafter, in step S118, when it is determined that all the pixels of the upper hierarchical image have been determined as the attention area, the attention area determination circuit 92 outputs a control signal for turning on the switch 24. When the switch 24 is turned on, the upper-layer image with the optimized pixel values stored in the upper-layer image memory 21 is output to the subsequent upper-layer image memory 3.

As described above, when processing is performed in units of areas, an image with a higher S / N can be obtained.

FIG. 26 shows a second example of the configuration of an encoder to which the present invention is applied. This encoder 111
This is obtained by adding a prediction coefficient expansion circuit 112 to a stage subsequent to the convergence determination circuit 10 of the encoder 1 shown in FIG. The prediction coefficient extension circuit 112 outputs the 4
The prediction coefficient table corresponding to the class codes of 16 bits (16 types) is made to correspond to the extended class codes of 5 bits (32 types).

A detailed configuration example of the prediction coefficient expansion circuit 112 will be described with reference to FIG. Convergence determination circuit 10
Is input to the prediction coefficient expansion circuit 11
2, the prediction tap extracting circuit 122, the class code reading circuit 123, and the spatial class code determining circuit 12
4 is supplied. The pixel-of-interest determination circuit 121 sequentially determines the pixels of the upper layer image as the pixel of interest, and uses the position information as the prediction tap extraction circuit 122 and the class code reading circuit 12.
3, and output to the space class code determination circuit 124.

The prediction tap extraction circuit 122 extracts the pixel value (4 bits on the MSB side of the pixel data) of the prediction tap of 5 × 5 pixels centering on the pixel of interest and outputs it to the normal equation generation circuit 126. The class code reading circuit 123 reads the class code of the pixel of interest (4 bits on the LSB side of the pixel data) and outputs it to the space class code determination circuit 124.

The space class code determination circuit 124 determines the space class code of the pixel of interest. That is, FIG.
As shown in (2), the pixel value of the pixel of interest is compared with the pixel value of the pixel in the vicinity thereof (for example, the pixel on the left side). Is determined to be 0, and when the pixel value of the neighboring pixel is larger, the space class code is determined to be 1. Furthermore, the space class code determination circuit 124
An extended class code (5 bits) is generated by adding a space class code to the MSB side of the 4-bit class code input from 23, and is output to the normal equation generating circuit 126.

Note that the number of bits of the space class code is 1
It is not limited to bits, but may be a plurality of bits. Further, instead of determining the space class code by the above-described method, for example, the space class code may be determined from 1-bit ADRC processing, a differential value, or a difference value.

The teacher data extraction circuit 125 extracts the teacher data (true values of the pixel values of the lower hierarchical image generated using the prediction taps) from the original image, and
6 is output. The normal equation generating circuit 126 generates a normal equation including the known teacher data and the pixel value of the prediction tap, and the prediction coefficient as a variable for each extended class code of the pixel of interest.
Output to

The prediction coefficient calculation circuit 127 applies the least squares method to the input normal equation to obtain 32 types (5 bits).
The prediction coefficient corresponding to the extended class code is calculated and output to the subsequent stage. When the normal equation from the normal equation generation circuit 126 cannot be solved, the prediction coefficient calculation circuit 127
16 types (4 bits) supplied from the prediction coefficient memory 4
The prediction coefficient table corresponding to the class code is output to the subsequent stage.

Next, the operation of the prediction coefficient expansion circuit 112 will be described with reference to the flowchart in FIG. This prediction coefficient expansion process is started when the optimum upper layer image is input from the convergence determination circuit 10.

In step S121, the pixel-of-interest determination circuit 121 determines one pixel of the upper hierarchical image as the pixel of interest, and uses the position information as the prediction tap extraction circuit 122, the class code reading circuit 123, and the space class code determination. Output to the circuit 124.

In step S122, the class code reading circuit 123 reads the class code of the pixel of interest (4 bits on the LSB side of the pixel data) and outputs it to the space class code determining circuit. In step S123, the space class code determination circuit 124 compares the pixel value of the pixel of interest with the pixel values of neighboring pixels. If the pixel value of the pixel of interest is larger, the space class code (1 bit) is determined. When it is determined to be 0 and the pixel value of the neighboring pixel is larger, the space class code is determined to be 1. Furthermore, the space class code determination circuit 124
An extended class code (5 bits) is generated by adding a space class code to the MSB side of the 4-bit class code input from 23, and is output to the normal equation generating circuit 126.

In step S124, the prediction tap extraction circuit 122 extracts the pixel value (4 bits on the MSB side of the pixel data) of the prediction tap of 5 × 5 pixels centering on the pixel of interest, and sends it to the normal equation generation circuit 126. Output.

In step S125, the normal equation generating circuit 126 generates a normal equation including the known teacher data, the pixel value of the prediction tap, and the prediction coefficient as a variable for each class code of the pixel of interest. Output to the prediction coefficient calculation circuit 127.

In step S126, the pixel-of-interest determination circuit 121 determines whether or not all the pixels of the upper hierarchical image have been set as the pixel of interest. If it is determined that not all the pixels have been set as the pixel of interest, the process returns to step S121. , And the subsequent processes are repeated. If it is determined in step S126 that all the pixels have been set as the target pixel, the process proceeds to step S127.

In step S127, the prediction coefficient calculation circuit 127 applies the least square method to the normal equation generated by the normal equation generation circuit 126 in step S125, and
A prediction coefficient corresponding to two types (5 bits) of extended class codes is calculated, and the obtained prediction coefficient (prediction coefficient table)
Is output to the subsequent stage.

As described above, in the prediction coefficient extension circuit 112, the prediction coefficient is made to correspond to the extension class code (5 bits). However, the extension class code is not written in the 5 bits on the LSB side of the pixel data of the target pixel. Since the pixel value of the pixel of interest (4 bits on the MSB side of the pixel data) is not changed, there is no lack of gradation in the lower hierarchical image generated from the upper hierarchical image.

FIG. 30 shows the encoder 11 shown in FIG.
1 shows a configuration example of a decoder that restores an original image (generates a lower-layer image) from the upper-layer image generated in 1.

This decoder 131 is obtained by adding a space class code judging circuit 132 to the decoder 81 shown in FIG. In the space class code determination circuit 132,
Upper layer image from encoder 1, pixel-of-interest determination circuit 8
2 and the class code from the class code reading circuit 83 (of the pixel data of the pixel of interest).
LSB side 4 bits) is input.

The space class code determination circuit 132 determines the pixel value of the pixel of interest and its predetermined neighborhood (in this case, the left side)
Are compared, and if the pixel value of the pixel of interest is larger, the space class code (1 bit) is determined to be 0. If the pixel value of a neighboring pixel is larger, the space class code is 1 is determined. Further, the space class code determining circuit 132 generates an extended class code (5 bits) by adding a space class code to the MSB side of the 4-bit class code input from the class code reading circuit 83, Output. Note that components other than the space class code determination circuit 132 are the same as those shown in FIG. 18, and a description thereof will be omitted.

Next, the operation of the decoder 131 will be described with reference to the flowchart in FIG. This decoding process is performed on the upper layer image in frame units that is sequentially input after the prediction coefficient table output from the encoder 111 is supplied to the mapping circuit 85.

In step S131, the pixel-of-interest determination circuit 82 determines one pixel of the upper hierarchical image as the pixel of interest, and uses the position information as the class code reading circuit 83, the prediction tap extraction circuit 84, and the spatial class code. Output to the judgment circuit 132. In step S132, the class code reading circuit 83 reads the class code of the pixel of interest (the 4 bits on the LSB side of the pixel data) from the upper hierarchical image and outputs it to the spatial class code determination circuit 132.

In step S133, the space class code determination circuit 132 compares the pixel value of the pixel of interest with the pixel value of the pixel on the left, and if the pixel value of the pixel of interest is larger, the space class code ( (1 bit) is determined to be 0, and when the pixel value of a neighboring pixel is larger, the space class code is determined to be 1. Further, the space class code determining circuit 132 generates an extended class code (5 bits) by adding a space class code to the MSB side of the 4-bit class code input from the class code reading circuit 83, Output.

In step S134, the prediction tap extracting circuit 84 determines the pixel value (4 bits on the MSB side of the pixel data) included in the 5.times.5 pixel prediction tap centered on the pixel of interest from the input upper hierarchical image. ) Is extracted and output to the mapping circuit 85.

In step S135, the mapping circuit 85 reads a prediction coefficient corresponding to the extended class code input from the space class code determination circuit 132 from the prediction coefficient table, and reads out the read prediction coefficient and each pixel of the prediction tap. The linear primary combination with the pixel value is calculated, and the calculation result is output to the lower layer image memory 86 as the pixel value of the lower layer image.

In step S136, the pixel-of-interest determination circuit 82 determines whether or not all the pixels of the upper hierarchical image have been determined as the pixel of interest. Until all the pixels have been determined to be the pixel of interest, steps S131 to S136 are performed. Is repeated. If it is determined in step S137 that all the pixels have been set as the pixel of interest, the pixel value of the lower hierarchical image is output from the lower hierarchical image memory 86 to the monitor (not shown) in frame units in step S137.

FIG. 32 shows the encoder 11 shown in FIG.
30 using the upper layer image generated in step 1 with respect to the original image of the lower layer image restored by the decoder 131 shown in FIG.
It shows the result of a simulation that calculates S / N.
As can be seen from the figure, after the S / N has converged by repeatedly executing the update using the LSB side 4 bits as the class code (up to the 20th update), 32 kinds of prediction coefficients (5 Bit), the S / N is further improved.

In this embodiment, the pixel data is 8 bits, and the 4 bits on the MSB side are the pixel value, L
Although the four bits on the SB side are used as the class code, the number of these bits may be changed. Further, the MSB side may be used as a class code, and the LSB side may be used as a pixel value. Further, the pixel value and the class code may be stored separately.

The shapes of the class tap, the prediction tap, and the area for updating the pixel value are not limited to those described above, and may be, for example, circular or discontinuous.
In addition, the shape may be symmetric or asymmetric.

In the present embodiment, the original image is converted into an image having a smaller number of pixels which can be restored to the original image. For example, the original image is converted into an image having the same number of pixels as the original image. When the information amount per pixel (for example, 5 bits) is smaller than the information amount per pixel (for example, 8 bits) of the original image, and the image is converted to an image that can be restored to the original image. In addition, the present invention can be applied.

Various modifications and application examples can be considered without departing from the gist of the present invention. Therefore,
The gist of the present invention is not limited to the present embodiment.

Note that the computer program for performing each of the above-described processes can be provided to the user via a network medium such as the Internet or a digital satellite in addition to a medium such as an information recording medium such as a magnetic disk or a CD-ROM. it can.

[0193]

As described above, according to the image encoding device of the first aspect, the image encoding method of the second aspect, and the providing medium of the third aspect, the pixel value of the original image And the pixel value of the second image, and according to the comparison result,
Since the stored pixel values are updated on a region-by-region basis, it is possible to generate an upper layer image capable of generating an image closer to the original image.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a first configuration example of an encoder 1 to which the present invention has been applied.

FIG. 2 is a flowchart illustrating an operation of the encoder 1 of FIG.

FIG. 3 is a flowchart illustrating an operation of a preprocessing circuit 2 of FIG. 1;

FIG. 4 is a flowchart illustrating details of step S11 in FIG. 3;

FIG. 5 is a diagram illustrating an arrangement of pixels.

FIG. 6 is a diagram illustrating pixel data.

FIG. 7 is a flowchart illustrating details of step S12 in FIG. 3;

FIG. 8 is a block diagram illustrating a first configuration example of a pixel value updating circuit 6 of FIG. 1;

9 is a block diagram illustrating a configuration example of an optimum pixel value determination circuit 22 in FIG.

FIG. 10 is a flowchart illustrating an operation of the pixel value updating circuit 6 of FIG. 8;

FIG. 11 is a diagram illustrating an arrangement of pixels.

12 is a block diagram illustrating a configuration example of a prediction coefficient updating circuit 7 in FIG.

FIG. 13 is a flowchart illustrating the operation of the prediction coefficient update circuit 7 of FIG.

FIG. 14 is a block diagram illustrating a configuration example of a class code selection circuit 8 of FIG. 1;

15 is a flowchart illustrating the operation of the class code selection circuit 8 in FIG.

FIG. 16 is a block diagram illustrating a configuration example of a decoding circuit 9 in FIG. 1;

17 is a flowchart illustrating the operation of the decoding circuit 9 in FIG.

FIG. 18 is a block diagram showing a configuration example of a decoder 81 corresponding to the encoder 1 shown in FIG.

FIG. 19 is a flowchart illustrating the operation of the decoder 81 in FIG. 18;

20 is a flowchart illustrating another operation of the encoder 1 in FIG.

FIG. 21 is a diagram showing a simulation result.

FIG. 22 is a block diagram illustrating a second configuration example of the pixel value updating circuit 6 of FIG. 1;

FIG. 23 is a block diagram illustrating a configuration example of a region optimization circuit 91 in FIG. 22;

24 is a flowchart illustrating an operation of the pixel value updating circuit 6 in FIG.

FIG. 25 is a diagram for explaining an influence error function matrix.

FIG. 26 is a block diagram illustrating a second configuration example of an encoder to which the present invention has been applied.

27 is a block diagram illustrating a configuration example of a prediction coefficient expansion circuit 112 in FIG. 26.

FIG. 28 is a diagram for explaining an extended class code.

FIG. 29 is a flowchart illustrating the operation of the prediction coefficient expansion circuit 112 in FIG. 27;

30 is a corresponding decoder 1 of the encoder 1 of FIG.
FIG. 3 is a block diagram illustrating a configuration example of a first embodiment.

FIG. 31 is a flowchart illustrating the operation of the decoder 131 in FIG. 30;

FIG. 32 is a diagram showing a simulation result.

[Explanation of symbols]

1 encoder, 2 preprocessing circuit, 3 upper layer image memory, 4 prediction coefficient memory, 5 selector,
6 pixel value update circuit, 7 prediction coefficient update circuit, 8
Class code selection circuit, 9 decoding circuit, 10
Convergence determination circuit, 11 update counter, 22 optimal pixel value determination circuit, 81 decoder, 82 pixel determination circuit of interest, 83 class code reading circuit, 84 prediction tap extraction circuit, 85 mapping circuit, 86 lower layer image memory, 91 area optimization Circuit, 111
Encoder, 112 prediction coefficient extension circuit, 124
Space class code decision circuit, 131 decoder, 1
32 space class code judgment circuit

 ──────────────────────────────────────────────────続 き Continuing on the front page (72) Inventor Kenji Takahashi 6-7-35 Kita-Shinagawa, Shinagawa-ku, Tokyo Inside Sony Corporation (72) Inventor Naoki Kobayashi 6-35, Kita-Shinagawa, Shinagawa-ku, Tokyo Sony Corporation F-term (reference) 5C059 MA34 MB08 MB12 MB14 SS20 SS26 TA29 TB08 TC02 TD11 UA02 UA38 5C063 CA11 CA34 5C076 AA40 BA09 BA10 5C078 BA64 DA01 DA02 DA11 DA12 DB13 DB14 9A001 BB02 BB03 BB04 EE02H16 GG04

Claims (3)

[Claims] 1. An image coding apparatus for converting an original image composed of a plurality of pixels into a first image composed of a smaller number of pixels, wherein the first image is converted using m pixel values of the original image. A pixel value generation unit that generates n pixel values of: a prediction coefficient generation unit that generates a prediction coefficient using a pixel value of the original image and a pixel value of the first image; A pixel data storage unit that stores a predetermined part of the pixel data as a pixel value and stores the other part as a class code; a prediction coefficient storage unit that stores the prediction coefficient in association with the class code; The pixel value of the second image is calculated using the pixel value that is a predetermined part of the pixel data of the image of the above and the prediction coefficient corresponding to the class code that is the other part of the pixel data of the first image. Arithmetic means and Comparing means for comparing the pixel value of the original image with the pixel value of the second image; and a prediction for updating the prediction coefficient stored in the prediction coefficient storage means in accordance with a comparison result of the comparison means. A coefficient updating unit, an area pixel value updating unit that updates the pixel value stored in the pixel data storage unit on an area basis in accordance with a comparison result of the comparing unit, and a comparison result of the comparison unit. And a class code updating means for updating the class code stored in the pixel data storage means. 2. An image encoding method of an image encoding device for converting an original image composed of a plurality of pixels into a first image composed of a smaller number of pixels, comprising: using m pixel values of the original image, A pixel value generation step of generating n pixel values of the first image; a prediction coefficient generation step of generating prediction coefficients using the pixel values of the original image and the pixel values of the first image; A pixel data storing step of storing a predetermined part of the pixel data of the first image as a pixel value and storing the other part as a class code; and a prediction coefficient storing step of storing the prediction coefficient in association with the class code Using a pixel value that is a predetermined part of the pixel data of the first image and a prediction coefficient corresponding to a class code that is another part of the pixel data of the first image, A calculating step of calculating a pixel value; a comparing step of comparing a pixel value of the original image with a pixel value of the second image; and a prediction coefficient storing step corresponding to the comparison result of the comparing step. A prediction coefficient update step of updating the prediction coefficient, and an area pixel value update step of updating the pixel value stored in the pixel data storage step in units of areas in accordance with a comparison result of the comparison step. A class code updating step of updating the class code stored in the pixel data storing step in accordance with the comparison result of the comparing step. 3. An image coding apparatus for converting an original image composed of a plurality of pixels into a first image composed of a smaller number of pixels, wherein the first image is encoded using m pixel values of the original image. A pixel value generating step of generating n pixel values of: a prediction coefficient generating step of generating a prediction coefficient using a pixel value of the original image and a pixel value of the first image; A pixel data storing step of storing a predetermined part of the pixel data as a pixel value and storing the other part as a class code; a prediction coefficient storing step of storing the prediction coefficient in association with the class code; The pixel value of the second image is calculated using the pixel value that is a predetermined part of the pixel data of the image of the above and the prediction coefficient corresponding to the class code that is the other part of the pixel data of the first image. Calculation Step; a comparing step of comparing the pixel value of the original image with the pixel value of the second image; and updating the prediction coefficient stored in the prediction coefficient storing step in accordance with the comparison result of the comparing step A prediction coefficient update step, and an area pixel value update step of updating the pixel value stored in the pixel data storage step in units of area in accordance with the comparison result of the comparison step, and a comparison result of the comparison step. Correspondingly, a providing medium which provides a computer-readable program for executing a process including a class code updating step of updating the class code stored in the pixel data storing step.