JP2000316085A - Image converter, image converting method, learning unit and method and recording medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To simultaneously optimize pixel values of a plurality of pixels in order to generate a low-order image from a high-order image obtained by compression. SOLUTION: An image reduction circuit 1 reduces a received original image. A high-order image memory 2 stores a received high-order image. A prediction tap acquisition circuit 3 extracts a prediction tap from the high-order image stored in the high-order image memory 2 and outputs it to a prediction coefficient arithmetic circuit 4, a pixel value update circuit 5, and a mapping circuit 6. The prediction coefficient arithmetic circuit 4 generates an observation equation by using the prediction tap for pupil data and using pixels of a corresponding original image for teacher data and generates a prediction coefficient by solving the observation equation. The pixel value update circuit 5 generates an observation equation by using the prediction coefficient from the prediction coefficient arithmetic circuit 4 for pupil data and using the corresponding original image data for teacher data. Optimum values of a plurality of update pixel values with respect to the given coefficient can simultaneously be obtained by solving the observation equation.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image conversion apparatus and method, a learning apparatus and method, and a recording medium, and more particularly, to an image conversion apparatus and method for generating a compressed image capable of restoring an image almost identical to an original image, The present invention relates to a learning device and method, and a recording medium.

[0002]

2. Description of the Related Art The inventor of the present invention disclosed in Japanese Patent Laid-Open No.
As disclosed in Japanese Patent Application Publication No. 980, a technique for generating a high-resolution image using a low-resolution image has been proposed, and an original image is generated using a low-resolution image obtained by reducing a high-resolution original image. It is said that almost the same high-resolution image can be restored. In this proposal, for example, as shown in FIG. 1, a pixel i of a high-resolution image (restored image) at a position corresponding to a target pixel I of a low-resolution image (upper layer image)
The pixel values of the 3 × 3 pixels a to i centered on the pixel are set to a plurality of pixels (for example, 3
X 3 pixels A to I) and a predetermined linear prediction coefficient are calculated to calculate a linear primary combination. Further, an error between the pixel value of the restored image and the pixel value of the original image is calculated,
The updating of the pixel value of the low-resolution image and the prediction coefficient are repeated according to the result.

[0003]

By the way, the updating of the pixel value of the conventional low-resolution image is performed under the condition that the pixel value of the neighboring pixel is fixed for each pixel. That is, as shown in FIG. 1, the pixel value of the target pixel I of the low-resolution image is obtained by fixing the pixel values of the eight pixels A to H centered on the target pixel I and the value of the predetermined prediction coefficient. It was updated to the optimal value below.

Therefore, when the pixel value of the pixel D is updated after the pixel value of the pixel I is updated, the pixel D
Are updated when the pixel value is updated, the pixel value of the pixel I updated earlier is not optimal for the updated pixel D. Therefore, when the pixel values of the low-resolution image (upper-layer image) are sequentially updated for each pixel, the low-resolution image (upper-layer image) in which all the pixel values are finally updated is not necessarily the optimal image in which the original image can be restored. There was a problem that could not be said to be a problem.

[0005] This problem can be solved by simultaneously updating the pixel values of a plurality of adjacent pixels of the low-resolution image (upper hierarchical image) to the optimum value. It takes time and the scale of the arithmetic circuit increases, which is practically impossible.

The present invention has been made in view of such a situation, and by simultaneously updating the pixel values of a plurality of adjacent pixels, it is possible to restore a high-resolution image substantially the same as the original image. This is to enable a low-resolution image to be obtained in a short time.

[0007]

According to a first aspect of the present invention, there is provided an image processing apparatus comprising:
An image data conversion device for converting the second image data into lower quality image data than the first image data.
An intermediate image data generation unit that generates intermediate image data substantially the same as the second image data; a storage unit that stores the intermediate image data; and a plurality of blocks for each block that is a part of one screen from the intermediate image data. A block extraction unit for extracting pixel data, a prediction coefficient generation unit for outputting a generated or previously obtained prediction coefficient, and extraction by the block extraction unit based on the prediction coefficient, the intermediate image data, and the first image data A pixel value updating unit that updates the pixel value of the processed intermediate image data,
A predicted image data generation unit that generates predicted image data that is substantially the same as the first image data based on the intermediate image data whose pixel values have been updated by the pixel value update unit and the prediction coefficients;
An image data conversion device, comprising: an error detection unit that detects an error between image data of the image data and predicted image data; and a control unit that determines whether or not to use intermediate image data as an output image based on the error. It is.

According to a sixth aspect of the present invention, the first image data is
An image data conversion method for converting second image data into lower quality second image data than the first image data, comprising the steps of: generating intermediate image data of substantially the same quality as the second image data from the first image data; Extracting a plurality of pixel data for each block which is a part of one screen, outputting a generated or previously obtained prediction coefficient, predicting coefficient, intermediate image data, and first image data. Updating the pixel value of the intermediate image data extracted by the block extracting unit based on the above, and predicting substantially the same quality as the first image data based on the intermediate image data having the updated pixel value and the prediction coefficient. Generating image data, detecting an error between the first image data and the predicted image data, and using the intermediate image data as an output image based on the error An image data conversion method characterized by a step of determining whether.

According to an eleventh aspect of the present invention, there is provided a learning apparatus for learning a pixel value of a second image data when converting the first image data into a second image data having a lower quality than the first image data. An intermediate image data generating unit that generates intermediate image data substantially the same as the second image data from the first image data, a storage unit that stores the intermediate image data, and a part of one screen from the intermediate image data A block extraction unit that extracts a plurality of pixel data for each block, a prediction coefficient generation unit that outputs a generated or previously obtained prediction coefficient, a prediction coefficient, intermediate image data, and first image data. A pixel value updating unit that updates a pixel value of the intermediate image data extracted by the block extracting unit based on the first image based on the intermediate image data whose pixel value is updated by the pixel value updating unit and the prediction coefficient De A predicted image data generating unit that generates predicted image data substantially the same as the data, an error detecting unit that detects an error between the first image data and the predicted image data, and intermediate image data as an output image based on the error. The pixel value updating unit updates the pixel value of the intermediate image data by the least squares method using the prediction coefficient as the student data and the corresponding first image data as the teacher data. The learning device is characterized in that:

According to a twelfth aspect of the present invention, there is provided a learning method for learning pixel values of second image data when converting the first image data into second image data having a lower quality than the first image data. Generating intermediate image data of substantially the same quality as the second image data from the first image data; extracting a plurality of pixel data for each block that is a part of one screen from the intermediate image data; Outputting the generated or previously obtained prediction coefficients;
Updating a pixel value of the intermediate image data extracted based on the prediction coefficient, the intermediate image data, and the first image data; and performing a first operation based on the updated intermediate image data and the prediction coefficient based on the pixel value. Generating predicted image data having substantially the same quality as the image data, detecting an error between the first image data and the predicted image data, and determining whether or not the intermediate image data is an output image based on the error. And updating the pixel value. The step of updating the pixel value includes updating the pixel value of the intermediate image data by the least square method using the prediction coefficient as the student data and the corresponding first image data as the teacher data. This is a featured learning method.

According to a thirteenth aspect of the present invention, there is provided a recording medium on which a computer-controllable program for converting image data for converting first image data into second image data of lower quality than the first image data is recorded. In the program, the program generates intermediate image data of substantially the same quality as the second image data from the first image data, and converts the plurality of pixel data for each block that is a part of one screen from the intermediate image data. Extracting, generating a prediction coefficient based on the extracted intermediate image and first image data at a position corresponding to the extracted intermediate image data, predicting coefficient, intermediate image data, and first image Updating a pixel value of the intermediate image data extracted based on the data and a first coefficient based on the intermediate image data having the updated pixel value and the prediction coefficient. Generating predicted image data having substantially the same quality as the image data, detecting an error between the first image data and the predicted image data, and determining whether to use the intermediate image data as an output image based on the error A recording medium.

According to the present invention, the prediction coefficient calculation unit generates an observation equation using the prediction tap as student data, the corresponding original image pixel as teacher data, and solves the observation equation to generate a prediction coefficient. The pixel value updating unit creates an observation equation using the prediction coefficient from the prediction coefficient calculation unit as student data and using the corresponding original image data as teacher data.
By solving this observation equation, optimal values of a plurality of updated pixel values for a given coefficient can be obtained at the same time.

[0013]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described below. FIG. 2 shows a configuration of an embodiment of an image processing apparatus to which the present invention is applied.

The transmitting apparatus 101 is supplied with digitized image data. The transmission device 101 forms an average value for each of a plurality of pixels of the input image data, and compresses the data amount by replacing the plurality of pixels with the average value, and converts the resulting encoded data into an optical disk, a magnetic tape, or the like. Or transmitted via a transmission line 103 such as a broadcast line (satellite broadcast), a telephone line, the Internet, or the like.

The receiving device 104 reproduces the encoded data recorded on the recording medium 102 or
Receiving the encoded data transmitted through the. And decoding the encoded data. That is, the values of the thinned pixels are restored. The decoded image obtained from the receiving device 104 is supplied to a display (not shown) and displayed on the display.

FIG. 3 shows an example of the transmitting apparatus 101. I
The / F (interface) 111 performs a process of receiving image data supplied from the outside and a process of transmitting encoded data to the transmitter / recording device 116. ROM 112
Stores an IPL (Initial Program Loading) program and the like. The RAM 113 stores the external storage device 11
5 (OS (Operati
ng System)) and application programs, and data necessary for the operation of the CPU 114.

The CPU 114 loads a system program and an application program from the external storage device 115 to the RAM 113 in accordance with the IPL program stored in the ROM 112, and executes the application program under the control of the system program. That is, an encoding process described later is performed on the image data supplied from the interface 111.

The external storage device 115 is, for example, a hard disk, and stores a system program, an application program, and data. Transmitter / storage device 11
6 records the encoded data supplied from the interface 111 on the recording medium 102 or transmits the encoded data via the transmission path 103. Interface 111, ROM 11
2. The RAM 113, the CPU 114, and the external storage device 115 are connected to each other via a bus.

In the transmitting apparatus 101 having the above-described configuration, when image data is supplied to the interface 111, the image data is supplied to the CPU 114.
The CPU 114 encodes the image data and supplies the resulting encoded data to the interface 111. The interface 111 transmits the encoded data to the transmitter /
The data is recorded on the recording medium 102 via the recording device 116 or transmitted to the transmission path 103.

FIG. 4 shows a functional configuration of the transmitting apparatus 101 of FIG. 3 other than the transmitting / recording apparatus 116, that is, the encoder. The encoder is hardware,
It can be realized by software or a combination of both. For example, by mounting a recording medium in which a program for an encoding process as shown in a flowchart described later is stored in a drive, the program can be installed in the external storage device 115 and a function as an encoder can be realized.

As a recording medium for providing a user with a computer program for performing the above-described processing,
In addition to recording media such as magnetic disks, CD-ROMs, and solid-state memories, communication media such as networks and satellites can be used.

In the encoder shown in FIG. 4, the input original image data is supplied to the image reduction circuit 1, the prediction coefficient operation circuit 4, the pixel value update circuit 5, and the error operation circuit 7 as the image data. ing. Image reduction circuit 1
Shows the supplied original image (high-resolution image) as shown in FIG.
As shown in the above, the image is divided into blocks of 3 × 3 pixels, and the average value of the pixel values of 9 pixels in each block is used as the pixel value of the pixel of the upper hierarchical image (low resolution image) located at the center of the block An initial upper layer image is generated and output to the upper layer image memory 2. Therefore, the upper hierarchical image (hereinafter, referred to as the upper image) is obtained by reducing the vertical and horizontal sizes of the original image to １ ／.

When forming an initial upper image, in addition to the average value, the value of the pixel located at the center of each block, the intermediate value of the plurality of pixel values of each block, and the largest number of the plurality of pixel values of each block A value, a pixel formed by thinning, or the like may be used.

The upper hierarchical image memory 2 includes an image reducing circuit 1
Is stored. The upper layer image memory 2 updates the stored pixel value of the upper image using the pixel value input from the pixel value updating circuit 5. Further, the upper hierarchical image memory 2 outputs the stored upper image data to the frame memory 9 via the switch 8.

The prediction tap acquisition circuit 3 sequentially determines the pixels of the upper image stored in the upper layer image memory 2 as the pixel of interest, and calculates the pixel values of the pixel of interest and the pixels in the vicinity of the pixel of interest. , A pixel value updating circuit 5 and a mapping circuit 6.

For the sake of simplicity, a case where the size of the updated pixel value block (prediction tap) is 3 × 3 and the size of the prediction coefficient tap is 3 × 3 as shown in FIG. 5 will be described. The update pixel value block (prediction tap) means a block composed of a plurality of pixels extracted for prediction. Further, the prediction coefficient tap means a plurality of coefficient groups used for prediction. In FIG. 5, x indicates a pixel to be updated, and X indicates a pixel whose pixel value is fixed. Also,
C, which will be described later, means a coefficient, Y ′ indicates a predicted value, and Y indicates a pixel value of an original image.

For example, when the pixel x5 shown in FIG. 5 is determined as the target pixel, the prediction coefficient calculation circuit 4 sends the target pixel x5
A prediction tap consisting of 3 × 3 pixels (pixels x1 to x9) centered at 5 is provided. The pixel value update circuit 5 includes all the prediction taps including any of the three 3 × 3 pixels centered on the target pixel x5 (see FIG. 7x7 centered on pixel x5
Pixel). The mapping circuit 6 includes 40 (= 49-9) pixels (pixels X1 to X40) obtained by removing 3 × 3 pixels centered on the target pixel x5 from 7 × 7 pixels centered on the target pixel x5. Supplied.

The prediction coefficient calculation circuit 4 uses the prediction taps (pixels x1 to x9) centered on the pixel of interest x5 supplied from the prediction tap acquisition circuit 3 as learning data (student data), and assigns corresponding pixels of the original image. By generating an observation equation as teacher data and solving the observation equation by the least squares method, prediction coefficients for nine modes (modes 1 to 9) are calculated as shown in FIG. 5,
And a mapping circuit 6.

The prediction coefficient tap of each mode composed of 3 × 3 prediction coefficients is a 3 × 3 pixel centered on a pixel of a lower hierarchical image (hereinafter referred to as a lower image) at a position corresponding to a target pixel. It is used when predicting the pixel value of each pixel.

More specifically, in the pixel arrangement of FIG. 1, the prediction coefficient tap used when predicting pixel a is the prediction coefficient tap of mode 1, and the prediction coefficient tap used when predicting pixel b is used. The tap is a prediction coefficient tap in mode 2, a prediction coefficient tap used when predicting pixel c is a prediction coefficient tap in mode 3, and a prediction coefficient tap used when predicting pixel h is prediction in mode 4. The prediction coefficient tap used when predicting the pixel i is the prediction coefficient tap of mode 5, and the prediction of the pixel d
The prediction coefficient tap used when predicting is the prediction coefficient tap of mode 6, the prediction coefficient tap used when predicting the pixel g is the prediction coefficient tap of mode 7,
The prediction coefficient tap used when predicting the pixel f is the prediction coefficient tap in mode 8, and the prediction coefficient tap used when predicting the pixel e is the prediction coefficient tap in mode 9.

FIG. 7A shows a lower image predicted from the upper image shown in FIG. For example, the prediction coefficient taps (prediction coefficients c11 to c19) in mode 1 and the pixels (pixels X11, X12,
X13, x2, x3, X17, x5, x6, X21) and a linear primary combination with the pixel value of the lower-level image at the position corresponding to the target pixel (pixel Y5 'in FIG. 7B). The pixel value of (pixel Y1 'in FIG. 7B) is calculated. Further, the prediction coefficient taps in mode 9 (prediction coefficients c91 to c9)
9) and the pixel adjacent to the lower right image of the pixel of the lower image at the position corresponding to the target pixel by a linear linear combination of the pixel value of the pixel forming the prediction tap centered on the pixel x3 (FIG. 7B).
The pixel value of the pixel Y9 ') is calculated.

The pixel value updating circuit 5 simultaneously updates the 3 × 3 pixel values centered on the target pixel, and outputs the updated pixel values to the upper layer image memory 2 and the mapping circuit 6. I have.

FIG. 8 shows a detailed configuration example of the pixel value updating circuit 5. The normal equation generation circuit 11 calculates a prediction coefficient input from the prediction coefficient calculation circuit 4, a pixel value of a pixel forming a prediction tap input from the prediction tap acquisition circuit 3,
A normal equation is generated from a predicted value and a true value (pixel value of the original image) using the corresponding pixel value of the original image, and is output to the pixel value determination circuit 12. Pixel value determination circuit 1
2 is centered on the target pixel of the upper image, which minimizes the error between the predicted value and the true value by the input normal equation.
The pixel values (update values) of × 3 pixels are calculated simultaneously. In the following, nine pixels that are updated simultaneously are referred to as updated pixel value taps.

Here, the generated normal equation will be described. The normal equation is generated using pixel values in a range where the updated pixel value tap and the prediction coefficient tap partially overlap. For example, 3 × 3 with the pixel x5 shown in FIG.
When the pixel values of the pixels (pixels x1 to x9) are updated (updated pixel value taps), the pixel values of the pixels (pixels X1 to X40) other than the updated pixel value taps in the area surrounded by the broken line, and all Are fixed, and the prediction coefficient taps are moved within the region surrounded by the broken line to predict the pixel value of the lower image.

For example, the center of the prediction coefficient tap is pixel x3
Is moved to a position overlapping with the prediction coefficient tap (prediction coefficients c11 to c19) of the mode 1 and the pixel x3
3 × 3 pixels (pixels X11, X12, X13, x2
, X3, X17, x5, x6, X21), the pixel value (predicted value) of the pixel Y1 'at the upper left of the pixel Y5' of the lower image corresponding to the position of the pixel x3 is calculated. Is done. This pixel value Y1 'can be expressed by the following equation (1).

Y1 '= c11X11 + c12X12 + c13X13 + c14x2 + c15x3 + c16X17 + c17x5 + c18x6 + c19X21 (1) Similarly, the pixel values Y2' to Y9 'are also calculated by using the prediction coefficients and the pixel values of the upper image. When expressed by linear linear combinations and the obtained nine equations are rewritten using a matrix, an observation equation such as the following equation is established.

Y '= cX where Y' is a matrix composed of a set of pixel values Y1 'to Y9', c is a matrix composed of a set of prediction coefficients c11 to c99, and X is the pixel value of the upper image. It is a matrix consisting of a set.

Next, it is considered that a least square method is applied to this observation equation to obtain a predicted value Y 'close to the pixel value of the original image.

Here, paying attention again to the equation (1) which is the basis of the observation equation, the difference between the predicted value Y1 'and the corresponding pixel value Y1 of the original image is expressed by the following equation (2). become.

Y1−Y1 ′ = Y1− (c11X11 + c12X12 + c13X13 + c14x2 + c15x3 + c16X17 + c17x5 + c18x6 + c19X21) (2) By rearranging the right side of the equation (2), the following equation (2) is obtained. Obtain 3).

Y1−Y1 ′ = Y1− (c11X11 + c12X12 + c13X13 + c16X17 + c19X21) − (c14 × 2 + c15 × 3 + c17 × 5 + c18 × 6) (3) The predicted value Y1 ′ and the corresponding pixel of the original image The difference from the value Y1,
That is, the following equation (4) is obtained by rearranging the left-hand side of equation (3) as a residual and moving the constant term on the right-hand side to the left-hand side.

Y1− (c11X11 + c12X12 + c13X13 + c16X17 + c19X21) + e1 = (c14x2 + c15x3 + c17x5 + c18x6) (4) Further, other modes of the prediction coefficient tap (modes 2 to 9) ), The following equations (5) to (12) similar to the equation (4) are obtained from Yn-Yn '(n is 2 to 9).

Y2- (c21X11 + c22X12 + c23X13 + c26X17 + c29X21) + e2 = (c24x2 + c25x3 + c27x5 + c28x6) (5) Y3- (c31X11 + c32X12 + c33X13 + c36X17 + c39X21) + e3 = (c34x2 + c35x3 + c37x5 + c38x6) ... (6) Y4-(c41X11 + c42X12 + c43X13 + c46X17 + c49X21) + e4 = (c44x2 + c45x3 + c47x5 + c48x6) ... (7) Y5 -(c51X11 + c52X12 + c53X13 + c56X17 + c59X21) + e5 = (c54x2 + c55x3 + c57x5 + c58x6) ... (8) Y6-(c61X11 + c62X12 + c63X13 + c66X17 + c69X21) + e6 = (c64x2) + c65x3 + c67x5 + c68x6) (9) Y7-(c71X11 + c72X12 + c73X13 + c76X17 + c79X21) + e7 = (c74x2 + c75x3 + c77x5 + c78x6) ... (10) Y8- (c81X11 +) c82X12 + c83X13 + c86X17 + c89X21) + e8 = (c84x2 + c85x3 + c87x5 + c88x6) (11) Y9- (c91X11 + c92X12) + c93X13 + c96X17 + c99X21) + e9 = (c94x2 + c95x3 + c97x5 + c98x6) (12) Similarly, the position of the prediction coefficient tap is moved within the area enclosed by the broken line in FIG. That is, the center of the prediction coefficient tap is moved so as to sequentially overlap with all the pixels (25 pixels) in the rectangular area having the vertices of the pixels X9, X13, X28, and X32, and all the modes of the prediction coefficient tap are used. 225
(= 9 × 25) Equations similar to Equations (4) to (12) are obtained.

If these 225 equations are represented by a matrix, as shown in the following equation (13), [teacher data] + [residual error e] =
The residual equation has the form of [learning data c] × [prediction pixel value x].

[0045]

(Equation 1)

However, Expression (13) shows only the parts corresponding to Expressions (4) to (12) in order to simplify the notation. Also, aij (i = 1, 2,..., M (= 2
25), j = 1, 2,..., 9) are equal to data existing in the i-th row and j-th column of the matrix [learning data c].

In this case, the predicted pixel value xi for obtaining the predicted value Y 'close to the pixel value Y of the original image can be obtained by minimizing the following square error.

[0048]

(Equation 2)

Therefore, when the above-mentioned squared error is differentiated by the predicted pixel value xi, the result becomes 0, that is, a predicted value Y 'that satisfies the following equation is close to the pixel value Y of the original image. It is the optimal value for

[0050]

(Equation 3)

Therefore, first, equation (13) is calculated using the predicted pixel value x
By differentiating with i, the following equation is established.

[0052]

(Equation 4)

From equations (14) and (16), equation (1)
7) is obtained.

[0054]

(Equation 5)

Further, the teacher data (pixel value Y-constant term of the original image) of equation (13) is Y ″, and the relationship between teacher data Y ″, prediction coefficient c, prediction pixel value x, and residual e is Is considered, a normal equation such as the following equation (18) can be obtained from the equation (17).

[0056]

(Equation 6)

By solving the obtained normal equation by applying, for example, a sweeping-out method (Gauss-Jordan elimination method), the updated pixel value corresponding to the prediction coefficient tap supplied from the prediction coefficient calculation circuit 4 is obtained. The optimum pixel value of the tap can be obtained.

Returning to the description of FIG. The mapping circuit 6
The pixel values of the nine pixels of the update pixel value tap centered on the target pixel supplied from the pixel value update circuit 5 and the 7 × 7 centered on the target pixel supplied from the prediction tap acquisition circuit 3
40 excluding 3 × 3 pixels centering on the target pixel from the pixel
The pixel values of the pixels and the prediction coefficients of the prediction coefficient taps for the nine modes input from the prediction coefficient calculation circuit 4 are represented by a linear 1
Subsequent combining locally decodes the pixel values of the lower image (partly the range affected by the pixels of the updated pixel value taps). The pixel value of the locally decoded lower image is supplied to the error calculation circuit 7.

The error calculation circuit 7 calculates an error between the pixel value of the locally decoded lower image from the mapping circuit 6 and the corresponding pixel value of the original image. In the following description, S / N is used as the error. S / N = 20log ₁₀ (255 / er
r) (err: standard deviation of error). If the S / N is equal to or larger than the threshold value, it is determined that an optimum pixel has been generated, and the switch 8 is controlled to be turned on.
In this case, the S / N may be evaluated over the entire image instead of evaluating the S / N on the locally decoded image.

The frame memory 9 updates and stores a partially optimized upper image inputted from the upper layer image memory 2 via the switch 8 every time it is inputted. Therefore, the upper layer image memory 2
After all the pixels of the upper image stored in are stored as the target pixel, the frame memory 9 stores the optimal upper image in which all the pixels are optimized.

The optimal upper-order image stored in the frame memory 9 is output to a decoder (described later with reference to FIG. 13) at a predetermined timing together with prediction coefficient taps for nine modes. A control unit 10 is provided to control the processing of the encoder described below. The control unit 10 receives the output of the error calculation circuit 7,
A signal for controlling the switch 8 is generated. In addition, various control signals are supplied to each block in order to perform processing of the encoder.

Next, an outline of the optimum upper pixel value generation processing of the encoder will be described with reference to the flowchart of FIG. The processing described below is described corresponding to the configuration of FIG. However, the present invention is not limited to the hardware realized by the hardware having the configuration shown in FIG. 4, but may be installed from the outside or may be used in accordance with a software program stored in the ROM 112 shown in FIG.
14 may be performed. In that case, the processing of each step is performed by the CPU 1 according to the software program.
It is performed under the control of fourteen.

In step S1, the image reduction circuit 1
Divides the supplied original image (high-resolution image) into blocks of 3 × 3 pixels, and averages the pixel values of 9 pixels in each block to the upper-level image (low-resolution image) located at the center of the block. An upper-level image is generated as a pixel value of the pixel of (1) and stored in the upper-level image memory 2.

The prediction tap obtaining circuit 3 sequentially determines the pixels of the upper image stored in the upper layer image memory 2 as the pixel of interest, and determines from the upper layer image memory 2 a 7 × 7 pixel centered on the pixel of interest. Get the pixel value of a pixel. Acquired 4
The 3 × 3 pixel values centered on the pixel of interest among the nine pixel values are supplied to the prediction coefficient calculation circuit 4. Also,
The pixel value updating circuit 5 is supplied with all the acquired pixel values. Further, 40 (= 49−9) obtained by removing 3 × 3 pixels centering on the target pixel from the 49 obtained pixel values.
The pixel values of the pixels are supplied to the mapping circuit 6. For example, when the pixel x5 shown in FIG. 5 is determined as the target pixel, the pixel values of the prediction coefficient taps of 3 × 3 pixels (pixels x1 to x9) centered on the target pixel x5 are supplied to the prediction coefficient calculation circuit 4. The pixel value of the 7 × 7 pixels centered on the target pixel x5 is supplied to the pixel value updating circuit 5, and the target pixel x5
The pixel values of 40 (= 49−9) pixels obtained by removing 3 × 3 pixels centered on the target pixel x5 from 7 × 7 pixels centered on 5 are supplied to the mapping circuit 6.

In step S 2, the prediction coefficient calculation circuit 4 uses the prediction tap of 3 × 3 pixels centered on the pixel of interest supplied from the prediction tap acquisition circuit 3 as learning data (student data), and calculates the corresponding original image. An observation equation is generated using pixels as teacher data, and a prediction coefficient tap for nine modes is obtained by applying the least squares method to solve the equation, and is supplied to the pixel value updating circuit 5 and the mapping circuit 6. When obtaining the prediction coefficient, an equation is established for all the pixels in the screen.

In step S3, the pixel value updating circuit 5
The normal equation generation circuit 11 includes a prediction coefficient tap input from the prediction coefficient calculation circuit 4, a 7 × 7 pixel value centered on the pixel of interest supplied from the prediction tap acquisition circuit 3,
Using the pixel values of the original image and the corresponding pixel values of the original image, an observation equation as shown in Expression (13) is generated.
Output to The pixel value determination circuit 12 solves the input observation equation by applying the least square method, and outputs the obtained pixel value of the updated pixel value tap to the upper layer image memory 2 and the mapping circuit 6.

The upper layer image memory 2 uses the pixel value of the updated pixel value tap input from the pixel value updating circuit 5 to
The pixel value of the corresponding pixel of the upper image stored so far is updated. The mapping circuit 6 calculates the pixel value of the updated pixel value tap inputted from the pixel value updating circuit 5 and the 7 × 7 centered on the pixel of interest inputted from the prediction tap acquiring circuit 3.
40 excluding 3 × 3 pixels centering on the target pixel from the pixel
A linear primary combination of the pixel value of the pixel and the prediction coefficient taps for the nine modes input from the prediction coefficient calculation circuit 4 is calculated, and the pixel value of the lower image is partially locally decoded. The pixel value of the locally decoded lower image is supplied to the error calculation circuit 7.

In step S4, the error calculation circuit 7
Calculates the S / N of the pixel value of the locally decoded lower image from the mapping circuit 6 and the corresponding pixel value of the original image, and determines whether the S / N is equal to or greater than a predetermined threshold value . If it is determined that S / N is not greater than or equal to a predetermined threshold,
Steps S2 to S4 are repeated. If it is determined that S / N is equal to or greater than the predetermined threshold, the process proceeds to step S5.

In step S 5, the switch 8 is turned on under the control of the error calculation circuit 7, and a partially optimized upper image is output from the upper layer image memory 2 to the frame memory 9 via the switch 8.

By executing this optimum upper pixel value generation processing on all the pixels of the upper image stored in the upper layer image memory 2, all the pixels are optimized in the frame memory 9. The optimum top image is stored. The stored optimal upper image is output to the decoder at a predetermined timing together with the prediction coefficient taps for the nine modes.

Some examples of the processing of the encoder shown in FIG. 9 will be described. The first method shown in the flowchart of FIG. 10 is an example in which each pixel is updated once for each update of the prediction coefficient.

In step S21, the encoder
An upper image is generated by reducing the original image. Then, the encoder updates the prediction coefficients of all pixels on the entire screen (Step S22). Next step S23
In, the encoder updates the pixel value of the block (synonymous with the update pixel value tap). In step S24, it is determined whether or not processing of all blocks has been completed.
If not, the process returns to step S22.
The process is repeated.

When it is determined in step S24 that updating of the pixel values of all blocks has been completed, the encoder
The updated upper image is mapped (locally decoded), and an S / N indicating an error from the lower image is calculated (step S25). In step S26, the encoder determines whether S / N is greater than or equal to a threshold. If S / N is equal to or larger than the threshold, the updated upper image is output to the frame memory 9 and the prediction coefficient is output (step S27). If S / N is smaller than the threshold value in step S26, the process returns to step S22, and the processes in and after step S22 are repeated.

FIG. 11 is a flowchart showing the second method. The second method is to update the pixel value of only one block for one update of the prediction coefficient. Therefore, when the updating of the pixel values of all the blocks is not completed, the process returns to step S22 (update of the prediction coefficients of the entire screen) instead of step S23 (update of the pixel values) only in FIG. This is different from the flowchart.

FIG. 12 is a flowchart showing the third method. In the third method, the evaluation of the updated pixel value is performed after the prediction coefficient is updated and after the pixel value is updated.

In FIG. 12, the original image is reduced (step S
After 31), the prediction coefficients of all pixels are updated (step S32). The encoder maps the updated upper image and calculates the S / N, which is the error from the lower image.
(Step S33). In step S34, it is determined whether S / N is equal to or greater than a threshold. If S / N is above the threshold,
The encoder outputs the updated upper image to the frame memory 9 and outputs the prediction coefficient (step S3).
5).

If the S / N is smaller than the threshold value in step S34, the process proceeds to step S36, and the process proceeds to step S36.
At 36, the encoder updates the pixel values of the block. In step S37, it is determined whether or not the processing of all blocks has been completed.
Returning to step S36, the process is repeated.

If it is determined in step S37 that the updating of the pixel values of all the blocks has been completed, the encoder
The updated upper image is mapped to calculate an S / N, which is an error from the lower image (step S38). Step S
At 39, it is determined whether the S / N is greater than or equal to a threshold. S / N
Is greater than or equal to the threshold, the encoder
Then, the updated upper image is output, and the prediction coefficient is output (step S35). If the S / N is smaller than the threshold value in step S39, the process returns to step S32, and the above-described processing after step S32 is repeated.

In the above-described embodiment of the present invention, both the prediction coefficient and the pixel value of the upper image are optimized. However, in the present invention, it is also possible to optimize only the pixel values by obtaining the prediction coefficients in advance. In this case, the prediction coefficients are generated in advance by performing the same processing as the prediction coefficient generation processing in the encoder using the digital image for coefficient determination. In addition, since the prediction coefficient is shared by the encoder and the decoder, recording or transmission to a recording medium is not required.

Next, an example of the configuration of a decoder for restoring the original image (predicting the lower image) from the optimal upper image output from the encoder will be described with reference to FIG. In this decoder, the optimal upper-layer image input from the encoder is stored in the optimal upper-layer image memory 21,
The prediction coefficient taps for the nine modes are supplied to the mapping circuit 23.

The prediction tap acquisition circuit 22 sequentially determines the pixels of the optimal upper-layer image stored in the optimal upper-layer image memory 21 as the pixel of interest, and determines from the optimal upper layer image memory 21 the center of the pixel of interest. A prediction tap of × 3 pixels is acquired and output to the mapping circuit 23.

The mapping circuit 23 calculates a linear linear combination of the pixel values of the nine pixels forming the prediction taps input from the prediction tap acquisition circuit 22 and the prediction coefficient taps for the nine modes supplied from the encoder. By doing so, 3 × centering on the pixel of the lower image corresponding to the position of the pixel of interest
The pixel values of three pixels are predicted (the pixels of the original image are restored). The predicted pixel value of 3 × 3 pixels of the lower image is output to the frame memory 24 and stored. The pixel values of the lower image stored in the frame memory 24 are output to a display (not shown) at a predetermined timing for each frame.

The original image restoration processing of this decoder is as follows.
This will be described with reference to FIG. This original image restoration process is started after the optimal upper image generated by the encoder is stored in the optimal upper layer image memory 21 and the prediction coefficient taps for nine modes are supplied to the mapping circuit 23.

In step S 11, the prediction tap acquisition circuit 22 determines one pixel of the pixels of the optimal upper image stored in the optimal upper hierarchical image memory 21 as a target pixel. In step S <b> 12, the prediction tap acquisition circuit 22 acquires a 3 × 3 pixel prediction tap centered on the pixel of interest from the optimal upper layer image memory 21, and outputs the prediction tap to the mapping circuit 23.

In step S13, the mapping circuit 23 calculates a linear 1 between the pixel values of the nine pixels forming the prediction taps input from the prediction tap acquisition circuit 22 and the prediction coefficient taps for the nine modes supplied from the encoder. By calculating the next combination, the pixel value of 3 × 3 pixels centered on the pixel of the lower image corresponding to the position of the target pixel is predicted (the pixel of the original image is restored). Predicted lower image 3
The pixel value of the x3 pixel is output to the frame memory 24,
It is memorized.

In step S14, the prediction tap acquisition circuit 22 determines whether or not all the pixels of the optimal upper-layer image stored in the optimal upper-layer image memory 21 have been determined as the target pixels, and determines all the pixels of interest. Steps S11 to S14 are repeated until it is determined that the pixel is determined. If it is determined that all the pixels have been determined as the target pixel, the process proceeds to step S15.

In step S15, the pixel value of the lower image stored in the frame memory 24 is output to a display (not shown) at a predetermined timing for each frame.

According to the present embodiment, it is possible to obtain a high-order image in which the S / N of the restored image is large as compared with the conventional method.

[0089]

As described above, according to the present invention, it is possible to simultaneously optimize the pixel values of a plurality of pixels in block units. Thereby, the processing can be simplified and the processing time can be reduced.

[Brief description of the drawings]

FIG. 1 is a schematic diagram showing an arrangement of pixels for explaining an encoding proposed above.

FIG. 2 is a block diagram showing an overall configuration of an image data conversion device to which the present invention is applied.

FIG. 3 is a block diagram illustrating a functional configuration example of a transmission device in FIG. 2;

FIG. 4 is a block diagram illustrating a configuration example of an encoder to which the present invention has been applied.

FIG. 5 is a diagram for explaining processing of a prediction tap acquisition circuit 3 of FIG. 4;

FIG. 6 is a diagram illustrating prediction coefficient taps.

FIG. 7 is a diagram illustrating a lower hierarchical image.

8 is a block diagram illustrating a configuration example of a pixel value updating circuit 5 in FIG.

FIG. 9 is a flowchart illustrating an outline of an optimum pixel value generation process of the encoder in FIG. 4;

FIG. 10 is a flowchart illustrating an example of an optimum pixel value generation process of the encoder in FIG. 4;

11 is a flowchart illustrating another example of the optimum pixel value generation processing of the encoder in FIG.

12 is a flowchart illustrating still another example of the optimum pixel value generation processing of the encoder in FIG.

13 is a block diagram illustrating a configuration example of a decoder for restoring an original image from an optimum upper-level image generated by the encoder in FIG.

FIG. 14 is a flowchart illustrating an original image restoration process of the decoder in FIG. 13;

[Explanation of symbols]

1 ... image reduction circuit, 2 ... upper layer image memory,
3 Prediction tap acquisition circuit, 4 Prediction coefficient operation circuit, 5 Pixel value update circuit, 6 Mapping circuit, 7 Error operation circuit, 11 Normal equation generation circuit , 12... Pixel value determination circuit

 ──────────────────────────────────────────────────続 き Continuation of the front page (72) Inventor Kenji Takahashi 6-7-35 Kita-Shinagawa, Shinagawa-ku, Tokyo Inside Sony Corporation (72) Inventor Yoshinori Watanabe 6-35, Kita-Shinagawa, Shinagawa-ku, Tokyo Sony Corporation F term (reference) 5C059 KK15 MA19 MA32 SS11 SS20 SS26 TA00 TA06 TA30 TB08 TC03 TD16 UA33 UA39 5C076 AA21 AA22 BB24 BB40 5C078 BA21 CA00 CA25 CA31 DA00 DA02 DB04 DB13 9A001 BB06 EE04 GG05 H23

Claims (13)

[Claims] 1. An image data conversion device for converting first image data into second image data having a lower quality than said first image data, comprising: converting said first image data to said second image data; An intermediate image data generation unit that generates substantially the same intermediate image data; a storage unit that stores the intermediate image data; and a plurality of pixel data for each block that is a part of one screen from the intermediate image data A block extracting unit that outputs a generated or previously obtained prediction coefficient; and a block extraction unit that extracts based on the prediction coefficient, the intermediate image data, and the first image data. A pixel value updating unit for updating a pixel value of the intermediate image data obtained, and the first image based on the intermediate image data whose pixel value has been updated by the pixel value updating unit and the prediction coefficient. A predicted image data generation unit that generates predicted image data substantially the same as the data; an error detection unit that detects an error between the first image data and the predicted image data; and, based on the error, the intermediate image data. An image data conversion device, comprising: a control unit for determining whether to output an image. 2. The prediction coefficient generation unit generates a prediction coefficient based on the intermediate image extracted by the block extraction unit and first image data at a position corresponding to the extracted intermediate image data. The image data conversion device according to claim 1, wherein: 3. The apparatus according to claim 1, wherein the error detection unit is configured to detect the first image for one screen.
2. The image data conversion apparatus according to claim 1, wherein an error between the image data of the image data and the predicted pixel data for one screen is detected. 4. The normal value generating section that generates a normal equation based on the prediction coefficient, the intermediate image data, and the first image data, and solves the normal equation. The image data conversion device according to claim 1, further comprising a pixel value determination unit that determines an updated pixel value of the intermediate image data. 5. The image data conversion device according to claim 3, wherein the pixel value determination unit solves the normal equation using a least squares method. 6. An image data conversion method for converting first image data into second image data having a lower quality than the first image data, wherein the first image data has substantially the same quality as the second image data. Generating intermediate image data; extracting a plurality of pixel data for each block that is a part of one screen from the intermediate image data; outputting a generated or previously obtained prediction coefficient Updating a pixel value of the intermediate image data extracted by the block extracting unit based on the prediction coefficient, the intermediate image data, and the first image data; and an intermediate image having the updated pixel value. Generating predicted image data having substantially the same quality as the first image data based on the data and the prediction coefficient; and generating the first image data and the predicted image data. An image data conversion method, comprising the steps of: detecting an error in the data; and determining whether to use the intermediate image data as an output image based on the error. 7. The method according to claim 6, wherein the step of outputting the prediction coefficient is performed based on the extracted intermediate image and first image data at a position corresponding to the extracted intermediate image data. The image data conversion method described in 1. 8. The image according to claim 6, wherein the step of detecting an error detects an error between the first image data for one screen and the predicted pixel data for one screen. Data conversion method. 9. The method of updating the pixel value, comprising: generating a normal equation based on the prediction coefficient, the intermediate image data, and the first image data; and solving the normal equation, Determining the updated pixel value of the intermediate image data. 10. The method according to claim 8, wherein the step of determining the pixel value solves the normal equation using a least squares method. 11. A learning apparatus for learning pixel values of the second image data when converting the first image data into second image data having a lower quality than the first image data. An intermediate image data generating unit for generating intermediate image data having substantially the same quality as the second image data from the image data; a storage unit for storing the intermediate image data; A block extraction unit that extracts a plurality of pixel data for each block, which is a unit; a prediction coefficient generation unit that outputs a generated or previously obtained prediction coefficient; a prediction coefficient, the intermediate image data, and the first A pixel value updating unit that updates a pixel value of the intermediate image data extracted by the block extracting unit based on the image data; and an intermediate image data whose pixel value is updated by the pixel value updating unit. A predicted image data generation unit that generates predicted image data substantially the same as the first image data based on the prediction coefficient; and an error detection unit that detects an error between the first image data and the predicted image data. A control unit that determines whether or not to use the intermediate image data as an output image based on the error, wherein the pixel value updating unit sets the prediction coefficient as student data,
A learning device, wherein the pixel value of the intermediate image data is updated by a least square method using the corresponding first image data as teacher data. 12. A learning method for learning pixel values of the second image data when converting the first image data into second image data having a lower quality than the first image data. Generating intermediate image data having substantially the same quality as the second image data from the image data; and extracting a plurality of pixel data for each block that is a part of one screen from the intermediate image data. Or outputting a prediction coefficient obtained in advance; updating a pixel value of intermediate image data extracted based on the prediction coefficient, the intermediate image data, and the first image data; Generating predicted image data having substantially the same quality as the first image data based on the intermediate image data whose values have been updated and the prediction coefficient; Detecting an error in the predicted image data; and determining whether to use the intermediate image data as an output image based on the error. The step of updating a pixel value comprises: Using the first image data as the teacher data and updating the pixel values of the intermediate image data by the least squares method. 13. A recording medium on which a computer-controllable program for converting image data for converting first image data into second image data lower in quality than said first image data is recorded. Generating intermediate image data of substantially the same quality as the second image data from the first image data; and extracting a plurality of pixel data for each block that is a part of one screen from the intermediate image data Generating a prediction coefficient based on the extracted intermediate image and first image data at a position corresponding to the extracted intermediate image data; and calculating the prediction coefficient, the intermediate image data, and the Updating the pixel values of the extracted intermediate image data based on the first image data; and updating the pixel values of the extracted intermediate image data with the updated intermediate image data. Generating predicted image data having substantially the same quality as the first image data based on the prediction coefficient; detecting an error between the first image data and the predicted image data; And determining whether or not the intermediate image data is an output image.