JPH1093981A - Image encoder and image encoding method, image decoder and image decoding method, transmitting method and recording medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To obtain a decoded image of high image quality from data of the lowest hierarchy that is acquired in hierarchical encoding. SOLUTION: Images of the 2nd and the 3rd hierarchies are formed by successively thinning an image of the 1st hierarchy which is the original image in thinning circuits 11 and 12. Then, the image of the 2nd hierarchy is corrected in the optimum correction data calculating part 13, a predictive value of the image of the 1st hierarchy is predicted from corrected data that is acquired from the result, and correction data of the image of the 2nd hierarchy that makes the predictive error of the predictive value equal to or lower than a prescribed value is sought. Similarly, in the optimum correction data calculating circuit 14, correction data of the image of the 3rd hierarchy is sought.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

The present invention relates to an image encoding device and an image encoding method, an image decoding device and an image decoding method, a transmission method, and a recording medium. In particular, the present invention relates to an image encoding device and an image encoding method for performing hierarchical encoding for dividing an image into a plurality of layers having different numbers of pixels, an image decoding device and an image decoding method, a transmission method, and a recording medium.

[0002]

2. Description of the Related Art An original image (original image) is used as an image of a first hierarchy (highest hierarchy), an image of a second hierarchy in which the number of pixels is sequentially reduced (resolution is sequentially reduced), and an image of a third hierarchy. Encoding for forming images,... (Hereinafter referred to as hierarchical encoding as appropriate) has been proposed.

According to the hierarchical coding, a plurality of hierarchical images are transmitted from a transmitting side to a receiving side, and the receiving side displays the images of each hierarchical level on a monitor corresponding to each of the plurality of hierarchical levels. can do.

Further, in hierarchical coding, for example,
Data about the lowest hierarchical image (the image with the least number of pixels) is subjected to a stronger error correction process than the other hierarchical images. When an error that cannot be corrected for a hierarchical image occurs, at worst, only the data for the lowest hierarchical image can be obtained on the receiving side in a normal manner. As a result, on the receiving side, a higher-layer image can be obtained from the data of the lowest-layer image by, for example, an interpolation process. Therefore, according to the hierarchical coding, robustness against errors is improved. be able to.

FIG. 47 shows the configuration of an example of a conventional image encoding apparatus that performs the above-described hierarchical encoding.
The image data to be encoded is the data of the first hierarchy (the highest hierarchy) as the thinning circuit 11 and the signal processing circuit 50.
1.

In the thinning circuit 11, the number of pixels of the image data of the first hierarchy is thinned to form the image data of the second hierarchy one level lower and supplied to the thinning circuit 12 and the signal processing circuit 501. . In the thinning circuit 12, the number of pixels of the image data of the second hierarchy is thinned,
Further, image data of the third hierarchy lower by one level is formed and supplied to the signal processing circuit 501.

The signal processing circuit 501 performs, for example, error correction processing and other necessary signal processing on the image data of the first to third layers, and thereafter multiplexes the image data.
Output as encoded data. Note that the signal processing circuit 5
In No. 01, image data of the third hierarchy, which is the lowest hierarchy, is subjected to stronger error correction than data of other hierarchies.

FIG. 48 shows a configuration of an example of a conventional image decoding apparatus for hierarchically decoding encoded data output from the image encoding apparatus of FIG.

In the signal processing circuit 601, the encoded data is separated into image data of the first to third layers, further subjected to error correction processing and the like, and output. The image data of the first layer is directly output as a decoded image of the first layer. Further, the image data of the second hierarchy is supplied to the interpolation circuit 602. The interpolation circuit 602
By performing interpolation processing on the image data of the second hierarchy, image data having the same number of pixels as the image data of the first hierarchy one level higher is generated, and is output as a decoded image of the first hierarchy.

[0010] The image data of the third hierarchy is supplied to an interpolation circuit 603.
Supplied to The interpolation circuit 603 performs an interpolation process on the image data of the third hierarchy, thereby obtaining the next higher second image data.
Image data having the same number of pixels as the image data of the hierarchy, that is,
A decoded image of the second layer is generated and output to the interpolation circuit 604. The interpolation circuit 604 performs the same interpolation processing on the output of the interpolation circuit 603 as in the case of the interpolation circuit 602, thereby generating image data having the same number of pixels as the image data of the next higher-level first hierarchy. Then, the decoded image is output as a first-layer decoded image.

Therefore, in the image decoding apparatus, the first
Even if the image data of the hierarchy cannot be obtained for some reason, the decoded image of the first hierarchy can be obtained from the image data of the second hierarchy. Similarly, even if image data of the first and second layers cannot be obtained for some reason, decoded images of the first and second layers can be obtained from image data of the third layer. .

[0012]

However, the decoded image of the upper layer obtained only from the image data of the third layer, which is the lowest layer, has a significantly deteriorated image quality.

The present invention has been made in view of such a situation, and it is an object of the present invention to obtain a high-quality decoded image only from encoded data of the lowest hierarchy.

[0014]

According to the first aspect of the present invention, there is provided an image coding apparatus, comprising: a second layer image data having a smaller number of pixels than the first layer image data; Correction means for correcting the image data of the first hierarchy, outputting correction data, prediction means for predicting the predicted value of the image data of the first hierarchy from the correction data, and prediction for the image data of the first hierarchy Calculating means for calculating a prediction error of the value, determining means for determining the appropriateness of the correction data output by the correcting means based on the prediction error, and correcting the correction data in accordance with the determination result by the determining means. Output means for outputting as hierarchical image data.

According to the image encoding method of the present invention, the image data of the second layer having a smaller number of pixels than the image data of the first layer is formed, and the image data of the second layer is corrected. , Outputting correction data, predicting a prediction value of the image data of the first layer from the correction data, calculating a prediction error of the prediction value with respect to the image data of the first layer, and calculating a prediction error of the first layer. The determination of the correctness of the correction data based on the prediction error is repeated, and the corrected correction data is used as the image data of the second hierarchy.

According to a twelfth aspect of the present invention, there is provided an image decoding apparatus, an image decoding method according to the eighteenth aspect, a transmission method according to the nineteenth aspect, and a recording medium according to the twentieth aspect, wherein encoded data is Forming image data of a second layer having a smaller number of pixels than image data of the first layer, correcting the image data of the second layer, outputting correction data, and converting the corrected data to the first layer Of the image data of the first hierarchy, the prediction error of the prediction value for the image data of the first hierarchy is calculated, and based on the prediction error of the first hierarchy,
The determination of the correctness of the correction data is repeated, and the corrected correction data is included as the image data of the second hierarchy.

According to a twenty-first aspect of the present invention, there is provided an image coding apparatus, comprising: a classifying means for classifying pixels constituting an image of a first hierarchy into a predetermined class according to its properties; and a predetermined mapping coefficient for each class. By performing a predetermined calculation using a mapping coefficient storage unit that stores a target pixel in a first layer image and a mapping coefficient corresponding to the class of the target pixel. Computing means for calculating image data of the second layer in which the number of pixels of the image data of the first layer is reduced.

According to the image coding method of the present invention, the pixels constituting the image of the first hierarchy are classified into predetermined classes according to their properties, and a predetermined mapping coefficient is stored for each class. From the stored mapping coefficient storage means,
By reading a mapping coefficient of a class corresponding to a target pixel of interest in the image of the hierarchy of, and performing a predetermined operation using the mapping coefficient and the target pixel,
It is characterized in that image data of a second hierarchy in which the number of pixels of the image data of the first hierarchy is reduced is calculated.

An image decoding apparatus according to claim 29, an image decoding method according to claim 35, a transmission method according to claim 36, and a recording medium according to claim 37, wherein the encoded data is The pixels constituting the image of the first hierarchy are classified into any one of predetermined classes according to their properties, and the mapping coefficient storage means storing the predetermined mapping coefficients for each class is used for the first class. A first layer image obtained by reading a mapping coefficient of a class corresponding to a target pixel of interest in an image of the first layer, and performing a predetermined operation using the mapping coefficient and the target pixel. Is characterized by including an image of the second hierarchy in which the number of pixels is reduced.

In the image encoding apparatus according to the first aspect, the forming means forms the image data of the second layer having a smaller number of pixels than the image data of the first layer, and the correcting means forms the image data of the second layer. Is corrected and the corrected data is output. The prediction means predicts a predicted value of the image data of the first hierarchy from the correction data, and the calculation means calculates a prediction error of the predicted value with respect to the image data of the first hierarchy. The determining means determines the appropriateness of the correction data output by the correcting means based on the prediction error, and the output means outputs the correction data as second-layer image data in accordance with the determination result by the determining means. It has been made to be.

In the image coding method according to the eleventh aspect, image data of a second layer having a smaller number of pixels than image data of the first layer is formed, and the image data of the second layer is corrected. Output the correction data, and from the correction data,
The prediction value of the image data of the first layer is predicted, the prediction error of the prediction value for the image data of the first layer is calculated, and the appropriateness of the correction data is determined based on the prediction error of the first layer. Is repeated, and the corrected data that becomes appropriate is used as the image data of the second hierarchy.

In the image decoding device according to the twelfth aspect, the image decoding method according to the eighteenth aspect, the transmission method according to the nineteenth aspect, and the recording medium according to the twentieth aspect, the encoded data is Forming image data of a second layer having a smaller number of pixels than image data of the first layer,
The image data of the hierarchy of is corrected, the correction data is output,
From the correction data, a prediction value of the image data of the first layer is predicted, a prediction error of the prediction value for the image data of the first layer is calculated, and correction is performed based on the prediction error of the first layer. The determination of the appropriateness of the data is repeated, and the corrected data that has become appropriate is included as the image data of the second hierarchy.

In the image coding apparatus according to the twenty-first aspect, the classification means classifies the pixels constituting the image of the first hierarchy into a predetermined class according to their properties, and the mapping coefficient storage means comprises: A predetermined mapping coefficient is stored for each class. The calculating means performs a predetermined calculation using the focused pixel of interest in the first hierarchical image and a mapping coefficient corresponding to the class of the focused pixel, thereby obtaining the first hierarchical image. The image data of the second hierarchy in which the number of pixels of the data is reduced is calculated.

In the image coding method according to the present invention, the pixels constituting the image of the first hierarchy are classified into predetermined classes according to their properties, and a predetermined mapping coefficient is stored for each class. A mapping coefficient of a class corresponding to a target pixel of interest in an image of the first layer is read from a mapping coefficient storage unit that performs the operation, and a predetermined operation is performed using the mapping coefficient and the target pixel. Thus, the image data of the second hierarchy in which the number of pixels of the image data of the first hierarchy is reduced is calculated.

In the image decoding device according to the present invention, the image decoding method according to the present invention, the transmission method according to the present invention, and the recording medium according to the present invention, the encoded data is The pixels constituting the image of the first hierarchy are classified into any one of predetermined classes according to their properties, and the mapping coefficient storage means storing the predetermined mapping coefficient for each class, The mapping coefficient of the class corresponding to the target pixel of interest in the image of the first layer is read out, and a predetermined operation is performed using the mapping coefficient and the pixel of interest, and the first layer is obtained. It contains an image of the second hierarchy in which the number of pixels of the image is reduced.

[0026]

DESCRIPTION OF THE PREFERRED EMBODIMENTS Embodiments of the present invention will be described below, but before that, the correspondence between each means of the invention described in the claims and the following embodiments will be clarified. For this reason, the features of the present invention are described as follows by adding the corresponding embodiment (however, an example) in parentheses after each means.

That is, an image coding apparatus according to claim 1 is an image coding apparatus for performing hierarchical coding, wherein image data of a second layer having a smaller number of pixels than image data of the first layer. (For example, the thinning circuits 11 and 12 shown in FIG. 2) and a correction unit (for example, FIG. 5) which corrects image data of the second layer and outputs correction data.
And a prediction unit (for example, a prediction unit that predicts a predicted value of the image data of the first layer from the correction data).
A local decoding unit 22 shown in FIG. 5), a calculating unit (for example, an error calculating unit 23 shown in FIG. 5) for calculating a prediction error of a prediction value of the image data of the first hierarchy, A determination unit (for example, the determination unit 24 shown in FIG. 5) that determines the appropriateness of the correction data output by the correction unit based on the correction data, and stores the correction data in the second hierarchy in accordance with the determination result by the determination unit. An output unit (for example, the determination unit 24 shown in FIG. 5) for outputting the image data is provided.

According to a second aspect of the present invention, in the image coding apparatus, the predicting means includes a classifying means (for example, a class classifying circuit 45 shown in FIG. 9) for classifying the correction data into a predetermined class according to the nature of the correction data. , A prediction value calculation means (for example, an adaptive processing circuit 46 shown in FIG. 9) for obtaining a prediction value corresponding to each class.

According to a third aspect of the present invention, in the image encoding apparatus, the prediction means calculates a prediction coefficient for calculating a prediction value by linear combination with the correction data.
It is characterized by having an adaptive processing circuit 46 shown in FIG. 9) and predicted value calculating means for obtaining a predicted value from a prediction coefficient and correction data (for example, the adaptive processing circuit 46 shown in FIG. 9).

According to a fourth aspect of the present invention, in the image coding apparatus, the predicting means includes a classifying means (for example, a class classifying circuit 45 shown in FIG. 9) for classifying the correction data into a predetermined class according to the property thereof. The prediction coefficient calculation means (for example, the adaptive processing circuit 46 shown in FIG. 9) for obtaining a prediction coefficient for calculating a prediction value by a linear combination with the correction data for each class and the correction data class are obtained. It is characterized by having prediction value calculation means (for example, an adaptive processing circuit 46 shown in FIG. 9) for obtaining a prediction value from a prediction coefficient and its correction data.

According to a sixth aspect of the present invention, in the image coding apparatus, the prediction means stores a prediction coefficient for calculating a prediction value by a linear combination with the correction data for each predetermined class. (For example, the prediction coefficient RO shown in FIG.
M88 and the like, and a classifying means (for example, FIG. 2) for classifying the correction data into one of predetermined classes according to its properties.
2) and prediction value calculating means (for example, a prediction circuit 89 shown in FIG. 22) for obtaining a prediction value from a prediction coefficient for a class of correction data and the correction data. It is characterized by the following.

According to a ninth aspect of the present invention, in the image encoding apparatus, the correcting means stores a correction value for correcting the image data of the second hierarchy (for example, the correction value ROM 33 shown in FIG. 7). And the like, and the image data of the second hierarchy is corrected using the correction value.

An image decoding apparatus according to a twelfth aspect of the present invention is an image decoding apparatus for decoding encoded data obtained by hierarchical encoding, and a receiving means for receiving the encoded data (for example, FIG. 18). And a decoding unit (for example, a prediction unit 72 or 73 shown in FIG. 18) that decodes the encoded data, and the encoded data is the image data of the first layer. Second with fewer pixels than
Forming the image data of the first layer, correcting the image data of the second layer, outputting the corrected data, predicting the predicted value of the image data of the first layer from the corrected data, Calculate the prediction error of the prediction value for the image data,
The determination of the correctness of the correction data based on the prediction error for the first layer is repeated, and the corrected data that has become appropriate is included as image data of the second layer.

According to a thirteenth aspect of the present invention, in the image decoding apparatus, the decoding means classifies the image data of the second hierarchy into a predetermined class according to the property thereof (for example, FIG. 19).
And a prediction value calculating means (for example, a prediction circuit 86 shown in FIG. 19) for obtaining a predicted value of the image data of the first layer corresponding to the class. And

According to a fourteenth aspect of the present invention, in the image decoding apparatus, the encoded data also includes a prediction coefficient for calculating a prediction value by a linear combination with the image data of the second hierarchy. , And a prediction value calculating means (for example, a prediction circuit 86 shown in FIG. 19) for calculating a prediction value of the image data of the first layer from the prediction coefficients and the image data of the second layer.

According to a fifteenth aspect of the present invention, the coded data includes a prediction coefficient for each predetermined class for calculating a predicted value by a linear combination with the image data of the second hierarchy. A decoding unit that classifies the image data of the second hierarchy into a predetermined class according to the property (for example, a class classification circuit 85 shown in FIG. 19);
And a prediction coefficient for the class of the image data of the second layer and the image data of the second layer, a predicted value calculating means for obtaining a predicted value of the image data of the first layer (for example, FIG. Prediction circuit 86).

According to a sixteenth aspect of the present invention, in the image decoding apparatus, the decoding means stores a prediction coefficient for calculating a prediction value by a linear combination with the image data of the second hierarchy for each predetermined class. Prediction coefficient storage means (for example, a prediction coefficient ROM 87 shown in FIG. 31), and a classification means (for example, for classifying image data of the second hierarchy into one of predetermined classes according to the property thereof). A classifying circuit 85 shown in FIG. 31) and a prediction coefficient for the class of the image data of the second layer are read out from the prediction coefficient storage means,
A prediction value calculating means (for example, a prediction circuit 86 shown in FIG. 31) for calculating a prediction value of the image data of the first hierarchy from the prediction coefficient and the image data of the second hierarchy is characterized. .

An image coding apparatus according to a twenty-first aspect is an image coding apparatus for hierarchically encoding an image, wherein pixels constituting an image of a first hierarchical level are classified into a predetermined class according to their properties. Classification means for classifying (for example, class classification circuit 113 shown in FIG. 34) and mapping coefficient storage means for storing predetermined mapping coefficients for each class (for example, mapping coefficient memory 114 shown in FIG. 34)
By performing a predetermined operation using a target pixel of interest in the image of the first layer and a mapping coefficient corresponding to the class of the pixel of interest, the image data of the first layer is obtained. A calculating means for calculating the image data of the second hierarchy in which the number of pixels is reduced (for example, the calculating circuit 1 shown in FIG. 34)
16 etc.).

An image encoding method according to claim 28 is an image encoding method for hierarchically encoding an image, wherein pixels constituting an image of a first hierarchical level are classified into a predetermined class according to their properties. A mapping coefficient storage unit that classifies and stores predetermined mapping coefficients for each class (for example, FIG. 34)
From the mapping coefficient memory 114 shown in FIG. 1), the mapping coefficient of the class corresponding to the target pixel of interest in the image of the first hierarchy is read, and a predetermined operation is performed using the mapping coefficient and the target pixel. By performing the calculation, the image data of the second layer in which the number of pixels of the image data of the first layer is reduced is calculated.

An image decoding apparatus according to a twenty-ninth aspect is an image decoding apparatus for decoding encoded data obtained by hierarchical encoding, wherein the receiving means (for example, FIG. 18) receives encoded data. And a decoding unit (for example, a prediction unit 72 or 73 shown in FIG. 18) for decoding the encoded data, and the encoded data is used to convert the image of the first layer. The constituent pixels are classified into one of the first classes according to their properties, and a mapping coefficient storage unit (for example, shown in FIG. 34) that stores a predetermined mapping coefficient for each first class. From the mapping coefficient memory 114), a first class mapping coefficient corresponding to the target pixel of interest in the image of the first layer is read, and a predetermined function is performed using the mapping coefficient and the target pixel. Obtained by performing, characterized in that it comprises an image of the second hierarchy with a reduced number of pixels of the image of the first hierarchy.

According to a thirtieth aspect of the present invention, in the image decoding apparatus, the decoding means calculates a prediction coefficient for calculating a prediction value of the first layer image data by linear combination with the second layer image data. The prediction coefficient storage means (for example, the prediction coefficient ROM 87 shown in FIG. 31) for each of the second classes and the image data of the second hierarchy are stored in the second class in accordance with the properties thereof. Classification means (for example, a class classification circuit 85 shown in FIG. 31) for classifying the
A prediction coefficient for a second class of image data of the second layer from the prediction coefficient storage means, and a prediction value for obtaining a prediction value of an image of the first layer from the prediction coefficient and the image data of the second layer And a calculating means (for example, the prediction circuit 86 shown in FIG. 31).

An image decoding method according to a thirty-fifth aspect of the present invention is an image decoding method for decoding coded data obtained by hierarchical coding, wherein the coded data forms an image of a first hierarchy. The pixels to be classified are classified into predetermined classes according to their properties, and mapping coefficient storage means (for example, FIG. 34) storing predetermined mapping coefficients for each predetermined class.
From the mapping coefficient memory 114 shown in FIG. 1), the mapping coefficient of the class corresponding to the target pixel of interest in the image of the first hierarchy is read, and a predetermined operation is performed using the mapping coefficient and the target pixel. It is characterized by including an image of the second layer obtained by performing the operation, in which the number of pixels of the image of the first layer is reduced.

A transmission method according to a thirty-sixth aspect is a transmission method for transmitting coded data obtained by hierarchical coding, wherein the coded data replaces pixels constituting an image of a first layer with the pixels. The data is classified into predetermined classes according to the properties, and for each predetermined class, a mapping coefficient storage unit (for example, a mapping coefficient memory 114 shown in FIG. 34) that stores a predetermined mapping coefficient stores the first class. The number of pixels of the first layer image obtained by reading a mapping coefficient of a class corresponding to a target pixel of interest in the image and performing a predetermined operation using the mapping coefficient and the target pixel. The image is characterized by including an image of the second hierarchy in which the number of images is reduced.

A recording medium according to a thirty-seventh aspect of the present invention is a recording medium on which encoded data obtained by hierarchical encoding is recorded, wherein the encoded data includes pixels constituting an image of a first hierarchy. , Classified into predetermined classes according to their properties,
From a mapping coefficient storage unit (for example, mapping coefficient memory 114 shown in FIG. 34) that stores a predetermined mapping coefficient for each predetermined class, a target pixel of interest in the image of the first layer And a second layer image obtained by reading a mapping coefficient of a class corresponding to the first layer and performing a predetermined operation using the mapping coefficient and the pixel of interest and having a reduced number of pixels of the first layer image. It is characterized by the following.

Of course, this description does not mean that each means is limited to those described above.

FIG. 1 shows an image processing system to which the present invention is applied (a system refers to a system in which a plurality of devices are logically assembled, and it does not matter whether the devices of each configuration are in the same housing or not. 1) shows the configuration of an embodiment.

The transmitting apparatus 1 is supplied with digitized image data. Transmission device 1
Is to hierarchically encode the input image data and record the resulting encoded data on a recording medium 2 such as an optical disk, a magneto-optical disk, a magnetic tape, a phase change disk, or The signal is transmitted via a terrestrial wave, a satellite line, a telephone line, a CATV network, the Internet or another transmission path 3.

In the receiving device 4, the encoded data recorded on the recording medium 2 is reproduced, or the encoded data transmitted via the transmission path 3 is received, and the encoded data is hierarchically decoded. You. Then, the decoded image obtained as a result is supplied to a display (not shown) and displayed.

The image processing system described above is
For example, an optical disk device, a magneto-optical disk device, a magnetic tape device, and other devices for recording / reproducing images,
Alternatively, the present invention is applied to, for example, a videophone device, a television broadcasting system, a CATV system, and other devices that transmit images. Further, since the data amount of the encoded data output from the transmission device 1 is small, the image processing system in FIG. 1 can be applied to a low transmission rate, for example, a mobile phone or other portable terminal that is convenient for movement. is there.

FIG. 2 shows an example of the configuration of the transmitting device 1.

The digital image data to be encoded, that is, the image data of the first hierarchy (highest hierarchy) is supplied to the signal processing circuit 15 as it is as the encoded data of the first hierarchy, and the thinning circuit 11 and the optimal The correction data is supplied to the correction data calculation circuit 13. In the thinning circuit 11, the number of pixels of the image data of the first hierarchy is thinned to form the image data of the second hierarchy one level lower. That is, in the thinning circuit 11, for example, as shown in FIG.
Is simply thinned out (thin in both the horizontal and vertical directions), thereby forming the image data of the second hierarchy (the portion indicated by the triangle in the figure). The image data of the second hierarchy is supplied to the thinning circuit 12 and the optimum correction data calculation circuit 13.

In the thinning circuit 12, the number of pixels of the image data of the second hierarchy is thinned out to form the image data of the third hierarchy which is lower by one. That is, in the thinning circuit 12, for example, similarly to the case of the thinning circuit 11, the image data of the second hierarchy is simply thinned to 1/9,
Thereby, the image data of the third hierarchy indicated by the mark x in FIG. 3 is formed. The image data of the third hierarchy is supplied to the optimum correction data calculation circuit 14.

As described above, in the thinning circuits 11 and 12, as shown in FIG. 4, the second layer and the third layer image data are formed from the first layer image data (original image).

In the optimum correction data calculation circuit 13, optimum correction data optimal for obtaining a decoded image of the first layer from the image data of the second layer (hereinafter, appropriately referred to as optimum correction data of the second layer). Is calculated and output to the signal processing circuit 15 as encoded data of the second hierarchy. The optimum correction data calculation circuit 14 calculates the optimum correction data (hereinafter, appropriately referred to as the third-layer optimum correction data) suitable for obtaining the optimum correction data output from the optimum correction data calculation circuit 14, The encoded data of the hierarchy is supplied to the signal processing circuit 15.

Here, the optimal correction data of the third hierarchy is optimal for obtaining the optimal correction data of the second hierarchy.
Further, since the second-layer optimal correction data is optimal for obtaining the first-layer decoded image, the third-layer optimal correction data is the same as the second-layer optimal correction data.
It can be said that this is optimal for obtaining a hierarchically decoded image.

The signal processing circuit 15 takes out a prediction coefficient (first-layer prediction coefficient), which will be described later, included in the second-layer encoded data, and includes it in the third-layer encoded data. Then, the signal processing circuit 15 performs, for example, error correction processing and other necessary signal processing on the encoded data of the first to third layers, and then multiplexes the final encoded data. Is output as In addition,
In the signal processing circuit 15, the coded data of the third hierarchy, which is the lowest hierarchy, is subjected to stronger error correction than the coded data of the other hierarchies.

The encoded data output from the signal processing circuit 13 as described above is recorded on the recording medium 2 or transmitted via the transmission path 3.

In the above description, the thinning circuits 11 and 12 perform thinning at the same rate. However, the thinning rates performed by the thinning circuits 11 and 12 need not be the same.

FIG. 5 shows an example of the configuration of the optimum correction data calculation circuit 13 shown in FIG. Note that the optimum correction data calculation circuit 14 is configured similarly to the optimum correction data calculation circuit 13, and a description thereof will be omitted.

The second layer image data from the thinning circuit 11 is supplied to the correction unit 21, and the first layer image data is supplied to the local decoding unit 22.
To be supplied. Further, the image data of the first hierarchy is also supplied to the error calculator 23.

The correction unit 21 converts the image data of the second hierarchy into
The correction is performed according to the control from the determination unit 24. The correction data obtained as a result of the correction in the correction unit 21 is supplied to the local decoding unit 22 and the determination unit 24.

The local decoding unit 22 predicts a predicted value of the first hierarchy one level higher by one based on the correction data from the correction unit 21, ie, the correction result of the image data of the second hierarchy, and calculates an error. 23.

As will be described later, the local decoding unit 22 calculates a prediction coefficient for each class for calculating a prediction value of the first hierarchy obtained by linear combination with the correction data, and outputs the correction data and the first hierarchy of the first hierarchy. A process is performed using image data (original image), and an adaptive process is performed to obtain a first-layer predicted value based on a prediction coefficient for each class. Are also supplied to the determination unit 24.

The error calculator 23 calculates a prediction error of a predicted value from the local decoder 22 with respect to the first-layer image data (original image) input thereto. This prediction error is supplied to the determination unit 24 as error information.

The determination section 24 converts the correction data output from the correction section 21 based on the error information from the error calculation section 23 into
It is determined whether or not the original image (here, the image of the first layer) is to be a layer coding result.
When the determining unit 24 determines that the correction data output from the correcting unit 21 is not appropriate as the result of the hierarchical coding of the original image, the determining unit 24 controls the correcting unit 21 and further performs
The image data of the second hierarchy is corrected, and new correction data obtained as a result is output. When the determining unit 24 determines that the correction data output from the correcting unit 21 is appropriate as the hierarchical encoding result of the original image, the determining unit 24 determines the correction data supplied from the correcting unit 21 as: The correction coefficient is supplied to the multiplexing unit 25 as the optimum correction data (the second-layer optimum correction data), and the prediction coefficient for each class supplied from the local decoding unit 22 is also supplied to the multiplexing unit 25.

The multiplexing unit 25 multiplexes the optimum correction data from the judging unit 24 and the prediction coefficient for each class, and outputs the multiplexed result as coded data of the second hierarchy. .

Next, referring to the flowchart of FIG.
The operation will be described. For the correction unit 21, the second
When the image data of the hierarchy is supplied, the correction unit 21 outputs the image data of the second hierarchy to the local decoding unit 22 and the determination unit 24 without performing any correction at first in step S1. In step S2, the local decoding unit 22 locally decodes the correction data from the correction unit 21 (initially, the second-layer image data itself as described above).

That is, in step S 2, a prediction coefficient for each class for calculating a prediction value of the first hierarchy one level higher by linear combination with the correction data from the correction unit 21 is calculated by using the correction data and the first data. Processing for obtaining the image data is performed using the image data of the hierarchy. Further, a predicted value of the image of the first layer is obtained based on the prediction coefficient for each class, and is supplied to the error calculating unit 23.

The error calculating section 23 is a local decoding section 2
When the predicted value of the first layer is received from Step 2, Step S3
In, the prediction error of the prediction value from the local decoding unit 22 with respect to the image data of the first hierarchy is calculated and supplied to the determination unit 24 as error information. Upon receiving the error information from the error calculation unit 23, the determination unit 24 uses the correction data output by the correction unit 21 as a layer coding result of the first layer image based on the error information in step S4. Is determined.

That is, in step S4, for example,
It is determined whether the error information is equal to or less than a predetermined threshold ε. If it is determined in step S4 that the error information is not smaller than or equal to the predetermined threshold value ε, it is recognized that it is not appropriate to use the correction data output by the correction unit 21 as an image of the second hierarchy, and the process proceeds to step S5. The unit 24 includes the correction unit 21
To thereby correct the image data of the second layer. The correction unit 21 operates according to the control of the determination unit 24.
The image data of the second layer is corrected by changing the correction amount (correction value す る described later), and the resulting correction data is output to the local decoding unit 22 and the determination unit 24. Then, the process returns to step S2, and thereafter, the same processing is repeated.

On the other hand, if it is determined in step S4 that the error information is equal to or smaller than the predetermined threshold ε,
It is recognized that it is proper to use the correction data output by the second layer as an image of the second hierarchy, and the determination unit 24 sets the correction data obtained when error information equal to or smaller than a predetermined threshold ε is obtained as the optimum correction data. , Together with the prediction coefficients for each class. In step S6, the multiplexing unit 25 multiplexes the optimal correction data from the determination unit 24 and the prediction coefficient for each class, and outputs the multiplexed result as coded data of the second hierarchy. finish.

As described above, the correction data when the error information becomes equal to or less than the predetermined threshold value ε is used as the image data of the second hierarchy. Based on the (correction data), it is possible to obtain an image substantially the same as the original image (the image of the first hierarchy).

In the optimum correction data calculation circuit 13, the process of obtaining the prediction coefficient for each class for calculating the prediction value of the first hierarchy one level higher by linear combination with the correction data has been described above. As described above, the processing is performed using the image data of the first hierarchy. In the optimum correction data calculation circuit 14, the processing is performed using the optimum correction data output from the optimum correction data calculation circuit 13. However, the optimal correction data calculation circuit 14 may perform an adaptive process (a process of obtaining a predicted value based on a prediction coefficient) using the second layer image output from the thinning circuit 11 as it is. It is.

FIG. 7 shows an example of the configuration of the correction unit 21 shown in FIG.

The image data of the second hierarchy is supplied to the correction circuit 32. The correction circuit 32 corrects the correction value R in accordance with the control signal from the determination section 24 (FIG. 5).
An address is given to the OM 33, whereby the correction value △ is read. Then, the correction circuit 32
The correction data is generated by adding, for example, a correction value 画像 from the correction value ROM 33 to the image data of the second hierarchy, and is supplied to the local decoding unit 22 and the determination unit 24. The correction value ROM 33 stores a combination of various correction values △ for correcting the image data of the second hierarchy (for example, a combination of correction values for correcting the image data of the second hierarchy for one frame). And
The combination of the correction value △ corresponding to the address supplied from the correction circuit 32 is read and supplied to the correction circuit 32.

Next, referring to FIG. 8, the correction unit 21 shown in FIG.
Will be described.

For example, when the image data of the second hierarchy such as one frame is supplied to the correction circuit 32, the correction circuit 3
2 receives the image data of the second hierarchy in step S11, and determines in step S12
It is determined whether a control signal has been received from (FIG. 5).
If it is determined in step S12 that the control signal has not been received, steps S13 and S14 are skipped, the process proceeds to step S15, and the correction circuit 32
The hierarchical image data is directly output as correction data to the local decoding unit 22 and the determination unit 24, and the process returns to step S12.

That is, as described above, the determination section 24 controls the correction section 21 (correction circuit 32) based on the error information, and the correction circuit 32 receives the image data of the second hierarchy. Immediately after the determination, no error information is obtained yet (because the error information is not output from the error calculation unit 23), and thus no control signal is output from the determination unit 24. For this reason, immediately after receiving the image data of the second hierarchy, the correction circuit 32 does not correct the image data of the second hierarchy (corrects by adding 0), but directly uses the local decoding unit as the correction data. 22 and the determination unit 24.

On the other hand, in step S12, the judgment unit 2
If it is determined that the control signal from the control signal 4 has been received, the correction circuit 32 outputs an address according to the control signal to the correction value ROM 33 in step S13. As a result, in step S13, a combination (set) of the correction values 補正 for correcting the one-frame image data of the second hierarchy stored at the address is read from the correction value ROM 33, and the correction circuit 32. Upon receiving the combination of the correction values か ら from the correction value ROM 33, the correction circuit 32 adds the corresponding correction value △ to each of the image data of the second layer of one frame in step S14, and thereby, Correction data obtained by correcting the image data is calculated. Thereafter, the process proceeds to step S15, in which the correction data is output from the correction circuit 32 to the local decoding unit 22 and the determination unit 24, and returns to step S12.

As described above, under the control of the determination unit 24, the correction unit 21 repeatedly outputs the correction data obtained by correcting the image data of the second hierarchy to various values.

When the coding of one frame image is completed, the judging section 24 supplies a control signal indicating this to the correcting section 21. In step S12, the correction unit 21 outputs a control signal indicating that encoding of one frame of image has been completed to the determination unit 2
It is also determined whether or not the frame has been received. If it is determined that the frame has been received, the processing for that frame is terminated. Then, after waiting for the next frame to be supplied, the process returns to step S11, and again returns to step S11.
The processing from step 11 is repeated.

FIG. 9 shows an example of the configuration of the local decoding unit 22 shown in FIG.

The correction data from the correction unit 21 is supplied to a class classification blocking circuit 41 and a predicted value calculation blocking circuit 42. The class classification blocking circuit 41 is a unit for classifying the correction data into a predetermined class according to the property thereof, and converts the correction data into a class classification block centering on the correction data of interest (correction data of interest). It is made to block.

That is, for example, in FIG.
It is assumed that the pixel indicated by the mark constitutes the image of the first layer, and the pixel indicated by the mark ● constitutes the image (correction data) of the second layer, i-th from the top and j from the left.
Assuming that the second correction data (or pixel) is represented as X _ij , the classifying blocking circuit 41 _calculates the upper left, upper, upper right, left, right, lower left, and lower left of the pixel of interest (correction data of interest) X _ij
The eight pixels X _{(i-1) (j-1)} adjacent to the lower right and lower right
X _{(i-1) j} , X _{(i-1) (j + 1)} , Xi _(j-1) , Xi _{(j + 1)} , X
_{(i-1) (j- 1)} , X _{(i-1) j} , X _{(i + 1) (j + 1)}
A class classification block composed of a total of 9 pixels (9 correction data) is configured. The classification block is supplied to the classification adaptive processing circuit 43.

In this case, the class classification block is constituted by a square block composed of 3 × 3 pixels. However, the shape of the class classification block does not need to be a square. For example, the shape can be a rectangle, a cross, or any other shape. Also,
The number of pixels constituting the class classification block is not limited to 3 × 3 (horizontal × vertical) nine pixels.

The prediction value calculation blocking circuit 42 blocks the correction data into a prediction value calculation block centering on the target correction data, which is a unit for calculating the prediction value of the image of the first hierarchy. It has been made like that. That is, in FIG. 10, the pixel values of 9 pixels of 3 × 3 in the original image (here, the image of the first hierarchy) centered on the correction data X _ij (the portion indicated by a mark in the figure). From the left to right and from top to bottom, Y _ij (1), Y _ij
(2), Y _ij (3), Y _ij (4), Y _ij (5), Y
_{ij (6), Y ij (} 7), Y ij (8), when the expressed and Y _ij (9), for the calculation of the predicted value of the pixel Y _ij (1) through Y _ij (9), the predicted value The calculation blocking circuit 42 is, for example, a 5 × 5 25 pixel X centered on the pixel X _{ij.
(i-2) (j-2)} , X _{(i-2) ( j- 1)} , X _{(i-2) j} , X _{(i-2) (j + 1)} ,
X _{(i-2) (j + 2)} , X _{(i-1) (j-2)} , X _{(i-1) (j-1)} ,
X _{(i-1) j} , X _{(i-1) (j + 1)} , X _{(i-1) (j + 2)} , Xi _(j-2) , X
_{i (j-1)} , _Xij , Xi _{(j + 1)} , Xi _{(j + 2)} , X _{(i + 1) (j-2)} , X
_{(i + 1) (j-1)} , X _{(i + 1) j} , X _{(i + 1) (j + 1)} , X _{(i + 1) ( j + 2)} ,
X _{(i + 2) (j-2)} , X _{(i + 2) (j-1)} , X _{(i + 2) j} ,
A square-shaped prediction value calculation block composed of X _{(i + 2) (j + 1)} and X _{(i + 2) (j + 2) (} a prediction using pixel X _ij as a target pixel (target correction data)) (Value calculation block).

Specifically, for example, in order to calculate the predicted values of the nine pixels Y ₃₃ (1) to Y ₃₃ (9) in the image of the first layer, which is surrounded by a rectangle in FIG. _{X 11, X 12, X 13} , X 14, X 15, X 21, X 22,
_{X 23, X 24, X 25} , X 31, X 32, X 33, X 34, X 35, X
₄₁ , _X42 , _X43 , _X44 , _X45 , _X51 , _X52 , _X53 ,
X ₅₄ and X ₅₅ constitute a prediction value calculation block.

The prediction value calculation block obtained in the prediction value calculation block circuit 42 is supplied to the class classification adaptive processing circuit 43.

Note that the predicted value calculation block is also
As in the case of the class classification block, the number and shape of the pixels are not limited to those described above. However, the number of pixels constituting the predicted value calculation block is
It is desirable that the number of pixels is larger than the number of pixels constituting the class classification block.

In the case of performing the above-described blocking (the same applies to processing other than blocking), there is a case where there is no corresponding pixel near the image frame of the image. In this case, for example, The processing is performed assuming that the same pixel as the pixel forming the image frame exists outside the pixel.

The class classification adaptive processing circuit 43 uses the ADRC
(Adaptive Dynamic Range Coding) is configured by a processing circuit, a class classification circuit 45, and an adaptive processing circuit 46, and performs class classification adaptive processing.

The class classification adaptive process is a process of classifying an input signal into several classes based on the characteristics thereof and performing an appropriate adaptive process on the input signal of each class for the class. It is divided into processing and adaptive processing.

Here, the classification process and the adaptation process will be briefly described.

First, the class classification processing will be described.

Now, for example, as shown in FIG. 11A, a block of 2 × 2 pixels (class classification block) is formed by a certain pixel of interest and three adjacent pixels. A pixel is represented by one bit (takes any level of 0 or 1). In this case, as shown in FIG. 11B, a 2 × 2 block of four pixels is divided into 16 blocks according to the level distribution of each pixel.
(= (2 ¹ ) ⁴ ) patterns. That is, the pixel of interest constituting the block can be classified into patterns. Such pattern classification is a class classification process, and is performed in the class classification circuit 45.

The class classification process can be performed in consideration of the activity (complexity of the image) (the degree of change) of the image (the image in the block).

Here, usually, for example, about 8 bits are assigned to each pixel. Further, in the present embodiment, as described above, the class classification block is 3 × 3
And 9 pixels. Therefore, if the classification processing is performed on such a classification block,
(2 ⁸ ) It will be classified into a huge number of classes of ⁹ . Therefore, in the present embodiment, in the ADRC processing circuit 44, A
The DRC process is performed, whereby the number of classes is reduced by reducing the number of bits of pixels constituting the class classification block.

That is, for simplicity, for example, as shown in FIG. 12A, a block composed of four pixels arranged on a straight line is considered. Are detected as the maximum value MAX and the minimum value MIN. Then, DR = MAX−MIN is set as a local dynamic range of the block, and based on the dynamic range DR, the pixel values of the pixels forming the block are requantized to K bits.

[0099] That is, from each pixel value in the block, subtracts the minimum value MIN, dividing the subtracted value by DR / 2 ^K.
Then, the code (A) corresponding to the resulting division value
DRC code). Specifically, for example, K
In the case where = 2, as shown in FIG. 12 (B), it is determined whether the divided value belongs to any range obtained by equally dividing the dynamic range DR by 4 (= 2 ² ).
The range of the lowest level, the range of the second lowest level,
In the case of belonging to the range of the third level from the bottom or the range of the top level, for example, 00B, 0
It is encoded into two bits such as 1B, 10B, or 11B (B represents a binary number). Then, on the decoding side, the ADRC codes 00B, 01B, 10
B or 11B is the center value L ₀₀ of the range of the lowest level obtained by dividing the dynamic range DR into four, the center value L ₀₁ of the range of the second level from the bottom, and the center value L _{01 of} the third level from the bottom. The conversion is performed to the center value L ₁₀ of the range or the center value L ₁₁ of the range of the highest level, and the minimum value MIN is added to the value to perform decoding.

Here, such ADRC processing is called non-edge matching.

The details of the ADRC processing are disclosed in, for example, Japanese Patent Application Laid-Open No. 3-53778 filed earlier by the present applicant.

AD for performing requantization with a smaller number of bits than the number of bits allocated to the pixels constituting the block
As described above, the number of classes can be reduced by performing the RC processing.
The processing is performed in the DRC processing circuit 44.

In the present embodiment, in the class classification circuit 45, A is output from the ADRC processing circuit 44.
The class classification process is performed based on the DRC code. The class classification process includes, for example, DPCM (Predictive Coding), BTC (Block Truncation Coding), and VTC.
Q (vector quantization), DCT (discrete cosine transform),
It is also possible to perform processing on data that has been subjected to Hadamard transformation or the like.

Next, the adaptive processing will be described.

For example, now, the pixel value y of the image of the first layer
Of the predicted values E [y] of the pixels are corrected by the pixel values of some pixels around the pixels (here, correction data obtained by correcting the image data of the second layer) (hereinafter, appropriately referred to as learning data) x ₁ , x ₂ ,・ ・
.. And a linear first-order combination model defined by a linear combination of predetermined prediction coefficients w ₁ , w ₂ ,... In this case, the predicted value E [y] can be expressed by the following equation.

E [y] = w ₁ x ₁ + w ₂ x ₂ +... (1)

For generalization, a matrix W composed of a set of prediction coefficients w, a matrix X composed of a set of learning data, and a matrix Y ′ composed of a set of predicted values E [y]

(Equation 1) Defines the following observation equation.

XW = Y '(2)

Then, the least square method is applied to this observation equation, and a prediction value E close to the pixel value y of the image of the first hierarchy is obtained.
Consider finding [y]. In this case, a matrix Y composed of a set of pixel values y of the image of the first hierarchy (hereinafter, appropriately referred to as teacher data) and a residual e of the predicted value E [y] with respect to the pixel value y of the image of the first hierarchy A matrix E consisting of a set is

(Equation 2) From equation (2), the following residual equation is established.

XW = Y + E (3)

In this case, the first-layer prediction coefficient w for obtaining the predicted value E [y] close to the pixel value y of the first-layer image
_i is the squared error

(Equation 3) Can be obtained by minimizing.

Therefore, when the above-mentioned square error is differentiated by the prediction coefficient w _i , the result becomes 0, that is, the prediction coefficient w _i that satisfies the following equation is the prediction value E near the pixel value y of the image of the first layer
This is an optimum value for obtaining [y].

[0113]

(Equation 4) ... (4)

Then, first, the following equation is established by differentiating equation (3) with the prediction coefficient w _i of the first hierarchy.

[0115]

(Equation 5) ... (5)

From equations (4) and (5), equation (6) is obtained.

[0117]

(Equation 6) ... (6)

Further, the learning data x, the prediction coefficient w, the teacher data y, and the residual e in the residual equation of the equation (3) are obtained.
In consideration of the relationship, the following normal equation can be obtained from Expression (6).

[0119]

(Equation 7) ... (7)

The normal equation of the equation (7) can be set by the same number as the number of the prediction coefficients w to be obtained. Therefore, by solving the equation (7), the optimum prediction coefficient w can be obtained. . In solving equation (7), for example,
A sweeping method (Gauss-Jordan elimination method) or the like can be applied.

As described above, the optimal first-layer prediction coefficient w is obtained for each class, and further, using the prediction coefficient w, the equation (1) is used to approximate the pixel value y of the first-layer image. The adaptive processing is to obtain the predicted value E [y], and the adaptive processing is performed in the adaptive processing circuit 46.

In the adaptive processing, the components contained in the original image (here, the first layer image) which are not included in the thinned image (here, the second layer image) are reproduced. Is different from the interpolation processing. That is, in the adaptive processing,
As far as looking only at equation (1), the interpolation processing is the same as that using a so-called interpolation filter, but a prediction coefficient w corresponding to a tap coefficient of the interpolation filter is obtained by learning using teacher data y, so to speak. Since it is obtained, the components contained in the original image can be reproduced. From this, it can be said that the adaptive processing has a so-called image creation action.

Next, the processing of the local decoding unit 22 in FIG. 9 will be described with reference to the flowchart in FIG.

In the local decoding unit 22, first, in step S21, the correction data from the correction unit 21 is divided into blocks. That is, in the class classification blocking circuit 41, the correction data is divided into 3 × 3 pixel class classification blocks centered on the target correction data, supplied to the class classification adaptive processing circuit 43, and In the calculation blocking circuit 42, the correction data is 5 × 5 centered on the target correction data.
The block is divided into prediction value calculation blocks for pixels and supplied to the classification adaptive processing circuit 43.

As described above, the class classification adaptive processing circuit 43 is supplied with the first class image data in addition to the class classification block and the predicted value calculation block. Is supplied to the ADRC processing unit 44, and the prediction value calculation block and the image data of the first hierarchy are supplied to the adaptive processing circuit 46.

When the ADRC processing circuit 44 receives the class classification block, in step S22, the ADRC processing circuit 44 applies, for example, 1-bit AD to the class classification block.
An RC (ADRC for performing requantization with one bit) process is performed, whereby the correction data is converted (coded) to one bit and output to the class classification circuit 45. In step S23, the class classification circuit 45 performs a class classification process on the class classification block on which the ADRC process has been performed. That is, the state of the level distribution of each pixel constituting the classifying block is detected, and the class to which the classifying block belongs (the class of the target correction data constituting the classifying block) is determined. The result of this class determination is supplied to the adaptive processing circuit 46 as class information.

In the present embodiment, the class classification processing is performed on the class classification block composed of 3 × 3 9 pixels that have been subjected to the 1-bit ADRC processing. The classification block is 512 (= (2
¹ ) It will be classified into one of the classes of ⁹ ).

Then, the process proceeds to a step S 24, wherein the adaptive processing circuit 46 performs an adaptive process for each class based on the class information from the class classification circuit 45, thereby obtaining a prediction coefficient for each class of the first hierarchy. And a predicted value for one frame is calculated.

That is, in the present embodiment, 25 × 9 prediction coefficients for each class are calculated from the correction data of one frame and the image data of the first hierarchy. Then, for example, when attention is paid to one piece of correction data, a total of nine pixels of the pixel of the first layer image corresponding to the attention correction data and eight pixels adjacent around the pixel are provided. Is 5 ×, centered on the correction data of interest.
It is calculated by performing adaptive processing using a prediction value calculation block composed of five pixels.

More specifically, for example, 3 × 3 correction data X ₂₂ , X ₂₃ , X ₂₄ , X ₃₂ , X ₃₃ , X ₃₄ , and X ₃₄ centered on the correction data X ₃₃ shown in FIG. ₄₂ , X ₄₃ , X
Class information C on the class classification block ₄₄ (a class classification block in which the correction data X ₃₃ is configured as the target correction data) is output from the class classification circuit 45, and corresponds to the class classification block. As prediction value calculation blocks, correction data X ₁₁ , X ₁₂ , X ₁₃ , X ₁₄ , and 5 × 5 pixels centered on the correction data X ₃₃ .
_{X 15, X 21, X 22} , X 23, X 24, X 25, X 31, X 32, X
₃₃ , _X34 , _X35 , _X41 , _X42 , _X43 , _X44 , _X45 ,
A prediction value calculation block consisting of X ₅₁ , X ₅₂ , X ₅₃ , X ₅₄ , and X ₅₅ (a prediction value calculation block using the correction data X ₃₃ as target correction data) is output from the prediction value calculation blocking circuit 42. When those, first, the correction data for constituting the predicted values calculating block, with the learning data, in the image of the first hierarchy, the 3 × 3 pixels (Fig. 10 around the correction data X ₃₃ Using the pixel values Y ₃₃ (1) to Y ₃₃ (9) of the rectangle (portion surrounded by a rectangle) as teacher data, a normal equation shown in Expression (7) is established.

Further, for example, a normal equation is similarly set for another predicted value calculation block corresponding to a class classification block classified into the same class information C in one frame. , Pixel value Y ₃₃ (k)
(Here, k = 1, 2,..., 9) predicted value E [Y
₃₃ (k)] to obtain prediction coefficients w ₁ (k) to w ₂₅
(K) (In the present embodiment, 25 pieces of learning data are used to obtain one prediction value.
When the number of normal equations sufficient to calculate 25 prediction coefficients w is obtained (therefore, until such a number of normal equations are obtained, step S24 is performed).
Then, the process of establishing a normal equation is performed.) By solving the normal equation, the class information C is most suitable for obtaining the predicted value E [Y ₃₃ (k)] of the pixel value Y ₃₃ (k). Various prediction coefficients w ₁ (k) to w ₂₅ (k) are calculated.

This process is performed for each class, whereby 25 × 9 prediction coefficients are calculated for each class. Then, using the prediction coefficient corresponding to the class information C and the predicted terrain calculation block, a predicted value E [Y ₃₃ (k)] is obtained according to the following equation corresponding to equation (1).

E [Y ₃₃ (k)] = w ₁ (k) X ₁₁ + w ₂ (k) X ₁₂ + w ₃ (k) X ₁₃ + w ₄ (k) X ₁₄ + w ₅ (k) X ₁₅ + w ₆ ( _{k) X 21 + w 7 (} k) X 22 + w 8 (k) X 23 + w 9 (k) X 24 + w 10 (k) X 25 + w 11 (k) X 31 + w 12 (k) X 32 + w 13 (k ) X ₃₃ + w ₁₄ (k) X ₃₄ + w ₁₅ (k) X ₃₅ + w ₁₆ (k) X ₄₁ + w ₁₇ (k) X ₄₂ + w ₁₈ (k) X ₄₃ + w ₁₉ (k) X ₄₄ + w ₂₀ (k) X ₄₅ + w ₂₁ (k) X ₅₁ + w ₂₂ (k) X ₅₂ + w ₂₃ (k) X ₅₃ + w ₂₄ (k) X ₅₄ + w ₂₅ (k) X ₅₅ (8)

In step S24, as described above,
A 25 × 9 prediction coefficient is obtained for each class, and a prediction value of a 3 × 3 original image pixel centering on the target correction data is obtained using the prediction coefficient for each class.

Thereafter, the flow advances to step S25, where the 25 × 9 prediction coefficients for each class are supplied to the judgment section 24,
The prediction value for one frame obtained in units of three pixels is supplied to the error calculation unit 23. Then, the process returns to step S21, and the same process is repeated for each frame as described above, for example.

Next, FIG. 14 shows an example of the configuration of the error calculator 23 in FIG.

The original image data, that is, the image data of the first hierarchy here, is supplied to the blocking circuit 51.
Divides the image data into blocks each of which corresponds to a first-layer prediction value output from the local decoding unit 22 and outputs the resulting 3 × 3 pixel block to the square error calculation circuit 52. It has been made to be. As described above, the square error calculating unit 52 is supplied with the blocks from the blocking circuit 51, and receives the predicted values of the first hierarchy from the local decoding unit 22 in units of nine (3 × 3 pixel block units). The square error calculation circuit 52 calculates a square error as a prediction error of a prediction value of the image of the first hierarchy, and calculates the square error.
5.

That is, the square error calculation circuit 52 is composed of arithmetic units 53 and 54. The computing unit 53 subtracts a corresponding predicted value from each of the blocked image data from the blocking circuit 51 and supplies the subtracted value to the computing unit 54. Arithmetic unit 54
Is configured to square the output of the arithmetic unit 53 (the difference between the image data of the first hierarchy and its predicted value) and supply it to the integrating unit 55.

When receiving the squared error from the squared error calculating circuit 52, the integrating section 55 reads the stored value of the memory 56, adds the stored value and the squared error, and supplies the stored value to the memory 56 again for storage. By repeating this, the integrated value (error variance) of the square error is obtained. Further, when the integration of the square error for a predetermined amount (for example, for one frame) is completed, the integrating unit 55 reads out the integrated value from the memory 56 and supplies the integrated value to the determining unit 24 as error information. ing. Each time the processing for one frame is completed, the memory 56
The output value of the integrating unit 55 is stored while clearing the stored value.

Next, the operation will be described with reference to the flowchart of FIG. In the error calculator 23, first, in step S31, the stored value of the memory 56 is cleared to, for example, 0, and the process proceeds to step S32.
In the blocking circuit 51, the image data of the first hierarchy is blocked as described above, and the resulting block is supplied to the square error calculation circuit 52. In the square error calculation circuit 52, in step S33, the image data of the image of the first hierarchy, which constitutes the block supplied from the blocking circuit 51, and the local decoding unit 22
Is calculated with respect to the first-layer prediction value supplied from.

That is, in step S 33, the corresponding predicted value is subtracted from each of the blocked first-layer image data supplied from the blocking circuit 51 in the computing unit 53 and supplied to the computing unit 54. . Further, in step S33, the output of the arithmetic unit 53 is squared in the arithmetic unit 54 and supplied to the integrating unit 55.

Upon receiving the squared error from the squared error calculation circuit 52, the integrating unit 55 reads the stored value of the memory 56 in step S34 and adds the stored value and the squared error to obtain the integrated value of the squared error. Ask for. Accumulator 5
5 is stored in the memory 56
And is stored by overwriting the previous stored value.

Then, in the integrating section 55, step S35
In, it is determined whether or not the integration of the square error for a predetermined amount, for example, for one frame is completed. If it is determined in step S35 that the integration of the square error for one frame has not been completed, the process returns to step S32, and the processing from step S32 is repeated again. If it is determined in step S35 that the integration of the square error for one frame has been completed, the process proceeds to step S36, where the integration unit 55
Then, the integrated value of the squared error for one frame stored in 6 is read out and output to the determination unit 24 as error information. Then, the process returns to step S31, and again returns to step S31.
The processing from 31 is repeated.

Accordingly, the error calculating section 23 sets the image data of the first hierarchy to Y _ij (k), and sets the predicted value of the first hierarchy generated from the correction data in the local decoding section 22 to E [Y _ij (k). k)], the error information Q is calculated by performing the calculation according to the following equation.

Q = {(Y _ij (k) −E [Y _ij (k)]) ² Here, Σ means summation for one frame.

Next, FIG. 16 shows an example of the configuration of the determination section 24 of FIG.

The prediction coefficient memory 61 stores prediction coefficients supplied from the local decoding unit 22. The correction data memory 62 stores the correction data supplied from the correction unit 21.

It should be noted that the correction data memory 62 stores the correction unit 2
In 1, the image data of the second layer is newly corrected,
As a result, when new correction data is supplied,
New correction data is stored in place of the correction data already stored (previous correction data). After the correction data is updated to a new one, the local decoding unit 22 outputs a new set of prediction coefficients for each class corresponding to the new correction data. When a new prediction coefficient for each class is thus supplied to the coefficient memory 61 as well, the new prediction coefficient for each class (the previous prediction coefficient for each class) is replaced with the new prediction coefficient. The prediction coefficients for each class are stored.

The error information memory 63 stores the error information supplied from the error calculator 23.
The error information memory 63 stores the error information supplied last time in addition to the error information supplied this time from the error calculation unit 23 (even if new error information is supplied, Until new error information is provided,
The error information already stored is retained). Further, the error information memory 63 is cleared each time a process for a new frame is started.

The comparison circuit 64 compares the current error information stored in the error information memory 63 with a predetermined threshold value ε.
Further, if necessary, the current error information is compared with the previous error information. The comparison result in the comparison circuit 64 is supplied to the control circuit 65.

The control circuit 65 determines, based on the comparison result of the comparison circuit 64, whether the correction data stored in the correction data memory 62 is appropriate (optimal) as the image encoding result, and is not optimal. When it is recognized (determined), a control signal requesting the output of new correction data is supplied to the correction unit 21 (FIG. 5). When the control circuit 65 recognizes that it is optimal to use the correction data stored in the correction data memory 62 as the image encoding result, the control circuit 65 stores the correction data for each class stored in the prediction coefficient memory 61. The prediction coefficient is read out, and the multiplexing unit 2
5, the correction data stored in the correction data memory 62 is read out, and is also supplied to the multiplexing unit 25 as optimal correction data. Further, in this case, the control circuit 65 outputs a control signal indicating that encoding of the image of one frame has been completed,
This is output to the correction unit 21, thereby causing the correction unit 21 to start processing for the next frame, as described above.

Next, the operation of the determination section 24 will be described with reference to FIG. In the determination unit 24, first, in step S41, the comparison circuit 64 determines whether or not error information has been received from the error calculation unit 23.
If it is determined that the error information has not been received, the process returns to step S41. If it is determined in step S41 that the error information has been received, that is, if the error information has been stored in the error information memory 63, the process proceeds to step S42, where the comparison circuit 64 stores the error information in the error information memory 63. The obtained error information (current error information) is compared with a predetermined threshold value ε to determine which is larger.

If it is determined in step S42 that the current error information is equal to or larger than the predetermined threshold ε, the comparison circuit 64 reads the previous error information stored in the error information memory 63. Then, the comparison circuit 64
In step S43, the previous error information is compared with the current error information to determine which is larger.

Here, when the processing for one frame is started and error information is first supplied, the error information memory 63 does not store the previous error information. In step 24, step S
The processing after 43 is not performed, and the control circuit 65 outputs a control signal for controlling the correction circuit 32 (FIG. 7) so as to output a predetermined initial address.

If it is determined in step S43 that the current error information is equal to or less than the previous error information, that is, if the error information is reduced by correcting the image data of the second hierarchy, the process proceeds to step S44. Then, the control circuit 65 outputs to the correction circuit 32 a control signal for instructing the correction value △ to be changed in the same manner as the previous time, and
Return to 41. If it is determined in step S43 that the current error information is larger than the previous error information, that is, if the error information is increased by correcting the image data of the second hierarchy, the process proceeds to step S45. The control circuit 65 outputs to the correction circuit 32 a control signal for instructing the correction value △ to be changed in the reverse of the previous time, and returns to step S41.

When the error information, which has been decreasing, starts to rise at a certain timing, the control circuit 6
Reference numeral 5 denotes a control signal for instructing the correction value で to be changed, for example, by half the size of the previous case, in the opposite manner to the previous time.

By repeating the processing of steps S41 to S45, the error information decreases. As a result, if it is determined in step S42 that the current error information is smaller than the predetermined threshold value ε, the process proceeds to step S46.
The control circuit 65 reads the prediction coefficient for each class stored in the prediction coefficient memory 61, reads the correction data for one frame stored in the correction data memory 62 as optimal correction data, and multiplexes the data. And the process is terminated.

After that, after the error information for the next frame is supplied, the processing according to the flowchart shown in FIG. 17 is repeated.

It is to be noted that the correction circuit 32 can correct the image data of the second layer for all data of one frame or only for a part of the data. Can also. In a case where correction is performed only on a part of the data, the control circuit 65 may be made to detect, for example, a pixel having a strong influence on the error information, and only the data on such a pixel may be corrected. Pixels having a strong influence on error information can be detected, for example, as follows. That is, first, the error information is obtained by performing processing using the image data of the second hierarchy as it is.
Then, a control signal for causing the image data of the second hierarchy to be corrected by the same correction value △ pixel by pixel is output from the control circuit 65 to the correction circuit 32, and the resulting error information is output. It is sufficient to compare the error information obtained when the image data of the second hierarchy is used as it is, and detect a pixel whose difference is equal to or more than a predetermined value as a pixel having a strong influence on the error information.

As described above, the correction of the image data of the second hierarchy is repeated until the error information is made smaller than the predetermined threshold ε (below), and when the error information becomes smaller than the predetermined threshold ε. Since the correction data is output as a result of encoding the image, the receiving device 4 (FIG. 1) sets the pixel values of the pixels constituting the decimated image to the most appropriate values for restoring the original image. From the corrected data (optimized correction data), it is possible to obtain the same (almost the same) decoded image as the original image.

Further, since the image is compressed not only by the thinning process but also by the ADRC process, the class classification adaptive process, etc., it is possible to obtain encoded data with a very high compression ratio. The above-described encoding processing in the transmission device 1 realizes high-efficiency compression by using the compression processing by thinning and the classification adaptive processing in an organically integrated manner. From this, it can be said that it is an integrated coding process.

Next, FIG. 18 shows a configuration example of the receiving apparatus 4 of FIG.

The encoded data recorded on the recording medium 2 is reproduced, or the encoded data transmitted via the transmission path 3 is received and supplied to the signal processing circuit 71. The signal processing circuit 71 receives the encoded data supplied thereto, separates the encoded data into the first to third layers of encoded data, and performs error correction processing and the like.
Further, the signal processing unit 71 copies the prediction coefficient of the first layer from the coded data of the third layer, and includes it in the coded data of the second layer. Is output.

The coded data of the first hierarchy is directly stored in the first layer.
It is output as a hierarchically decoded image. The encoded data of the second or third layer is output to the prediction unit 72 or 73, respectively.

In the prediction section 72, a predicted value of the image of the first layer is obtained based on the encoded data of the second layer.
It is output as a hierarchically decoded image. In addition, the prediction unit 72 also calculates the first layer based on the encoded data of the third layer.
The predicted value of the hierarchy is obtained and output.

Next, FIG. 19 shows an example of the configuration of the prediction section 72.

The coded data of the second hierarchy from the signal processing circuit 71 (FIG. 18) is supplied to a separating section 81, where
The optimal correction data of the second hierarchy and the prediction coefficients for each class of the first hierarchy are separated and supplied to the decoding unit 80.

The decoding unit 80 includes a class classification blocking circuit 82, a predicted value calculation blocking circuit 83, an ADR
It comprises a C processing circuit 84, a class classification circuit 85, and a prediction circuit 86. The optimum correction data of the second hierarchy from the separation unit 81 is divided into a class classification block circuit 82 and a predicted value calculation block circuit 83. Is entered into
The prediction coefficient for each class of the first hierarchy is input to the prediction circuit 86. Then, in the prediction circuit 86, the prediction coefficient for each class of the first hierarchy is stored in the built-in memory 86A.

The classifying blocking circuit 82, the predicted value calculating blocking circuit 83, the ADRC processing circuit 84, or the class classifying circuit 85 are the class classifying blocking circuit 41, the predicted value calculating blocking circuit in FIG. 4
2. ADRC processing circuit 44 or class classification circuit 45
Therefore, in these blocks, the same processing as in the case of FIG. 9 is performed, whereby the predicted value calculation blocking circuit 83 outputs the optimum correction data of the second hierarchy. The constructed prediction value calculation block is output, and the class classification circuit 85 outputs class information. These prediction value calculation block and class information are supplied to the prediction circuit 86.

In the prediction circuit 86, a 25 × 9 prediction coefficient corresponding to the class information supplied from the class classification circuit 85 is read from the memory 86A, and the 25 × 9 prediction coefficient and the prediction value calculation block are read out. Using the optimal correction data constituting the prediction value calculation block of 5 × 5 pixels supplied from the conversion circuit 83, the prediction value of the first hierarchy is calculated in 3 × 3 pixel units according to the equation (1). To go. And
When a prediction value for one frame is obtained, this is output as a first-layer decoded image.

The optimum correction data of the second hierarchy is such that the error between the predicted value of the first hierarchy obtained by performing the prediction and the image of the first hierarchy is equal to or smaller than the threshold ε as described above. Therefore, according to the optimum correction data of the second layer, it is possible to obtain a decoded image of the first layer having a high resolution.

Next, FIG. 20 shows an example of the configuration of the prediction unit 73 in FIG.

The coded data of the third hierarchy from the signal processing circuit 71 (FIG. 18) is supplied to a separating section 81, where
The data is separated into the optimum correction data of the third hierarchy, the prediction coefficient of each class of the second hierarchy, and the prediction coefficient of each class of the first hierarchy. The optimal correction data of the third hierarchy and the prediction coefficients of each class of the second hierarchy are supplied to the decoding unit 222,
The prediction coefficient for each class in the hierarchy is supplied to the decoding unit 223.

The decoding section 222 has the same configuration as that of the decoding section 80 in FIG. 19. Therefore, the decoding section 222 determines the optimum correction data of the third hierarchy and the prediction coefficient for each class of the second hierarchy. In the same manner as described above, the predicted value of the image of the second layer is calculated, and the decoded image (predicted value) for one frame of the second layer composed of such predicted values is sent to the decoding unit 223. Supplied. The decoding unit 223 is also configured in the same manner as the decoding unit 80. Therefore, the prediction value of the first layer is calculated from the prediction value of the second layer and the prediction coefficient of each class of the first layer,
One frame of image composed of such predicted values is
It is output as a decoded image (prediction value) of the first layer.

The prediction coefficient for each class in the second hierarchy is shown in FIG.
The optimum correction data calculation circuit 14 calculates the optimum correction data as the second-layer optimum correction data, and according to the second-layer optimum correction data, as described above, Therefore, it is possible to obtain a first-layer decoded image having a high resolution even with the third-layer encoded data.

As described above, even if some or all of the encoded data of the first and second layers are lost for some reason, the encoded data of the third layer, which is the lowest layer, ie, the third layer , And a prediction coefficient for each class of the first layer and the second layer, it is possible to obtain a high-resolution decoded image.

The receiving side can receive only the coded data of the second and third layers (the optimal correction data of the second and third layers and the prediction coefficients for each class of the first and second layers). However, a high-resolution decoded image (first-layer image) can be obtained. In addition, the third
Even if the receiving apparatus can only receive the encoded data of the hierarchy, it is still possible to obtain a high-resolution decoded image.

In the above-described case, the second-layer encoded data includes the second-layer optimal correction data. However, the second-layer encoded data includes the second-layer optimal data. It is possible to include the image data itself of the second hierarchy instead of the optimum correction data of the hierarchy.

Further, the encoded data output from the signal processing circuit 15 includes the encoded data of the first to third layers. However, as described above, the encoded data of the third layer ( Since a high-quality decoded image can be obtained only from the third-layer optimal correction data and the prediction coefficients for each of the first and second classes, the encoded data output from the signal processing circuit 15 , Only the encoded data of the third layer can be included.

Further, in FIG. 2, the optimum correction data calculating circuit 14 is supplied with the optimum correction data of the second hierarchy output from the optimum correction data calculating circuit 13 to obtain the optimum correction data of the second hierarchy. Although the optimal prediction coefficient for each class is determined, the optimal correction data calculation circuit 14
It is also possible to supply the second-layer image data itself instead of the second-layer optimal correction data, and to obtain the prediction coefficient for each class that is optimal for obtaining the second-layer image data. It is.

In the above case, the local decoding unit 22 shown in FIG. 9 uses the prediction coefficients for each class of the first hierarchy (the local decoding unit 22 constituting the optimum correction data calculation circuit 14 uses the prediction coefficients for each class of the second hierarchy). Was calculated, and the prediction value of the first layer was calculated using the prediction coefficient. However, the prediction value was calculated without calculating the prediction coefficient for each class (the prediction coefficient obtained in advance by learning). (Using).

That is, FIG. 21 shows a second configuration example of the optimum correction data calculation circuit 13 (14) in FIG. In the figure, the same reference numerals are given to portions corresponding to the case in FIG. That is, the optimum correction data calculation circuit 13 replaces the local decode unit 22 with
The configuration is the same as that in FIG. 5 except that a local decoding unit 1022 is provided.

In FIG. 5, however, the original image data (the first hierarchical image in the optimal correction data calculation circuit 13 and the optimal correction data
In FIG. 21, the optimal correction data of the second hierarchy is supplied, but in FIG. 21, the original image data is not supplied to the local decoding unit 1022.

FIG. 22 is a block diagram of the local decoding unit 1 shown in FIG.
022 shows a configuration example. Note that, in the figure, portions corresponding to those in FIG. 9 are denoted by the same reference numerals. That is, the local decoding unit 1022 is configured similarly to the local decoding unit 22 in FIG. 9 except that a prediction coefficient ROM 88 and a prediction circuit 89 are provided instead of the adaptive processing circuit 46.

The prediction coefficient ROM 88 stores 25 × 9 prediction coefficients for each class obtained by performing learning (described later) in advance, receives the class information output from the class classification circuit 45, The prediction coefficient stored at the address corresponding to the class information is read and supplied to the prediction circuit 89.

The prediction circuit 89 uses the 5 × 5 pixel prediction value calculation block from the prediction value calculation blocking circuit 42 and the 25 × 9 prediction coefficient from the prediction coefficient ROM 88 to obtain the equation (1). (Specifically, for example, the linear linear expression shown in Expression (8) is calculated, and thereby, the predicted value of the original image is calculated in 3 × 3 pixel units.

Therefore, according to the classification adaptive processing circuit 43 of FIG. 22, the predicted value is calculated without using the original image (original image). For this reason, as described above, the original image (the image of the first hierarchy or the optimal correction data of the second hierarchy) is not supplied to the local decoding unit 1022 (there is no need to supply the original image, so the local decoding unit 1022 is not supplied) ).

Next, the processing of the local decoding unit 1022 in FIG. 22 will be further described with reference to the flowchart in FIG.

In the local decoding section 1022,
First, in steps S1021 to S1023, the same processes as those in steps S21 to S23 of FIG. 13 are performed, whereby the class information is output from the class classification circuit 45, and the prediction value calculation block is formed. The circuit 42 outputs a prediction value calculation block. The class information or the predicted value calculation block is supplied to the prediction coefficient ROM 88 or the prediction circuit 89, respectively.

Upon receiving the class information, the prediction coefficient ROM 88 reads out a 25 × 9 prediction coefficient corresponding to the class information from the stored prediction coefficients for each class and sends it to the prediction circuit 89 in step S1024. Supply.

The prediction circuit 89 is supplied with 25 × 9 prediction coefficients from the prediction coefficient ROM 88 and, as described above, also receives a prediction value calculation block of 5 × 5 pixels from the prediction value calculation blocking circuit. Supplied. Then, in the prediction circuit 89, in step S1025, the prediction coefficient RO
The adaptive processing is performed using the 25 × 9 prediction coefficient from M88 and the 5 × 5 pixel prediction value calculation block from the prediction value calculation blocking circuit 42, that is, specifically, By performing the calculation according to Expression (1) (or Expression (8)), the correction data of interest (here,
The predicted value of the pixel of the 3 × 3 original image centered on the pixel at the center of the predicted value calculation block) is obtained.

Thereafter, for example, when a predicted value for one frame is obtained, the process proceeds to step S1026, where the prediction coefficient R
The 25 × 9 prediction coefficient for each class stored in the OM 88 is read and supplied to the determination unit 24, and the prediction value obtained in step S1025 is supplied to the error calculation unit 23. Then, the process returns to step S1021, and thereafter, the same processing is repeated, for example, in units of one frame.

In this embodiment, as the prediction coefficient for each class, the one stored in the prediction coefficient ROM 88 is used, and the value does not change.
In 1025, the prediction coefficient for each class is determined by the determination unit 2
After once being supplied to 4, there is basically no need to supply again.

Also, even when the optimum correction data calculating circuits 13 and 14 are configured as shown in FIG. 21, the receiving device 4 is configured as shown in FIG. The same decoded image can be obtained.

Next, FIG. 24 shows the prediction coefficient ROM of FIG.
9 shows a configuration example of an image processing apparatus that performs learning to obtain a prediction coefficient for each class stored in 88.

The learning blocking circuit 91 and the teacher blocking circuit 92 are supplied with learning image data (learning images) for obtaining prediction coefficients applicable to any image. . When a prediction coefficient for each class of the first hierarchy is obtained, as a learning image,
When the image data of the first hierarchy obtains the prediction coefficient for each class of the second hierarchy, the image data of the second hierarchy is used as the learning image by the learning blocking circuit 91 and the teacher blocking circuit. 92.

The learning blocking circuit 91 extracts from the input image data, for example, 25 pixels (5 × 5 pixels) centered on the pixel of interest having the positional relationship indicated by a circle in FIG. The block composed of 25 pixels is supplied to the ADRC processing 93 and the learning data memory 96 as a learning block.

In the teacher blocking circuit 92, for example, a block composed of 3 × 3 9 pixels centered on the pixel of interest is generated from the input image data, and the block is composed of the 9 pixels. The blocks are supplied to the teacher data memory 98 as teacher blocks.

When the learning block forming circuit 91 generates, for example, a learning block composed of 25 pixels having a positional relationship indicated by a circle in FIG. 3 shown in the figure with a square
A teacher block of × 3 pixels is generated. In this case, the pixel of interest is the pixel X _33.

The ADRC processing circuit 93 extracts the central nine pixels (3 × 3 pixels) from the 25 pixels constituting the learning block, and applies the ADRC processing circuit shown in FIG. As in 44, 1
A bit ADRC process is performed. The block of 3 × 3 pixels on which the ADRC process has been performed is supplied to the class classification circuit 94. In the class classification circuit 94, as in the case of the class classification circuit 45 of FIG.
3 is subjected to the class classification processing, and the obtained class information is obtained through the terminal a of the switch 95.
The learning data memory 96 and the teacher data memory 98 are supplied.

In the learning data memory 96 or the teacher data memory 98, the learning block from the learning blocking circuit 91 or the teacher block from the teacher blocking circuit 92 is added to the address corresponding to the class information supplied thereto. Are respectively stored.

Therefore, in the learning data memory 96,
For example, if a block composed of 5 × 5 pixels indicated by a mark in FIG. 10 is stored as a learning block at a certain address, in the teacher data memory 98, the same address as that address is used. A block of 3 × 3 pixels surrounded by a rectangle is stored as a teacher block.

Thereafter, the same processing is repeated for all learning images prepared in advance, whereby the learning block and the local decoding unit 1 shown in FIG.
022, a teacher block composed of 9 pixels for which a predicted value is obtained using a predicted value calculation block composed of 25 correction data having the same positional relationship as the 25 pixels constituting the learning block; Are stored at the same address in the learning data memory 96 and the teacher data memory 98.

In the learning data memory 96 and the teacher data memory 98, a plurality of pieces of information can be stored at the same address, whereby a plurality of learning blocks can be stored at the same address. And a teacher block can be stored.

When the learning block and the teacher block for all the learning images are stored in the learning data memory 96 and the teacher data memory 98, the switch 95 which has selected the terminal a is switched to the terminal b. This allows
The output of the counter 97 is supplied to the learning data memory 96 and the teacher data memory 98 as an address. The counter 97 counts a predetermined clock and outputs the count value. In the learning data memory 96 or the teacher data memory 98, a learning block or a teacher block stored at an address corresponding to the count value is stored. The data is read out and supplied to the arithmetic circuit 99.

Therefore, the arithmetic circuit 99 includes the counter 97
, A set of learning blocks of a class corresponding to the count value and a set of teacher blocks are supplied.

Upon receiving the set of learning blocks and the set of teacher blocks for a certain class, the arithmetic circuit 99 calculates a prediction coefficient that minimizes an error by using the least squares method.

That is, for example, assume that the pixel values of the pixels constituting the learning block are x ₁ , x ₂ , x ₃ ,.
When the prediction coefficients to be obtained are w ₁ , w ₂ , w ₃ ,..., A linear primary combination of these is used to obtain the pixel value y of a certain pixel forming the teacher block.
_{1, w 2, w 3,} ··· , it is necessary to satisfy the following equation.

Y = w ₁ x ₁ + w ₂ x ₂ + w ₃ x ₃ +...

Therefore, in the arithmetic circuit 99, the prediction value w ₁ x ₁ + w ₂ x ₂ + w ₃ x ₃ +... With respect to the true value y is obtained from the learning block of the same class and the corresponding teacher block.
Prediction coefficients w ₁ , w ₂ , w ₃ , which minimize the square error of
Is obtained by setting up and solving the normal equation shown in the above equation (7). The above processing is performed for each class, so that 25 × 9 prediction coefficients are calculated for each class.

The prediction coefficients for each class obtained in the arithmetic circuit 99 are supplied to the memory 100. The memory 100 is supplied with the count value from the counter 97 in addition to the prediction coefficient from the arithmetic circuit 99,
In the memory 100, the prediction coefficient from the arithmetic circuit 99 is stored at an address corresponding to the count value from the counter 97.

As described above, the memory 100 stores, at the address corresponding to each class, 25 × 9 prediction coefficients optimal for predicting 3 × 3 pixels of the block of the class.

The prediction coefficient ROM 88 of FIG. 22 stores the prediction coefficients for each class stored in the memory 100 as described above.

FIG. 25 shows a third configuration example of the optimum correction data calculation circuit 13 (14) of FIG. In the figure, the same reference numerals are given to portions corresponding to the case in FIG. That is, the transmission device 1
Is replaced with the local decoding unit 2022 or the determining unit 2024 instead of the local decoding unit 22 or the determining unit 24.
Are provided, and the configuration is the same as that in FIG. 5 except that the multiplexing unit 25 is not provided.

However, in this case, as in the case of FIG. 22, original image data (image data of the first hierarchy or optimal correction data of the second hierarchy) is not supplied to the local decoding unit 2022. I have. Further, FIG.
In the fifth embodiment, the local decoding unit 2022
The prediction coefficient is not output to the determination unit 2024.

Next, the operation will be described with reference to the flowchart of FIG.

When the image data (the image data of the second hierarchy or the image data of the third hierarchy) is supplied to the compression unit 21, the compression unit 21 proceeds to step S1001.
The image data is compressed by thinning it out, and is initially output to the local decoding unit 2022 and the determination unit 2024 without correction. In step S1002, the local decoding unit 2022 locally decodes the correction data from the compression unit 21 (initially, as described above, the compressed data itself obtained by simply thinning out the image data).

That is, in step S1002, the compression unit 2
The correction data from 1 is locally decoded, for example, in the same manner as in the case of the local decoding unit 1022 shown in FIG. 22, whereby the predicted value of the original image is calculated. This predicted value is supplied to the error calculator 23.

[0219] In the error calculating section 23, step S1003 is executed.
, A prediction error is calculated in the same manner as in step S3 of FIG.
24. The determination unit 2024 determines whether the error
In step S1004, it is determined whether or not the error information from the error calculator 23 is equal to or less than a predetermined threshold ε, as in step S4 of FIG. If it is determined in step S1004 that the error information is not smaller than or equal to the predetermined threshold ε, the process proceeds to step S1005, and the determination unit 2024 controls the compression unit 21 to thereby correct the image data of the second hierarchy. Then, the compression unit 21 corrects the image data of the second hierarchy and waits for new correction data to be output, and then returns to step S1002.

On the other hand, if it is determined in step S1004 that the error information is equal to or smaller than the predetermined threshold value ε, the determination unit 2024 determines that the correction data obtained when the error information is equal to or smaller than the predetermined threshold value ε The data is output to the signal processing circuit 15 (FIG. 2), and the process ends.

Therefore, here, the prediction coefficient for each class is not output.

Next, FIG. 27 shows a configuration example of the local decoding unit 2022 of FIG. Note that in FIG.
The same reference numerals are given to the portions corresponding to the case in FIG. That is, the local decoding unit 2022
The configuration is basically the same as that of the local decoding unit 1022 in FIG.

However, in FIG. 22, the prediction coefficient ROM
88 represents the prediction coefficient for each class stored therein,
27, the prediction coefficient ROM 88 stores the prediction coefficient for each class in FIG.
It is not supplied to the determination unit 2024.

Next, the operation will be described with reference to the flowchart in FIG.

In the local decoding unit 2022, in steps S2021 to S2025, the same processing as in steps S1021 to 1025 in FIG. 23 is performed. Then, in step S2026, only the predicted value obtained in step S2025 is supplied to the error calculation unit 23, and the process returns to step S2021.
Hereinafter, the same processing is repeated for each frame, for example. That is, as described above, the prediction coefficient for each class is not output here.

Next, FIG. 29 is a block diagram showing the determination unit 2024 of FIG.
Is shown. Note that, in the figure, parts corresponding to those in FIG. 16 are denoted by the same reference numerals. That is, as described above, the local decoding unit 202
Since no prediction coefficient is supplied from No. 2, the judgment unit 2024
Has a prediction coefficient memory 6 for storing the prediction coefficient.
1 is not provided, except for that, basically,
The configuration is the same as that of the determination unit 24 in FIG.

Next, the operation will be described with reference to the flowchart of FIG.

In the judgment section 2024, step S2041 is executed.
In steps S41 to S45 in FIG.
Processing similar to that in the case of 45 is performed, whereby the compression unit 21 is controlled so that error information decreases.

Then, steps S2041 to S204
By repeating the process of step 5, the error information decreases. If it is determined in step S2042 that the current error information is smaller than the predetermined threshold ε, the process proceeds to step S2046, where the control circuit 65 The correction data of one frame stored in the data memory 62 is
The data is read out as the optimum correction data, supplied to the signal processing circuit 15, and the processing ends. That is, the local decoding unit 2
022, no prediction coefficient is supplied, so that step S
At 2046, only the correction data is output.

[0230] Thereafter, after the error information for the next frame is supplied, the same processing is repeated again.

Next, FIG. 31 shows an example of the configuration of the prediction section 72 in the receiving apparatus 4 of FIG. 18 when the optimum correction data calculation circuits 13 and 14 are configured as shown in FIG. In the figure, parts corresponding to those in FIG. 19 are denoted by the same reference numerals. That is, FIG.
1 prediction unit 72 includes a class classification circuit 85 and a prediction circuit 86
, A prediction coefficient ROM 87 is newly provided, the prediction circuit 86 does not include the memory 86A, and the separation unit 8
The configuration is basically the same as that in FIG. 19 except that 1 is not provided.

When the optimum correction data calculation circuit 13 (14) is configured as shown in FIG. 25, as described above, the prediction coefficient for each class of the second hierarchy is included in the encoded data of the second hierarchy. Not included, the coded data of the second layer is basically composed of only the optimum correction data of the second layer. Therefore, the second-layer optimal correction data as the second-layer encoded data from the signal processing circuit 71 (FIG. 18) is directly sent to the class-classifying blocking circuit 82 and the predicted value calculating blocking circuit 83. (Supplied without passing through the separation unit 81 (FIG. 19)).

The classifying blocking circuit 82, the predicted value calculating blocking circuit 83, the ADRC processing circuit 84, or the class classifying circuit 85 includes the class classifying blocking circuit 41 and the predicted value calculating block circuit shown in FIG. 4
2. ADRC processing circuit 44 or class classification circuit 45
The same processing as that described above is performed, whereby the prediction value calculation block of 5 × 5 pixels is output from the prediction value calculation blocking circuit 83, and the class information is output from the class classification circuit 85. . The prediction value calculation block is supplied to the prediction circuit 86, and the class information is stored in the prediction coefficient ROM 8.
7 is supplied.

In the prediction coefficient ROM 87, the same prediction coefficients as those stored in the prediction coefficient ROM 88 of FIG. 27 are stored. A 25 × 9 prediction coefficient corresponding to the information is read and supplied to the prediction circuit 86.

The prediction circuit 86 stores the 25 × 9 prediction coefficients from the prediction coefficient ROM 87 and the correction data constituting the prediction value calculation block of 5 × 5 pixels supplied from the prediction value calculation blocking circuit 83. Using the equation (1), a predicted value of 3 × 3 pixels of the original image is calculated, and an image of one frame including such predicted values is output as a decoded image.

Next, FIG. 32 shows an example of the configuration of the prediction section 73 in the receiving apparatus 4 of FIG. 18 when the optimum correction data calculation circuits 13 and 14 are configured as shown in FIG.

In this case, the prediction unit 73
22 and 1223.

When the optimum correction data calculation circuit 13 (14) is configured as shown in FIG. 25, as described above, the encoded data of the third layer is also included in the encoded data of the first and second layers for each class. No prediction coefficient is included, so the encoded data of the third hierarchy is basically composed of only the optimal correction data of the third hierarchy. The third-layer optimal correction data as such third-layer encoded data is supplied from the signal processing circuit 71 to the decoding unit 1222.

[0239] The decoding section 1222 is provided by the prediction section 7 shown in FIG.
Therefore, the decoding unit 1222 calculates the predicted value of the image of the second layer from the optimal correction data of the third layer in the same manner as described above, and A decoded image (predicted value) for one frame of the second layer composed of the predicted value is supplied to the decoding unit 1223. The decoding unit 1223 also includes the prediction unit 72 in FIG.
Therefore, the prediction value of the first hierarchy is calculated from the prediction value of the second hierarchy, and the image of one frame composed of such prediction values is converted into the image of the first hierarchy. It is output as a decoded image (predicted value).

Note that the decoding units 1222 and 1223
Basically, as described above, each of them is configured in the same manner as the prediction unit 72. However, the prediction coefficient ROM 87 stores in the decoding unit 1222 each class learned using the second layer image as a learning image. The prediction coefficient of each class is stored in the decoding unit 1223, and the prediction coefficient of each class learned using the image of the first hierarchy as the learning image is stored.

The optimum correction data calculation circuits 13 and 14 are configured as shown in FIG.
Of FIG. 31 or FIG. 32 respectively.
25x9 for each class when configured as shown in
Since it is not necessary to transmit and receive the prediction coefficient, the transmission capacity or the recording capacity can be reduced accordingly.

The prediction coefficient ROM 87 (FIG. 31) and 8
In FIG. 8 (FIGS. 22 and 27), it is possible to store the average value of the pixel values constituting the teacher block, instead of storing the prediction coefficient at the address corresponding to each class. is there. In this case, when the class information is given, the pixel value corresponding to the class is output, and the local decoding unit 1022 in FIG.
In the local decoding unit 2022, the prediction value calculation blocking circuit 42 and the prediction circuit 89 need not be provided. Also, in the prediction unit 72 of FIG.
This eliminates the need to provide the prediction value calculation blocking circuit 83 and the prediction circuit 86. Further, in this case, it is not necessary to transmit the prediction coefficients in FIG.

When the optimum correction data calculating circuits 13 and 14 are configured as shown in FIG. 25,
As described above, in the transmission device 1 shown in FIG. 2, it is not necessary to supply the first-level image data to the optimum correction data calculation circuit 13 and the optimum correction data calculation circuit 13
4 does not need to supply the second-layer optimum correction data output from the optimum correction data calculation circuit 13. In this case, since the prediction coefficient for each class is not obtained, the encoded data output from the signal processing circuit 15 does not include the prediction coefficient for each class as described above.

Therefore, in this case, the prediction unit 72 or 73 shown in FIG.
Alternatively, it can be configured without the 221 (FIG. 20).

Hereafter, as described with reference to FIG. 9, a method of sequentially obtaining prediction coefficients to obtain a predicted value will be referred to as a sequential method, and as described with reference to FIG. 22 and FIG. Stored in the prediction coefficient ROM 88 in advance,
A method of obtaining a predicted value by using this is called a ROM method.

Next, FIG. 33 shows another configuration example of the transmitting apparatus 1 of FIG. Note that, in the figure, parts corresponding to those in FIG. 2 are denoted by the same reference numerals.

That is, in this embodiment, only the first-layer image data is supplied to the optimum correction data calculation circuit 101, where the first-layer image data is converted to the second-layer image data. The optimum correction data of the hierarchy is calculated, and the signal processing circuit 15
To be supplied. In addition, the optimal correction data calculation circuit 102 outputs the second
Only the image data of the hierarchy is supplied, where the optimum correction data of the third hierarchy is calculated from the image data of the second hierarchy, and the encoded data of the third hierarchy is sent to the signal processing circuit 15 as the encoded data of the third hierarchy. It is made to be supplied.

Next, FIG. 34 shows a configuration example of the optimum correction data calculation circuit 101 of FIG. Note that the optimum correction data calculation circuit 102 is also the optimum correction data calculation circuit 10.
1, the description is omitted.

The first layer image data (the second layer image data in the case of the optimum correction data calculation circuit 102) is input to the blocking circuit 111. , Which is a unit for classifying the image data of the first hierarchy into a predetermined class according to its property, is divided into a class classification block centering on a pixel of interest and supplied to the ADRC processing circuit 112 and the delay circuit 115 It has been made to be.

The ADRC processing circuit 112 performs ADRC processing on the block (class classification block) from the blocking circuit 111, and supplies the resulting block composed of the ADRC code to the class classification circuit 113. It has been made like that.

The class classification circuit 113 performs a class classification process of classifying the block from the ADRC processing circuit 112 into a predetermined class according to its property, and determines whether the block (the target pixel constituting the block) Is supplied to the mapping coefficient memory 114 as class information.

The mapping coefficient memory 114 stores mapping coefficients obtained by learning (mapping coefficient learning) described below for each class information, and uses the class information supplied from the class classification circuit 113 as an address. The mapping coefficient stored in the address is read and supplied to the arithmetic circuit 116.

The delay circuit 115 includes a blocking circuit 111
Is stored in the mapping coefficient memory 1 in a mapping coefficient corresponding to the class information of the block.
The data is delayed until the data is read from the memory 14, and is supplied to the arithmetic circuit 116.

The arithmetic circuit 116 performs a predetermined arithmetic operation using the pixel values of the pixels constituting the block supplied from the delay circuit 115 and the mapping coefficients supplied from the mapping coefficient memory 114 and corresponding to the class of the block. Is performed so as to calculate encoded data (the second-layer optimal correction data in which the number of pixels of the first-layer image is reduced) by encoding the image by thinning out (reducing) the number of pixels. It has been done. That is, the arithmetic circuit 116 sets the pixel values of each pixel constituting the block output by the blocking circuit 111 to y ₁ , y ₂ ,... When the corresponding mapping coefficients are k ₁ , k ₂ ,..., Predetermined function values f (y ₁ , y ₂ ,.
.., k ₁ , k ₂ ,...), And the function value f (y
_{1, y 2, ···, k} 1, k 2, ···) , and block output from the blocking circuit 111 (classification block)
Is output, for example, as the pixel value of the center pixel among the pixels constituting

Therefore, assuming that the number of pixels constituting the class classification block output from the blocking circuit 111 is N pixels, the arithmetic circuit 16 thins out the image data to 1 / N and outputs this as coded data. It has been made like that.

The encoded data output from the arithmetic circuit 116 is not obtained by a so-called simple thinning-out process in which the center pixel of a block composed of N pixels is extracted and output. As described above, the function value f (y ₁ ,
y ₂ ,..., k ₁ , k ₂ ,...), and the function value f (y ₁ , y ₂ ,..., k ₁ , k ₂ ,. In other words, it can be considered that the pixel value of the pixel at the center of the block, which is obtained by a simple thinning-out process, is corrected to an optimal value for obtaining the original image based on the pixel values of the surrounding pixels. Therefore, the data obtained as a result of the operation of the mapping coefficients and the pixels constituting the block is hereinafter also referred to as optimal correction data as appropriate.

In the arithmetic processing in the arithmetic circuit 116, the pixel value of each pixel constituting the class classification block output from the blocking circuit 111 is converted into a function value f (y ₁ ,
y ₂ ,..., k ₁ , k ₂ ,...). Therefore, the coefficients k ₁ , k ₂ ,... Used for such processing are called mapping coefficients.

Next, the operation will be described with reference to the flowchart in FIG.

For example, the first layer image data is supplied to the blocking circuit 111 in units of one frame, and the blocking circuit 111 determines in step S101 that the first layer image data of one frame is in the first layer. The image is blocked into classifying blocks. That is, the blocking circuit 111 classifies the image data of the first hierarchy into a class of 9 pixels of 3 × 3 (horizontal × vertical) centering on the pixel of interest as shown by a rectangle in FIG. , And is sequentially supplied to the ADRC processing circuit 112 and the delay circuit 115.

In this case, the class classification block is formed of a square block composed of 3 × 3 pixels. However, the shape of the class classification block does not need to be a square. For example, the shape can be a rectangle, a cross, or any other shape. Also,
The number of pixels constituting the class classification block is also 3 × 3 = 9.
It is not limited to pixels. In addition, the class classification block may be configured not with adjacent pixels but with separated pixels. However, the shape and the number of pixels need to match those at the time of learning (mapping coefficient learning) described later.

When the ADRC processing circuit 112 receives the class classification block from the blocking circuit 111, the ADRC processing circuit 112 performs, for example, 1-bit ADRC processing on the block in step S102, whereby the block is represented by 1 bit. Block composed of pixels. A
The classification block subjected to the DRC processing is supplied to the classification circuit 113.

In the classifying circuit 113, step S1
At 03, the class classification block from the ADRC processing circuit 112 is classified, and the resulting class information is supplied to the mapping coefficient memory 114 as an address. Thereby, in step S104, the mapping coefficient corresponding to the class information supplied from the class classification circuit 113 is read from the mapping coefficient memory 114 and supplied to the arithmetic circuit 116.

On the other hand, the delay circuit 115 delays the class classification block from the blocking circuit 111 and waits until the mapping coefficient corresponding to the class information of the block is read from the mapping coefficient memory 114. 116. In step S 105, the computing unit 116 uses the pixel value of each pixel constituting the class classification block from the delay circuit 115 and the mapping coefficient from the mapping coefficient memory 114 to obtain the function value f (·) (this The parentheses in the function f
Pixel values y ₁ , y ₂ ,... And mapping coefficients k ₁ , k ₂ ,.
.. Are calculated, thereby calculating optimal correction data in which the pixel value of the central pixel (central pixel) constituting the class classification block is corrected to an optimal value. This optimum correction data is stored in step S10.
At 6, it is output to the signal processing circuit 15 (FIG. 33) as encoded data of the second hierarchy.

Then, the flow advances to step S107 to determine whether the processing for the image data of the first hierarchy for one frame has been completed. In step S107,
If it is determined that the processing for the image data of the first layer for one frame has not been completed yet, step S
Returning to step 102, the processing from step S102 is repeated for the next block for class classification. If it is determined in step S107 that the processing on the first layer of image data for one frame has been completed, the process returns to step S101, and the processing from step S101 onward is repeated for the next frame.

Next, FIG. 36 shows an example of the configuration of an image processing apparatus for performing a learning process (mapping coefficient learning) for calculating the mapping coefficients stored in the mapping coefficient memory 114 of FIG.

The memory 121 stores one or more frames of first-layer image data suitable for learning (hereinafter, appropriately referred to as learning images). When calculating a mapping coefficient to be stored in the mapping coefficient memory 114 constituting the optimum correction data calculating circuit 102 (FIG. 33), the memory 121 stores the image data of the second hierarchy.

The blocking circuit 122 reads out the image data stored in the memory 121 and forms the same block as the class classification block output from the blocking circuit 111 in FIG. It is supplied to the arithmetic circuit 126.

The ADRC processing circuit 123 or the class classification circuit 124 performs the same processing as in the case of the ADRC processing circuit 112 or the class classification circuit 113 in FIG. Therefore, the class classification circuit 124 outputs the class information of the block output by the blocking circuit 122. The class information is stored in the mapping coefficient memory 131.
Is supplied as an address.

The arithmetic unit 126 uses the pixels constituting the block supplied from the blocking circuit 122 and the mapping coefficients supplied from the mapping coefficient memory 131 to perform the same operation as in the arithmetic circuit 116 of FIG. Is performed, and the resulting correction data (function value f
(•)) is supplied to the local decoding unit 127.

[0270] The local decoding unit 127 is
26, the predicted value of the original learning image (the predicted value of the pixel value of the pixel forming the block output by the blocking circuit 122) is calculated based on the correction data supplied from the RO.
It is predicted (calculated) by the M method or the sequential method, and supplied to the error calculation unit 128. When the prediction value is obtained by the sequential method, the learning image stored in the memory 121 is supplied to the local decoding unit 127 to obtain the prediction coefficient for each class (here, for example, And the ROM method, the learning image is stored in the local decoding unit 127.
Not supplied).

The error calculating section 128 reads from the memory 121 the pixel value (true value) of the learning image corresponding to the prediction value supplied from the local decoding section 127, and calculates the prediction value of the learning image with respect to the pixel value of the learning image. A prediction error is calculated (detected), and the prediction error is used as error information by the determination unit 12.
9.

The judgment section 129 compares the error information from the error calculation section 128 with a predetermined threshold value ε1, and controls the mapping coefficient setting circuit 130 in accordance with the comparison result. Mapping coefficient setting circuit 130
Sets (changes) a set of mapping coefficients of the same number as the number of classes obtained as a result of the classification in the classification circuit 124 according to the control of the determination unit 129, and supplies the same to the mapping coefficient memory 131. I have.

The mapping coefficient memory 131 temporarily stores the mapping coefficients supplied from the mapping coefficient setting circuit 30. Note that the mapping coefficient memory 131 has storage areas capable of storing mapping coefficients (sets of mapping coefficients) of the number of classes classified by the class classification circuit 124. In each storage area, When a new mapping coefficient is supplied from the mapping coefficient setting circuit 130, the new mapping coefficient is stored instead of the already stored mapping coefficient.

The mapping coefficient memory 131 reads out the mapping coefficient stored at the address corresponding to the class information supplied from the class classification circuit 124 and supplies it to the arithmetic circuit 126.

Next, the operation will be described with reference to the flowchart in FIG.

First, the mapping coefficient setting circuit 13
In step S151, the number of sets of the initial values of the mapping coefficients is set to the number of classes to be classified by the class classification circuit 124 in step S151. In the mapping coefficient memory 131,
The mapping coefficient (initial value) from the mapping coefficient setting circuit 130 is stored at the address of the corresponding class.

Then, in step S152, the blocking circuit 122 converts all the learning images stored in the memory 121 into 3 × 3 pixels centered on the pixel of interest in the same manner as in the blocking circuit 111 of FIG. Block into blocks of pixels. Further, the blocking circuit 1
21 reads the block from the memory 121,
It is sequentially supplied to the ADRC processing circuit 123 and the arithmetic circuit 126.

In the ADRC processing circuit 123, step S
At 153, the block from the blocking circuit 122 is subjected to 1-bit ADRC processing as in the case of the ADRC processing circuit 112 in FIG. 34, and supplied to the class classification circuit 124. In the class classification circuit 124, in step S154, the ADRC processing circuit 12
3 is determined, and the class information is stored in the mapping coefficient memory 1 as an address.
31. Thereby, in step S155, the mapping coefficient is read from the address corresponding to the class information supplied from the class classification circuit 124 in the mapping coefficient memory 131, and supplied to the arithmetic circuit 126.

The operation circuit 126 is composed of a blocking circuit 122
When the block is received from the mapping coefficient memory 131 and the mapping coefficient corresponding to the class of the block is received from the mapping coefficient memory 131, the mapping coefficient and the pixel of the pixel constituting the block supplied from the blocking circuit 122 are supplied in step S156. The above function value f (·) is calculated using the values. The calculation result is supplied to the local decoding unit 127 as correction data obtained by correcting the pixel value of the central pixel of the block supplied from the blocking circuit 122.

That is, for example, assuming that a block of 3 × 3 pixels as shown by a rectangle in FIG. 10 is output from the block forming circuit 122, the arithmetic circuit 126 outputs Correction data obtained by correcting the pixel value of the pixel indicated by
27.

Therefore, in the arithmetic circuit 126, the number of pixels constituting the learning image is reduced to 1/9 and supplied to the local decoding unit 127.

After the correction data has been calculated in step S156, the flow advances to step S157 to determine whether correction data has been obtained for all the learning images stored in the memory 121. If it is determined in step S157 that the correction data for all the learning images has not been obtained yet, step S153 is performed.
Then, the processing of steps S153 to S157 is repeated until the correction data for all the learning images is obtained.

If it is determined in step S157 that correction data for all learning images has been obtained, that is, a thinned image obtained by thinning out all the learning images stored in the memory 121 to 1/9. Is obtained (however, the thinned image is not an image in which the learning image is simply thinned out to 1/9, but an image whose pixel value is obtained by an operation with a mapping coefficient) in step S
Proceeding to 158, the local decoding unit 127 locally decodes the thinned image to calculate a predicted value of the original learning image. This predicted value is supplied to the error calculator 128.

In the error calculating section 128, step S159
In, the learning image is read from the memory 121,
The prediction error of the prediction value supplied from the local decoding unit 127 for the learning image is calculated. That is, the pixel value of the learning image is represented by Y _ij, and the predicted value output from the local decoding unit 127 is represented by E [Y _ij ].
In the expression, the error calculation unit 128 calculates an error variance (sum of squares of error) Q represented by the following equation, and supplies this to the determination unit 129 as error information.

Q = Σ (Y _ij -E [Y _ij ]) ^{2 In} the above equation, Σ represents the summation for all the pixels of the learning image.

Upon receiving the error information from error calculating section 128, determining section 129 compares the error information with a predetermined threshold ε1, and determines the magnitude relation in step S160. In step S160, when it is determined that the error information is equal to or larger than the threshold ε1, that is, it is not recognized that the image composed of the predicted values obtained in the local decoding unit 127 is the same as the original learning image. In this case, the determination unit 129 outputs a control signal to the mapping coefficient setting circuit 130. Mapping coefficient setting circuit 130
In step S161, the mapping coefficient is changed in accordance with the control signal from the determination unit 129, and the changed mapping coefficient is newly stored in the mapping coefficient memory 131.

Then, the process returns to step S153, and the processes from step S153 are repeated again using the changed mapping coefficient stored in the mapping coefficient memory 131.

Here, the mapping coefficient setting circuit 130 may change the mapping coefficient at random, or if the current error information is smaller than the previous error information, the same as the previous time. Change with the tendency of
When the current error information becomes larger than the previous error information, the error information may be changed in a reverse tendency.

Further, the change of the mapping coefficient can be performed for all classes, or can be performed for only some of the classes. In the case of changing only the mapping coefficients of some classes, for example, a class having a strong influence on the error information may be detected, and only the mapping coefficients of such classes may be changed. The class having a strong influence on the error information can be detected, for example, as follows. That is, first, processing is performed using the initial value of the mapping coefficient to obtain the error information. Then, the mapping coefficient is changed by the same amount for each class, and the resulting error information is compared with the error information obtained when the initial value is used, and the difference is equal to or more than a predetermined value. Class
What is necessary is just to detect as a class which has a strong influence on error information.

The mapping coefficient is k1,
In the case where a plurality is set as one set such as k2,..., it is also possible to change only those having a strong influence on the error information.

Further, in the above-described case, the mapping coefficient is set for each class. However, the mapping coefficient may be set independently for each block, or may be set for each adjacent block. It is possible to set it with.

However, for example, when the mapping coefficients are independently set for each block, a plurality of sets of mapping coefficients may be obtained for a certain class (inversely, a plurality of sets of mapping coefficients may be obtained). , There may be a class for which no set of mapping coefficients can be obtained.) Since the mapping coefficients must be finally determined for each class, as described above, when a plurality of sets of mapping coefficients are obtained for a certain class,
It is necessary to determine one set of mapping coefficients by performing some processing on a plurality of sets of mapping coefficients.

On the other hand, when it is determined in step S160 that the error information is smaller than the threshold value ε1, that is, the image composed of the predicted values obtained by the local decoding unit 127 is the same as the original learning image. If accepted, the process ends.

At this point, the mapping coefficient memory 131
The mapping coefficient of FIG. 34 is determined as the optimal one for obtaining the correction data that can restore the decoded image (prediction value) that is recognized as the same as the original image. It is set in the memory 114.

Therefore, according to the optimum correction data obtained by using such mapping coefficients, it is possible to obtain an image substantially the same as the original image.

In the embodiment shown in FIG. 36, as described above, in the blocking circuit 122, the image is divided into 3 × 3 9 pixels centering on the target pixel, and the ADRC processing circuit is used. In 123, the 1-bit ADRC process is performed, so that the class classification circuit 12
The number of classes obtained by class classification by 4 is 512
(= (2 ¹ ) ⁹ ), so that 512 sets of mapping coefficients are obtained.

Next, FIG. 38 shows another example of the configuration of an image processing apparatus for performing learning (mapping coefficient learning) processing for calculating mapping coefficients stored in the mapping coefficient memory 114 of FIG. .

According to the image processing apparatus shown in FIG. 36, in addition to the case where the function f is expressed by a linear linear equation, the case where the function f is expressed by a non-linear equation or a quadratic or higher equation, Although the optimum prediction coefficient can be obtained, the image processing apparatus in FIG. 38 can obtain the optimum prediction coefficient only when the function f is represented by a linear linear expression.

That is, in the image processing apparatus of FIG. 38, in FIG. 34, the pixel values of the pixels constituting the 3 × 3 pixel block output by the blocking circuit 111 are represented by y ₁ , y ₂ ,.
, Y ₉ and the mapping coefficient memory 114
K _1, k ₂ but the mapping coefficients outputted, ..., in a case where a k _9, the arithmetic circuit 116, the function value f (y _1, y ₂ according to the following equation, ..., k _1, k ₂ ,...) Can be used when calculating correction data.

F (·) = k ₁ y ₁ + k ₂ y ₂ +... + K ₉ y ₉

The thinning circuit 171 is supplied with first-layer image data as learning images suitable for learning, for example, on a frame-by-frame basis. The image data of the second hierarchy is formed by thinning out the number of pixels. The optimum correction data calculation circuit 102 (FIG. 3)
When calculating the mapping coefficient to be stored in the mapping coefficient memory 114 constituting 3), the thinning circuit 17 is used.
1 is supplied with the image data of the second hierarchy, whereby the image data of the third hierarchy is formed.

The image data of the second hierarchy obtained in the thinning circuit 171 is supplied to the optimum correction data calculating section 170. Optimal correction data calculation unit 17
0 is the correction unit 21 and the local decoding unit 2 in FIG.
2. Error calculating unit 23 or determining unit 24 (or correcting unit 21 and local decoding unit 102 in FIG. 21)
2, the error calculating unit 23 or the determining unit 24, and the correcting unit 172 and the local decoding unit 173 each configured similarly to the correcting unit 21, the local decoding unit 2022, the error calculating unit 23, or the determining unit 2024 in FIG. , An error calculation unit 174 or a determination unit 175, in which an image input thereto, that is, an optimal image for a second layer image in which the number of pixels of the first layer image is reduced. Correction data is generated, and the latch circuit 1
76. In the case where the local decoding unit 173 obtains a prediction value by a sequential method, an original image (here, a first image as a learning image) is used to obtain a prediction coefficient for each class.
(Hierarchical image data) is supplied to the local decoding unit 173.

The latch circuit 176 has a built-in memory 176A, and stores the optimum correction data supplied from the optimum correction data calculation section 170 in the memory 176A. Further, the latch circuit 176 reads out the optimum correction data stored in the memory 176A corresponding to the central pixel of the block read from the memory 177A of the blocking circuit 177, and
80. When one frame of correction data is stored in the memory 176A, the latch circuit 176 outputs a control signal indicating this to the blocking circuit 177.

The blocking circuit 177 includes the thinning circuit 1
The same image data as the image data supplied to 71, that is, here, the image data of the first hierarchy is supplied in units of one frame. Blocking circuit 1
77 has a built-in memory 177A.
The learning image supplied thereto is stored in 77A. Further, upon receiving the control signal from the latch circuit 176, the blocking circuit 177
The learning image stored in 7A is divided into blocks each including a pixel of interest and 3 × 3 pixels at the center, as in the case of the blocking circuit 111 in FIG. The data is supplied to the processing circuit 178 and the memory 180.

When reading a block from the built-in memory 177A, the blocking circuit 177 sends a control signal indicating the position of the block to the latch circuit 176.
To be supplied. The latch circuit 176 recognizes a 3 × 3 pixel block read from the memory 177A based on the control signal, and reads the optimal correction data corresponding to the center pixel of the block from the memory 176A as described above. It has been made like that. That is, a block of a certain 3 × 3 pixel and the optimum correction data corresponding to the block are simultaneously supplied to the memory 180.

The ADRC processing circuit 178 or the class classification circuit 179 has the same configuration as the ADRC processing circuit 112 or the class classification circuit 113 in FIG. 34, respectively. Then, the class information about the block from the blocking circuit 177 output from the class classification circuit 179 is supplied to the memory 180 as an address.

The memory 180 stores the optimum correction data supplied from the latch circuit 176 and the block supplied from the blocking circuit 177 in association with the address corresponding to the class information supplied from the class classification circuit 179. It has been made to be. The memory 180 has 1
A plurality of pieces of information can be stored in one address, whereby a plurality of sets of optimal correction data and blocks corresponding to certain class information can be stored.

The arithmetic circuit 181 is associated with the nine pixels y ₁ , y ₂ ,..., Y ₉ constituting the 3 × 3 block of the learning image stored in the memory 180 and associated with the block. By reading the optimal correction data y ′ and applying the least squares method to them, mapping coefficients k _{1 to} k ₉ are obtained for each class and supplied to the memory 182. The memory 182 stores the mapping coefficients k _{1 to} k ₉ for each class supplied from the arithmetic circuit 181 at addresses corresponding to the classes.

Next, the operation will be described with reference to the flowchart in FIG.

When the image data of the first hierarchy as the learning image is input, the learning image is converted to the block circuit 1.
The data is stored in the memory 177A of the memory 77 and supplied to the thinning circuit 171. In the thinning circuit 171, image data of the second hierarchy is formed from image data of the first hierarchy,
It is supplied to the optimum correction data calculation section 170.

Upon receiving the image data of the second hierarchical level, the optimum correction data calculating section 170 calculates the optimal correction data of the second hierarchical level in step S131, and
The data is supplied to and stored in the memory 176A at 76.

When the one-frame optimal correction data is stored in the memory 176A, the latch circuit 176 outputs a control signal to the blocking circuit 177. Upon receiving the control signal from the latch circuit 176, the blocking circuit 177 determines in step S132 that the memory 17
The learning image stored in 7A is divided into blocks each including 3 × 3 pixels. Then, the blocking circuit 177
Reads the block of the learning image stored in the memory 177A and reads the ADRC processing circuit 178 and the memory 1
80.

At the same time, the blocking circuit 177
When reading a block from the memory 177A, a control signal indicating the position of the block is supplied to the latch circuit 176, and the latch circuit 176 responds to the control signal by:
The block of 3 × 3 pixels read from the memory 177A is recognized, and the optimum correction data corresponding to the central pixel of the block is read and supplied to the memory 180.

Then, the flow advances to step S133 to execute ADR
In the C processing circuit 178, the block from the blocking circuit 177 is subjected to ADRC processing, and further, in the class classification circuit 179, the block is classified.
This classification result is stored in the memory 180 as an address.
Supplied to

In the memory 180, in step S134, at the address corresponding to the class information supplied from the class classification circuit 179, the optimal correction data supplied from the latch circuit 176 and the block supplied from the blocking circuit 177 (learning). ) Are stored in association with each other.

The flow advances to step S135 to determine whether the memory 180 has stored the blocks for one frame and the optimum correction data. Step S13
In 5, when it is determined that the block for one frame and the optimal correction data are not yet stored in the memory 180, the next block is read from the blocking circuit 177 and the corresponding block is read from the latch circuit 176. The optimal correction data to be read is read out, and the process returns to step S133, and thereafter, the processing after step S133 is repeated.

If it is determined in step S135 that the blocks for one frame and the optimal correction data have been stored in the memory 180, the flow advances to step S136 to determine whether the processing has been completed for all the learning images. You. If it is determined in step S136 that the processing for all the learning images has not been completed, the process returns to step S131, and the processing from step S131 is repeated for the next learning image.

On the other hand, if it is determined in step S136 that the processing for all the learning images has been completed, the process proceeds to step S137, where the arithmetic circuit 181 determines the optimum correction data and the block stored in the memory 180,
The data is read out for each class, and a normal equation as shown in Expression (7) is established by these. Further, the arithmetic circuit 181
Calculates the mapping coefficient for each class that minimizes the error by solving the normal equation in step S138. This mapping coefficient is calculated in step S13.
At 9, the data is supplied to and stored in the memory 182, and the processing is terminated.

When the function f is represented by a linear linear expression, the mapping coefficient stored in the memory 182 as described above is stored in the mapping coefficient memory 114 shown in FIG.
And optimal correction data can be obtained using this.

In some classes, it is not possible to obtain as many normal equations as can obtain mapping coefficients. In such a case, the arithmetic circuit 1 in FIG.
At 16, 3 output from the blocking circuit 111 is output.
For example, mapping coefficients for outputting, for example, an average value of nine pixels constituting a block of × 3 pixels, that is, k _{1 to} k ₉ = 1/9 are set as default values.

Here, when transmitting apparatus 1 is configured as shown in FIG. 33, receiving apparatus 4 is configured as shown in FIG. However, in the receiving device 4 of FIG. 18, the prediction unit 72 or 73 is configured as shown in FIG. 31 or 32, respectively. In this case, the prediction unit 72
The prediction coefficients used in the prediction unit 73 are basically the same as those of the local decoding unit 127 in FIG.
It is necessary to adopt the same one used in the local decoding unit 173 of FIG.

By the way, in the case of FIG. 2, first,
As shown in FIG. 40 (A), the second-layer image data is corrected so that a prediction value of the first layer can be obtained in which an error (error information) from the image data of the first layer is equal to or less than a predetermined threshold ε. The optimum correction data obtained is obtained by the optimum correction data calculation circuit 13, and thereafter, as shown in FIG. 3B, the error (error information) from the optimum correction data of the second hierarchy is set to a predetermined threshold value ε or less. The optimum correction data obtained by correcting the image data of the third hierarchy, which gives the predicted value of the second hierarchy, is obtained by the optimum correction data calculation circuit 14. The optimum correction data of the third hierarchy, which is the lowest hierarchy, is In addition, for example, it is also possible to obtain as shown in FIG.

That is, the prediction value of the second hierarchy is obtained from the correction data obtained by correcting the image data of the third hierarchy, and the prediction value of the first hierarchy is obtained by using the prediction value of the second hierarchy as it is. Then, the correction data that makes the prediction error (error information) of the prediction value of the first hierarchy equal to or smaller than the predetermined threshold ε is set as the optimum correction data of the third hierarchy.

FIG. 42 shows an example of the configuration of the transmitting apparatus 1 for obtaining the optimum correction data of the third hierarchy as described above. Note that, in the figure, parts corresponding to those in FIG. 2 are denoted by the same reference numerals.

In this embodiment, the image data of the first to third layers is supplied to the optimum correction data calculating circuit 201, where the image data of the third layer is corrected. A predicted value of the second layer is obtained from the correction data, and further, a predicted value of the first layer is obtained using the predicted value of the second layer as it is. Then, the optimum correction data calculation circuit 201 calculates the prediction error (error information) of the prediction value of the first layer by a predetermined threshold ε.
When the following correction data is obtained, the correction data is set as the optimum correction data of the third hierarchy, and is output to the signal processing circuit 15 as the encoded data of the third hierarchy.

In the embodiment shown in FIG. 42, the image data of the first hierarchy is directly supplied to the signal processing circuit 15 as the encoded data of the first hierarchy.
In the embodiment shown in FIG. 42, the second layer image data is not supplied to the signal processing circuit 15. However, similar to the first layer, the second layer image data is used as the second layer encoded data. It is also possible to supply 15.

FIG. 43 shows an example of the configuration of the optimum correction data calculation circuit 201 shown in FIG. In the figure, the same reference numerals are given to portions corresponding to the case in FIG. That is, the optimum correction data calculation circuit 201
Has basically the same configuration as the optimal correction data calculation circuit 13 in FIG. 5 except that a local decoding unit 231 is newly provided.

In this embodiment, the image data of the third hierarchy is input to the correction unit 21 and the third
Correction data obtained by correcting the hierarchical image data is output. The local decoding unit 22 is configured to receive the correction data output from the correction unit 21 and the image data of the second hierarchy, where the predicted value of the second hierarchy is calculated and output. It has been made to be. Further, the local decoding unit 231 is configured, for example, in the same manner as the local decoding unit 22. The local decoding unit 231 receives the prediction value of the second hierarchy output from the local decoding unit 22 and the image data of the first hierarchy. It has been made like that. Then, in the local decoding unit 231,
The prediction value of the first hierarchy is calculated and output.

Therefore, in this embodiment, the local decoders 22 and 231 calculate the predicted value in a sequential manner. However, the local decoding units 22 and 231 are configured, for example, like the local decoding unit 1022 shown in FIG.
It is also possible to calculate a predicted value by a method.

Next, the operation will be described with reference to the flowchart in FIG.

When the image data of the third hierarchy is supplied to the correction unit 21, the correction unit 21 first converts the image data of the third hierarchy into the local data without performing any correction in step S201. Decoding unit 22 and determination unit 2
4 is output. In step S202, the local decoding unit 22 corrects the correction data from the correction unit 21 (at first, as described above, the image data itself of the third hierarchy).
Is locally decoded.

That is, in step S202, a prediction coefficient for each class of the second hierarchy is obtained in the same manner as described with reference to FIG. 9, and further, a prediction coefficient for the second hierarchy is calculated based on the prediction coefficient for each class. The value is obtained and supplied to the local decoding unit 231 (however, the local decoding unit 2
2 is configured as the local decoding unit 1022, whereby when the predicted value of the second hierarchy is obtained by the ROM method, the prediction coefficient of the second hierarchy is read from the ROM, and based on the prediction coefficient, , The predicted value of the second hierarchy is obtained).

[0333] In step S203, the local decoding unit 231 outputs the second
The predicted value of the hierarchy is locally decoded.

That is, in step S203, the prediction coefficient for each class of the first hierarchy is obtained in the same manner as in the case of the local decoding unit 22, and the prediction coefficient for the first hierarchy is further determined based on the prediction coefficient for each class. The value is obtained and supplied to the error calculation unit 23 (however, when the local decoding unit 231 is configured like the local decoding unit 1022, and thus the prediction value of the first hierarchy is obtained by the ROM method, The first-layer prediction coefficient is read from the ROM, and the first-layer prediction value is obtained based on the prediction coefficient.)

Thereafter, in steps S204 to S207, the same processes as those in steps S3 to S6 in FIG. 6 are performed, whereby the third-layer optimal correction for making the predicted value of the first hierarchy equal to or smaller than the threshold ε is performed. Data is required.

Therefore, according to the third-layer optimal correction data obtained in this way, as in the case described above,
A high quality decoded image can be obtained.

In the embodiment of FIG. 42, the coded data of the second hierarchy is not supplied to the signal processing circuit 15, but the coded data of the second hierarchy is supplied to the signal processing circuit 15. In the above, as described above, the second
In addition to using the image data itself of the hierarchy as the encoded data of the second hierarchy, for example, the predicted value of the second hierarchy output by the local decoding unit 22 can be the encoded data of the second hierarchy. Further, for example, the optimum correction data calculation circuit 13 shown in FIG.
It is also possible to use the optimum correction data of the hierarchy as the encoded data of the second hierarchy.

Also in the case of the embodiment shown in FIG. 42, the encoded data output from the signal processing circuit 15 does not include all of the encoded data of the first to third layers. It is possible to include only the encoded data of the hierarchy.

Note that the encoded data output from the transmitting device 1 shown in FIG. 42 can be decoded by the receiving device 4 shown in FIG. However, in the receiving device 4 of FIG.
Alternatively, they need to be configured as shown in FIG.

FIG. 45 shows another example of the configuration of the optimum correction data calculation circuit 201 shown in FIG. Note that, in the figure, parts corresponding to the case in FIG. 43 are denoted by the same reference numerals. That is, the optimum correction data calculation circuit 201 shown in FIG.
1 instead of the local decoder 3022 or 323
1 is provided, and the configuration is basically the same as that in FIG. 43 except that the multiplexing unit 25 is not provided.

The local decoding unit 3022 calculates the predicted value of the second hierarchy from the correction data obtained by correcting the image data of the third hierarchy input thereto, for example, by a ROM method, and supplies the predicted value to the local decoding unit 3231. Has been made. The local decoding unit 3231 calculates the first-layer predicted value from the second-layer predicted value from the local decoding unit 3022, for example, again by the ROM method, and supplies the predicted value to the error calculating unit 23.

In the present embodiment, both local decoding units 3022 and 3231
The configuration is the same as that of the local decoding unit 2022 shown in FIG. 27, and therefore, the prediction coefficient is not output to the determination unit 24.

Next, the operation will be described with reference to the flowchart in FIG.

In steps S3201 to S3206, basically, steps S201 to S20 in FIG.
6, the same processing is performed. However, in step S3202, the local decoding unit 30
22, the prediction coefficient RO is calculated as described with reference to FIG.
From M88, a prediction coefficient of a required class of the second hierarchy is read, and a prediction value of the second hierarchy is obtained based on the prediction coefficient, and is supplied to the local decoding unit 3231. Also in step S3203, step S320
2, the local decoding unit 3231
In, a prediction coefficient of a required class of the first hierarchy is read, and a prediction value of the first hierarchy is obtained based on the prediction coefficient, and is supplied to the error calculation unit.

If it is determined in step S3205 that the error information is equal to or less than the threshold value ε, that is, if the third-layer optimal correction data that makes the predicted value of the first hierarchy equal to or less than the threshold value ε is obtained, Proceeding to step S3207, the determination unit 24 outputs the third-layer optimal correction data to the signal processing circuit 15, and ends the processing. That is, since the prediction coefficients are not output from the local decoding units 3022 and 3231 configured similarly to the local decoding unit 2022, only the optimal correction data is output in step S3207.

A high-quality decoded image can be obtained from the third-layer optimal correction data obtained as described above, as in the case described above.

When the optimum correction data calculation circuit 201 of the transmitting apparatus 1 of FIG. 42 is configured as shown in FIG. 45, since the prediction coefficients are not transmitted from the above, the receiving apparatus of FIG. In 4, the prediction unit 72 or 7
3 needs to be configured as shown in FIG. 31 or FIG. 32, for example.

Although the image processing system to which the present invention is applied has been described above, such an image processing apparatus has a large data amount in addition to encoding a standard television signal such as the NTSC system. This is particularly effective when encoding a so-called high-vision television signal or the like.

In the present embodiment, the block is formed for one frame of image. However, the block collects pixels at the same position in, for example, a plurality of frames continuous in time series. It is also possible to configure it.

In this embodiment, the sum of the squares of the error is used as the error information.
It is possible to use the sum or the like of the raised values. Which one to use as error information is determined, for example, by
It is possible to determine based on the convergence and the like.

Furthermore, in the present embodiment, the image is encoded into three layers, but the number of layers is not limited to three.

As the optimum correction data of the third hierarchy, which is the lowest hierarchy, in addition to the above-described case, for example, a correction error for which the prediction error of the prediction value of each hierarchy is obtained and the total value thereof is equal to or less than a predetermined value It is also possible to ask for.

Further, for example, in the embodiment of FIG. 13, a normal equation is set up for each frame, and the prediction coefficient for each class is obtained. It is also possible to establish a normal equation in units of one field or a plurality of frames. The same applies to other processes.

The present invention can also be implemented by hardware or by reading the application program from a recording medium such as a hard disk on which the application program for performing the above-described processing is recorded, and causing the computer to execute the program. Is feasible.

[0355]

According to the image encoding apparatus of the first aspect and the image encoding method of the eleventh aspect, the image data of the second layer having a smaller number of pixels than the image data of the first layer Is corrected, and a predicted value of the image data of the first layer is predicted from the corrected data. Further, a prediction error of a prediction value of the image data of the first layer is calculated, and the appropriateness of the correction data is determined based on the prediction error of the first layer. Then, the correction data determined to be appropriate is set as the image data of the second hierarchy. Therefore, it is possible to obtain high-quality first-layer image data from the second-layer image data.

According to the image decoding apparatus of claim 12, the image decoding method of claim 18, the transmission method of claim 19, and the recording medium of claim 20, encoded data Forms image data of a second layer having a smaller number of pixels than image data of the first layer, corrects the image data of the second layer, outputs correction data, and outputs the first data from the correction data. The prediction value of the image data of the first layer, calculate the prediction error of the prediction value for the image data of the first layer, and determine the appropriateness of the correction data based on the prediction error of the first layer. Repeat to do
The corrected correction data is included as image data of the second hierarchy. Therefore, a high-quality decoded image can be obtained from the encoded data.

[0357] According to the image coding apparatus of the present invention and the image coding method of the present invention, mapping of a class corresponding to a target pixel of interest in an image of the first hierarchy. By performing a predetermined calculation using the coefficient and the pixel of interest, image data of the second layer in which the number of pixels of the image data of the first layer is reduced is calculated. Therefore, it is possible to obtain high-quality first-layer image data from the second-layer image data.

According to the image decoding apparatus of claim 29, the image decoding method of claim 35, the transmission method of claim 36, and the recording medium of claim 37, the encoded data Classifies the pixels constituting the image of the first hierarchy into any one of predetermined classes according to their properties, and for each class, from a mapping coefficient storage unit storing a predetermined mapping coefficient, A first coefficient obtained by reading a mapping coefficient of a class corresponding to a focused pixel of interest in an image of the first layer and performing a predetermined operation using the mapping coefficient and the focused pixel is obtained.
The image of the second hierarchy in which the number of pixels of the image of the second hierarchy is reduced is included. Therefore, a high-quality decoded image can be obtained from the encoded data.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a configuration of an embodiment of an image processing system to which the present invention has been applied.

FIG. 2 is a block diagram illustrating a configuration of a first embodiment of the transmission device 1 of FIG. 1;

FIG. 3 is a diagram for explaining processing of thinning circuits 11 and 12 in FIG. 2;

FIG. 4 is a diagram for explaining processing of the thinning circuits 11 and 12 in FIG. 2;

FIG. 5 is a block diagram showing a first configuration example of an optimum correction data calculation circuit 13 (14) in FIG. 2;

FIG. 6 is a flowchart for explaining processing of an optimum correction data calculation circuit 13 of FIG. 5;

FIG. 7 is a block diagram illustrating a configuration example of a correction unit 21 in FIG. 5;

8 is a flowchart for explaining the operation of the correction unit 21 in FIG.

9 is a block diagram illustrating a configuration example of a local decoding unit 22 in FIG.

FIG. 10 is a diagram for explaining the processing of the class classification blocking circuit 41 of FIG. 9;

FIG. 11 is a diagram illustrating a class classification process.

FIG. 12 is a diagram for explaining an ADRC process.

FIG. 13 is a flowchart for explaining the operation of the local decoding unit 22 in FIG. 9;

14 is a block diagram illustrating a configuration example of an error calculation unit 23 in FIG.

FIG. 15 is a flowchart for explaining the operation of the error calculator 23 in FIG. 14;

16 is a block diagram illustrating a configuration example of a determination unit 24 in FIG.

FIG. 17 is a flowchart for explaining the operation of the determination unit 24 in FIG. 16;

FIG. 18 is a block diagram showing a configuration of an embodiment of the receiving device 4 of FIG.

19 is a block diagram illustrating a configuration example of a prediction unit 72 in FIG.

20 is a block diagram illustrating a configuration example of a prediction unit 73 in FIG.

FIG. 21 is an optimum correction data calculation circuit 13 (14) in FIG. 2;
It is a block diagram which shows the 2nd example of a structure.

FIG. 22 is a block diagram illustrating a configuration example of a local decoding unit 1022 in FIG. 21.

FIG. 23 is a flowchart illustrating a process of a local decoding unit 1022 in FIG. 22.

FIG. 24 is a block diagram illustrating a configuration example of an image processing apparatus that performs a learning process for obtaining a prediction coefficient.

FIG. 25 is an optimum correction data calculation circuit 13 (14) of FIG. 2;
It is a block diagram which shows the 3rd example of a structure.

26 is a diagram showing the optimum correction data calculation circuit 13 (1
It is a flowchart for demonstrating the process of 4).

27 is a block diagram illustrating a configuration example of a local decoding unit 2022 in FIG.

FIG. 28 is a flowchart illustrating a process of a local decoding unit 2022 in FIG. 27;

29 is a block diagram illustrating a configuration example of a determination unit 2024 in FIG.

FIG. 30 is a flowchart for explaining processing of a determination unit 2024 in FIG. 29;

FIG. 31 is a block diagram illustrating another configuration example of the prediction unit 72 in FIG. 18;

32 is a block diagram illustrating another configuration example of the prediction unit 73 in FIG.

FIG. 33 is a block diagram illustrating a configuration of a second embodiment of the transmission device 1 of FIG. 1;

FIG. 34 shows an optimum correction data calculation circuit 101 (1) shown in FIG.
FIG. 2B is a block diagram illustrating a configuration example.

FIG. 35 is a flowchart for explaining the operation of the optimum correction data calculation circuit 101 of FIG. 34;

FIG. 36 is a block diagram illustrating a configuration of a first embodiment of an image processing apparatus that performs learning for obtaining mapping coefficients.

FIG. 37 is a flowchart illustrating the operation of the image processing apparatus of FIG. 36;

FIG. 38 is a block diagram illustrating a configuration of a second embodiment of the image processing apparatus that performs learning for obtaining mapping coefficients.

FIG. 39 is a flowchart for explaining the operation of the image processing apparatus of FIG. 38;

FIG. 40 is a diagram for describing a first hierarchical coding method according to the present invention.

FIG. 41 is a diagram for explaining a second hierarchical coding method according to the present invention.

FIG. 42 is a block diagram illustrating a configuration of a third embodiment of the transmission device 1 of FIG. 1;

FIG. 43 is a block diagram illustrating a configuration example of an optimum correction data calculation circuit 201 in FIG. 42;

FIG. 44 is a flowchart illustrating the operation of the optimum correction data calculation circuit 201 of FIG. 43.

FIG. 45 is a block diagram illustrating another configuration example of the optimum correction data calculation circuit 201 in FIG. 42;

FIG. 46 is a flowchart for explaining the operation of the optimum correction data calculation circuit 201 of FIG. 45;

FIG. 47 is a block diagram illustrating a configuration of an example of a conventional image encoding device that performs hierarchical encoding.

FIG. 48 is a block diagram illustrating a configuration of an example of a conventional image decoding device that performs hierarchical decoding.

[Explanation of symbols]

1 transmission device, 2 recording medium, 3 transmission line, 4
Receiver, 11, 12 Decimation circuit, 13, 14
Optimal correction data calculation circuit, 15 signal processing circuit, 2
1 correction unit, 22 local decoding unit, 23 error calculation unit, 24 determination unit, 25 multiplexing unit, 32
Correction circuit, 33 correction value ROM, 41 class classification blocking circuit, 42 predicted value calculation blocking circuit, 43 class classification adaptive processing circuit, 44 ADRC
Processing circuit, 45 class classification circuit, 46 adaptive processing circuit, 51 blocking circuit, 52 square error calculation circuit, 53, 54 arithmetic unit, 55 integrator, 56
Memory, 61 prediction coefficient memory, 62 correction data memory, 63 error information memory, 64 comparison circuit,
65 control circuit, 71 signal processing circuit, 72, 7
3 prediction unit, 80 decoding unit, 81 separation unit, 8
2 Blocking circuit for class classification, 83 Blocking circuit for prediction value calculation, 84 ADRC processing circuit, 85
Classification circuit, 86 prediction circuit, 86A memory,
87,88 prediction coefficient ROM, 89 prediction circuit,
91 Blocking circuit for learning, 92 Blocking circuit for teacher, 93 ADRC processing circuit, 94 class classification circuit, 95 switch, 96 learning data memory,
97 counter, 98 teacher data memory, 99
Arithmetic circuit, 100 memory, 101, 102 optimal correction data calculation circuit, 111 blocking circuit,
112 ADRC processing circuit, 113 class classification circuit, 114 mapping coefficient memory, 115 delay circuit, 116 arithmetic circuit, 121 memory, 12
2 Blocking circuit, 123 ADRC processing circuit,
124 class classification circuit, 126 arithmetic circuit, 12
7 local decoding unit, 128 error calculating unit, 1
29 determination unit, 130 mapping coefficient setting circuit,
131 mapping coefficient memory, 141 blocking circuit for class classification, 142 blocking circuit for calculating predicted values, 143 adaptive processing circuit for class classification, 144 A
DRC processing circuit, 145 class classification circuit, 146
Prediction coefficient ROM, 147 prediction circuit, 170 optimal correction data calculation unit, 171 thinning circuit, 172
Correction unit, 173 Local decoding unit, 174
Error calculating section, 175 determining section, 176 latch circuit, 176A memory, 177 blocking circuit,
177A memory, 178 ADRC processing circuit,
179 Classification circuit, 180 memory, 181
Arithmetic circuit, 182 memory, 201 optimal correction data calculation circuit, 221 separation unit, 222, 223
Decoding unit, 231, 1022 Local decoding unit, 1222, 1223 Decoding unit, 2022
Local decoding unit, 2024 judgment unit, 302
2,3231 local decoding unit

Claims (37)

[Claims] 1. An image coding apparatus for performing hierarchical coding, comprising: forming means for forming image data of a second layer having a smaller number of pixels than image data of a first layer; Correction means for correcting the image data of the first layer, outputting correction data, prediction means for predicting a predicted value of the image data of the first layer from the correction data, and a correction means for the image data of the first layer Calculating means for calculating a prediction error of a predicted value; determining means for determining adequacy of the correction data output by the correction means based on the prediction error; and An output unit that outputs the correction data as the second-layer image data. 2. The predicting unit includes: a classifying unit that classifies the correction data into a predetermined class according to a property thereof; and a prediction value calculating unit that obtains the predicted value corresponding to the class. The image encoding device according to claim 1, wherein: 3. The prediction means comprises: a prediction coefficient calculation means for obtaining a prediction coefficient for calculating the prediction value by a linear combination with the correction data; and a prediction for obtaining the prediction value from the prediction coefficient and the correction data. 2. The apparatus according to claim 1, further comprising a value calculating unit.
An image encoding device according to claim 1. 4. The predicting unit: a classifying unit that classifies the correction data into a predetermined class according to a property thereof; and a prediction coefficient for calculating the prediction value by a linear combination with the correction data, A prediction coefficient calculation unit for obtaining the prediction value from the prediction coefficient obtained for the class of the correction data and the correction data; and a prediction value calculation unit for obtaining the prediction value from the correction data. Item 2. The image encoding device according to Item 1. 5. The image encoding apparatus according to claim 4, wherein the output unit outputs the prediction coefficient for each class together with the correction data. 6. A prediction coefficient storage unit that stores, for each predetermined class, a prediction coefficient for calculating the prediction value by a linear combination with the correction data, wherein the correction data includes: Classifying means for classifying the correction data into one of the predetermined classes according to the property thereof; and prediction value calculating means for obtaining the prediction value from the prediction coefficient for the class of the correction data and the correction data. The image encoding device according to claim 1, wherein: 7. The prediction coefficient for each class stored in the prediction coefficient storage means is generated by performing learning using learning image data. An image encoding device according to claim 1. 8. The image encoding apparatus according to claim 7, wherein the output unit outputs the prediction coefficient for each class together with the correction data. 9. The image processing apparatus according to claim 1, wherein the correction unit includes a storage unit that stores a correction value for correcting the image data of the second layer, and uses the correction value to store an image of the second layer. The image encoding device according to claim 1, wherein the data is corrected. 10. The determination means determines whether the correction data is appropriate based on whether or not the prediction error is equal to or less than a predetermined value. The output means determines whether the correction error is equal to or less than a predetermined value. The image encoding device according to claim 1, wherein the correction data is output. 11. An image encoding method for performing hierarchical encoding, comprising: forming image data of a second layer having a smaller number of pixels than image data of a first layer; And outputs correction data. From the correction data, predicts a predicted value of the image data of the first layer, and calculates a prediction error of the predicted value for the image data of the first layer. Repeating the determination of the correctness of the correction data based on the prediction error of the first layer, and setting the corrected correction data as the image data of the second layer. Image encoding method. 12. An image decoding apparatus for decoding encoded data obtained by hierarchical encoding, comprising: a receiving unit that receives the encoded data; and a decoding unit that decodes the encoded data. Wherein the encoded data forms image data of a second layer having a smaller number of pixels than image data of the first layer, corrects the image data of the second layer, and outputs corrected data Predicting a predicted value of the image data of the first layer from the correction data; calculating a prediction error of the predicted value with respect to the image data of the first layer; An image decoding apparatus comprising: repeating the determination of the correctness of the correction data based on an error; and including the corrected correction data as image data of the second hierarchy. 13. The classifying means for classifying the image data of the second layer into a predetermined class according to the nature of the image data, and the image data of the first layer corresponding to the class. 13. The image decoding apparatus according to claim 12, further comprising: a predicted value calculating unit that calculates a predicted value of the data. 14. The coded data also includes a prediction coefficient for calculating the prediction value by a linear combination with the image data of the second hierarchy, wherein the decoding unit includes the prediction coefficient and the prediction coefficient 13. The image decoding apparatus according to claim 12, further comprising a predicted value calculating unit that calculates a predicted value of the image data of the first layer from the image data of the second layer. 15. The encoded data also includes a prediction coefficient for each predetermined class for calculating the predicted value by linear combination with the image data of the second hierarchy. A classifying unit that classifies the image data of the second layer into a predetermined class according to its property; a prediction coefficient for the class of the image data of the second layer; 13. The image decoding apparatus according to claim 12, further comprising: a prediction value calculating unit that calculates a prediction value of the image data of the first hierarchy from the image data. 16. A prediction coefficient storage means for storing, for each predetermined class, a prediction coefficient for calculating the prediction value by a linear combination with the image data of the second hierarchy. A classifying unit that classifies the image data of the second hierarchy into one of the predetermined classes according to the property thereof; and the prediction coefficient of the class of the image data of the second hierarchy is 13. A prediction value calculation means for reading out from the prediction coefficient storage means and calculating a prediction value of the image data of the first hierarchy from the prediction coefficient and the image data of the second hierarchy. 5. The image decoding device according to claim 1. 17. The method according to claim 16, wherein the prediction coefficient for each class stored in the prediction coefficient storage means is generated by performing learning using image data for learning. 5. The image decoding device according to claim 1. 18. An image decoding method for decoding encoded data obtained by hierarchical encoding, wherein the encoded data is a second layer having a smaller number of pixels than image data of a first layer. Forming the image data of the second layer, correcting the image data of the second layer, outputting correction data, predicting a predicted value of the image data of the first layer from the correction data, For the image data of the hierarchy, the prediction error of the prediction value is calculated, and based on the prediction error for the first hierarchy, the determination of the appropriateness of the correction data is repeated. , And image data of the second hierarchy. 19. A transmission method for transmitting encoded data obtained by hierarchical encoding, wherein the encoded data is a second layer of image data having a smaller number of pixels than the first layer of image data. And correcting the second layer image data, outputting correction data, predicting a predicted value of the first layer image data from the correction data, For the data, calculating the prediction error of the prediction value, based on the prediction error for the first hierarchy, repeatedly determining the correctness of the correction data, the correct correction data, the A transmission method characterized in that the transmission method includes image data of two layers. 20. A recording medium on which encoded data obtained by hierarchical encoding is recorded, wherein the encoded data of a second layer having a smaller number of pixels than image data of a first layer. Forming image data, correcting the second layer image data, outputting correction data, predicting a predicted value of the first layer image data from the correction data, For the image data of, the prediction error of the prediction value is calculated, and based on the prediction error for the first layer, the determination of the correctness of the correction data is repeated. A recording medium comprising the image data as the second layer of image data. 21. An image encoding apparatus that hierarchically encodes an image, comprising: a classifying unit that classifies pixels constituting an image of a first hierarchy into a predetermined class according to a property thereof; A mapping coefficient storage unit that stores a predetermined mapping coefficient, and a target pixel of interest in the image of the first hierarchy, and the mapping coefficient corresponding to the class of the target pixel. An image encoding apparatus comprising: an arithmetic unit that calculates a second hierarchical image data in which the number of pixels of the first hierarchical image data is reduced by performing a predetermined arithmetic operation. 22. The arithmetic unit performs a predetermined arithmetic operation using a plurality of pixels of the image of the first hierarchy including the pixel of interest and the mapping coefficient corresponding to the class of the pixel of interest. The image encoding device according to claim 21, wherein: 23. The image coding apparatus according to claim 21, wherein the mapping coefficient is generated by performing learning using image data for learning. 24. The mapping coefficient of a prediction result of predicting an image of a first layer from an image of the second layer,
22. The image encoding apparatus according to claim 21, wherein the image encoding apparatus is obtained by performing learning so that a prediction error with respect to the image of the first hierarchy is minimized. 25. The mapping coefficient such that a prediction error of a prediction result of predicting the image of the first layer from the image of the second layer with respect to the image of the first layer is equal to or less than a predetermined value. 22. The image encoding apparatus according to claim 21, wherein the image encoding apparatus is obtained by performing learning. 26. The mapping coefficient, comprising: classifying pixels constituting an image of a first layer for learning into one of the classes according to a property thereof; By performing a predetermined operation using an attention learning pixel of interest in the image and a predetermined coefficient corresponding to the class of the attention learning pixel, the pixel of the first layer image for learning is obtained. Calculating a second-layer image data for learning with a reduced number, predicting a predicted value of the first-layer image for learning based on the second-layer image for learning; Calculating a prediction error of a prediction value of the learning first layer image with respect to the learning first layer image, and changing the predetermined coefficient based on the prediction error; Obtained by repeating until the coefficient of The image coding apparatus according to claim 21, characterized in that said predetermined factor of the optimum value. 27. The learning apparatus according to claim 27, wherein the mapping coefficient forms an image of a second layer for learning having a smaller number of pixels than an image of the first layer for learning, and corrects the image of the second layer for learning. Outputting learning correction data, predicting an image of the learning first layer based on the learning correction data, and outputting a predicted value thereof; and learning the first layer of learning. Calculating the prediction error of the predicted value of the image of the first layer for learning with respect to the image of (i), and determining the appropriateness of the learning correction data based on the prediction error; Is obtained by using the corrected learning correction data obtained by repeating the process until it becomes proper, and the image of the first layer for learning. Item 22. The image encoding device according to Item 21. 28. An image encoding method for hierarchically encoding an image, comprising: classifying pixels constituting an image of a first hierarchy into a predetermined class according to a property thereof; From the mapping coefficient storage unit that stores the mapping coefficient, the mapping coefficient of the class corresponding to the target pixel of interest in the image of the first hierarchy is read, and the mapping coefficient and the pixel of interest are read. And performing a predetermined operation to calculate image data of a second layer in which the number of pixels of the image data of the first layer is reduced. 29. An image decoding device for decoding encoded data obtained by hierarchical encoding, comprising: a receiving unit for receiving the encoded data; a decoding unit for decoding the encoded data; Wherein the encoded data classifies the pixels constituting the image of the first layer into one of first classes according to their properties, and a predetermined mapping is performed for each of the first classes. From the mapping coefficient storage means storing coefficients, the first class of mapping coefficients corresponding to the target pixel of interest in the image of the first hierarchy is read, and the mapping coefficient and the target pixel are read out. And an image decoding apparatus including a second layer image obtained by performing a predetermined operation using the first layer image and having a reduced number of pixels of the first layer image. 30. The decoding means stores, for each second class, a prediction coefficient for calculating a prediction value of the image of the first layer by a linear combination with the image of the second layer. Prediction coefficient storage means, classification means for classifying the image of the second hierarchy into one of the second classes according to the property thereof, and second classification of the image of the second hierarchy. Prediction value calculation means for reading out the prediction coefficient for the class of (i) from the prediction coefficient storage means, and calculating a prediction value of the image of the first hierarchy from the prediction coefficient and the image of the second hierarchy. 30. The image decoding apparatus according to claim 29, wherein: 31. The image decoding apparatus according to claim 30, wherein the prediction coefficient is generated by performing learning using a learning image. 32. The image decoding apparatus according to claim 29, wherein the mapping coefficient is generated by performing learning using a learning image. 33. The mapping coefficient is learned so that a prediction error of a prediction result of the image of the first layer from an image of the second layer with respect to the image of the second layer is minimized. 30. The image decoding device according to claim 29, wherein the image decoding device is obtained by performing the operation. 34. The mapping coefficient such that a prediction error of a prediction result of predicting the image of the first layer from the image of the second layer with respect to the image of the first layer is equal to or less than a predetermined value. 30. The image decoding apparatus according to claim 29, wherein the image decoding apparatus is obtained by performing learning. 35. An image decoding method for decoding coded data obtained by hierarchical coding, wherein the coded data sets pixels constituting an image of a first layer in accordance with the property thereof. Classifying into a predetermined class, and for each of the predetermined classes, a class corresponding to a target pixel of interest in the image of the first hierarchy is stored in a mapping coefficient storage unit that stores a predetermined mapping coefficient. A second hierarchical image obtained by reading out the mapping coefficient of the above and performing a predetermined operation using the mapping coefficient and the target pixel, wherein the number of pixels of the first hierarchical image is reduced. An image decoding method characterized by the following. 36. A transmission method for transmitting coded data obtained by hierarchical coding, wherein the coded data is obtained by classifying pixels constituting an image of a first hierarchy into a predetermined class according to the property thereof. From the mapping coefficient storage means storing a predetermined mapping coefficient for each of the predetermined classes, from the mapping of the class corresponding to the target pixel of interest in the image of the first hierarchy. A second layer image obtained by reading a coefficient, performing a predetermined operation using the mapping coefficient and the target pixel, and reducing the number of pixels of the first layer image. Transmission method to do. 37. A recording medium on which encoded data obtained by hierarchical encoding is recorded, wherein the encoded data includes a pixel that forms an image of a first layer, and a predetermined number of pixels corresponding to a property of the pixel. From the mapping coefficient storing means storing a predetermined mapping coefficient for each of the predetermined classes, from a class corresponding to a target pixel of interest in the image of the first hierarchy. Including a second layer image obtained by reading out the mapping coefficient and performing a predetermined operation using the mapping coefficient and the target pixel, wherein the second layer image has a reduced number of pixels. Characteristic recording medium.