JP4807349B2 - Learning apparatus and method - Google Patents

Description

The present invention relates to a learning device and Methods, in particular, efficiently and accurately, to the learning apparatus and Methods were to be able to make predictions.

  Conventionally, an image having a low spatial resolution is converted into an image having a higher spatial resolution and displayed. In this case, more pixel data is interpolated (generated) from pixel data having a low spatial resolution.

  Conventionally, in order to perform such interpolation processing, when pixel data with low spatial resolution is composed of, for example, R, G, and B component signals, the interpolation processing is performed independently for each component signal. It was.

  That is, R pixel data having a high spatial resolution is generated from R pixel data having a low spatial resolution, and G pixel data having a high spatial resolution is generated from G pixel data having a low spatial resolution. The B pixel data is generated from the B pixel data having a low spatial resolution.

  As a result, not only the efficiency is deteriorated, but it is difficult to obtain good accuracy.

  The present invention has been made in view of such a situation, and is intended to improve efficiency and accuracy.

The learning device according to claim 1, a component signal that constitutes the second pixel data, and a plurality of types of component signals that constitute each of the plurality of first pixel data located in the vicinity of the second pixel data, and It is characterized by comprising a calculation means for performing a calculation for obtaining a prediction coefficient that minimizes an error from a predicted value of a component signal constituting the second pixel data predicted using the prediction coefficient.

The learning method according to claim 3, wherein a component signal constituting the second pixel data and a plurality of types of component signals constituting each of the plurality of first pixel data located in the vicinity of the second pixel data, and A calculation step is provided for performing a calculation for obtaining a prediction coefficient that minimizes an error from a predicted value of a component signal that constitutes the second pixel data predicted using the prediction coefficient .

The learning device according to claim 1 and the learning method according to claim 3, wherein the component signal constituting the second pixel data and a plurality of types of first pixels located in the vicinity of the second pixel data. By performing an operation for obtaining a prediction coefficient that minimizes an error from a predicted value of a component signal that constitutes second pixel data that is predicted using a plurality of component signals and prediction coefficients that constitute each of the data, A prediction coefficient is determined .

  According to the present invention, it is possible to efficiently perform prediction processing with high accuracy.

  FIG. 1 shows a configuration example of a system in which image data is thinned and transmitted from a transmission side, and thinned pixels are generated and reproduced on a reception side. Digital video data to be transmitted is input from the input terminal 201 of the transmission apparatus 1 to the sub-sampling circuit 202, and every other pixel data is thinned out in the horizontal direction so that the amount of data to be transmitted is halved. Yes. The encoder 203 performs high-efficiency encoding on the data supplied from the sub-sampling circuit 202 using, for example, an orthogonal transform code such as DCT (Discrete Cosine Transform) or ADRC (Adaptive Dynamic Range Coding) to further reduce the data amount. Has been made. The transmission processing circuit 204 performs processing such as error correction encoding, framing, and channel encoding on the output of the encoder 203, and outputs the output from the output terminal 205 to the transmission path 3 or records on an optical disk, magnetic disk, or the like. Recorded on the medium 2.

  The data supplied from the transmission path 3 or the recording medium 2 is input to the reception processing circuit 212 from the input terminal 211 of the receiving device 4 so that channel encoding decoding processing, frame decomposition processing, error correction processing, and the like are performed. Has been made. The decoder 213 performs a decoding process corresponding to the encoder 203 on the transmission device 1 side. The output of the decoder 213 is supplied to the synchronization circuit 215 and the synthesis circuit 214.

  The synchronization circuit 215 adjusts the output of the decoder 213 so that pixel data to be processed is generated at the same timing, and outputs the adjusted data to the ADRC processing circuit 216 and the data generation circuit 219. . The ADRC processing circuit 216 performs ADRC processing on the data supplied from the synchronization circuit 215 with one bit, and outputs the processing result to the class classification circuit 217. The class classification circuit 217 performs class classification processing corresponding to the data supplied from the ADRC processing circuit 216 and outputs a signal indicating the classified class to a ROM (Read Only Memory) 218 as an address.

  The ROM 218 reads the coefficient data stored at the address corresponding to the class supplied from the class classification circuit 217 and outputs the coefficient data to the data generation circuit 219. The data generation circuit 219 multiplies the data supplied from the synchronization circuit 215 by the coefficient data supplied from the ROM 218 to generate new pixel data, which is output to the synthesis circuit 214. The combining circuit 214 combines the originally existing pixel data supplied from the decoder 213 and the pixel data generated by the data generation circuit 219, and outputs them to the CRT (not shown) from the output terminal 220 for display. Has been made.

  Next, the operation will be described. For example, as shown in FIG. 2, the digital image data input from the input terminal 201 is thinned out by the sub-sampling circuit 202 every other horizontal direction. In FIG. 2, ◯ represents pixel data that has been thinned out and X represents pixel data that has been thinned and is not transmitted. Thereby, the pixel data to be transmitted is halved.

  The pixel data is encoded by the encoder 203, subjected to predetermined processing by the transmission processing circuit 204, and transmitted from the output terminal 205 to the transmission path 3 or the recording medium 2.

  The reception processing circuit 212 receives transmission data from the transmission path 3 or the recording medium 2 from the input terminal 211 and outputs it to the decoder 213. The decoder 213 decodes the input data, and outputs pixel data (pixel data indicated by ◯ in FIG. 2) obtained as a result of the decoding to the synthesizing circuit 214 and the synchronization circuit 215.

The synchronization circuit 215 performs a process of applying a predetermined delay so that pixel data to be processed is generated at the same timing. Thereby, for example, the pixel data X _{1 to} X ₄ located above (X ₁ ), left (X ₂ ), right (X ₃ ) and below (X ₄ ) of the omitted pixel Y ₁ shown in FIG. Are supplied to the ADRC processing circuit 216 and the data generation circuit 219 at the same timing.

The ADRC processing circuit 216 executes ADRC processing of one block composed of the input four pieces of pixel data X _{1 to} X ₄ . In this embodiment, as shown in FIG. 3, each pixel data X is composed of vectors (X _R , X _G , X _B ) on a color space defined by R, G, B components. Yes. X _R , X _G , and X _B represent the component components of R, G, and B of the pixel data X, respectively, for example, each represented by 8 bits. Since the ADRC processing circuit 216 executes 1-bit ADRC processing, for example, the R component X _R1 of the pixel data X ₁ is represented by 1 bit, the G component X _G1 is represented by 1 bit, and the B component X _B1 is represented by 1 bit. Represented by That is, the pixel data X ₁ originally represented by 24 bits (= 3 × 8) is assumed to be 3 bits (= 3 × 1). The other pixel data X _{2 to} X ₄ are similarly converted into 3-bit pixel data, and each of them is classified into pixel data (X ₁ , X ₂ , X ₃ , X ₄ ) represented by 3 bits. To be supplied.

The class classification circuit 217 classifies the input data of a total of 12 bits (= 4 × 3) into a class, generates a class data signal representing the classified class, and outputs it to the ROM 218. That is, in this embodiment, the class is represented by 12 bits, so that there are 4096 (= 2 ¹² ) classes.

  The ROM 218 stores a prediction coefficient w for each class. When a signal representing a predetermined class is supplied from the class classification circuit 217, the prediction coefficient w stored in an address corresponding to the class is read. And supplied to the data generation circuit 219.

The data generation circuit 219 uses the prediction coefficient w supplied from the ROM 218 and the pixel data X _{1 to} X ₄ supplied from the synchronization circuit 215 to perform an operation as shown in the following equation, as shown in FIG. Pixel data Y ₁ is generated.
Y _R1 = w ₁ (R) X _R1 + w ₂ (R) X _G1 + w ₃ (R) X _B1
+ W ₄ (R) X _R2 + w ₅ (R) X _G2 + w ₆ (R) X _B2
+ W ₇ (R) X _R3 + w ₈ (R) X _G3 + w ₉ (R) X _B3
+ W ₁₀ (R) X _R4 + w ₁₁ (R) X _G4 + w ₁₂ (R) X _B4

Y _G1 = w ₁ (G) X _R1 + w ₂ (G) X _G1 + w ₃ (G) X _B1
+ W ₄ (G) X _R2 + w ₅ (G) X _G2 + w ₆ (G) X _B2
+ W ₇ (G) X _R3 + w ₈ (G) X _G3 + w ₉ (G) X _B3
+ W ₁₀ (G) X _R4 + w ₁₁ (G) X _G4 + w ₁₂ (G) X _B4

Y _B1 = w ₁ (B) X _R1 + w ₂ (B) X _G1 + w ₃ (B) X _B1
+ W ₄ (B) X _R2 + w ₅ (B) X _G2 + w ₆ (B) X _B2
+ W ₇ (B) X _R3 + w ₈ (B) X _G3 + w ₉ (B) X _B3
+ W ₁₀ (B) X _R4 + w ₁₁ (B) X _G4 + w ₁₂ (B) X _B4

Note that w _i (R), w _i (G), and w _i (B) represent prediction coefficients for R, G, and B, respectively.

As apparent from the above equation, in this embodiment, the R component Y _R1 of pixel data Y ₁ is not only the R component X _R1 to X _R4 of surrounding pixels X ₁ to X _4, or G component X _G1 X _G4 and B components X _{B1 to} X _{B4 are} generated. Similarly, the G component Y _G1 and the B component Y _B1 of the pixel data Y ₁ are not only the corresponding components of the pixel data X _{1 to} X ₄ but also all the components X _{R1 to} X _R4 , X _{G1 to} X _G4. , X _{B1 to} X _B4 .

  An image, particularly a natural image taken using a television camera, has a correlation, and a relatively close pixel has a stronger correlation. Therefore, when new pixel data is generated by calculation, it is possible to generate new pixel data more efficiently and with high accuracy by performing the calculation based on pixel data in the vicinity. .

That is, as shown in the above formula, for example, to determine the R component Y _R1 of pixel data Y _1, although the uses a total of 12 pieces of data X _R1 to X _B4, for example, the Y _R1 In order to use only 12 R components for calculation, since there is only one R component for each pixel, it is necessary to use R components of a total of 12 pixels after all. In this way, it is inevitably necessary to use pixel data that is farther away from the pixel Y _{1 of} interest, and the efficiency and accuracy deteriorate.

  Therefore, if the R component (the G component and the B component are the same) of the target pixel is generated using the R, G, and B components of each pixel as in this embodiment, the pixels at closer positions The required number of data can be obtained from the data. Therefore, highly accurate pixel data can be generated efficiently.

  The synthesis circuit 214 synthesizes the new pixel data Y generated by the data generation circuit 219 and the originally existing pixel data X supplied from the decoder 213 as described above, and outputs them from the output terminal 220. Accordingly, the pixel data output from the output terminal 220 is an image having a higher spatial resolution than the image constituted by the pixel data X received by the receiving circuit 212 (subsampled by the subsampling circuit 202 in FIG. 1). The image has the same resolution as the previous image).

  The ROM 218 stores the prediction coefficient w of the above-described formula. The table of the prediction coefficient w can be obtained from the apparatus shown in FIG. 4, for example.

That is, in the embodiment of FIG. 4, a digital video signal is input from the input terminal 231 and supplied to the synchronization circuit 232. The digital video signal input to the input terminal 231 is preferably a standard signal necessary for creating a table (thus, a high-resolution image signal before being thinned). A signal composed of a still image of a picture can be employed. Similar to the synchronization circuit 215 in FIG. 1, the synchronization circuit 232 performs timing adjustment so that the pixel data X _{1 to} X ₄ shown in FIG. 2 are output simultaneously. Pixel data output from the synchronization circuit 232 is supplied to the ADRC processing circuit 233 and the data memory 237.

  The ADRC processing circuit 233 performs ADRC processing on the input pixel data with 1 bit, and outputs the processed pixel data to the class classification circuit 234. The class classification circuit 234 classifies the data input from the ADRC processing circuit 233 and supplies a signal corresponding to the classified class to the data memory 237 via the contact A of the switch 235 as an address. That is, the synchronization circuit 232, the ADRC processing circuit 233, and the class classification circuit 234 perform the same processing as in the synchronization circuit 215, the ADRC processing circuit 216, and the class classification circuit 217 in FIG.

  The counter 236 counts the clock CK supplied from a circuit (not shown), and supplies the count value as an address to the data memory 237 via the contact C of the switch 235.

When the address is supplied from the class classification circuit 234 via the switch 235, the data memory 237 writes the data supplied from the synchronization circuit 232 to the address, and the address is supplied from the counter 236 via the switch 235. Data stored in the address is read out and output to the least squares method arithmetic circuit 238. The least square method arithmetic circuit 238 performs an operation based on the least square method on the pixel data supplied from the data memory 237, calculates a prediction coefficient w _i , and outputs it to the memory 239. The memory 239 writes the prediction coefficient w _i supplied from the least squares method arithmetic circuit 238 to the address supplied from the counter 236 via the switch 235.

  Next, the operation will be described. Digital video data for learning for determining a prediction coefficient is synchronized in the synchronization circuit 232, 1-bit ADRC processing is performed in the ADRC processing circuit 233, and then input to the class classification circuit 234. Is done. In this case, as in the case of FIG. 1, four pixels are set as one block for class classification, and each pixel is subjected to ADRC processing with 1 bit for each R, G, B component in the ADRC processing circuit 233. Therefore, 12-bit class data is supplied from the class classification circuit 234 to the data memory 237 as an address via the contact A of the switch 235. The data memory 237 stores the pixel data supplied from the synchronization circuit 232 at this address.

Here, the pixel data to be stored is pixel data of an image having a higher spatial resolution before being subsampled by the subsampling circuit 202 in FIG. Accordingly, not only the pixel data X _i marked with ○ in FIG. 2, but also the image data Y _i indicated with x is stored.

As shown in the above equation, there are _twelve coefficients w ₁ (R) to w ₁₂ (R) for calculating image data of one component, for example, Y _R1 . Therefore, in order to obtain these 12 prediction coefficients, 12 simultaneous equations having 12 prediction coefficients as unknowns are required in each class. The data memory 237 stores at least the number of pixel data necessary to solve the simultaneous equations.

After the necessary number of pixel data is stored in the data memory 237, the switch 235 is switched to the contact C side. Since the counter 236 counts the clock CK and outputs the count value, a value incremented by 1 is input to the data memory 237 as a read address. The data memory 237 reads out pixel data corresponding to the input read address and outputs the pixel data to the least squares operation circuit 238. The least square method arithmetic circuit 238 applies specific data to the above-described equation, generates a simultaneous equation having the prediction coefficient w _i as a variable, solves the simultaneous equation, and obtains the prediction coefficient w _i .

Then, predetermined pixel data (for example, the R component Y _R1 of the pixel data Y ₁ described above) is obtained (predicted) using the prediction coefficient w _i obtained by the calculation. Then, the error between the value of Y _R1 obtained by calculation (prediction) and the actual pixel data Y _R1 is calculated, and the prediction coefficient w _i is calculated so that the square of the error is minimized. The prediction coefficient w _i obtained by the calculation is written in the address of the memory 239 corresponding to the address of the pixel data read from the data memory 237 now. In this way, the prediction coefficient w _i is stored in the memory 239. The stored contents are written in the ROM 218 shown in FIG.

In the above embodiment, the prediction coefficient w _i is written in the ROM 218 (memory 239), but it is also possible to write the data itself after being multiplied by the coefficient. In this way, the data generation circuit 219 in FIG. 1 becomes unnecessary.

  FIG. 5 shows another configuration example of the transmission device 1.

  The I / F (InterFace) 11 performs reception processing of image data supplied from the outside and transmission processing of encoded data to the transmitter / recording device 16. A ROM (Read Only Memory) 12 stores a program for IPL (Initial Program Loading) and others. A RAM (Random Access Memory) 13 stores system programs (OS (Operating System)) and application programs recorded in the external storage device 15, and data necessary for the operation of a CPU (Central Processing Unit) 14. It is made to memorize. In accordance with the IPL program stored in the ROM 12, the CPU 14 expands the system program and application program from the external storage device 15 to the RAM 13, and executes the application program under the control of the system program. An encoding process as described later is performed on the supplied image data. The external storage device 15 is, for example, a magnetic disk device or the like, and stores system programs and application programs executed by the CPU 14 as well as data necessary for the operation of the CPU 14 as described above. The transmitter / recording device 16 records the encoded data supplied from the I / F 11 on the recording medium 2 or transmits it via the transmission path 3.

  The I / F 11, the ROM 12, the RAM 13, the CPU 14, and the external storage device 15 are connected to each other via a bus.

  In the transmission apparatus 1 configured as described above, when image data is supplied to the I / F 11, the image data is supplied to the CPU 14. The CPU 14 encodes the image data and supplies the encoded data obtained as a result to the I / F 11. When the encoded data is received, the I / F 11 supplies the encoded data to the transmitter / recording device 16. In the transmitter / recording device 16, the encoded data from the I / F 11 is recorded on the recording medium 2 or transmitted via the transmission path 3.

  FIG. 6 is a functional block diagram of a portion excluding the transmitter / recording device 16 of the transmission device 1 of FIG.

  The image data to be encoded is supplied to the compression unit 21, the local decoding unit 22, and the error calculation unit 23. The compression unit 21 compresses the image data by simply thinning out the pixels, and corrects the resulting compressed data (image data after the thinning is performed) according to the control from the determination unit 24. Has been made. Correction data obtained as a result of correction in the compression unit 21 is supplied to the local decoding unit 22 and the determination unit 24.

  The local decoding unit 22 predicts the original image based on the correction data from the compression unit 21 and supplies the predicted value to the error calculation unit 23. As will be described later, the local decoding unit 22 performs an adaptive process for obtaining a prediction coefficient for calculating a prediction value by linear combination with correction data, and obtains a prediction value based on the prediction coefficient. As described above, the prediction value is supplied to the error calculation unit 23, and the prediction coefficient obtained at that time is supplied to the determination unit 24.

  The error calculation unit 23 is configured to calculate the prediction error of the prediction value from the local decoding unit 22 with respect to the original image data (original image) input thereto. This prediction error is supplied to the determination unit 24 as error information.

  Based on the error information from the error calculation unit 23, the determination unit 24 determines the appropriateness of using the correction data output from the compression unit 21 as the encoding result of the original image. If the determination unit 24 determines that the correction data output from the compression unit 21 is not appropriate as the encoding result of the original image, the determination unit 24 controls the compression unit 21 and further corrects the compressed data. Thus, new correction data obtained as a result is output. In addition, when the determination unit 24 determines that the correction data output from the compression unit 21 is appropriate as the encoding result of the original image, the determination unit 24 uses the correction data supplied from the compression unit 21 as the optimum result. The compressed data (hereinafter referred to as optimum compressed data as appropriate) is supplied to the multiplexing unit 25 and the prediction coefficient supplied from the local decoding unit 22 is supplied to the multiplexing unit 25.

  The multiplexing unit 25 multiplexes the optimum compressed data (correction data) from the determination unit 24 and the prediction coefficient, and supplies the multiplexed result to the transmitter / recording device 16 (FIG. 5) as encoded data. It is made to do.

  Next, the operation will be described with reference to the flowchart of FIG. When image data is supplied to the compression unit 21, the compression unit 21 compresses the image data by thinning out in step S <b> 1. At first, without performing correction, the local decoding unit 22 and the determination unit are compressed. 24. In the local decoding unit 22, in step S2, the correction data from the compression unit 21 (initially, the compressed data obtained by simply thinning out the image data as described above) is locally decoded.

  That is, in step S2, an adaptive process for obtaining a prediction coefficient for calculating the predicted value of the original image is performed by linear combination with the correction data from the compression unit 21, and the predicted value is calculated based on the predicted coefficient. Desired. The prediction value obtained in the local decoding unit 22 is supplied to the error calculation unit 23, and the prediction coefficient is supplied to the determination unit 24.

  Here, the image composed of the prediction values output from the local decoding unit 22 is the same as the decoded image obtained on the receiving device 4 side.

  When receiving the predicted value of the original image from the local decoding unit 22, the error calculating unit 23 calculates the prediction error of the predicted value from the local decoding unit 22 with respect to the original image data in step S3, and as error information , Supplied to the determination unit 24. When the determination unit 24 receives the error information from the error calculation unit 23, in step S4, based on the error information, the appropriateness of using the correction data output from the compression unit 21 as the encoding result of the original image. Determine.

  That is, in step S4, it is determined whether the error information is equal to or less than a predetermined threshold value ε. If it is determined in step S4 that the error information is not equal to or less than the predetermined threshold ε, it is recognized that the correction data output from the compression unit 21 is not appropriate to be the encoded data of the original image, and the process proceeds to step S5. The determination unit 24 controls the compression unit 21 and thereby corrects the compressed data. The compression unit 21 changes the correction amount (correction value Δ described later) under the control of the determination unit 24 to correct the compressed data, and outputs the correction data obtained as a result to the local decoding unit 22 and the determination unit 24. . And it returns to step S2 and the same process is repeated hereafter.

  On the other hand, if it is determined in step S4 that the error information is equal to or less than the predetermined threshold ε, it is recognized that the correction data output from the compression unit 21 is appropriate to be the encoding result of the original image. The determination unit 24 outputs the correction data when error information equal to or less than the predetermined threshold ε is obtained to the multiplexing unit 25 as the optimal compression data together with the prediction coefficient. In step S6, the multiplexing unit 25 multiplexes the optimum compressed data and the prediction coefficient from the determination unit 24, outputs the encoded data obtained as a result, and ends the process.

  As described above, the correction data obtained by correcting the compressed data when the error information is equal to or less than the predetermined threshold value ε is used as the encoding result of the original image. Based on the correction data, it is possible to obtain an image substantially identical to the original image (original image).

  Next, FIG. 8 shows a configuration example of the compression unit 21 of FIG.

  The image data to be encoded is input to the thinning circuit 31. The thinning circuit 31 thins the input image data to 1 / N (in this case, 1/2). . Accordingly, the thinning circuit 31 outputs compressed data obtained by compressing image data to 1 / N. This compressed data is supplied from the thinning circuit 31 to the correction circuit 32.

  The correction circuit 32 gives an address to the correction value ROM 33 in accordance with a control signal from the determination unit 24 (FIG. 6), thereby reading the correction value Δ. Then, the correction circuit 32 generates correction data by adding, for example, the correction value Δ from the correction value ROM 33 to the compressed data from the thinning circuit 31 and supplies the correction data to the local decoding unit 22 and the determination unit 24. It is made to do. The correction value ROM 33 stores various combinations of correction values Δ (for example, combinations of correction values for correcting compressed data for one frame) for correcting the compressed data output by the thinning circuit 31. The combination of correction values Δ corresponding to the addresses supplied from the correction circuit 32 is read out and supplied to the correction circuit 32.

  Next, the processing of the compression unit 21 in FIG. 8 will be described with reference to FIG.

  For example, when image data for one frame (field) or the like is supplied to the thinning circuit 31, the thinning circuit 31 thins the image data to 1 / N in step S11, and the compressed data obtained as a result is Output to the correction circuit 32.

  Here, as shown in FIG. 2, the thinning circuit 31 thins out the image data by 1/2 for each line, for example. Note that the thinning circuit 31 performs the above-described processing, for example, in units of one frame (field). Therefore, one frame of image data is supplied from the thinning circuit 31 to the correction circuit 32 as compressed data that has been thinned by half. However, the thinning process in the thinning circuit 31 can also be performed in units of blocks by dividing an image of one frame into several blocks.

  When receiving the compressed data from the thinning circuit 31, the correction circuit 32 determines whether or not a control signal has been received from the determination unit 24 (FIG. 6) in step S12. If it is determined in step S12 that the control signal has not been received, the process proceeds to step S15, and the correction circuit 32 directly uses the compressed data from the thinning circuit 31 as correction data to the local decoding unit 22 and the determination unit 24. Output, and return to step S12.

  That is, as described above, the determination unit 24 is configured to control the compression unit 21 (correction circuit 32) based on the error information, and immediately after the compressed data is output from the thinning circuit 31, Since error information is not obtained (because error information is not output from the error calculation unit 23), no control signal is output from the determination unit 24. For this reason, immediately after the compressed data is output from the thinning circuit 31, the correction circuit 32 does not correct the compressed data (corrected by adding 0), and directly uses the local decoding unit 22 and the determination as the corrected data. To the unit 24.

  On the other hand, when it is determined in step S12 that the control signal from the determination unit 24 has been received, in step S13, the correction circuit 32 outputs an address according to the control signal to the correction value ROM 33. As a result, in step S 13, a combination (set) of correction values Δ for correcting the compressed data for one frame stored in the address is read from the correction value ROM 33 and supplied to the correction circuit 32. The When the correction circuit 32 receives the combination of the correction values Δ from the correction value ROM 33, in step S14, the correction circuit 32 adds the corresponding correction value Δ to each compressed data of one frame, thereby correcting the corrected data by correcting the compressed data. calculate. Thereafter, the process proceeds to step S15, and the correction data is output from the correction circuit 32 to the local decoding unit 22 and the determination unit 24, and the process returns to step S12.

  As described above, the compression unit 21 repeatedly outputs correction data obtained by correcting the compressed data into various values under the control of the determination unit 24.

  When the determination unit 24 finishes encoding the image of one frame, the determination unit 24 supplies a control signal indicating that to the compression unit 21, and the compression unit 21 receives the control signal. The processing according to the flowchart of FIG. 9 is performed on the image of the next frame.

  In the above case, the thinning circuit 31 is made to generate the compressed data by extracting the pixel data (pixel value) at a rate of one for every two pixels. It is also possible to calculate the average value of the pixels and generate the compressed data using the average value as the pixel value of the center pixel of 3 × 3 pixels.

  Next, FIG. 10 shows a configuration example of the local decoding unit 22 of FIG.

  The correction data from the compression unit 21 is supplied to the class classification blocking circuit 41 and the predicted value calculation blocking circuit 42. The class classification blocking circuit 41 is configured to block the correction data into class classification blocks which are units for classifying the correction data into a predetermined class according to the property.

That is, the class classification blocking circuit 41 is configured to constitute a class classification block composed of a total of four pixels, ie, four pixels X ₁ , X ₂ , X ₃ , and X ₄ shown in FIG. Yes. This class classification block is supplied to the class classification adaptive processing circuit 43.

  In this case, the class classification block is configured by a cross-shaped block of 4 pixels. However, the class classification block may have other shapes such as a rectangle, a square, and other arbitrary shapes. Is possible. Further, the number of pixels constituting the class classification block is not limited to four pixels.

The predicted value calculation blocking circuit 42 is configured to block the correction data into predicted value calculation blocks that are units for calculating the predicted value of the original image. In this embodiment, the block is the same as the block for class classification, and the block is constituted by the pixel data X _{1 to} X ₄ in FIG.

  Thus, in the case of this embodiment, since the predicted value calculation blocking circuit 42 blocks the same range as the class classification blocking circuit 41, both may be shared.

  The predicted value calculation block obtained in the predicted value calculation blocking circuit 42 is supplied to the class classification adaptive processing circuit 43.

  Note that the number of pixels and the shape of the prediction value calculation block are not limited to those described above, as in the case of the class classification block. However, it is desirable that the number of pixels constituting the prediction value calculation block is equal to or greater than the number of pixels constituting the class classification block.

  Further, when the above-described blocking is performed (the same applies to processes other than blocking), there may be no corresponding pixel near the image frame of the image. In this case, for example, the image frame The processing is performed on the assumption that the same pixel as that constituting the pixel exists outside.

  The class classification adaptive processing circuit 43 includes an ADRC (Adaptive Dynamic Range Coding) processing circuit, a class classification circuit 45, and an adaptive processing circuit 46, and performs class classification adaptive processing.

  Class classification adaptation processing classifies input signals into several classes based on their characteristics, and applies appropriate adaptation processing to the input signals of each class. It is divided into processing.

  Here, the class classification process and the adaptation process will be briefly described.

  First, the class classification process will be described.

Now, for example, as shown in FIG. 11A, a block (class classification block) composed of 2 × 2 pixels is configured by a certain target pixel and three pixels adjacent to the target pixel. It is expressed by 1 bit (takes a level of 0 or 1). In this case, the 2 × 2 4-pixel block can be classified into 16 (= (2 ¹ ) ⁴ ) patterns as shown in FIG. 11B according to the level distribution of each pixel. Such pattern division is class classification processing and is performed in the class classification circuit 45.

  The class classification processing can be performed in consideration of the activity (complexity of the image) (severity of change) of the image (image in the block).

Here, normally, for example, about 8 bits are assigned to each pixel. In this embodiment, as described above, the class classification block is composed of 9 pixels of 3 × 3. Therefore, if the class classification process is performed on such a class classification block, it is classified into an enormous number of classes of (2 ⁸ ) ⁹ .

  Therefore, in this embodiment, the ADRC processing circuit 44 performs ADRC processing on the class classification block, thereby reducing the number of bits of the pixels constituting the class classification block. By doing so, the number of classes has been reduced.

  That is, for example, in order to simplify the description, as shown in FIG. 12A, when a block composed of four pixels arranged on a straight line is considered, in ADRC processing, the maximum value of the pixel value is considered. MAX and minimum value MIN are detected. Then, DR = MAX−MIN is set as the local dynamic range of the block, and the pixel values of the pixels constituting the block are requantized to K bits based on the dynamic range DR.

That is, from each pixel value in the block, subtracts the minimum value MIN, dividing the subtracted value by DR / 2 ^K. Then, it is converted into a code (ADRC code) corresponding to the division value obtained as a result. Specifically, for example, when K = 2, as shown in FIG. 12B, which range the division value belongs to is obtained by dividing the dynamic range DR into 4 (= 2 ² ) equally? And the division value belongs to the range of the lowest level, the range of the second level from the bottom, the range of the third level from the bottom, or the range of the highest level, respectively, It is coded into 2 bits such as 00B, 01B, 10B, or 11B (B represents a binary number). On the decoding side, the ADRC code 00B, 01B, 10B, or 11B is the center value L _{00 of} the lowest level range obtained by equally dividing the dynamic range DR into four, and the second level range from the bottom. center value L ₀₁ of the is converted into a center value L ₁₁ of the central value L ₁₀ or most of the upper level range, the range of the third level from the bottom, to the value, decoded by the minimum value MIN is added Is done.

  Here, such ADRC processing is called non-edge matching. For such non-edge matching, as shown in FIG. 12C, the average value MIN ′ of the pixel values belonging to the lowest level range obtained by dividing the dynamic range DR into four equal parts, or the most A level at which the ADRC code 00B or 11B is converted into the average value MAX ′ of the pixel values belonging to the upper level range and the dynamic range DR ′ defined by MAX′−MIN ′ is equally divided (three equal parts). In addition, there is an ADRC process in which the ADRC code is decoded by converting the ADRC codes 01B and 10B, which is called edge matching.

  The details of the ADRC processing are disclosed in, for example, Japanese Patent Application Laid-Open No. 3-53778 filed by the applicant of the present application.

  By applying ADRC processing that performs requantization with a smaller number of bits than the number of bits allocated to the pixels constituting the block, as described above, the number of classes can be reduced. This is performed in the ADRC processing circuit 44.

  In the present embodiment, the class classification circuit 45 performs the class classification process based on the ADRC code output from the ADRC processing circuit 44. The class classification process is performed by other methods such as DPCM (predictive coding). It is also possible to perform the processing on data subjected to BTC (Block Truncation Coding), VQ (Vector Quantization), DCT (Discrete Cosine Transform), Hadamard Transform, or the like.

  Next, the adaptation process will be described.

For example, now, the predicted value E [y] of the pixel value y of the original image is changed to pixel values (hereinafter referred to as learning data) x ₁ , x ₂ ,. Consider a linear primary combination model defined by a linear combination of predetermined prediction coefficients w ₁ , w ₂ ,. In this case, the predicted value E [y] can be expressed by the following equation.

E [y] = w ₁ x ₁ + w ₂ x ₂ +...
... (1)

で定義すると、次のような観測方程式が成立する。 Therefore, in order to generalize, a matrix W composed of a set of prediction coefficients w, a matrix X composed of a set of learning data, and a matrix Y ′ composed of a set of predicted values E [y],

Then, the following observation equation holds.

XW = Y '
... (2)

で定義すると、式（２）から、次のような残差方程式が成立する。 Then, it is considered to apply the least square method to this observation equation to obtain a predicted value E [y] close to the pixel value y of the original image. In this case, a matrix Y consisting of a set of pixel values y of the original image (hereinafter referred to as teacher data as appropriate) y and a set of residuals e of predicted values E [y] for the pixel values y of the original image. E

From the equation (2), the following residual equation is established.

XW = Y + E
... (3)

を最小にすることで求めることができる。 In this case, the prediction coefficient w _i for obtaining the predicted value E [y] close to the pixel value y of the original image is a square error.

Can be obtained by minimizing.

Accordingly, when the above-mentioned square error differentiated by the prediction coefficient w _i becomes 0, that is, the prediction coefficient w _i satisfying the following equation obtains the prediction value E [y] close to the pixel value y of the original image. Therefore, it is an optimum value.

・・・（４）

... (4)

Therefore, first, the following equation is established by differentiating the equation (3) by the prediction coefficient w _i .

・・・（５）

... (5)

  From equations (4) and (5), equation (6) is obtained.

・・・（６）

... (6)

  Further, considering the relationship among the learning data x, the prediction coefficient w, the teacher data y, and the residual e in the residual equation of Equation (3), the following normal equation can be obtained from Equation (6). .

・・・（７）

... (7)

  The normal equation of Expression (7) can be established by the same number as the number of prediction coefficients w to be obtained. Therefore, the optimal prediction coefficient w can be obtained by solving Expression (7). In solving equation (7), for example, a sweep-out method (Gauss-Jordan elimination method) or the like can be applied.

  As described above, the optimum processing is to obtain the optimum prediction coefficient w, and further to obtain the prediction value E [y] close to the pixel value y of the original image by using the prediction coefficient w and the equation (1). The adaptive processing is performed in the adaptive processing circuit 46.

  Note that the adaptive processing is different from the interpolation processing in that a component included in the original image that is not included in the thinned image is reproduced. In other words, the adaptive process is the same as the interpolation process using a so-called interpolation filter as long as only Expression (1) is seen, but the prediction coefficient w corresponding to the tap coefficient of the interpolation filter uses the teacher data y. In other words, since it is obtained by learning, the components included in the original image can be reproduced. From this, it can be said that the adaptive process is a process having an image creating action.

  Next, the processing of the local decoding unit 22 in FIG. 10 will be described with reference to the flowchart in FIG.

  In the local decoding unit 22, first, in step S21, the correction data from the compression unit 21 is blocked. That is, in the class classification blocking circuit 41, the correction data is blocked into a 4-pixel class classification block and supplied to the class classification adaptive processing circuit 43, and the prediction value calculation blocking circuit 42 corrects the correction data. The data is divided into four-pixel prediction value calculation blocks and supplied to the class classification adaptive processing circuit 43.

  As described above, the class classification adaptive processing circuit 43 is supplied with original image data in addition to the class classification block and the prediction value calculation block. The class classification block is the ADRC processing unit 44. The predicted value calculation block and the original image data are supplied to the adaptive processing circuit 46.

  Upon receiving the class classification block, the ADRC processing circuit 44 performs, for example, 1-bit ADRC (ADRC that performs re-quantization by 1 bit) on the class classification block in step S22, thereby The correction data is converted (encoded) into 1 bit and output to the class classification circuit 45. In step S23, the class classification circuit 45 performs class classification processing on the class classification block that has been subjected to ADRC processing, and determines the class to which the class classification block belongs. The class determination result is supplied to the adaptive processing circuit 46 as class information.

In the present embodiment, the class classification process is performed on the class classification block composed of four pixels in which the R, G, and B components have been subjected to the 1-bit ADRC process. The class classification block is classified into any one of 4096 (= (2 ³ ) ⁴ ) classes.

  In step S24, the adaptive processing circuit 46 performs adaptive processing for each class based on the class information from the class classification circuit 45, thereby calculating the prediction coefficient and the predicted value of the original image data. Is done.

  That is, in this embodiment, for example, when attention is paid to a certain pixel, the adaptive processing is performed using a prediction value calculation block including four pixels adjacent to the target pixel.

Specifically, for example, the class classification circuit 45 outputs the class information C for the class classification block composed of the four correction data X ₁ , X ₂ , X ₃ , and X ₄ shown in FIG. In addition, it is assumed that a prediction value calculation block composed of four pixel correction data X ₁ , X ₂ , X ₃ , and X ₄ is output from the prediction value calculation block forming circuit 42 as a prediction value calculation block. First, the correction data constituting the prediction value calculation block is used as learning data, and the correction data Y ₁ in the original image is used as teacher data, and the normal equation shown in Expression (7) is established. .

Further, for other prediction value calculation blocks classified into the class information C, a normal equation is established in the same manner, and the prediction coefficient w ₁ for obtaining the prediction value E [Y _R1 ] of the pixel value Y _R1. When the number of normal equations that can calculate (R) to w ₁₂ (R) is obtained (therefore, until such number of normal equations is obtained, in step S24, the process of generating the normal equations is performed). The prediction coefficients w ₁ (R) to w ₁₂ (R) optimal for obtaining the predicted value E [Y _R1 ] of the pixel value Y _R1 for the class information C by solving the normal equation. Is calculated. Then, a predicted value E [Y _R1 ] is obtained according to the following equation corresponding to equation (1). The same applies to Y _G1 , Y _B1 and the like.

E [Y _R1 ] = w ₁ (R) X _R1 + w ₂ (R) X _G1 + w ₃ (R) X _B1
+ W ₄ (R) X _R2 + w ₅ (R) X _G2 + w ₆ (R) X _B2
+ W ₇ (R) X _R3 + w ₈ (R) X _G3 + w ₉ (R) X _B3
+ W ₁₀ (R) X _R4 + w ₁₁ (R) X _G4 + w ₁₂ (R) X _B4

E [Y _G1 ] = w ₁ (G) X _R1 + w ₂ (G) X _G1 + w ₃ (G) X _B1
+ W ₄ (G) X _R2 + w ₅ (G) X _G2 + w ₆ (G) X _B2
+ W ₇ (G) X _R3 + w ₈ (G) X _G3 + w ₉ (G) X _B3
+ W ₁₀ (G) X _R4 + w ₁₁ (G) X _G4 + w ₁₂ (G) X _B4

E [Y _B1 ] = w ₁ (B) X _R1 + w ₂ (B) X _G1 + w ₃ (B) X _B1
+ W ₄ (B) X _R2 + w ₅ (B) X _G2 + w ₆ (B) X _B2
+ W ₇ (B) X _R3 + w ₈ (B) X _G3 + w ₉ (B) X _B3
+ W ₁₀ (B) X _R4 + w ₁₁ (B) X _G4 + w ₁₂ (B) X _B4

  In step S24, when the prediction coefficients for the R, G, and B components of each pixel are obtained as described above, the prediction values are output to the error calculation unit 23 and the prediction coefficients are output to the determination unit 24. Then, the process returns to step S21, and the same processing is repeated thereafter.

  Next, FIG. 14 shows a configuration example of the error calculation unit 23 of FIG.

The original image data is supplied to the blocking circuit 51, and the blocking circuit 51 blocks the pixel data in units of pixels corresponding to the predicted values output from the local decoding unit 22. Then, the pixel of the block obtained as a result (in this case, this block is composed of one pixel (Y _{1 in} FIG. 2)) is output to the square error calculation circuit 52. As described above, the square error calculation unit 52 is supplied with pixel data from the blocking circuit 51 and is also supplied with pixel data as a predicted value from the local decoding unit 22 to calculate square error. The circuit 52 calculates a square error as a prediction error of the prediction value for the original image and supplies the square error to the integrating unit 55.

  That is, the automatic error calculation circuit 52 is composed of computing units 53 and 54. The computing unit 53 subtracts the corresponding predicted value from each of the blocked image data from the blocking circuit 51 and supplies the subtracted value to the computing unit 54. The computing unit 54 squares the output of the computing unit 53 (difference between the original image data and the predicted value) and supplies the square to the integrating unit 55.

  When receiving the square error from the square error calculation circuit 52, the accumulating unit 55 reads the stored value in the memory 56, adds the stored value and the square error, and repeatedly supplies the stored value to the memory 56 for storage. Thus, an integrated value (error variance) of the square error is obtained. Furthermore, when the integration of the square error for a predetermined amount (for example, for one frame) is completed, the integration unit 55 reads the integration value from the memory 56 and supplies it to the determination unit 24 as error information. ing. The memory 56 is configured to store the output value of the integrating unit 55 while clearing the stored value every time processing for one frame is completed.

  Next, the operation will be described with reference to the flowchart of FIG. In the error calculation unit 23, first, in step S31, the stored value of the memory 56 is cleared to, for example, 0, and the process proceeds to step S32. In the blocking circuit 51, the image data is blocked as described above. The block obtained as a result is supplied to the square error calculation circuit 52. In step S33, the square error calculation circuit 52 calculates a square error between the image data of the original image constituting the block supplied from the blocking circuit 51 and the predicted value supplied from the local decoding unit 22. .

  That is, in step S <b> 33, the computing unit 53 subtracts the corresponding predicted value from each of the blocked image data supplied from the blocking circuit 51, and supplies it to the computing unit 54. Further, in step S <b> 33, the computing unit 54 squares the output of the computing unit 53 and supplies it to the integrating unit 55.

  When the square error is received from the square error calculation circuit 52, the integrating unit 55 reads the stored value of the memory 56 and adds the stored value and the square error in step S34, thereby obtaining an integrated value of the square error. The integrated value of the square error calculated in the integrating unit 55 is supplied to the memory 56 and stored by being overwritten on the previous stored value.

  Then, in step S35, the integrating unit 55 determines whether or not the integration of the square error for one frame, for example, as a predetermined amount has ended. If it is determined in step S35 that the square error accumulation for one frame has not been completed, the process returns to step S32, and the processing from step S32 is repeated again. If it is determined in step S35 that the integration of the square error for one frame has been completed, the process proceeds to step S36, where the integration unit 55 stores the integrated value of the square error for one frame stored in the memory 56. Is output to the determination unit 24 as error information. And it returns to step S31 and repeats the process from step S31 again.

Therefore, in the error calculation unit 23, when the original image data is Y _i and the predicted value is E [Y _i ], the error information Q is calculated by performing an operation according to the following equation. The

Q = (Σ (Y _i ) −E [Y _i ]) ²
However, Σ means summation for one frame.

  Next, FIG. 16 illustrates a configuration example of the determination unit 24 of FIG.

  The prediction coefficient memory 61 is configured to store the prediction coefficient supplied from the local decoding unit 22. The correction data memory 62 stores correction data supplied from the compression unit 21.

  In the correction data memory 62, when the compressed data is newly corrected in the compression unit 21 and new correction data is supplied as a result, the correction data memory 62 stores the correction data already stored (previous correction data). Instead, new correction data is stored. In addition, at the timing when the correction data is updated to a new one in this way, the local decoding unit 22 outputs a new set of prediction coefficients corresponding to the new correction data. Also in 61, when a new prediction coefficient is supplied in this way, the new prediction coefficient is stored in place of the previously stored prediction coefficient (previous prediction coefficient).

  The error information memory 63 is configured to store error information supplied from the error calculation unit 23. The error information memory 63 stores the error information supplied last time from the error calculation unit 23 in addition to the error information supplied this time (even if new error information is supplied, Until new error information is supplied, the already stored error information is held). The error information memory 63 is cleared every time processing for a new frame is started.

  The comparison circuit 64 compares the current error information stored in the error information memory 63 with a predetermined threshold value ε, and further compares the current error information with the previous error information as necessary. Has been made. The comparison result in the comparison circuit 64 is supplied to the control circuit 65.

  Based on the comparison result in the comparison circuit 64, the control circuit 65 determines the appropriateness (optimum) of using the correction data stored in the correction data memory 62 as the encoding result of the original image. In the case of recognition (determination), a control signal for requesting output of new correction data is supplied to the compression unit 21 (correction circuit 32) (FIG. 8). When the control circuit 65 recognizes that it is optimal to use the correction data stored in the correction data memory 62 as the encoding result of the original image, the control circuit 65 predicts the prediction data stored in the prediction coefficient memory 61. The coefficient is read and output to the multiplexing unit 25, and the correction data stored in the correction data memory 62 is read and supplied to the multiplexing unit 25 as optimum compressed data. Further, in this case, the control circuit 65 outputs a control signal indicating that the encoding of one frame image has been completed to the compression unit 21, and as described above, the control circuit 65 causes the compression unit 21 to The process for the frame is started.

  Next, the operation of the determination unit 24 will be described with reference to FIG. In the determination unit 24, first, in step S41, whether or not the error information is received from the error calculation unit 23 is determined by the comparison circuit 64. If it is determined that the error information is not received, the process proceeds to step S41. Return. If it is determined in step S41 that the error information has been received, that is, if the error information is stored in the error information memory 63, the process proceeds to step S42, and the comparison circuit 64 stores the error information in the error information memory 63. The determined error information (current error information) is compared with a predetermined threshold ε to determine which is larger.

  If it is determined in step S42 that the current error information is greater than or equal to the predetermined threshold ε, the previous error information stored in the error information memory 63 is read in the comparison circuit 64. In step S43, the comparison circuit 64 compares the previous error information with the current error information and determines which is larger.

  Here, when the processing for one frame is started and error information is supplied for the first time, the error information memory 63 does not store the previous error information. The processing after step S43 is not performed, and a control signal for controlling the correction circuit 32 (FIG. 8) is output in the control circuit 65 so as to output a predetermined initial address.

  If it is determined in step S43 that the current error information is equal to or less than the previous error information, that is, if the error information is reduced by correcting the compressed data, the process proceeds to step S44, where the control circuit 65 A control signal instructing to change the correction value Δ in the same manner as the previous time is output to the correction circuit 32, and the process returns to step S41. If it is determined in step S43 that the current error information is larger than the previous error information, that is, if the error information has increased by correcting the compressed data, the process proceeds to step S45, where the control circuit 65 Then, a control signal for instructing to change the correction value Δ opposite to the previous time is output to the correction circuit 32, and the process returns to step S41.

  When the error information that has continued to decrease starts to increase at a certain timing, the control circuit 65 sets the correction value Δ to the previous value, for example, at a magnitude that is 1/2. Conversely, a control signal instructing to change is output.

  Then, by repeating the processes of steps S41 to S45, the error information decreases, and when it is determined in step S42 that the current error information is smaller than the predetermined threshold value ε, the process proceeds to step S46, and control is performed. The circuit 65 reads the prediction coefficient stored in the prediction coefficient memory 61, reads the correction data stored in the correction data memory 62, supplies the correction data to the multiplexing unit 25, and ends the process.

  Thereafter, after waiting for the error information for the next frame to be supplied, the processing according to the flowchart shown in FIG. 17 is repeated again.

  The correction circuit 32 can correct the compressed data for all the compressed data of one frame or only a part of the compressed data. In the case of performing correction only for a part of the compressed data, for example, the control circuit 65 can detect pixels having a strong influence on the error information and correct only the compressed data for such pixels. . A pixel having a strong influence on error information can be detected as follows, for example. That is, first, the error information is obtained by performing processing using the compressed data for the pixels remaining after thinning out as they are. Then, a control signal for performing a process of correcting the compressed data for the pixels remaining after the thinning out one by one by the same correction value Δ is output from the control circuit 65 to the correction circuit 32 and obtained as a result. The error information may be compared with the error information obtained when the compressed data is used as it is, and a pixel whose difference is a predetermined value or more may be detected as a pixel having a strong influence on the error information.

  As described above, the correction of the compressed data is repeated until the error information is made smaller (below) than the predetermined threshold ε, and the correction data when the error information becomes smaller than the predetermined threshold ε is the code of the image. Therefore, in the receiving device 4, the same value as that of the original image is obtained from the correction data in which the pixel values of the pixels constituting the thinned image are set to values most suitable for restoring the original image ( It is possible to obtain substantially the same decoded image.

  In addition to being compressed by thinning processing, the image is also compressed by ADRC processing, class classification adaptation processing, and the like, so that encoded data with a very high compression rate can be obtained. Note that the encoding process as described above in the transmission apparatus 1 achieves high-efficiency compression by using so-called organic integration of the compression process by thinning and the class classification adaptation process, From this, it can be said that it is an integrated encoding process.

  Next, FIG. 18 shows still another configuration example of the receiving device 4 of FIG.

  In the receiver / reproducing apparatus 71, the encoded data recorded on the recording medium 2 is reproduced or the encoded data transmitted via the transmission path 3 is received and supplied to the separation unit 72. In the separation unit 72, the encoded data is separated into correction data and a prediction coefficient, the correction data is supplied to the class classification blocking circuit 73 and the prediction value calculation blocking circuit 77, and the prediction coefficient is the prediction circuit 76. To be supplied.

  The class classification blocking circuit 73, the ADRC processing circuit 74, the class classification circuit 75, or the prediction value calculation blocking circuit 77 are the class classification blocking circuit 41, the ADRC processing circuit 44, the class classification circuit 45, FIG. Alternatively, each block is configured in the same manner as each of the predicted value calculation block forming circuits 42. Therefore, in these blocks, the same processing as in FIG. 10 is performed. The prediction value calculation block is output, and the class classification circuit 75 outputs class information. These prediction value calculation blocks and class information are supplied to the prediction circuit 76.

  The prediction circuit 76 uses the prediction coefficient corresponding to the class information and the correction data constituting the prediction value calculation block supplied from the prediction value calculation blocking circuit 77 to calculate the prediction value according to the equation (1). A one-frame image composed of such predicted values is output as a decoded image. As described above, this decoded image is almost the same as the original image.

  Note that, on the receiving side, even if the receiving device 4 is not as shown in FIG. 18, the decoding is performed by performing normal interpolation without using the prediction coefficient by a device that decodes the thinned image by simple interpolation. An image can be obtained. However, the decoded image obtained in this case has deteriorated image quality (resolution).

  By the way, in the above-described case, the local decoding unit 22 in FIG. 6 obtains the prediction coefficient and uses this to calculate the prediction value. However, the local decoding unit 22 does not obtain the prediction coefficient, It is possible to calculate the predicted value.

  That is, FIG. 19 shows another configuration example of the local decoding unit 22 of FIG. In the figure, parts corresponding to those in FIG. 10 are denoted by the same reference numerals. That is, the local decoding unit 22 in FIG. 19 is configured in the same manner as in FIG. 10 except that a prediction coefficient ROM 81 and a prediction circuit 82 are provided instead of the adaptive processing circuit 46.

  The prediction coefficient ROM 81 stores a prediction coefficient for each class obtained in advance by learning (described later), receives the class information output from the class classification circuit 45, and is stored at an address corresponding to the class information. The prediction coefficient is read and supplied to the prediction circuit 82.

  The prediction circuit 82 uses the prediction value calculation block from the prediction value calculation blocking circuit 42 and the prediction coefficient from the prediction coefficient ROM 81, and uses Equation (1) (specifically, for example, Equation (8)). ) Is calculated, and thereby the predicted value of the original image is calculated.

  Therefore, according to the class classification adaptive processing circuit 43 of FIG. 19, the predicted value is calculated without using the original image.

  Next, FIG. 20 shows a configuration example of an image processing apparatus that performs learning for obtaining a prediction coefficient stored in the prediction coefficient ROM 81 of FIG.

  The learning block circuit 91 and the teacher block circuit 92 are supplied with learning image data (learning image) for obtaining a prediction coefficient applicable to any image (that is, before the thinning process is performed). It is made to be done.

The learning blocking circuit 91 extracts, for example, four pixels (for example, X _{1 to} X _{4 in} FIG. 2) from the input image data, and a block composed of these four pixels is used as a learning block. The ADRC process 93 and the learning data memory 96 are supplied.

Further, the teacher block forming circuit 92 generates a block composed of, for example, one pixel (Y _{1 in} FIG. 2) from the input image data, and the block composed of this one pixel is the teacher block. Is supplied to the teacher data memory 98.

  Note that when the learning block forming circuit 91 generates a learning block including a predetermined number of pixels, the teacher blocking circuit 92 generates a teacher block for the corresponding pixel. ing.

  The ADRC processing circuit 93 performs a 1-bit ADRC process on the block of four pixels constituting the learning block, as in the ADRC processing circuit 44 of FIG. The 4-pixel block subjected to ADRC processing is supplied to the class classification circuit 94. In the class classification circuit 94, the blocks from the ADRC processing circuit 93 are classified, and class information obtained thereby is supplied to the learning data memory 96 and the teacher data memory 98 via the terminal a of the switch 95.

  In the learning data memory 96 or the teacher data memory 98, the learning block from the learning blocking circuit 91 or the teacher block from the teacher blocking circuit 92 is respectively assigned to the address corresponding to the class information supplied thereto. Remembered.

Therefore, in the learning data memory 96, if a block of four pixels (X _{1 to} X _{4 in} FIG. 2) is stored as a learning block at a certain address, the teacher data memory 98 has the same address as that address. In addition, a corresponding block of one pixel (Y _{1 in} FIG. 2) is stored as a teacher block.

  Thereafter, the same processing is repeated for all learning images prepared in advance, whereby the learning block and the local decoding unit 22 shown in FIG. 19 are the same as the four pixels constituting the learning block. A teacher block composed of one pixel from which a predicted value is obtained using a predicted value calculation block composed of four correction data having a positional relationship includes a learning data memory 96 and teacher data. In the memory 98, they are stored at the same address.

  In the learning data memory 96 and the teacher data memory 98, a plurality of information can be stored at the same address, whereby a plurality of learning blocks and a teacher data are stored at the same address. Blocks can be stored.

  When the learning blocks and the teacher blocks for all the learning images are stored in the learning data memory 96 and the teacher data memory 98, the switch 95 that has selected the terminal a is switched to the terminal b. The output of the counter 97 is supplied to the learning data memory 96 and the teacher data memory 98 as addresses. The counter 97 counts a predetermined clock and outputs the count value. In the learning data memory 96 or the teacher data memory 98, a learning block or a teacher block stored at an address corresponding to the count value is stored. It is read out and supplied to the arithmetic circuit 99.

  Accordingly, the arithmetic circuit 99 is supplied with a set of learning blocks of a class corresponding to the count value of the counter 97 and a set of teacher blocks.

  When the arithmetic circuit 99 receives a set of learning blocks and a set of teacher blocks for a certain class, the arithmetic circuit 99 uses them to calculate a prediction coefficient that minimizes the error by the least square method.

That is, for example, the pixel values of the pixels constituting the learning block are now x ₁ , x ₂ , x ₃ ,... And the prediction coefficients to be obtained are w ₁ , w ₂ , w ₃ ,. Then, in order to obtain the pixel value y of a certain pixel constituting the teacher block by these linear linear combinations, the prediction coefficients w ₁ , w ₂ , w ₃ ,... There is.

y = w ₁ x ₁ + w ₂ x ₂ + w ₃ x ₃ +...

Therefore, the arithmetic circuit 99 calculates a square error of the predicted value w ₁ x ₁ + w ₂ x ₂ + w ₃ x ₃ +... With respect to the true value y from the learning block of the same class and the corresponding teacher block. The minimum prediction coefficients w ₁ , w ₂ , w ₃ ,... Are obtained by building and solving the normal equation shown in the above equation (7).

  The prediction coefficient for each class obtained by the arithmetic circuit 99 is supplied to the memory 100. In addition to the prediction coefficient from the arithmetic circuit 99, the memory 100 is supplied with a count value from the counter 97. Thus, in the memory 100, the prediction coefficient from the arithmetic circuit 99 becomes the count value from the counter 97. Stored at the corresponding address.

  As described above, the memory 100 stores the optimum prediction coefficient for predicting the pixel of the block of the class at the address corresponding to each class.

  The prediction coefficient ROM 81 in FIG. 19 stores the prediction coefficient stored in the memory 100 as described above.

  Note that the prediction coefficient ROM 81 can store an average value of pixel values constituting a teacher block instead of storing a prediction coefficient at an address corresponding to each class. In this case, when class information is given, a pixel value corresponding to the class is output, and the local decoding unit 22 in FIG. 19 does not include the prediction value calculation blocking circuit 42 and the prediction circuit 82. It will be over.

  In the case where the local decoding unit 22 is configured as shown in FIG. 19, the receiving device 4 shown in FIG. 18 is connected to the class classification adaptive processing circuit 43 in FIG. What is necessary is just to make it comprise similarly.

  In this embodiment, the error sum of squares is used as the error information. However, as the error information, for example, the sum of absolute values of errors and the sum of the third and higher powers are used. It is possible to Which one is used as error information can be determined based on, for example, its convergence.

  In this embodiment, the correction of the compressed data is repeatedly performed until the error information becomes equal to or less than the predetermined threshold value ε. However, an upper limit may be set for the number of corrections of the compressed data. It is. That is, for example, when image transmission is performed in real time, processing for one frame needs to be completed within a predetermined period, but error information converges within such a predetermined period. Not always. Therefore, by setting an upper limit on the number of corrections, if the error information does not converge below the threshold ε within a predetermined period, the processing for that frame is terminated (the correction data at that time is used as the encoding result). ), It is possible to start processing for the next frame.

  Furthermore, in the present embodiment, a block is configured from an image of one frame, but the block may also be configured from pixels at the same position in a plurality of frames that are continuous in time series, for example. Is possible.

  In the present embodiment, the compression unit 21 simply thins out the image, that is, extracts one pixel every two pixels and uses it as compressed data. In addition, for example, the average value of a plurality of pixels constituting the block is obtained, and the average value is set as the pixel value of the central pixel in the block, thereby reducing the number of pixels (decimation), It is also possible to use compressed data.

  FIG. 21 shows a configuration example of the transmission apparatus 1 in this case.

  Image data to be encoded is input to the block forming circuit 111. The block forming circuit 111 is a class classification that is a unit for classifying image data into predetermined classes according to their properties. These blocks are divided into blocks for use and supplied to the ADRC processing circuit 112 and the delay circuit 115.

  The ADRC processing circuit 112 performs ADRC processing on the block (class classification block) from the blocking circuit 111 and supplies a block composed of the resulting ADRC code to the class classification circuit 113. ing.

  By this ADRC processing, the number of bits of pixels constituting the class classification block is reduced.

  The class classification circuit 113 performs a class classification process for classifying the block from the ADRC processing circuit 112 into a predetermined class according to its property, and uses as a class information which class the block belongs to as a mapping coefficient memory. 114 is provided.

  The mapping coefficient memory 114 stores mapping coefficients obtained by learning (mapping coefficient learning) as described later for each class information, and stores the class information supplied from the class classification circuit 113 as an address. The mapping coefficient thus read is read out and supplied to the arithmetic circuit 116.

  The delay circuit 115 delays the block supplied from the block forming circuit 111 until the mapping coefficient corresponding to the class information of the block is read from the mapping coefficient memory 114, and supplies the block to the arithmetic circuit 116. .

The arithmetic circuit 116 performs a predetermined operation using the pixel values of the pixels constituting the block supplied from the delay circuit 115 and the mapping coefficient corresponding to the class of the block supplied from the mapping coefficient memory 114. Thus, encoded data obtained by encoding the image with the number of pixels thinned out (reduced) is calculated. That is, the arithmetic circuit 116 sets the pixel values (pixel values of the original image) of each pixel constituting the block output from the blocking circuit 111 to y ₁ , y ₂ ,. when outputting, the mapping coefficients corresponding to the class of the block k _1, k _2, and ..., predetermined function value f to their arguments _{(y 1, y 2, ···} , k 1, k ₂ ,...) and the function value f (y ₁ , y ₂ ,..., k ₁ , k ₂ ,. For example, the pixel value of the central pixel among the pixels constituting the block) is output.

  Accordingly, assuming that the number of pixels constituting the class classification block output from the blocking circuit 111 is N pixels, the arithmetic circuit 116 thins out the image data to 1 / N and outputs this as encoded data. ing.

Note that the encoded data output from the arithmetic circuit 116 is not obtained by a simple thinning process in which one pixel at the center of a block composed of N pixels is selected and output. Thus, the function value f (y ₁ , y ₂ ,..., K ₁ , k ₂ ,...) Defined by the N pixels constituting the block is the function value f (y ₁ , y ₂ ,..., k ₁ , k ₂ ,..., based on the surrounding pixel values, the pixel value of the pixel at the center of the block, which is obtained by simple decimation processing. It can be considered as corrected. Therefore, the encoded data, which is data obtained as a result of the operation of the mapping coefficient and the pixels constituting the block, is also referred to as correction data as appropriate hereinafter.

Further, the arithmetic processing in the arithmetic circuit 116 is performed by converting the pixel value of each pixel constituting the class classification block output from the blocking circuit 111 into the function value f (y ₁ , y ₂ ,..., K ₁ , k _2. ,...)) Can also be considered as mapping (mapping) processing. Therefore, the coefficients k ₁ , k ₂ ,... Used for such processing are called mapping coefficients.

  The transmitter / recording device 117 records correction data supplied as encoded data from the arithmetic circuit 116 on the recording medium 2 or transmits it via the transmission path 3.

  Next, the operation will be described with reference to the flowchart of FIG.

  For example, image data is supplied to the block forming circuit 111 in units of one frame (field). In the block forming circuit 111, in step S61, one frame image is blocked in a class classification block. It becomes. That is, the blocking circuit 111 divides the block into class classification blocks each including, for example, 5 pixels, and supplies the block to the ADRC processing circuit 112 and the delay circuit 115.

  In this case, the class classification block is configured by a cross-shaped block of 5 pixels. However, the class classification block may have other shapes such as a rectangle, a square, and other arbitrary shapes. Is possible. Further, the number of pixels constituting the class classification block is not limited to five pixels. Furthermore, the class classification block may be configured not with adjacent pixels but with distant pixels. However, the shape and the number of pixels need to match those in the case of learning (mapping coefficient learning) described later.

When the ADRC processing circuit 112 receives the block for class classification from the blocking circuit 111, in step S62, the ADRC processing circuit 112 includes four pixels (X _{1 in} FIG. 2) excluding the central pixel (Y _{1 in} FIG. 2) of the block. To X ₄ ), for example, a 1-bit ADRC process is performed so that each of the R, G, and B pixels is a block composed of pixels represented by 1 bit. The class classification block subjected to ADRC processing is supplied to the class classification circuit 113.

  In the class classification circuit 113, the class classification block from the ADRC processing circuit 112 is classified in step S63, and the class information obtained as a result is supplied to the mapping coefficient memory 114 as an address. As a result, the mapping coefficient corresponding to the class information supplied from the class classification circuit 113 is read from the mapping coefficient memory 114 and supplied to the arithmetic circuit 116.

On the other hand, the delay circuit 115 delays the 5-pixel data of the class classification block from the blocking circuit 111 and waits for the mapping coefficient corresponding to the class information of the block to be read from the mapping coefficient memory 114 before performing the calculation. Is supplied to the vessel 116. In step S64, the arithmetic unit 116 uses the pixel value of each pixel constituting the class classification block from the delay circuit 115 and the mapping coefficient from the mapping coefficient memory 114, and uses the function value f (•) (this · In parentheses of the function f represents a set of pixel values X ₁ , X ₂ ,... And mapping coefficients k ₁ , k ₂ ,. Correction data obtained by correcting the pixel value of the center pixel (center pixel) constituting the block for use is calculated. In this case, one piece of pixel data at the position of the pixel data Y ₁ (X ₅ ) is generated from the pixel data X _{1 to} X ₄ and the pixel data Y ₁ (X ₅ ) in FIG. This blocking is performed on the pixel data in an overlapping manner, and finally ½ pixel data is thinned out. Also in this process, as described above, in order to generate the R component (G component or B component), not only the R component (G component or B component) but also the G component and B (R component and B component, or R component and G component) are used. This correction data is supplied to the transmitter / recording device 117 as encoded data obtained by encoding an image.

  In the transmitter / recording device 117, the encoded data from the arithmetic circuit 116 is recorded on the recording medium 2 or transmitted via the transmission path 3 in step S 65.

  In step S66, it is determined whether or not the processing for one frame of image data has been completed. If it is determined in step S66 that the processing for the image data for one frame has not been completed, the process returns to step S62, and the processing in step S62 and subsequent steps is repeated for the next class classification block. If it is determined in step S66 that the processing for the image data for one frame has been completed, the process returns to step S61, and the processing in step S61 and subsequent steps is repeated for the next frame.

  Next, FIG. 23 shows a configuration example of an image processing apparatus that performs learning (mapping coefficient learning) processing for calculating the mapping coefficient stored in the mapping coefficient memory 114 of FIG.

  The memory 121 stores one or more frames of digital image data suitable for learning (hereinafter referred to as learning images as appropriate). The blocking circuit 122 reads the image data stored in the memory 121, configures the same block as the class classification block output from the blocking circuit 111 in FIG. 21, and includes an ADRC processing circuit 123 and an arithmetic circuit 126. It is made to supply to.

  The ADRC processing circuit 123 or the class classification circuit 124 performs the same processing as that in the ADRC processing circuit 112 or the class classification circuit 113 of FIG. Therefore, the class classification circuit 124 outputs the class information of the block output from the blocking circuit 122. This class information is supplied to the mapping coefficient memory 131 as an address.

  The arithmetic circuit 126 performs the same operation as in the arithmetic circuit 116 of FIG. 21 using the pixels constituting the block supplied from the blocking circuit 122 and the mapping coefficient supplied from the mapping coefficient memory 131, The correction data (function value f (·)) obtained as a result is supplied to the local decoding unit 127.

  Based on the correction data supplied from the arithmetic circuit 126, the local decoding unit 127 predicts the predicted value of the original learning image (the predicted value of the pixel value of the pixels constituting the block output by the blocking circuit 122). (Calculated) and supplied to the error calculation unit 128. The error calculation unit 128 reads out the pixel value (true value) of the learning image corresponding to the prediction value supplied from the local decoding unit 127 from the memory 121, and calculates the prediction error of the prediction value with respect to the pixel value of the learning image. Calculation (detection) is performed, and the prediction error is supplied to the determination unit 129 as error information.

  The determination unit 129 compares the error information from the error calculation unit 128 with a predetermined threshold value ε1, and controls the mapping coefficient setting circuit 130 in accordance with the comparison result. The mapping coefficient setting circuit 130 sets (changes) the same number of mapping coefficient sets as the number of classes obtained as a result of class classification in the class classification circuit 124 according to the control of the determination unit 129 and supplies the set to the mapping coefficient memory 131. It is made like that.

  The mapping coefficient memory 131 temporarily stores the mapping coefficient supplied from the mapping coefficient setting circuit 130. The mapping coefficient memory 131 has storage areas that can store as many mapping coefficients (sets of mapping coefficients) as the number of classes classified in the class classification circuit 124. In each storage area, When a new mapping coefficient is supplied from the mapping coefficient setting circuit 130, the new mapping coefficient is stored instead of the already stored mapping coefficient.

  The mapping coefficient memory 131 is also configured to read the mapping coefficient stored at the address corresponding to the class information supplied from the class classification circuit 124 and supply the mapping coefficient to the arithmetic circuit 126.

  Next, the operation will be described with reference to the flowchart of FIG.

  First, the mapping coefficient setting circuit 130 sets a set of initial values of mapping coefficients for the number of classes classified in the class classification circuit 124 in step S71 and supplies the set to the mapping coefficient memory 131. In the mapping coefficient memory 131, the mapping coefficient (initial value) from the mapping coefficient setting circuit 130 is stored at the address of the corresponding class.

Then, in step S72, the blocking circuit 122 converts all the learning images stored in the memory 121 into five pixels (X _{1 to} X ₄ , FIG. 2 in the same manner as in the blocking circuit 111 in FIG. 21). Block into Y ₁ ) blocks. Further, the blocking circuit 121 reads the block from the memory 121 and sequentially supplies the block to the ADRC processing circuit 123 and the arithmetic circuit 126.

In the ADRC processing circuit 123, in step S 73, four pixels (X _{1 to} X _{4 in} FIG. 2) in the block from the blocking circuit 122 are processed as in the case of the ADRC processing circuit 112 in FIG. Bit ADRC processing is performed and supplied to the class classification circuit 124. In step S74, the class classification circuit 124 determines the class of the block supplied from the ADRC processing circuit 123, and supplies the class information to the mapping coefficient memory 131 as an address. As a result, in step S 75, the mapping coefficient is read from the address corresponding to the class information supplied from the class classification circuit 124 in the mapping coefficient memory 131 and supplied to the arithmetic circuit 126.

The arithmetic circuit 126 receives the five pixels (X _{1 to} X ₄ , Y _{1 in} FIG. 2) of the block from the blocking circuit 122 and receives the mapping coefficient corresponding to the class of the block from the mapping coefficient memory 131. Then, in step S76, the function value f (•) described above is calculated using the mapping coefficient and the pixel values of the five pixels constituting the block supplied from the blocking circuit 122. This calculation result is supplied to the local decoding unit 127 as correction data obtained by correcting the pixel value of the central pixel of the block supplied from the blocking circuit 122.

That is, for example, in FIG. 2 described above, assuming that the blocks X _{1 to} X ₄ and Y ₁ are output from the blocking circuit 122, the arithmetic circuit 126 obtains correction data obtained by correcting the pixel value. Is output to the local decoding unit 27.

  However, in the arithmetic circuit 26, the blocking in the blocking circuit 122 is performed on the pixel data in an overlapping manner, and the number of pixels constituting the learning image is thinned out to ½, and the local decoding unit 27 Supplied.

  Referring back to FIG. 24, after the correction data is calculated in step S76, the process proceeds to step S77, and it is determined whether correction data for all the learning images stored in the memory 121 has been obtained. If it is determined in step S77 that the correction data for all the learning images has not yet been obtained, the process returns to step S73, and the correction data for all the learning images is obtained in steps S73 to S77. Repeat the process.

  If it is determined in step S77 that correction data for all learning images has been obtained, that is, a thinned image obtained by thinning all the learning images stored in the memory 121 to ½ is obtained. If this is the case (however, this thinned image is not a learning image that is simply thinned by half, but a pixel value obtained by calculation with a mapping coefficient), the process proceeds to step S78, where In the decoding unit 127, the predicted value of the original learning image is calculated by locally decoding the thinned image. This predicted value is supplied to the error calculation unit 128.

  Here, an image composed of predicted values obtained in the local decoding unit 127 (however, as described later, when the error information output from the error calculation unit 128 becomes smaller than the threshold ε1) is received. This is the same as the decoded image obtained on the device 4 side.

In step S79, the error calculation unit 128 reads the learning image from the memory 121, and calculates the prediction error of the prediction value supplied from the local decoding unit 127 for the learning image. That is, when the pixel value of the learning image is expressed as Y _ij and the predicted value output from the local decoding unit 127 is expressed as E [Y _ij ], the error calculation unit 128 calculates the error variance expressed by the following equation: (Sum of squares of error) Q is calculated, and this is supplied to the determination unit 129 as error information.

Q = Σ (Y _ij −E [Y _ij ]) ²
However, in the above equation, Σ represents summation for all the pixels of the learning image.

  When receiving the error information from the error calculation unit 128, the determination unit 129 compares the error information with a predetermined threshold value ε1, and determines the magnitude relationship in step S80. If it is determined in step S80 that the error information is greater than or equal to the threshold ε1, that is, the image composed of the prediction values obtained in the local decoding unit 127 is not recognized as being the same as the original learning image. In this case, the determination unit 129 outputs a control signal to the mapping coefficient setting circuit 130. In step S <b> 81, the mapping coefficient setting circuit 130 changes the mapping coefficient in accordance with the control signal from the determination unit 129, and newly stores the changed mapping coefficient in the mapping coefficient memory 131.

  Then, the process returns to step S73, and the processing after step S73 is repeated again using the changed mapping coefficient stored in the mapping coefficient memory 131.

  Here, the mapping coefficient setting circuit 130 may change the mapping coefficient at random, and if the current error information becomes smaller than the previous error information, the same tendency as the previous time is observed. When the error information of this time becomes larger than the previous error information, it can be changed with a tendency reverse to that of the previous time.

  Further, the mapping coefficient can be changed for all classes or only for some of the classes. In the case of changing only the mapping coefficients for some classes, for example, a class having a strong influence on error information can be detected, and only the mapping coefficients for such classes can be changed. The class having a strong influence on the error information can be detected as follows, for example. That is, first, the error information is obtained by performing processing using the initial value of the mapping coefficient. Then, the mapping coefficient is changed by the same amount for each class, and the error information obtained as a result is compared with the error information obtained when the initial value is used, and the difference becomes a predetermined value or more. The class may be detected as a class having a strong influence on the error information.

  In addition, when there are a plurality of mapping coefficients such as k1, k2,... Described above, only the one having a strong influence on the error information can be changed. .

  Furthermore, in the above-described case, the mapping coefficient is set for each class. However, the mapping coefficient is set separately for each block, for example, or in units of adjacent blocks. It is possible to make it.

  However, when mapping coefficients are set independently for each block, for example, multiple sets of mapping coefficients may be obtained for a certain class (conversely, mapping coefficients). However, there may be classes that cannot be obtained in one set). Since the mapping coefficients need to be finally determined for each class, as described above, when multiple sets of mapping coefficients are obtained for a certain class, multiple sets of mapping coefficients are targeted. It is necessary to determine one set of mapping coefficients by performing some processing.

  On the other hand, when it is determined in step S80 that the error information is smaller than the threshold value ε1, that is, when the image composed of the prediction values obtained in the local decoding unit 127 is recognized to be the same as the original learning image. The process is terminated.

  At this time, it is optimal to obtain correction data that can restore a decoded image (predicted value) in which the mapping coefficient for each class stored in the mapping coefficient memory 131 is recognized to be the same as the original image. As a thing, it is set to the mapping coefficient memory 114 of FIG.

  Therefore, by generating correction data using such a mapping coefficient, on the receiving device 4 side, it is possible to obtain an image that is almost the same as the original image.

  In the embodiment shown in FIG. 23, as described above, the block forming circuit 122 blocks the image into four pixels, and the ADRC processing circuit 123 performs 1-bit ADRC processing. The number of classes obtained by the class classification by the circuit 124 is 4096, and thus 4096 sets of mapping coefficients are obtained.

  Next, FIG. 25 illustrates a configuration example of the local decoding unit 127 of FIG.

  The correction data from the arithmetic circuit 126 is supplied to the class classification blocking circuit 141 and the predicted value calculation blocking circuit 142. The class classification blocking circuit 141 is configured to block the correction data into class classification blocks, which are units for classifying the correction data into predetermined classes according to their properties.

  Note that the class classification block obtained in the class classification blocking circuit 141 in FIG. 25 is configured to determine a class of a block for which a prediction value is obtained, and in this respect, a block for calculating correction data. Is different from that generated by the blocking circuit 111 of FIG.

  The predicted value calculation blocking circuit 142 blocks the correction data into predicted value calculation blocks which are units for calculating the predicted value of the original image (here, the learning image). .

  The prediction value calculation block obtained in the prediction value calculation block forming circuit 142 is supplied to the prediction circuit 146.

  Note that the number of pixels and the shape of the prediction value calculation block are not limited to those described above, as in the case of the class classification block. However, in the local decoding unit 127, it is desirable that the number of pixels constituting the prediction value calculation block is larger than the number of pixels constituting the class classification block.

  Further, when the above-described blocking is performed (the same applies to processes other than blocking), there may be no corresponding pixel near the image frame of the image. In this case, for example, the image frame The processing is performed on the assumption that the same pixel as that constituting the pixel exists outside.

  The ADRC processing circuit 143 performs, for example, 1-bit ADRC processing on a block (class classification block) output from the class classification blocking circuit 141 and supplies the block to the class classification circuit 144. The class classification circuit 144 classifies the blocks from the ADRC processing circuit 143, and supplies class information as a classification result to the prediction coefficient ROM 145. The prediction coefficient ROM 145 stores a prediction coefficient. When class information is received from the class classification circuit 144, the prediction coefficient ROM 145 reads the prediction coefficient stored at an address corresponding to the class information and supplies the prediction coefficient to the prediction circuit 146. ing. Note that the prediction coefficients stored in the prediction coefficient ROM 145 are obtained by learning (prediction coefficient learning) described later.

  The prediction circuit 146 calculates (predicts) a predicted value of the original image (learning image) using the predicted value calculation block from the predicted value calculation blocking circuit 142 and the prediction coefficient from the prediction coefficient ROM 145. It is made to do.

  Next, the operation will be described with reference to the flowchart of FIG.

In the local decoding unit 127, first, in step S91, the correction data from the arithmetic circuit 126 is sequentially received and blocked. That is, in the class classification blocking circuit 141, the correction data is blocked into four pixel classification blocks (X _{1 to} X _{4 in} FIG. 2) and supplied to the ADRC processing circuit 143, and a predicted value calculation is performed. In the block forming circuit 142, the correction data is divided into blocks for predicting value calculation of 4 pixels and supplied to the predicting circuit 146.

  The class classification blocking circuit 141 and the prediction value calculation blocking circuit 142 generate corresponding class classification blocks and prediction value calculation blocks.

  Upon receiving the class classification block, the ADRC processing circuit 143 performs, for example, 1-bit ADRC (ADRC that performs re-quantization with 1 bit) on the class classification block in step S92. The correction data is converted (encoded) into 1 bit and output to the class classification circuit 144. In step S93, the class classification circuit 144 performs class classification processing on the class classification block that has been subjected to ADRC processing, and determines the class to which the class classification block belongs. This class determination result is supplied to the prediction coefficient ROM 145 as class information.

In the embodiment of FIG. 25, class classification processing is performed on a class classification block composed of four pixels in which R, G, and B components are each subjected to 1-bit ADRC processing. Therefore, each class classification block is classified into one of 4096 (= 2 ¹² ) classes.

  In step S94, the prediction coefficient is read from the address corresponding to the class information from the class classification circuit 144 in the prediction coefficient ROM 145. In step S95, the prediction circuit 146 calculates the prediction coefficient and the prediction value. Using the four pixel values constituting the prediction value calculation block from the blocking circuit 142, for example, according to the following linear linear expression, the prediction value E [y] of the pixel value y of the original image is obtained. calculate.

E [y] = w ₁ x ₁ + w ₂ x ₂ +...
However, w _1, w _2, ··· represents a prediction coefficient, x _1, x _2, ··· represent the pixel values of the pixels constituting the predicted values calculation block (correction data). However, x _1, x _2, · · · includes R, G, and B components, respectively, w _1, w _2, · · · also, R, G, consists of coefficients for B.

  Here, in the embodiment of FIG. 25, as described above, the predicted value of one pixel is calculated from the four pixels constituting the predicted value calculation block.

That is, for example, the class information C about the class classification block composed of the correction data X _{1 to} X ₄ shown in FIG. 2 is output from the class classification circuit 144, and X _{1 is} used as a predicted value calculation block. It is assumed that a predicted value calculation block consisting of through X ₄ is output from the predicted value calculation block forming circuit 142.

Further, in the prediction coefficient ROM 145, w ₁ (R) to w ₁₂ (R), w ₁ (G) to w ₁₂ (G), w ₁ ( Assuming that B) to w ₁₂ (B) are stored, for example, similarly to the case described above, for example, the predicted values E [Y _Ri ], Y of each component Y _Ri , Y _Gi , Y _Bi of each pixel, E [Y _Gi ], E [Y _Bi ] are calculated.

  When the predicted value is obtained as described above in step S94, the process returns to step S91, and thereafter, the processing of steps S91 to S94 is repeated, whereby the predicted value is obtained in units of 4 pixels.

  The image processing apparatus that performs learning (prediction coefficient learning) for obtaining the prediction coefficient stored in the prediction coefficient ROM 145 of FIG. 25 has the same configuration as that shown in FIG. Therefore, this description is omitted.

  Next, FIG. 27 shows another configuration example of the image processing apparatus that performs learning (mapping coefficient learning) processing for calculating the mapping coefficient stored in the mapping coefficient memory 114 of FIG.

  Note that, according to the image processing apparatus of FIG. 23, the optimal prediction is possible not only when the function f is expressed by, for example, a linear linear expression but also by a nonlinear expression or a quadratic expression or higher. Although the coefficient can be obtained, the image processing apparatus in FIG. 27 can obtain the optimum prediction coefficient only when the function f is expressed by a linear linear expression.

That is, the image processing apparatus of FIG. 27, in FIG. 21, y ₁ pixel values of four pixels constituting the block outputted from the blocking circuit _{111 (X 1, X 2,} X 3, X 4 in FIG. 2), y ₂ , y ₃ , y ₄ (each having R, G, B components), and mapping coefficients output by the mapping coefficient memory 114 are k ₁ , k ₂ , k ₃ , k ₄ (respectively, R, G, B components), the arithmetic circuit 116 calculates the function value f (y ₁ , y ₂ ,..., K ₁ , k ₂ ,...) According to the following equation: It can be used when correction data is obtained.

f (·) = k ₁ y ₁ + k ₂ y ₂ + k ₃ y ₃ + k ₄ y ₄

  The optimum correction data calculation unit 170 is supplied with a learning image suitable for learning, for example, in units of one frame. The optimal correction data calculation unit 170 includes a compression unit 171, a correction unit 172, a local decoding unit 173, an error calculation unit 174, and a determination unit 175, and reduces the number of pixels from the learning image input thereto. A pixel value (hereinafter referred to as optimum correction data as appropriate) that constitutes an image that is compressed and is optimal for predicting the original image is calculated and supplied to the latch circuit 176.

  That is, the learning image supplied to the optimum correction data calculation unit 170 is supplied to the compression unit 171 and the error calculation unit 174. The compression unit 171 simply thins out the learning image at the same rate as the arithmetic circuit 116 of FIG. 21 thins out the pixels. In other words, the compression unit 171 simply thins out the learning image in half in this embodiment. Thereby, the learning image is compressed and supplied to the correction unit 172.

  The correction unit 172 corrects data supplied from the compression unit 171 and compressed by simple thinning (hereinafter, referred to as compressed data as appropriate) according to control from the determination unit 175. Data obtained as a result of correction in the correction unit 172 (This data is also obtained by correcting the pixel value of the central pixel of the five-pixel block in the same manner as the output of the arithmetic circuit 116 in FIG. 21. Data) is supplied to the local decoding unit 173.

  The local decoding unit 173 predicts the original image (learning image) based on the correction data from the correction unit 172 in the same manner as the case of the local decoding unit 127 in FIG. The unit 174 is supplied.

  The error calculation unit 174 calculates the prediction error of the prediction value from the local decoding unit 173 for the original image data input thereto, in the same manner as in the error calculation unit 128 of FIG. Yes. This prediction error is supplied to the determination unit 175 as error information.

  Based on the error information from the error calculation unit 174, the determination unit 175 determines whether or not the correction data output from the correction unit 172 is the compression result of the original image. If the determination unit 175 determines that the correction data output from the correction unit 172 is not appropriate as the compression result of the original image, the determination unit 175 controls the correction unit 172 to further correct the compressed data. The new correction data obtained as a result is output. In addition, when the determination unit 175 determines that the correction data output from the correction unit 172 is appropriate as the compression result of the original image, the determination unit 175 uses the correction data supplied from the correction unit 172 as an optimal correction. The data is supplied to the latch circuit 176.

  The latch circuit 176 has a built-in memory 176A, and the optimum correction data supplied from the correction unit 172 is stored in the memory 176A. Further, the latch circuit 176 reads out the optimum correction data stored in the memory 176A corresponding to the central pixel of the block read from the memory 177A of the blocking circuit 177 and supplies it to the memory 180. . Note that when the correction data for one frame is stored in the memory 176A, the latch circuit 176 outputs a control signal indicating that to the blocking circuit 177.

  As with the optimum correction data calculation unit 170, the learning circuit 177 is supplied with learning images in units of one frame. The block forming circuit 177 has a built-in memory 177A, and the learning image supplied thereto is stored in the memory 177A. Further, when receiving the control signal from the latch circuit 176, the blocking circuit 177 converts the learning image stored in the memory 177A into a block composed of 5 pixels as in the blocking circuit 111 in FIG. The blocks are divided and the blocks are sequentially read out and supplied to the ADRC processing circuit 178 and the memory 180.

  The blocking circuit 177 supplies a control signal indicating the position of the block to the latch circuit 176 when reading the block from the built-in memory 177A. Based on this control signal, the latch circuit 176 recognizes the 5-pixel block read from the memory 177A, and as described above, the optimum correction data corresponding to the central pixel of the block is read from the memory 176A. Has been made. That is, as a result, a block of 5 pixels and optimum correction data corresponding to the block are simultaneously supplied to the memory 180.

  The ADRC processing circuit 178 or the class classification circuit 179 is configured similarly to the ADRC processing circuit 112 or the class classification circuit 113 of FIG. The class information about the block from the blocking circuit 177 output from the class classification circuit 179 is supplied to the memory 180 as an address.

  The memory 180 stores the optimum correction data supplied from the latch circuit 176 and the block supplied from the blocking circuit 177 in association with the address corresponding to the class information supplied from the class classification circuit 179. Has been made. Note that the memory 180 can store a plurality of pieces of information at one address, so that a plurality of sets of optimum correction data and blocks corresponding to certain class information can be stored. Has been made.

The arithmetic circuit 181 stores the five pixels y ₁ , y ₂ , y ₃ , y ₄ , and y ₅ that constitute a 5-pixel block of the learning image stored in the memory 180, and the optimum associated with the block. By reading the correction data y ′ and applying the least square method thereto, the mapping coefficients k _{1 to} k ₅ are obtained for each class and supplied to the memory 182. The memory 182 stores the mapping coefficients k _{1 to} k ₅ for each class supplied from the arithmetic circuit 181 at addresses corresponding to the classes.

  Next, the operation will be described with reference to the flowchart of FIG.

  When the learning image is input, the learning image is stored in the memory 177 A of the blocking circuit 177 and supplied to the optimum correction data calculation unit 170. When receiving the learning image, the optimum correction data calculating unit 170 calculates optimum correction data for the learning image in step S101.

  The processing in step S101 is the same as the processing in the flowchart in FIG. That is, first, in step S1, the compression unit 171 generates compressed data by thinning out the learning image by ½, and the correction unit 172 first performs local correction without performing correction. The data is output to the decoding unit 173. In step S2, the local decoding unit 173 calculates the predicted value of the original image based on the correction data from the correction unit 172 (initially, compressed data obtained by simply thinning out image data as described above). (Local decoding is performed). The predicted value is supplied to the error calculation unit 174.

  When the error calculation unit 174 receives the predicted value of the original image from the local decoding unit 173, in step S3, the error calculation unit 174 calculates a prediction error of the predicted value from the local decoding unit 173 with respect to the original image data as error information. , And supplied to the determination unit 175. When the determination unit 175 receives the error information from the error calculation unit 174, in step S4, the determination unit 175 determines whether the correction data output from the correction unit 172 is the compression result of the original image based on the error information. judge.

  That is, in step S4, it is determined whether the error information is equal to or less than a predetermined threshold value ε. If it is determined in step S4 that the error information is not equal to or less than the predetermined threshold ε, it is recognized that the correction data output from the correction unit 172 is not appropriate to be the compression result of the original image, and the process proceeds to step S5. The determination unit 175 controls the correction unit 172, thereby correcting the compressed data output from the compression unit 171. The correction unit 172 changes the correction amount (correction value Δ) according to the control of the determination unit 175, corrects the compressed data, and outputs the correction data obtained as a result to the local decoding unit 173. And it returns to step S2 and the same process is repeated hereafter.

  The correction of the compressed data can be performed, for example, in the same manner as the mapping coefficient change described with reference to FIG.

  On the other hand, if it is determined in step S4 that the error information is equal to or smaller than the predetermined threshold ε, it is recognized that it is appropriate to use the correction data output from the correction unit 172 as the compression result of the original image. The unit 175 outputs correction data when error information equal to or less than a predetermined threshold ε is obtained as optimum correction data from the correction unit 172 to the latch circuit 176, stores it in the built-in memory 176A, and returns. .

  As described above, the correction data obtained by correcting the compressed data when the error information is equal to or smaller than the predetermined threshold ε is stored in the memory 176A as the optimum correction data. Since this optimum correction data has error information that is equal to or less than a predetermined threshold value ε, by using this to calculate a predicted value, an image that is substantially the same as the original image (original image) is obtained. be able to.

  Returning to FIG. 28, the latch circuit 176 outputs a control signal to the blocking circuit 177 after storing the optimum correction data for one frame in the memory 176A. When receiving the control signal from the latch circuit 176, the blocking circuit 177 divides the learning image stored in the memory 177A into blocks each composed of five pixels in step S102. Then, the blocking circuit 177 reads out the learning image block stored in the memory 177 A and supplies it to the ADRC processing circuit 178 and the memory 180.

  At the same time, when the block forming circuit 177 reads the block from the memory 177A, the block forming circuit 177 supplies a control signal indicating the position of the block to the latch circuit 176, and the latch circuit 176 The 5-pixel block read from 177A is recognized, and optimum correction data corresponding to the central pixel of the block is read and supplied to the memory 180.

  In step S103, the ADRC processing circuit 178 performs ADRC processing on the block from the blocking circuit 177, and the class classification circuit 179 classifies the block. The classification result is supplied to the memory 180 as an address.

  In the memory 180, the optimum correction data supplied from the latch circuit 176 and the block (learning data) supplied from the blocking circuit 177 at the address corresponding to the class information supplied from the class classification circuit 179 in step S104. Are stored in association with each other.

  Then, the process proceeds to step S105, where it is determined whether or not the block and the optimum correction data for one frame are stored in the memory 180. In step S105, when it is determined that the block and the optimum correction data for one frame are not yet stored in the memory 180, the next block is read from the blocking circuit 177 and the block is read from the latch circuit 176. The corresponding optimum correction data is read out, and the process returns to step S103, and thereafter, the processes after step S103 are repeated.

  If it is determined in step S105 that the block and the optimal correction data for one frame have been stored in the memory 180, the process proceeds to step S106, and it is determined whether or not the processing has been completed for all the learning images. If it is determined in step S106 that the processing for all the learning images has not been completed yet, the process returns to step S101, and the processing from step S101 is repeated for the next learning image.

  On the other hand, if it is determined in step S106 that the processing for all the learning images has been completed, the process proceeds to step S107, and the arithmetic circuit 181 reads out the optimum correction data and blocks stored in the memory 180 for each class. Thus, a normal equation as shown in Equation (7) is established. Further, in step S108, the arithmetic circuit 181 calculates a mapping coefficient for each class that minimizes the error by solving the normal equation. This mapping coefficient is supplied to and stored in the memory 12 in step S109, and the process ends.

  When the function f is expressed by a linear linear expression, the mapping coefficient stored in the memory 182 as described above is stored in the mapping coefficient memory 114 in FIG. 21 and is used to encode an image. It can be performed.

Note that, depending on the class, there may be cases where the number of normal equations that can determine the mapping coefficient cannot be obtained. In such a case, in the arithmetic circuit 116 in FIG. 21, a mapping coefficient that outputs, for example, an average value of the five pixels constituting the block output from the blocking circuit 111, that is, k _{1 to} k ₅ is output. = 1/5 is set as a default value.

  Next, FIG. 29 illustrates a configuration example of the reception device 4 corresponding to the transmission device of FIG.

  In the receiver / reproduction device 191, the encoded data recorded on the recording medium 2 is reproduced, or the encoded data transmitted via the transmission path 3 is received and supplied to the decoding unit 192.

  The decoding unit 192 includes class classification blocking circuits 193 to 198 corresponding to the class classification blocking circuit 141 to the prediction circuit 146 in the local decoding unit 127 shown in FIG. In the unit 192, a prediction value is obtained from the correction data in the same manner as in the local decoding unit 127 in FIG.

  The correction data has error information that is equal to or less than a predetermined threshold value. Therefore, the receiving apparatus 4 can obtain an image that is almost the same as the original image.

  On the receiving side, even if the receiving device 4 is not as shown in FIG. 29, a decoded image can be obtained by performing normal interpolation with a device that decodes the thinned image by interpolation. However, the decoded image obtained in this case has deteriorated image quality (resolution).

  In the above, pixel data is expressed using R, G, and B component components. However, as component signals, a luminance signal Y, a color signal I, and Combination of color signal Q, luminance signal Y, color difference signal RY, and color difference signal BY, or C (cyan), M (magenta), Y (yellow) mainly used in the field of printing ), And a combination of K (black) added as necessary.

I = 0.60R-0.28G-0.32B
Q = 0.21R-0.52G + 0.31B

RY = 0.7R-0.59G-0.11B
BY = -0.3R-0.59G + 0.89B

C = 255-R
M = 255-G
Y = 255-B
However, C, M, and R are expressed by an additive color mixture with R, G, and B being 8 bits each.

It is a block diagram which shows the structural example of the system which applied the image processing apparatus of this invention. It is a figure explaining operation | movement of the subsampling circuit of FIG. It is a figure explaining the pixel data in the Example of FIG. It is a block diagram which shows the structural example of the apparatus which produces | generates the memory content of ROM218 of FIG. It is a block diagram which shows the other structural example of the transmitter 1 of FIG. FIG. 6 is a block diagram illustrating a functional configuration example of the transmission device 1 of FIG. 5. It is a flowchart for demonstrating operation | movement of the transmitter 1 of FIG. It is a block diagram which shows the structural example of the compression part 21 of FIG. It is a flowchart for demonstrating operation | movement of the compression part 21 of FIG. It is a block diagram which shows the structural example of the local decoding part 22 of FIG. It is a figure for demonstrating a classification process. It is a figure for demonstrating an ADRC process. It is a flowchart for demonstrating operation | movement of the local decoding part 22 of FIG. It is a block diagram which shows the structural example of the error calculation part 23 of FIG. It is a flowchart for demonstrating operation | movement of the error calculation part 23 of FIG. It is a block diagram which shows the structural example of the determination part 24 of FIG. 17 is a flowchart for explaining the operation of the determination unit 24 in FIG. 16. It is a block diagram which shows the further another structural example of the receiver 4 of FIG. It is a block diagram which shows the other structural example of the local decoding part 22 of FIG. It is a block diagram which shows the structure of one Example of the image processing apparatus which calculates the prediction coefficient memorize | stored in the prediction coefficient ROM81 of FIG. It is a block diagram which shows the other structural example of the transmitter 1 of FIG. It is a flowchart for demonstrating operation | movement of the transmission apparatus of FIG. It is a block diagram which shows the structure of 1st Example of the image processing apparatus which performs learning for obtaining a mapping coefficient. It is a flowchart for demonstrating operation | movement of the image processing apparatus of FIG. FIG. 24 is a block diagram illustrating a configuration example of a local decoding unit 127 in FIG. 23. It is a flowchart for demonstrating the process of the local decoding part 127 of FIG. It is a block diagram which shows the structure of 2nd Example of the image processing apparatus which performs learning for obtaining a mapping coefficient. It is a flowchart for demonstrating operation | movement of the image processing apparatus of FIG. It is a block diagram which shows the other structural example of the receiver 4 of FIG.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 Transmitter, 2 Recording medium, 3 Transmission path, 4 Receiver, 11 I / F, 12 ROM, 13 RAM, 14 CPU, 15 External storage device, 16 Transmitter / recorder, 21 Compression part, 22 Local decoding part , 23 Error calculation unit, 24 determination unit, 25 multiplexing unit, 31 decimation circuit, 32 correction circuit, 33 correction value ROM, 41 class classification blocking circuit, 42 prediction value calculation blocking circuit, 43 class classification adaptive processing Circuit, 44 ADRC processing circuit, 45 class classification circuit, 46 adaptive processing circuit, 51 blocking circuit, 52 square error calculation circuit, 53, 54 computing unit, 55 accumulator, 56 memory, 61 prediction coefficient memory, 62 correction data memory , 63 error information memory, 64 comparison circuit, 65 Control circuit, 71 receiver / reproducing device, 72 separation unit, 73 class classification blocking circuit, 74 ADRC processing circuit, 75 class classification circuit, 76 prediction circuit, 77 prediction value calculation blocking circuit, 81 prediction coefficient ROM, 82 prediction circuit, 91 learning block circuit, 92 teacher block circuit, 93 ADRC processing circuit, 94 class classification circuit, 95 switch, 96 learning data memory, 97 counter, 98 teacher data memory, 99 arithmetic circuit, 100 memory

Claims (3)

An image processing apparatus for generating a second image having a higher spatial resolution from a first image having a lower spatial resolution composed of pixel data composed of a plurality of component signals combined to represent pixel data. The acquisition means for acquiring the first pixel data of the first image and the component signal constituting the first pixel data are located in the vicinity of the second pixel data of the second image. A second combination of the second image is obtained by linear combination of a plurality of types of the component signals constituting each of the plurality of first pixel data of the first image and a prediction coefficient obtained by learning in advance. And a prediction means for predicting a component signal constituting the pixel data. In the learning device for the,
Prediction using the component signal constituting the second pixel data and a plurality of types of component signals and prediction coefficients constituting each of the plurality of first pixel data located in the vicinity of the second pixel data A learning apparatus comprising: a calculation unit that performs a calculation for obtaining the prediction coefficient that minimizes an error from a predicted value of the component signal that constitutes the second pixel data. Class classification means for classifying the first pixel data into classes;
The learning apparatus according to claim 1, wherein the calculation unit performs the calculation using the first pixel data for each class. An image processing apparatus for generating a second image having a higher spatial resolution from a first image having a lower spatial resolution composed of pixel data composed of a plurality of component signals combined to represent pixel data. The acquisition means for acquiring the first pixel data of the first image and the component signal constituting the first pixel data are located in the vicinity of the second pixel data of the second image. A second combination of the second image is obtained by linear combination of a plurality of types of the component signals constituting each of the plurality of first pixel data of the first image and a prediction coefficient obtained by learning in advance. And a prediction means for predicting a component signal constituting the pixel data. In learning how to,
Prediction using the component signal constituting the second pixel data and a plurality of types of component signals and prediction coefficients constituting each of the plurality of first pixel data located in the vicinity of the second pixel data A learning method comprising: an operation step of performing an operation for obtaining the prediction coefficient that minimizes an error from a predicted value of the component signal constituting the second pixel data .

Images