JP4770704B2 - Image processing apparatus, image processing method, and program - Google Patents

Description

  The present invention relates to an image processing device, an image processing method, and a program, and in particular, for example, an image processing device, an image processing method, and an image processing method capable of effectively removing noise and obtaining high-quality image data. And program.

For example, a decoded image obtained by encoding and decoding an image for each predetermined block by an encoding and decoding method such as MPEG-2 (Moving Picture Experts Group phase 2), such as block distortion and mosquito noise It becomes a noise image including noise. As a noise removal method for removing noise from such a noise image, a technique for performing noise removal processing using an 8 × 8 pixel (horizontal × vertical) block, which is a processing unit for encoding and decoding, as a processing unit for noise removal, It is disclosed in Patent Documents 1 and 2.
JP 2003-324738 A JP 2004-7607 A

  By the way, in terrestrial digital broadcasting, as shown in FIG. 1, on the broadcast station 31 side, the original image of 1920 × 1080 pixels (horizontal × vertical) to be broadcast is reduced by 3/4 times in the horizontal direction. The reduced image obtained as a result is encoded by an encoding method such as MPEG-2 in units of 8 × 8 pixel blocks, and transmitted to each home as an MPEG encoded image.

  On the other hand, each home side receives the MPEG encoded image transmitted from the broadcasting station 31, and the decoder 32 installed in each home uses the same 8 × 8 pixel block unit as MPEG- By performing decoding corresponding to an encoding method such as 2, an MPEG decoded image obtained by decoding an MPEG encoded image is obtained.

  In addition, the decoder 32 enlarges the MPEG decoded image to a 1920 × 1080 pixel enlarged MPEG decoded image having the same number of pixels as the original image by multiplying the MPEG decoded image by 4/3 in the horizontal direction. It is displayed on a television receiver (not shown).

  In an enlarged MPEG decoded image, an 8 × 8 pixel block, which is an encoding and decoding processing unit based on MPEG-2 or the like, is multiplied by 4/3 in the horizontal direction to be 10.666. ..) It is a block of 8 pixels (horizontal x vertical).

  Therefore, the expanded MPEG decoded image has mosquito noise and block distortion at the boundary (near the block) of 10.666 × 8 pixels, but the noise removal method for removing the noise generated in the expanded MPEG decoded image For example, when noise removal processing using the techniques disclosed in Patent Documents 1 and 2 described above is employed, noise may not be effectively removed.

  That is, the techniques disclosed in Patent Documents 1 and 2 perform noise removal processing for noise generated at the boundary of an 8 × 8 pixel block that matches the processing unit of MPEG-2 encoding and decoding, for example. The noise removal process is not performed in units of 10.666 × 8 pixel blocks.

  Then, an 8 × 8 pixel block to be subjected to noise removal processing using the techniques disclosed in Patent Documents 1 and 2, and a 10.666 × 8 pixel block as a unit in which noise is generated on the enlarged MPEG decoded image, If there is a phase shift that does not match, in other words, there is a phase shift, compared to a case where there is no phase shift where the block subject to noise removal matches the block where the noise is generated, the noise is effectively reduced. Sometimes it cannot be removed.

  The present invention has been made in view of such a situation, and is capable of effectively removing noise and obtaining high-quality image data.

According to an embodiment of the present invention is an image processing apparatus for processing the output image data an image data encoded by a predetermined block unit, a decoder for decoding at the predetermined block unit is output, the output image Determining means for determining a block size of the predetermined block in the data; and first reduction / enlargement means for reducing or enlarging the output image data to first reduced / enlarged image data having a predetermined number of pixels. , the block size of the predetermined block in the output image data, the decoder in the case where the block size of the predetermined block in the decoded image data obtained by performing the decoding are different, the decrypt image data with a predetermined High-quality image data higher than the decoded image data, which is obtained by a prediction calculation using a coefficient. A predicted value measured was the use a high-quality image data and the predetermined coefficients obtained by learning to minimize an error between said first scaled image data, the first scaling the image data The second reduced / enlarged image data with higher image quality than the first reduced / enlarged image data, and when the block size of the predetermined block in the decoded image data in the output image data matches, Image conversion means for converting the output image data into image data having higher image quality than the output image data using a predetermined coefficient and the output image data; and the second reduced / enlarged image data as the output Second reduction / enlargement means for reducing or enlarging image data having the same number of pixels as the image data is provided.

The image converting means converts the first image data into second image data having a higher image quality than the first image data, and the block size of the predetermined block in the output image data; When the block size of the predetermined block in the decoded image data is different, the first reduced / enlarged image data is first image data, the second reduced / enlarged image data is second image data, When the block size of the predetermined block in the output image data matches the block size of the predetermined block in the decoded image data, the output image data is the first image data, and the output image data and high-quality image data and the second image data than the a designated pixel of the first image picture data of interest Corresponding to element, a plurality of pixels used for the prediction computation for predicting the pixel of the second image picture data, the prediction tap extraction means for extracting as a prediction tap from the first image picture data, the pixel of interest a plurality of pixels used for classification to classify into any class of a plurality of classes, and class tap extracting means for extracting as a class tap from the first images data, extracted by the class tap extracting means And classifying means for classifying the pixel of interest based on the class tap, and corresponding to the class of the pixel of interest from the coefficients corresponding to each of the plurality of classes obtained in advance by the learning. A coefficient output means for outputting a coefficient; a coefficient output by the coefficient output means; and a coefficient extracted by the prediction tap extraction means. By the prediction operation using said predictive tap, corresponding to the pixel of interest may have a predictive calculation means for calculating a pixel of the second images data.

In the decoder, the image data obtained by reducing or enlarging the original image to the predetermined number of pixels is decoded in the predetermined block unit for the encoding result obtained by encoding the predetermined block unit, the decoded image data obtained by the decoding, the image data obtained by reducing or enlarging the image data of the original image and the same number of pixels can be output as the output image data.

An image processing method or program according to an aspect of the present invention is an image processing apparatus that processes output image data output from a decoder that decodes image data encoded in predetermined block units. processing method or a predetermined image data encoded in units of blocks, the program for executing the image processing of the image processing apparatus to a computer of a decoder for decoding at the predetermined block unit processes the output image data to be output, the A block size of the predetermined block in the output image data is determined, the output image data is reduced or enlarged to first reduced / enlarged image data having a predetermined number of pixels, and the predetermined image data in the output image data is reduced. The block size of the block and the decoded image obtained by decoding by the decoder When the block size of the predetermined block in the data are different, determined by the prediction calculation using said decrypt the image data and a predetermined coefficient was predicted high-quality high-quality image data than the decoded image data Using the predetermined coefficient obtained by learning that minimizes an error between the predicted value and the high-quality image data and the first reduced / enlarged image data , the first reduced / enlarged image data is Converting the second reduced / enlarged image data having higher image quality than the first reduced / enlarged image data, the block size of the predetermined block in the output image data, and the block size of the predetermined block in the decoded image data; When the output image data is higher than the output image data using the predetermined coefficient and the output image data. Of converting the image data, the second scaling image data, the image data of the output image data and the same number of pixels, comprising the steps of reduced or enlarged.

In one aspect of the present invention, a block size of the predetermined block in the output image data is determined, and the output image data is reduced or enlarged to first reduced / enlarged image data having a predetermined number of pixels. is the block size of the predetermined block in the output image data, wherein when the decoder and the block size of the predetermined block in the decoded image data obtained by performing the decoding are different, the decrypt image data with a predetermined Obtained by a prediction calculation using the coefficient of the image, the prediction value obtained by predicting high-quality image data with higher image quality than the decoded image data, and the learning obtained by learning that minimizes an error between the high-quality image data by using the predetermined coefficient and the first scaled image data, the first scaling the image data, the first condensation Than the enlarged image data is converted into the second scaling image data of high image quality, when the block size of the predetermined block in the output image data, and the block size of the predetermined block in the decoded image data match In addition, the output image data is converted into image data with higher image quality than the output image data using the predetermined coefficient and the output image data . Further, the second reduced / enlarged image data is reduced or enlarged to image data having the same number of pixels as the output image data.

  According to one aspect of the present invention, it is possible to effectively remove noise and obtain high-quality image data.

  Embodiments of the present invention will be described below. Correspondences between the constituent elements of the present invention and the embodiments described in the specification or the drawings are exemplified as follows. This description is intended to confirm that the embodiments supporting the present invention are described in the specification or the drawings. Therefore, even if there is an embodiment which is described in the specification or the drawings but is not described here as an embodiment corresponding to the constituent elements of the present invention, that is not the case. It does not mean that the form does not correspond to the constituent requirements. Conversely, even if an embodiment is described here as corresponding to a configuration requirement, that means that the embodiment does not correspond to a configuration requirement other than the configuration requirement. Not something to do.

An image processing apparatus according to one aspect of the present invention includes:
Output image data (for example, the decoder 32 of FIG. 1) output by the decoder (for example, the decoder 32 of FIG. 1) that decodes image data (for example, the MPEG encoded image of FIG. 1) encoded for each predetermined block in units of the predetermined blocks. In the image processing apparatus (for example, the image processing apparatus 221 in FIG. 9 ) for processing the enlarged MPEG decoded image in FIG.
Determination means for determining the block size of the predetermined block in the output image data (for example, the block size determination unit 252 in FIG. 9);
First reduction / enlargement means (for example, the image reduction filter 101 in FIG. 9) for reducing or enlarging the output image data to first reduced / enlarged image data having a predetermined number of pixels;
And the block size of the predetermined block in the output image data, if the block size is different from the predetermined block in the decoded image data obtained by the decoder performs decoding, the decrypt image data and a predetermined coefficient And the predetermined value obtained by learning that minimizes an error between the predicted value obtained by predicting high-quality image data having a higher image quality than the decoded image data and the high-quality image data. Using the coefficient and the first reduced / enlarged image data , the first reduced / enlarged image data is converted into second reduced / enlarged image data with higher image quality than the first reduced / enlarged image data ,
When the block size of the predetermined block in the output image data matches the block size of the predetermined block in the decoded image data, the output image is used by using the predetermined coefficient and the output image data. Image conversion means (for example, the image conversion unit 102 and the image conversion unit 253 in FIG. 9) for converting the data into image data having higher image quality than the output image data ;
Second reduction / enlargement means (for example, the image enlargement filter 103 in FIG. 9 ) for reducing or enlarging the second reduced / enlarged image data to image data having the same number of pixels as the output image data.

The image conversion means (for example, the image conversion unit 102 in FIG. 4) converts the first image data into second image data having a higher image quality than the first image data.
When the block size of the predetermined block in the output image data is different from the block size of the predetermined block in the decoded image data, the first reduced / enlarged image data is set as first image data, and the first 2 reduced and enlarged image data as second image data,
When the block size of the predetermined block in the output image data matches the block size of the predetermined block in the decoded image data, the output image data is the first image data, and the output image data Higher image quality image data as the second image data,
Corresponding to the pixel of interest is a designated pixel of said first image picture data, a plurality of pixels used for the prediction computation for predicting the pixel of the second image picture data, the first images data Prediction tap extraction means (for example, the prediction tap extraction unit 131 in FIG. 4) that extracts as a prediction tap from
A plurality of pixels used for classification to classify into any of the classes of the target pixel a plurality of classes, the class tap extracting means for extracting as a class tap from the first image picture data (e.g., in FIG. 4 A class tap extraction unit 132);
Class classification means (for example, the class classification unit 133 in FIG. 4) for classifying the pixel of interest based on the class tap extracted by the class tap extraction means;
Coefficient output means (for example, coefficient storage unit 134 in FIG. 4) for outputting a coefficient corresponding to the class of the target pixel from among the coefficients corresponding to each of the plurality of classes obtained in advance by the learning;
By the prediction computation using the coefficients output by the coefficient output unit, and the prediction tap extracted by the prediction tap extraction means, corresponding to the pixel of interest, determine the pixels of the second images data And a prediction calculation means (for example, the prediction calculation unit 135 in FIG. 4).

  The decoder (for example, the decoder 32 in FIG. 1) obtains image data (for example, the reduced image in FIG. 1) obtained by reducing or enlarging the original image (for example, the original image in FIG. 1) to the predetermined number of pixels. , Decoding is performed in units of the predetermined block for the encoding result (for example, the MPEG encoded image of FIG. 1) encoded for each predetermined block, and decoded image data (for example, FIG. Image data obtained by reducing or enlarging one MPEG decoded image) to image data having the same number of pixels as the original image (for example, the enlarged MPEG decoded image in FIG. 1) is output as the output image data.

An image processing method or program according to one aspect of the present invention includes:
Output image data (for example, the decoder 32 of FIG. 1) output by the decoder (for example, the decoder 32 of FIG. 1) that decodes image data (for example, the MPEG encoded image of FIG. 1) encoded for each predetermined block in units of the predetermined blocks. Image processing method of an image processing apparatus for processing (enlarged MPEG decoded image in FIG. 1), or output image data output by a decoder that decodes image data encoded for each predetermined block in units of the predetermined block In a program for causing a computer to execute image processing of an image processing apparatus to process,
A block size of the predetermined block in the output image data is determined (for example, step S132 in FIG. 10),
The output image data is reduced or enlarged to first reduced / enlarged image data having a predetermined number of pixels (for example, step S 133 in FIG. 10 ),
And the block size of the predetermined block in the output image data, if the block size is different from the predetermined block in the decoded image data obtained by the decoder performs decoding, the decrypt image data and a predetermined coefficient And the predetermined value obtained by learning that minimizes an error between the predicted value obtained by predicting high-quality image data having a higher image quality than the decoded image data and the high-quality image data. Using the coefficient and the first reduced / enlarged image data , the first reduced / enlarged image data is converted into second reduced / enlarged image data having a higher image quality than the first reduced / enlarged image data (for example, , Step S134 in FIG.
When the block size of the predetermined block in the output image data matches the block size of the predetermined block in the decoded image data, the output image is used by using the predetermined coefficient and the output image data. The data is converted into image data with higher image quality than the output image data (for example, step S136 in FIG. 10),
The second reduced / enlarged image data is reduced or enlarged to image data having the same number of pixels as the output image data (for example, step S 135 in FIG. 10 ).
Includes steps.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  FIG. 2 is a block diagram showing an embodiment of an image processing apparatus to which the present invention is applied.

  The image processing device 61 in FIG. 2 includes an image reduction filter 101, an image conversion unit 102, and an image enlargement filter 103.

  The image processing device 61 in FIG. 2 performs image conversion processing for converting output image data such as an enlarged MPEG decoded image output from the decoder 32 in FIG. 1 into image data with higher image quality than the output image data. .

  That is, the image processing device 61 in FIG. 2 performs a noise removal process for removing block distortion and mosquito noise generated in the output image data output from the decoder 32 in FIG.

  Here, the output image data output by the decoder 32 of FIG. 1 is 1920 × 4 × 4 × 1 × 4 × 1 × 4, which is obtained directly by MPEG-2 decoding. It is a 1080-pixel enlarged MPEG decoded image (FIG. 1).

  Accordingly, in the output image data, an 8 × 8 pixel block, which is an MPEG-2 encoding and decoding processing unit (hereinafter referred to as an MPEG processing unit), is also multiplied by 4/3 in the horizontal direction to be 10.666 × It is a block of 8 pixels.

  The image processing apparatus 61 in FIG. 2 receives 1920 × 1080 pixel output image data from the decoder 32 in FIG. 1 in which an 8 × 8 pixel block in the MPEG processing unit is expanded to a 10.666 × 8 pixel block. The output image data is supplied to the image reduction filter 101.

  The image reduction filter 101 reduces the 1920 × 1080 pixel output image data supplied thereto to 1440 × 1080 pixel reduced image data by multiplying the output image data by 3/4 in the horizontal direction.

  In the reduced image data of 1440 × 1080 pixels obtained by the output image data of 1920 × 1080 pixels being multiplied by 3/4 in the horizontal direction in the image reduction filter 101, 10.666 × 8 pixels of the output image data This block is also multiplied by 3/4 in the horizontal direction to form an 8 × 8 pixel block in units of MPEG processing.

  The image reduction filter 101 supplies reduced image data of 1440 × 1080 pixels obtained by reducing the output image data of 1920 × 1080 pixels to the image conversion unit 102.

  The image conversion unit 102 removes noise such as block distortion generated at the boundary of an 8 × 8 pixel block of the MPEG processing unit from the MPEG decoded image directly obtained by decoding by MPEG-2. An image conversion process for converting to a removed image is performed.

  That is, the image conversion unit 102 performs image conversion processing on the reduced image data of 1440 × 1080 pixels supplied from the image reduction filter 101, and the reduced image data of 1440 × 1080 pixels from the image reduction filter 101 The image data is converted into high-quality reduced image data of 1440 × 1080 pixels that has higher image quality than the reduced image data, and is supplied to the image enlargement filter 103.

  The image enlargement filter 103 multiplies the 1440 × 1080 pixel high-quality reduced image data supplied from the image conversion unit 102 by a factor of 4/3 in the horizontal direction, thereby having the same number of pixels as the output image data, 1920 × The image data is enlarged and output to 1080 pixel image data.

  For example, a Lanczos filter can be employed as the image reduction filter 101 that reduces an image and the image enlargement filter 103 that enlarges an image.

  The filter coefficient Lanczos (x) of the Lanczos filter is expressed by the following equation (1), where x is the horizontal position of the pixel relative to the pixel to be filtered.

・・・（１）

... (1)

  Here, FIG. 3 shows the filter coefficient Lanczos (x) of the Lanczos filter expressed by the equation (1).

  Next, FIG. 4 is a block diagram illustrating a detailed configuration example of the image conversion unit 102 of FIG.

  The image conversion unit 102 includes a prediction tap extraction unit 131, a class tap extraction unit 132, a class classification unit 133, a coefficient storage unit 134, and a prediction calculation unit 135, and is described in, for example, Japanese Patent Application Laid-Open No. 2003-324738. Thus, an image conversion process for converting the MPEG decoded image into a noise-removed image is performed.

  The prediction tap extraction unit 131 and the class tap extraction unit 132 receive 1440 × 1080 pixel blocks from the image reduction filter 101 in FIG. 2 in which the block of the MPEG processing unit is the same 8 × 8 pixel block as the MPEG decoded image. Reduced image data is supplied.

  The prediction tap extraction unit 131 sequentially sets the 8 × 8 pixel block of the reduced image data supplied from the image reduction filter 101 as a target block, and sequentially sets each pixel of the target block in, for example, a so-called raster scan order. The pixel of interest.

  In addition, the prediction tap extraction unit 131 selects a pixel corresponding to the target pixel from among pixels of high-quality reduced image data corresponding to the noise-removed image obtained by removing noise from the reduced image data corresponding to the target pixel. ) Are extracted as prediction taps from the reduced image data.

  That is, the prediction tap extraction unit 131, for example, out of the pixel of the target block, the pixel adjacent to the target pixel, the upper, lower, left, right, upper left, lower left, upper right, and lower right pixels of the target pixel, Of the pixels adjacent to the boundary of the target block in the block adjacent above the block, the pixels in the same column as the target pixel, the pixel adjacent to the boundary of the target block in the block adjacent below the target block Of the pixels in the same column as the target pixel, the pixels adjacent to the boundary of the target block in the block adjacent to the left of the target block, the pixel in the same row as the target pixel, and to the right of the target block Among the pixels adjacent to the boundary of the target block in the adjacent block, the pixels in the same row as the target pixel, the pixels in the frame before and after the target pixel frame, etc., in the same position as the target pixel As prediction taps.

  The prediction tap extraction unit 131 supplies the prediction tap for the target pixel to the prediction calculation unit 135.

  The class tap extraction unit 132 classifies a plurality of pixels used for class classification for classifying the target pixel into one of a plurality of classes from the reduced image data supplied from the image reduction filter 101. Extract as

  That is, the class tap extraction unit 132, for example, among the pixels of the target block, the target pixel, seven pixels other than the target pixel in the same column as the target pixel, and other than the target pixel in the same row as the target pixel In addition, among the pixels adjacent to the boundary of the target block of the block adjacent above the target block, the pixel in the same column as the target pixel, the target of the block adjacent below the target block Of the pixels adjacent to the block boundary, the pixel in the same column as the target pixel, and the pixel adjacent to the target block boundary of the block adjacent to the left of the target block are in the same row as the target pixel. Among the pixels adjacent to the boundary of the target block of the pixel adjacent to the right of the target block, the pixels in the same row as the target pixel are extracted as class taps.

  Note that the prediction tap and the class tap may be a plurality of pixels having the same tap structure (positional relationship with respect to the target pixel), or may be a plurality of pixels having different tap structures.

  The class tap extraction unit 132 supplies the class tap of the target pixel to the class classification unit 133.

The class classification unit 133 classifies the target pixel based on the class tap from the class tap extraction unit 132, and outputs a class code corresponding to the class obtained as a result.

  Here, as a method of classifying, for example, ADRC (Adaptive Dynamic Range Coding) or the like can be employed.

  In the method using ADRC, the pixel values of the pixels constituting the class tap are subjected to ADRC processing, and the class of the target pixel is determined according to the ADRC code obtained as a result.

Note that, in the K-bit ADRC, for example, the maximum value MAX and the minimum value MIN of the pixels constituting the class tap are detected, and DR = MAX−MIN is set as a local value of the set of pixels constituting the class tap. Based on this dynamic range DR, the pixels (the pixel values) constituting the class tap are requantized to K bits. That is, the pixel value of each pixel forming the class taps, the minimum value MIN is subtracted, and the subtracted value is divided (quantized) by DR / 2 ^K. A bit string obtained by arranging the pixel values of the K-bit pixels constituting the class tap in a predetermined order is output as an ADRC code. Therefore, for example, when the class tap is subjected to 1-bit ADRC processing, the pixel value of each pixel constituting the class tap is the difference between the maximum value MAX and the minimum value MIN after the minimum value MIN is subtracted. The pixel value of each pixel is set to 1 bit (binarized). Then, a bit string in which the 1-bit pixel values are arranged in a predetermined order is output as an ADRC code.

  In addition, in the class classification unit 133, for example, an absolute value of a difference between pixel values of adjacent pixels among the pixels constituting the class tap (hereinafter, referred to as an inter-pixel difference) is obtained, and the difference between each pixel is determined. It is possible to classify a bit string in which 1-bit flags representing magnitudes are arranged as many as the number of pixel differences as a class code.

  The coefficient storage unit 134 stores a set of tap coefficients for each class obtained in advance by learning described later, and further, a class code supplied from the class classification unit 133 in the stored set of tap coefficients. The tap coefficient stored in the address corresponding to (the tap coefficient corresponding to the class of the pixel of interest represented by the class code supplied from the class classification unit 133) is output.

  The prediction calculation unit 135 acquires the prediction tap output from the prediction tap extraction unit 131 and the tap coefficient output from the coefficient storage unit 134, and uses the prediction tap and the tap coefficient to correspond to the target pixel. A predetermined prediction calculation is performed to obtain the pixel value (predicted value) of the pixel of the image quality reduced image data.

  And the prediction calculation part 135 outputs the pixel value of the pixel of the high quality reduced image data obtained by performing a predetermined prediction calculation.

  Next, image processing in the image processing apparatus 61 in FIG. 2 will be described with reference to the flowchart in FIG.

  Output image data output from the decoder 32 of FIG. 1 is supplied to the image reduction filter 101 of the image processing device 61 of FIG. At this time, the image reduction filter 101 outputs 1920 × 1080 pixel output image data in which the MPEG processing unit 8 × 8 pixel block is enlarged to 10.666 × 8 pixel block from the decoder 32 of FIG. Is supplied.

  In step S31, the image reduction filter 101 reduces the output image data of 1920 × 1080 pixels from the decoder 31 of FIG. 1 by 3/4 in the horizontal direction to reduce it to reduced image data of 1440 × 1080 pixels. In the reduced image data of 1440 × 1080 pixels, the 10.666 × 8 pixel block of the MPEG processing unit becomes an 8 × 8 pixel block.

  The image reduction filter 101 supplies the reduced image data of 1440 × 1080 pixels obtained by reducing the output image data of 1920 × 1080 pixels to the image conversion unit 102, and the processing proceeds from step S31 to step S32. move on.

  In step S32, the image conversion unit 102 performs image conversion processing on the reduced image data of 1440 × 1080 pixels supplied from the image reduction filter 101, thereby converting the reduced image data of 1440 × 1080 pixels into block distortion or the like. It is converted into high-quality reduced image data of 1440 × 1080 pixels from which noise has been removed, and is supplied to the image enlargement filter 103.

  After the process of step S32 is completed, the process proceeds to step S33, and the image enlargement filter 103 multiplies the 1440 × 1080 pixel high-quality reduced image data supplied from the image conversion unit 102 by 4/3 in the horizontal direction. Thus, the image data is enlarged to 1920 × 1080 pixel image data having the same number of pixels as the output image data, the image data is output, and the process ends.

  Next, the image conversion process in step S32 in FIG. 5 performed by the image conversion unit 102 in FIG. 2 will be described with reference to the flowchart in FIG.

  In step S61, the prediction tap extraction unit 131 sequentially selects, from the reduced image data supplied from the image reduction filter 101, blocks in the reduced image data, that is, blocks of 8 × 8 pixels in the MPEG processing unit in the reduced image data. , A target block, and each pixel of the target block is sequentially set as a target pixel in the raster scan order.

  After the process of step S61 ends, the process proceeds to step S62, and the prediction tap extraction unit 131 extracts a plurality of pixels that are prediction taps of the target pixel from the reduced image data from the image reduction filter 101, and performs a prediction calculation. The process proceeds to step S63.

  In step S63, the class tap extraction unit 132 extracts a plurality of pixels as the class tap of the target pixel from the reduced image data supplied from the image reduction filter 101, and supplies the extracted pixels to the class classification unit 133. Proceed to step S64.

  In step S64, the class classification unit 133 classifies the pixel of interest based on the class tap from the class tap extraction unit 132, and outputs a class code corresponding to the resulting class to the coefficient storage unit 134. The process proceeds to step S65.

  In step S65, the coefficient storage unit 134 stores the tap coefficient (attention) stored in the address corresponding to the class code supplied from the class classification unit 133 among the tap coefficients for each class obtained in advance by learning described later. The tap coefficient corresponding to the pixel class) is output to the prediction calculation unit 135.

  Thereafter, after the process of step S65 ends, the process proceeds to step S66, and the prediction calculation unit 135 acquires the prediction tap output by the prediction tap extraction unit 131 and the tap coefficient output by the coefficient storage unit 134, Using the prediction tap and the tap coefficient, a predetermined prediction calculation for obtaining the pixel value of the pixel of the high-quality reduced image data corresponding to the target pixel is performed. Further, in step S66, the prediction calculation unit 135 outputs the pixel value of the pixel of the high-quality reduced image data obtained by performing the predetermined prediction calculation, and the process proceeds to step S67.

  In step S <b> 67, the prediction tap extraction unit 131 determines whether or not all the pixels of the target block are the target pixels.

  If it is determined in step S67 that all the pixels of the target block are not the target pixel, the process returns to step S61, and the prediction tap extraction unit 131 sets the target pixel among the pixels of the target block. The process proceeds to step S62 with a pixel that has not been processed as a target pixel, and the same process is repeated thereafter.

  On the other hand, if it is determined in step S67 that all the pixels of the target block are the target pixels, the process proceeds to step S68.

  In step S <b> 68, the prediction tap extraction unit 131 determines whether all the 8 × 8 pixel blocks of the reduced image data supplied from the image reduction filter 101 have been used as the target block.

  If it is determined in step S68 that all the 8 × 8 pixel blocks of the reduced image data supplied from the image reduction filter 101 are not the target block, the process returns to step S61, and the prediction tap extraction unit 131 From the reduced image data supplied from the image reduction filter 101, a block that has not yet been selected as a target block is newly selected as a target block, and a target pixel is selected from the pixels of the target block. In step S62, the same processing is repeated thereafter.

  On the other hand, if it is determined in step S68 that all the 8 × 8 pixel blocks of the reduced image data supplied from the image reduction filter 101 are the target blocks, the process returns to step S32 in FIG.

  In the image processing apparatus 61 of FIG. 2, as described above, output image data of 1920 × 1080 pixels in which an 8 × 8 pixel block of the MPEG processing unit is a block of 10.666 × 8 pixels is horizontally displayed. By multiplying by 3/4, the 10.666 × 8 pixel block of the MPEG processing unit is reduced to reduced image data of 1440 × 1080 pixels, which becomes an 8 × 8 pixel block, and the reduced image data is processed as MPEG. Image conversion processing for converting the decoded image into a noise-removed image from which noise such as block distortion has been removed is performed to obtain high-quality reduced image data of 1440 × 1080 pixels. Therefore, an 8 × 8 pixel block in the MPEG processing unit is obtained. Effectively removes noise such as block distortion that occurs at the boundary of the 10.666 × 8 pixel block from the output image data that is a 10.666 × 8 pixel block Can.

  Next, prediction calculation in the prediction calculation unit 135 in FIG. 4 and learning of tap coefficients stored in the coefficient storage unit 134 will be described.

  Now, a prediction tap is extracted from an MPEG decoded image decoded in block units of 8 × 8 pixels, and noise such as block distortion is removed from the MPEG decoded image using the prediction tap and tap coefficient. Consider obtaining (predicting) a pixel value of a pixel of a removed image by a predetermined prediction calculation.

  As the predetermined prediction calculation, for example, when linear primary prediction calculation is adopted, a pixel value y of a pixel of a high-quality noise-removed image (hereinafter, appropriately referred to as a high-quality pixel) is expressed by the following linear linear expression Will be required.

・・・（２）

... (2)

However, in Expression (2), x _n represents the pixel value of the pixel of the n-th MPEG decoded image (hereinafter referred to as a decoded pixel as appropriate) that constitutes the prediction tap for the high-quality pixel y, and w _n is It represents the nth tap coefficient multiplied by the nth decoded pixel (the pixel value thereof). In equation (2), the prediction tap is assumed to be composed of _N decoded pixels x ₁ , x ₂ ,..., X _N.

  Here, the pixel value y of the high-quality pixel can be obtained not by the linear primary expression shown in Expression (2) but by a higher-order expression of the second or higher order.

Now, when the true value of the pixel value of the high-quality pixel of the k-th sample is expressed as y _k and the predicted value of the true value y _k obtained by Expression (2) is expressed as y _k ′, the prediction error e _k Is expressed by the following equation.

・・・（３）

... (3)

Now, since the predicted value y _k ′ of equation (3) is obtained according to equation (2), the following equation is obtained by replacing y _k ′ of equation (3) according to equation (2).

・・・（４）

... (4)

In Equation (4), x _{n, k} represents the nth decoded pixel that constitutes the prediction tap for the high quality pixel of the kth sample.

Tap coefficient w _n for the prediction error e _k 0 of the formula (4) (or formula (3)) is, is the optimal for predicting the high-quality pixel, for all the high-quality pixel, such In general, it is difficult to obtain a large tap coefficient w _n .

Therefore, as the standard for indicating that the tap coefficient w _n is optimal, for example, when adopting the method of least squares, optimal tap coefficient w _n, the sum E of square errors expressed by the following formula It can be obtained by minimizing.

・・・（５）

... (5)

However, in Equation (5), K is a decoded pixel x _{1, k} , x _{2, k} ,..., X _N, which constitutes a prediction tap for the high quality pixel y _k and the high quality pixel y _{k . This represents the} number of samples in the set with _k (the number of samples for learning).

The minimum value of the sum E of square errors of Equation (5) (minimum value), as shown in equation (6) is given a material obtained by partially differentiating the sum E with the tap coefficient w _n by w _n to 0.

・・・（６）

... (6)

On the other hand, when the partial differentiation of the above formula (4) with the tap coefficient w _n, the following equation is obtained.

・・・（７）

... (7)

  From the equations (6) and (7), the following equation is obtained.

・・・（８）

... (8)

By substituting equation (4) into e _k in equation (8), equation (8) can be expressed by the normal equation shown in equation (9).

・・・（９）

... (9)

Normal equation of Equation (9), for example, by using a like sweeping-out method (Gauss-Jordan elimination method) can be solved for the tap coefficient w _n.

The normal equation of Equation (9), by solving for each class, the optimal tap coefficient (here, the tap coefficient that minimizes the sum E of square errors) to w _n, can be found for each class .

  In the image conversion unit 102 in FIG. 4, the reduced image data of 1440 × 1080 pixels is converted into high image quality of 1440 × 1080 pixels by performing the calculation of Expression (2) using the tap coefficients for each class as described above. Converted to reduced image data.

Next, FIG. 7 is a block diagram showing a configuration example of a learning apparatus that performs learning for determining the tap coefficient w _n by solving the normal equations in equation (9) for each class.

  The learning device 161 includes a learning image storage unit 191, a noise addition unit 192, a prediction tap extraction unit 193, a class tap extraction unit 194, a class classification unit 195, an addition unit 196, and a tap coefficient calculation unit 197.

The learning image storage unit 191, the learning image is stored as an image used for learning of the tap coefficient w _n.

  Here, the learning image is a teacher image that is an image corresponding to the above-described noise-removed image.

  The noise adding unit 192 reads out the teacher image from the learning image storage unit 191, MPEG-encodes the teacher image, and further performs MPEG decoding so that the image conversion unit 102 is an object of the image conversion process. A student image which is an image corresponding to an MPEG decoded image in which the block is an 8 × 8 pixel block is generated.

  In addition, the noise adding unit 192 supplies the student image to the prediction tap extracting unit 193 and the class tap extracting unit 194.

  The prediction tap extraction unit 193 sequentially selects the 8 × 8 pixel block of the student image supplied from the noise addition unit 192 as a target block, and further selects each pixel of the target block in the raster scan order, for example. Sequentially, the pixel of interest is selected.

  Furthermore, the prediction tap extraction unit 193 extracts a predetermined one of the pixels constituting the student image supplied from the noise addition unit 192 for the target pixel, so that the prediction tap extraction unit 131 in FIG. A prediction tap having the same tap structure as the prediction tap obtained for the prediction tap (prediction tap composed of pixels of the student image having the same positional relationship as the pixels constituting the prediction tap obtained by the prediction tap extraction unit 131 in FIG. 4 for the target pixel) is obtained. Then, it is supplied to the adding portion 196.

  The class tap extraction unit 194 extracts a predetermined pixel among the pixels constituting the student image supplied from the noise addition unit 192 for the target pixel, so that the class tap extraction unit 132 in FIG. 4 obtains the target pixel. Obtaining a class tap having the same tap structure as the class tap (a class tap made up of student image pixels having the same positional relationship as the pixels constituting the class tap obtained by the class tap extraction unit 132 of FIG. 4 for the target pixel), The data is supplied to the class classification unit 195.

  The class classification unit 195 performs the same class classification as the class classification unit 133 in FIG. 4 based on the class tap output from the class tap extraction unit 194, and adds the class code corresponding to the resulting class to the addition unit 196. Output to.

  The adding unit 196 reads out the teacher image from the learning image storage unit 191, the pixel of the teacher image corresponding to the pixel of interest (hereinafter referred to as “teacher pixel” as appropriate), and the pixel of interest supplied from the prediction tap extraction unit 193. For each class code supplied from the class classification unit 195, addition is performed on the pixel of the student image (hereinafter referred to as “student pixel” as appropriate) constituting the prediction tap.

That is, of the teacher image pixels stored in the learning image storage unit 191, the teacher pixel (its pixel value) y _k corresponding to the target pixel and the prediction tap extraction unit 193 output to the adding unit 196. The student pixel (pixel value) x _{n, k} constituting the prediction tap and the class code representing the class of the pixel of interest output from the class classification unit 195 are supplied.

Then, the adding unit 196 uses the prediction tap (student pixel) x _{n, k} for each class corresponding to the class code supplied from the class classification unit 195 _, and uses the student pixels in the matrix on the left side of Equation (9). (X _{n, k} x _{n ′, k} ) and computation corresponding to summation (Σ).

Furthermore, the adding unit 196 also uses the prediction tap (student pixel) x _{n, k} and the teacher pixel y _k for each class corresponding to the class code supplied from the class classification unit 195, and the equation (9). Multiplication (x _{n, k} y _k ) of the student pixel x _{n, k} and teacher pixel y _k in the right-hand vector and an operation corresponding to summation (Σ) are performed.

That is, the adding unit 196 performs the left-side matrix component (Σx _{n, k} x _{n ′, k} ) and the right-side vector component (Σx _{n, k} ) in Equation (9) previously obtained for the target _pixel. y _k ) is stored in its built-in memory (not shown), and the matrix component (Σx _{n, k} x _{n ′, k} ) or vector component (Σx _{n, k} y _k ) The corresponding component x _{n, k + 1} x _{n ′, k + 1} or x _{n, k} calculated for the new pixel of interest using the teacher pixel y _{k + 1} and the student pixel x _{n, k + 1 +1} y _{k + 1} is added respectively (addition represented by the summation of Expression (9) is performed).

  Then, the addition unit 196 performs the above-described addition using all the student pixels of the student images generated from the teacher images stored in the learning image storage unit 191 as the target pixel, thereby obtaining an expression ( When the normal equation shown in 9) is established, the normal equation is supplied to the tap coefficient calculation unit 197.

Tap coefficient calculation unit 197 solves the normal equations for each class supplied from the adder 196, for each class, and outputs the determined optimal tap coefficient w _n.

  Next, the learning process in the learning device 161 in FIG. 7 will be described with reference to the flowchart in FIG.

  In step S <b> 101, the noise adding unit 192 reads out the teacher image from the learning image storage unit 191, MPEG-encodes the teacher image, and further performs MPEG decoding so that the image conversion unit 102 can perform image conversion processing. Then, a student image that is an image corresponding to an MPEG decoded image in which the block of the MPEG processing unit is an 8 × 8 pixel block is generated.

  The noise adding unit 192 supplies the student image to the prediction tap extracting unit 193 and the class tap extracting unit 194, and the process proceeds from step S101 to step S102.

  In step S102, the prediction tap extraction unit 193 sequentially selects the 8 × 8 pixel block of the student image supplied from the noise addition unit 192 as a target block, and further selects each pixel of the target block as, for example, The pixels are selected as the target pixel sequentially in the raster scan order, and the process proceeds to step S103.

  In step S <b> 103, the prediction tap extraction unit 193 extracts a predetermined one of the pixels constituting the student image supplied from the noise addition unit 192 for the target pixel, so that the prediction tap extraction unit 131 of FIG. A prediction tap having the same tap structure as the prediction tap obtained for the target pixel is obtained and supplied to the adding unit 196.

  After the process of step S103 ends, the process proceeds to step S104, and the class tap extraction unit 194 extracts a predetermined one of the pixels constituting the student image supplied from the noise addition unit 192 for the target pixel. As a result, the class tap having the same tap structure as the class tap obtained by the class tap extraction unit 132 in FIG. 4 for the target pixel is obtained and supplied to the class classification unit 195.

  Thereafter, the process proceeds from step S104 to step S105, and the class classification unit 195 performs the same class classification as the class classification unit 133 in FIG. 4 based on the class tap output by the class tap extraction unit 194, and obtains the result. The class code corresponding to the class to be output is output to the adding unit 196, and the process proceeds to step S106.

  In step S <b> 106, the adding unit 196 reads the teacher image from the learning image storage unit 191, and configures the teacher pixel corresponding to the target pixel and the prediction tap for the target pixel supplied from the prediction tap extraction unit 193. The addition for pixels is performed for each class code supplied from the class classification unit 195, and the process proceeds to step S107.

  In step S107, the prediction tap extraction unit 193 determines whether or not all the pixels of the target block are the target pixels.

  If it is determined in step S107 that all the pixels of the target block are not the target pixels, the process returns to step S102, and the prediction tap extraction unit 193 determines that the target pixel is still the target pixel among the pixels of the target block. The process proceeds to step S103 with the pixel that has not been processed as the target pixel, and the same process is repeated thereafter.

  On the other hand, when it is determined in step S107 that all pixels of the target block are the target pixels, the process proceeds to step S108.

  In step S <b> 108, the prediction tap extraction unit 193 determines whether all the 8 × 8 pixel blocks of the student image supplied from the noise addition unit 192 have been set as the target block.

  If it is determined in step S108 that all the 8 × 8 pixel blocks of the student image supplied from the noise adding unit 192 are not the target block, the process returns to step S102, and the prediction tap extracting unit 193 From the student image supplied from the adding unit 192, a block not yet selected as a target block is newly selected as a target block, and a target pixel is selected from the pixels of the target block. The same processing is repeated thereafter.

  On the other hand, in step S108, when it is determined that all the 8 × 8 pixel blocks of the student image supplied from the noise adding unit 192 are the target blocks, that is, in the adding unit 196, the learning image storage unit When the normal equation shown in Expression (9) is established for each class by performing the above addition using all the student pixels of the student image generated from the teacher image stored in 191 as the target pixel. The adding unit 196 supplies the normal equation to the tap coefficient calculating unit 197, and the process proceeds to step S109.

In step S109, the tap coefficient calculation unit 197 obtains and outputs the optimum tap coefficient w _n for each class by solving the normal equation for each class supplied from the addition unit 196, and the process ends. To do.

  FIG. 9 is a block diagram showing another embodiment of an image processing apparatus to which the present invention is applied.

  In the figure, portions corresponding to those in FIG. 2 are denoted by the same reference numerals, and description thereof will be omitted as appropriate.

  That is, the image processing apparatus 221 in FIG. 9 is provided with a block size detection unit 251, a block size determination unit 252, and an image conversion unit 253 that performs the same processing as the image conversion unit 102 in FIG. The configuration is the same as that of the image processing apparatus 61 of FIG.

  The image processing apparatus 221 in FIG. 9 is supplied with output image data output from a decoder that performs MPEG decoding in units of 8 × 8 pixel blocks.

  Here, a decoder that performs MPEG decoding includes a decoder that outputs an MPEG decoded image directly obtained by MPEG decoding as output image data, and an MPEG decoded image that is directly obtained by MPEG decoding as in the decoder 32 of FIG. There is a decoder that outputs, as output image data, an enlarged MPEG decoded image in which an 8 × 8 pixel block in an MPEG processing unit is a 10.666 × 8 pixel block.

  Output image data supplied from the decoder to the image processing device 221 is supplied to the block size detection unit 251.

  The block size detection unit 251 detects the block size of the block in the output image data supplied thereto, and supplies the block size determination unit 252 together with the output image data.

  The block size determination unit 252 acquires the output image data supplied from the block size detection unit 251 and the block size of the block in the output image data.

  Further, the block size determination unit 252 matches the block size of the block in the output image data acquired from the block size detection unit 251 with the block size of the 8 × 8 pixel block in the MPEG decoded image, or the expanded MPEG decoding. It is determined whether or not it matches the block size of a 10.666 × 8 pixel block in the image.

  When the block size of the block in the output image data matches the block size of the 8 × 8 pixel block in the MPEG decoded image, the block size determination unit 252 performs image conversion on the output image data acquired from the block size detection unit 251. Supplied to the unit 253.

  On the other hand, when the block size of the block in the output image data matches the block size of the block of 10.666 × 8 pixels in the enlarged MPEG decoded image, the block size determination unit 252 outputs the output image acquired from the block size detection unit 251. Data is supplied to the image reduction filter 101.

  Similar to the image conversion unit 102 in FIG. 2, the image conversion unit 253 performs an image conversion process on the output image data from the block size determination unit 252, that is, the MPEG decoded image, so that the block size determination unit 252. Is converted into high-quality image data from which noise such as block distortion has been removed from the output image data.

  Next, image processing performed by the image processing apparatus 221 in FIG. 9 will be described with reference to the flowchart in FIG.

  In step S131, the block size detection unit 251 detects the block size of the block in the output image data supplied thereto, and supplies the block size determination unit 252 together with the output image data.

  The block size determination unit 252 acquires the output image data supplied from the block size detection unit 251 and the block size of the block in the output image data, and the process proceeds from step S131 to step S132.

  In step S132, the block size determination unit 252 determines that the block size of the block in the output image data acquired from the block size detection unit 251 is the block size of an 8 × 8 pixel block in the MPEG decoded image or the expanded MPEG decoded image. It is determined whether or not it matches the block size of the 10.666 × 8 pixel block.

  In step S132, when the block size of the block in the output image data matches the block size of the block of 10.666 × 8 pixels in the enlarged MPEG decoded image, the block size determination unit 252 acquires from the block size detection unit 251. The output image data is supplied to the image reduction filter 101, and the process proceeds to step S133. In steps S133 to S135, the same processes as in steps S31 to S33 in FIG. 5 are performed, and the process ends. .

  On the other hand, in step S132, when the block size of the block in the output image data matches the block size of the 8 × 8 pixel block in the MPEG decoded image, the block size determination unit 252 outputs the output acquired from the block size detection unit 251. The image data is supplied to the image conversion unit 253, and the process proceeds to step S136.

  In step S136, the image conversion unit 253 performs image conversion processing on the output image data from the block size determination unit 252, that is, the MPEG decoded image, in the same manner as the image conversion unit 102 in FIG. The output image data supplied from the size determination unit 252 is converted into high-quality image data from which noise such as block distortion has been removed from the output image data, and the processing ends.

  As described above, the image processing apparatus 221 in FIG. 9 determines the block size of the block in the output image data, and if the block size of the block in the output image data matches the block size of the block in the MPEG decoded image, If the output image data is subjected to image conversion processing as it is and is different, that is, if it matches the block size of the block in the enlarged MPEG decoded image, the output image data is reduced, and the reduced image is subjected to image conversion processing. Since the output image data is either an MPEG decoded image or an enlarged MPEG decoded image, noise such as block distortion and mosquito noise can be effectively removed.

  In FIG. 9, in addition to the image conversion unit 102, an image conversion unit 253 that performs the same image conversion processing as the image conversion unit 102 is provided. However, the block size of the block in the output image data is expanded MPEG decoding. When the block size of the block of 10.666 × 8 pixels in the image matches, the output image data is supplied from the block size determination unit 252 to the image reduction filter 101, and the block size of the block in the output image data is MPEG decoded. When the block size of the block of 8 × 8 pixels in the image matches, the output image data is supplied from the block size determination unit 252 to the image conversion unit 102, bypassing the image reduction filter 101, and image conversion is performed. The image data obtained as a result of the image conversion processing of the unit 102 is converted into an image enlargement filter 103. By bypassing to so as to output, without providing the image converting unit 253, it is possible to configure the image processing apparatus 221.

  11 and 12 show a case where image conversion processing by the image conversion unit 102 is performed on output image data having a block size of 10.666 × 8, that is, an enlarged MPEG decoded image, and an image by the image processing device 61. It is a figure which shows the result of the simulation with the case where a process is performed.

  In the simulation, original images pic1 to pic7 of seven contents are prepared, and the original images pic1 to pic7 are processed as described with reference to FIG. 1 to obtain an enlarged MPEG decoded image as output image data. Thus, the image conversion processing by the image conversion unit 102 and the image processing by the image processing apparatus 61 are set as targets.

  The simulation was performed on two types of enlarged MPEG decoded images with a bit rate of 12 Mbps (mega bits per second) and 15 Mbps.

  FIG. 11 illustrates an image obtained by performing image conversion processing by the image conversion unit 102 and image processing by the image processing device 61 (compared with an original image) for an enlarged MPEG decoded image having a bit rate of 12 Mbps. ) Indicates SNR (signal to noise ratio).

  FIG. 12 illustrates an image obtained by performing image conversion processing by the image conversion unit 102 and image processing by the image processing device 61 for an enlarged MPEG decoded image having a bit rate of 15 Mbps (compared with the original image). ) Indicates SNR.

  From FIG. 11 and FIG. 12, the image processing by the image processing device 61 can obtain an image with a higher SNR (higher image quality) than the image conversion processing by the image conversion unit 102. Therefore, block distortion and mosquito It can be seen that noise such as noise is effectively removed.

  Next, the series of processing performed by the image processing device 61 in FIG. 2, the image processing device 221 in FIG. 9, and the learning device 161 in FIG. 7 can be executed by dedicated hardware or executed by software. It can also be made. When a series of processing is executed by software, a program constituting the software can execute various functions by installing a so-called embedded computer or various programs. For example, it is installed from a program storage medium in a general-purpose computer or the like.

  FIG. 13 is a block diagram illustrating a configuration example of a computer that executes the above-described series of processing by a program.

  A CPU (Central Processing Unit) 701 executes various processes according to a program stored in a ROM (Read Only Memory) 702 or a storage unit 708. A RAM (Random Access Memory) 703 appropriately stores programs executed by the CPU 701, data, and the like. These CPU 701, ROM 702, and RAM 703 are connected to each other by a bus 704.

  An input / output interface 705 is also connected to the CPU 701 via a bus 704. Connected to the input / output interface 705 are an input unit 706 composed of a keyboard, mouse, microphone, and the like, and an output unit 707 composed of a display, a speaker, and the like. The CPU 701 executes various processes in response to commands input from the input unit 706. Then, the CPU 701 outputs the processing result to the output unit 707.

  A storage unit 708 connected to the input / output interface 705 includes, for example, a hard disk, and stores programs executed by the CPU 701 and various data. A communication unit 709 communicates with an external device via a network such as the Internet or a local area network.

  A program may be acquired via the communication unit 709 and stored in the storage unit 708.

  A drive 710 connected to the input / output interface 705 drives a removable medium 711 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, and drives the program or data recorded therein. Get etc. The acquired program and data are transferred to and stored in the storage unit 708 as necessary.

  As shown in FIG. 13, a program storage medium that stores a program that is installed in a computer and can be executed by the computer includes a magnetic disk (including a flexible disk), an optical disk (CD-ROM (Compact Disc-Read Only). Memory), DVD (including Digital Versatile Disc)), magneto-optical disc (including MD (Mini-Disc)), or removable media 711 which is a package medium made of semiconductor memory or the like, or the program is temporary or permanent ROM 702 stored in the hard disk, a hard disk constituting the storage unit 708, and the like. The program is stored in the program storage medium using a wired or wireless communication medium such as a local area network, the Internet, or digital satellite broadcasting via a communication unit 709 that is an interface such as a router or a modem as necessary. Done.

  In the present specification, the step of describing the program stored in the program storage medium is not limited to the processing performed in time series according to the described order, but is not necessarily performed in time series. Or the process performed separately is also included.

  Here, the image processing device 61 in FIG. 2 and the image processing device 221 in FIG. 9 are devices that handle output image data output by a decoder that performs decoding in predetermined block units such as MPEG decoding, for example, a television, for example. The present invention can be applied to John receivers, hard disk recorders, DVD recorders, and the like.

  The present embodiment is not limited to the above-described embodiment, and various modifications can be made without departing from the gist of the present invention.

It is a figure explaining the process performed in terrestrial digital broadcasting. It is a block diagram which shows the image processing apparatus 61 to which this invention is applied. It is a figure which shows the Lanczos filter used when enlarging or reducing an image. FIG. 3 is a block diagram illustrating a detailed configuration example of an image conversion unit 102 in FIG. 2. 6 is a flowchart for explaining image processing in the image processing apparatus 61 in FIG. 2. 3 is a flowchart for describing image conversion processing performed by an image conversion unit 102 in FIG. 2. It is a block diagram showing a learning device 161 that performs learning for determining a tap coefficient w _n. It is a flowchart explaining the learning process in the learning apparatus 161 of FIG. It is a block diagram which shows the image processing apparatus 221 to which this invention is applied. 10 is a flowchart illustrating image processing performed by the image processing apparatus 221 in FIG. 9. It is a figure which shows the result of simulation. It is another figure which shows the result of simulation. It is a block diagram which shows the structural example of a computer.

Explanation of symbols

  61 image processing device, 101 image reduction filter, 102 image conversion unit, 103 image enlargement filter, 131 prediction tap extraction unit, 132 class tap extraction unit, 133 class classification unit, 134 coefficient storage unit, 135 prediction calculation unit, 161 learning device , 191 Learning image storage unit, 192 Noise addition unit, 193 Prediction tap extraction unit, 194 Class tap extraction unit, 195 Class classification unit, 196 Addition unit, 197 Tap coefficient calculation unit, 221 Image processing device, 251 Block size detection Section, 252 block size determination section, 253 image conversion section

Claims (5)

In an image processing apparatus that processes output image data output from a decoder that decodes image data encoded in predetermined block units in predetermined block units,
Determination means for determining a block size of the predetermined block in the output image data;
First reduction / enlargement means for reducing or enlarging the output image data to first reduced / enlarged image data having a predetermined number of pixels;
And the block size of the predetermined block in the output image data, if the block size is different from the predetermined block in the decoded image data obtained by the decoder performs decoding, the decrypt image data and a predetermined coefficient And the predetermined value obtained by learning that minimizes an error between the predicted value obtained by predicting high-quality image data having a higher image quality than the decoded image data and the high-quality image data. Using the coefficient and the first reduced / enlarged image data , the first reduced / enlarged image data is converted into second reduced / enlarged image data with higher image quality than the first reduced / enlarged image data ,
When the block size of the predetermined block in the output image data matches the block size of the predetermined block in the decoded image data, the output image is used by using the predetermined coefficient and the output image data. Image conversion means for converting data into image data with higher image quality than the output image data ;
An image processing apparatus comprising: a second reduction / enlargement unit that reduces or enlarges the second reduced / enlarged image data to image data having the same number of pixels as the output image data. The image conversion means converts the first image data into second image data having a higher image quality than the first image data,
When the block size of the predetermined block in the output image data is different from the block size of the predetermined block in the decoded image data, the first reduced / enlarged image data is set as first image data, and the first 2 reduced and enlarged image data as second image data,
When the block size of the predetermined block in the output image data matches the block size of the predetermined block in the decoded image data, the output image data is the first image data, and the output image data Higher image quality image data as the second image data,
Corresponding to the pixel of interest is a designated pixel of said first image picture data, a plurality of pixels used for the prediction computation for predicting the pixel of the second image picture data, the first images data A prediction tap extracting means for extracting as a prediction tap from:
A plurality of pixels used for classification to classify into any of the classes of the target pixel a plurality of classes, and class tap extracting means for extracting as a class tap from the first images data,
Class classification means for classifying the pixel of interest based on the class tap extracted by the class tap extraction means;
Coefficient output means for outputting a coefficient corresponding to the class of the target pixel from the coefficients corresponding to each of the plurality of classes obtained in advance by the learning;
By the prediction computation using the coefficients output by the coefficient output unit, and the prediction tap extracted by the prediction tap extraction means, corresponding to the pixel of interest, determine the pixels of the second images data The image processing apparatus according to claim 1, further comprising: a prediction calculation unit. The decoder performs decoding in units of the predetermined blocks with respect to an encoding result obtained by encoding image data obtained by reducing or expanding the original image to the predetermined number of pixels in units of the predetermined blocks, the decoded image data obtained by the decoding, the image processing apparatus according the image data obtained by reducing or enlarging the original image the same number of pixels and the image data, to claim 1 for outputting as the output image data. In an image processing method of an image processing apparatus for processing output image data output by a decoder that decodes image data encoded in predetermined block units in predetermined block units,
Determining a block size of the predetermined block in the output image data;
Reducing or enlarging the output image data into first reduced / enlarged image data having a predetermined number of pixels;
And the block size of the predetermined block in the output image data, if the block size is different from the predetermined block in the decoded image data obtained by the decoder performs decoding, the decrypt image data and a predetermined coefficient And the predetermined value obtained by learning that minimizes an error between the predicted value obtained by predicting high-quality image data having a higher image quality than the decoded image data and the high-quality image data. Using the coefficient and the first reduced / enlarged image data , the first reduced / enlarged image data is converted into second reduced / enlarged image data with higher image quality than the first reduced / enlarged image data ,
When the block size of the predetermined block in the output image data matches the block size of the predetermined block in the decoded image data, the output image is used by using the predetermined coefficient and the output image data. The data is converted into image data with higher image quality than the output image data,
An image processing method including a step of reducing or enlarging the second reduced / enlarged image data to image data having the same number of pixels as the output image data. In a program for causing a computer to execute image processing of an image processing apparatus that processes output image data output by a decoder that decodes image data encoded in predetermined block units.
Determining a block size of the predetermined block in the output image data;
Reducing or enlarging the output image data into first reduced / enlarged image data having a predetermined number of pixels;
And the block size of the predetermined block in the output image data, if the block size is different from the predetermined block in the decoded image data obtained by the decoder performs decoding, the decrypt image data and a predetermined coefficient And the predetermined value obtained by learning that minimizes an error between the predicted value obtained by predicting high-quality image data having a higher image quality than the decoded image data and the high-quality image data. Using the coefficient and the first reduced / enlarged image data , the first reduced / enlarged image data is converted into second reduced / enlarged image data with higher image quality than the first reduced / enlarged image data ,
When the block size of the predetermined block in the output image data matches the block size of the predetermined block in the decoded image data, the output image is used by using the predetermined coefficient and the output image data. The data is converted into image data with higher image quality than the output image data,
A program that causes a computer to execute image processing including a step of reducing or enlarging the second reduced / enlarged image data to image data having the same number of pixels as the output image data.

Images