JP2003324738A5

JP2003324738A5 -

Info

Publication number: JP2003324738A5
Application number: JP2002129440A
Authority: JP
Filing date: 2002-04-30
Publication date: 2005-09-08
Anticipated expiration: 2022-04-30

Claims

In an image signal processing apparatus that performs a predetermined estimation operation on an input first image and converts the first image into a second image,
A region cutout means for cutting out the pixel of interest and a plurality of pixels around it from the first image as a prediction tap;
A difference value calculating means for calculating a difference value of pixel values between pixels having a predetermined positional relationship among the prediction taps;
Class classification means for classifying the pixel of interest into one of a plurality of classes based on the difference value of the prediction tap;
Reading means for reading a prediction coefficient corresponding to a class in which the pixel of interest is classified;
Image signal processing, comprising: an estimation calculation means for converting the pixel value of the pixel of interest into a pixel value of a second image by the predetermined estimation calculation from the read prediction coefficient and the prediction tap apparatus.

The class classification means determines that an edge exists between the pixels when the difference value is equal to or greater than a first threshold value, and classifies the class based on the edge determination result. Item 2. The image signal processing device according to Item 1.

3. The image signal processing apparatus according to claim 2, wherein the region cutout unit cuts out line data including the target pixel in the first image as the prediction tap.

The class classification means classifies the pixel of interest into one of a plurality of spatial classes when the difference value of a prediction tap configured from the line data is equal to or greater than a predetermined threshold. 3. The image signal processing device according to 3.

4. The image signal processing apparatus according to claim 3, wherein the line data is horizontal line data in a horizontal direction including the target pixel and vertical line data in a vertical direction including the target pixel.

6. The difference value calculating means calculates difference values between two pixels adjacent in the horizontal direction of the horizontal line data and between two pixels adjacent in the vertical direction of the vertical line data. The image signal processing apparatus described.

2. The image signal processing apparatus according to claim 1, wherein the difference value calculation means calculates the difference value based on a luminance signal of the first image or a color difference signal of the first image.

4. The image signal processing apparatus according to claim 3, further comprising: a statistic calculating unit that extracts a feature of the line data from a statistic of the difference value in the line data calculated by the difference value calculating unit.

It has a statistic calculation means for calculating a dynamic range that is a difference between the maximum value and the minimum value of the difference value from the statistics of the difference value in the line data calculated by the difference value calculation means. The image signal processing apparatus according to claim 3.

10. The image signal processing apparatus according to claim 9, wherein the statistic calculation means calculates the first threshold value in the inter-pixel edge determination from the maximum value, the minimum value, and the dynamic range of the difference values. .

A block that receives block data for each block of the first image and calculates a difference value of pixel values between pixels adjacent in the horizontal direction and the vertical direction in the target block to which the target pixel belongs as a block pixel difference value The image signal processing apparatus according to claim 2, further comprising an inter-pixel difference value calculation unit.

When the difference value between the block pixels in the block of interest is less than a second threshold, the block of interest is determined to be flat, and block feature amount storage means for storing the block unevenness determination result is provided. The image signal processing apparatus according to claim 11.

12. The image signal processing according to claim 11, further comprising block feature amount storage means for determining whether the block of interest has a field DCT structure or a frame DCT structure, and storing the block type determination result. apparatus.

12. A block statistic calculating unit for extracting a feature of the target block from a statistic of the inter-block pixel difference value in the target block calculated by the block pixel difference value calculating unit. Image signal processing apparatus.

Block statistics for calculating a dynamic range that is a difference between the maximum value and the minimum value of the difference value between the block pixels from the statistics of the difference value between the block pixels in the target block calculated by the difference value calculation means between the block pixels 12. The image signal processing apparatus according to claim 11, further comprising a quantity calculation unit.

The image according to claim 15, wherein the block statistic calculation unit calculates the second threshold value in the block unevenness determination from the maximum value, the minimum value, and the dynamic range of the block pixel difference values. Signal processing device.

The region cutout means receives data for each block of the first image, and outputs horizontal line data and vertical line data in the horizontal direction and vertical direction including the target pixel in the target block to which the target pixel belongs, respectively. The image signal processing apparatus according to claim 1, wherein the image signal processing apparatus is cut out as a prediction tap.

The area cutout means receives data for each block of the first image, and includes horizontal line data and vertical line data in the horizontal direction and vertical direction, respectively, including the target pixel in the target block to which the target pixel belongs, 2. The image signal according to claim 1, wherein in the adjacent block adjacent to the target block, the adjacent block pixel having the same horizontal or vertical position as the target pixel and closest to the target block is cut out as the prediction tap. Processing equipment.

The class classification means determines that there is an edge between the pixels when each difference value between adjacent pixels in the horizontal line data and the vertical line data is equal to or greater than a predetermined threshold value. 19. The image signal processing apparatus according to claim 18, wherein the target pixel is classified into one of a plurality of spatial classes based on a determination result.

When the difference value between the adjacent block pixel and the pixel closest to the adjacent block pixel is equal to or greater than a predetermined value, the class classification unit may determine whether the target block and the adjacent block to which the adjacent block pixel belongs are between 19. The image signal processing apparatus according to claim 18, wherein it is determined that an edge exists, and the target pixel is classified into one of a plurality of spatial classes based on the edge determination result.

12. The image signal processing apparatus according to claim 11, wherein when the difference value between block pixels is less than a predetermined threshold, the entire target block to which the target pixel belongs is classified into a flat block class.

An edge exists between the target pixel and a pixel adjacent on one side of the target pixel, and between the target pixel and a pixel adjacent on the other side of the target pixel; and 5. The image signal processing apparatus according to claim 4, wherein when the difference values between the pixels have the same sign, the target pixel is classified into an on-edge class.

An edge exists between the target pixel and a pixel adjacent on one side of the target pixel, and between the target pixel and a pixel adjacent on the other side of the target pixel; and 5. The image signal processing apparatus according to claim 4, wherein when the difference values between the pixels have different signs, the target pixel is classified into a fine line class.

If there are edges in both directions between the pixels arranged on one side of the pixel of interest and between the pixels arranged on the other side of the pixel of interest, the pixel of interest is complex 5. The image signal processing apparatus according to claim 4, wherein the image signal processing apparatus is classified into classes.

If there is an edge on either side between the pixels arranged on one side of the target pixel and between the pixels arranged on the other side from the target pixel, the target 5. The image signal processing apparatus according to claim 4, wherein the pixels are classified into a simple edge class.

26. The image signal processing apparatus according to claim 25, wherein the simple edge class is further classified into different classes based on edge positions.

A plurality of frame memories for storing the first image for each frame;
The region extraction means receives an attention frame including the attention pixel and a plurality of frames before and after the attention frame, and extracts a pixel at the same position as the attention pixel as the prediction tap in all the frames. The image signal processing apparatus according to claim 1, wherein:

The class classifying unit classifies the target pixel into any one of a plurality of time classes based on the difference value of the prediction tap composed of pixels at the same position as the target pixel in all the frames. 28. The image signal processing apparatus according to claim 27.

Block pixel difference value calculation that receives block data for each frame and calculates a difference value of pixel values between pixels adjacent to each other in the horizontal direction and the vertical direction in the target block to which the target pixel belongs. Means,
Block feature value calculating means for calculating a block threshold based on statistics of the difference value between block pixels in the block of interest calculated by the difference value calculating means between block pixels;
29. The time class of the pixel of interest according to claim 28, wherein the class classification means classifies the time class of the pixel of interest based on a difference value between pixels of adjacent frames in the prediction tap and a maximum value of the block threshold value for each frame. Image signal processing device.

If the inter-pixel difference value of the adjacent frame in the prediction tap is larger than the maximum value of the block threshold for each frame, the pixel of the frame in a direction away from the frame of interest is calculated from the frame from which the inter-pixel difference value is calculated. Unusable tap
30. The image signal processing according to claim 29, wherein pixels other than the unusable taps are usable taps, and the class classification unit classifies the time class based on the number of usable taps and the number of unusable taps. apparatus.

Learning means for learning the prediction coefficient in advance;
The learning means is an image generating means for generating the first image to be a student image obtained by adding distortion to a teacher image, and a prediction tap for a target pixel and a plurality of first pixels around it from the first image. A region cutout unit that cuts out as: a difference value calculation unit that calculates a difference value between pixels having a predetermined positional relationship among the prediction taps; and the prediction tap is selected from any of a plurality of classes based on the difference value Class classifying means for classifying the second image, prediction coefficient calculating means for calculating a prediction coefficient for predicting the second image from the teacher image and the prediction tap, and storage for storing the prediction coefficient for each class 31. The image signal processing apparatus according to claim 30, further comprising: means.

The prediction tap includes a plurality of taps in the future direction and the past direction in time from the attention frame to which the pixel of interest belongs, and one or more usable taps exist in the future direction, and 32. The image signal processing apparatus according to claim 31, wherein when there are one or more usable taps, the target pixel is classified into a first time class.

The prediction tap includes a plurality of taps in the future direction and the past direction from the attention frame to which the pixel of interest belongs, and when all the taps in the future direction are the unusable taps, The image signal processing apparatus according to claim 30, wherein the image signal processing apparatus is classified into two time classes.

The prediction tap includes a plurality of taps in the future direction and the past direction from the attention frame to which the pixel of interest belongs, and when all the taps in the past direction are the unusable taps, The image signal processing device according to claim 30, wherein the image signal processing device is classified into three time classes.

The prediction tap includes a plurality of taps in the future direction and the past direction from the attention frame to which the pixel of interest belongs, and when all the taps are unusable taps, the pixel of interest is a fourth time. The image signal processing apparatus according to claim 30, wherein the image signal processing apparatus is classified into classes.

The prediction tap in the learning means includes a plurality of taps in a future direction and a past direction in time from the target frame to which the target pixel belongs, and when all the prediction taps are usable taps, the target pixel 33. The image signal processing device according to claim 32, wherein the image signal processing device is classified into the first time class.

The prediction coefficient of the prediction tap in the learning means that is classified and learned in the first time class corresponding to the unusable tap in the prediction tap that is classified into the first time class at the time of the estimation calculation, 37. The image signal processing apparatus according to claim 36, wherein the prediction signal is distributed to all the usable taps in the prediction tap at the time of estimation calculation, and the prediction coefficient of the unusable tap is set to zero.

The prediction tap in the learning means includes a plurality of taps in the future direction and the past direction in time from the attention frame to which the pixel of interest belongs, and even one of all taps in the future direction has the unusable tap. 34. The image signal processing device according to claim 33, wherein the target pixel is classified into a second time class.

The prediction coefficient of the prediction tap in the learning means that is classified and learned in the second time class corresponding to the unusable tap in the prediction tap that is classified into the second time class at the time of the estimation calculation, 39. The image signal processing apparatus according to claim 38, wherein the prediction signal is distributed to all the usable taps in the prediction tap at the time of estimation calculation, and the prediction coefficient of the unusable tap is set to zero.

The prediction tap in the learning means includes a plurality of taps in the future direction and the past direction from the attention frame to which the pixel of interest belongs, and one of all the taps in the past direction has the unusable tap. 35. The image signal processing apparatus according to claim 34, wherein the target pixel is classified into a third time class.

The prediction coefficient of the prediction tap in the learning means that is classified and used in the third time class corresponding to the unusable tap in the prediction tap that is classified into the third time class at the time of the estimation calculation, 41. The image signal processing apparatus according to claim 40, wherein the prediction signal is distributed to all usable taps in the prediction tap at the time of estimation calculation, and the prediction coefficient of the unusable tap is set to zero.

The prediction tap in the learning means includes a plurality of taps in the future direction and the past direction in time from the attention frame to which the pixel of interest belongs, and even one of all taps in the future direction has the unusable tap. 35. The image signal processing device according to claim 34, wherein if there is at least one unusable tap among all taps in the past direction, the pixel of interest is classified into a fourth time class.

12. The image signal processing apparatus according to claim 11, wherein the class classification unit performs class classification based on a position of the target pixel in the block data.

2. The image signal processing apparatus according to claim 1, wherein the first image is a decoded image data or an MPEG decoded image encoded by DCT transform and DCT coefficient quantization.

The first image includes a luminance signal and a color difference signal, and the class classification unit classifies a class of a target pixel of the color difference signal from the luminance signal corresponding to the color difference signal. The image signal processing apparatus according to 1.

46. The image signal processing apparatus according to claim 45, further comprising: feature amount storage means for determining whether the first image is an interlaced method or a progressive method and recording the determination result.

The area extraction means includes a first area extraction means for extracting a class tap used for class classification by the class classification means, and a second area extraction for extracting a prediction tap used by the estimation calculation means. The image processing apparatus according to claim 1, further comprising: means.

In a prediction coefficient learning device that learns a prediction coefficient for performing a predetermined estimation operation on a first image and converting the first image into a second image,
Image generating means for generating a first image by adding distortion to the original image;
A region cutout means for cutting out the target pixel and the plurality of first pixels around it from the first image as prediction taps;
Difference value calculating means for calculating a difference value between pixels having a predetermined positional relationship among the prediction taps;
Class classification means for classifying the prediction tap into any one of a plurality of classes based on the difference value;
Prediction coefficient calculating means for calculating a prediction coefficient for predicting the second image from the prediction tap;
A prediction coefficient learning apparatus comprising: a storage unit that stores a prediction coefficient for each class.

In an image signal processing method for performing a predetermined estimation operation on an input first image and converting the first image into a second image,
A region cutting step of cutting out the target pixel and a plurality of pixels around it from the first image as a prediction tap;
A difference value calculating step of calculating a difference value between pixels having a predetermined positional relationship among the prediction taps;
A class classification step for classifying the pixel of interest into one of a plurality of classes based on the difference value of the prediction tap;
A reading step of reading a prediction coefficient corresponding to the class in which the pixel of interest is classified;
An image signal processing comprising: an estimation calculation step of converting the pixel value of the pixel of interest into a pixel value of a second image by the predetermined estimation calculation from the read prediction coefficient and the prediction tap Method.

In a prediction coefficient learning method for learning a prediction coefficient for performing a predetermined estimation operation on a first image and converting the first image into a second image,
An image generation step of generating a first image by adding distortion to the original image;
A region cutting out step of cutting out the target pixel and the plurality of first pixels around it from the first image as a prediction tap;
A difference value calculating step of calculating a difference value between pixels having a predetermined positional relationship among the prediction taps;
A class classification step for classifying the prediction tap into one of a plurality of classes based on the difference value;
A prediction coefficient calculation step for calculating a prediction coefficient for predicting the second image from the prediction tap;
A prediction coefficient learning method comprising: storing a prediction coefficient for each class.