JP2012175411A - Device and method for image quality evaluation, and computer program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image quality evaluation device, capable of evaluating image quality of a deteriorated image, by simply using the deteriorated image without referring to information of a decoding process.SOLUTION: The image quality evaluation device includes: an evaluation target frame extraction unit 11 for extracting an evaluation target frame from a moving image; a neighbor frame extraction unit 12 for extracting a neighbor frame to the evaluation target frame; a difference information calculation unit 13 for calculating difference information representing a difference between information extracted from each pixel set included in the evaluation target frame and information extracted from a corresponding pixel set, which is a pixel set in a position corresponding to the pixel set in the neighbor frame; and an image quality evaluation unit 14 for calculating an image quality evaluation value of the moving image, based on the difference information calculated for each pixel set.

Description

  The present invention relates to an image quality evaluation apparatus that evaluates the image quality of an image, and more particularly to an image quality evaluation apparatus that evaluates the degree of degradation of a moving image.

  In order to objectively evaluate the degree of image degradation, it is common to compare a degraded image (hereinafter referred to as a degraded image) with an original image before the degraded image is degraded. For example, Non-Patent Document 1 describes an image quality evaluation apparatus that performs image quality evaluation by direct comparison between a deteriorated image and an original image.

  Non-Patent Document 2 describes an image quality evaluation apparatus that performs image quality evaluation using a deteriorated image and a feature amount extracted from an original image.

  Patent Document 1 describes an image quality evaluation apparatus that performs image quality evaluation on a single degraded image in response to a case where an original image cannot be acquired. The image quality evaluation apparatus described in Patent Document 1 performs image quality evaluation using information obtained in the process of decoding an encoded degraded image and image information after decoding.

“Objective perceptual video quality measurement techniques for digital cable television in the presence of a full reference”, ITU-T Recommendation J.144, 2003. “Perceptual video quality measurement techniques for digital cable television in the presence of a reduced reference”, ITU-T Recommendation J.249, 2010.

WO2008 / 093714

  However, the image quality evaluation apparatuses described in Non-Patent Document 1 and Non-Patent Document 2 require the presence of the original image because the image quality evaluation is performed by comparison with the original image. Therefore, there is a problem that image quality cannot be evaluated when there is no original image.

  In addition, the image quality evaluation apparatus described in Patent Document 1 performs image quality evaluation using information obtained in the process of decoding an encoded degraded image, and thus needs to refer to information inside the decoding apparatus. However, if the decoding has already been completed, or if the decoding device is in the hardware such as a set-top box and the internal information cannot be referred to, information on the decoding process cannot be acquired. As described above, there is a problem that the image quality evaluation cannot be performed when the decoding process information cannot be acquired.

  The present invention has been made to solve the above-described problems, and is an image quality evaluation apparatus, an image evaluation method, and a computer that enable image quality evaluation of a deteriorated image without referring to information on a decoding process with a single deteriorated image.・ The purpose is to provide a program.

  An image quality evaluation apparatus according to the present invention includes an evaluation target frame extraction unit that extracts an evaluation target frame from a moving image, a neighboring frame extraction unit that extracts a neighboring frame of the evaluation target frame, and each pixel set included in the evaluation target frame A difference information calculation unit that calculates difference information representing a difference between information extracted from the information extracted from a corresponding pixel set that is a pixel set at a position corresponding to the pixel set in the neighboring frame, and each pixel An image quality evaluation unit that calculates an image quality evaluation value of the moving image based on the difference information calculated for the set.

  Further, the image quality evaluation method of the present invention extracts an evaluation target frame from a moving image, extracts neighboring frames of the evaluation target frame, information extracted from each pixel set included in the evaluation target frame, and the neighborhood Difference information representing a difference from information extracted from a corresponding pixel set that is a pixel set at a position corresponding to the pixel set in a frame is calculated, and based on the difference information calculated for each pixel set, the moving image An image quality evaluation value is calculated.

  The computer program according to the present invention includes an evaluation target frame extraction step for extracting an evaluation target frame from a moving image, a neighboring frame extraction step for extracting a neighboring frame of the evaluation target frame, and each of the evaluation target frames. A difference information calculation step for calculating difference information representing a difference between information extracted from a pixel set and information extracted from a corresponding pixel set that is a pixel set at a position corresponding to the pixel set in the neighboring frame; And causing the computer device to execute an image quality evaluation step of calculating an image quality evaluation value of the moving image based on the difference information calculated for each pixel set.

  The present invention can provide an image quality evaluation apparatus, an image evaluation method, and a computer program that enable image quality evaluation of a deteriorated image without referring to information on a decoding process by itself.

It is a functional block block diagram of the image quality evaluation apparatus as the 1st Embodiment of this invention. It is a flowchart explaining operation | movement of the image quality evaluation apparatus as the 1st Embodiment of this invention. It is a functional block block diagram of the image quality evaluation apparatus as the 2nd Embodiment of this invention. It is a figure explaining an example of the calculation method of the block distortion amount in the 2nd Embodiment of this invention. It is a figure explaining an example of the calculation method of the blur amount in the 2nd Embodiment of this invention. It is a flowchart explaining operation | movement of the image quality evaluation apparatus as the 2nd Embodiment of this invention. It is a functional block block diagram of the image quality evaluation apparatus as the 3rd Embodiment of this invention. It is a flowchart explaining operation | movement of the image quality evaluation apparatus as the 3rd Embodiment of this invention. It is a functional block block diagram of the image quality evaluation apparatus as the 4th Embodiment of this invention. It is a flowchart explaining operation | movement of the image quality evaluation apparatus as the 4th Embodiment of this invention.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

(First embodiment)
FIG. 1 shows a functional block configuration of an image quality evaluation apparatus 1 as a first embodiment of the present invention.

  In FIG. 1, the image quality evaluation apparatus 1 includes an evaluation target frame extraction unit 11, a neighboring frame extraction unit 12, a difference information calculation unit 13, and an image quality evaluation unit 14.

  Here, the image quality evaluation apparatus 1 is configured by a general-purpose computer device including a CPU (Central Processing Unit), a RAM (Random Access Memory), a ROM (Read Only Memory), and an auxiliary storage device such as a hard disk drive. Has been. Each functional block of the image quality evaluation apparatus 1 is configured by a CPU that reads a computer program module stored in a ROM or an auxiliary storage device into a RAM and executes it. Note that the hardware configuration of each functional block of the image quality evaluation apparatus 1 is not limited to the above configuration.

  The evaluation target frame extraction unit 11 extracts an evaluation target frame from the frames constituting the moving image. Specifically, the evaluation target frame extraction unit 11 acquires a deteriorated moving image (hereinafter also referred to as a deteriorated image) that is an object of image quality evaluation. Then, the evaluation target frame extraction unit 11 selects an evaluation target frame that is an evaluation target from among the frames that constitute the acquired degraded image. Further, the evaluation target frame extraction unit 11 extracts the value of each pixel constituting the selected evaluation target frame (hereinafter referred to as pixel value). This pixel value may be expressed, for example, in the form of luminance (Y) or color difference (Cb, Cr). Alternatively, the pixel values may be expressed in other formats such as RGB composed of red, blue, and green.

  Note that the evaluation target frame extraction unit 11 may acquire a deteriorated image from an auxiliary storage device of a computer device constituting the image quality evaluation device 1. Alternatively, the evaluation target frame extraction unit 11 may acquire a deteriorated image from the outside via a peripheral device connection interface or a network interface.

  Further, the evaluation target frame extraction unit 11 may extract other frames constituting the deteriorated image as evaluation target frames after the processing by other functional blocks for the extracted evaluation target frames is completed. For example, the evaluation target frame extraction unit 11 may extract all the frames constituting the deteriorated image as the evaluation target frames in order from the top. Alternatively, the evaluation target frame extraction unit 11 may extract some frames constituting the deteriorated image as evaluation target frames in a predetermined order.

  The neighborhood frame extraction unit 12 extracts a neighborhood frame of the evaluation target frame. Specifically, the neighboring frame extraction unit 12 extracts each pixel value of a frame whose temporal position is close to the evaluation target frame. For example, the neighboring frame extraction unit 12 may extract each pixel value of a frame that is temporally adjacent to the evaluation target frame. Further, the neighboring frame extraction unit 12 may extract one or more neighboring frames for one evaluation target frame. For example, the neighboring frame extraction unit 12 may extract two neighboring frames that are temporally adjacent to the evaluation target frame. Alternatively, the neighboring frame extraction unit 12 may extract a plurality of neighboring frames that are temporally continuous with respect to the evaluation target frame, or may extract a part of the frames.

  The difference information calculation unit 13 calculates a difference between information extracted from each pixel set included in the evaluation target frame and information extracted from a corresponding pixel set that is a pixel set at a position corresponding to the pixel set in a neighboring frame. The difference information to represent is calculated. For example, the difference information calculation unit 13 may calculate difference information from a corresponding pixel set for a pixel set in each of one or more predetermined regions included in the evaluation target frame.

  Specifically, the difference information calculation unit 13 estimates a corresponding pixel set of a neighboring frame corresponding to each pixel set included in the evaluation target frame. That is, the difference information calculation unit 13 estimates a corresponding pixel set in which the object shown in each pixel set included in the evaluation target frame is shown in the neighboring frame. That is, the difference information calculation unit 13 may estimate the movement of each pixel set of the evaluation target frame. In this case, the difference information calculation unit 13 can estimate the corresponding pixel set using a general algorithm related to motion estimation in a moving image.

  Further, the difference information calculation unit 13 calculates difference information representing a difference between information extracted from the pixel set and information extracted from the corresponding pixel set for each pixel set included in the evaluation target frame. The difference information may be the following values. That is, the difference information may be, for example, a sum of absolute differences between each pixel value included in the pixel set of the evaluation target frame and each pixel value included in the corresponding pixel set. Alternatively, the difference information may be a sum of squared differences between each pixel value of the pixel set of the evaluation target frame and each pixel value of the corresponding pixel set. Alternatively, the difference information may be a difference between the average pixel value of the pixel set of the evaluation target frame and the average pixel value of the corresponding pixel set. Alternatively, the difference information may be a difference between the variance of the pixel values of the pixel set of the evaluation target frame and the variance of the pixel values of the corresponding pixel set. Alternatively, the difference information may be a difference between the activity value of the pixel set of the evaluation target frame and the activity value of the corresponding pixel set. Here, the activity value is a value calculated as follows, for example. First, the average of the pixel values in the corresponding pixel set is obtained. Then, a difference from the pixel value average is obtained for each pixel value in the corresponding pixel set. The average of the absolute values of these differences becomes the activity value of the corresponding pixel set. Note that the difference information calculation unit 13 may use a luminance value of each pixel, a value representing RGB, or a value representing color difference as a pixel value used when calculating the difference information.

  The image quality evaluation unit 14 calculates an image quality evaluation value based on the difference information calculated between each pixel set of the evaluation target frame and the corresponding pixel set. For example, the image quality evaluation unit 14 may use the average value or the maximum value of the difference information as the image quality evaluation value.

  The operation of the image quality evaluation apparatus 1 configured as described above will be described with reference to FIG.

  First, the evaluation target frame extraction unit 11 acquires a degraded image that is an object of image quality evaluation (step S101).

  For example, the evaluation target frame extraction unit 11 may acquire a deteriorated image designated via the input device from the auxiliary storage device, or obtain it from an external device connected via a peripheral device interface or a network. Also good.

  Next, the evaluation target frame extraction unit 11 extracts an evaluation target frame (step S102).

  For example, the evaluation target frame extraction unit 11 may extract each pixel value of the frame using the first frame of the moving image as the evaluation target frame.

  Next, the neighborhood frame extraction unit 12 extracts a neighborhood frame of the evaluation target frame extracted in step S102 (step S103).

  For example, the neighboring frame extraction unit 12 may extract each pixel value of the frame using a frame adjacent to the evaluation target frame as a neighboring frame.

  Next, the difference information calculation unit 13 selects one of the pixel sets included in the evaluation target frame as a processing target (step S104).

  For example, the difference information calculation unit 13 may select a pixel set corresponding to one of one or more predetermined regions included in the evaluation target frame as a processing target.

  Next, the difference information calculation unit 13 estimates the position of the corresponding pixel set in the neighboring frame corresponding to the pixel set to be processed (step S105).

  For example, the difference information calculation unit 13 is a position where the difference absolute value sum between the luminance value of each pixel included in the pixel set to be processed and the luminance value of each pixel included in an arbitrary pixel set in the neighboring frame is the smallest. May be estimated as the corresponding pixel set.

  Next, the difference information calculation unit 13 calculates difference information representing a difference between information extracted from the processing target pixel set and information extracted from the corresponding pixel set (step S106).

  For example, the difference information calculation unit 13 calculates the difference between the activity value of the pixel set to be processed and the activity value of the corresponding pixel set as the difference information.

  Next, the image quality evaluation apparatus 1 determines whether or not there is an unprocessed pixel set to be processed in the evaluation target frame (step S107).

  For example, the image quality evaluation apparatus 1 may determine whether or not processing has been performed on all pixel sets corresponding to predetermined areas defined in the evaluation target frame.

  Here, when it is determined that there is an unprocessed pixel set, the image quality evaluation apparatus 1 executes the processes of steps S104 to S106 for the next pixel set.

  On the other hand, when the process for all pixel sets to be processed is completed, the image quality evaluation apparatus 1 determines whether or not there is another frame to be evaluated in the degraded image acquired in step S101 (step S108). ).

  For example, when the image quality evaluation apparatus 1 processes all the frames constituting the deteriorated image as the evaluation target frames in order from the top, the neighboring frame immediately after the next frame (that is, two frames after the frame that has been processed). It may be determined whether or not there is a frame.

  Here, when it is determined that there is another frame to be evaluated, the image quality evaluation apparatus 1 executes the processing from step S102 for the next evaluation target frame.

  On the other hand, when the processing for all the frames to be evaluated is completed, the image quality evaluation unit 14 calculates an image quality evaluation value based on the difference information calculated for each pixel set in step S106 (step S109).

  For example, the image quality evaluation unit 14 may calculate the average of the activity difference values calculated for each pixel set in step S106 as the image quality evaluation value. Thereby, it can be determined that the larger the image quality evaluation value, the higher the degree of image quality degradation.

  Thus, the image quality evaluation apparatus 1 ends the operation.

  Next, effects of the first exemplary embodiment of the present invention will be described.

  The image quality evaluation apparatus according to the first embodiment of the present invention enables image quality evaluation of a deteriorated image without referring to information on a decoding process with a single deteriorated image.

  The reason is that the difference information calculation unit obtains difference information of information that can be extracted between each pixel set included in the evaluation target frame and the corresponding pixel set corresponding to the neighboring frame, and the image quality evaluation unit This is because the image quality evaluation value is calculated based on the difference information. As a result, the image quality evaluation apparatus as the first embodiment of the present invention estimates that a small area included in each frame and an object projected in the small area are projected in a neighboring frame. It is possible to calculate an image quality evaluation value indicating that the degree of deterioration is larger as the variation of the pixel value between the regions is larger. That is, the difference in information that can be extracted between the corresponding regions of neighboring frames increases as the manner of occurrence of deterioration differs from frame to frame. Therefore, the image quality evaluation apparatus as the first embodiment of the present invention can evaluate the deterioration of the image quality from the difference.

(Second Embodiment)
Next, a second embodiment of the present invention will be described in detail with reference to the drawings. Note that, in each drawing referred to in the description of the present embodiment, the same reference numerals are given to the same configuration and steps that operate in the same manner as in the first embodiment of the present invention, and the detailed description in the present embodiment. Description is omitted.

  First, FIG. 3 shows a functional block configuration of an image quality evaluation apparatus 2 as a second embodiment of the present invention.

  In FIG. 3, the image quality evaluation apparatus 2 is different from the image quality evaluation apparatus 1 according to the first embodiment of the present invention in that a difference information calculation unit 23 and an image quality evaluation unit 14 are replaced with the difference information calculation unit 13. The image quality evaluation unit 24 is different.

  Here, the image quality evaluation apparatus 2 is configured by a general-purpose computer device including a CPU, a RAM, a ROM, and an auxiliary storage device such as a hard disk drive. Each functional block of the image quality evaluation apparatus 2 is constituted by a CPU that reads a computer program module stored in a ROM or an auxiliary storage device into a RAM and executes it. Note that the hardware configuration of each functional block of the image quality evaluation apparatus 2 is not limited to the above configuration.

  The difference information calculation unit 23 divides the evaluation target frame into one or more pixel sets. Then, the difference information calculation unit 23 calculates difference information between each divided pixel set and a corresponding corresponding pixel set. Note that the difference information calculation unit 23 may not calculate difference information for all of the divided pixel sets. For example, the difference information calculation unit 23 may calculate difference information by thinning out the divided pixel sets.

  Further, the difference information calculation unit 23 estimates the position of the corresponding pixel set based on the difference between information based on each pixel set obtained by dividing the evaluation target frame and information based on an arbitrary pixel set in the neighboring frame. For example, the difference information calculation unit 23 calculates a sum of absolute differences between each pixel value of the pixel set of the evaluation target frame and each pixel value included in an arbitrary pixel set of the same size in the neighboring frame. Then, the difference information calculation unit 23 may estimate, as a corresponding pixel set, a pixel set at a position where the difference absolute value sum with each pixel value of the pixel set of the evaluation target frame is minimum in the neighboring frame. Alternatively, the difference information calculation unit 23 may estimate, as a corresponding pixel set, a pixel set at a position where the sum of squared differences with each pixel value included in the pixel set of the evaluation target frame is the smallest in the neighboring frame. Alternatively, the difference information calculation unit 23 estimates, as a corresponding pixel set, a pixel set at a position having an average of pixel values that minimizes a difference from an average of pixel values of the pixel set of the evaluation target frame in the neighboring frame. Also good. Alternatively, the difference information calculation unit 23 estimates, as a corresponding pixel set, a pixel set at a position having a variance of pixel values that minimizes a difference from a variance of pixel values of the pixel set of the evaluation target frame in the neighboring frame. Also good. Alternatively, the difference information calculation unit 23 may estimate, as a corresponding pixel set, a pixel set at a position having an activity value that minimizes the difference from the activity value of the pixel set of the evaluation target frame in the neighboring frame.

  At this time, the difference information calculation unit 23 may estimate the corresponding pixel set in a region where the relative position of each frame of the evaluation target frame is within a predetermined range. For example, it is assumed that the evaluation target frame is divided into a pixel set having a size of 16 pixels × 16 pixels. In this case, the difference information calculation unit 23 determines that the sum of the absolute differences of the pixel values described above is minimum in the region of the neighboring frame in the range of ± 32 pixels in the horizontal and vertical directions with respect to the position of each pixel set in the evaluation target frame. You may search for the position of the pixel set.

  Further, the difference information calculation unit 23 does not have to calculate difference information for a pixel set that satisfies a predetermined condition when calculating difference information between each pixel set of the evaluation target frame and the corresponding pixel set. Also good. Here, the predetermined condition is a case where, when estimating a corresponding pixel set corresponding to each pixel set, a pixel set whose difference from information based on each pixel set is smaller than a threshold value cannot be detected in a neighboring frame. In such a case, it can be considered that there is no corresponding pixel set in the neighboring frame in which the object shown in the corresponding pixel set in the evaluation target frame is shown. Therefore, if the difference information calculation unit 23 cannot detect a pixel set whose difference from the information based on the pixel set of the evaluation target frame is smaller than the threshold in the neighboring frame, the difference information calculation unit 23 omits the calculation of the difference information for the corresponding pixel set. May be.

  Note that the difference information calculation unit 23 uses a luminance value, a value representing color difference, or a value representing RGB as a pixel value used when estimating the corresponding pixel set and a pixel value used when calculating the difference information. It may be used.

  The image quality evaluation unit 24 calculates an image quality evaluation value after weighting the difference information calculated between each pixel set of the evaluation target frame and the corresponding pixel set. Thereby, the image quality evaluation unit 24 can increase the correlation between the image quality evaluation value and the subjective evaluation.

  Further, the image quality evaluation unit 24 can calculate the weighting coefficient for the difference information calculated for each pixel set from the corresponding pixel set based on information related to the corresponding pixel set. Here, the information related to the pixel set may be information representing a spatial frequency, a temporal frequency, a specific color, block distortion, a transmission error, or a blur. In addition, the information related to such a pixel set may be information calculated based on the content of each frame including the pixel set, or information calculated based on the content of each pixel set. May be.

  For example, block distortion, transmission error, or blur is itself a factor of deterioration. Such deterioration tends to occur uniformly in the frame. For this reason, the image quality evaluation unit 24 may calculate each value such as block distortion, transmission error, or blur for the evaluation target frame, and determine a larger weighting coefficient as the calculated value increases. The image quality evaluation unit 24 calculates the image quality evaluation value by applying the weighting coefficient calculated based on the evaluation target frame including the pixel set to the difference information calculated for each pixel set. Note that the image quality evaluation unit 24 further calculates each value such as block distortion, transmission error, or blur in each frame adjacent to the corresponding evaluation target frame, and based on the average value of these values per frame. The weighting factor may be determined.

The block distortion amount can be calculated based on, for example, an activity value for each pixel area having a predetermined size. More specifically, the image quality evaluation unit 24 uses the expression shown in FIG. 4B for the pixel set block 1 and block 2 of two 8 pixels × 8 pixels adjacent in the horizontal direction shown in FIG. Therefore, the average value Act _{Ave of the} activity is calculated. Further, the image quality evaluation unit 24 calculates DiffBound, which is an average value of the absolute difference values of the luminance values of the pixels in the portion where the two pixel set blocks 1 and 2 are in contact, according to the formula shown in FIG. . Then, the image quality evaluation unit 24 calculates a block distortion amount estimation parameter Block representing the ratio between the average value Act _Ave and the average value DiffBound according to the equation shown in FIG. In this way, the image quality evaluation unit 24 sets the block distortion amount estimation parameter Block _{i, j} in each of the M pixel areas (areas including the block 1 and the block 2) in the N neighboring frames including the corresponding frame. Ask. Then, the image quality evaluation unit 24 may calculate the average value Block _Ave as the block distortion amount of the corresponding frame according to the equation shown in FIG.

  Also, the amount of blur can be calculated based on, for example, the edge width detected from each frame. More specifically, the image quality evaluation unit 24 detects edges by using a Sobel filter, a high-pass filter, or the like. Then, the image quality evaluation unit 24 obtains the edge width as the number of continuous pixels in which the pixel value in the edge region monotonously increases (or monotonously decreases). Here, as shown in FIG. 5, the amount of blur is inversely proportional to the sharpness of the edge. That is, in FIG. 5A, since the edge is steep (the edge width is small), there is little blur. On the other hand, in FIG. 5B, the edge is not steep (the width of the edge is large), so there is much blur. Therefore, the image quality evaluation unit 24 can calculate the amount of blur in the corresponding frame based on the width of each edge detected in the frame.

  In addition, for example, when the spatial frequency or the temporal frequency is high, it tends to be difficult for the viewer to detect deterioration. That is, it is difficult for the viewer to perceive deterioration in a fine pattern area or an area where movement is intense in the video. Further, such spatial frequency or temporal frequency tends to be different for each pixel set. For this reason, the image quality evaluation unit 24 may calculate each value of the spatial frequency or the temporal frequency for each pixel set, and determine a weighting coefficient having a smaller value as the calculated value is higher. Of course, the image quality evaluation unit 24 may calculate such a spatial frequency or temporal frequency per frame and determine a weighting coefficient for each evaluation target frame. Note that the spatial frequency and the temporal frequency can be calculated by, for example, discrete cosine transform.

  In addition, the specific color may affect the ease with which the viewer perceives deterioration. For example, an area with a lot of skin color has a high possibility of human beings. For this reason, a region with a lot of flesh color tends to be watched by the viewer, and deterioration is easily detected by the viewer. Such colors also tend to be different for each pixel set. Therefore, the image quality evaluation unit 24 may calculate a ratio of a specific color (for example, skin color) for each pixel set, and may determine a large weighting coefficient for a pixel set with a large specific color ratio. Of course, the image quality evaluation unit 24 may calculate the ratio of such a specific color per frame and determine a weighting coefficient for each evaluation target frame.

  The operation of the image quality evaluation apparatus 2 configured as described above will be described with reference to FIG.

  First, the evaluation target frame extraction unit 11 acquires a degraded image as an image quality evaluation target (step S101).

  Next, the evaluation target frame extraction unit 11 matches the frame pointer indicating the evaluation target frame with one of the frames constituting the deteriorated image acquired in step S101 (step S202).

  For example, the evaluation target frame extraction unit 11 may align the frame pointer with the head of the frame constituting the deteriorated image.

  Next, the evaluation target frame extraction unit 11 extracts each pixel value of the evaluation target frame indicated by the frame pointer (step S203).

  Next, the neighboring frame extraction unit 12 extracts a neighboring frame of the evaluation target frame extracted in step S203 (step S204).

  For example, the neighboring frame extraction unit 12 may extract each pixel value of a frame that is one frame later in time with respect to the evaluation target frame.

  Note that each pixel value extracted in steps S203 and S204 may be luminance (Y) information. Alternatively, each pixel value may be information representing a color difference (Cb, Cr) or information representing RGB.

  Next, the difference information calculation unit 23 divides the evaluation target frame into a set of pixels of a predetermined size (step S205). For example, the difference information calculation unit 23 may divide the evaluation target frame into a pixel set of 16 pixels × 16 pixels.

Next, the difference information calculation unit 23 matches the pixel set pointer indicating the pixel set to be processed with one of the pixel sets divided in step S205 (step S206).
For example, the difference information calculation unit 23 may align the pointer with the upper left pixel set of the frame.

  Next, the difference information calculation unit 23 obtains a difference between the information based on the pixel set indicated by the pixel set pointer and the information based on an arbitrary pixel set extracted in step S204 (step S207).

  At this time, the difference information calculation unit 23 may obtain a difference between the relative position with respect to the pixel set indicated by the pixel set pointer and information based on an arbitrary pixel set within a predetermined range in the neighboring frame.

  For example, the difference information calculation unit 23 obtains the sum of absolute differences between the luminance value of each pixel included in the pixel set of the evaluation target frame and the luminance value of each pixel included in an arbitrary pixel set of the neighboring frame. Also good.

  Next, the difference information calculation unit 23 detects the position of the corresponding pixel set in the neighboring frame based on the difference obtained in step S207 (step S208).

  For example, the difference information calculation unit 23 may estimate a pixel set at a position where the sum of absolute differences of luminance values obtained in step S207 is minimum as a corresponding pixel set.

  Next, the difference information calculation unit 23 determines whether or not the difference calculated in step S207 for the corresponding pixel set estimated in step S208 is equal to or smaller than a threshold (step S209).

  If the difference calculated in step S207 is larger than the threshold value, the image quality evaluation apparatus 2 omits the process for this pixel set and proceeds to step S211.

  On the other hand, when the difference calculated in step S207 is equal to or smaller than the threshold value, the difference information calculation unit 23 calculates difference information between information extracted from the pixel set indicated by the pixel set pointer and information extracted from the corresponding pixel set. (Step S210).

  For example, the difference information calculation unit 23 may calculate the difference between the aforementioned activity values as the difference information.

  Then, the image quality evaluation apparatus 2 advances the pixel set pointer to the next pixel set (step S211).

  Here, if there is an unprocessed pixel set among the pixel sets obtained by dividing the evaluation target frame (Yes in step S212), the image quality evaluation apparatus 2 repeatedly executes the processing from step S207.

  On the other hand, when the processing of all pixel sets obtained by dividing the evaluation target frame is completed (No in step S212), the image quality evaluation unit 24 determines a weighting coefficient in the evaluation target frame (step S213).

  For example, the image quality evaluation unit 24 determines the weighting coefficient based on information representing the spatial frequency, temporal frequency, specific color, block distortion, transmission error, blur, etc. in the evaluation target frame. If the weighting coefficient is determined based on the contents of each pixel set, the image quality evaluation unit 24 may execute the process of step S213 between steps S210 and S211.

  Then, the image quality evaluation apparatus 2 advances the frame pointer to the next frame (step S214).

  Next, when there is an unprocessed frame to be evaluated in the degraded image (Yes in step S215), the image quality evaluation device 2 repeatedly executes the processing from step S203.

  For example, when the image quality evaluation apparatus 2 processes all the frames constituting the deteriorated image as the evaluation target frames in order from the top, the neighboring frame of the frame next to the evaluation target frame that has been processed (that is, the evaluation target that has been processed) It may be determined whether or not there is a second frame after the frame.

  On the other hand, when the processing for all frames to be evaluated in the degraded image is completed (No in step S215), the image quality evaluation unit 24 adds the weighting coefficient calculated in step S213 to the difference information calculated for each pixel set in step S210. The image quality evaluation value is calculated based on the value to which (S216) is applied.

  For example, the image quality evaluation unit 24 may calculate the average of the difference information to which the weighting coefficient is applied as the image quality evaluation value.

  Thus, the image quality evaluation apparatus 2 ends the operation.

  Next, an operation example of the image quality evaluation apparatus 2 will be described. In the following description, it is assumed that the image of the image quality evaluation target is a deteriorated image of SDTV (Standard Definition Television) size (horizontal 720 pixels × vertical 480 pixels, 30 frames per second).

  First, the evaluation target frame extraction unit 11 acquires this deteriorated image (step S201).

  Next, the evaluation target frame extraction unit 11 first extracts the luminance value using the first frame as the evaluation target frame (steps S202 and S203).

  Next, the neighboring frame extraction unit 12 extracts a frame adjacent to the evaluation target frame one time after (step S204).

  Next, the difference information calculation unit 23 divides the evaluation target frame into a pixel set of 16 pixels × 16 pixels. Then, the difference information calculation unit 23 searches each pixel set for a corresponding pixel set at a position where the sum of absolute differences of luminance values in neighboring frames is minimum within a range of ± 32 pixels in each of horizontal and vertical directions (step S205 to S208).

  Next, the difference information calculation unit 23 calculates the difference between the activity value of each pixel set and the activity value of the corresponding pixel set in the neighboring frame (step S210).

  When the activity difference is calculated for all the pixel sets obtained by dividing the evaluation target frame, the image quality evaluation unit 24 calculates the blur amount for each of the 30 neighboring frames including the evaluation target frame, and calculates the average value of the calculated blur amount per frame. Based on this, a weighting coefficient is determined (step S213).

  Next, the image quality evaluation apparatus 2 advances the frame pointer to the next frame to be evaluated (step S214), and repeats the processing of steps S204 to S213 for the evaluation target frame indicated by the frame pointer.

  When the calculation of the difference information for each pixel set is completed in all frames to be evaluated in the degraded image (No in step S215), the image quality evaluation unit 24 adds all the difference information calculated in step S210 to step S213. The image quality evaluation value is calculated after applying the weighting coefficient calculated in (Step S216).

  Above, description of the operation example of the image quality evaluation apparatus 2 is complete | finished.

  Next, the effect of the second exemplary embodiment of the present invention will be described.

  The image quality evaluation apparatus according to the second embodiment of the present invention can perform image quality evaluation of a deteriorated image with higher accuracy without referring to information on a decoding process with a single degraded image.

  The reason is that if the difference information calculation unit cannot estimate a corresponding pixel set whose difference value with each pixel set obtained by dividing the evaluation target frame is smaller than the threshold value in the neighboring frame, the difference information is calculated with the corresponding pixel set. This is because it is not calculated. Thereby, the image quality evaluation apparatus as the second exemplary embodiment of the present invention provides difference information between each pixel set and a corresponding pixel set that is unlikely to show the same object in the neighboring frame. There is no calculation. Such difference information is likely not to be an effective index for image quality evaluation. Therefore, the image quality evaluation apparatus according to the second embodiment of the present invention does not use difference information that is highly likely not to be an effective index for calculating the image quality evaluation value.

  Furthermore, the image quality evaluation apparatus as the second embodiment of the present invention can reflect the viewer's subjectivity in the image quality evaluation of a single degraded image.

  This is because the image quality evaluation unit calculates the image quality evaluation value after weighting the difference information calculated for each pixel set. For example, the image quality evaluation unit applies a larger weighting coefficient to difference information calculated for a pixel set included in a frame having a large value such as block distortion, transmission error, or blur of a frame that affects deterioration itself. It is. Further, for example, the image quality evaluation unit applies a smaller weighting coefficient to the difference information calculated for a pixel set that has a high spatial frequency or temporal frequency and is less likely to be detected by the viewer. Further, for example, the image quality evaluation unit applies a larger weighting coefficient to the difference information calculated for the color pixel set that is easily watched by the viewer.

  In the present embodiment, the operation example has been described in which the difference information is calculated using all pixel sets divided into a predetermined size in all frames of the degradation image to be evaluated. In addition to such an operation example, the image quality evaluation apparatus as the present embodiment may calculate the above-described difference information by thinning out the evaluation target frame and the pixel set to be processed, and calculate the image quality evaluation value. .

(Third embodiment)
Next, a third embodiment of the present invention will be described in detail with reference to the drawings. Note that, in each drawing referred to in the description of the present embodiment, the same reference numerals are given to the same configuration and steps that operate in the same manner as in the second embodiment of the present invention, and the detailed description in the present embodiment. Description is omitted.

  First, FIG. 7 shows a functional block configuration of an image quality evaluation apparatus 3 as a third embodiment of the present invention. In FIG. 7, the image quality evaluation apparatus 3 includes an evaluation target frame extraction unit 31 instead of the evaluation target frame extraction unit 11 in addition to the image quality evaluation apparatus 2 as the second embodiment of the present invention. The difference is that it includes a digitized frame detector 35.

  Here, the image quality evaluation device 3 is configured by a general-purpose computer device including a CPU, a RAM, a ROM, and an auxiliary storage device such as a hard disk drive. Each functional block of the image quality evaluation device 3 is configured by a CPU that reads a computer program module stored in a ROM or an auxiliary storage device into a RAM and executes it. Note that the hardware configuration of each functional block of the image quality evaluation apparatus 3 is not limited to the above configuration.

  The intra-coded frame detection unit 35 checks whether or not the frame included in the deteriorated image has been intra-coded. The intra-coded frame detection unit 35, together with the evaluation target frame extraction unit 31, constitutes an embodiment of the evaluation target frame extraction unit of the present invention.

  In general, each frame of an image encoded by the MPEG-2 (MPEG2) method or the like uses an intra-coded frame (I picture) that is encoded using information of only the frame and prediction between frames. And inter-coded frames (P picture, B picture) to be encoded. The intra encoded frame detection unit 35 determines whether or not the frame selected by the evaluation target frame extraction unit 31 is an intra encoded frame.

  Specifically, the intra-coded frame detection unit 35 uses frame type information previously associated with the frame, and whether or not the type information indicating that the frame is an intra-coded frame is associated with the frame. May be determined. In this case, the intra-coded frame detection unit 35 may acquire such frame type information from the outside. Note that such frame type information may be output at the stage where an image is encoded by an image encoding device, for example. Alternatively, such frame type information may be output at the stage of decoding an image by an image decoding device.

  The evaluation target frame extraction unit 31 extracts an intra-coded frame as an evaluation target frame from among frames constituting the deteriorated image. Specifically, the evaluation target frame extraction unit 31 determines whether or not the frame constituting the deteriorated image is an intra-encoded frame using the intra-encoded frame detection unit 35.

  The operation of the image quality evaluation apparatus 3 configured as described above will be described with reference to FIG.

  First, the evaluation target frame extraction unit 31 acquires a degraded image as an image quality evaluation target (step S101).

  Next, the intra encoded frame detection unit 35 acquires frame type information of each frame constituting the acquired degraded image (step S401).

  Next, the evaluation target frame extraction unit 31 aligns the frame pointer with one of the frames constituting the deteriorated image acquired in step S101 (step S202).

  For example, the evaluation target frame extraction unit 31 may align the frame pointer with the head of the deteriorated image.

  Next, the intra encoded frame detection unit 35 determines whether or not the frame indicated by the frame pointer is an intra encoded frame using the frame type information (step S402).

  Here, when the corresponding frame is not an intra-encoded frame, the image quality evaluation apparatus 3 advances the process to step S214 to advance the frame pointer.

  On the other hand, when the corresponding frame is an intra-coded frame, the evaluation target frame extraction unit 31 extracts each pixel value using the frame indicated by the frame pointer as the evaluation target frame (step S203).

  Next, the neighboring frame extraction unit 12 extracts one or more frames as neighboring frames from frames consecutive before and after the evaluation target frame extracted in step S203 (step 404).

  For example, the neighboring frame extraction unit 12 may extract a total of four frames, two frames before and after the evaluation target frame.

  The image quality evaluation apparatus 3 operates in the same manner as the image quality evaluation apparatus 2 as the second embodiment of the present invention for each neighboring frame from step S205 to S212. As a result, the image quality evaluation apparatus 3 calculates difference information between each pixel set obtained by dividing the frame determined to be an intra-coded frame and the corresponding pixel set in each neighboring frame.

  Next, the image quality evaluation apparatus 3 calculates a weighting coefficient for the evaluation target frame, similarly to the image quality evaluation apparatus 2 as the second embodiment of the present invention (step S213).

  Then, the image quality evaluation apparatus 3 advances the frame pointer (step S214), and calculates difference information from each neighboring frame for all frames determined to be intra-coded frames among the frames constituting the deteriorated image (step S402). S203 to S215).

  Next, similarly to the image quality evaluation apparatus 2 as the second embodiment of the present invention, the image quality evaluation apparatus 3 applies the weighting coefficient calculated in step S213 to each difference information calculated in step S210, and An evaluation value is calculated (step S216).

  Thus, the image quality evaluation apparatus 3 ends the operation.

  Next, effects of the third exemplary embodiment of the present invention will be described.

  The image quality evaluation apparatus according to the third embodiment of the present invention can perform image quality evaluation of a deteriorated image with higher accuracy without referring to information on the decoding process with a single deteriorated image.

  The reason will be described. In general, the manner in which deterioration occurs differs between an intra-coded frame and an inter-coded frame. For example, an intra-coded frame is encoded so that deterioration is reduced, and an inter-coded frame is encoded so that deterioration is slightly increased. In addition, it is conceivable that degradation occurs substantially uniformly in inter-coded frames. Therefore, there is a high possibility that calculating difference information between inter-coded frames will not be an effective index for image quality evaluation. In the image quality evaluation apparatus as the third embodiment of the present invention, the evaluation target frame extraction unit extracts an intra-coded frame as an evaluation target frame using the intra-coded frame detection unit. At this time, inter-coded frames are continuous before and after the intra-coded frame. Therefore, the image quality evaluation apparatus as the third embodiment of the present invention calculates difference information of each pixel set between an intra-coded frame and an inter-coded frame that is a neighboring frame. Therefore, the difference information calculated for each pixel set between the intra-coded frame and the inter-coded frame is an effective index for evaluating the degree of deterioration. As described above, the image quality evaluation apparatus according to the third embodiment of the present invention may be an effective index without using difference information between inter-coded frames that are not likely to be an effective index. This is because the image quality evaluation value is calculated using difference information between a high intra-coded frame and an inter-coded frame.

  In each embodiment of the present invention, the operation example has been described in which difference information is calculated for all pixel sets divided into a predetermined size in all intra-coded frames in the degraded image to be evaluated. In addition to such an operation example, the image quality evaluation apparatus according to the present embodiment may calculate the above-described difference information by thinning out the evaluation target frame or the divided pixel set to calculate the image quality evaluation value.

  In the embodiment of the present invention, the example in which the neighboring frame extraction unit extracts a total of four frames before and after the evaluation target frame as the neighboring frames has been described. However, the number of neighboring frames is not limited. . In addition to such an operation example, the image quality evaluation apparatus according to the present embodiment may extract one or more consecutive frames before and after the evaluation target frame as neighboring frames. In addition, the evaluation target frame and the neighboring frame do not necessarily have to be adjacent to each other, such as two or around three or three frames around the evaluation target frame. However, it is preferable that the neighborhood frame extraction unit extracts the inter-coded frame as a neighborhood frame.

(Fourth embodiment)
Next, a fourth embodiment of the present invention will be described in detail with reference to the drawings. Note that in each drawing referred to in the description of the present embodiment, the same reference numerals are given to the same configuration and steps that operate in the same manner as in the third embodiment of the present invention, and the detailed description in the present embodiment. Description is omitted.

  First, FIG. 9 shows a functional block configuration of an image quality evaluation apparatus 4 as a fourth embodiment of the present invention. In FIG. 9, the image quality evaluation apparatus 4 includes an intra-coded frame detection unit 45 instead of the intra-coded frame detection unit 35 with respect to the image quality evaluation apparatus 3 as the third embodiment of the present invention. Is different.

  Here, the image quality evaluation device 4 is configured by a general-purpose computer device including a CPU, a RAM, a ROM, and an auxiliary storage device such as a hard disk drive. Each functional block of the image quality evaluation apparatus 4 is constituted by a CPU that reads a computer program module stored in a ROM or an auxiliary storage device into a RAM and executes it. Note that the hardware configuration of each functional block of the image quality evaluation apparatus 4 is not limited to the above configuration.

  The intra-coded frame detection unit 45 determines whether or not the corresponding frame is an intra-coded frame by analyzing information on the frame included in the degraded image.

  For example, the intra-coded frame detection unit 45 may determine whether or not the frame is an intra-coded frame based on the remaining amount of the high frequency component of the spatial frequency.

  Here, the reason why it can be determined whether or not the frame is an intra-coded frame based on the remaining amount of the high frequency component will be described. In general, in an intra-coded frame, the amount of information is compressed by cutting information on high frequency components of the spatial frequency of luminance and color difference information. On the other hand, in an inter-coded frame, the information amount is compressed by removing information on differences between frames. Therefore, in the intra-coded frame, there is a tendency that high frequency components are few. Therefore, the intra-coded frame detection unit 45 determines that a frame in which the remaining amount of high-frequency components is smaller than the threshold is an intra-coded frame. Alternatively, the intra-coded frame detection unit 45 obtains an average value per remaining frame of the remaining amount of the high frequency component in one or more consecutive frames immediately adjacent to the evaluation target frame, and a frame having a smaller remaining amount than the obtained average value. May be determined as an intra-coded frame.

  The remaining amount of the high frequency component can be calculated as the sum of alternating current components as a result of the discrete cosine transform for each pixel set of a predetermined size (8 pixels × 8 pixels or the like), but other calculation methods may be used. Good.

  Further, for example, the intra-coded frame detection unit 45 may determine whether the frame is an intra-coded frame based on the block distortion amount.

  Here, the reason why it can be determined whether or not the frame is an intra-coded frame based on the block distortion amount will be described. In general, an intra-coded frame is a prediction signal in inter-frame prediction of subsequent frames, and is thus coded so as to reduce image quality degradation. Therefore, the amount of block distortion tends to be small in an intra-coded frame. Therefore, the intra-coded frame detection unit 45 determines that a frame having a block distortion amount smaller than the threshold is an intra-coded frame. Alternatively, the intra-coded frame detection unit 45 obtains an average value of the block distortion amount per frame in one or more frames that are immediately adjacent to the evaluation target frame, and intra-frames that have a smaller block distortion amount than the obtained average value are intra-framed. It may be determined as an encoded frame.

  The operation of the image quality evaluation apparatus 4 configured as described above will be described with reference to FIG.

  First, the evaluation target frame extraction unit 31 acquires a degraded image as an image quality evaluation target (step S101).

  Next, the evaluation target frame extraction unit 31 aligns the frame pointer with one of the frames constituting the deteriorated image acquired in step S101 (step S202).

  For example, the evaluation target frame extraction unit 31 may align the frame pointer with the head of the frame constituting the deteriorated image.

  Next, the evaluation target frame extraction unit 31 extracts each pixel value of the frame indicated by the frame pointer (step S203).

  Next, the intra encoded frame detection unit 45 determines whether or not the frame is an intra encoded frame based on information calculated from the corresponding frame (step S602).

  For example, as described above, the intra-coded frame detection unit 45 calculates the residual amount of the high frequency component of the spatial frequency or the block distortion amount from the corresponding frame, and when the calculated value is smaller than the threshold, It may be determined that the frame is a frame.

  Here, if the corresponding frame is not an intra-encoded frame, the image quality evaluation device 4 advances the process to step S214.

  On the other hand, when the corresponding frame is an intra-coded frame, the image quality evaluation apparatus 4 operates in the same manner as the image quality evaluation apparatus 3 as the third embodiment of the present invention from step S404, S205 to S216. The image quality evaluation value is calculated.

  Thus, the image quality evaluation device 4 ends the operation.

  Next, effects of the fourth exemplary embodiment of the present invention will be described.

  The image quality evaluation apparatus according to the fourth embodiment of the present invention can perform the image quality evaluation of a deteriorated image with high accuracy without referring to other external information with a single deteriorated image.

  The reason will be described. In the third embodiment of the present invention, detection of an intra-coded frame, which is a process for performing high-quality image evaluation with a single degraded image, is performed by acquiring frame type information from the outside.

  In contrast, in the image quality evaluation apparatus as the fourth exemplary embodiment of the present invention, the intra-coded frame detection unit determines whether the frame is an intra-coded frame by analyzing the corresponding frame information. is doing. Thereby, the image quality evaluation apparatus as the fourth exemplary embodiment of the present invention can detect an appropriate intra-coded frame as an evaluation target frame without referring to other external information. Then, the image quality evaluation apparatus according to the fourth embodiment of the present invention is configured such that each pixel set is between an intra-coded frame detected without referring to external information and an inter-coded frame that is a neighboring frame. Difference information is calculated. Therefore, the image quality evaluation apparatus as the third exemplary embodiment of the present invention detects difference information between an intra-coded frame and an inter-coded frame by detecting an intra-coded frame without referring to external information. This is because the image quality evaluation value can be calculated using.

  In each of the above-described embodiments, the example in which the neighboring frame extracting unit extracts the frame immediately after the evaluation target frame and the example of extracting the two frames before and after the center has been described. It is not intended to limit.

  Moreover, in the above-described second to fourth embodiments, the example in which the difference information calculation unit divides the evaluation target frame into a pixel set of 16 pixels × 16 pixels has been described. However, the difference information calculation unit of the present invention The size of the pixel set to be handled is not limited.

  In the second to fourth embodiments described above, as in the first embodiment, information such as luminance, color difference, and RGB can be applied as pixel values.

  Also in the second to fourth embodiments described above, as in the first embodiment, the difference information between each pixel set of the evaluation target frame and the corresponding pixel set of the neighboring frame is used as the difference information. The sum of absolute differences, the sum of squares of differences between pixel values, the difference of average pixel values, the difference of pixel value dispersion, the activity difference, and the like can be applied.

  Also in the third to fourth embodiments described above, as in the second embodiment, as the difference value used when estimating the corresponding pixel set, the sum of absolute differences between pixel values, the pixel value Difference sum of squares, average difference of pixel values, difference of dispersion of pixel values, activity difference, and the like can be applied.

  Also in the third to fourth embodiments described above, as in the second embodiment, the weighting coefficient is information representing spatial frequency, temporal frequency, specific color, block distortion, transmission error, blur, or the like. Can be calculated based on

  In each embodiment of the present invention described above, the operation of the image quality evaluation apparatus described with reference to each flowchart is stored in a storage device (storage medium) of a computer apparatus as a computer program of the present invention. Such a computer program may be read and executed by the CPU. In such a case, the present invention is constituted by the code of the computer program or a storage medium.

  Moreover, each embodiment mentioned above can be implemented in combination as appropriate.

  The present invention is not limited to the above-described embodiments, and can be implemented in various modes.

Moreover, although a part or all of said embodiment can be described also as the following additional remarks, it is not restricted to the following.
(Appendix 1)
An evaluation target frame extraction unit that extracts an evaluation target frame from the moving image;
A neighborhood frame extraction unit for extracting a neighborhood frame of the evaluation target frame;
Difference information representing the difference between information extracted from each pixel set included in the evaluation target frame and information extracted from a corresponding pixel set that is a pixel set at a position corresponding to the pixel set in the neighboring frame is calculated. A difference information calculation unit,
An image quality evaluation unit that calculates an image quality evaluation value of the moving image based on the difference information calculated for each pixel set;
An image quality evaluation device.
(Appendix 2)
The image quality evaluation apparatus according to appendix 1, wherein the evaluation target frame extraction unit extracts an intra-coded frame as the evaluation target frame.
(Appendix 3)
The evaluation target frame extraction unit selects a frame associated with type information indicating an intra-coded frame based on frame type information associated with each frame included in the moving image in advance. The image quality evaluation apparatus according to appendix 2, wherein the image quality evaluation apparatus is extracted as a frame.
(Appendix 4)
The evaluation target frame extraction unit determines whether the frame is an intra-encoded frame based on information calculated from each frame included in the moving image, and determines that the frame is an intra-encoded frame The image quality evaluation apparatus according to appendix 2, wherein a frame is extracted as the evaluation target frame.
(Appendix 5)
The evaluation target frame extraction unit calculates a residual amount of a high-frequency component of a spatial frequency in each frame included in the moving image, and based on the calculated residual amount, whether or not the frame is an intra-coded frame The image quality evaluation apparatus according to appendix 4, characterized by:
(Appendix 6)
The evaluation target frame extraction unit calculates a block distortion amount of each frame included in the moving image, and determines whether the frame is an intra-coded frame based on the calculated block distortion amount. The image quality evaluation apparatus according to appendix 4, which is characterized.
(Appendix 7)
From the supplementary note 1, the difference information calculation unit calculates the difference information between each pixel set obtained by dividing the evaluation target frame into a predetermined size and the corresponding pixel set. The image quality evaluation apparatus according to any one of appendix 6.
(Appendix 8)
The difference information calculation unit estimates a position of the corresponding pixel set based on a difference between information based on each pixel set of the evaluation target frame and information based on an arbitrary pixel set in the neighboring frame. The image quality evaluation apparatus according to any one of appendix 1 to appendix 7.
(Appendix 9)
The difference information calculation unit does not calculate the difference information when a difference between information based on each pixel set of the evaluation target frame and information based on the corresponding pixel set is larger than a threshold value. The image quality evaluation apparatus according to appendix 8.
(Appendix 10)
The difference information calculation unit estimates the position of the corresponding pixel set in an area where the relative position of the evaluation target frame with respect to each pixel set is in a predetermined range. Image quality evaluation device.
(Appendix 11)
The difference information calculation unit calculates, as the difference information, a sum of absolute differences between a pixel value included in each pixel set of the evaluation target frame and a pixel value included in the corresponding pixel set. The image quality evaluation apparatus according to any one of appendix 1 to appendix 10.
(Appendix 12)
The difference information calculation unit calculates, as the difference information, a sum of squares of a difference between a pixel value included in each pixel set of the evaluation target frame and a pixel value included in the corresponding pixel set. The image quality evaluation apparatus according to any one of 1 to Appendix 10.
(Appendix 13)
The difference information calculation unit calculates, as the difference information, a difference between an average of pixel values included in each pixel set of the evaluation target frame and an average of pixel values included in the corresponding pixel set. The image quality evaluation apparatus according to any one of appendix 1 to appendix 10.
(Appendix 14)
The difference information calculation unit calculates, as the difference information, a difference between a variance of pixel values included in each pixel set of the evaluation target frame and a variance of pixel values included in the corresponding pixel set. The image quality evaluation apparatus according to any one of appendix 1 to appendix 10.
(Appendix 15)
The difference information calculation unit, as the difference information, an activity value representing an average of absolute values of differences between an average of pixel values and each pixel value included in each pixel set of the evaluation target frame;
11. The image quality evaluation apparatus according to any one of appendix 1 to appendix 10, wherein a difference from the activity value of the corresponding pixel set is calculated.
(Appendix 16)
The image quality evaluation apparatus according to any one of Supplementary Note 11 to Supplementary Note 15, wherein the difference information calculation unit uses a luminance value as the pixel value.
(Appendix 17)
The image quality evaluation unit weights the difference information calculated for each pixel set based on information related to the pixel set, and uses the weighted difference information to calculate the image quality evaluation value. The image quality evaluation apparatus according to any one of Supplementary Note 1 to Supplementary Note 16, which is calculated.
(Appendix 18)
18. The image quality evaluation according to appendix 17, wherein the image quality evaluation unit performs the weighting on the difference information calculated for the pixel set using information calculated for each frame including the pixel set. apparatus.
(Appendix 19)
18. The image quality evaluation apparatus according to appendix 17, wherein the image quality evaluation unit weights difference information calculated for the pixel set using information calculated for each pixel set.
(Appendix 20)
20. The image quality evaluation apparatus according to any one of appendix 17 to appendix 19, wherein the image quality evaluation unit performs the weighting based on a spatial frequency associated with each pixel set.
(Appendix 21)
The image quality evaluation apparatus according to any one of appendix 17 to appendix 20, wherein the image quality evaluation unit performs the weighting based on a time frequency associated with each pixel set.
(Appendix 22)
The image quality evaluation apparatus according to any one of appendix 17 to appendix 21, wherein the image quality evaluation unit performs the weighting based on a color associated with each pixel set.
(Appendix 23)
The image quality evaluation apparatus according to any one of appendix 17 to appendix 22, wherein the image quality evaluation unit performs the weighting based on a block distortion amount related to each pixel set.
(Appendix 24)
The image quality evaluation apparatus according to any one of appendix 17 to appendix 23, wherein the image quality evaluation unit performs the weighting based on a transmission error amount related to each pixel set.
(Appendix 25)
25. The image quality evaluation apparatus according to any one of appendix 17 to appendix 24, wherein the image quality evaluation unit performs the weighting based on an amount of blur associated with each pixel set.
(Appendix 26)
Extract the frame to be evaluated from the video,
Extracting neighboring frames of the evaluation target frame;
Difference information representing the difference between information extracted from each pixel set included in the evaluation target frame and information extracted from a corresponding pixel set that is a pixel set at a position corresponding to the pixel set in the neighboring frame is calculated. And
An image quality evaluation method for calculating an image quality evaluation value of the moving image based on the difference information calculated for each pixel set.
(Appendix 27)
27. The image quality evaluation method according to attachment 26, wherein an intra-coded frame is extracted as the evaluation target frame.
(Appendix 28)
An evaluation target frame extraction step for extracting an evaluation target frame from the moving image;
A neighboring frame extracting step of extracting a neighboring frame of the evaluation target frame;
Difference information representing the difference between information extracted from each pixel set included in the evaluation target frame and information extracted from a corresponding pixel set that is a pixel set at a position corresponding to the pixel set in the neighboring frame is calculated. Difference information calculation step to perform,
An image quality evaluation step of calculating an image quality evaluation value of the moving image based on the difference information calculated for each pixel set;
Is a computer program that causes a computer device to execute.
(Appendix 29)
29. The computer program according to appendix 28, wherein in the evaluation target frame extraction step, an intra-coded frame is extracted as the evaluation target frame.

1, 2, 3, 4 Image quality evaluation apparatus 11, 31 Evaluation target frame extraction unit 12 Neighboring frame extraction unit 13, 23 Difference information calculation unit 14, 24 Image quality evaluation unit 35, 45 Intra encoded frame detection unit

Claims (10)

An evaluation target frame extraction unit that extracts an evaluation target frame from the moving image;
A neighborhood frame extraction unit for extracting a neighborhood frame of the evaluation target frame;
Difference information representing the difference between information extracted from each pixel set included in the evaluation target frame and information extracted from a corresponding pixel set that is a pixel set at a position corresponding to the pixel set in the neighboring frame is calculated. A difference information calculation unit,
An image quality evaluation unit that calculates an image quality evaluation value of the moving image based on the difference information calculated for each pixel set;
An image quality evaluation device.   The image quality evaluation apparatus according to claim 1, wherein the evaluation target frame extraction unit extracts an intra-coded frame as the evaluation target frame.   The evaluation target frame extraction unit determines whether the frame is an intra-encoded frame based on information calculated from each frame included in the moving image, and determines that the frame is an intra-encoded frame The image quality evaluation apparatus according to claim 2, wherein a frame is extracted as the evaluation target frame.   The difference information calculation unit estimates a position of the corresponding pixel set based on a difference between information based on each pixel set of the evaluation target frame and information based on an arbitrary pixel set in the neighboring frame. The image quality evaluation apparatus according to any one of claims 1 to 3.   The difference information calculation unit does not calculate the difference information when a difference between information based on each pixel set of the evaluation target frame and information based on the corresponding pixel set is larger than a threshold value. The image quality evaluation apparatus according to claim 4. The difference information calculation unit, as the difference information, an activity value representing an average of absolute values of differences between an average of pixel values and each pixel value included in each pixel set of the evaluation target frame;
6. The image quality evaluation apparatus according to claim 1, wherein a difference from an activity value of the corresponding pixel set is calculated.   The image quality evaluation unit weights the difference information calculated for each pixel set based on information related to the pixel set, and uses the weighted difference information to calculate the image quality evaluation value. The image quality evaluation apparatus according to claim 1, wherein the image quality evaluation apparatus calculates the image quality.   8. The image quality evaluation unit according to claim 7, wherein the image quality evaluation unit performs the weighting based on at least one of a block distortion amount, a transmission error amount, and a blur amount related to each pixel set. Image quality evaluation device. Extract the frame to be evaluated from the video,
Extracting neighboring frames of the evaluation target frame;
Difference information representing the difference between information extracted from each pixel set included in the evaluation target frame and information extracted from a corresponding pixel set that is a pixel set at a position corresponding to the pixel set in the neighboring frame is calculated. And
An image quality evaluation method for calculating an image quality evaluation value of the moving image based on the difference information calculated for each pixel set. An evaluation target frame extraction step for extracting an evaluation target frame from the moving image;
A neighboring frame extracting step of extracting a neighboring frame of the evaluation target frame;
Difference information representing the difference between information extracted from each pixel set included in the evaluation target frame and information extracted from a corresponding pixel set that is a pixel set at a position corresponding to the pixel set in the neighboring frame is calculated. Difference information calculation step to perform,
An image quality evaluation step of calculating an image quality evaluation value of the moving image based on the difference information calculated for each pixel set;
Is a computer program that causes a computer device to execute.

Images