JP5013487B2 - Video objective image quality evaluation system - Google Patents

Description

  The present invention relates to a video objective image quality evaluation apparatus, and more particularly to a video objective image quality evaluation apparatus that evaluates the quality of an image deteriorated by compression encoding of a moving image without a reference image.

  When storing and transmitting digital video, the amount of information is usually reduced by compression encoding. Here, compression coding generally means lossy compression. Lossy compression means that when the encoded information (encoded bitstream) is decoded, the original image before encoding is not completely reconstructed and the visual deterioration is sufficiently suppressed, that is, the image quality is sufficiently high. It is a compression format that reduces the amount of information under the condition of keeping. Typical examples of lossy compression include MPEG-2 and H.264 (Non-Patent Documents 1 and 2 below).

  In these lossy compressions, encoding is performed after sufficiently suppressing visual deterioration as described above, but the deterioration is visually recognized as the compression rate increases, that is, as the bit rate decreases. Become so. Further, even if the compression rate is the same, there is a property that the degree of degradation visually recognized varies depending on the characteristics of the image such as the fineness, the magnitude of movement, and the complexity of the object in the screen. For this reason, there is a need for a technique for quantitatively measuring image quality degradation accompanying lossy compression.

  Conventional measurement of image quality has been performed by a technique called subjective evaluation. This is a collection of about 20 subjects, video is presented to the subjects, a score is assigned according to the subjectivity of the subject, and a numerical value obtained by statistically processing the score (eg average of the scores) is defined as the quality of the video. It is. Representative methods of the subjective evaluation method are defined in ITU-R recommendation BT.500-11, ITU-T recommendation P.910, etc. (Non-Patent Documents 3 and 4). However, subjective evaluation is not a simple means of evaluating video quality, such as satisfying the strict viewing conditions stipulated by the recommendation and having to recruit a large number of subjects.

  Therefore, objective image quality evaluation for extracting one or a plurality of numerical indexes indicating the feature of the video, which is called a video feature amount, by analyzing the video signal and deriving the quality of the video from the video feature amount has been studied. Yes. The image quality derived by objective image quality evaluation is an estimate of subjective image quality, and is intended to be used as an alternative to subjective image quality evaluation.

  ITU-T J.143 (Non-Patent Document 5) defines a framework for objective image quality evaluation methods. The objective evaluation method framework is classified into the following three categories depending on which stage of transmission or storage is used for evaluation.

  (1) Full Reference (FR) type: A method of using baseband information of an original image and a decoded image (when stored) before compression encoding, or a transmitted image and a received image (when transmitted).

  (2) No Reference (NR) type: A method that uses only baseband information of a decoded image or received image (information of an original image or a transmitted image is not used).

  (3) Reduced Reference (RR) type: A method of using an image feature amount of an original image or transmission image with limited information amount and baseband information of a decoded image or reception image.

  Since the full reference type can use baseband images before and after storage or transmission, the subjective image quality estimation accuracy is the highest among the three frameworks. On the other hand, since the No Reference type uses only the baseband image after storage or transmission, it is inferior to Full Reference in terms of accuracy. The Reduced Reference type uses the image feature quantity of the original image or transmission image in addition to the baseband information of the decoded image or reception image used in the No Reference type. Here, the image feature amount is about several tens to several hundred kbps, and is limited to an information amount that is sufficiently smaller than the baseband information of the original image. In the RR type, for the purpose of improving the estimation accuracy of subjective image quality compared to the NR type, the image feature quantity on the transmitting side is transmitted to the receiving side using a data line prepared separately from the video line during video transmission. Yes.

  Among the above three types of frameworks, the objective evaluation method based on the FR type includes ITU-T recommendation J.144 (Non-patent document 6), ITU-T recommendation J.267 (Non-patent document 7), and JP2008. -35357 (Patent Document 11) and the like exist. Non-Patent Document 6 is an objective image quality evaluation method for standard TV (SDTV) encoding degradation, and Non-Patent Document 7 and Patent Document 11 are objective image quality images that are often used in multimedia applications. The evaluation method is shown.

  As an objective evaluation method based on the RR type, ITU-T recommendation J.246 (Non-Patent Document 8) is known. Non-Patent Document 8 discloses an objective evaluation method based on a video format for multimedia applications.

On the other hand, as for the objective evaluation method based on the NR type, methods for still image compressed images have been proposed by Kawasaki et al. (Non-Patent Document 9), Farias et al. (Non-Patent Document 10), etc. There is still no established method for evaluation.
ITU-T Recommendation H.262, “Information technology-Generic coding of moving pictures and associated audio information: Video” ITU-T Recommendation H.264, “Advanced video coding for generic audiovisual services” Recommendation ITU-R BT.500-11, “Methodology for the subjective assessment of the quality of television pictures” ITU-T Recommendation P.910, “Subjective video quality assessment methods for multimedia applications” ITU-T Recommendation J.143, “User requirements for objective perceptual video quality measurements in digital cable television” ITU-T Recommendation J.144, “Objective perceptual video quality measurement techniques for digital cable television in the presence of a full reference” ITU-T Recommendation J.247, “Objective perceptual multimedia video quality measurement in the presence of a full reference” ITU-T Recommendation J.246, “Perceptual audiovisual quality measurement techniques for multimedia services over digital cable television networks in the presence of a reduced bandwidth reference” Mylene CQ Farias et al, “NO-REFERENCE VIDEO QUALITY METRIC BASED ON ARTIFACT MEASUREMENTS”, ICIP 2005, cr2987 Kawasaki et al. "NR image quality evaluation model of coded video using image restoration algorithm", PCSJ 2004, P-2-02 JP 2008-35357 A

  As described above, the NR type image quality evaluation is inferior to the FR type in terms of subjective image quality estimation accuracy. However, since only the decoded image or the received image is input, there is an advantage that the system configuration is easy. This method is expected to be put to practical use for the purpose of video surveillance. In this NR type image quality evaluation, the establishment of an NR type evaluation technique for moving images is an issue.

  In view of the above problems, the present invention has an object to provide an NR-type objective image quality evaluation device for video that accurately estimates subjective image quality from only a baseband signal of a lossy-encoded video image. Yes.

  In order to achieve the above object, the present invention performs Hadamard transform on a pixel block of a decoded image in an objective image quality evaluation apparatus for video that estimates subjective image quality using only a decoded image without using a reference image, and the conversion A block distortion feature amount calculation unit for obtaining a block distortion feature amount from power of a component representing a signal change at a coding block boundary of coefficients, and a pixel variance in a pixel block of the decoded image is lower than a given threshold, Objectively based on a flicker feature quantity calculation unit for calculating the inter-frame difference in the pixel block, and an approximation function having two arguments, the average of the block distortion feature quantity of each pixel block in the decoded image and the sum of the flicker feature quantities. It is characterized in that it has an objective evaluation scale calculator for deriving image quality.

  According to the present invention, it is possible to perform NR objective image quality evaluation that accurately estimates the subjective image quality of an encoded video without using a reference image (transmission image or original image). Thereby, it is possible to realize high-accuracy image quality evaluation and image quality monitoring with a simpler system configuration.

  Hereinafter, the present invention will be described in detail with reference to the drawings. FIG. 1 is a functional block diagram showing the configuration of an embodiment of the present invention. In the following, the present invention will be described with an example in which an objective evaluation scale is output using the luminance signal of the decoded image divided into 16 × 16 pixel blocks as an input, but the present invention is not limited to 16 × 16 pixel blocks, and other It can also be applied to blocks of the size.

  As shown in the figure, the automatic monitoring apparatus according to the present embodiment includes a block distortion feature amount calculation unit 1, a flicker feature amount calculation unit 2, and an objective evaluation scale calculation unit 3.

  First, the function of the block distortion feature value calculation unit 1 will be described. Block distortion is a common degradation factor in encoded images that are processed in units of blocks such as MPEG-2 and H.264, and the degree of degradation is considered to have a high correlation with subjective image quality. Block distortion is caused by a large change in signal value at a block boundary. It can be said that a block is recognized as degraded when the luminance signal contains many stepped waveform components at the block boundary.

Therefore, in this embodiment, the evaluation image is divided into 16 × 16 pixel blocks, and the power of the [8, 8] component when the Hadamard transform is applied to each block is used as a block distortion measure. Frame f of the evaluation image, a matrix representing the X _f of the pixel values of the block _B, and the _B, X _f, transform coefficient P _f of _{B, B} is determined as follows.
P _{f, B} = H ₁₆ X _{f, B} H ₁₆ ^T

Here, H ₁₆ is a 16 × 16 Hadamard matrix. Hadamard matrix is the basic matrix

とするとき、下記の再帰式で定義される行列である。

Is a matrix defined by the following recursive formula.

H₁₆は上式において、ｎ＝４とした場合の行列となる。
H₁₆の基底画像は、図２に示すように、様々な細かさを持った矩形画像の集合となっている。ここで、白色が画素値 +1/16、黒色が画素値 -1/16を示している。

H ₁₆ is a matrix in the above equation where n = 4.
As shown in FIG. 2, the base image of H ₁₆ is a set of rectangular images having various finenesses. Here, white indicates a pixel value of +1/16, and black indicates a pixel value of −1/16.

  Among these, the (8,8) component has a boundary of the 8th line and the 8th pixel in the block, as shown in FIG. 3, so that the signal value changes stepwise at the 8 × 8 pixel block boundary. It represents the size and is suitable as a block distortion measure.

Finally, the block distortion feature amount Blk _{f, b} in the block is defined as follows.

In the above equation, normalization is performed with the DC component P _{f, b} [0, 0] in consideration of the occurrence of blur due to the improvement of the compression ratio and the increase of the DC component due thereto.

The block distortion feature quantity calculation unit 1 performs the above processing, and the block distortion feature quantity calculation unit 1 outputs the block distortion feature quantity Blk _{f, b} .

  Next, the function of the flicker feature quantity calculation unit 2 will be described. Flicker is a degradation that is detected when there is a large quality fluctuation at every intra-frame insertion period of motion compensation predictive coding, and is perceived by a sudden change in luminance between successive frames. This is one of video features that have a high correlation with subjective image quality as well as block distortion.

In the present embodiment, in order to capture a temporal change in luminance value in a 16 × 16 pixel block, the luminance inter-frame difference in each pixel block is defined as a flicker feature amount. However, it is known that flicker tends to be visually detected in a flat pattern, but is less likely to be detected in a complex pattern area than the flat area even when the same luminance value change occurs. Therefore, in this embodiment, the average value in the sequence (for example, the average value for 10 to 15 seconds) of the inter-frame difference signal of the block whose luminance signal variance var is equal to or less than the given threshold V _th is extracted as the flicker feature amount. . When the (i, j) element in the signal X _{f, b} in the 16 × 16 pixel block is expressed as X _{f, b} [i, j], the flicker feature amount Flk _{f, b} of the frame f, block _b is Are defined as follows.

  Here, the variance value var can be set as follows, for example. When one pixel is expressed by 8 bits, the pixel value has a range of 0 to 255. When the fluctuation of the pixel value in the block falls within about 10% of the entire width (256 levels) of the pixel value, that is, about 25 levels, it is visually recognized as a flat picture. In this case, since the dispersion value var in the block is usually in the range of about 100 to 1000, it is preferable to set the threshold value to an appropriate value within this range.

  Next, the function of the objective evaluation scale calculation unit 3 will be described. The objective evaluation scale calculation unit 3 receives the average value within the sequence of the block distortion feature amount and the average value within the sequence of the flicker feature amount, and outputs the objective evaluation scale.

  In the objective evaluation scale calculation unit 3, first, the average value (Blockiness) of the block distortion feature value and the total average value (Flickerness) of the flicker feature value in the sequence are defined as follows.

Here, N _B and N _F are the total number of 16 × 16 pixel blocks in the frame and the total number of frames in the sequence. Next, the objective evaluation value Q _obj is approximated using Blockiness and Flickerness. In general, the objective evaluation value is approximated in the following format.

  Where f () represents a given function. Since the optimum approximate expression varies depending on conditions such as the image format to be evaluated, the encoding method, and the encoding bit rate, a value that maximizes the correlation with the subjective evaluation value is selected under these conditions.

  As an example of the definition of f (), there is a method in which a and b are constants and expressed by a weighted sum as in the following equation.

As another method, when a, b, g ₁ and g ₂ are constants, there is an approximation using the following equation.

Constants a ₁ , b ₂ , g ₁ , and g ₂ in the above equation are set so that the correlation between the objective evaluation value Q _obj and the subjective evaluation value is maximized. The correlation between the objective evaluation value and the subjective evaluation value is obtained by performing regression analysis on a series of objective evaluation values and a series of subjective evaluation values obtained using a plurality of evaluation videos.

  FIG. 4 shows an example of regression analysis. When each data series is plotted with the objective evaluation value on the horizontal axis and the subjective evaluation value on the vertical axis, both can be approximated by a certain regression curve. As a regression curve, in addition to a linear function, a nonlinear function such as a high-order polynomial or a logistic function is applied. The objective of objective image quality evaluation is to estimate the subjective evaluation value, and the higher the accuracy of approximation by the regression curve (that is, the shorter the distance between each plot point on the graph and the regression curve), the higher the performance.

  Next, an example of an experimental result by the present inventor is shown in FIG. FIG. 5 shows the relationship between the subjective image quality and the objective evaluation value according to the present invention in an HDTV (1920 × 1080 pixels, 30 frames / second) image encoded at H.264 4.0, 6.0, and 8.0 Mbps.

  Objective evaluation value

と定義した場合、主観画質と客観評価値は、ロジスティック関数

The subjective image quality and objective evaluation value are logistic functions

で近似することができる。このときの主観画質と客観評価値の相関係数は0.75であり、高い相関を得られていることが分かる。

Can be approximated by At this time, the correlation coefficient between the subjective image quality and the objective evaluation value is 0.75, which indicates that a high correlation is obtained.

It is a functional block diagram which shows the structure of one Embodiment of this invention. It is a diagram illustrating a base image example of H _16. It is an enlarged view of the (8,8) component of FIG. It is a figure which shows an example of a regression analysis. It is a figure which shows the experimental example of the relationship between subjective image quality and the objective evaluation value by this invention.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... Block distortion feature-value calculation part, 2 ... Flicker feature-value calculation part, 3 ... Objective evaluation scale calculation part

Claims (5)

In an objective image quality evaluation apparatus for video that estimates subjective image quality using only decoded images without using reference images,
A block distortion feature amount calculation unit that performs Hadamard transform on a pixel block of a decoded image and obtains a block distortion feature amount from power of a component that represents a signal change at an encoding block boundary among the transform coefficients;
A flicker feature amount calculation unit that calculates an inter-frame difference in the pixel block when the pixel variance in the pixel block of the decoded image is lower than a given threshold;
An objective evaluation scale calculation unit for deriving an objective image quality based on an approximate function having two arguments, an average of block distortion feature values and a sum of flicker feature values of each pixel block in the decoded image. An objective image quality evaluation device for video. The objective image quality evaluation device for video according to claim 1,
The block distortion feature amount calculation unit performs a Hadamard transform in units of 16 × 16 pixel blocks to calculate a block distortion feature amount, and calculates a square of a ratio between the (8,8) component and the (0,0) component. An objective image quality evaluation apparatus for video, characterized by using block distortion feature quantities. The objective image quality evaluation device for video according to claim 1 or 2,
The flicker feature quantity calculation unit uses the average value in the sequence of the inter-frame difference signal as the flicker feature quantity when the intra-block variance of the luminance signal of the decoded image is a given threshold value or less. Image quality evaluation device. The objective image quality evaluation apparatus for video according to any one of claims 1 to 3,
The objective evaluation scale calculation unit uses a weighted sum of an average of block distortion feature values and a sum of flicker feature values of each pixel block in a decoded image as an approximate function for deriving an objective image quality. Objective image quality evaluation device. The objective image quality evaluation apparatus for video according to any one of claims 1 to 3,
The objective evaluation scale calculation unit, as an approximation function for derivation of objective image quality, is a number obtained by raising the average of block distortion feature values of each pixel block of a decoded image to a power that is a predetermined power and a flicker feature of each pixel block of the decoded image. An objective image quality evaluation apparatus for video, characterized by using a weighted sum of numbers obtained by raising a sum of quantities to a power that is a predetermined power and further using a power that is a power that is different from the power.

Images