JP5450279B2 - Image quality objective evaluation apparatus, method and program - Google Patents

Description

  The present invention relates to a video quality objective evaluation apparatus, method, and program, and more particularly to objectively evaluate video quality in an IPTV (Internet Protocol TeleVisiton) service, a video distribution service using a mobile phone, a videophone service, and the like. The present invention relates to an image quality objective evaluation apparatus, method, and program.

  With the increase in communication line speed and bandwidth and the diversification of terminals that receive broadcast waves, an environment in which video services can be enjoyed using various terminals is being prepared.

  In the video service, deterioration such as block distortion or blur occurs in the video. The cause is that encoding is performed using the fact that the transmission band is generally limited, and bit errors and packet loss occur due to the deterioration of the quality of the transmission path.

  Therefore, in order to confirm that the video service is provided with good quality, the quality of the video experienced by the user is measured before or during service provision, and the quality of the video provided to the user is determined. It is important to confirm and monitor that it is good.

  Therefore, a video quality evaluation technique that can appropriately evaluate the video quality experienced by the user is necessary.

  Methods for evaluating video quality include a subjective evaluation method (for example, see Non-Patent Document 1) and an objective evaluation method (for example, Non-Patent Document 2).

  In the subjective evaluation method, it is necessary to have a large number of users evaluate videos in a facility where the evaluation environment (room illumination, room noise, etc.) can be reproduced. For this reason, the cost is high and the evaluation takes time, so that it is not suitable for use in which quality is evaluated in real time. Therefore, it is desired to develop a video quality evaluation method that derives a physical feature amount from a video signal and outputs a video quality evaluation value.

  Conventional objective evaluation methods include an evaluation technique for deriving video quality from features that affect video quality by comparing the video signals (pixel information) of the reference video and degraded video as input (for example, Non-patent document 2).

ITU-T Recommendation P. 910 "Subjective video quality assessment methods for multimedia applications". ITU-T recommendation J.247 "Objective perceptual multimedia video quality measurement in the presence of a full reference".

  However, the technique of Patent Document 2 is based on the premise that the resolution and format of the reference video and the degraded video match, and there is a problem that quality evaluation cannot be performed if they do not match. Specifically, in the 1Seg service of digital terrestrial television broadcasting service for mobile terminals, the standard video is HDTV (High Definition TeleVision) format video, but it has been abolished with QVGA (Quarter Video Graphics Array) size (320 X240 pixels), but the quality of the video displayed on the mobile device may be evaluated differently because the resolution of the reference video differs from that of the degraded video, such as the resolution and aspect ratio differ by manufacturer and model. Have difficulty.

  Here, a method of matching the reference video and degraded video by applying signal processing so as to match the resolution and format is also conceivable. However, this method has the problem that the effect of the applied signal processing comes out, proper matching cannot be achieved, and quality estimation cannot be performed.

  The present invention has been made in view of the above points, and is capable of estimating a highly accurate video quality evaluation value even when the resolution, format format, and display rate of the reference video and the deteriorated video are different. It is an object of the present invention to provide a reference type video quality objective evaluation apparatus, method and program.

  In the present invention, in order to achieve the above object, the pixel values of the reference image and the deteriorated image are not directly compared as in the prior art. A video quality objective evaluation apparatus, method, and program capable of evaluating video quality even when the resolution and the format are different by deriving the relative deterioration feature quantity to be extracted and using the feature quantity are provided.

The present invention (Claim 1) is a video quality objective evaluation apparatus for objectively evaluating video quality,
The relative degradation feature amount extracting relative deterioration characteristic amount obtained by normalizing the respective difference feature quantity of difference feature quantity as a pixel average between frames of the deteriorated video difference feature quantity between frames of criteria video Extraction means;
Video quality estimation means for deriving a video quality evaluation value based on the relative deterioration feature quantity extracted from the relative deterioration feature quantity extraction means;
It is characterized by having.

Further, the present invention (Claim 2) is the relative deterioration feature quantity extraction unit of Claim 1,
A first deterioration feature amount comparison unit that synchronizes the reference image and the deteriorated video using a timing at which both changes in the difference feature amount between the frames of the reference image and the deteriorated image both increase;
Or
A second deterioration feature amount comparison unit that compares a change amount for each corresponding frame when the resolution of the image of the reference image and the deterioration image is different;
Or
Third deterioration feature amount comparison means for comparing a change amount per unit time when the display frame rate of the reference image and the deteriorated image are different;
Or
As the difference feature amount between the frames of the reference image and the deteriorated image, the square of the difference between the pixel average of the difference feature amount between the frames of the reference image and the pixel average of the difference feature amount between the frames of the deteriorated image A ratio of pixel values of feature amounts, a value obtained by equalizing a pixel difference amount with a pixel value of a reference image, or a value obtained by statistically processing them, a fourth degradation feature amount comparison unit,
Have one of the following.

Further, according to the present invention (Claim 3), in the relative deterioration feature amount extraction unit according to Claim 1,
Reference video feature amount extraction means for deriving a difference feature amount between frames of the reference video;
Degrading image feature amount extraction means for deriving a difference feature amount between frames of the deteriorated image,
The reference image feature amount extraction unit and the deteriorated image feature amount extraction unit include:
When deriving a difference feature amount between frames of the reference image and the deteriorated image , pixel thinning or information on a specific area is used.

Further, according to the present invention (Claim 4), in the relative deterioration feature amount extraction unit according to Claim 1,
Encoding processing means is provided for outputting a video obtained by performing encoder processing on the inputted reference video as a new reference video.

Further, the present invention (Claim 5) is the video quality estimation means according to Claim 1 ,
Information obtained by reading from the storage means storing the parameters of the video to be evaluated extracted in advance or by directly comparing pixel information between the reference video and the degraded video is used.

The present invention (Claim 6) is a video quality objective evaluation method for objectively evaluating video quality,
The relative degradation feature amount extracting relative deterioration characteristic amount obtained by normalizing the respective difference feature quantity of difference feature quantity as a pixel average between frames of the deteriorated video difference feature quantity between frames of criteria video An extraction step;
And a video quality estimation step of deriving a video quality evaluation value based on the relative deterioration feature amount extracted from the relative deterioration feature amount extraction means.

Further, according to the present invention (Claim 7), in the relative deterioration feature amount extraction step of Claim 6,
A first deterioration feature amount comparison step for synchronizing the reference image and the deteriorated image using a timing at which both changes in the difference feature amount between frames of the reference image and the deteriorated image both increase;
Or
A second deterioration feature amount comparison step of comparing the amount of change for each corresponding frame when the resolution of the image of the reference image and the deterioration image is different;
Or
A third deterioration feature amount comparison step for comparing a change amount per unit time when the display frame rates of the reference image and the deterioration image are different;
Or
As the difference feature amount between the frames of the reference image and the deteriorated image, the square of the difference between the pixel average of the difference feature amount between the frames of the reference image and the pixel average of the difference feature amount between the frames of the deteriorated image Either a ratio of pixel values of feature amounts, a value obtained by equalizing pixel difference amounts with pixel values of a reference image, or a fourth deterioration feature amount comparison step using a value obtained by statistically processing them is performed.

The present invention (Claim 8) is characterized in that in the relative deterioration feature amount extraction step of Claim 6,
A reference image feature extraction step for deriving a difference feature between frames of the reference image;
Performing a degraded video feature amount extraction step for deriving a difference feature amount between frames of the degraded video,
In the reference image feature extraction step and the deteriorated image feature extraction step,
When deriving a difference feature amount between frames of the reference image and the deteriorated image , pixel thinning or information on a specific area is used.

Further, according to the present invention (Claim 9), in the relative deterioration feature amount extraction step of Claim 6,
A step of outputting a video obtained by performing encoder processing on the inputted reference video as a new reference video is performed.

Further, the present invention (Claim 10) is the video quality estimation step according to Claim 6 ,
Information obtained by reading from the storage means storing the parameters of the video to be evaluated extracted in advance or by directly comparing pixel information between the reference video and the degraded video is used.

  Moreover, this invention (Claim 11) is the video quality objective evaluation program for functioning a computer as each means which comprises the video quality objective evaluation apparatus of any one of Claim 1 thru | or 5.

  As described above, according to the present invention, even when the resolution, format, and display rate of the reference image and the deteriorated image are different, the difference in the image deterioration amount between the frames before and after the reference image and the deteriorated image is determined. By using the extracted relative deterioration feature quantity, it is possible to improve the point that it has been difficult to evaluate the quality by comparing the pixels directly, and to estimate the video quality value in the video service simply and with high accuracy.

  Accordingly, since the video service provider can monitor the video quality of the video service that the user actually views according to the present invention, the provider can check whether the provided service is provided with an appropriate quality to the user. .

  Thereby, the provider of the video service can appropriately grasp and manage the actual quality of the service before and during the provision.

It is a block diagram of the video quality objective evaluation apparatus in the 1st Embodiment of this invention. It is an example of steady quality DB in the 1st Embodiment of this invention. It is a figure which shows the interframe difference feature-value in the 1st Embodiment of this invention. 3 is a method for synchronizing reference feature / degraded inter-frame difference feature values in the first exemplary embodiment of the present invention. It is a unit time comparison method of the reference feature / degraded interframe difference feature value in the first embodiment of the present invention. It is a block diagram of the video quality objective evaluation apparatus in the 2nd Embodiment of this invention.

  Embodiments of the present invention will be described below with reference to the drawings.

[First Embodiment]
As shown in FIG. 1, the video quality objective evaluation apparatus according to the embodiment of the present invention uses a reference video signal 10 and a degraded video signal 20 to derive a video quality evaluation value 60, which is a full-reference type video quality objective. An evaluation device is realized.

  FIG. 1 shows a video quality objective evaluation apparatus according to the first embodiment of the present invention, which includes a relative deterioration feature quantity extraction unit 100A, a quality estimation unit 200, and a steady quality database 300.

  As shown in FIG. 2, the steady quality database 300 stores steady quality data including encoder conditions, steady quality values, coefficients, and evaluation formulas.

  The relative deterioration feature amount extraction unit 100A includes a reference image feature amount extraction unit 110, a deterioration image feature amount extraction unit 120, and a deterioration feature amount comparison unit 130.

  When the reference video signal 10 is input, the reference video feature amount extraction unit 110 extracts inter-frame difference information as illustrated in FIG. 3 and outputs the reference inter-frame difference feature amount 15 to the degradation feature amount comparison unit 130. . FIG. 3A shows the difference between reference video frames, and FIG. 3B shows the difference between deteriorated video frames. Here, color difference information or RGB information may be used as the reference inter-frame difference feature 15. Here, color difference information or RGB information may be used.

上記の、△R(ｉ，ｊ，ｋ)はｋフレーム目の画素（ｉ，ｊ）の基準映像の差分輝度値、R(ｉ，ｊ，ｋ)はｋフレーム目の画素（ｉ，ｊ）の基準映像の輝度値を表す。

ΔR (i, j, k) is the difference luminance value of the reference image of the pixel (i, j) in the kth frame, and R (i, j, k) is the pixel (i, j) in the kth frame. Represents the luminance value of the reference video.

  When the degraded video signal 20 is input, the degraded video feature quantity extraction unit 120 outputs the degraded inter-frame difference feature quantity 25 to the degradation feature quantity comparison unit 130 by the same processing as the reference video feature quantity extraction unit 110. Here, the following difference frame using a luminance value is used as the degraded inter-frame difference feature 25. Here, color difference information or RGB information may be used.

ここで、△P(ｉ，ｊ，ｋ)はｋフレーム目の画素（ｉ，ｊ）の基準映像の差分輝度値、Ｐ（ｉ，ｊ，ｋ）はｋフレーム目の画素（ｉ，ｊ）の基準映像の輝度値を表す。

Here, ΔP (i, j, k) is the difference luminance value of the reference image of the pixel (i, j) in the kth frame, and P (i, j, k) is the pixel (i, j) in the kth frame. Represents the luminance value of the reference video.

  Note that, in the derivation of inter-frame difference information performed by the reference video feature quantity extraction unit 110 and the degraded video feature quantity extraction unit 120, it is assumed that all pixels are used, but the ease of implementation can be reduced by reducing the processing amount. In order to promote the securing and the improvement of the processing speed and to achieve real time, pixel thinning or pixels in a specific video area can be used.

  The deterioration feature amount comparison unit 130 outputs the relative deterioration feature amount 30 by comparing the reference inter-frame difference feature amount 15 and the deteriorated inter-frame difference feature amount 25.

  The deterioration feature amount comparison unit 130 needs to synchronize the reference image with the deteriorated image in order to compare the difference feature amount 15 between the reference frames and the difference feature amount 25 between the deterioration frames. For this reason, as shown in FIG. 4, synchronization is performed using timing at which a large change occurs in the deterioration feature amounts of the frames before and after the reference image and the deteriorated image due to a scene change or the like. 4A shows the peak A of the feature amount of the reference image, and FIG. 4B shows the peak B of the feature amount of the deteriorated image. In the figure, the peak A of the feature amount generated by the scene change of the reference video is synchronized with the peak B of the feature amount generated by the scene change generated by the degraded video.

  The unit for comparing the reference inter-frame difference feature 15 and the deteriorated inter-frame difference feature 25 is, as shown in FIG. 5, the reference inter-frame difference feature and the deteriorated inter-frame difference feature every 1/15 seconds. Compare the amount of change per unit time. Accordingly, evaluation can be performed even when the display frame rate of the reference image and the deteriorated image are different or when a frame is lost due to packet loss or the like.

  Further, as the relative deterioration feature amount 30, the following feature amounts (1) to (3) and combinations thereof are derived. These feature amounts are normalized by using the difference value between frames as a pixel average, and the influence of the difference in the image size between the reference image and the deteriorated image is taken into consideration.

  (1) Difference amount between previous and next frame pixel values (MSE):

但し、ｋフレーム目を対象とする。また、基準映像の解像度は水平方向Ｘ画素、垂直方向にＹ画素とし、劣化映像の解像度は水平方向Ｘ'画素、垂直方向にＹ'画素とする。

However, the target is the kth frame. Further, the resolution of the reference image is X pixels in the horizontal direction and Y pixels in the vertical direction, and the resolution of the deteriorated image is X ′ pixels in the horizontal direction and Y ′ pixels in the vertical direction.

  (2) Ratio of pixel values of previous and next frames:

但し、ｋフレーム目を対象とする。また、基準映像の解像度は水平方向Ｘ画素、垂直方向にＹ画素とし、劣化映像の解像度は水平方向Ｘ'画素、垂直方向にＹ'画素とする。

However, the target is the kth frame. Further, the resolution of the reference image is X pixels in the horizontal direction and Y pixels in the vertical direction, and the resolution of the deteriorated image is X ′ pixels in the horizontal direction and Y ′ pixels in the vertical direction.

  (3) Signal to noise ratio (PSNR) of the front and rear frames:

但し、PSNRは、画素差分量を基準映像の画素値で均一化した値であり、MAXは△Rがとりうる最大画素値であり、１輝度値を何bitで構成しているかによって変化する。複数フレームのＳＮ比はフレーム毎のＳＮ比の平均として求まるが、例えば、式（５）のようにフレーム毎に導出するのではなく、式(６)に示すように複数フレームPSNRを計算する際に複数フレームの差分和を先に計算してから導出することもできる。

However, PSNR is a value obtained by equalizing the pixel difference amount with the pixel value of the reference image, and MAX is the maximum pixel value that can be taken by ΔR, and changes depending on how many bits constitute one luminance value. The SN ratio of a plurality of frames is obtained as an average of the SN ratios for each frame. For example, instead of deriving for each frame as shown in Equation (5), when calculating the multiple frames PSNR as shown in Equation (6) It is also possible to derive after calculating the difference sum of a plurality of frames first.

但し、Ｍは測定対象とするフレーム数、MAXは△Rがとりうる最大画素値であり、１輝度値を何bitで構成しているかによって変化する。

However, M is the number of frames to be measured, MAX is the maximum pixel value that can be taken by ΔR, and varies depending on how many bits constitute one luminance value.

  Further, as the feature amount, a statistic such as the variance value, the maximum value, or the minimum value can be used.

  The quality estimation unit 200 receives the relative degradation feature quantity 30 from the degradation feature quantity comparison unit 130, and sends the known degradation video signal encoding conditions and service conditions to the steady quality database 300, and includes a steady quality value and coefficients. The quality data 40 is received and the following calculation is performed.

但し、Ｑは映像品質評価値３０、α_SはＳ番目の相対劣化特徴量でＳの最大値はＰとする。また、βは定常品質値、ａ，ｂ，ｃは係数とする。

However, Q is the video quality evaluation value 30, α _S is the S-th relative deterioration feature amount, and the maximum value of S is P. Β is a steady quality value, and a, b, and c are coefficients.

  Further, the steady quality database 300 may return only the steady quality value, but may pass the evaluation formula itself. Furthermore, as long as the encoding scheme remains unchanged, the steady quality data 40 derivation request may not be made at any time.

[Second Embodiment]
Hereinafter, a second embodiment will be described with reference to the drawings.

  In the following, only the parts different from the first embodiment will be described.

  FIG. 6 shows the configuration of the video quality objective evaluation apparatus in the second embodiment of the present invention. In the figure, the same components as those in FIG.

  The video quality objective evaluation apparatus according to the present embodiment includes a relative deterioration feature amount extraction unit 100B, a quality estimation unit 200, and a steady quality extraction unit 400.

  The relative deterioration feature amount extraction unit 100B includes an encoding processing unit 105, a reference image feature amount extraction unit 110, a deterioration image feature amount extraction unit 120, and a deterioration feature amount comparison unit 130.

  When the reference video signal 10 is input, the encoding processing unit 105 outputs an encoded processing reference video signal 12 that is encoded to the reference video feature amount extraction unit 110. This is to improve that the influence of deterioration due to encoding may not be appropriately taken with the difference feature quantity of the reference image itself as a comparison target of the deteriorated image. Although the encoding processing unit 105 performs the encoding process, an appropriate filtering process corresponding to the encoding may be used.

  When the encoding process reference video signal 12 is input, the reference video feature quantity extraction unit 110 outputs the reference interframe difference feature quantity 15 by performing the same processing as in the first embodiment.

  The steady quality extraction unit 400 derives the steady quality data 50 by directly comparing the pixel information of the reference video signal 10 and the degraded video signal 20 in order to derive a reference for the relative deterioration feature amount. This method is difficult to directly estimate quality due to the influence of video resolution, but uses average quality information. The feature quantity used in the steady quality extraction unit 400 is the reference video signal 10 and the degraded video signal instead of the reference frame difference feature quantity 15 and the degraded inter-frame difference feature quantity 25 which are the original inputs of the feature quantity comparison unit 130. 20 and the steady quality extraction unit 400 directly derives the pixel difference from these. In addition, when the video resolution and the display frame rate are different, the steady quality extraction unit 400 performs a process for matching the resolution and the rate.

  Note that the components of the video quality objective evaluation device in the first and second embodiments described above are constructed as a program and installed in a computer used as the video quality objective evaluation device, or executed via a network. Can be distributed.

  Further, the constructed program can be stored in a portable storage medium such as a hard disk, a flexible disk, or a CD-ROM, and can be installed or distributed in a computer.

  The present invention is not limited to the above-described embodiment, and various modifications and applications can be made within the scope of the claims.

  The present invention can be applied to estimation of video quality of transmitted video at the head of the video distribution base, quality confirmation before transmission in the VOD service, and estimation and confirmation after reception on the terminal side. It is.

10 Reference video signal 20 Degraded video signal 12 Encoding reference video signal 15 Reference interframe difference feature 25 Degraded interframe difference feature 30 Relative deterioration feature 40 Steady quality data requirement 50 Steady quality data 60 Video quality evaluation value 100A, 100B Relative deterioration feature amount extraction unit 105 Encoding processing unit 110 Reference video feature amount extraction unit 120 Degraded video feature amount extraction unit 130 Deterioration feature amount comparison unit 200 Quality estimation unit 300 Steady quality DB
400 Stationary quality extraction unit

Claims (11)

A video quality objective evaluation device for objectively evaluating video quality,
The relative degradation feature amount extracting relative deterioration characteristic amount obtained by normalizing the respective difference feature quantity of difference feature quantity as a pixel average between frames of the deteriorated video difference feature quantity between frames of criteria video Extraction means;
Video quality estimation means for deriving a video quality evaluation value based on the relative deterioration feature quantity extracted from the relative deterioration feature quantity extraction means;
A video quality objective evaluation apparatus characterized by comprising: The relative deterioration feature amount extraction means includes:
A first deterioration feature amount comparison unit that synchronizes the reference image and the deteriorated video using a timing at which both changes in the difference feature amount between the frames of the reference image and the deteriorated image both increase;
Or
A second deterioration feature amount comparison unit that compares a change amount for each corresponding frame when the resolution of the image of the reference image and the deterioration image is different;
Or
Third deterioration feature amount comparison means for comparing a change amount per unit time when the display frame rate of the reference image and the deteriorated image are different;
Or
As the difference feature amount between the frames of the reference image and the deteriorated image, the square of the difference between the pixel average of the difference feature amount between the frames of the reference image and the pixel average of the difference feature amount between the frames of the deteriorated image A ratio of pixel values of feature amounts, a value obtained by equalizing a pixel difference amount with a pixel value of a reference image, or a value obtained by statistically processing them, a fourth degradation feature amount comparison unit,
The video quality objective evaluation apparatus according to claim 1, comprising: The relative deterioration feature amount extraction means includes:
Reference video feature amount extraction means for deriving a difference feature amount between frames of the reference video;
Degrading image feature amount extraction means for deriving a difference feature amount between frames of the deteriorated image,
The reference image feature amount extraction unit and the deteriorated image feature amount extraction unit include:
The video quality objective evaluation apparatus according to claim 1, wherein when deriving a difference feature amount between frames of the reference video and the deteriorated video, pixel thinning or information on a specific area is used. The relative deterioration feature amount extraction means includes:
The video quality objective evaluation apparatus according to claim 1, further comprising: an encoding processing unit that outputs a video obtained by performing an encoder process on the input reference video as a new reference video. The video quality estimation means includes
The video according to claim 1, wherein information obtained by reading from a storage unit storing parameters of an evaluation target video extracted in advance or by directly comparing pixel information between the reference video and the deteriorated video is used. Quality objective evaluation device. A video quality objective evaluation method for objectively evaluating video quality,
The relative degradation feature amount extracting relative deterioration characteristic amount obtained by normalizing the respective difference feature quantity of difference feature quantity as a pixel average between frames of the deteriorated video difference feature quantity between frames of criteria video An extraction step;
A video quality estimation step for deriving a video quality evaluation value based on the relative degradation feature quantity extracted from the relative degradation feature quantity extraction unit;
A video quality objective evaluation method characterized by In the relative deterioration feature amount extraction step,
A first deterioration feature amount comparison step for synchronizing the reference image and the deteriorated image using a timing at which both changes in the difference feature amount between frames of the reference image and the deteriorated image both increase;
Or
A second deterioration feature amount comparison step of comparing the amount of change for each corresponding frame when the resolution of the image of the reference image and the deterioration image is different;
Or
A third deterioration feature amount comparison step for comparing a change amount per unit time when the display frame rates of the reference image and the deterioration image are different;
Or
As the difference feature amount between the frames of the reference image and the deteriorated image, the square of the difference between the pixel average of the difference feature amount between the frames of the reference image and the pixel average of the difference feature amount between the frames of the deteriorated image A fourth deterioration feature amount comparison step using a ratio of pixel values of feature amounts, a value obtained by equalizing pixel difference amounts with pixel values of a reference image, or a value obtained by statistically processing them;
7. The video quality objective evaluation method according to claim 6, wherein any one of the above is performed. In the relative deterioration feature amount extraction step,
A reference image feature extraction step for deriving a difference feature between frames of the reference image;
Performing a degraded video feature amount extraction step for deriving a difference feature amount between frames of the degraded video,
In the reference image feature extraction step and the deteriorated image feature extraction step,
The video quality objective evaluation method according to claim 6, wherein pixel decimation or information on a specific area is used when deriving a difference feature amount between frames of the reference video and the deteriorated video. In the relative deterioration feature amount extraction step,
7. The video quality objective evaluation method according to claim 6, wherein a step of outputting a video obtained by performing an encoder process on the inputted reference video as a new reference video is performed. In the video quality estimation step,
7. The video according to claim 6, wherein information obtained by reading from a storage means storing parameters of a video to be evaluated that has been extracted in advance or by directly comparing pixel information between the reference video and the degraded video is used. Quality objective evaluation method.   A video quality objective evaluation program for causing a computer to function as each means constituting the video quality objective evaluation apparatus according to any one of claims 1 to 5.

Images