JP2014229931A - Video quality evaluation apparatus, video quality evaluation method and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To weight the quality of each hierarchy by using the quality of a video frame unit, and to derive the user perceived quality by using the quality of each hierarchy thus weighted.SOLUTION: A video quality evaluation apparatus for evaluating the quality of video has a hierarchy quality derivation unit for deriving a hierarchy quality, i.e., the quality of each hierarchy, on the basis of the quality of a video frame unit derived from an input signal, and the hierarchy of encoding derived from an input signal, and a video quality evaluation unit for deriving the evaluation value of the quality of video, on the basis of the hierarchy quality derived from the hierarchy quality derivation unit.

Description

  The present invention relates to a video quality evaluation apparatus, a video quality evaluation method, and a program. In particular, the present invention relates to a video quality evaluation technique for objectively evaluating video quality in an IPTV (IP Television) service, a video distribution service, or the like performed via an IP (Internet Protocol) network such as the Internet.

  With the increase in the speed and bandwidth of broadcast waves and Internet access lines, video communication services for transferring video media between terminals or between a server and terminals via broadcast waves and the Internet have become widespread.

  In broadcasting, CATV (Cable Television), and the Internet, the original video is encoded and transmitted, so degradation due to encoding occurs. For this reason, the quality (user experience quality: QoE (Quality of Experience)) perceived by the user with respect to video media and the like deteriorates.

  Specifically, when the original video is encoded, the video signal in the frame is deteriorated by processing in units of blocks, or high-frequency components of the video signal are lost, so that the fineness of the entire video is lowered. As a result, the user perceives blur, blur, and mosaic distortion in the received video.

  In order to confirm that the video communication service having the above characteristics is provided to the user with good quality, the quality of the video experienced by the user is measured during the provision of the video communication service, and On the other hand, it is necessary to monitor the quality of the video provided. Therefore, there is a need for an objective video quality evaluation technique that can appropriately evaluate the quality of video experienced by the user.

  Conventionally, as a method for evaluating video quality, there are a subjective quality evaluation method (for example, see Non-Patent Document 1) and an objective quality evaluation method (for example, Non-Patent Document 2-7).

  The subjective quality evaluation method is a quality scale (very good, good, normal, bad, very good) of the quality experienced by multiple users who actually watched the video and experienced the quality (may be 9 or 11 levels) Bad) and disturbance scale (no degradation is observed at all, degradation is observed but not bothered, slightly concerned about degradation, concerned about degradation, very concerned about degradation), etc. Quality evaluation values of each video averaged by the number of users (for example, video with a packet loss rate of 0% and a bit rate of 20 Mbps) are defined as MOS (Mean Opinion Score) values, DMOS (Degradation Mean Opinion Score) values, etc. .

  However, subjective quality evaluation not only requires special equipment (monitor, etc.) and an evaluation facility that can adjust the evaluation environment (room illuminance, room noise, etc.), but many users actually evaluate the video. There is a need. For this reason, it takes time for the user to actually complete the evaluation, which is not suitable for evaluating the quality in real time.

  Therefore, feature quantities that affect video quality (for example, (1) feature quantities derived from packets such as bit rates and bit quantities in units of frames, and (2) encoded data (bits such as quantization parameters and motion vectors). Development of an objective quality evaluation method that outputs video quality evaluation values using (3) feature amounts derived from stream information) and (3) feature amounts derived from pixel signals such as noise amount and edge amount) is desired. ing.

  A technique for deriving a video quality evaluation value from a feature amount extracted from a pixel signal of a video in a conventional objective quality evaluation method (see, for example, Non-Patent Documents 2, 3, and 4) Is a technique for deriving a video quality evaluation value from bitstream information (for example, see Non-Patent Documents 5 and 6).

  As described above, many conventional objective quality evaluation methods estimate a video quality evaluation value using a video pixel signal, a packet header, and a bit stream.

ITU-T recommendation P.910 (04/2008) ITU-T recommendation J.341 (01/2011) ITU-T recommendation J.247 (08/2008) ITU-T Recommendation J.144 (03/2004) ITU-T Recommendation P.1201.1 (10/2012) ITU-T recommendation P.1201.2 (10/2012) ITU-T recommendation P.1202.1 (10/2012) ITU-T recommendation H.264 (01/2012) ITU-T recommendation H.265 (04/2013) ITU-T Recommendation J.340 (06/2010) Z. Wang, et al., "Image Quality Assessment: From Error Visibility to Structural Similarity," IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 13, NO. 4, APRIL 2004 Z. Wang, et al., "MULTI-SCALE STRUCTURAL SIMILARITY FOR IMAGE QUALITY ASSESSMENT," IEEE Asilomar Conference on Signals, Systems and Computers, Pacific Grove, CA, Nov. 9-12, 2003.

  Although the conventional techniques of Non-Patent Documents 2-7 differ in input information, basically, feature quantities are derived in units of video frames (pictures), and quality and parameter values are calculated in units of video frames based on them. A value obtained by performing statistical processing such as averaging processing is derived as a video quality evaluation value for a certain time (for example, 10 seconds).

  However, Non-Patent Document 8 and Non-Patent Document 9 introduce the concept of hierarchical frame coding as shown in FIG. 1 to increase the compression rate. Here, FIG. 1 shows the order of POC (Picture Order Counter) indicating the playback order of video frames (the decoding order is also shown). J in FIG. 1 indicates a hierarchy. However, although j = 1 and j = 2 are located at the same height, the quality of the I frame is usually higher than the quality of other frames, so it is classified into two layers, j = 1 and j = 2. Yes.

  The basic concept of hierarchical frame coding is as follows. In general, humans cannot detect short-term quality degradation. Therefore, as shown in FIG. 1, every time the hierarchy is deepened (as j is increased), the quality of the video frame unit is lowered (specifically, the quantization step width is increased), and the compression efficiency is increased. Yes.

  Therefore, with hierarchical frame coding, the deeper the hierarchical structure, the higher the quality difference between layers, and the user experience quality is derived simply by averaging the values for each video frame. In this case, the value estimated from the objective quality evaluation technique is low. In addition, since the depth of the hierarchical structure is not uniquely determined and depends on the service form, when the averaging process is simply performed, the output from the objective quality evaluation technology is the same when comparing the one with few layers with the one with many layers. Even so, the user experience quality may be significantly different.

  As described above, in the video frame in which the concept of the hierarchical frame coding is introduced and the compression rate is increased, the objective quality evaluation technique simply averaging the values for each video frame is insufficient.

  The present invention has been made in view of the above points. In order to solve the above-described problem, the quality for each layer (hierarchy quality) is weighted using the quality of each video frame, and the weighted quality for each layer is provided. An object of the present invention is to provide a video quality evaluation apparatus, a video quality evaluation method, and a program for deriving the quality (video quality evaluation value) that the user feels using.

An image quality evaluation apparatus according to an aspect of the present invention is provided.
An image quality evaluation device for evaluating image quality,
A layer quality deriving unit for deriving a layer quality that is a quality for each layer based on the quality of the video frame unit derived from the input signal and the encoding layer derived from the input signal;
A video quality evaluation unit for deriving an evaluation value of video quality based on the hierarchical quality derived from the hierarchical quality deriving unit;
It is characterized by having.

In addition, a video quality evaluation method according to an aspect of the present invention includes:
An image quality evaluation method in an image quality evaluation apparatus for evaluating image quality,
The hierarchical quality deriving unit of the video quality evaluation apparatus derives a hierarchical quality that is a quality for each hierarchy based on the quality of each video frame derived from the input signal and the encoding hierarchy derived from the input signal. And steps to
A step of deriving an evaluation value of the quality of the video based on the hierarchical quality derived from the hierarchical quality deriving unit;
It is characterized by having.

A program according to an aspect of the present invention is
A program for causing a computer to function as a video quality evaluation apparatus for evaluating video quality,
Hierarchy quality deriving means for deriving layer quality that is quality for each layer based on the quality of each video frame derived from the input signal and the encoding layer derived from the input signal, and the layer quality deriving unit Video quality evaluation means for deriving an evaluation value of video quality based on the hierarchical quality derived from
It is made to function as.

  According to the present invention, it is possible to perform objective quality evaluation in consideration of a quality difference for each layer, and it is possible to improve the accuracy of video quality evaluation.

Conceptual diagram of hierarchical frame coding The block diagram of the video quality evaluation apparatus which concerns on the Example of this invention. The flowchart of the video quality evaluation method in the video quality evaluation apparatus based on 1st Example of this invention. The figure which shows the relationship between POC and PSNR in the hierarchical frame coding of FIG. Flowchart of video quality evaluation method in video quality evaluation apparatus according to second embodiment of the present invention. Flowchart of video quality evaluation method in video quality evaluation apparatus according to the third embodiment of the present invention.

  Embodiments of the present invention will be described below with reference to the drawings.

  In an embodiment of the present invention, a video quality evaluation apparatus for performing objective quality evaluation in consideration of quality differences between hierarchies to which video frames belong will be described.

  FIG. 2 is a block diagram of the video quality evaluation apparatus 100 according to the embodiment of the present invention. The video quality evaluation apparatus 100 includes a video frame quality deriving unit 101, a hierarchy deriving unit 102, a hierarchy quality deriving unit 103, and a video quality evaluating unit 104.

  The video frame quality deriving unit 101 derives video frame quality (video frame quality) from the input signal. For example, the video frame quality estimation unit 101 derives a video frame quality Fi (i: video frame number) in units of video frames from any one of a pixel signal, a packet header, and a bit stream, or a combination thereof. For example, the video frame quality deriving unit 101 derives the video frame quality based on the feature amount extracted from the input signal. In addition, when the correlation between the feature quantity extracted from the input signal and the video frame quality is high, the feature quantity extracted from the input signal can be used as the video frame quality.

  The hierarchy deriving unit 102 derives an encoding hierarchy j (j: hierarchy number) from the input signal. For example, the hierarchy deriving unit 102 derives a hierarchy for each video frame by considering a feature amount extracted from the input signal. As will be described in detail in the embodiments below, the hierarchy derivation method varies depending on the type of the input signal, but any derivation method may be used including the hierarchy derivation method according to the prior art.

  The hierarchical quality deriving unit 103 uses the video frame quality Fi derived from the video frame quality deriving unit 101 and the hierarchy j derived from the hierarchical deriving unit 102 to derive a quality (hierarchy quality) Hj for each layer. For example, the hierarchical quality Hj is derived as follows.

ただし、Ｉは測定区間の映像フレーム数、Njは階層jの映像フレーム数である。

Here, I is the number of video frames in the measurement section, and Nj is the number of video frames in layer j.

  The video quality evaluation unit 104 derives the video quality evaluation value V from the hierarchical quality Hj derived from the hierarchical quality deriving unit 103. For example, the video quality evaluation value V is derived by weighting the hierarchical quality Hj as follows.

ただし、ajは階層毎の重み係数、Jは階層数である。

Here, aj is a weighting factor for each layer, and J is the number of layers.

  The video quality evaluation value V derived in this way may be used as video quality (user experience quality) experienced by the user.

  Note that either or either one of the video frame quality deriving unit 101 and the hierarchy deriving unit 102 may be provided in an apparatus outside the video quality evaluation apparatus 100. That is, the video quality evaluation apparatus 100 includes a hierarchical quality deriving unit 103 and a video quality evaluation unit 104, and derives a video quality evaluation value using the video frame quality or the hierarchical quality derived by an external device. Also good.

  Next, the operation of the video quality evaluation apparatus 100 according to the embodiment of the present invention will be described based on the embodiment. In the first embodiment, a case where a bit stream is used as an input signal will be described. In a second embodiment, a case where a pixel signal will be used as an input signal will be described. In the third embodiment, a packet header will be used as an input signal. The case where it is used will be described. Further, in the fourth embodiment, application to these combinations will be mentioned.

<First embodiment>
In the first embodiment, a specific video quality evaluation method when a bit stream is used as an input signal will be described with reference to FIG. FIG. 3 is a flowchart of the video quality evaluation method in the video quality evaluation apparatus according to the first embodiment of the present invention.

  When a bit stream is used as an input signal, the video frame playback order (POC: Picture Order Counter), slice type (ST: Slice Type), coding unit (CU: Coding Unit), prediction unit (PU: Prediction Unit) Information such as a transform unit (TU), a quantization parameter (QP), and a motion vector (MV) can be extracted as a feature quantity.

  The video frame quality deriving unit 101 can estimate the video frame quality from these pieces of information. For example, the QP corresponds to the quantization step width, and the video frame quality Fi is estimated using the fact that the QP value has a high correlation with the video frame quality (step S101). For example, the video frame quality Fi is derived by a model in which the relationship between the value obtained by averaging the QP values for each block in the video frame (average QP value) and the video frame quality is formulated. This model is obtained by formulating a subjective quality evaluation value and an average QP value derived by subjective quality evaluation. For example, the model may be formulated by a polynomial or an exponential function. In this embodiment, an example in which the video frame quality is derived from the QP is shown. However, the scope of the present invention is not limited to the quality estimation in units of video frames by the method shown in the embodiment.

  The hierarchy deriving unit 102 derives a hierarchy j for each video frame by considering the POC, ST, block reference structure, and the like (step S102). Specifically, as shown in FIG. 1, a hierarchy can be derived by using POC, decoding order, and video frame type. Decoding order 0th I frame (POC = 0) is the lowest hierarchy (j = 1), and next decoding order (1st) B frame (POC = 8) is the next hierarchy (j = 2) ) At this time, the video frames in the subsequent decoding order have POC = 1-7 numbers.

  The B frame (POC = 4), which is the next decoding order (second), has a POC smaller than that of the previous B frame, and is therefore the next layer (j = 3). The B frame (POC = 2), which is the next decoding order (third), has a POC smaller than that of the previous B frame, and is therefore the next layer (j = 4). The B frame (POC = 1), which is the next decoding order (fourth), has a POC smaller than that of the previous B frame, and is therefore the next layer (j = 5). The next decoding order (fifth) B frame (POC = 5) has the same POC (j = 5) as the previous video frame because the POC is larger than the previous B frame. In the same way, the hierarchy is derived.

  The hierarchy quality deriving unit 103 averages the video frame quality for each hierarchy from the relationship between the video frame quality Fi and the hierarchy j for each video frame, and derives the hierarchy quality Hj (step S103). Regarding the derivation of the layer quality Hj, not only the average value as described above but also the average value and variance of the video frame quality Fi are derived, and the layer quality Hj is derived from the average value and variance. Good. However, the scope of the present invention is not limited to the quality estimation of the hierarchical quality by the method shown in the embodiment.

  The video quality evaluation unit 104 weights the hierarchical quality Hj, derives a video quality evaluation value V, and ends the process. (Step S104). In this embodiment, an example is shown in which the hierarchical quality is weighted and the video quality evaluation value is derived, but the scope of the present invention is not limited to this derivation method.

<Second embodiment>
In the second embodiment, a specific video quality evaluation method when a pixel signal is used as an input signal will be described with reference to FIGS. FIG. 4 is a diagram showing the relationship between POC and PSNR in the hierarchical frame coding of FIG. FIG. 5 is a flowchart of the video quality evaluation method in the video quality evaluation apparatus according to the second embodiment of the present invention.

  When pixel signals are used, various feature amounts can be extracted for each video frame.

  The video frame quality deriving unit 101 can estimate the video frame quality from these feature quantities. For example, peak signal-to-noise ratio (PSNR) (see Non-Patent Document 10), structural similarity (SSIM: Structural SIMilarity) (see Non-Patent Document 11), multi-scale SSIM (MS-) Utilizing the fact that SSIM (Multi Scale SSIM) (see Non-Patent Document 12) and the like have high correlation with video frame quality, video frame quality Fi is derived using these feature quantities as video frame quality (step S201). Note that the video frame quality Fi may be a feature quantity itself such as PSNR, or may be derived from a model that formulates the relationship between the feature quantity such as PSNR and the video frame quality. In this embodiment, an example in which video frame quality is derived from PSNR, SSIM, MS-SSIM, etc. has been shown. However, the scope of the present invention is limited to quality estimation in units of video frames by the method shown in the embodiment. It is not something.

  The hierarchy deriving unit 102 derives a hierarchy j for each video frame by using the increase / decrease in the feature amount (eg, PSNR, SSIM, MS-SSIM) of each video frame. As can be seen from FIG. 4, for example, the PSNR has a higher value as the hierarchy is lower. For example, PSNR of a video frame with POC = 1 in FIG. 1 is 47.61, PSNR of a video frame with POC = 8 is 46.29, PSNR of a video frame with POC = 4 is 44.35, PSNR of a video frame with POC = 2 is 43.36, POC PSNR of the video frame of = 1 = 40.59, PSNR of the video frame of POC = 3 = 40.57, PSNR of the video frame of POC = 6 = 43.38, etc., and the quality differs from layer to layer. Here, PSNR is used as the quality value, but the same applies to other parameters and quality values. Using the relationship between the feature amount of each video frame and the hierarchy, the hierarchy introduction unit 102 determines the hierarchy (step S202). However, the lower the feature amount, the higher the quality may mean. In that case, the reverse process may be performed. Regarding the derivation of the hierarchy j for each video frame, the feature amounts may be classified by clustering analysis. However, the scope of the present invention is not limited to the hierarchy derivation method by the method shown in the embodiment.

  The hierarchical quality deriving unit 103 averages the video frame quality for each hierarchy from the relationship between the video frame quality Fi and the hierarchy j for each video frame, and derives the hierarchical quality Hj (step S203). In the present embodiment, the example in which the layer quality is derived from the video frame quality for each layer has been shown, but the scope of the present invention is not limited to this derivation method.

  The video quality evaluation unit 104 weights the hierarchical quality Hj, derives a video quality evaluation value V, and ends the process. (Step S204). In the present embodiment, an example is shown in which the hierarchical quality is weighted and the video quality evaluation value is derived, but the scope of the present invention is not limited to this derivation method.

<Third embodiment>
In the third embodiment, a specific video quality evaluation method when a packet header is used as an input signal will be described with reference to FIG. FIG. 6 is a flowchart of the video quality evaluation method in the video quality evaluation apparatus according to the third embodiment of the present invention.

  When the packet header is used, the bit amount, packet number, payload, and byte number for each video frame can be extracted as the feature amount.

  The video frame quality deriving unit 101 uses the bit amount of each video frame as a feature amount and uses the correlation that the quality is high if the bit amount is large, and estimates the video frame quality Fi from the bit amount (step S301). Note that the video frame quality Fi may be a feature amount itself such as a bit amount or may be derived by a model that formulates a relationship between a feature amount such as a bit amount and a video frame quality. In this embodiment, an example in which the video frame quality is derived from the bit amount which is a feature amount for each video frame is shown. However, the scope of the present invention is limited to the quality estimation in units of video frames by the method shown in the embodiment. It is not something.

  The hierarchy deriving unit 102 derives a hierarchy j for each video frame by using the increase / decrease of the video frame quality based on the feature amount such as the bit amount, packet number, payload, and byte number of each video frame. For example, as with the PSNR shown in FIG. 4, the bit amount has a higher value as the hierarchy is lower, and the hierarchy for each video frame is determined (step S302).

  The hierarchical quality deriving unit 103 averages the video frame quality for each hierarchy from the relationship between the video frame quality Fi and the hierarchy j for each video frame, and derives the hierarchical quality Hj (step S303). In the present embodiment, the example in which the layer quality is derived from the video frame quality for each layer has been shown, but the scope of the present invention is not limited to this derivation method.

  The video quality evaluation unit 104 weights the hierarchical quality Hj, derives a video quality evaluation value V, and ends the process. (Step S304). In the present embodiment, an example is shown in which the hierarchical quality is weighted and the video quality evaluation value is derived, but the scope of the present invention is not limited to this derivation method.

<Fourth embodiment>
In the fourth embodiment, a case where any set of a bit stream, pixel information, and packet header is used as an input signal will be described.

  When any set of bitstream, pixel information, and packet header is input to the video quality evaluation apparatus 100, by using the method shown in the first to third embodiments according to the type of the input signal, Video quality evaluation value can be derived.

<Effect of the embodiment of the present invention>
As described above, according to the embodiment of the present invention, the video frame quality is derived for each video frame, the quality value is derived for each layer, and after the layer quality is derived, weighting processing and the like are performed, Since the video quality evaluation value of the frame is derived, the video quality evaluation value can be appropriately evaluated.

  Conventional objective quality evaluation technology derives quality and parameter values for each video frame, averages these values, etc., and derives video quality evaluation values. In the video quality evaluation, the estimation accuracy is reduced and the video quality evaluation value cannot be derived properly. Specifically, when hierarchical frame coding is performed, for example, as shown in FIG. 4, the quality differs for each hierarchy. Further, the quality value for each layer shows a close value if there is no significant change in the scene. As described above, such quality degradation is performed in consideration of human perceptual characteristics, and it is possible to evaluate quality appropriately by deriving the quality of multiple video frames by performing simple averaging processing etc. Can not. In addition, since the depth of the hierarchical structure depends on the service, an appropriate quality value cannot be derived if the averaging process is performed without considering the depth of the hierarchy.

  On the other hand, according to the embodiment of the present invention, based on the value obtained by weighting the quality value (hierarchy quality) for each layer using the video frame quality of each video frame derived from a plurality of video frames. Since the video quality evaluation value in consideration of the depth of the hierarchy is derived, the video quality evaluation value can be appropriately evaluated.

  Therefore, it is possible to easily determine whether the service being provided maintains a certain level of quality for the user, and it becomes possible to grasp and manage the actual quality of the service being provided in real time. .

  As described above, it is possible to improve the quality of the services that are being provided, and to improve the quality of video quality with higher accuracy compared to the quality assessment based on the conventional technology. It becomes.

  The video quality evaluation apparatus 100 in the above embodiment is realized by installing a computer program in a computer including a CPU (Central Processing Unit), a memory, and an interface. The various functions of the video quality evaluation apparatus 100 described above are as follows. Various hardware resources of the computer and the computer program (software) may be realized in cooperation.

  The operations of the components of the video quality evaluation apparatus 100 shown in FIG. 2 can be constructed as a program and installed in a computer used as the video quality evaluation apparatus 100 to be executed or distributed via a network. is there.

  Furthermore, the constructed program can be stored in a portable storage medium such as a hard disk, a flexible disk, or a CD-ROM, and installed in a computer or distributed.

  The present invention is not limited to the above-described embodiments, and various modifications and applications can be made within the scope of the claims.

  The present invention can be used for an apparatus for estimating a video quality evaluation value of video communication such as an IPTV service and a video distribution service performed via an IP network.

DESCRIPTION OF SYMBOLS 100 Video quality evaluation apparatus 101 Video frame quality deriving part 102 Hierarchy deriving part 103 Hierarchy quality deriving part 104 Video quality evaluating part

Claims (8)

An image quality evaluation device for evaluating image quality,
A layer quality deriving unit for deriving a layer quality that is a quality for each layer based on the quality of the video frame unit derived from the input signal and the encoding layer derived from the input signal;
A video quality evaluation unit for deriving an evaluation value of video quality based on the hierarchical quality derived from the hierarchical quality deriving unit;
A video quality evaluation apparatus.   The video quality evaluation apparatus according to claim 1, wherein the hierarchical quality deriving unit averages the quality of each video frame for each hierarchy and derives the hierarchical quality.   The video quality evaluation apparatus according to claim 1, wherein the video quality evaluation unit weights hierarchical quality using a weighting factor for each hierarchy and derives an evaluation value of video quality.   The video quality evaluation apparatus according to claim 1, further comprising a hierarchy deriving unit that derives a coding hierarchy from the input signal.   5. The video quality evaluation apparatus according to claim 1, further comprising a video frame quality deriving unit that derives quality in units of video frames from a feature amount extracted from an input signal. 6.   6. The video quality evaluation apparatus according to claim 5, wherein the video frame quality deriving unit derives the quality of the video frame unit by using the correlation between the feature amount extracted from the input signal and the quality of the video frame unit. An image quality evaluation method in an image quality evaluation apparatus for evaluating image quality,
The hierarchical quality deriving unit of the video quality evaluation apparatus derives a hierarchical quality that is a quality for each hierarchy based on the quality of each video frame derived from the input signal and the encoding hierarchy derived from the input signal. And steps to
A step of deriving an evaluation value of the quality of the video based on the hierarchical quality derived from the hierarchical quality deriving unit, the video quality evaluating unit of the video quality evaluation device;
A video quality evaluation method. A program for causing a computer to function as a video quality evaluation apparatus for evaluating video quality,
Hierarchy quality deriving means for deriving layer quality that is quality for each layer based on the quality of each video frame derived from the input signal and the encoding layer derived from the input signal, and the layer quality deriving unit Video quality evaluation means for deriving an evaluation value of video quality based on the hierarchical quality derived from
Program to function as.

Images