JP5851332B2 - 3D video quality evaluation apparatus, method and program - Google Patents

Description

  The present invention relates to a 3D video quality evaluation apparatus, method, and program, and more particularly to objectively evaluate the quality of 3D video in an IPTV service and video distribution service performed via an IP (Internet Protocol) network such as the Internet. The present invention relates to a 3D video quality evaluation apparatus, method, and program to be used.

  As broadcast waves and Internet access lines increase in speed and bandwidth, 3D video communication services that transfer video media between terminals or between a server and terminals via broadcast waves and the Internet are expected to become widespread.

  In broadcasting, CATV, and the Internet, the original video is encoded and the video is transmitted, so degradation due to encoding occurs. Since communication quality is not always guaranteed for a network, when communication is performed using video media, etc., it is a bit due to weak antenna reception sensitivity, weak signals, or narrow network bandwidth between user terminals. Packet loss and packet transfer delay occur due to rate reduction and line congestion, and the quality perceived by the user (Quality of Experience (QoE)) for the received video media is degraded. .

  Specifically, when the original video is encoded, the video signal in the frame is deteriorated by processing in units of blocks, or high-frequency components of the video signal are lost, so that the fineness of the entire video is lowered. Further, when the encoded video data is transmitted as a packet to the user terminal via the network, if a packet loss or discard occurs, the frame is deteriorated and the quality of the entire video is lowered.

  As a result, the user perceives blur, blur, and mosaic distortion in the received video.

  In order to confirm that the video communication service as described above is provided with good quality, the quality of the video provided to the user is measured by measuring the quality of the video experienced by the user during the service provision. It is important to monitor that is high.

  Therefore, there is a need for 3D video quality objective evaluation technology that can appropriately express the quality of the video experienced by the user.

  Conventionally, as a method for evaluating video quality, there are a subjective quality evaluation method (for example, see Non-Patent Document 1) and an objective quality evaluation method (for example, Non-Patent Document 2).

  The subjective quality evaluation method is a quality scale (very good, good, normal, bad, very good) of the quality experienced by multiple users who actually watched the video and experienced the quality (may be 9 or 11 levels) Bad) and disturbance scale (no degradation is observed, degradation is observed but not bothered, slightly concerned about degradation, concerned about degradation, very concerned about degradation), etc. Average the quality evaluation value of each video (for example, video with a packet loss rate of 0% and a bit rate of 20 Mbps) by the number of users, and define the value as MOS (Mean Opinion Score) value or DMOS (Degradation Mean Opinion Score) value doing.

  However, subjective quality evaluation not only requires special dedicated equipment (such as a monitor) and an evaluation facility that can adjust the evaluation environment (such as room illuminance and room noise), but many users actually evaluate the video. There is a need. For this reason, it takes time for the user to actually complete the evaluation, which is not suitable for evaluating the quality in real time.

  Therefore, feature quantities that affect video quality (for example, (1) feature quantities derived from packet information such as bit rate, bit quantity per frame, packet loss information, (2) quantization parameters, motion vectors, etc. Objective quality evaluation that outputs video quality evaluation values using feature values derived from encoded data (bitstream information) and (3) feature values derived from video pixel signals such as noise and edge amounts) The development of a law is desired.

  On the other hand, as one of conventional objective quality evaluation methods, there is a technique for deriving a video quality evaluation value from a feature amount extracted from a video pixel signal (see, for example, Non-Patent Documents 2, 3, and 4). In particular, many of the conventional objective quality evaluation methods for evaluating 3D video quality estimate the 2D video quality evaluation value using the 2D video pixel signal as described above, and calculate the 3D video quality from the left and right 2D video quality. It was an estimate. However, there is a problem in that the amount of calculation is large because the amount of data is large when the pixel values of the 2D video are read and calculated one by one. Therefore, in 2D video quality evaluation, various techniques for evaluating 2D video quality from packet header information and encoded data (bit stream) information have been studied.

ITU-T recommendation P.910 ITU-T recommendation J.341 ITU-T recommendation J.247 ITU-T recommendation J.144

  When coding 3D video, there is a technique called Inter-view coding (see Fig. 1) that takes into account the similarity between the left and right video, and simply extracts parameters from the 2D video bitstream. The conventional technology for evaluating video quality does not consider the presence or absence of Inter-view encoding, and therefore cannot correctly evaluate 3D video quality.

  First, Inter-view coding increases the coding efficiency for video on one side (usually the right-eye video), so how much improvement is achieved by Inter-view coding Therefore, it is necessary to evaluate the 3D video quality in consideration of whether it is encoded.

  Second, when Inter-view coding is used, it is necessary to appropriately consider the fact that when a video such as packet loss enters the video of the reference frame, the degradation propagates to the video frame. . In other words, at the same time (note: on the display using polarized glasses, the left and right eye images are displayed at the same time, but on the display using the frame sequential, the image is precisely shifted by 1 / frame rate (for example, It is necessary to consider that one or both of the left and right eye images that are played back at 16.6 ms in the case of 60-fps video for both eyes are degraded.

  The present invention has been made in view of the above points. In order to solve the above-described problems, the improvement of coding efficiency by Inter-view coding is considered, and further, degradation of packet loss or the like is caused in a reference video. When entering, in addition to capturing the phenomenon that the degradation propagates to video frames using Inter-view coding, the amount of temporal degradation that block-like degradation and freeze propagate to the following frames is derived, An object is to provide a 3D video quality evaluation apparatus, method, and program for deriving a video quality evaluation value.

In order to solve the above problems, the present invention is a 3D video quality evaluation apparatus that objectively evaluates the quality of 3D video,
Video parameter extraction means for extracting video quality parameters from input video data;
And 3D video quality calculation means for calculating a 3D image quality before Kipa parameters,
Have
The video parameter extraction unit determines a video view of 3D video, determines a video frame type for each view of the 3D video, and if there is degradation in the video frame, the video frame referring to the video frame is degraded. A video frame deterioration amount calculating unit for calculating a video frame deterioration amount as a frame and storing the video frame deterioration amount in a first storage unit;
A video frame information amount calculating means for calculating a statistic of the amount of information for each video frame and storing it in a second storage means;
Including
The 3D video quality calculating means includes:
Encoded video quality calculation means for calculating an encoded video quality evaluation value immediately after 3D video is encoded, using a statistic of the amount of information for each video frame stored in the second storage means .

  Conventionally, in 2D video encoding, the number of video views (in the case of Stereoscopic 3D for the left and right eyes, the number of video views is 2. Usually, the view number of the left eye video is 0 (Base view), and the view number of the right eye video is 1 (Dependent view)), there is no concept of Inter-view coding. However, video coding schemes represented by MVC (Multi-view coding) have introduced an Inter-view coding technique in which coding is performed in consideration of the similarity between left and right videos. Therefore, the perception of encoding distortion after video encoding and packet loss distortion when packet loss occurs differs from 2D video. Against this background, 3D video quality cannot be correctly evaluated even when video quality evaluation technology used for 2D video quality evaluation is applied.

  On the other hand, according to the present invention, since the 3D video quality can be evaluated in consideration of the improvement of the encoding efficiency by the Inter-view encoding and the propagation state of the degradation, the appropriate 3D video quality evaluation can be performed.

  Accordingly, the present invention makes it possible to monitor the video quality value of the video of the video communication service that is actually viewed by the user. Therefore, it is possible to determine whether the service being provided maintains a certain level of quality for the user. Judgment can be made without carrying out the evaluation.

  For this reason, it becomes possible to improve the point that the quality of the service being provided could not be evaluated with the prior art.

It is a conceptual diagram of Inter-view encoding. It is a block diagram of the 3D video quality evaluation apparatus in one embodiment of this invention. It is a block diagram of the video view discrimination | determination part in one embodiment of this invention. It is a block diagram of the 3D video quality calculation part in one embodiment of this invention. It is a conceptual diagram of the reference structure of the video frame in one embodiment of this invention. It is a flowchart of operation | movement of the 3D video quality evaluation apparatus in one embodiment of this invention.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

The 3D video quality evaluation apparatus according to this embodiment of the present invention extracts video parameters that extract parameters related to video quality from input video data. 3D video quality is objectively evaluated from the extracted parameters.

FIG. 2 shows a configuration of a 3D video quality evaluation apparatus according to an embodiment of the present invention.

As shown in the figure, the 3D video quality evaluation apparatus 100 includes a video parameter extraction unit 1 and a 3D video quality calculation unit 2.

  The video parameter extraction unit 1 is one of the video view determination unit 11, the video frame type determination unit 12, the lost video frame determination unit 13, the video frame deterioration amount calculation unit 14, the video frame information amount calculation unit 15, and the memories 16 and 17. Or you may have all. The video view determination unit 11 may include an Inter-view encoding determination unit 111 as shown in FIG. In the following description, it is assumed that the video parameter extraction unit 1 is configured by all these components. The memory 16 stores the determination results of the video view determination unit 11, the video frame type determination unit 12, the lost video frame determination unit 13, and the calculation result of the video frame deterioration amount calculation unit 14. It is assumed that the calculation result of the video frame information amount calculation unit 15 is stored.

  Here, the video view determination unit 11 determines a video view (a left-eye image or a right-eye image) based on the view_id indicated in the bitstream information. In the case of stereoscopic 3D video, one of the left and right eye images is 0 and the other is 1, and the video view is determined from this information. Typically, the left-eye video is 0 and the right-eye video is 1 (hereinafter, video view 0 is the left-eye video and video view 1 is the right-eye video).

  The inter-view encoding determination unit 111 determines whether inter-view encoding is used in the right-eye video from inter_view_flag, and stores the result in the memory 16.

  The video frame type discriminating unit 12 discriminates I, B, and P frames based on primary_pic_type described in AUD (Access unit delimiter) and stores them in the memory 16. The video frame type determination unit 12 outputs the bit stream information to the video frame information amount calculation unit 15.

  The lost video frame determination unit 13 determines whether there is missing data due to packet loss or the like in each video frame, sets a flag for the lost video frame, and stores the flag in the memory 16.

  The video frame degradation amount calculation unit 14 uses the information of the lost video frame for which the flag of the memory 16 is set (see FIG. 4 for the reference frame structure of the video frame in the time direction, Inter-view The video frame degradation amount (DF) is calculated in consideration of the propagation state of degradation of the reference frame structure in encoding (see FIG. 1) and stored in the memory 16. However, at the same time (Note. On a display that uses polarized glasses, the left and right eye images are displayed simultaneously, but on a display that uses frame sequential, the images are strictly shifted by 1 / frame rate (for example, In the case of 60fps images for both eyes, the image frame at the time when the misalignment of both eyes was corrected) is considered to be degraded, and either the left or right or both the left and right are degraded video slices. If there is, the number of deteriorated frames is counted, and the amount of image frame deterioration (that is, the left-eye image frame deteriorates and the deterioration propagates to the right-eye image frame, taking into account the deterioration at the same time, The number of degraded video frames may be calculated as 1). Also, the output of the video frame degradation amount calculation unit 14 is not the number of degraded video frames, but the degraded video frame rate (number of degraded video frames / total number of video frames), degradation time length (for example, 30 fps video is degraded by one frame) In such a case, the deterioration duration may be 33 ms). Furthermore, the amount of deterioration in the time direction (for example, the amount of deterioration when several frames are propagated) and the amount of deterioration in the spatial direction (the amount of deterioration when several frames are propagated within the same video frame) are given to the 3D video quality evaluation value. Since the influences are different, the amount of degradation in the time direction and the amount of degradation in the spatial direction may be weighted to obtain the video frame degradation amount. At this time, in order to estimate the degree of deterioration in the spatial direction, the spatial loss amount may be estimated from the data amount of one lost video frame and the lost data amount.

  The video frame information amount calculation unit 15 is a statistical amount (for example, average value, variance, maximum value, minimum value, etc.) of information amount (bit amount) for each video frame type input via the video frame type determination unit 12. Are extracted as parameters and stored in the memory 17.

  As shown in FIG. 5, the 3D video quality calculation unit 2 may include an encoded video quality calculation unit 21 and a video quality integration unit 22. The 3D video quality calculation unit 2 can include both or either of the encoded video quality calculation unit 21 and the video quality integration unit 22, but in the present embodiment, both are provided. Will be described.

  The encoded video quality calculation unit 21 derives an encoded video quality evaluation value (VQc) from the statistical amount of the information amount for each video frame type acquired from the memory 17 in consideration of the deterioration amount caused only by encoding.

  For example, the left-eye video bit rate (BRL) and the right-eye video bit rate (BRR) are derived from the statistics of the amount of information for each type of video frame (typically when using Inter-view coding, The bit rate of the right eye video is lower than the bit rate of the left eye video), and the encoded video quality evaluation of the left eye and right eye video from the left eye video bit rate (BRL) and the right eye video bit rate (BRR) Values (VQcL and VQcR) are derived as follows:

ただし、VQcL及びVQcRの算出方法はこれに限ったものではなく、映像フレーム種別毎の情報量の統計量から算出されていれば良く以下のような実施もある。

However, the calculation method of VQcL and VQcR is not limited to this, and it may be calculated from the statistics of the information amount for each video frame type, and the following implementation is also possible.

  The video quality of the I frame affects the video quality of the P and B frames, and the video quality of the P frame affects the video quality of the B frame. If the video quality of the I frame is low, the video quality of the P and B frames usually falls because a large amount of bits cannot be allocated due to the bit rate limitation. Furthermore, under the restriction of the bit rate, for example, in the case of a video with a lot of motion, it is encoded so that the bit amount of the I frame is lowered, the bit amount of the P and B frames is raised, and the overall video quality is improved. Yes. On the other hand, in the case of a video with little motion, it is encoded such that the bit amount of the I frame is increased and the bit amount of the P and B frames is decreased to improve the overall video quality. Taking these qualitative characteristics into consideration, the left and right eyes are calculated from the average value (BRLI, BRLP, BRLB, BRRI, BRRP, BRRB) of the bit amount for each video frame type (I, P, B frame) for each of the left and right eyes. The encoded video quality evaluation values (VQcL and VQcR) of the video may be derived as follows.

VQcL = 1 + vl exp (−BRLI / il) exp (−BRLP / pl) exp (−BRLB / bl)
VQcR = 1 + vr exp (−BRRI / ir) exp (−BRRP / pr) exp (−BRRB / br)
As for the mathematical formula, it is only necessary to consider the above-described qualitative characteristics, and the mathematical formula is not limited to the mathematical formula based on the exponential function (for example, it may be formulated with a polynomial).

  Next, in consideration of the quality difference between VQcL and VQcR, the encoded video quality evaluation value (VQc) is derived by the following equation (1).

ここで、x，xl，xr，y，yl，yr，zi，zl，zr，vl，il，pl，bl，vr，ir，pr，brは主観品質評価値との対応関係に基づき、線形もしくは非線形回帰分析等により導出される係数である。
映像品質統合部２２は、符号化映像品質算出部２１で求められた符号化映像品質評価値及びメモリ１６から読み出した映像フレーム劣化量（DF）から、符号化及びパケット損失劣化を考慮した３Ｄ映像品質評価値（VQ）を導出する。具体的には、次式のように、３Ｄ映像品質評価値（VQ）を算出する。ただし、aは主観品質評価値と式(2)との対応関係に基づき、線形もしくは非線形回帰分析等により導出される係数である。

Where x, xl, xr, y, yl, yr, zi, zl, zr, vl, il, pl, bl, vr, ir, pr, br are linear or A coefficient derived by nonlinear regression analysis or the like.
The video quality integration unit 22 uses the encoded video quality evaluation value obtained by the encoded video quality calculation unit 21 and the video frame deterioration amount (DF) read from the memory 16 to 3D video considering encoding and packet loss deterioration. Deriving the quality evaluation value (VQ). Specifically, a 3D video quality evaluation value (VQ) is calculated as in the following equation. However, a is a coefficient derived by linear or nonlinear regression analysis or the like based on the correspondence between the subjective quality evaluation value and the formula (2).

VQ = 1 + (VQc − 1) exp (−DF / a) Equation (2)
Although the 3D video quality evaluation value was formulated with the exponential function of Equation (2), it is sufficient to use a mathematical formula that has a good correspondence with the subjective quality evaluation value of 3D video. For example, two or more exponential functions as shown in Equation (3) A 3D video quality evaluation value may be formulated and a 3D video quality evaluation may be formulated by a polynomial. However, b, c, and d are coefficients derived by linear or nonlinear regression analysis or the like based on the correspondence relationship between the subjective quality evaluation value and Equation (3).

VQ = 1 + (VQc−1) {(1−b) exp (−DF / c) + b exp (−DF / d)} Equation (3)
Hereinafter, a series of operations of the 3D video quality evaluation apparatus 100 will be described.

FIG. 6 is a flowchart of the operation of the 3D video quality evaluation apparatus according to the embodiment of the present invention.

Video data is input to the video parameter extraction unit 1 of the 3D video quality evaluation apparatus 100 (S101).

  The video view determination unit 11 determines a video view based on the view_id of the bitstream information (S102).

  The Inter-view encoding determination unit 111 determines whether or not Inter-view encoding is performed based on inter_view_flag of the bitstream information (S103).

  The video frame type determination unit 12 determines the video frame type for each determined video view (S104).

  The lost video frame discriminating unit 13 discriminates whether degradation has occurred for each video frame, sets a flag for the lost video frame, and stores it in the memory 16 (S105).

  The video frame deterioration amount calculation unit 14 detects a video frame that refers to a video frame in which deterioration has occurred with a flag set in the memory 16, derives the video frame deterioration amount, and stores it in the memory 16 (S 106). .

  The video frame information amount calculation unit 15 calculates the statistical amount of information for each video frame type and stores it in the memory 17 (S107).

  The encoded video quality calculation unit 21 derives an encoded video quality evaluation value from the statistics of the information amount for each video frame type acquired from the memory 17 and outputs it to the video quality integration unit 22 (S108).

  The video quality integration unit 22 derives a 3D video quality evaluation value from the encoded video quality evaluation value acquired from the encoded video frequent appearance calculation unit 21 and the video frame degradation amount acquired from the memory 16 (S109), and ends the processing. To do.

  As described above, according to the present embodiment, the video frame type for each video view (viewpoint) is determined from the video data, and the reference structure of the video frame and the reference structure between the video views by Inter-view coding are considered. It becomes possible to evaluate the 3D video quality.

  Therefore, it is possible to easily determine whether the service being provided maintains a certain level of quality for the user, and it becomes possible to grasp and manage the actual quality of the service being provided in real time. .

Incidentally, 3D video quality assessing apparatus 100 in the above embodiment, CPU (central processing unit) and a memory, is achieved by installing a computer program on a computer consisting interface, various 3D video quality assessing apparatus 100 described above The function is realized by cooperation of various hardware resources of the computer and the computer program (software).

The operation of the constituent elements of the 3D video quality evaluation apparatus 100 shown in FIG. 3 can be constructed as a program, installed in a computer used as the 3D video quality evaluation apparatus, executed, or distributed via a network. It is.

  Furthermore, the constructed program can be stored in a portable storage medium such as a hard disk, a flexible disk, or a CD-ROM, and installed in a computer or distributed.

  The present invention is not limited to the above-described embodiment, and various modifications and applications can be made within the scope of the claims.

  The present invention can be used for a 3D video quality evaluation apparatus that estimates a 3D video quality evaluation value of 3D video communication such as an IPTV service and a video distribution service performed via an IP network.

DESCRIPTION OF SYMBOLS 1 Video parameter extraction part 2 3D video quality calculation part 11 Video view discrimination | determination part 12 Video frame classification discrimination part 13 Lost video frame discrimination | determination part 14 Video frame degradation amount calculation part 15 Video frame information amount calculation part 16 Memory 17 Memory 21 Encoded video Quality calculation unit 22 Video quality integration unit 100 3D video quality evaluation device 111 Inter-view coding discrimination unit

Claims (8)

A 3D video quality evaluation apparatus that objectively evaluates the quality of 3D video,
Video parameter extraction means for extracting video quality parameters from input video data;
And 3D video quality calculation means for calculating a 3D image quality before Kipa parameters,
Have
The video parameter extraction unit determines a video view of 3D video, determines a video frame type for each view of the 3D video, and if there is degradation in the video frame, the video frame referring to the video frame is degraded. A video frame deterioration amount calculating unit for calculating a video frame deterioration amount as a frame and storing the video frame deterioration amount in a first storage unit;
A video frame information amount calculating means for calculating a statistic of the amount of information for each video frame and storing it in a second storage means;
Including
The 3D video quality calculating means includes:
Encoded video quality calculation means for calculating an encoded video quality evaluation value immediately after 3D video is encoded, using a statistic of the amount of information for each video frame stored in the second storage means 3D video quality evaluation apparatus characterized by the above. The video parameter extraction means includes
The 3D video quality evaluation apparatus according to claim 1, further comprising an Inter-view encoding determination unit that determines whether or not Inter-view encoding is used. The 3D video quality calculating means includes:
Video quality integration for calculating a 3D video quality evaluation value using the video frame deterioration amount stored in the first storage means and the encoded video quality evaluation value calculated by the encoded video quality calculation means The 3D video quality evaluation apparatus according to claim 1, further comprising means. The video frame deterioration amount calculating means includes
Means for counting the number of degraded video frames, and setting the value as the video frame degradation amount;
Or
Means for determining the video frame degradation amount as a degraded video frame rate obtained by dividing the number of degraded video frames by the total number of video frames;
Means for setting a weighted value to the deterioration amount when several frames are propagated within the same video frame as the deterioration amount when the deterioration propagates several frames in the time direction ;
The 3D video quality evaluation apparatus according to claim 1, comprising any one of the following. The video frame information amount calculating means includes
Means for extracting a statistic of information amount for each video frame type as the parameter;
The encoded video quality calculation means includes:
The 3D video quality evaluation apparatus according to claim 1, further comprising means for obtaining an encoded video quality evaluation value of a left eye video and a right eye video of the video view from the statistics of the information amount. A 3D video quality evaluation method for objectively evaluating the quality of 3D video,
A video parameter extraction step for extracting video quality parameters from the input video data;
And 3D video quality calculation step of calculating a 3D image quality before Kipa parameters,
Have
In the video parameter extraction step,
A video view of 3D video is determined, a video frame type is determined for each view of the 3D video, and when there is degradation in the video frame, a video frame referring to the video frame is set as a degraded video frame, and the video frame degradation amount And calculating a video frame deterioration amount that is stored in the first storage means;
Calculating a statistic of the amount of information for each video frame and storing it in a second storage means;
And
In the 3D video quality calculation step,
An encoded video quality calculation step of calculating an encoded video quality evaluation value immediately after the 3D video is encoded, using a statistic of the amount of information for each video frame stored in the second storage unit;
Video quality integration for calculating a 3D video quality evaluation value using the video frame deterioration amount stored in the first storage means and the encoded video quality evaluation value calculated in the encoded video quality calculation step Steps,
A 3D video quality evaluation method comprising: In the video parameter extraction step,
The 3D video quality evaluation method according to claim 6, wherein it is determined whether or not Inter-view coding is used. Computer
A 3D video quality evaluation program for causing each unit of the 3D video quality evaluation apparatus according to any one of claims 1 to 5 to function.

Images