JP4460549B2 - User experience quality estimation apparatus, method, and program - Google Patents

Description

  The present invention relates to a communication quality estimation technique, and more particularly to a technique for estimating user experience quality of a real-time application.

  In recent years, with the rapid development of packet communication technology, packet networks are being used in real-time applications that communicate media (contents) such as audio and video one-to-one or between multiple points. In the in-service quality management of the packet network, quality management for such an application is also required. Especially in real-time applications, quality management using user experience quality such as subjective or objective quality at the user level that actually uses the application is regarded as important, and technology that estimates user experience quality from network quality Have also been proposed (see, for example, Non-Patent Document 1).

  The inventors have also proposed a technique for estimating the subjective quality of a real-time application based on invalid packets (see, for example, Patent Document 1). In this technique, a lost packet that has not arrived at the receiving terminal among the test packets and a late arrival packet whose arrival interval between the previous arrival packets exceeds the allowable fluctuation absorption time of the real-time application are determined as invalid packets. The number of invalid packets is counted, and the ratio of the number of invalid packets to the total number of test packets transmitted from the transmission side is calculated as the invalid packet rate. The subjective quality evaluation value corresponding to the invalid packet rate is estimated with reference to the quality estimation model that shows the relationship.

  In contrast, the inventors have proposed a technique for estimating the subjective quality of a real-time application based on the degree of video quality degradation estimated from lost packets and the duration of video quality degradation (for example, patents). Reference 2 etc.). In this technique, for each video packet sent to a user terminal, the frame type and the frame generation rule at the time of high-efficiency compression encoding to which the packet belongs are embedded. At the user terminal, the degree of quality degradation within one video frame is estimated from the generation number of the lost packet, and the video quality degradation is determined from the frame type and the frame generation rule at the time of high-efficiency compression coding to which the lost packet belongs. The duration is estimated, and the subjective quality of the video decoded at the user terminal is estimated based on the estimated degree of video quality degradation and the duration of the video quality degradation. Thereby, the estimation accuracy was improved with a simple configuration.

Japanese Patent No. 3579334 JP 2006-033722 A ITU-T Recommendation G.107, "The E-Model, a computational model for use in transmission planning", May 2000

  However, in such a conventional technique, it is necessary to embed video frame information such as a frame type and a frame generation rule at the time of high-efficiency compression encoding to which the packet belongs in a media packet used for communication of media data. For video communications that cannot use video frame information of packets, there is a problem that the user experience quality cannot be estimated.

  In video communication, video frame information may not be acquired due to factors such as encryption of media packets. In addition, a forward error control method (FEC: Forward Error Correction) in which an error correction code packet for complementing a loss packet is generated, transmitted, and reconstructed on the receiving side requires a packet interpolation algorithm of the method, Video frame information cannot be used easily. Therefore, in such a case, the video frame information of the media packet cannot be used, and it is difficult to estimate the user experience quality.

  The present invention is for solving such problems. Even when video frame information of media packets cannot be used, the user experience quality value of a real-time application is highly accurate without requiring complicated processing. It is an object to provide a user experience quality estimation apparatus, method, and program that can be estimated by the above.

In order to achieve such an object, a user experience quality estimation apparatus according to the present invention is a media such as video and audio between a transmission side and a reception side application terminal connected via a packet network by a real-time application. A user experience quality estimation device for estimating a user experience quality that a user feels about a medium received and reproduced by a reception side application terminal when each frame obtained by encoding is transmitted and received as a media packet, A quality information acquisition unit for acquiring transmission quality information indicating the transmission status of media packets transmitted from the application terminal, and reception quality information indicating the reception status of media packets received by the receiving-side application terminal; and transmission quality information And sent in unit measurement period based on reception quality information From the invalid frame conversion model that shows the relationship between the number of invalid packets and the invalid frame rate, count the number of packets that could not be received normally among the media packets and calculate the invalid frame rate corresponding to the number of invalid packets. The user experience quality value corresponding to the invalid frame rate calculated by the quality analysis unit is derived from the quality analysis unit to be derived and the quality estimation model indicating the relationship between the invalid frame rate and the user experience quality value, and the user experience of the application And a user experience quality estimation unit that outputs the estimated quality value.

  At this time, the quality analysis unit uses the media quality packet that has been confirmed to be transmitted using the transmission quality information, the packet that is not confirmed to be received according to the reception quality information, and the media packet that is confirmed to be received according to the reception quality information. A packet whose delay fluctuation time indicating the arrival interval with the packet exceeds the allowable packet delay time of the application, and a packet whose transmission order and reception order are inconsistent among media packets that have been confirmed to be received by the reception quality information, You may make it determine with an invalid packet.

  Further, as an invalid frame conversion model, a function for calculating an invalid frame rate from a product of a unit invalid frame rate and the number of invalid packets indicating the number of invalid frames that can be generated according to one invalid packet in a unit measurement period is used. May be.

  In addition, the invalid frame conversion model is determined based on the frame configuration used for encoding the media extracted from the transmission quality information by the quality analysis unit, the number of transmission packets for each frame type related to media packets, and the frame rate of the media. A conversion model generation unit to be generated is further provided, and the conversion model generation unit calculates, for each frame type, the number of frame influences of frames that are invalid due to the influence of deterioration of one frame based on the frame configuration. A unit frame transmitted per unit measurement period from a product of a frame rate and a unit measurement period, a packet occupancy ratio calculation unit that calculates a packet occupancy ratio for each frame type based on the number of transmitted packets for each frame type Unit frame number calculation means for calculating the number of Unit invalid frame rate calculation means is provided for calculating the unit invalid frame rate by dividing the number of invalid frames per unit measurement period obtained by multiplying and multiplying the number of frame influences and the packet occupation rate by the number of unit frames. May be.

  Further, the quality information acquisition unit acquires the quality information from the transmission side application terminal received by the packet reception unit of the reception side application terminal as transmission quality information, and is received by the packet reception unit of the reception side application terminal. Quality information obtained from the media packet may be acquired as reception quality information.

  In addition, the quality information acquisition unit acquires the quality information from the reception side application terminal received by the packet reception unit of the transmission side application terminal as reception quality information and is transmitted by the packet transmission unit of the transmission side application terminal. Quality information obtained from the media packet may be acquired as transmission quality information.

  Moreover, you may further provide the quality information notification part which notifies the transmission quality information obtained in the packet transmission part of the transmission side application terminal to a reception side application terminal.

  Moreover, you may further provide the quality information notification part which notifies the receiving quality information obtained in the packet receiving part of the receiving side application terminal to a transmitting side application terminal.

In addition, the user experience quality estimation method according to the present invention is obtained by encoding media such as video and audio by a real-time application between a transmission side and a reception side application terminal connected via a packet network. A user experience quality estimation method used in a user experience quality estimation device for estimating a user experience quality that a user feels about a media received and played by a receiving application terminal when a frame is transmitted / received as a media packet, comprising: Quality information acquisition for acquiring transmission quality information indicating the transmission status of media packets transmitted from the transmission-side application terminal and reception quality information indicating the reception status of media packets received by the reception-side application terminal by the acquisition unit Step and quality analysis department The number of packets that could not be received normally among the media packets sent during the unit measurement period based on the information and the reception quality information is counted as the number of invalid packets, and the relationship between the number of invalid packets and the invalid frame rate is shown. A quality analysis step for deriving an invalid frame rate corresponding to the number of invalid packets from the invalid frame conversion model, and a quality analysis from a quality estimation model indicating a relationship between the invalid frame rate and the user experience quality value by the user experience quality estimation unit. A user experience quality estimation step of deriving a user experience quality value corresponding to the invalid frame rate calculated by the unit and outputting the user experience quality value as an estimated value of the user experience quality of the application.

Further, the program according to the present invention uses each frame obtained by encoding media such as video and audio by a real-time application between the transmission side and reception side application terminals connected via the packet network. When the packet is transmitted / received in a packet, the quality information acquisition unit transmits the user experience quality estimation device that estimates the user experience quality that the user feels about the media received and played by the reception side application terminal. A quality information acquisition step for acquiring transmission quality information indicating the transmission status of the media packet and reception quality information indicating the reception status of the media packet received by the receiving-side application terminal; Unit measurement based on reception quality information Corresponding to the number of invalid packets from the invalid frame conversion model that shows the relationship between the number of invalid packets and the invalid frame rate, counting the number of packets that could not be received normally among media packets sent in between Corresponding to the invalid frame rate calculated by the quality analysis unit based on the quality analysis step for deriving the invalid frame rate and the quality estimation model indicating the relationship between the invalid frame rate and the user experience quality value by the user experience quality estimation unit A user experience quality estimation step of deriving a user experience quality value and outputting it as an estimated value of the user experience quality of the application is executed.

  According to the present invention, the number of packets that could not be normally received among the media packets transmitted during the unit measurement period based on the transmission quality information and the reception quality information is counted as the number of invalid packets by the quality analysis unit. The invalid frame rate corresponding to the number of invalid packets is derived from the invalid frame conversion model indicating the relationship between the number of invalid packets and the invalid frame rate, and the relationship between the invalid frame rate and the user experience quality value is calculated by the user experience quality estimation unit. The user experience quality value corresponding to the invalid frame rate calculated by the quality analysis unit is derived from the quality estimation model indicating and is output as an estimate value of the user experience quality of the application.

Thus, the user experience quality value of real-time applications can be obtained from extremely simple and easily measurable information such as packet loss information and delay fluctuation information without checking the contents of media packets such as payload information of IP packets. Can be estimated with high accuracy.
Therefore, when the video frame information of the media packet cannot be used, for example, when the video frame information cannot be acquired due to factors such as encryption of the media packet, or a complicated algorithm such as a forward error control method (FEC) is applied. Even if it exists, the user experience quality value of a real-time application can be estimated with high precision, without requiring complicated processing.

Next, embodiments of the present invention will be described with reference to the drawings.
[First Embodiment]
First, a user experience quality estimation apparatus according to a first embodiment of the present invention will be described with reference to FIG. FIG. 1 is a block diagram showing the configuration of the user experience quality estimation apparatus according to the first embodiment of the present invention.

The user experience quality estimation apparatus 1 includes an information processing apparatus having a communication function, and is connected to the application terminal 2 and the packet network 5 to estimate the user experience quality of a real-time application executed on the application terminal 2.
The user experience quality estimation apparatus 1 includes a quality information acquisition unit 11, a quality analysis unit 12, a user experience quality estimation unit 14, an estimation result notification unit 17, and a quality information notification unit 18 as main functional units. Yes.

In this embodiment, the quality information acquisition unit 11 transmits transmission quality information indicating the transmission status of media packets used for communication of media data transmitted from the transmission-side application terminal, and media packets received by the reception-side application terminal. Reception quality information indicating the reception status of the packet, and the quality analysis unit 12 receives a packet of media packets transmitted during the unit measurement period based on the transmission quality information and the reception quality information, which has not been normally received. The invalid frame rate corresponding to the number of invalid packets is derived from the invalid frame conversion model indicating the relationship between the number of invalid packets and the invalid frame rate. Calculated by the quality analysis unit 12 from a quality estimation model indicating the relationship between the rate and the user experience quality value. It was derived QoE value corresponding to the invalid frame rate, in which to output as an estimate 16 of the QoE applications.

[User experience quality estimation device]
Next, the user experience quality estimation apparatus according to the first embodiment of the present invention will be described in detail with reference to FIG.
Each functional unit of the user experience quality estimation apparatus 1 includes a dedicated circuit unit, an arithmetic processing unit, and a storage unit. In particular, the arithmetic processing unit (computer) includes a microprocessor such as a CPU and its peripheral circuits, and reads and executes a program stored in a memory or a storage unit in the peripheral circuit, whereby the hardware and program To realize each functional unit. The storage unit includes a storage device such as a memory and a hard disk, and stores various processing information used for processing in each functional unit.

The quality information acquisition unit 11 has a function of acquiring transmission quality information indicating the transmission status of the media packet transmitted from the transmission-side application terminal from the packet reception unit 21 of the application terminal 2, and for the media received by the reception-side application terminal. It has a function of acquiring reception quality information indicating the reception status of packets from the packet receiving unit 21 of the application terminal 2.

The quality analysis unit 12 converts the invalid frame rate corresponding to the number of invalid packets in the unit measurement period counted from the transmission quality information and the reception quality information into an invalid frame conversion model 13 indicating the relationship between the number of invalid packets and the invalid frame rate. And a function derived based on this.
As main functional means constituting the quality analysis unit 12, there are an invalid packet determination means 12A and an invalid frame conversion means 12B.

The invalid packet determination unit 12A has a function of determining a packet that could not be normally received from media packets based on the transmission quality information and the reception quality information acquired by the quality information acquisition unit 11 as an invalid packet, and unit measurement. A function of counting the number of packets determined to be invalid within the period as the number of invalid packets.
The invalid frame conversion unit 12B has a function of deriving an invalid frame rate corresponding to the number of invalid packets counted by the invalid packet determination unit 12A from the invalid frame conversion model 13 indicating the relationship between the number of invalid packets and the invalid frame rate. is doing.

  An invalid packet is a packet that cannot be normally received by the receiving real-time application and cannot be reproduced as media among media packets that contain media (content) data sent from the sending real-time application. is there. Specifically, a packet lost on a transmission path connecting the transmission side application terminal and the reception side application terminal via the packet network 5, a late arrival packet whose arrival interval could not be adjusted by the fluctuation absorbing buffer of the reception side application, Furthermore, a packet in which the arrival order of media packets is reversed in a real-time application in which the arrival order of media packets is defined is targeted.

  The invalid frame conversion model 13 is a model showing the relationship between the number of invalid packets and the invalid frame rate. From the function having the actually measured invalid packet number as an input variable and a desired invalid frame rate as an output variable. Become. As a specific example of the invalid frame conversion model 13, a product of a unit invalid frame rate indicating the number of invalid frames that can occur in response to one invalid packet in the unit measurement period and the number of invalid packets that are input variables is obtained. A function for calculating the invalid frame rate may be used.

  The user experience quality estimation unit 14 derives a user experience quality value corresponding to the invalid frame rate calculated by the quality analysis unit 12 from the quality estimation model 15 indicating the relationship between the invalid frame rate and the user experience quality value. It has a function of outputting as a quality estimation value 16.

The estimation result notifying unit 17 uses the control packet to the user experience quality estimation device (not shown) and the quality management server (not shown) connected to the opposite application terminal via the packet network 5. It has a function of notifying the estimation result information including quality factor information such as transmission quality information used for calculation of the quality estimation value 16 obtained by the apparatus and the user experience quality value periodically or as necessary. Yes.
The quality information notification unit 18 has a function of notifying the transmission side application terminal of the transmission quality information obtained by the packet transmission unit 22 of the application terminal 2.

[Invalid frame conversion model]
Next, an invalid frame conversion model used in the user experience quality estimation apparatus according to the first embodiment of the present invention will be described in detail with reference to FIG. 2 and FIG. FIG. 2 is an explanatory diagram showing the relationship between packets and frames. FIG. 3 is an explanatory diagram showing the influence on other frames due to frame deterioration.

A packet and a video frame used in video communication have a relationship as shown in FIG. Here, the relationship between the amount of packets and a video frame in a unit measurement period (in this example, 10 sec) in which the user experience quality estimation is performed is shown.
The MPEG (Moving Picture Experts Group) GOP (Group of picture) configuration used in this video communication service is N = 15 (cycle in which I frames appear), M = 3 (cycle in which I or P frames appear), and the frame rate is In the case of 30 fps, 300 frames / 7200 packets are transmitted within the unit measurement period as shown in FIG. Each frame is composed of a plurality of packets. For example, the first transmitted I (1) frame is composed of 25 packets, and the second B (2) frame is composed of 3 packets. Yes.

When the packets transmitted within the unit measurement period are counted for each frame type (I, B, P) of the frame constituting the packet, each packet occupancy rate is 20% of the whole with I frame = 1440 packets, B frame = 3168 packets, 44% of the total, and P frame = 2592 packets, which is 36% of the total.
In addition, when the frames transmitted within the unit measurement period are counted for each frame type, the respective frame occupancy rates are 6.7% of the total when I frame = 20 frames, and the total when B frame = 200 frames. 66.7% of the total, P frame = 80 frames, and 26.7% of the total.

  Further, in the video communication (GOP configuration: N = 15, M = 3, frame rate = 30 fps) used in FIG. 2, since an encoding process is performed over a plurality of frames, when an arbitrary packet is lost, the packet This not only deteriorates the frame composed of the frames, but also affects the decoding of other frames. The influence on other frames varies depending on the frame type and the frame position of the frame formed by the invalid packet.

For example, as shown in FIG. 3, when the I (1) frame transmitted first deteriorates, the 15 (B) frame located 15 frames before (in the past) of the I (1) frame 15 A total of 17 frames up to the P (15) frame transmitted the second time are affected.
Further, when the B (3) frame transmitted third is deteriorated, only the relevant frame (one frame) is affected.

  On the other hand, the influence of the P frame differs depending on the deteriorated frame position. For example, the P (4) frame transmitted fourth affects a total of 12 frames from the frame to the P (15) frame, but the P (7) frame transmitted seventh is from the frame. A total of 9 frames up to P (15) frame are affected. Similarly, the P (10) frame transmitted tenth affects a total of six frames from the frame to the P (15) frame, and the P (13) frame transmitted thirteenth is the frame. A total of 3 frames from P to (15) frames are affected.

  In this way, when the number of frames affected when various types of frames are deteriorated is counted as one of the influence numbers, the number of influences of I frame = 17, B frame = 1, P frame = 30/4 = 7.5 is obtained. When the number of influences of each frame type and the packet occupancy rate of each frame type are summed up for each frame type, if one media packet cannot be received normally and becomes an invalid packet during the unit measurement period, the effect is In response, the number of frames that deteriorate to become invalid frames, that is, the number of unit invalid frames IVFe (expected value) can be calculated.

Further, the unit invalid frame rate IVFRe (expected value) can be calculated by dividing the number of unit invalid frames IVFe by the total number of frames transmitted in the unit measurement period. Therefore, if the number of invalid packets IVP _SUM is counted in the unit measurement period from actual video communication, the invalid frame rate can be calculated from the product of the number of invalid packets IVP _SUM and the unit invalid frame rate IVFRe.
In this embodiment, an invalid frame conversion model 13 indicating the relationship between the number of invalid packets and the invalid frame rate, in this case, a function f (IVP _SUM ) = IVFRe × IVP _SUM is obtained in advance, and this invalid frame is obtained. Based on the conversion model 13, the invalid frame rate corresponding to the actually measured number of invalid packets is calculated.

[Operation of First Embodiment]
Next, the operation of the user experience quality estimation apparatus according to the first embodiment of the present invention will be described with reference to FIG. FIG. 4 is a flowchart showing a user experience quality estimation process in the user experience quality estimation apparatus according to the first embodiment of the present invention.

  Here, when the application terminal 2 is the receiving side and the user experience quality value regarding the media packet of the video media transmitted from the opposite application terminal (not shown) on the transmitting side and received via the packet network 5 is estimated. Will be described as an example. At this time, when transmission quality information obtained by the packet transmission unit of the opposite application terminal is notified to the application terminal 2 by a control packet from a user experience quality estimation device (not shown) connected to the opposite application terminal Will be described as an example.

[User experience quality estimation processing]
The quality information acquisition unit 11 of the user experience quality estimation apparatus 1 receives the transmission quality information transmitted by the control packet such as RTCP XR (RTP Control Protocol) from the opposite application terminal side by the packet reception unit 21 of the application terminal 2. The acquired one is acquired (step 100). Further, the reception quality information obtained from the media packet from the opposite application terminal is acquired by the packet receiving unit 21 of the application terminal 2 (step 101).

  The main quality information included in the transmission quality information acquired by the quality information acquisition unit 11 of the user experience quality estimation device 1 includes a sequence number indicating a transmission order, a video frame type, and a video frame for each transmission media packet. There are a number, the number of transmitted packets, and the number of transmitted frames. The main quality information included in the reception quality information includes a sequence number indicating the transmission order and a delay fluctuation time indicating the arrival time difference from the immediately preceding media packet for each received media packet.

Next, the quality analysis unit 12 determines an invalid packet for each media packet whose transmission is confirmed by the transmission quality information by the invalid packet determination unit 12A, and counts the number of invalid packets IVP _SUM (step 102). ). The invalid packet is a packet that cannot be reproduced as media by the receiving-side real-time application among the media packets transmitted from the transmitting-side real-time application. Specifically, a packet lost on a transmission line connecting the transmission side application terminal and the reception side application terminal via the packet network 5, a late arrival packet whose arrival interval could not be adjusted by the fluctuation absorbing buffer of the reception side application, Furthermore, a packet in which the arrival order of media packets is reversed in a real-time application in which the arrival order of media packets is defined is targeted.

Next, the quality analysis unit 12 uses the invalid frame conversion unit 12B to obtain an invalid packet from the invalid frame conversion model 13 indicating the relationship between the number of invalid packets stored in the storage unit (not shown) and the invalid frame rate. An invalid frame rate corresponding to the invalid packet count IVP _SUM counted by the determination unit 12A is derived (step 103).
For example, when the invalid frame conversion model 13 has a function of f (IVP _SUM ) = IVFRe × IVP _SUM using the unit invalid frame rate IVFRe, the invalid packet number IVP _SUM counted by the invalid packet determination unit 12A and the unit invalid frame. A desired invalid frame rate is calculated from the product of the rate IVFRe.

  Next, the user experience quality estimation unit 14 obtains the quality analysis unit 15 from the quality estimation model 15 indicating the relationship between the invalid frame rate and the user experience quality value prepared in advance and stored in the storage unit (not shown). A user experience quality value corresponding to the invalid frame rate calculated in step 12 is derived (step 104), and is output to the storage unit as a quality estimate value 16 of the user experience quality of the real-time application.

In the creation of a quality estimation model, the relationship between the intermediate parameters and the user experience quality consisting of MOS values (Mean Opinion Score average opinion scores), which are subjective evaluation values obtained by actually performing the opinion evaluation, is expressed by a regression curve or a table. Model it.
Conventionally, the invalid packet rate of lost packets lost on the packet network 5 is used as an intermediate parameter, and the MOS value is estimated from the invalid packet rate using an estimation model indicating the relationship between the invalid packet rate and the MOS value. (Refer patent document 1 etc.). Such an estimation model can guarantee a certain degree of estimation accuracy, but is not sufficient when high estimation accuracy is required because media packets and frames do not correspond one-to-one.

  In this embodiment, the MOS value is estimated by the quality estimation model 15 using the invalid frame rate instead of the invalid packet rate. FIG. 5 is a quality estimation model showing the relationship between the invalid frame rate and the MOS value. According to the quality estimation model 15 as described above, the user experience quality value can be estimated with good accuracy with less variation compared to the conventional estimation model indicating the relationship between the invalid packet rate or the packet loss rate and the MOS value. .

As the quality estimation model 15 that is actually used by the user experience quality estimation unit 14, the regression curve of FIG. 5 may be used, or an estimation model obtained by tabulating the regression curve may be used.
Specifically, in FIG. 5, for example, when the minimum unit that is the minimum input unit of the input parameter (specifically, the minimum input unit of the invalid frame rate) is 0.01, it is output in 0.01 units. A table is created by averaging the measured values. In other words, by averaging the MOS values that fall within the minimum time range, a table of the relationship between the invalid frame rate and the MOS is created for every minimum time from 0. However, when the number of evaluation samples is small, there is a minimum data with no data. In such a case, complementation is performed using data having data before and after. This complementation may be linear or regression may be used. In order to minimize such a phenomenon, it is desirable to acquire a large number of data samples when creating a model.

Thereafter, the estimation result notifying unit 17 reads the quality estimation value 16 estimated by the user experience quality estimation unit 14 from the storage unit, stores it in a control packet such as RTCP XR, and the counter application via the packet network 5. It transmits with respect to the user experience quality estimation apparatus (not shown) and quality management server (not shown) connected to the terminal (step 105), and complete | finishes a series of user experience quality estimation processes.
At this time, the estimation result notification unit 17 may notify the result of statistical processing for obtaining the average value of the quality estimation value 16 and the variance indicating the variation at regular intervals.

As described above, in the present embodiment, the quality information acquisition unit 11 receives the transmission quality information indicating the transmission status of the media packet used for communication of the media data transmitted from the transmission-side application terminal, and the reception-side application terminal. Reception quality information indicating the reception status of the received media packet, and the quality analysis unit 12 can normally receive the media packet transmitted in the unit measurement period based on the transmission quality information and the reception quality information. The number of packets that did not exist is counted as the number of invalid packets, and an invalid frame rate corresponding to the number of invalid packets is derived from the invalid frame conversion model indicating the relationship between the number of invalid packets and the invalid frame rate. From the quality estimation model that shows the relationship between the invalid frame rate and the user experience quality value, quality analysis Deriving a QoE value corresponding to the invalid frame rate calculated in 12, and outputs as an estimate 16 of the QoE applications.

Thus, the user experience quality value of real-time applications can be obtained from extremely simple and easily measurable information such as packet loss information and delay fluctuation information without checking the contents of media packets such as payload information of IP packets. Can be estimated with high accuracy.
Therefore, when the video frame information of the media packet cannot be used, for example, when the video frame information cannot be acquired due to factors such as encryption of the media packet, or a complicated algorithm such as a forward error control method (FEC) is applied. Even if it exists, the user experience quality value of a real-time application can be estimated with high precision, without requiring complicated processing.

[Second Embodiment]
Next, a user experience quality estimation apparatus according to a second embodiment of the present invention will be described with reference to FIG. FIG. 6 is a flowchart showing invalid packet determination processing of the user experience quality estimation device according to the second exemplary embodiment of the present invention.

In the present embodiment, a specific example of invalid packet determination processing in the invalid packet determination means 12A of the first embodiment described above will be described.
First, the invalid packet determination unit 12A arbitrarily selects an undetermined transmission media packet from the transmission quality information (step 110), and confirms whether or not a media packet having the same sequence number is received with the reception quality information ( Step 111). If no media packet having the same sequence number is received (step 111: NO), the media packet is an invalid packet since it can be determined that the media packet is a lost packet lost in the transmission path. (Step 115).

  On the other hand, when the same media packet as the sequence number has been received (step 111: YES), the delay fluctuation time of the transmission media packet is confirmed with the reception quality information, and the allowable delay time possessed by the real-time application Compare (step 112). Here, if the delay fluctuation time exceeds the allowable delay time (step 112: NO), it can be determined that the media packet is a late arrival packet that arrives after being delayed in the transmission path or terminal. Therefore, it is determined that the packet is invalid (step 115).

  On the other hand, when the delay fluctuation time is equal to or shorter than the allowable delay time (step 112: YES), only when the real-time application considers the arrival order reverse packet as a loss (step 113: YES), and the media packet It is confirmed from the reception quality information whether the order of arrival and the sequence number match. Here, when the packet cannot be confirmed and the arrival order is reversed (step 114: YES), it is determined that the packet is an invalid packet (step 115).

When the arrival order is not reversed (step 114: NO), or when the real-time application does not regard the arrival order reversed packet as a loss (step 113: NO), it is determined that the packet is not an invalid packet. (Step 116).
In this way, after the invalid packet determination is performed on the selected transmission media packet, if there is a transmission packet that has not been determined to be invalid (step 117: YES), the process returns to step 110 described above. The invalid packet determination for the remaining received media packets is repeated. On the other hand, when the invalid packet determination is completed for all the transmission media packets (step 117: NO), the series of invalid packet determination processing ends.

  As described above, in the present embodiment, the invalid packet determination unit 12A determines a media packet that has not been confirmed to be received in the reception quality information from among the media packets that have been confirmed to be transmitted in the transmission quality information, and the reception quality information. Media packets whose arrival time difference from the previous media packet obtained from the arrival time of each media packet exceeds the allowable packet delay time of the application are judged as invalid packets. Packets lost due to arrival can also be determined as invalid packets, and packets that cannot be reproduced as media by the real-time application on the receiving side can be accurately determined.

  Further, since the invalid packet determination means 12A determines that the media packet whose reception order is inconsistent with the transmission order of the media packet confirmed by the reception quality information is an invalid packet, the real-time application arrives. When it is considered that the order-reversed packet is a loss, it is possible to more accurately determine a packet that cannot be reproduced as media by the receiving-side real-time application.

[Third Embodiment]
Next, a user experience quality estimation apparatus according to the third embodiment of the present invention will be described with reference to FIGS. FIG. 7 is a block diagram showing the configuration of the user experience quality estimation device according to the third exemplary embodiment of the present invention. The same or equivalent parts as those in FIG. FIG. 8 is a block diagram illustrating the configuration of the conversion model generation unit of the user experience quality estimation device according to the third embodiment of the present invention.

In the first embodiment, the case where the invalid frame conversion model 13 is stored in advance in the storage unit as a function expression has been described as an example. This Embodiment demonstrates the case where the function part which produces | generates the invalid frame conversion model 13 is provided in the user experience quality estimation apparatus 1. FIG.
Compared to the first embodiment (see FIG. 1), the user experience quality estimation device 1 according to the present embodiment includes a conversion model including a dedicated circuit unit and an arithmetic processing unit, as with other functional units. A generation unit 19 is provided. Further, the quality analysis unit 12 is provided with a frame type determination unit 12C, a transmission packet number recording unit 12D, and a frame rate calculation unit 12E as new functional units. Other configurations are the same as those in the first embodiment, and a detailed description thereof is omitted here.

  The frame type determination means 12C of the quality analysis unit 12 has a function of determining the frame type from the header information of the media packet acquired by the quality information acquisition unit 11, and a function of recognizing the frame configuration of the video media. Yes. At this time, the quality information acquisition unit 11 acquires each transmission media packet including the payload from the application terminal 2, and extracts header information for media data decoding included in the packet.

  At this time, for example, in the case of an IP application to which MPEG-2TS encoding is applied, a TS (Transport) packet header included in the IP packet, a PES header included in the TS packet, and a PES (Packetized Elementary Stream) packet GOP header information included in the quality information acquisition unit 11 is acquired. The TS packet recognizes that the packet is a video packet, identifies the B frame and the other frames by the PES header, and identifies the other frame by the GOP header as one frame and a P frame. At this time, since the G frame configuration from the first recognition of the I frame to the recognition of the next I frame is one GOP configuration, the GOP configuration can also be recognized at this stage.

Based on the frame type determination result in the frame type determination unit 12C and further the GOP configuration recognition result, the transmission packet number counting unit 12D determines the media packet that constitutes the frame for each of the I, B, and P frame types. A function for counting the number of transmitted packets (n _I , n _B , n _P ), and a function for deriving a GOP configuration by recording the frame generation process from the order of the media packets and the frame type based on the count results have.

  The frame rate calculation means 12E has a function of calculating the frame rate of the video media from the header information of the media packet acquired by the quality information acquisition unit 11. For example, in the case of an IP application to which MPEG-2TS encoding is applied, time information for displaying a video frame such as a PTS (Presentation Time Stamp) is included in the PES header included in the payload of the IP packet. Based on such time information, it is possible to recognize the display interval between frames or the number of frames displayed per second based on the PTS time, thereby calculating the frame rate.

As shown in FIG. 8, the conversion model generation unit 19 includes unit frame number calculation means 19A, invalid frame influence number calculation means 19B, packet occupancy rate calculation means 19C, and unit invalid frame rate calculation means as main function means. 19D is provided.
The unit frame number calculation unit 19A multiplies the unit measurement period T (19E) stored in the storage unit in advance by the frame rate fr calculated by the frame rate calculation unit 12E of the quality analysis unit 12, for example. It has a function of calculating the number of unit frames fr × T indicating the number of frames to be transmitted per measurement period 19E.

The invalid frame influence number calculation means 19B is based on the GOP configuration obtained by the transmission packet number counting means 12D of the quality analysis unit 12, and for each frame type, the invalid frame influence number calculation means 19B is affected by the deterioration of one frame. It has a function of calculating the influence number (l _I , l _B , l _P ). In this case, as shown in FIG. 3 described above, the affected frames may be counted and calculated for each frame type based on the GOP configuration.
Based on the number of transmission packets (n _I , n _B , n _P ) obtained by the transmission packet number counting unit 12D of the quality analysis unit 12, the packet occupancy rate calculation unit 19C determines the packet occupancy rate (p _I for each frame type). , P _B , p _P ). In this case, the packet occupation ratio (p _I , p _B , p _P ) may be calculated from the ratio of the number of transmission packets (n _I , n _B , n _P ) to the total transmission media packets in the unit measurement time T. .

The unit invalid frame rate calculation means 19D includes the unit frame number fr × T obtained by the unit frame number calculation means 19A, and the invalid frame influence number (l _I , for each frame type obtained by the invalid frame influence number calculation means 19B. l _B , l _P ) and the packet occupancy rate (p _I , p _B , p _P ) for each frame type obtained by the packet occupancy rate calculation means 19C, the unit invalid frame rate used in the invalid frame conversion model 13 is determined. It has a function to calculate. In this case, the number of invalid frame influences and the packet occupancy rate are summed for each frame type to calculate the number of invalid frames IVFe = p _I × l _I + p _B × l _B + p _P × l _P per unit measurement period. . Then, the unit invalid frame rate IVFRe = IVFe / (fr × T) may be calculated by dividing the unit invalid frame number IVFe by the unit frame number fr × T.

[Operation of Third Embodiment]
Next, the conversion model generation operation of the user experience quality estimation device according to the third exemplary embodiment of the present invention will be described.
When generating the conversion model generation operation, the quality information acquisition unit 11 first acquires each transmission media packet including the payload from the application terminal 2, and extracts the header information for media data decoding included in the packet. .

The frame type determination unit 12C of the quality analysis unit 12 determines the frame type from the header information of the media packet acquired by the quality information acquisition unit 11, and recognizes the GOP configuration of the video media.
Based on the frame type determination result in the frame type determination unit 12C and further the GOP configuration recognition result, the transmission packet number counting unit 12D determines the media packet that constitutes the frame for each of the I, B, and P frame types. The number of transmission packets (n _I , n _B , n _P ) is counted. Then, based on these count results, the GOP configuration is derived by recording the frame generation process from the order of the media packets and the frame type.
The frame rate calculation means 12E calculates the frame rate of the video media from the header information of the media packet acquired by the quality information acquisition unit 11.

The unit frame number calculation unit 19A of the conversion model generation unit 19 multiplies the unit measurement period T (19E) stored in the storage unit in advance by the frame rate fr calculated by the frame rate calculation unit 12E of the quality analysis unit 12. As a result, the number of unit frames fr × T indicating the number of frames transmitted per unit measurement period 19E is calculated.
The invalid frame influence number calculation means 19B is based on the GOP configuration obtained by the transmission packet number counting means 12D of the quality analysis unit 12, and for each frame type, the invalid frame influence number calculation means 19B is affected by the deterioration of one frame. The influence number (l _I , l _B , l _P ) is calculated.

Based on the number of transmission packets (n _I , n _B , n _P ) obtained by the transmission packet number counting unit 12D of the quality analysis unit 12, the packet occupancy rate calculation unit 19C determines the packet occupancy rate (p _I for each frame type). , P _B , p _P ).
The unit invalid frame rate calculation means 19D includes the unit frame number fr × T obtained by the unit frame number calculation means 19A, and the invalid frame influence number (l _I , for each frame type obtained by the invalid frame influence number calculation means 19B. l _B , l _P ) and the packet occupancy rate (p _I , p _B , p _P ) for each frame type obtained by the packet occupancy rate calculation means 19C, the unit invalid frame rate used in the invalid frame conversion model 13 is determined. calculate.

  As described above, in this embodiment, as the invalid frame conversion model 13, the product of the unit invalid frame rate and the number of invalid packets indicating the number of invalid frames that can be generated according to one invalid packet in the unit measurement period is obtained. Since the function for calculating the invalid frame rate is used, a desired invalid frame rate corresponding to the number of invalid frames can be obtained by a very simple process.

  In addition, the conversion model generation unit 19 includes an invalid frame influence number calculating unit 19B that calculates, for each frame type, the frame influence number of a frame that is invalid due to the influence of deterioration of one frame based on the frame configuration, and the frame type. A unit for calculating the number of unit frames transmitted per unit measurement period from the packet occupancy rate calculation means 19C for calculating the packet occupancy rate for each frame type based on the number of transmitted packets for each unit and the product of the frame rate and the unit measurement period The unit invalid frame rate is obtained by dividing the number of invalid frames per unit measurement period obtained by multiplying the number of invalid frame influences and the packet occupancy rate for each frame type by the sum of frames calculation unit 19A and the unit frame number. The unit invalid frame rate calculation means 19D for calculating the Units can be derived invalid frame rate for GOP structure, can easily generate an invalid frame transformation model 13.

  In the present embodiment, the frame type determination unit 12C, the transmission packet number recording unit 12D, and the frame rate calculation unit 12E of the quality analysis unit 12 shown in FIG. 8, the unit frame number calculation unit 19A of the conversion model generation unit 19, Although the case where the invalid frame influence number calculating unit 19B, the packet occupation rate calculating unit 19C, and the unit invalid frame rate calculating unit 19D are provided in the user experience quality estimation apparatus 1 has been described as an example, the present invention is not limited thereto. Absent. For example, these configurations may be implemented in a conversion model generation device that is separate from the user experience quality estimation device 1, and the obtained invalid frame conversion model 13 may be stored in advance in the storage unit of the user experience quality estimation device 1. Good.

[Extension of each embodiment]
In each of the above embodiments, the application terminal 2 is the receiving side, and the user experience quality relating to the media packet of the video media transmitted from the opposite application terminal (not shown) on the transmitting side and received via the packet network 5 Although the case where the value is estimated has been described as an example, the present invention is not limited to this. That is, it is possible to estimate the user experience quality value related to the media packet of the video media received via the packet network 5 at the opposite application terminal (not shown) as the reception side, where the application terminal 2 is the transmission side.

In this case, the quality information acquisition unit 11 of the user experience quality estimation apparatus 1 uses the quality information obtained by the packet transmission unit 22 of the application terminal 2 as the transmission quality information described above, and the packet reception unit of the application terminal 2 The quality information from the opposite application terminal received at 21 may be used as the reception quality information described above.
At this time, when the similar user experience quality is also estimated in the opposite application terminal, instead of transmitting the transmission quality information obtained by the packet transmission unit 22 of the application terminal 2 by the quality information notification unit 18 described above, packet reception The reception quality information regarding the reception media packet obtained by the unit 21 may be transmitted to the opposite application terminal.

  The transmission / reception quality information may be directly exchanged between the transmission side and the reception side, but may be exchanged via a quality management server that manages the quality information. Further, the user experience quality estimation device 1 may receive quality information directly.

  In each of the above embodiments, the case where the user experience quality estimation device 1 is realized by a device separate from the application terminal 2 has been described as an example. However, the present invention is not limited to this and is parallel to a real-time application. Then, it may be realized by a quality estimation application executed on the application terminal 2. As a result, the hardware resources of the application terminal 2 can be shared and realized, and the user experience quality can be managed very easily.

  Further, in each of the above embodiments, as an invalid frame conversion model 13 used in the user experience quality estimation apparatus 1, a case where a function for calculating an invalid frame rate from the product of the unit invalid frame rate and the number of invalid packets is used as an example. However, the present invention is not limited to this, and an invalid frame conversion model 13 composed of other functions is used as long as the number of invalid packets is an input variable and a function having a desired invalid frame rate as an output variable. May be. Further, other models than functions, such as a black box model in which only input / output characteristics are specified, such as a neural network or a case base, may be used.

  In each of the above embodiments, the unit invalid frame rate is calculated on the assumption that invalid packets occur at a rate of one per frame. This is because the probability that a plurality of invalid packets are actually generated in the same frame is extremely small and the influence on the estimation accuracy is small, and the invalid frame conversion model 13 is simplified by omitting such a phenomenon. And the simplification of the invalid frame rate derivation process is realized. In addition, processing such as calculating the number of unit invalid frames by finding the probability of multiple invalid packets actually occurring in the same frame for each frame type, is invalid when multiple invalid packets occur in the same frame It may be included in the frame rate derivation process.

It is a block diagram which shows the structure of the user experience quality estimation apparatus concerning the 1st Embodiment of this invention. It is explanatory drawing which shows the relationship between a packet and a frame. It is explanatory drawing which shows the influence on other frames by frame degradation. It is a flowchart which shows the user experience quality estimation process of the user experience quality estimation apparatus concerning the 1st Embodiment of this invention. It is a quality estimation model showing the relationship between the invalid frame rate and the MOS value. It is a flowchart which shows the invalid packet determination process of the user experience quality estimation apparatus concerning the 2nd Embodiment of this invention. It is a block diagram which shows the structure of the user experience quality estimation apparatus concerning the 3rd Embodiment of this invention. It is a block diagram which shows the structure of a conversion model production | generation part.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... User experience quality estimation apparatus, 11 ... Quality information acquisition part, 12 ... Quality analysis part, 12A ... Invalid packet determination means, 12B ... Invalid frame conversion means, 12C ... Frame type determination means, 12D ... Transmission packet number count means, 12E ... Frame rate calculation means, 13 ... Invalid frame conversion model, 14 ... User experience quality estimation unit, 15 ... Quality estimation model, 16 ... Quality estimation value, 17 ... Estimation result notification unit, 18 ... Quality information notification unit, 19 ... Conversion model generation unit, 19A: unit frame number calculation means, 19B ... invalid frame influence number calculation means, 19C ... packet occupancy rate calculation means, 19D ... unit invalid frame rate calculation means, 19E ... unit measurement period, 2 ... application terminal, 21: Packet receiver, 22: Packet transmitter, 5: Packet network.

Claims (10)

When each frame obtained by encoding media such as video and audio by a real-time application is transmitted / received as a media packet between a transmission-side and reception-side application terminal connected via a packet network, the reception is performed. A user experience quality estimation device that estimates a user experience quality that a user feels about media received and played by a side application terminal,
A quality information acquisition unit that acquires transmission quality information indicating a transmission status of a media packet transmitted from the transmission-side application terminal and reception quality information indicating a reception status of the media packet received by the reception-side application terminal. When,
Based on the transmission quality information and the reception quality information, the number of packets that could not be normally received among the media packets transmitted during the unit measurement period is counted as the number of invalid packets, and the number of invalid packets and the invalid frame rate A quality analysis unit for deriving an invalid frame rate corresponding to the number of invalid packets from the invalid frame conversion model indicating the relationship between
From the quality estimation model indicating the relationship between the invalid frame rate and the user experience quality value, a user experience quality value corresponding to the invalid frame rate calculated by the quality analysis unit is derived, and as an estimate of the user experience quality of the application A user experience quality estimation device comprising: an output user experience quality estimation unit. In the user experience quality estimation apparatus according to claim 1,
The quality analysis unit includes: a media packet whose transmission is confirmed with the transmission quality information, a packet whose reception is not confirmed with the reception quality information, and a media immediately before among the media packets whose reception is confirmed with the reception quality information Packets in which the delay fluctuation time indicating the arrival interval with the packet for use exceeds the allowable packet delay time of the application, and packets in which the transmission order and the reception order are inconsistent among the media packets confirmed to be received with the reception quality information Each of which is determined to be the invalid packet. In the user experience quality estimation apparatus according to claim 1,
The invalid frame conversion model is a function for calculating the invalid frame rate from a product of a unit invalid frame rate indicating the number of invalid frames that can occur in response to one invalid packet in the unit measurement period and the invalid packet number. A user experience quality estimation device comprising: In the user experience quality estimation apparatus according to claim 1,
The invalidity based on the frame configuration used for encoding the media, the number of transmission packets for each frame type related to the media packet, and the frame rate of the media extracted from the transmission quality information by the quality analysis unit A conversion model generation unit for generating a frame conversion model;
The conversion model generation unit
Invalid frame influence number calculating means for calculating, for each frame type, the frame influence number of a frame that is invalid due to the influence of deterioration of one frame based on the frame configuration;
A packet occupancy rate calculating means for calculating a packet occupancy rate for each frame type based on the number of transmitted packets for each frame type;
Unit frame number calculating means for calculating the number of unit frames transmitted per unit measurement period from the product of the frame rate and the unit measurement period;
The unit invalid frame rate is calculated by dividing the number of invalid frames per unit measurement period obtained by multiplying the invalid frame influence number and the packet occupancy rate for each frame type by the unit frame number. A user experience quality estimation device comprising: a unit invalid frame rate calculation unit. In the user experience quality estimation apparatus according to claim 1,
The quality information acquisition unit acquires, as the transmission quality information, the quality information from the transmission side application terminal received by the packet reception unit of the reception side application terminal, and at the packet reception unit of the reception side application terminal. A user experience quality estimation device, wherein quality information obtained from a received media packet is acquired as the reception quality information. In the user experience quality estimation apparatus according to claim 1,
The quality information acquisition unit acquires, as the reception quality information, quality information from the reception-side application terminal received by the packet reception unit of the transmission-side application terminal, and a packet transmission unit of the transmission-side application terminal. A user experience quality estimation device, wherein quality information obtained from a transmitted media packet is acquired as the transmission quality information. In the user experience quality estimation apparatus according to claim 1,
The user experience quality estimation apparatus further comprising a quality information notification unit that notifies the reception side application terminal of the transmission quality information obtained by the packet transmission unit of the transmission side application terminal. In the user experience quality estimation apparatus according to claim 1,
The user experience quality estimation apparatus further comprising a quality information notification unit for notifying the reception side application terminal of the reception quality information obtained by the packet reception unit of the reception side application terminal. When each frame obtained by encoding media such as video and audio by a real-time application is transmitted / received as a media packet between a transmission-side and reception-side application terminal connected via a packet network, the reception is performed. A user experience quality estimation method used in a user experience quality estimation device that estimates a user experience quality that a user feels about the media received and played by a side application terminal,
Transmission quality information indicating the transmission status of the media packet transmitted from the transmission-side application terminal and reception quality information indicating the reception status of the media packet received by the reception-side application terminal by the quality information acquisition unit. A quality information acquisition step to be acquired;
The quality analysis unit counts the number of packets that could not be normally received among the media packets transmitted during the unit measurement period based on the transmission quality information and the reception quality information as the number of invalid packets. A quality analysis step of deriving an invalid frame rate corresponding to the invalid packet number from an invalid frame conversion model indicating a relationship between the number and the invalid frame rate;
The user experience quality estimation unit derives a user experience quality value corresponding to the invalid frame rate calculated by the quality analysis unit from a quality estimation model indicating a relationship between the invalid frame rate and the user experience quality value, and A user experience quality estimation method comprising: a user experience quality estimation step of outputting as an estimate value of the user experience quality. When each frame obtained by encoding media such as video and audio by a real-time application is transmitted / received as a media packet between a transmission-side and reception-side application terminal connected via a packet network, the reception is performed. In the computer of the user experience quality estimation device that estimates the user experience quality that the user feels about the media received and played by the side application terminal,
Transmission quality information indicating the transmission status of the media packet transmitted from the transmission-side application terminal and reception quality information indicating the reception status of the media packet received by the reception-side application terminal by the quality information acquisition unit. A quality information acquisition step to be acquired;
The quality analysis unit counts the number of packets that could not be normally received among the media packets transmitted during the unit measurement period based on the transmission quality information and the reception quality information as the number of invalid packets. A quality analysis step of deriving an invalid frame rate corresponding to the invalid packet number from an invalid frame conversion model indicating a relationship between the number and the invalid frame rate;
The user experience quality estimation unit derives a user experience quality value corresponding to the invalid frame rate calculated by the quality analysis unit from a quality estimation model indicating a relationship between the invalid frame rate and the user experience quality value, and A program for executing a user experience quality estimation step that is output as an estimated value of the user experience quality.

Images