JP4460523B2 - User experience quality estimation apparatus, method, and program - Google Patents

Description

  The present invention relates to a communication quality estimation technique, and more particularly to a technique for estimating user experience quality of a real-time application.

  2. Description of the Related Art In recent years, with the rapid development of packet communication technology, packet networks represented by the Internet are being used by real-time applications that communicate media such as audio and video one-to-one or between multiple points. Therefore, in-service quality management of the packet network also requires quality management for such applications. Especially in real-time applications, quality management using user experience quality that shows subjective or objective quality at the user level that actually uses the application is regarded as important, and technology that estimates user experience quality from network quality Has also been proposed (see, for example, Non-Patent Documents 1-3)

  Conventionally, the inventors have proposed a technique for estimating the subjective quality of a real-time application (see, for example, Patent Document 1). In this technique, a lost packet that has not arrived at the receiving terminal among the test packets and a late arrival packet whose arrival interval between the previous arrival packets exceeds the allowable fluctuation absorption time of the real-time application are determined as invalid packets. The number of invalid packets is counted, and the ratio of the number of invalid packets to the total number of test packets transmitted from the transmission side is calculated as the invalid packet rate. The subjective quality evaluation value corresponding to the invalid packet rate is estimated with reference to the quality estimation model that shows the relationship.

JP 2000-247421 A ITU-T Recommendation G.107, "The E-Model, a computational model for use in transmission planning", May.2000 A. Clark, `` Modeling the Effects of Burst Packet Loss and Recency on Subjective Voice Quality '', IP Telephony Workshop 2001, Apr. 2001 R. Caceres and A. Clark, "RTP Control Protocol Extended Reports (RTCP XR)", IETF RFC3611, May. 2003

  However, in such a conventional technique, since the subjective quality is estimated based on the invalid packet rate focusing on the media packet used for media data communication, a certain degree of estimation accuracy can be guaranteed, but high estimation accuracy is required. There was a problem that it was not enough in some cases.

  In real-time applications, media signals such as highly efficient encoded video signals and audio signals are decomposed into frames, which are further converted into media packets for exchange. For this reason, when the media signal is decoded by the application terminal, a frame is assembled from the received media packets, and the media signal is decoded using the preceding and following frame information.

  Therefore, when packet loss occurs in the packet network, the quality deterioration of the media signal due to the loss of one packet may extend not only to the frame but to a plurality of frames. For example, if there is no packet loss in the next frame where packet loss has occurred, the loss of the previous frame affects the decoding of the next frame, and quality degradation occurs in the decoded media signal, but at the packet level Since it is determined that there is no quality degradation, the estimation result is different from the quality degradation status in an actual real-time application.

  In addition, human perceptual characteristics are not sensitive to deterioration of media such as video and audio, and there are aspects that require a certain period of time to recognize the deterioration. For this reason, in the reproduced media, the user does not recognize all frames affected by the packet loss as quality degradation. Therefore, an error may occur between the subjective quality evaluation of the user with respect to the media and the estimation result estimated from the quality degradation state in the actual real-time application.

  The present invention is intended to solve such problems, and an object thereof is to provide a user experience quality estimation apparatus, method, and program capable of estimating a user experience quality value of a real-time application with high accuracy. .

In order to achieve such an object, a user experience quality estimation apparatus according to the present invention is used between a transmission side and a reception side application terminal connected via a packet network to encode media such as video and audio. A user experience quality estimation device for estimating a user experience quality for a real-time application that transmits and receives each frame obtained by converting to a media packet, and transmits a transmission status of a media packet transmitted from a transmission-side application terminal and transmission quality information indicating a quality information acquiring unit that acquires reception quality information indicating the reception status of the media packets received by the receiving application terminal, and receive a media packet based on the transmission quality information and reception quality information Each frame is affected by the loss of media packets. A quality analysis unit that determines invalid frames, calculates a ratio of equivalent invalid frames selected by weighting these invalid frames with a predetermined number of weighted frames as an equivalent invalid frame rate, and an equivalent invalid frame rate and user experience quality A user experience quality estimation unit that derives a user experience quality value corresponding to the equivalent invalid frame rate calculated by the quality analysis unit from a quality estimation model indicating a relationship with the value and outputs the user experience quality value as an estimated value of the user experience quality of the application; It has.

  At this time, the quality analysis unit corresponds to the invalid packet based on the transmission quality information, and invalid packet determination means for determining the lost packet among the media packets based on the transmission quality information and the reception quality information as an invalid packet. Frame identification means for identifying a frame as a target frame, invalid frame determination means for determining whether each frame is an invalid frame based on the frame configuration and target frame used for encoding the media, and one or more consecutive Equivalent invalid frame selection means for selecting invalid frames that exceed the number of weighted frames from the invalid frames as equivalent invalid frames, and the ratio of equivalent invalid frames to the total number of frames transmitted as media packets is calculated as the equivalent invalid frame rate Equivalent invalid frame rate calculating means may be provided.

  In addition, the invalid packet determination means indicates the arrival interval of media packets whose reception quality information is not confirmed among the media packets whose transmission quality is confirmed by the transmission quality information, and each media packet of the reception quality information. A media packet whose delay fluctuation time exceeds the allowable packet delay time of the application may be determined as an invalid packet.

  Further, the invalid packet determination means may determine a media packet in which the transmission order and the reception order of the media packet confirmed to be received by the reception quality information are inconsistent as an invalid packet.

  Further, the frame identification means acquires the frame number and frame type of the target frame from the invalid packet header information included in the transmission quality information, and the invalid frame determination means determines that the target frame is an invalid frame among the frames. At the same time, when an invalid frame is included in another frame used for decoding the frame according to the arrangement of the frame types defined in the frame configuration, the frame may be determined as an invalid frame.

  Further, the quality information acquisition unit acquires the transmission quality information from the transmission side application terminal side received by the packet reception unit of the reception side application terminal as transmission quality information and is received by the packet reception unit of the reception side application terminal. Reception quality information obtained from the media packet may be acquired as reception quality information.

  Further, the quality information acquisition unit acquires the reception quality information from the reception side application terminal received by the packet reception unit of the transmission side application terminal as reception quality information and is transmitted by the packet transmission unit of the transmission side application terminal. Transmission quality information obtained from the media packet may be acquired as transmission quality information.

  Moreover, you may further provide the quality information notification part which notifies the transmission quality information obtained in the packet transmission part of the transmission side application terminal to a reception side application terminal.

  You may further provide the quality information notification part which notifies the transmission quality application terminal of the reception quality information obtained by the packet reception part of the reception side application terminal.

In addition, the user experience quality estimation method according to the present invention is used between a transmission-side and reception-side application terminal connected via a packet network, and each frame obtained by encoding media such as video and audio is encoded. A user experience quality estimation method used in a user experience quality estimation apparatus for estimating a user experience quality for a real-time application that transmits and receives with a media packet, the transmission status of a media packet transmitted from a transmission-side application terminal being and transmission quality information indicating a quality information acquiring step of acquiring reception quality information indicating the reception status of the media packets received by the receiving application terminal, and receive a media packet based on the transmission quality information and reception quality information For loss of media packets out of each frame A quality analysis step for determining an invalid frame affected by the above, and calculating a ratio of equivalent invalid frames selected by weighting the invalid frames with a predetermined number of weight frames as an equivalent invalid frame rate, and an equivalent invalid frame The user who derives the user experience quality value corresponding to the equivalent invalid frame rate calculated in the quality analysis step from the quality estimation model indicating the relationship between the rate and the user experience quality value, and outputs it as the estimated value of the user experience quality of the application A bodily sensation quality estimation step.

Further, the program according to the present invention is used between a transmission side and a reception side application terminal connected via a packet network, and each frame obtained by encoding media such as video and audio is a media packet. For the real-time application to be transmitted and received, the computer of the user experience quality estimation device that estimates the user experience quality, the transmission quality information indicating the transmission status of the media packet transmitted from the transmission side application terminal, and the reception side application terminal The quality information acquisition step for acquiring the reception quality information indicating the reception status of the received media packet, and the influence of the loss of the media packet among the frames transmitted and received by the media packet based on the transmission quality information and the reception quality information. No percentage of invalid frames received A quality analysis step of determining a frame rate and calculating a ratio of equivalent invalid frames selected by weighting these invalid frames with a predetermined number of weight frames as an equivalent invalid frame rate, and an equivalent invalid frame rate and user experience quality A user experience quality estimation step of deriving a user experience quality value corresponding to the equivalent invalid frame rate calculated in the quality analysis step from a quality estimation model indicating a relationship with the value, and outputting as an estimate value of the user experience quality of the application; It is a program that executes.

  According to the present invention, the ratio of invalid frames affected by the loss of the media packet among the frames transmitted and received by the media packet based on the transmission quality information and the reception quality information is set as the invalid frame rate by the quality analysis unit. The calculated user experience quality estimation unit derives a user experience quality value corresponding to the invalid frame rate calculated by the quality analysis unit from the quality estimation model indicating the relationship between the invalid frame rate and the user experience quality value. It is output as an estimated value of the user experience quality of the application.

  Therefore, when estimating the user experience quality, the frame formed by each media packet is considered in comparison with the case where the user experience quality value is estimated from the packet loss rate or invalid packet rate obtained as objective quality information. Thus, the user experience quality value can be estimated, and the user experience quality value of the real-time application can be estimated with high accuracy.

Next, embodiments of the present invention will be described with reference to the drawings.
[First Embodiment]
First, a user experience quality estimation apparatus according to a first embodiment of the present invention will be described with reference to FIG. FIG. 1 is a block diagram showing the configuration of the user experience quality estimation apparatus according to the first embodiment of the present invention.

The user experience quality estimation apparatus 1 includes an information processing apparatus having a communication function, and is connected to the application terminal 2 and the packet network 5 to estimate the user experience quality of a real-time application executed on the application terminal 2.
The user experience quality estimation apparatus 1 includes a quality information acquisition unit 11, a quality analysis unit 12, a user experience quality estimation unit 13, an estimation result notification unit 16, and a quality information notification unit 17 as main functional units. Yes.

In this embodiment, the quality information acquisition unit 11 transmits transmission quality information indicating the transmission status of media packets used for communication of media data transmitted from the transmission-side application terminal, and media packets received by the reception-side application terminal. The reception quality information indicating the reception status of the frame is acquired, and the quality analysis unit 12 receives the invalidity influenced by the loss of the media packet among the frames transmitted and received by the media packet based on the transmission quality information and the reception quality information. Frames are determined, and the ratio of equivalent invalid frames selected by weighting these invalid frames with a predetermined number of weighted frames is calculated as an equivalent invalid frame rate. From the quality estimation model that shows the relationship with the user experience quality value, Deriving a QoE value corresponding to the equivalent invalid frame rate calculated by 2, in which to output as an estimate of the QoE applications.

[User experience quality estimation device]
Next, the user experience quality estimation apparatus according to the first embodiment of the present invention will be described in detail with reference to FIG.
Each functional unit of the user experience quality estimation apparatus 1 includes a dedicated circuit unit, an arithmetic processing unit, and a storage unit. In particular, the arithmetic processing unit has a microprocessor such as a CPU and its peripheral circuits, and reads and executes a program stored in a memory or a storage unit in the peripheral circuit, thereby cooperating the hardware and the program. Each function is realized by working. The storage unit includes a storage device such as a memory and a hard disk, and stores various processing information used for processing in each functional unit.

The quality information acquisition unit 11 has a function of acquiring transmission quality information indicating the transmission status of the media packet transmitted from the transmission-side application terminal from the packet reception unit 21 of the application terminal 2, and for the media received by the reception-side application terminal. It has a function of acquiring reception quality information indicating the reception status of packets from the packet receiving unit 21 of the application terminal 2.

The quality analysis unit 12 has a function of calculating, as an invalid frame rate, the ratio of invalid frames affected by the loss of media packets among the frames transmitted and received by media packets based on the transmission quality information and the reception quality information. is doing.
Functional means constituting the quality analysis unit 12 include an invalid packet determination unit 12A, a target frame identification unit 12B, an invalid frame determination unit 12C, an equivalent invalid frame selection unit 12D, and an equivalent invalid frame rate calculation unit 12E.

The invalid packet determination unit 12A has a function of determining a lost packet among media packets as an invalid packet based on the transmission quality information and the reception quality information.
The target frame identification unit 12B has a function of identifying a frame corresponding to an invalid packet as a target frame based on the transmission quality information.
The invalid frame determination unit 12C has a function of determining whether or not each frame is an invalid frame based on the frame configuration used for encoding the media and the target frame.

The equivalent invalid frame selection unit 12D has a function of selecting, as an equivalent invalid frame, an invalid frame exceeding a predetermined weight frame number 18 among one or more consecutive invalid frames for the invalid frame determined by the invalid frame determination unit 12C. is doing. The number of weighted frames 18 is a value indicating human perceptual characteristics, is defined by the number of consecutive invalid frames, and is registered in advance in the storage unit.
The equivalent invalid frame rate calculation means 12E has a function of calculating a ratio of the number of equivalent invalid frames to the total number of frames transmitted by the media packet as an equivalent invalid frame rate.

  The user experience quality estimation unit 13 derives a user experience quality value corresponding to the equivalent invalid frame rate calculated by the quality analysis unit from the quality estimation model 14 indicating the relationship between the equivalent invalid frame rate and the user experience quality value, It has a function of outputting as an estimated quality value 15 of the application.

The estimation result notifying unit 16 uses the control packet to the user experience quality estimation device (not shown) and the quality management server (not shown) connected to the opposite application terminal via the packet network 5. It has a function of notifying the estimation result information including quality factor information such as the quality estimation value 15 obtained by the apparatus and the transmission / reception quality information used for calculating the user experience quality value periodically or as necessary. Yes.
The quality information notification unit 17 has a function of notifying the reception side application terminal of the transmission quality information obtained by the packet transmission unit 22 of the application terminal 2.

[User experience quality estimation processing]
Next, the operation of the user experience quality estimation apparatus according to the first embodiment of the present invention will be described.
First, the user experience quality estimation operation of the user experience quality estimation apparatus according to the first embodiment of the present invention will be described with reference to FIG. FIG. 2 is a flowchart showing a user experience quality estimation process of the user experience quality estimation apparatus according to the first embodiment of the present invention.

  Here, when the application terminal 2 is the receiving side and the user experience quality value regarding the media packet of the video media transmitted from the opposite application terminal (not shown) on the transmitting side and received via the packet network 5 is estimated. Will be described as an example. At this time, when transmission quality information obtained by the packet transmission unit of the opposite application terminal is notified to the application terminal 2 by a control packet from a user experience quality estimation device (not shown) connected to the opposite application terminal Will be described as an example.

  The quality information acquisition unit 11 of the user experience quality estimation apparatus 1 receives the transmission quality information transmitted by the control packet such as RTCP XR (RTP Control Protocol) from the opposite application terminal side by the packet reception unit 21 of the application terminal 2. The acquired one is acquired (step 100). The main quality information included in the transmission quality information includes a sequence number indicating a transmission order, a video frame type, a video frame number, a transmission packet number, and a transmission frame number for each transmission media packet.

  Further, the quality information acquisition unit 11 acquires the reception quality information obtained from the media packet from the opposite application terminal by the packet reception unit 21 of the application terminal 2 (step 101). The main quality information included in the reception quality information includes a sequence number indicating the transmission order and a delay fluctuation time indicating an arrival time difference from the immediately preceding media packet for each received media packet.

  Next, the quality analysis unit 12 determines an invalid packet for each media packet whose transmission is confirmed by the transmission quality information by the invalid packet determination unit 12A (step 102). The invalid packet is a packet that cannot be reproduced as media by the receiving-side real-time application among the media packets transmitted from the transmitting-side real-time application. Specifically, a packet lost on a transmission path connecting the transmission side application terminal and the reception side application terminal via the packet network 5, a late arrival packet whose arrival interval could not be adjusted by the fluctuation absorbing buffer of the reception side application, Furthermore, a packet in which the arrival order of media packets is reversed in a real-time application in which the arrival order of media packets is defined is targeted.

  Next, the quality analysis unit 12 refers to the transmission quality information by the target frame identification unit 12B, acquires the frame type and frame number of the media packet determined to be invalid by the invalid packet determination unit 12A, and A target frame including a packet is identified (step 103).

  Subsequently, the quality analysis unit 12 refers to the transmission quality information and the reception quality information by the invalid frame determination unit 12C, and becomes an evaluation target with the frame type and frame number of the target frame identified by the target frame identification unit 12B. Based on the frame configuration used for encoding the media in the real-time application, an invalid frame that cannot be decoded, which is affected by the loss of the media packet, that is, the invalid packet is determined among the frames (step 104).

  For example, in video media, there is a compression encoding method such as MPEG (Moving Picture Experts Group) that transfers subsequent frames with difference data from a reference frame, and such a frame configuration is preliminarily indicated by GOD (Group of Picture) configuration information. It is prescribed. When such a compression coding method is used, even if the difference data is normally received, if the reference frame becomes an invalid frame, the subsequent frame cannot be decoded, and as a result, the subsequent frame also becomes an invalid frame.

Thereafter, the quality analysis unit 12 selects an equivalent invalid frame by weighting each invalid frame with the number of weighted frames 18 by the equivalent invalid frame selecting means 12D. FIG. 3 is an explanatory diagram illustrating an equivalent frame selection process and an equivalent invalid frame rate calculation process of the user experience quality estimation apparatus according to the first embodiment of the present invention.
The equivalent invalid frame selection means 12D first confirms the position of the invalid frame based on the frame number of the invalid frame in each frame and identifies one or more consecutive invalid frames as one unit (step 105). In the example of FIG. 3, unit A is composed of one invalid frame, unit B is composed of three consecutive invalid frames, and unit C is composed of two consecutive invalid frames.

  Next, the equivalent invalid frame selection means 12D calculates the equivalent invalid frame number by subtracting the weighted frame number from the invalid frame number included in each unit (step 106), and the obtained equivalent invalid frame number is negative. In this case (step 107: YES), the number of equivalent invalid frames is set to “0” (step 108).

At this time, the number of weighted frames 18 is a value indicating sensitivity to deterioration of media among human perceptual characteristics, and is defined by the number of consecutive invalid frames. Therefore, by performing the above subtraction, an invalid frame that can be recognized as degraded by the user in each unit, that is, an invalid frame exceeding the weighted frame number 18 among one or more consecutive invalid frames is selected as an equivalent invalid frame. Can do.
In FIG. 3, when the number of weighted frames 18 is “1”, the number of invalid frames in unit A is “1”, so the number of equivalent invalid frames in unit A is “0”. Similarly, the number of equivalent invalid frames of units B and C are “2” and “1”, respectively.

  In this way, after selecting the invalid equivalent frame and calculating the number of equivalent invalid frames for each unit, the quality analysis unit 12 adds the number of equivalent invalid frames for each unit by the equivalent invalid frame rate calculation means 12E. The total number of equivalent invalid frames is obtained (step 109), and this is divided by all the frames indicated by the transmission quality information. Thereby, the ratio of the equivalent invalid frame determined by the equivalent invalid frame selection means 12D among all the frames is calculated as the equivalent invalid frame rate (step 110). In the example of FIG. 3, since the total number of equivalent invalid frames is “3” and the total number of frames is “12”, the equivalent invalid frame rate is “0.25”.

  Next, the user experience quality estimation unit 13 obtains a quality analysis unit from the quality estimation model 14 indicating the relationship between the invalid frame rate and the user experience quality value prepared in advance and stored in the storage unit (not shown). A user experience quality value corresponding to the invalid frame rate calculated in step 12 is derived (step 111), and is output to the storage unit as a quality estimate value 15 of the user experience quality of the real-time application.

  The creation of a quality estimation model represents the relationship between the intermediate parameters and the user experience quality consisting of MOS values (Mean Opinion Score average opinion scores), which are subjective evaluation values obtained by actually performing opinion evaluation, using a regression curve or a table. Model it. Conventionally, the invalid packet rate of lost packets lost on the packet network 5 is used as an intermediate parameter, and the MOS value is calculated from the invalid packet rate using an estimation model as shown in FIG. 4 showing the relationship between the invalid packet rate and the MOS value. (See Patent Document 1 and the like). Such an estimation model can guarantee a certain degree of estimation accuracy, but is not sufficient when high estimation accuracy is required because media packets and frames do not correspond one-to-one.

  Also, FIG. 5 is an estimation model showing the relationship between the invalid frame rate and the MOS value. Although there is less variation compared to FIG. 4, human perceptual characteristics are not taken into account, and in an actual real-time application, The estimation result estimated from the quality degradation state of FIG. 5 includes an error from the subjective quality evaluation of the user with respect to the media.

In the present embodiment, an equivalent invalid frame rate considering sensitivity to media degradation among human perceptual characteristics is obtained and used as an intermediate parameter. FIG. 6 is a quality estimation model showing the relationship between the equivalent invalid frame rate and the MOS value. This equivalent invalid frame rate is derived from the quality information of the same media packet as that obtained when the invalid packet rate in FIG. 4 and the invalid frame rate in FIG. 5 are obtained. It can be seen that there are few.
As the quality estimation model 14 that is actually used by the user experience quality estimation unit 13, the regression curve of FIG. 5 may be used, or an estimation model obtained by tabulating the regression curve may be used.

  Specifically, in FIG. 6, for example, when the minimum input unit of the input parameter (specifically, the minimum input unit of the invalid frame rate) is 0.01, the output value is averaged in 0.01 increments. To create a table. That is, by subtracting the smallest step in parallel with the vertical axis of FIG. 6 and averaging the MOS values within the range, a table of the relationship between the invalid frame rate and the MOS is created for every smallest step from 0. However, when the number of evaluation samples is small as shown in FIG. 6, there may be a section where there is no data in the minimum time. In that case, complementation may be performed using a section in which data exists before and after. This complementation may be linear or regression may be used. In order to minimize such a phenomenon, it is desirable to acquire a large number of data samples when creating a model.

Thereafter, the estimation result notifying unit 16 reads the quality estimation value 15 estimated by the user experience quality estimation unit 13 from the storage unit, stores it in a control packet such as RTCP XR, and the counter application via the packet network 5. It transmits with respect to the user experience quality estimation apparatus (not shown) and quality control server (not shown) connected to the terminal (step 112), and complete | finishes a series of user experience quality estimation processes.
At this time, the estimation result notification unit 16 may notify the result of statistical processing for obtaining the average value of the quality estimation value 15 and the variance indicating the variation at regular intervals.

As described above, in the present embodiment, the quality information acquisition unit 11 receives the transmission quality information indicating the transmission status of the media packet transmitted from the transmission side application terminal, and the reception of the media packet received by the reception side application terminal. The reception quality information indicating the situation is acquired, and the quality analysis unit 12 determines an invalid frame affected by the loss of the media packet from among the frames transmitted and received by the media packet based on the transmission quality information and the reception quality information. The ratio of the equivalent invalid frames selected by weighting these invalid frames with a predetermined number of weighted frames is calculated as an equivalent invalid frame rate, and the user experience quality estimation unit 13 calculates the equivalent invalid frame rate and the user experience. Calculated by the quality analysis unit 12 from a quality estimation model indicating the relationship with the quality value Deriving a QoE value corresponding to the equivalent invalid frame rate, and outputs as the estimated value of the QoE applications.

  Therefore, when estimating the user experience quality, the frame formed by each media packet is considered in comparison with the case where the user experience quality value is estimated from the packet loss rate or invalid packet rate obtained as objective quality information. In addition to estimating the user experience quality value, it is possible to estimate the user experience quality value in consideration of the sensitivity to degradation of media among human perceptual characteristics, and to estimate the user experience quality value of a real-time application with high accuracy .

[Second Embodiment]
Next, a user experience quality estimation apparatus according to the second embodiment of the present invention will be described with reference to FIG. FIG. 7 is a flowchart showing invalid packet determination processing of the user experience quality estimation device according to the second exemplary embodiment of the present invention.

In the present embodiment, a specific example of invalid packet determination processing in the invalid packet determination means 12A of the first embodiment described above will be described.
First, the invalid packet determination unit 12A arbitrarily selects an undetermined transmission media packet from the transmission quality information (step 120), and confirms whether or not a media packet having the same sequence number is received with the reception quality information ( Step 121). Here, when the media packet having the same sequence number is not received (step 121: NO), the media packet is an invalid packet because it can be determined that the media packet is a lost packet lost in the middle transmission path. (Step 125).

  On the other hand, when the same media packet as the sequence number has been received (step 121: YES), the delay fluctuation time of the transmission media packet is confirmed with the reception quality information, and the allowable delay time possessed by the real-time application Compare (step 122). Here, if the delay fluctuation time exceeds the allowable delay time (step 122: NO), it can be determined that the media packet is a late arrival packet that arrives with a delay in the transmission path or terminal. From this, it is determined that the packet is an invalid packet (step 125).

  On the other hand, when the delay fluctuation time is equal to or shorter than the allowable delay time (step 122: YES), only when the real-time application considers the arrival order reversed packet as a loss (step 123: YES), and the media packet It is confirmed from the reception quality information whether the order of arrival and the sequence number match. Here, when the packet cannot be confirmed and the arrival order is reversed (step 124: YES), it is determined that the packet is an invalid packet (step 125).

When the arrival order is not reversed (step 124: NO), or when the real-time application does not regard the arrival order reversed packet as a loss (step 123: NO), it is determined that the packet is not an invalid packet. (Step 126).
In this way, after the invalid packet determination is performed on the selected transmission media packet, if there is a transmission packet that has not been determined to be invalid (step 127: YES), the process returns to step 120 described above. The invalid packet determination for the remaining received media packets is repeated. On the other hand, when the invalid packet determination is completed for all the transmission media packets (step 127: NO), the series of invalid packet determination processing ends.

  As described above, in the present embodiment, the invalid packet determination unit 12A determines a media packet that has not been confirmed to be received in the reception quality information from among the media packets that have been confirmed to be transmitted in the transmission quality information, and the reception quality information. Media packets whose arrival time difference from the previous media packet obtained from the arrival time of each media packet exceeds the allowable packet delay time of the application are judged as invalid packets. Packets lost due to arrival can also be determined as invalid packets, and packets that cannot be reproduced as media by the real-time application on the receiving side can be accurately determined.

  Further, since the invalid packet determining means determines that the media packet whose reception order is inconsistent with the transmission order of the media packet confirmed to be received with the reception quality information is an invalid packet, the real-time application receives the arrival order. When a reverse packet is regarded as a loss, a packet that cannot be reproduced as media by a real-time application on the receiving side can be more accurately determined.

[Third Embodiment]
Next, a user experience quality estimation apparatus according to the third embodiment of the present invention will be described with reference to FIG. FIG. 8 is a flowchart showing an invalid frame determination process of the user experience quality estimation apparatus according to the third embodiment of the present invention.

In the present embodiment, a specific example of the invalid frame determination process in the invalid frame determination means 12C of the first embodiment will be described.
First, the invalid frame determination unit 12C determines that a frame identified as the target frame by the target frame identification unit 12B among the frames whose transmission is confirmed by the transmission quality information is identified as the target frame because the target frame identification unit 12B includes the invalid packet (step 130).

  Next, with reference to the reception quality information, an invalid frame undetermined frame is selected according to the order of arrival at the receiving side (step 131), and the frame indicating the frame configuration used for media encoding in the real-time application By referring to the configuration information and the reception quality information, it is confirmed whether an invalid frame is included in another frame used for decoding the frame (step 132). The frame configuration information may be set in advance in the storage unit or may be acquired as transmission / reception quality information.

  If an invalid frame is included (step 132: YES), it is determined that the frame is an invalid frame because the frame cannot be decoded normally (step 133). On the other hand, if the invalid frame is not included (step 132: NO), since the frame can be normally decoded, it is determined that the frame is not an invalid frame (step 134).

  FIG. 9 shows an example of invalid frame determination processing. Here, the video media is compressed and encoded with a GOD configuration of “IBB” and divided into frames. The frame type I frame is a reference frame, and the original frame can be decoded only by the I frame. The frame type B frame is composed of difference data from the I frame, and the I frame is required for decoding.

As shown in FIG. 9, when the frames “1 to 9” are divided into media packets according to the GOD configuration of “IBB” and transmitted and received at the receiving side, the frames “2, 4, 7, 8” ”Is an invalid packet.
In this case, first, a target frame including an invalid packet is determined to be an invalid frame, and further, a frame used at the time of decoding is also determined to be an invalid frame.

  Specifically, the frame “4” is used for decoding the frame “5, 6”, but the frame “4” is an invalid frame, and therefore, these frames “5, 6” are also determined to be invalid frames. Similarly, frame “7” is used for decoding frame “9”, but since frame “7” is an invalid frame, frame “9” is also determined to be an invalid frame. Note that frame “1” is used for decoding frame “3”, but since frame “1” is not an invalid frame, frame “3” is determined not to be an invalid frame.

  In this manner, after the invalid frame determination is performed on the selected frame, if an invalid frame undecided frame remains (step 135: YES), the process returns to the above-described step 131 and the remaining frames are processed. Repeat invalid frame determination. On the other hand, when the invalid packet determination is completed for all transmission media packets (step 135: NO), the series of invalid frame determination processing is terminated.

  As described above, in this embodiment, the invalid frame determination unit 12C determines that the target frame among the frames is an invalid frame, and uses it for decoding the frame according to the frame type sequence defined in the frame configuration. When an invalid frame is included in the frame, it is determined that the frame is an invalid frame. An invalid frame can be accurately determined.

[Extension of each embodiment]
In each of the above embodiments, the application terminal 2 is the receiving side, and the user experience quality relating to the media packet of the video media transmitted from the opposite application terminal (not shown) on the transmitting side and received via the packet network 5 Although the case where the value is estimated has been described as an example, the present invention is not limited to this. That is, it is possible to estimate the user experience quality value related to the media packet of the video media received via the packet network 5 at the opposite application terminal (not shown) as the reception side, where the application terminal 2 is the transmission side.

In this case, the quality information acquisition unit 11 of the user experience quality estimation apparatus 1 uses the transmission quality information obtained by the packet transmission unit 22 of the application terminal 2 as the transmission quality information described above, and receives the packet of the application terminal 2. The reception quality information from the opposite application terminal received by the unit 21 may be used as the reception quality information described above.
At this time, when the similar user experience quality is also estimated in the opposite application terminal, instead of transmitting the transmission quality information obtained by the packet transmission unit 22 of the application terminal 2 by the quality information notification unit 17 described above, packet reception is performed. The reception quality information regarding the reception media packet obtained by the unit 21 may be transmitted to the opposite application terminal.

  The transmission / reception quality information may be directly exchanged between the transmission side and the reception side, but may be exchanged via a quality management server that manages the quality information. Further, the user experience quality estimation device 1 may receive quality information directly.

  In each of the above embodiments, the case where the user experience quality estimation device 1 is realized by a device separate from the application terminal 2 has been described as an example. However, the present invention is not limited to this and is parallel to a real-time application. Then, it may be realized by a quality estimation application executed on the application terminal 2. As a result, the hardware resources of the application terminal 2 can be shared and realized, and the user experience quality can be managed very easily.

It is a block diagram which shows the structure of the user experience quality estimation apparatus concerning the 1st Embodiment of this invention. It is a flowchart which shows the user experience quality estimation process of the user experience quality estimation apparatus concerning the 1st Embodiment of this invention. It is explanatory drawing which shows the equivalent frame selection process and equivalent invalid frame rate calculation process of the user experience quality estimation apparatus concerning the 1st Embodiment of this invention. It is a quality estimation model showing the relationship between the invalid packet rate and the MOS value. It is a quality estimation model showing the relationship between the invalid frame rate and the MOS value. It is a quality estimation model showing the relationship between the equivalent invalid frame rate and the MOS value. It is a flowchart which shows the invalid packet determination process of the user experience quality estimation apparatus concerning the 2nd Embodiment of this invention. It is a flowchart which shows the invalid frame determination process of the user experience quality estimation apparatus concerning the 3rd Embodiment of this invention. It is an example of an invalid frame determination process.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... User experience quality estimation apparatus, 11 ... Quality information acquisition part, 12 ... Quality analysis part, 12A ... Invalid packet determination means, 12B ... Object frame identification means, 12C ... Invalid frame determination means, 12D ... Equivalent invalid frame selection means, 12E: Equivalent invalid frame rate calculation means, 13 ... User experience quality estimation unit, 14 ... Quality estimation model, 15 ... Quality estimation value, 16 ... Estimation result notification unit, 17 ... Quality information notification unit, 2 ... Application terminal, 21 ... Packet receiver, 22 ... packet transmitter, 5 ... packet network.

Claims (11)

For real-time applications that use a media packet to transmit and receive each frame obtained by encoding media such as video and audio, which is used between transmitting and receiving application terminals connected via a packet network A user experience quality estimation device for estimating experience quality,
A quality information acquisition unit that acquires transmission quality information indicating a transmission status of a media packet transmitted from the transmission-side application terminal and reception quality information indicating a reception status of the media packet received by the reception-side application terminal. When,
Based on the transmission quality information and the reception quality information, an invalid frame that is affected by the loss of the media packet is determined among the frames transmitted and received by the media packet, and a predetermined number of weight frames are assigned to the invalid frame. A quality analysis unit that calculates a ratio of equivalent invalid frames selected by weighting in as an equivalent invalid frame rate;
A user experience quality value corresponding to the equivalent invalid frame rate calculated by the quality analysis unit is derived from a quality estimation model indicating a relationship between an equivalent invalid frame rate and a user experience quality value, and the user experience quality of the application is estimated. A user experience quality estimation device comprising: a user experience quality estimation unit that outputs the value as a value. In the user experience quality estimation apparatus according to claim 1,
The quality analysis unit
Invalid packet determination means for determining, as an invalid packet, a lost packet among the media packets based on the transmission quality information and the reception quality information;
Frame identifying means for identifying a frame corresponding to the invalid packet as a target frame based on the transmission quality information;
Invalid frame determination means for determining whether each frame is the invalid frame based on the frame configuration used for encoding the media and the target frame;
Equivalent invalid frame selection means for selecting, as an equivalent invalid frame, an invalid frame exceeding the weight frame number among one or more consecutive invalid frames;
Equivalent invalid frame rate calculation means for calculating a ratio of the number of equivalent invalid frames to the total number of frames transmitted in the media packet as the equivalent invalid frame rate. In the user experience quality estimation apparatus according to claim 2 ,
The invalid packet determination means includes an arrival interval of media packets that have not been confirmed to be received by the reception quality information among media packets that have been confirmed to be transmitted by the transmission quality information, and media packets of the reception quality information. A user experience quality estimation apparatus, wherein a media packet whose delay fluctuation time exceeds the allowable packet delay time of the application is determined as the invalid packet. In the user experience quality estimation apparatus according to claim 2 ,
The user experience quality estimation apparatus, wherein the invalid packet determination means determines a media packet in which a transmission order and a reception order of media packets confirmed to be received by the reception quality information are inconsistent as the invalid packet. In the user experience quality estimation apparatus according to claim 2 ,
The frame identification means obtains a frame number and a frame type of the target frame from header information of the invalid packet included in the transmission quality information,
The invalid frame determination means determines the target frame as an invalid frame among the frames, and includes an invalid frame in another frame used for decoding the frame according to a frame type sequence defined in the frame configuration. A user experience quality estimation device, wherein the frame is determined to be an invalid frame. In the user experience quality estimation apparatus according to claim 1,
The quality information acquisition unit acquires, as the transmission quality information, transmission quality information from the transmission side application terminal received by the packet reception unit of the reception side application terminal, and a packet reception unit of the reception side application terminal The user experience quality estimation device, wherein the reception quality information obtained from the media packet received in step (b) is acquired as the reception quality information. In the user experience quality estimation apparatus according to claim 1,
The quality information acquisition unit acquires, as the reception quality information, reception quality information from the reception side application terminal received by the packet reception unit of the transmission side application terminal, and a packet transmission unit of the transmission side application terminal A user experience quality estimation device, wherein the transmission quality information obtained from the media packet transmitted in (1) is acquired as the transmission quality information. In the user experience quality estimation apparatus according to claim 1,
The user experience quality estimation apparatus further comprising a quality information notification unit that notifies the reception side application terminal of the transmission quality information obtained by the packet transmission unit of the transmission side application terminal. In the user experience quality estimation apparatus according to claim 1,
The user experience quality estimation apparatus further comprising a quality information notification unit for notifying the reception side application terminal of the reception quality information obtained by the packet reception unit of the reception side application terminal. For real-time applications that use a media packet to transmit and receive each frame obtained by encoding media such as video and audio, which is used between transmitting and receiving application terminals connected via a packet network A user experience quality estimation method used in a user experience quality estimation apparatus for estimating experience quality,
Quality information acquisition step of acquiring transmission quality information indicating a transmission status of the media packet transmitted from the transmission side application terminal and reception quality information indicating a reception status of the media packet received by the reception side application terminal When,
Based on the transmission quality information and the reception quality information, an invalid frame that is affected by the loss of the media packet is determined among the frames transmitted and received by the media packet, and a predetermined number of weight frames are assigned to the invalid frame. A quality analysis step of calculating a ratio of equivalent invalid frames selected by weighting in as an equivalent invalid frame rate;
A user experience quality value corresponding to the equivalent invalid frame rate calculated in the quality analysis step is derived from a quality estimation model indicating a relationship between an equivalent invalid frame rate and a user experience quality value, and the user experience quality of the application is estimated. A user experience quality estimation method comprising: a user experience quality estimation step that outputs the value as a value. For real-time applications that use a media packet to transmit and receive each frame obtained by encoding media such as video and audio, which is used between transmitting and receiving application terminals connected via a packet network In the computer of the user experience quality estimation device that estimates the experience quality,
Quality information acquisition step of acquiring transmission quality information indicating a transmission status of the media packet transmitted from the transmission side application terminal and reception quality information indicating a reception status of the media packet received by the reception side application terminal When,
Based on the transmission quality information and the reception quality information, an invalid frame that is affected by the loss of the media packet is determined among the frames transmitted and received by the media packet, and a predetermined number of weight frames are assigned to the invalid frame. A quality analysis step of calculating a ratio of equivalent invalid frames selected by weighting in as an equivalent invalid frame rate;
A user experience quality value corresponding to the equivalent invalid frame rate calculated in the quality analysis step is derived from a quality estimation model indicating a relationship between an equivalent invalid frame rate and a user experience quality value, and the user experience quality of the application is estimated. A program for executing a user experience quality estimation step to output as a value.

Images