JP2007329778A - Apparatus, method and program for estimating user's sensory quality - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To highly accurately estimate a user's sensory quality value in consideration of the influence of invalid packets different in content class. <P>SOLUTION: An invalid packet decision part 12 decides a media packet which can not be normally received as an invalid packet. A quality analysis part 13 analyzes a contents class on the basis of a packet transmission state of a media packet detected in each frame sort, decides continuous invalid frames influenced by the invalid packet out of respective frames on the basis of a decision standard corresponding to the contents class and calculates the rate of equivalent invalid frames selected by weighting these continuous invalid frames by the predetermined number of weighting frames. A user's sensory quality estimation part 14 leads out a user's sensory quality value corresponding to the equivalent invalid frame rate from a quality estimation model 15 corresponding to the contents class and outputs the user's sensory quality value as an estimation value of user's sensory quality. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a communication quality estimation technique, and more particularly to a technique for estimating user experience quality of a real-time application.

  In recent years, with the rapid development of packet communication technology, packet networks are being used in real-time applications that communicate media (contents) such as audio and video one-to-one or between multiple points. In the in-service quality management of the packet network, quality management for such an application is also required. Especially in real-time applications, quality management using user experience quality such as subjective or objective quality at the user level that actually uses the application is regarded as important, and technology that estimates user experience quality from network quality Have also been proposed (see, for example, Non-Patent Document 1).

  The inventors have also proposed a technique for estimating the subjective quality of a real-time application based on invalid packets (see, for example, Patent Document 1). In this technique, a lost packet that has not arrived at the receiving terminal among the test packets and a late arrival packet whose arrival interval between the previous arrival packets exceeds the allowable fluctuation absorption time of the real-time application are determined as invalid packets. The number of invalid packets is counted, and the ratio of the number of invalid packets to the total number of test packets transmitted from the transmission side is calculated as the invalid packet rate. The subjective quality evaluation value corresponding to the invalid packet rate is estimated with reference to the quality estimation model that shows the relationship.

  In contrast, the inventors have proposed a technique for estimating the subjective quality of a real-time application based on the degree of video quality degradation estimated from lost packets and the duration of video quality degradation (for example, patents). Reference 2 etc.). In this technique, for each video packet sent to a user terminal, the frame type and the frame generation rule at the time of high-efficiency compression encoding to which the packet belongs are embedded. At the user terminal, the degree of quality degradation within one video frame is estimated from the generation number of the lost packet, and the video quality degradation is determined from the frame type and the frame generation rule at the time of high-efficiency compression coding to which the lost packet belongs. The duration is estimated, and the subjective quality of the video decoded at the user terminal is estimated based on the estimated degree of video quality degradation and the duration of the video quality degradation. Thereby, the estimation accuracy was improved with a simple configuration.

Japanese Patent No. 3579334 JP 2006-033722 A ITU-T Recommendation G.107, "The E-Model, a computational model for use in transmission planning", May 2000

  However, in such a conventional technique, the difference in the effect on the user experience quality due to the difference in contents such as video and audio transmitted in a real-time application is not much considered, and high estimation accuracy is required. There was a problem that it was not enough if it was possible.

  In real-time applications, media signals such as highly efficient encoded video signals and audio signals are decomposed into frames, which are further converted into media packets for exchange. For this reason, when the media signal is decoded by the application terminal, a frame is assembled from the received media packets, and the media signal is decoded using the preceding and following frame information. Therefore, when packet loss occurs in the packet network, the quality deterioration of the media signal due to the loss of one packet may extend not only to the frame but to a plurality of frames.

The influence on the user experience quality due to the lost frame that could not be normally decoded due to such packet loss differs depending on the content of the content reproduced in the frame. This is because the human perception characteristic of whether or not to perceive a lost frame as media degradation depends on the content of the content. The effect of frame loss is small in a few video conferences.
FIG. 13 is an explanatory diagram showing a difference in video frame loss rate-MOS value characteristics between different contents. The video frame loss rate is a value indicating the ratio of lost frames to frames transmitted in a unit measurement time, and even if the video frame loss rate is the same, the quality experienced by the user tends to differ depending on the content content. I understand that.

  In the prior art, the difference in the effect on the user experience quality due to the difference in the content contents is not considered so much, and the effect on the user experience quality is the same for any content. Since the quality was estimated, high estimation accuracy could not be obtained.

  The present invention is for solving such problems, and a user who can estimate the user experience quality value of a real-time application with high accuracy in consideration of the influence of the difference in content contents on the user experience quality. It is an object of the present invention to provide a bodily sensation quality estimation apparatus, method, and program.

  In order to achieve such an object, a user experience quality estimation apparatus according to the present invention is a media such as video and audio between a transmission side and a reception side application terminal connected via a packet network by a real-time application. A user experience quality estimation device for estimating a user experience quality that a user feels about a medium received and reproduced by a reception side application terminal when each frame obtained by encoding is transmitted and received as a media packet, A quality information acquisition unit for acquiring transmission quality information indicating the transmission status of the media packet transmitted from the application terminal, and reception quality information indicating the reception status of the media packet received by the reception-side application terminal; and transmission quality Packet for media based on information and received quality information An invalid packet determination unit that determines a packet that could not be received normally as an invalid packet, and analyzes the media content class based on the packet transmission status of the media packet detected for each frame type of the media packet from the transmission quality information. Based on the determination criteria according to the content class, consecutive invalid frames affected by invalid packets are determined from each frame, and these consecutive invalid frames are selected by weighting with a predetermined number of weight frames. Calculated by the quality analysis unit from the quality analysis unit that calculates the equivalent invalid frame rate as the equivalent invalid frame rate and the quality estimation model that shows the relationship between the equivalent invalid frame rate and the user experience quality value according to the content class. Deriving the user experience quality value corresponding to the equivalent invalid frame rate, And a QoE estimation unit for outputting as an estimate of the QoE of publication.

  At this time, the quality analysis unit determines that the normal frame is an invalid frame when the number of normal frames sandwiched between invalid frames is equal to or less than a predetermined number of invalid frames, and the number of consecutive invalid frames according to the content class Equivalent invalid frame selection means for selecting the number of invalid frames exceeding the number of weighted frames according to the content class among consecutive invalid frames as the number of equivalent invalid frames, and the number of equivalent invalid frames relative to the number of frames transmitted in the media packet You may provide the equivalent invalid frame rate calculation means which calculates a ratio as an equivalent invalid frame rate.

  In addition, the invalid packet determination unit uses a packet for media whose transmission quality has been confirmed to be received, a packet whose reception quality information has not been confirmed for reception, and a medium packet whose reception has been confirmed to have reception quality information for the immediately preceding media. A packet whose delay fluctuation time indicating the arrival interval with the packet exceeds the allowable packet delay time of the application, and a packet whose transmission order and reception order are inconsistent among media packets that have been confirmed to be received by the reception quality information, You may make it determine with an invalid packet.

  Also, the quality analysis unit determines that the target frame corresponding to the invalid packet among the frames is an invalid frame, and decodes the frame according to the frame type sequence defined in the frame configuration used for encoding the media. When an invalid frame is included in other frames to be used, the frame may be determined as an invalid frame.

  Further, the quality information acquisition unit acquires the quality information from the transmission side application terminal received by the packet reception unit of the reception side application terminal as transmission quality information, and is received by the packet reception unit of the reception side application terminal. Quality information obtained from the media packet may be acquired as reception quality information.

  In addition, the quality information acquisition unit acquires the quality information from the reception side application terminal received by the packet reception unit of the transmission side application terminal as reception quality information and is transmitted by the packet transmission unit of the transmission side application terminal. Quality information obtained from the media packet may be acquired as transmission quality information.

  Moreover, you may further provide the quality information notification part which notifies the transmission quality information obtained in the packet transmission part of the transmission side application terminal to a reception side application terminal.

  Moreover, you may further provide the quality information notification part which notifies the receiving quality information obtained in the packet receiving part of the receiving side application terminal to a transmitting side application terminal.

  In addition, the user experience quality estimation method according to the present invention is obtained by encoding media such as video and audio by a real-time application between a transmission side and a reception side application terminal connected via a packet network. A user experience quality estimation method used in a user experience quality estimation device for estimating a user experience quality that a user feels about a media received and played by a receiving application terminal when a frame is transmitted / received as a media packet, comprising: Quality information for acquiring transmission-side quality information indicating the transmission status of media packets transmitted from the transmission-side application terminal and reception quality information indicating the reception status of media packets received by the reception-side application terminal by the acquisition unit By the acquisition step and invalid packet determination unit An invalid packet determination step for determining a packet that could not be normally received among media packets based on the transmission quality information and the reception quality information, and a frame type of the media packet from the transmission quality information by the quality analysis unit The media content class is analyzed based on the packet transmission status of the media packet detected for each frame, and consecutive invalid frames that are affected by invalid packets are determined based on the judgment criteria according to the content class. The quality analysis step for calculating the ratio of equivalent invalid frames selected by weighting these consecutive invalid frames with a predetermined number of weight frames as the equivalent invalid frame rate, and the user experience quality estimation unit according to the content class Equivalent invalid frame rate and user body A user experience quality estimation step of deriving a user experience quality value corresponding to the equivalent invalid frame rate calculated by the quality analysis unit from a quality estimation model indicating a relationship with the quality value, and outputting it as an estimate of the user experience quality of the application And.

  Further, the program according to the present invention uses each frame obtained by encoding media such as video and audio by a real-time application between the transmission side and reception side application terminals connected via the packet network. When the packet is transmitted / received in a packet, the quality information acquisition unit transmits the user experience quality estimation device that estimates the user experience quality that the user feels about the media received and played by the reception side application terminal. The quality information acquisition step for acquiring the transmission side quality information indicating the transmission status of the media packet and the reception quality information indicating the reception status of the media packet received by the reception side application terminal, and the transmission quality by the invalid packet determination unit Information and received quality information The invalid packet determination step of determining a packet that could not be normally received among the media packets as an invalid packet, and the packet transmission status of the media packet detected by the quality analysis unit for each frame type of the media packet from the transmission quality information The media content class is analyzed based on the criteria, and based on the determination criteria according to the content class, consecutive invalid frames affected by invalid packets are determined among the frames. A quality analysis step for calculating a ratio of equivalent invalid frames selected by weighting with the number of weighted frames as an equivalent invalid frame rate, and an equivalent invalid frame rate and a user experience quality value according to a content class by a user experience quality estimation unit Quality estimation model showing the relationship between Derives the QoE value corresponding to the equivalent invalid frame rate calculated by the quality analysis unit, to execute a QoE estimation step of outputting as an estimate of the QoE applications.

  According to the present invention, the invalid packet determination unit determines that a packet that could not be normally received out of the media packets based on the transmission quality information and the reception quality information as an invalid packet, and the quality analysis unit transmits the transmission quality information. The content class of the media is analyzed based on the packet transmission status of the media packet detected for each frame type of the media packet, and is affected by the invalid packet of each frame based on the determination criteria according to the content class. The consecutive invalid frames are determined, and the ratio of equivalent invalid frames selected by weighting these consecutive invalid frames with a predetermined number of weighted frames is calculated as an equivalent invalid frame rate. Equivalent invalid frame rate and user body according to class From the quality estimation model showing the relationship between the quality value, the QoE value is derived which corresponds to the equivalent invalid frame rate calculated by the quality analysis unit, is output as an estimate of the QoE applications.

As a result, continuous invalid frames are determined based on the determination criteria corresponding to the content class of the content transmitted by the real-time application, the equivalent invalid frame rate is calculated, and the equivalent estimation is performed using the quality estimation model corresponding to the content class. A user experience quality value corresponding to the invalid frame rate is estimated.
Therefore, it is possible to accurately calculate the loss estimation scale used for the determination of the invalid frame and the estimation of the user experience quality in consideration of the influence of the content content on the user experience quality, and the estimation accuracy of the user experience quality is improved. Can be improved.

Next, embodiments of the present invention will be described with reference to the drawings.
[First Embodiment]
First, a user experience quality estimation apparatus according to a first embodiment of the present invention will be described with reference to FIG. FIG. 1 is a block diagram showing the configuration of the user experience quality estimation apparatus according to the first embodiment of the present invention.

The user experience quality estimation apparatus 1 includes an information processing apparatus having a communication function, and is connected to the application terminal 2 and the packet network 5 to estimate the user experience quality of a real-time application executed on the application terminal 2.
The user experience quality estimation device 1 includes, as main functional units, a quality information acquisition unit 11, an invalid packet determination unit 12, a quality analysis unit 13, a user experience quality estimation unit 14, an estimation result notification unit 17, and a quality information notification. A portion 18 is provided.

  In this embodiment, the quality information acquisition unit 11 transmits the transmission side quality information indicating the transmission status of the media packet used for communication of the media data transmitted from the transmission side application terminal, and the medium received by the reception side application terminal. The reception quality information indicating the reception status of the packet is acquired, and the invalid packet determination unit 12 determines, as an invalid packet, a packet that could not be normally received among the media packets based on the transmission quality information and the reception quality information. The quality analysis unit 13 analyzes the content class of the media packet based on the packet transmission status of the media packet detected for each frame type of the media packet from the transmission quality information, and based on the determination criterion according to the content class, Invalid among frames sent and received in media packets The user determines the consecutive invalid frames affected by the packet, calculates the equivalent invalid frame rate as the equivalent invalid frame rate by calculating the ratio of equivalent invalid frames selected by weighting these consecutive invalid frames with a predetermined number of frames. The estimation unit 14 calculates the user experience quality value corresponding to the equivalent invalid frame rate calculated by the quality analysis unit from the quality estimation model 15 indicating the relationship between the equivalent invalid frame rate and the user experience quality value according to the content class. Derived and output as an estimated value of the user experience quality of the application.

[User experience quality estimation device]
Next, the user experience quality estimation apparatus according to the first embodiment of the present invention will be described in detail with reference to FIG.
Each functional unit of the user experience quality estimation apparatus 1 includes a dedicated circuit unit, an arithmetic processing unit, and a storage unit. In particular, the arithmetic processing unit (computer) includes a microprocessor such as a CPU and its peripheral circuits, and reads and executes a program stored in a memory or a storage unit in the peripheral circuit, whereby the hardware and program To realize each functional unit. The storage unit includes a storage device such as a memory and a hard disk, and stores various processing information used for processing in each functional unit.

  The quality information acquisition unit 11 has a function of acquiring transmission-side quality information indicating the transmission status of media packets transmitted from the transmission-side application terminal from the packet reception unit 21 of the application terminal 2, and media received by the reception-side application terminal. A reception quality information indicating the reception status of the received packets from the packet receiving unit 21 of the application terminal 2.

  The invalid packet determination unit 12 has a function of determining, as an invalid packet, a packet that could not be normally received among media packets based on the transmission quality information and the reception quality information acquired by the quality information acquisition unit 11, and the invalid packet A function of classifying the invalid packet into one of different classes according to the packet generation process.

  The invalid packet is a packet that cannot be normally received by the receiving real-time application and cannot be reproduced as media among media packets transmitted from the transmitting real-time application. Specifically, a packet lost on a transmission path connecting the transmission side application terminal and the reception side application terminal via the packet network 5, a late arrival packet whose arrival interval could not be adjusted by the fluctuation absorbing buffer of the reception side application, Furthermore, a packet in which the arrival order of media packets is reversed in a real-time application in which the arrival order of media packets is defined is targeted.

The quality analysis unit 13 analyzes the content class of the media transmitted by the media packet from the transmission quality information acquired by the quality information acquisition unit 11, and determines the influence of the invalid packet based on the determination criterion according to the content class. It has a function of calculating the ratio of equivalent invalid frames selected from received invalid frames in consideration of human perceptual characteristics as an invalid frame rate.
Main functional means constituting the quality analysis unit 13 include content class analysis means 13A, invalid frame determination means 13B, equivalent invalid frame selection means 13C, and equivalent invalid frame rate calculation means 13D.

The content class analyzing means 13A has a function of analyzing the media content class based on the packet transmission status of the media packet detected for each frame type of the media packet from the transmission quality information.
The invalid frame determination means 13B has a function of determining a target frame corresponding to an invalid packet determined by the invalid packet determination unit 12 among the frames based on the transmission quality information, and a frame used for media encoding. According to the arrangement of frame types defined in the configuration, when an invalid frame is included in another frame used for decoding the frame, the frame has a function of determining the frame as an invalid frame.

  The equivalent invalid frame selection unit 13C determines that the invalid frame determined by the invalid frame determination unit 13B is an invalid frame when the number of normal frames sandwiched between the invalid frames is equal to or less than a predetermined number of invalid frame intervals. And a function for determining invalid frames that are continuous for a predetermined number of consecutive invalid frames as consecutive invalid frames among these invalid frames, and an invalid frame that is consecutive for a predetermined number of weighted frames among these consecutive invalid frames are equivalent. It has a function of selecting as an invalid frame.

The number of invalid frame intervals, the number of consecutive invalid frames, and the number of weighted frames are registered in advance in the storage unit as determination criteria for each content class. The equivalent invalid frame selection unit 13C selects an equivalent invalid frame based on the number of invalid frame intervals, the number of consecutive invalid frames, and the number of weighted frames specified by the determination criteria corresponding to the content class analyzed by the content class analysis unit 13A. To do.
The equivalent invalid frame rate calculation means 13D has a function of calculating the ratio of the number of equivalent invalid frames to the number of frames transmitted in the media packet as an equivalent invalid frame rate.

  The user experience quality estimation unit 14 derives a user experience quality value corresponding to the equivalent invalid frame rate calculated by the quality analysis unit 13 from the quality estimation model 15 indicating the relationship between the equivalent invalid frame rate and the user experience quality value. , And has a function of outputting the estimated quality value 16 of the application.

The estimation result notifying unit 17 uses the control packet to the user experience quality estimation device (not shown) and the quality management server (not shown) connected to the opposite application terminal via the packet network 5. It has a function of notifying the estimation result information including quality factor information such as transmission quality information used for calculation of the quality estimation value 16 obtained by the apparatus and the user experience quality value periodically or as necessary. Yes.
The quality information notification unit 18 has a function of notifying the transmission application terminal of the transmission quality information obtained by the packet transmission unit 22 of the application terminal 2.

[Content class analysis processing]
Next, with reference to FIGS. 2 to 5, the content class analysis processing in the user experience quality estimation apparatus according to the first embodiment of the present invention will be described in detail. FIG. 2 is an explanatory diagram showing content class determination criteria used in content analysis processing. FIG. 3 is an explanatory diagram showing another content class determination criterion used in content analysis processing. FIG. 4 is an explanatory diagram showing content class analysis processing. FIG. 5 is an explanatory diagram showing another content class analysis process.

The quality analysis unit 13 of the user experience quality estimation apparatus 1 uses the content analysis unit 13A to analyze the media content class based on the packet transmission status of the media packet detected for each frame type of the media packet from the transmission quality information. .
When a media is encoded into each frame, a real-time application efficiently transmits the media with a desired quality by adaptively changing the configuration of the frame type according to the characteristics of the media signal. Since the contents and the characteristics of the media signal are related in this way, the content class of the media can be analyzed by detecting the packet transmission status for each frame type transmitted by the media packet.

  The packet transmission status may be a packet transmission amount including the number of media packets and a packet occupancy rate detected for each frame type, or a variation amount of these packet transmission amounts. The fluctuation amount of the packet transmission amount is a value indicating a difference between the maximum value and the minimum value in each packet transmission state detected for each unit measurement period.

In the content class determination standard of FIG. 2, the number of media packets for each frame type (I, B, P) of video media (GOP (Group of Picture) configuration of MPEG (Moving Picture Exparts Group) encoding method) is taken as an example ( The content class is registered as a set for the packet occupancy rate), and the quality analysis unit 13 searches the content class determination criterion for the content class corresponding to the packet transmission status detected from the transmission quality information, and Determine the media content class.
In the content class determination criteria of FIG. 3, the content class is registered as a set for the amount of change in the number of media packets for each frame type (I, B, P) of the video media. The media content class is determined.

  As for the packet transmission status detection method, as shown in FIG. 4, a method of detecting the packet transmission status continuously for every unit measurement period for all media packets, and a unit measurement period as shown in FIG. There is a method of detecting the packet transmission status intermittently at every measurement cycle. In either method, the result of statistical processing of the packet transmission status obtained for each unit measurement period may be used for determination of the content class.

  In the present embodiment, the content analysis unit 13A of the quality analysis unit 13 analyzes the media content class based on the packet transmission status of the media packet detected for each frame type of the media packet from the transmission quality information. Therefore, it is possible to accurately analyze the content class with extremely simple processing.

[Equivalent invalid frame selection processing]
Next, an equivalent invalid frame selection process in the user experience quality estimation apparatus 1 according to the first embodiment of the present invention will be described in detail with reference to FIG. FIG. 6 is an explanatory diagram showing equivalent invalid frame selection processing.

  The quality analysis unit 13 of the user experience quality estimation apparatus 1 considers human perceptual characteristics from the invalid frame affected by the invalid packet based on the determination standard corresponding to the content class by the equivalent invalid frame selection unit 13C. The equivalent invalid frame is selected, and the equivalent invalid frame rate calculation means 13D calculates the ratio of the equivalent invalid frame to the frame transmitted by the media packet as the equivalent invalid frame rate.

  When selecting an equivalent invalid frame, it is first determined whether a normal frame sandwiched between invalid frames is perceived as degraded based on human perceptual characteristics. Human perception characteristics include a characteristic of perceiving a normal frame sandwiched between invalid frames as degradation when an invalid frame is intermittently generated. For this reason, when the number of normal frames is defined by the number of invalid frame intervals, and the number of normal frames sandwiched between invalid frames determined by the invalid frame determination means 13B is equal to or less than the number of invalid frame intervals. The normal frame is determined to be an invalid frame.

  Next, it is determined whether the invalid frame is perceived as degraded based on human perception characteristics. Human perception has a characteristic that perceives degradation only after a certain number of invalid frames continue. For this reason, the number of invalid frames perceived as degradation is defined as the number of consecutive invalid frames, and invalid frames that are consecutive in the number of invalid invalid frames are determined as consecutive invalid frames.

  Finally, it is determined whether the continuous invalid frame is perceived as a deteriorated frame based on human perception characteristics. The human perceptual characteristic has a characteristic that a certain amount of invalid frames are required before it is perceived as degradation among consecutive invalid frames. For this reason, the number of invalid frames required to perceive deterioration is defined as the number of weighted frames, and the invalid frames that continue from the weight invalid frame number among consecutive invalid frames are determined as equivalent invalid frames.

  FIG. 6 shows an example of selection of equivalent invalid frames for the frames F (0) to F (19). Of these, frames F (2), F (7), F (10), F (12) It is assumed that .about.F (16) is determined to be an invalid frame by the invalid frame determination means 13B. Also, it is assumed that the number of invalid frame intervals “2”, the number of consecutive invalid frames “5”, and the number of weighted frames “4” are defined in the determination criteria of the content class analyzed by the content class analyzing unit 13A. .

  First, the invalid frame F (2) is followed by at least three normal frames F (3) to F (5). The invalid frame interval “3” at this time is the number of invalid frame intervals “ Therefore, the normal frames F (3) and F (4) are not determined to be invalid frames. Accordingly, the continuous invalid frame length related to the invalid frame F (2) is “1”, which is less than the number of consecutive invalid frames “5” as a criterion, so that the invalid frame F (2) is determined as a continuous invalid frame. However, it is not subject to the equivalent invalid frame.

  On the other hand, the invalid frame F (7) includes the invalid frame F (10) after the two normal frames F (8) and F (9), and the invalid frame interval “2” at this time is the criterion. Since the number of invalid frame intervals is “2” or less, the normal frames F (8) and F (9) are determined to be invalid frames. Similarly, the invalid frame F (10) includes an invalid frame F (12) after one normal frame F (11). The invalid frame interval “1” at this time is the number of invalid frame intervals “2” as a criterion. Therefore, the normal frame F (11) is determined as an invalid frame.

  Further, invalid frames continue from frame F (12) to F (16), and at least three normal frames F (17) to F (19) follow the invalid frame F (16). Since the invalid frame interval “3” is larger than the invalid frame interval number “2” of the criterion, the normal frames F (17) and F (18) are not determined to be invalid frames. Therefore, the continuous invalid frame length related to the invalid frame F (7) is “10” up to the frame F (16), which is equal to or more than the number of consecutive invalid frames “5” as a criterion. F (16) is determined as a continuous invalid frame.

  Thereafter, since the number of consecutive frames of frames F (7) to F (16) determined to be consecutive invalid frames is “10”, the number of consecutive invalid invalid frames, “4” as the criterion invalid invalid frame number, continues. Six invalid frames, that is, frames F (11) to F (16) are selected as equivalent invalid frames.

  In the present embodiment, the equivalent invalid frame selection means 13C of the quality analysis unit 13 considers human perceptual characteristics from the invalid frame affected by the invalid packet based on the determination criterion according to the content class, and is equivalent invalid. Since the frame is selected, the equivalent invalid frame can be accurately selected in consideration of human perceptual characteristics that change for each content, and the quality estimation scale used for estimating the user experience quality, that is, the equivalent invalid The frame rate can be calculated with high accuracy.

[Operation of First Embodiment]
Next, the operation of the user experience quality estimation apparatus according to the first embodiment of the present invention will be described with reference to FIG. FIG. 7 is a flowchart showing a user experience quality estimation process in the user experience quality estimation apparatus according to the first embodiment of the present invention.

  Here, when the application terminal 2 is the receiving side and the user experience quality value regarding the media packet of the video media transmitted from the opposite application terminal (not shown) on the transmitting side and received via the packet network 5 is estimated. Will be described as an example. At this time, when transmission quality information obtained by the packet transmission unit of the opposite application terminal is notified to the application terminal 2 by a control packet from a user experience quality estimation device (not shown) connected to the opposite application terminal Will be described as an example.

  The main quality information included in the transmission quality information acquired by the quality information acquisition unit 11 of the user experience quality estimation device 1 includes a sequence number indicating a transmission order, a video frame type, and a video frame for each transmission media packet. There are a number, the number of transmitted packets, and the number of transmitted frames. The main quality information included in the reception quality information includes a sequence number indicating the transmission order and a delay fluctuation time indicating the arrival time difference from the immediately preceding media packet for each received media packet.

  The quality information acquisition unit 11 of the user experience quality estimation apparatus 1 receives the transmission quality information transmitted by the control packet such as RTCP XR (RTP Control Protocol) from the opposite application terminal side by the packet reception unit 21 of the application terminal 2. The acquired one is acquired (step 100). Further, the reception quality information obtained from the media packet from the opposite application terminal is acquired by the packet receiving unit 21 of the application terminal 2 (step 101).

  The invalid packet determination unit 12 determines an invalid packet for each media packet whose transmission is confirmed by the transmission quality information, and determines a media packet that cannot be normally received by the receiving application terminal as an invalid packet ( Step 102).

  Next, the quality analysis unit 13 analyzes the content class of the media based on the packet transmission status of the media packet detected from the transmission quality information for each frame type of the media packet by the content class analyzing unit 13A (step 103). ).

  Further, the quality analysis unit 13 determines the target frame corresponding to the invalid packet determined by the invalid packet determination unit 12 among the frames based on the transmission quality information by the invalid frame determination unit 13B (step 104). ). Further, when an invalid frame is included in another frame used for decoding the frame in accordance with the frame type sequence defined in the frame configuration used for encoding the media, the frame is determined to be an invalid frame ( Step 104).

  Subsequently, the quality analysis unit 13 performs the above-described equivalent invalid frame selection process of FIG. 6 by the equivalent invalid frame selection unit 13C based on the determination criterion corresponding to the content class analyzed by the content class analysis unit 13A. Thus, based on the invalid frame determined by the invalid frame determination means 13B, an equivalent invalid frame is selected from each frame according to the human perceptual characteristics (step 105).

  Thereafter, the quality analysis unit 13 divides the total number of invalid equivalent frames selected by the equivalent invalid frame selection unit 13C by the number of frames indicated by the transmission quality information by the equivalent invalid frame rate calculation unit 13D. Thereby, the ratio of the equivalent invalid frame determined by the equivalent invalid frame selection means 13C among the respective frames is calculated as the equivalent invalid frame rate (step 106).

  Next, the user experience quality estimation unit 14 prepares a content class analysis from among quality estimation models indicating the relationship between the invalid frame rate and the user experience quality value prepared in advance and stored in a storage unit (not shown). The quality estimation model 15 corresponding to the content class analyzed by the means 13A is selected (step 107), and the user experience quality value corresponding to the equivalent invalid frame rate calculated by the quality analysis unit 13 is selected from the quality estimation model 15. Derived (step 108) and output to the storage unit as the quality estimate 16 of the user experience quality of the real-time application.

The creation of a quality estimation model represents the relationship between the intermediate parameters and the user experience quality consisting of MOS values (Mean Opinion Score average opinion scores), which are subjective evaluation values obtained by actually performing opinion evaluation, using a regression curve or a table. Model it.
Conventionally, the invalid packet rate of lost packets lost on the packet network 5 is used as an intermediate parameter, and the MOS value is estimated from the invalid packet rate using an estimation model indicating the relationship between the invalid packet rate and the MOS value. (Refer patent document 1 etc.). Such an estimation model can guarantee a certain degree of estimation accuracy, but is not sufficient when high estimation accuracy is required because media packets and frames do not correspond one-to-one.

  In addition, the estimation model indicating the relationship between the invalid frame rate and the MOS value has less variation than the estimation model indicating the relationship between the invalid packet rate and the MOS value, but human perceptual characteristics are not considered. The estimation result estimated by such an estimation model based on the quality degradation situation in an actual real-time system application includes an error from the subjective quality evaluation of the user with respect to the media.

  In the present embodiment, an equivalent invalid frame rate considering sensitivity to media degradation among human perceptual characteristics is obtained and used as an intermediate parameter. The equivalent invalid frame rate is derived from the same media packet quality information as when the invalid packet rate and invalid frame rate were obtained, and the estimation model indicating the relationship between the invalid frame rate and the MOS value was used. In comparison, there is even less variation.

Further, as described above, the influence on the user experience quality due to the invalid frame varies depending on the content to be reproduced in the frame, and therefore, in this embodiment, a quality estimation model is prepared for each content class. .
FIG. 8 is an explanatory diagram showing a quality estimation model (vs. invalid frame rate) for each content class. Here, an invalid frame rate is used as a quality estimation measure. FIG. 9 is an explanatory diagram showing a quality estimation model (vs. equivalent invalid frame rate) for each content class. Here, an equivalent invalid frame rate is used as a quality estimation measure. In the present embodiment, as described above, by using the equivalent invalid frame rate as the quality estimation measure, the quality estimation in which the MOS value changes to the same extent as the quality estimation measure in each content class as shown in FIG. Each model is obtained.

As the quality estimation model 15 that is actually used by the user experience quality estimation unit 14, the regression curve of FIG. 9 may be used, or an estimation model obtained by tabulating the regression curve may be used.
Specifically, in FIG. 9, for example, when the minimum unit that is the minimum input unit of the input parameter (specifically, the minimum input unit of the equivalent invalid frame rate) is 0.01, the output is performed in 0.01 units. A table is created by averaging the values obtained. That is, by averaging the MOS values that fall within the minimum time range, a table is created for the relationship between the equivalent invalid frame rate and the MOS for every minimum time from 0. However, when the number of evaluation samples is small, there is a minimum data with no data. In that case, complementation is performed using data having data before and after. This complementation may be linear or regression may be used. It is desirable to acquire a large number of data samples when creating a model so that such a phenomenon is minimized.

Thereafter, the estimation result notifying unit 17 reads the quality estimation value 16 estimated by the user experience quality estimation unit 14 from the storage unit, stores it in a control packet such as RTCP XR, and the counter application via the packet network 5. It transmits with respect to the user experience quality estimation apparatus (not shown) and quality control server (not shown) connected to the terminal (step 109), and complete | finishes a series of user experience quality estimation processes.
At this time, the estimation result notification unit 17 may notify the result of statistical processing for obtaining the average value of the quality estimation value 16 and the variance indicating the variation at regular intervals.

  As described above, in the present embodiment, the quality information acquisition unit 11 uses the transmission side quality information indicating the transmission status of the media packet used for communication of the media data transmitted from the transmission side application terminal, and the reception side application terminal. The reception quality information indicating the reception status of the received media packet is acquired, and the invalid packet determination unit 12 invalidates the packet that could not be normally received among the media packets based on the transmission quality information and the reception quality information. It is determined as a packet, and the quality analysis unit 13 analyzes the content class of the media based on the packet transmission status of the media packet detected for each frame type of the media packet from the transmission quality information, and the determination criterion according to the content class Based on each frame sent and received in media packets. The consecutive invalid frames that are affected by invalid packets are determined, and the ratio of equivalent invalid frames selected by weighting these consecutive invalid frames with a predetermined number of frames is calculated as the equivalent invalid frame rate. The user corresponding to the equivalent invalid frame rate calculated by the quality analysis unit from the quality estimation model 15 indicating the relationship between the equivalent invalid frame rate and the user experience quality value according to the content class by the user experience quality estimation unit 14 A sensation quality value is derived and output as an estimated value of the user sensation quality of the application.

As a result, an invalid frame is determined based on a criterion according to the content class of the content transmitted in the real-time application, an equivalent invalid frame rate is calculated, and further, an equivalent frame is calculated using a quality estimation model corresponding to the content class. A user experience quality value corresponding to the invalid frame rate is estimated.
Therefore, it is possible to accurately calculate the loss estimation scale used for the determination of the invalid frame and the estimation of the user experience quality in consideration of the influence of the content content on the user experience quality, and the estimation accuracy of the user experience quality is improved. Can be improved.

  Further, in the present embodiment, when analyzing the content class, the content analysis unit 13A of the quality analysis unit 13 is based on the packet transmission status of the media packet detected for each frame type of the media packet from the transmission quality information. Since the media content class is analyzed, the content class can be analyzed accurately with extremely simple processing.

  In the present embodiment, when selecting an equivalent invalid frame, the equivalent invalid frame selecting unit 13C of the quality analysis unit 13 selects the invalid frame affected by the invalid packet based on the determination criterion according to the content class. Since the equivalent invalid frame is selected in consideration of human perceptual characteristics, the equivalent invalid frame can be accurately selected in consideration of human perceptual characteristics that change for each content, and the user experience quality can be improved. It is possible to accurately calculate a quality estimation scale used for estimation, that is, an equivalent invalid frame rate.

[Second Embodiment]
Next, a user experience quality estimation apparatus according to the second embodiment of the present invention will be described with reference to FIG. FIG. 10 is a flowchart showing invalid packet determination processing of the user experience quality estimation apparatus according to the second embodiment of the present invention.

In the present embodiment, a specific example of invalid packet determination processing in the invalid packet determination unit 12 of the first embodiment will be described.
The invalid packet determination unit 12 first arbitrarily selects an undetermined transmission media packet from the transmission quality information (step 120), and confirms whether or not the same media packet as the sequence number is received with the reception quality information ( Step 121). Here, when the media packet having the same sequence number is not received (step 121: NO), the media packet is an invalid packet because it can be determined that the media packet is a lost packet lost in the middle transmission path. (Step 125).

  On the other hand, when the same media packet as the sequence number has been received (step 121: YES), the delay fluctuation time of the transmission media packet is confirmed with the reception quality information, and the allowable delay time possessed by the real-time application Compare (step 122). Here, if the delay fluctuation time exceeds the allowable delay time (step 122: NO), it can be determined that the media packet is a late arrival packet that arrives after being delayed in the transmission path or terminal. From this, it is determined that the packet is an invalid packet (step 125).

  On the other hand, when the delay fluctuation time is equal to or shorter than the allowable delay time (step 122: YES), only when the real-time application considers the arrival order reversed packet as a loss (step 123: YES), and the media packet It is confirmed from the reception quality information whether the order of arrival and the sequence number match. Here, when the packet cannot be confirmed and the arrival order is reversed (step 124: YES), it is determined that the packet is an invalid packet (step 125).

When the arrival order is not reversed (step 124: NO), or when the real-time application does not regard the arrival order reversed packet as a loss (step 123: NO), it is determined that the packet is not an invalid packet. (Step 126).
In this way, after the invalid packet determination is performed on the selected transmission media packet, if there is a transmission packet that has not been determined to be invalid (step 127: YES), the process returns to step 120 described above. The invalid packet determination for the remaining received media packets is repeated. On the other hand, when the invalid packet determination is completed for all the transmission media packets (step 127: NO), the series of invalid packet determination processing ends.

  As described above, in the present embodiment, the invalid packet determination unit 12 determines a media packet that has not been confirmed to be received in the reception quality information from among the media packets that have been confirmed to be transmitted in the transmission quality information, and the reception quality information. Media packets whose arrival time difference from the previous media packet obtained from the arrival time of each media packet exceeds the allowable packet delay time of the application are judged as invalid packets. Packets lost due to arrival can also be determined as invalid packets, and packets that cannot be reproduced as media by the real-time application on the receiving side can be accurately determined.

  Further, since the invalid packet determination unit 12 determines that the media packet whose reception order is inconsistent with the transmission order of the media packet confirmed by the reception quality information is an invalid packet, the real-time application arrives. When it is considered that the order-reversed packet is a loss, it is possible to more accurately determine a packet that cannot be reproduced as media by the receiving side real-time application.

[Third Embodiment]
Next, with reference to FIG. 11, the user experience quality estimation apparatus concerning the 3rd Embodiment of this invention is demonstrated. FIG. 11: is a flowchart which shows the invalid frame determination process of the user experience quality estimation apparatus concerning the 3rd Embodiment of this invention.

In the present embodiment, a specific example of the invalid frame determination process in the invalid frame determination means 13B of the first embodiment will be described.
First, the invalid frame determination unit 13B determines that the frame identified as the target frame by the target frame identification unit 12B among the frames whose transmission is confirmed by the transmission quality information is identified as the invalid frame because the target frame identification unit 12B includes the invalid packet (Step S1). 130).

  Next, with reference to the reception quality information, an invalid frame undetermined frame is selected according to the order of arrival at the receiving side (step 131), and the frame indicating the frame configuration used for media encoding in the real-time application By referring to the configuration information and the reception quality information, it is confirmed whether an invalid frame is included in another frame used for decoding the frame (step 132). The frame configuration information may be set in advance in the storage unit or may be acquired as transmission / reception quality information.

  If an invalid frame is included (step 132: YES), it is determined that the frame is an invalid frame because the frame cannot be decoded normally (step 133). On the other hand, if the invalid frame is not included (step 132: NO), since the frame can be normally decoded, it is determined that the frame is not an invalid frame (step 134).

  FIG. 12 shows an example of invalid frame determination processing. Here, for example, on the basis of the MPEG encoding method, the video media is compressed and encoded with a GOP configuration of “IBB” and divided into frames. The frame type I frame is a reference frame, and the original frame can be decoded only by the I frame. The frame type B frame is composed of difference data from the I frame, and the I frame is required for decoding.

As shown in FIG. 12, when frames “1 to 9” are divided into media packets according to the GIB configuration of “IBB” and transmitted and received at the receiving side, frames “2, 4, 7, 8” ”Is an invalid packet.
In this case, first, a target frame including an invalid packet is determined to be an invalid frame, and further, a frame used at the time of decoding is also determined to be an invalid frame.

  Specifically, the frame “4” is used for decoding the frame “5, 6”, but the frame “4” is an invalid frame, and therefore, these frames “5, 6” are also determined to be invalid frames. Similarly, frame “7” is used for decoding frame “9”, but since frame “7” is an invalid frame, frame “9” is also determined to be an invalid frame. Note that frame “1” is used for decoding frame “3”, but since frame “1” is not an invalid frame, frame “3” is determined not to be an invalid frame.

  In this manner, after the invalid frame determination is performed on the selected frame, if an invalid frame undecided frame remains (step 135: YES), the process returns to the above-described step 131 and the remaining frames are processed. Repeat invalid frame determination. On the other hand, when the invalid packet determination is completed for all transmission media packets (step 135: NO), the series of invalid frame determination processing is terminated.

  As described above, in this embodiment, the invalid frame determination unit 13B determines that the target frame among the frames is an invalid frame and uses it for decoding the frame according to the arrangement of the frame types defined in the frame configuration. If an invalid frame is included in the frame, the frame is determined to be an invalid frame. An invalid frame can be accurately determined.

[Extension of each embodiment]
In each of the above embodiments, the application terminal 2 is the receiving side, and the user experience quality relating to the media packet of the video media transmitted from the opposite application terminal (not shown) on the transmitting side and received via the packet network 5 Although the case where the value is estimated has been described as an example, the present invention is not limited to this. That is, it is possible to estimate the user experience quality value related to the media packet of the video media received via the packet network 5 at the opposite application terminal (not shown) as the reception side, where the application terminal 2 is the transmission side.

In this case, the quality information acquisition unit 11 of the user experience quality estimation apparatus 1 uses the quality information obtained by the packet transmission unit 22 of the application terminal 2 as the transmission quality information described above, and the packet reception unit of the application terminal 2 The quality information from the opposite application terminal received at 21 may be used as the reception quality information described above.
At this time, when the similar user experience quality is also estimated in the opposite application terminal, instead of transmitting the transmission quality information obtained by the packet transmission unit 22 of the application terminal 2 by the quality information notification unit 18 described above, packet reception The reception quality information regarding the reception media packet obtained by the unit 21 may be transmitted to the opposite application terminal.

  The transmission / reception quality information may be directly exchanged between the transmission side and the reception side, but may be exchanged via a quality management server that manages the quality information. Further, the user experience quality estimation device 1 may receive quality information directly.

  In each of the above embodiments, the case where the user experience quality estimation device 1 is realized by a device separate from the application terminal 2 has been described as an example. However, the present invention is not limited to this and is parallel to a real-time application. Then, it may be realized by a quality estimation application executed on the application terminal 2. As a result, the hardware resources of the application terminal 2 can be shared and realized, and the user experience quality can be managed very easily.

It is a block diagram which shows the structure of the user experience quality estimation apparatus concerning the 1st Embodiment of this invention. It is explanatory drawing which shows the content class criteria used by a content analysis process. It is explanatory drawing which shows the other content class criteria used by a content analysis process. It is explanatory drawing which shows a content class analysis process. It is explanatory drawing which shows another content class analysis process. It is explanatory drawing which shows an equivalent invalid frame selection process. It is a flowchart which shows the user experience quality estimation process in the user experience quality estimation apparatus concerning the 1st Embodiment of this invention. It is explanatory drawing which shows the quality estimation model (vs. invalid frame rate) for every content class. It is explanatory drawing which shows the quality estimation model (vs. equivalent invalid frame rate) for every content class. It is a flowchart which shows the invalid packet determination process of the user experience quality estimation apparatus concerning the 2nd Embodiment of this invention. It is a flowchart which shows the invalid frame determination process of the user experience quality estimation apparatus concerning the 3rd Embodiment of this invention. It is an example of an invalid frame determination process. It is explanatory drawing which shows the difference in the image frame loss rate-MOS value characteristic between different contents.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... User experience quality estimation apparatus, 11 ... Quality information acquisition part, 12 ... Invalid packet determination part, 13 ... Quality analysis part, 13A ... Content class analysis means, 13B ... Invalid frame determination means, 13C ... Equivalent invalid frame selection means, 13D: Equivalent invalid frame rate calculation means, 14: User experience quality estimation unit, 15 ... Quality estimation model, 16 ... Quality estimation value, 17 ... Estimation result notification unit, 18 ... Quality information notification unit, 2 ... Application terminal, 21 ... Packet receiver, 22 ... packet transmitter, 5 ... packet network.

Claims (10)

When each frame obtained by encoding media such as video and audio by a real-time application is transmitted / received as a media packet between a transmission-side and reception-side application terminal connected via a packet network, the reception is performed. A user experience quality estimation device that estimates a user experience quality that a user feels about media received and played by a side application terminal,
Quality information acquisition for acquiring transmission-side quality information indicating the transmission status of media packets transmitted from the transmission-side application terminal and reception quality information indicating the reception status of the media packets received by the reception-side application terminal And
An invalid packet determination unit that determines, as an invalid packet, a packet that could not be normally received among the media packets based on the transmission quality information and the reception quality information;
Analyzing the media content class based on the packet transmission status of the media packet detected for each frame type of the media packet from the transmission quality information, and based on the determination criteria according to the content class, Judgment of consecutive invalid frames affected by the invalid packet among the frames, and calculating the ratio of equivalent invalid frames selected by weighting these consecutive invalid frames with a predetermined number of frames as the equivalent invalid frame rate A quality analysis department to
Deriving a user experience quality value corresponding to the equivalent invalid frame rate calculated by the quality analysis unit from a quality estimation model indicating a relationship between an equivalent invalid frame rate and a user experience quality value according to the content class, A user experience quality estimation device comprising: a user experience quality estimation unit that outputs an estimated value of the user experience quality of an application. In the user experience quality estimation apparatus according to claim 1,
The quality analysis unit
When the number of normal frames sandwiched between the invalid frames is equal to or less than a predetermined number of invalid frame intervals, the normal frame is determined as an invalid frame, and the invalid frames that are consecutive in the number of consecutive invalid frames according to the content class are determined. Equivalent invalid frame selection means for selecting the number of invalid frames exceeding the number of weight frames according to the content class as the equivalent invalid frame number,
Equivalent invalid frame rate calculation means for calculating a ratio of the number of equivalent invalid frames to the number of frames transmitted in the media packet as the equivalent invalid frame rate. In the user experience quality estimation apparatus according to claim 1,
The invalid packet determination unit includes: a media packet whose transmission is confirmed with the transmission quality information, a packet whose reception is not confirmed with the reception quality information, and a media immediately before the media packet whose reception is confirmed with the reception quality information. Packets in which the delay fluctuation time indicating the arrival interval with the packet for use exceeds the allowable packet delay time of the application, and packets in which the transmission order and the reception order are inconsistent among the media packets confirmed to be received with the reception quality information Each of which is determined to be the invalid packet. In the user experience quality estimation apparatus according to claim 1,
The quality analysis unit determines that the target frame corresponding to the invalid packet among the frames is an invalid frame, and determines the frame according to the frame type sequence defined in the frame configuration used for encoding the media. A user experience quality estimation apparatus comprising invalid frame determination means for determining that an invalid frame is included in another frame used for decoding, as an invalid frame. In the user experience quality estimation apparatus according to claim 1,
The quality information acquisition unit acquires, as the transmission quality information, the quality information from the transmission side application terminal received by the packet reception unit of the reception side application terminal, and at the packet reception unit of the reception side application terminal. A user experience quality estimation device, wherein quality information obtained from a received media packet is acquired as the reception quality information. In the user experience quality estimation apparatus according to claim 1,
The quality information acquisition unit acquires, as the reception quality information, quality information from the reception-side application terminal received by the packet reception unit of the transmission-side application terminal, and a packet transmission unit of the transmission-side application terminal. A user experience quality estimation device, wherein quality information obtained from a transmitted media packet is acquired as the transmission quality information. In the user experience quality estimation apparatus according to claim 1,
The user experience quality estimation apparatus further comprising a quality information notification unit that notifies the reception side application terminal of the transmission quality information obtained by the packet transmission unit of the transmission side application terminal. In the user experience quality estimation apparatus according to claim 1,
The user experience quality estimation apparatus further comprising a quality information notification unit for notifying the reception side application terminal of the reception quality information obtained by the packet reception unit of the reception side application terminal. When each frame obtained by encoding media such as video and audio by a real-time application is transmitted / received as a media packet between a transmission-side and reception-side application terminal connected via a packet network, the reception is performed. A user experience quality estimation method used in a user experience quality estimation device that estimates a user experience quality that a user feels about the media received and played by a side application terminal,
Transmission quality information indicating the transmission status of the media packet transmitted from the transmission-side application terminal by the quality information acquisition unit, and reception quality information indicating the reception status of the media packet received by the reception-side application terminal; A quality information acquisition step to acquire,
An invalid packet determination step by which an invalid packet determination unit determines, as an invalid packet, a packet that could not be normally received among the media packets based on the transmission quality information and the reception quality information;
The quality analysis unit analyzes the media content class based on the packet transmission status of the media packet detected for each frame type of the media packet from the transmission quality information, and sets the determination criterion according to the content class. Based on the above, the invalid invalid frames determined by determining the consecutive invalid frames affected by the invalid packet from the frames and weighting the consecutive invalid frames with a predetermined number of frames are equivalently invalid. A quality analysis step to calculate the frame rate;
A user experience corresponding to the equivalent invalid frame rate calculated by the quality analysis unit from a quality estimation model indicating a relationship between an equivalent invalid frame rate and a user experience quality value according to the content class by the user experience quality estimation unit. A user experience quality estimation method comprising: a user experience quality estimation step of deriving a quality value and outputting it as an estimated value of the user experience quality of the application. When each frame obtained by encoding media such as video and audio by a real-time application is transmitted / received as a media packet between a transmission-side and reception-side application terminal connected via a packet network, the reception is performed. In the computer of the user experience quality estimation device that estimates the user experience quality that the user feels about the media received and played by the side application terminal,
Transmission quality information indicating the transmission status of the media packet transmitted from the transmission-side application terminal by the quality information acquisition unit, and reception quality information indicating the reception status of the media packet received by the reception-side application terminal; A quality information acquisition step to acquire,
An invalid packet determination step by which an invalid packet determination unit determines, as an invalid packet, a packet that could not be normally received among the media packets based on the transmission quality information and the reception quality information;
The quality analysis unit analyzes the media content class based on the packet transmission status of the media packet detected for each frame type of the media packet from the transmission quality information, and sets the determination criterion according to the content class. Based on the above, each frame is determined as consecutive invalid frames affected by the invalid packets, and the ratio of equivalent invalid frames selected by weighting the consecutive invalid frames with a predetermined number of frames is equivalent. A quality analysis step to calculate the invalid frame rate;
A user experience corresponding to the equivalent invalid frame rate calculated by the quality analysis unit from a quality estimation model indicating a relationship between an equivalent invalid frame rate and a user experience quality value according to the content class by the user experience quality estimation unit. A program for deriving a quality value and executing a user experience quality estimation step of outputting the estimated value of the user experience quality of the application.

Images