JP5144593B2 - User experience quality estimation system and method - Google Patents

Description

  The present invention relates to a communication quality management technique, and more particularly to a technique for estimating a user experience quality for a video communication service based on a packet transfer situation that can be measured by a packet network or a receiving terminal.

In recent years, video-based IP services have begun to be provided in earnest throughout the world as IPTV services, and the market is expanding. On the other hand, since the quality of the packet network is not guaranteed, the quality management technology for applications that implement video communication services has attracted attention, and the quality measuring instrument market is also expanding.
In such quality management technology, it is becoming important to manage not only the quality of the packet network but also the quality of user experience (QoE: Quality of Experience) (see, for example, Non-Patent Document 1).

In order to manage user experience quality, conventionally, techniques for estimating user experience quality using various indexes that can be acquired from a network or a terminal have been studied.
Among these techniques, an approach using a video signal has been proposed as a user experience quality estimation technique for a video communication service (see, for example, Patent Document 1). However, in such an approach, since the video signal itself is used, the user experience quality cannot be estimated using information obtained from the network.

  On the other hand, an approach using information that can be acquired on the network such as a packet transfer status has also been proposed (see, for example, Patent Document 2). In addition, the inventors have provided a technology for estimating user experience quality using information that can be acquired by a packet network mainly for voice (VoIP) services, and also a network management technology using this. (For example, see Patent Document 3).

Japanese Patent No. 4257333 JP 2007-060475 A Japanese Patent No. 3579334

ITU-T Recommendation P.10 / G.100, "Definition of Quality of Experience (QoE)," Jan., 2007.

  However, in such a conventional technique, when estimating the user experience quality related to the video reproduced at the receiving terminal, the fluctuation of the video communication packet generated in the packet network, that is, the influence of the jitter on the user experience quality is sufficiently considered. The user experience quality cannot be estimated with high accuracy.

  That is, the above-described conventional technology mainly uses packet loss and jitter. However, with regard to the influence of jitter in the packet network, only the influence on the user experience quality due to the buffer underflow phenomenon in which the video communication packet arrives at the receiving terminal later than scheduled is considered, and the packet is earlier than the scheduled time. The influence on the user experience quality due to the buffer overrun phenomenon arriving at the receiving terminal is not considered. In practice, terminals and applications in which video quality deteriorates due to the effect that packets arrive at the terminal earlier are widely used.

  An object of the present invention is to provide a user quality estimation technique capable of estimating the user experience quality with high accuracy by sufficiently considering the packet jitter.

In order to achieve such an object, the user experience quality estimation system according to the present invention performs video communication performed between a transmission terminal and a reception terminal via a packet network on a video received and reproduced by a reception terminal. A user experience quality estimation system for estimating user experience quality perceived by a user, packet information storing packet information indicating packet transfer times measured for each packet for video communication transferred from a packet network to a receiving terminal A storage unit, a network quality management threshold storage unit that stores the number of allowable received packets indicating the number of packets that can be received by the receiving terminal per unit time as a network quality management threshold, and a packet loss rate that indicates a packet transfer status of video communication; In this case, a user experience quality estimation model indicating a relationship with the user experience quality value obtained by the video communication is stored. A model storage unit, the loss packet detecting unit that counts the number of lost packets by detecting a lost packet loss packet network based on the packet information for each unit time based on the transfer time information for each packet included in the packet information Count the number of transfer packets, and estimate the number of packets exceeding the allowable number of received packets as the number of discarded packets discarded at the receiving terminal, and the total of the number of lost packets and the number of discarded packets By dividing by the total number of transmitted packets transmitted from the transmitting terminal to the receiving terminal calculated from the information, the packet loss rate calculating unit for calculating the packet loss rate and the user experience quality estimation model are referred to. A user experience quality estimation unit for estimating a corresponding user experience quality value.

  At this time, a network quality control threshold value calculation device for calculating a network quality control threshold value is further provided, and the network quality control threshold value calculation device is provided for each video communication performed under a plurality of communication conditions with different jitter amounts in the packet network. A packet information storage unit for storing packet information classified by condition indicating packet transfer time measured for each video communication packet transferred from the packet network to the receiving terminal, and quality evaluation of the video reproduced by the receiving terminal for each communication condition Based on the steady-state packet information at the time of steady-state video communication at the time of steady-state video communication in which the quality of the quality evaluation result storage unit for storing the result and the video quality at the receiving terminal among the condition-specific packet information is not deteriorated. Based on a steady value calculation unit for calculating the number of steady received packets to be received, a steady unit time, and a steady received packet number, from within a predetermined verification range Assuming threshold generation unit for generating a plurality of sets of constant unit time and assumed allowable received packet number as an assumed threshold, and for each assumed threshold, based on the assumed unit time and the assumed allowable received packet number, By verifying the included packet transfer status, we estimate and evaluate the presence or absence of video quality degradation under the communication conditions, and compare the estimated evaluation with the quality evaluation results under the communication conditions, so that An evaluation accuracy rate calculation unit for calculating a rate, and an assumption unit time and an assumed allowable number of received packets regarding the assumption threshold with the highest accuracy rate among the evaluation accuracy rates of each assumption threshold are determined as a network quality management threshold for the network And a network quality control threshold value determination unit.

  Further, the apparatus further comprises a user experience quality estimation model creation device for creating a user experience quality estimation model, and the user experience quality estimation model creation device includes tests performed under a plurality of communication conditions with different numbers of packet losses in the packet network. For each video communication, a packet loss rate calculation unit that calculates the packet loss rate from the packet information obtained from the test video communication, and a quality evaluation result that acquires the quality evaluation result of the video played on the receiving terminal for each test video communication An acquisition unit and an estimation model creation unit that derives a user experience quality estimation model based on a correspondence relationship between the packet loss rate obtained for each test video communication and the quality evaluation result may be provided.

In addition, the user experience quality estimation method according to the present invention provides a user experience quality that a user feels with respect to a video received and reproduced by a reception terminal for video communication performed between a transmission terminal and a reception terminal via a packet network. Is a user experience quality estimation method used in a user experience quality estimation system that estimates a packet transfer time measured by the packet information storage unit for each video communication packet transferred from a packet network to a receiving terminal. A packet information storage step for storing information, and a network quality management threshold storage step in which the network quality management threshold storage unit stores an allowable number of received packets indicating the number of packets that can be received by the receiving terminal per unit time as a network quality management threshold. And the estimated model storage unit, the packet loss rate indicating the packet transfer status of video communication, and the video An estimated model storage step for storing a user experience quality estimation model indicating a relationship with a user experience quality value obtained by communication, and a lost packet detection unit detects a lost packet lost in the packet network based on the packet information and loss The lost packet detection step for counting the number of packets and the discarded packet estimation unit count the number of transferred packets per unit time based on the transfer time information of each packet included in the packet information . The discard packet estimation step for estimating the number of packets as the number of discarded packets discarded at the receiving terminal, and the packet loss rate calculation unit from the transmitting terminal that calculated the total of the number of lost packets and the number of discarded packets from the packet information to the receiving terminal The packet loss rate is calculated by dividing by the total number of transmitted packets. And Tsu DOO loss rate calculating step, the QoE estimation unit references the QoE estimation model, and a QoE estimation step of estimating the QoE value corresponding to the packet loss rate.

  At this time, the network quality management threshold value calculation device further includes a network quality management threshold value calculation step for calculating the network quality management threshold value, and the packet information storage unit differs in the jitter amount in the packet network in the network quality management threshold value calculation step. A packet information storage step for storing packet information classified by condition indicating a packet transfer time measured for each video communication packet transferred from the packet network to the receiving terminal for each video communication performed under a plurality of communication conditions; The quality evaluation result storage unit stores a quality evaluation result storage step for storing the quality evaluation result of the video reproduced by the receiving terminal for each communication condition, and the steady value calculation unit selects the video quality at the receiving terminal among the condition-specific packet information. Stationary reception that is received per stationary unit time that is set in advance based on steady packet information during steady video communication with no degradation The steady value calculation step for calculating the number of packets, and the assumption threshold generation unit, based on the steady unit time and the steady received packet number, set a set of the assumed unit time and the assumed allowable received packet number from a predetermined verification range. A hypothetical threshold generation step for generating a plurality of hypothetical thresholds, and an evaluation accuracy rate calculation unit, for each hypothetical threshold, based on the hypothetical unit time and the hypothetical allowable received packet number, the packet transfer status included in each condition packet information By verifying, the evaluation correctness that estimates and evaluates the presence or absence of video quality degradation under the communication conditions and compares the estimated evaluation with the quality evaluation results under the communication conditions to calculate the evaluation accuracy rate for the hypothetical threshold The rate calculation step and the network quality control threshold value determination unit determine the assumed unit time for the assumed threshold value with the highest accuracy rate among the evaluation accuracy rates for each assumed threshold value. Assumptions allowable number of received packets may include a network quality control threshold determination step of determining the network quality control threshold relating to the network.

  Further, the user experience quality estimation model creation device further includes a user experience quality estimation model creation step for creating a user experience quality estimation model, and the packet loss rate calculation unit includes a packet loss rate calculation unit in the packet network. A packet loss rate calculating step for calculating a packet loss rate from packet information obtained from the test video communication for each test video communication performed under a plurality of communication conditions with different packet loss numbers, and a quality evaluation result acquisition unit The quality evaluation result acquisition step for acquiring the quality evaluation result of the video reproduced by the receiving terminal for each test video communication, and the estimation model creation unit determines whether the packet loss rate and the quality evaluation result obtained for each test video communication are An estimation model creating step for deriving a user experience quality estimation model based on the correspondence relationship may be provided.

  According to the present invention, the buffer overrun phenomenon that occurs at the receiving terminal due to jitter in the packet network can be grasped as the number of discarded packets, and can be considered in estimating the user experience quality. Therefore, the user experience quality can be estimated with high accuracy.

It is a block diagram which shows the structure of the user experience quality estimation system concerning 1st Embodiment. It is a structural example of packet information. It is a structural example of a network quality control threshold value. It is a characteristic view which shows a user experience quality estimation model. It is a flowchart which shows the user experience quality estimation process of the user experience quality estimation system concerning 1st Embodiment. It is explanatory drawing which shows the discard packet in a receiving terminal. It is explanatory drawing which extracted the measurement example whose packet loss rate is zero from FIG. It is explanatory drawing which shows the correspondence of packet loss and packet discard, and a user experience quality. It is a block diagram which shows the structure of the network quality control threshold value apparatus concerning 2nd Embodiment. It is an example of a structure of packet information classified by conditions. It is a structural example of terminal information. It is a structural example of a quality evaluation result. It is a flowchart which shows the information acquisition process in the network quality management threshold value calculation apparatus concerning 2nd Embodiment. It is a structural example of a steady value. It is a flowchart which shows the network quality management threshold value determination process in the network quality management threshold value calculation apparatus concerning 2nd Embodiment. It is a structural example of an assumption threshold value. It is a flowchart which shows the evaluation correct rate calculation process in the network quality management threshold value calculation apparatus concerning 2nd Embodiment. It is an example of a calculation result of an evaluation correct answer rate. It is a graph which shows the calculation result of an evaluation correct answer rate. It is a block diagram which shows the structure of the user experience quality estimation model creation apparatus concerning 3rd Embodiment. It is a structural example of a quality evaluation result. It is a flowchart which shows the user experience quality estimation model creation process of the user experience quality estimation model creation apparatus concerning 3rd Embodiment.

Next, embodiments of the present invention will be described with reference to the drawings.
[First Embodiment]
First, a user experience quality estimation system according to a first embodiment of the present invention will be described with reference to FIG. FIG. 1 is a block diagram illustrating a configuration of a user experience quality estimation system according to the first embodiment.

  The user experience quality estimation system 10 as a whole is composed of information processing communication terminals such as general server devices and workstations, and is obtained from target video communication with reference to a user experience quality estimation model created in advance. Based on the packet information, it has a function of estimating the user experience quality that the user actually feels when the reception video of the video communication is reproduced on the reception terminal 3.

The video distribution apparatus 1 as a whole is composed of an information processing communication terminal such as a general server apparatus or workstation, and has a function of distributing video data in packets by operating as a transmission terminal in video communication. .
The receiving terminal 3 is composed of a communication terminal such as an STB (Set-Top-Box) which connects a television receiver to a data communication network and enables use of various communication services. It has a function of receiving video data in packets by performing video communication with the connected video distribution device 1 and reproducing and outputting the video data on a video display device 4 including a television receiver.

  The branching device 5 is composed of a communication device such as a tap or a hub, and is installed on a communication path between the packet network 2 and the receiving terminal 3 and transferred from the packet network 2 to the receiving terminal 3. It has a function to capture video communication packets.

The network quality management threshold value calculation device 20 is based on the packet information obtained by video communication performed in advance on a trial basis and the video quality evaluation at the receiving terminal 3 at that time, and the network for the packet network 2 used for the test. It has a function of calculating a quality control threshold value.
The user experience quality estimation model creation device 40 calculates the packet loss measured from any video communication based on the packet information obtained by video communication performed in advance on a trial basis and the video quality evaluation at the receiving terminal 3 at that time. It has a function of creating a user experience quality estimation model for estimating a user experience quality value according to the rate.

  In the present embodiment, the number of lost packets lost in the packet network 2 among the video communication packets transmitted from the transmission terminal is counted based on the packet information obtained from the target video communication, and the network quality control is performed. Based on the allowable number of received packets that can be received by the receiving terminal 3 per unit time specified by the threshold, the number of discarded packets discarded due to the buffer overrun phenomenon at the receiving terminal 3 is estimated, and the number of lost packets and discarded packets are estimated. The packet loss rate in the video communication is calculated from the number, and the user experience quality at the receiving terminal 3 is estimated with reference to the user experience quality estimation model created in advance from the packet loss rate.

Next, with reference to FIG. 1, the structure of the user experience quality estimation system 10 concerning this Embodiment is demonstrated in detail.
The user experience quality estimation system 10 includes, as main functional units, a packet information acquisition unit 11, a packet information storage unit 12, a lost packet detection unit 13, a network quality management threshold storage unit 14, a discarded packet estimation unit 15, a packet loss. A rate calculation unit 16, an estimated model storage unit 17, a user experience quality estimation unit 18, and a screen display unit 19 are provided.

The packet information acquisition unit 11 captures the quality estimation target video communication performed between the video distribution device 1 and the receiving terminal 3 in the in-service environment where the video communication service is provided. It has a function of acquiring packet information related to the video communication packet related to the video communication and a function of saving the acquired packet information in the packet information storage unit 12.
FIG. 2 is a configuration example of packet information. This packet information includes at least the packet number (sequence number) indicating the transmission order of the packets and the transfer time of each video communication packet transferred from the packet network 2 to the receiving terminal 3 through the branching unit 5. ing.

The packet transfer time of the packet information is described as relative time information from the first video communication packet that starts measurement, but may be absolute time information.
As for the packet transfer time, the reception time of each video communication packet captured by the branching device 5 and received by the packet information acquisition unit 11 may be used. The packet information acquisition unit 11 may acquire the transfer time from the branching device 5. Alternatively, the packet information acquisition unit 11 may acquire the transfer time of each video communication packet measured by the reception terminal 3 from the reception terminal 3.

  The lost packet detection unit 13 confirms the packet number (sequence number) of the packet information read from the packet information storage unit 12, and calculates the total number of transmitted packets from the packet number difference between the first packet and the last packet; Similarly, it has a function of confirming the packet number, detecting a discontinuous portion (jump number) not recorded as packet information as a lost packet lost in the packet network 2, and counting the number of lost packets. .

The discard packet estimation unit 15 indicates the packet transfer status included in the packet information read from the packet information storage unit 12, the unit time set as the network quality management threshold in the network quality management threshold storage unit 14, the allowable number of received packets, The number of discarded packets at the receiving terminal 3 is estimated by performing verification based on the above.
FIG. 3 is a configuration example of the network quality management threshold. The network quality control threshold is composed of unit time and the number of allowable received packets. A method for calculating the network quality control threshold will be described in a second embodiment described later.

The allowable number of received packets is a network quality management threshold value indicating the number of packets that can be received by the receiving terminal 3 per unit time. When a video communication packet exceeding the allowable number of received packets arrives at the receiving terminal 3 due to the jitter generated in the packet network 2, a buffer overrun phenomenon occurs at the receiving terminal 3, and the received packet is discarded. Is shown.
The discard packet estimation unit 15 counts the number of transfer packets per unit time based on the transfer time information of each packet included in the packet information read from the packet information storage unit 12, and determines the number of packets exceeding the allowable number of received packets. Count as the number of discarded packets.

  The packet loss rate calculation unit 16 adds the total number of lost packets obtained by the lost packet detection unit 13 and the number of discarded packets obtained by the discarded packet estimation unit 15 to the total transmission obtained by the lost packet detection unit 13. It has a function of calculating the packet loss rate by dividing by the number of packets.

  The user experience quality estimation unit 18 refers to the user experience quality estimation model in the estimation model storage unit 17 and obtains the user experience quality corresponding to the packet loss rate calculated by the packet loss rate calculation unit 16 to thereby estimate the quality. It has a function of estimating the user experience quality value of the target video quality and a function of displaying the estimated user experience quality value on the screen display unit 19.

FIG. 4 is a characteristic diagram showing a user experience quality estimation model, in which the horizontal axis indicates a packet loss rate (%), and the vertical axis indicates an estimated value of user experience quality (MOS defined in ITU-T recommendation P800: Mean Opinion Score).
In general, the user experience quality value in video communication has a characteristic of monotonically decreasing exponentially as the packet loss rate increases. A method for creating this user experience quality estimation model will be described in a third embodiment described later.

  In the user experience quality estimation system 10, as a general configuration provided in a normal information processing apparatus, in addition to the screen display unit 19, an arithmetic processing unit, a storage unit, a data communication interface unit, a data input / output interface unit, Functional units such as an operation input unit are provided, and the above-described functional units perform processing operations using these functional units.

Among the functional units of the user experience quality estimation system 10 described above, the packet information acquisition unit 11, the lost packet detection unit 13, the discarded packet estimation unit 15, the packet loss rate calculation unit 16, and the user experience quality estimation unit 18 are as follows. What is necessary is just to implement | achieve with an arithmetic processing part.
The arithmetic processing unit includes a microprocessor such as a CPU and its peripheral circuits, and implements various functional units by reading and executing a program stored in advance in the storage unit.

  Of the functional units of the user experience quality estimation system 10, the packet information storage unit 12, the network quality management threshold storage unit 14, and the estimated model storage unit 17 may be realized by a storage device such as a hard disk or a semiconductor memory. Good. Each of these storage units may be realized by separate storage devices, or any one or a plurality of storage units may be realized by one storage device.

[Operation of First Embodiment]
Next, the operation of the user experience quality estimation system according to the first exemplary embodiment of the present invention will be described with reference to FIG. FIG. 5 is a flowchart illustrating user experience quality estimation processing of the user experience quality estimation system according to the first embodiment.

  In the user experience quality estimation process of FIG. 5, first, the user experience quality estimation system 10 uses the packet information acquisition unit 11 in the in-service environment where the video communication service is provided, and the video distribution device 1 and the receiving terminal 3. Packet information related to the quality estimation target video communication performed between the two is acquired and stored in the packet information storage unit 12 (step 100).

Next, the user experience quality estimation system 10 calculates the total number of transmitted packets and the number of lost packets from the packet information read from the packet information storage unit 12 by the lost packet detection unit 13 (step 101).
Further, the user experience quality estimation system 10 includes a packet information storage unit based on the unit time set as the network quality management threshold value in the network quality management threshold value storage unit 14 by the discarded packet estimation unit 15 and the number of allowable received packets. The number of discarded packets in the receiving terminal 3 is calculated from the packet information read from 12 (step 102).

  Thereafter, the user experience quality estimation system 10 calculates the total of the number of lost packets obtained by the lost packet detector 13 and the number of discarded packets obtained by the discarded packet estimator 15 by the packet loss rate calculator 16. The packet loss rate is calculated by dividing by the total number of transmitted packets obtained by the lost packet detector 13 (step 103).

  Next, the user experience quality estimation system 10 corresponds to the packet loss rate calculated by the packet loss rate calculation unit 16 by referring to the user experience quality estimation model in the estimation model storage unit 17 by the user experience quality estimation unit 18. By obtaining the user experience quality to be estimated, the user experience quality value of the video quality to be the quality estimation target is estimated (step 104), and the estimated user experience quality value is displayed on the screen display unit (step 105), The user experience quality estimation process is terminated.

[Effect of the first embodiment]
As described above, in the present embodiment, the lost packet detector 13 detects the lost packets lost in the packet network 2 based on the packet information in the packet information storage unit 12, counts the number of lost packets, and estimates the discarded packets. The unit 15 estimates the number of discarded packets discarded by the receiving terminal 3 by verifying the packet information with the allowable number of received packets in the network quality management threshold value storage unit 14, and the packet loss rate calculating unit 16 estimates the number of lost packets. And the number of discarded packets are divided by the total number of transmitted packets transmitted from the video distribution apparatus 1 to the receiving terminal 3 to calculate a packet loss rate, and the user experience quality estimation unit 18 estimates the user experience quality. With reference to the model, the user experience quality value corresponding to the packet loss rate is estimated.

  For this reason, the buffer overrun phenomenon occurring at the receiving terminal 3 due to the jitter in the packet network 2 can be regarded as the number of discarded packets, and can be taken into account for estimation of the user experience quality. Therefore, the user experience quality can be estimated with high accuracy.

  FIG. 6 is an explanatory diagram showing a discarded packet at the receiving terminal, where the horizontal axis indicates the packet loss rate (%), and the vertical axis indicates the estimated value (MOS) of the user experience quality. As shown in FIG. 4, generally, the user experience quality value in video communication has a characteristic of monotonically decreasing exponentially as the packet loss rate increases. When is zero, good user experience quality can be obtained.

  On the other hand, in the measurement example of FIG. 6, the user experience quality value varies greatly even though the packet loss rate is zero. This is because a buffer overrun phenomenon occurs in the receiving terminal 3 due to jitter in the packet network 2, and the packet is discarded in the receiving terminal 3.

  FIG. 7 is an explanatory diagram in which a measurement example in which the packet loss rate is zero is extracted from FIG. 6. The horizontal axis indicates the estimated packet discard rate (%), and the vertical axis indicates the estimated value (MOS) of the user experience quality. ing. This estimated packet discard rate is obtained by dividing the number of discarded packets estimated by the aforementioned discarded packet estimation unit 15 by the total number of transmitted packets. As can be seen from FIG. 7, the user experience quality value monotonously decreases with an increase in the discarded packet rate, and a decrease in the user experience quality can be captured by considering the number of discarded packets.

FIG. 8 is an explanatory diagram showing a correspondence relationship between packet loss and packet discard and user experience quality, where the horizontal axis shows the packet loss rate (%), and the vertical axis shows the estimated value (MOS) of the user experience quality. ing.
Of each measurement example plotted in FIG. 8, the packet loss rate condition indicates a case where there is no packet loss at the receiving terminal 3 and only packet loss occurs in the packet network 2, and the estimated packet loss rate condition is This shows a case in which there is no packet loss in the packet network 2 and only packet discard at the receiving terminal 3 occurs. For any of the conditions, the packet loss rate is calculated by the calculation method in the packet loss rate calculation unit 16 described above.

As shown in FIG. 8, it can be understood that the estimated quality of packet loss rate also shows a tendency for the user experience quality value to monotonously decrease as the packet loss rate increases, similarly to the packet loss rate condition. Therefore, this suggests that each measurement example of the estimated packet discard rate condition can be expressed by the user experience quality estimation model derived from each measurement example of the packet loss rate condition.
In this embodiment, from this point of view, the number of lost packets and the number of discarded packets are handled in the same row, and the packet loss rate obtained from both is derived from each measurement example of the packet loss rate condition. To estimate the user experience quality value.

In order to create a user experience quality estimation model for a video communication service, it is necessary to actually evaluate the user experience quality of a video. However, this evaluation test requires a lot of work time and work load because the subject needs to actually view and evaluate the received video in accordance with the prescribed opinion evaluation method.
For this reason, changing the communication conditions and increasing the number of evaluation tests increase the time and work required for the evaluation. In an evaluation test, it is very difficult to control various jitter communication conditions with a traffic pattern similar to that of an actual packet network.

  According to the present embodiment, it is possible to exclude communication conditions related to jitter from the evaluation test. Therefore, it is possible to estimate the user experience quality value with high accuracy with the user experience quality estimation model that can be created with less work time and work load.

  In addition, because the number of lost packets is calculated considering the number of discarded packets based on the target video communication packet information, the influence of jitter is considered based on the packet information that can be easily obtained by the communication service provider. Thus, it is possible to estimate video quality related to individual video communication. Therefore, it is possible to provide a high-quality video communication service using only the facilities on the communication service provider side without requiring a user burden such as adding a special function to the receiving terminal.

[Second Embodiment]
Next, with reference to FIG. 9, a network quality management threshold value calculation apparatus according to the second embodiment of the present invention will be described. FIG. 9 is a block diagram illustrating a configuration of a network quality management threshold device according to the second embodiment.
The network quality control threshold value calculation device 20 as a whole is composed of an information processing communication terminal such as a general server device or workstation, and packet information obtained by video communication performed in advance on a trial basis and a receiving terminal at that time 3 has a function of calculating a network quality management threshold value for the packet network 2 used in the test.

  The network quality control device 6 is composed of a test device that gives an arbitrary amount of jitter to an input packet and outputs it. The network quality control device 6 is installed on the communication network between the packet network 2 and the receiving terminal 3 and the packet network 2 Has a function of giving a jitter amount corresponding to a designated communication condition to a video communication packet transferred from the terminal to the receiving terminal 3. Note that the video distribution device 1, the packet network 2, the receiving terminal 3, the video display device 4, and the branching device 5 in FIG. 9 have the same or similar configurations as those in FIG. Description is omitted.

In the present embodiment, as a network quality management threshold, a set of a unit time which is a frame reproduction required time for one frame at the receiving terminal 3 and an allowable number of received packets at the receiving terminal 3 in the unit time is used. It is determined whether or not quality degradation has occurred in the video communication by verifying whether or not the video communication packet transferred in actual video communication is transferred within the allowable number of received packets.
As a result, it is possible to calculate a network quality management threshold considering the influence on the user experience quality due to a buffer overrun phenomenon in which the packet arrives at the receiving terminal 3 earlier than the scheduled time.

  As a network quality control threshold value calculation method, video communication is experimentally performed in advance under a plurality of communication conditions with different amounts of jitter in the packet network 2 and transferred from the packet network to the receiving terminal for each video transmission. In addition to preparing packet information by condition indicating the packet transfer time measured for each video communication packet, a quality evaluation result of a video reproduced by the receiving terminal 3 under these communication conditions is prepared.

  Then, in the network quality management threshold value calculation device 20, a plurality of network quality management threshold values each composed of a set of a unit time that is a frame reproduction required time for one frame at the receiving terminal 3 and the allowable number of received packets in the unit time are assumed. For each of these hypothetical network quality control thresholds, the presence or absence of quality degradation in video communication under the communication conditions is evaluated by verifying the packet information for each condition. Assumed network quality management threshold is determined as the assumed network quality management threshold value that provides the closest evaluation to the actual quality evaluation result at the receiving terminal 3 for video communication.

[Configuration of Second Embodiment]
Next, the configuration of the network quality management threshold value calculation apparatus according to the present embodiment will be described in detail with reference to FIG.
As shown in FIG. 9, the network quality management threshold value calculation device 20 includes, as main functional units, a packet information acquisition unit 21, a packet information storage unit 22, a terminal information acquisition unit 23, a steady value calculation unit 24, and a steady value storage. A unit 25, a quality evaluation result acquisition unit 26, a quality evaluation result storage unit 27, an assumption threshold generation unit 30, an evaluation accuracy rate calculation unit 31, and a network quality management threshold determination unit 32 are provided.

  The packet information acquisition unit 21 branches the video communication that is generated by the network quality control device 6 and that is experimentally performed between the video distribution device 1 and the receiving terminal 3 under a plurality of communication conditions with different jitter amounts. A function for acquiring condition-specific packet information regarding the video communication packet regarding the video communication captured by the device 5 and a function for storing the acquired condition-specific packet information in the packet information storage unit 22.

FIG. 10 is a configuration example of condition-specific packet information. This condition-specific packet information includes at least the packet number (sequence number) and transfer time of each video communication packet that has passed through the branching unit 5 and transferred from the packet network 2 to the receiving terminal 3. ing.
At this time, the network quality control device 6 may vary the jitter amount, detect in advance the jitter amount at which the video quality starts to deteriorate, select several jitter amounts before and after that, and set the communication conditions. In the example of FIG. 10, the jitter amount at which the video quality starts to deteriorate is 90 ms, and the communication conditions are 60 ms to 120 ms before and after that in 1 ms increments.

The packet transfer time of the condition-specific packet information is described by relative time information from the first video communication packet that starts measurement, but may be absolute time information.
As for the packet transfer time, the reception time of each video communication packet captured by the branching device 5 and received by the packet information acquisition unit 21 may be used. The packet information acquisition unit 21 may acquire the transfer time from the branching device 5.

  As for the packet transfer time used as the condition-specific packet information, the most accurate network quality control threshold can be calculated by using the packet reception time at the receiving terminal 3. However, since the jitter generated in the communication path from the branching device 5 to the receiving terminal 3 is negligible as compared with the jitter generated in the packet network 2, the network quality is determined at the point where the branching device 5 is installed. If it is on the communication path between the control device 6 and the receiving terminal 3, for example, it is calculated even if it is installed adjacent to the network quality control device 6 at the connection point between the packet network 2 and the access line of the receiving terminal 3. The impact on the network quality control threshold is negligible.

  When managing the packet network 2 in an in-service environment where a video communication service is provided using a network quality control threshold during network management operation, a branching device 5 is provided in the same manner as the experimental video communication described above. Thus, it is necessary to obtain packet information from actual video communication packets. Therefore, it is desirable to provide the branching unit 5 at the edge node of the packet network 2 that is easy for the communication service provider to manage. The error from the management operation can be suppressed.

  The same applies to the jitter generated in the communication path from the branching device 5 to the network quality control threshold value calculation device 20, and the packet transfer time is measured by the packet information acquisition unit 21 even if it is measured by the branching device 5. The influence on the calculated network quality control threshold is negligible.

The terminal information acquisition unit 23 has a function of acquiring various terminal information used for video communication at the receiving terminal 3.
FIG. 11 is a configuration example of terminal information. As the terminal information acquired by the terminal information acquisition unit 23, there is a video frame display rate indicating the number of frames reproduced per second by the receiving terminal 3, as shown in FIG.
For the video frame display rate, a value measured at the time of video reproduction at the receiving terminal 3 may be used, or a value set as a specification of the receiving terminal 3 may be used.

  As a method for acquiring the terminal information, the terminal information acquiring unit 23 acquires the terminal information from data communication between the terminal information acquiring unit 23 and the receiving terminal 3 or from a recording medium on which the terminal information is recorded. Alternatively, the terminal information acquisition unit 23 may acquire terminal information input by an operator using a keyboard provided in the network quality management threshold value calculation device 20.

  The steady value calculation unit 24 calculates a frame reproduction required time required to reproduce one frame of video at the receiving terminal 3 from the video frame display rate acquired by the terminal information acquisition unit 23 as a steady unit time, Received by the receiving terminal 3 per steady unit time based on the steady packet information during steady video communication in which the video quality at the receiving terminal is not deteriorated among the packet information classified by conditions stored in the packet information storage unit 22 And a function of storing the steady unit time and the number of steady received packets in the steady value storage unit 25 as steady values.

The quality evaluation result acquisition unit 26 obtains the quality evaluation result of the video reproduced by the receiving terminal 3 for each of the communication conditions for the experimental video communication when the packet information classified by conditions acquired by the packet information acquisition unit 21 is obtained. It has a function to acquire and a function to save the acquired quality evaluation result in the quality evaluation result storage unit 27.
FIG. 12 is a configuration example of the quality evaluation result. The quality evaluation result may be information in which the presence or absence of video quality deterioration is detected, and an evaluation value quantified by a five-step evaluation or the like (for example, MOS: Mean Opinion defined in ITU-T recommendation P800) Score).

  The hypothetical threshold generation unit 30 uses a set of the hypothetical unit time and the hypothetical allowable reception packet number as a hypothetical threshold from a predetermined verification range based on the stationary unit time and the stationary reception packet number read from the stationary value storage unit 25. It has a function to generate multiple.

  The evaluation accuracy rate calculation unit 31 is included in each condition-specific packet information read from the packet information storage unit 22 for each assumption threshold generated by the assumption threshold generation unit 30 based on the assumed unit time and the number of assumed allowable received packets. The function of estimating the presence or absence of video quality degradation under the communication conditions by verifying the packet transfer status to be compared, and comparing these estimation evaluations with the quality evaluation results under the communication conditions read from the quality evaluation result storage unit 27 By doing so, it has a function of calculating an evaluation accuracy rate regarding the assumed threshold.

At this time, the evaluation accuracy rate calculation unit 31 compares the presence / absence of video quality degradation obtained for each of these hypothetical thresholds with the quality evaluation result under the communication condition read from the quality evaluation result storage unit 27, and both evaluations are performed. When both indicate deterioration or no deterioration, that is, when both evaluations match, it is determined that the estimated evaluation based on the assumed threshold is correct. In addition, when one of the two evaluations shows deterioration and the other shows no deterioration, that is, when the two evaluations do not coincide with each other, it is determined that the estimated evaluation based on the assumed threshold is correct.
Thereafter, the evaluation correct answer rate calculation unit 31 calculates the correct answer rate for each assumed threshold number by dividing the correct answer number by the number of condition-specific packet information.

  The network quality control threshold value determining unit 32 compares the correct answer rate for each assumed threshold number obtained by the evaluation correct answer rate calculating unit 31, and calculates the assumed unit time and the assumed allowable received packet number of the assumed threshold number having the highest correct answer rate. The function of determining the unit quality and the allowable number of received packets indicating the network quality control threshold, and the unit time and the allowable number of received packets are stored in the network quality control threshold storage unit 14 of the user experience quality estimation system 10 shown in FIG. It has the function to do.

  The network quality control threshold value calculation device 20 has functions such as an arithmetic processing unit, a storage unit, a data communication interface unit, a data input / output interface unit, and an operation input unit as a general configuration provided in a normal information processing apparatus. The functional units described above perform processing operations using these units.

Among the functional units of the network quality management threshold calculation device 20 described above, the packet information acquisition unit 21, the terminal information acquisition unit 23, the steady value calculation unit 24, the quality evaluation result acquisition unit 26, the assumed threshold generation unit 30, and the evaluation accuracy rate The calculation unit 31 and the network quality management threshold value determination unit 32 may be realized by an arithmetic processing unit.
The arithmetic processing unit includes a microprocessor such as a CPU and its peripheral circuits, and implements various functional units by reading and executing a program stored in advance in the storage unit.

  Of the functional units of the network quality management threshold value calculation device 20, the packet information storage unit 22, the steady value storage unit 25, and the quality evaluation result storage unit 27 may be realized by a storage device such as a hard disk or a semiconductor memory. . Each of these storage units may be realized by separate storage devices, or any one or a plurality of storage units may be realized by one storage device.

[Operation of Second Embodiment]
Next, the operation of the network quality management threshold value calculation apparatus 20 according to the present embodiment will be described.
Main processing operations in the network quality management threshold value calculation apparatus 20 include an information acquisition process, a network quality management threshold value determination process, and an evaluation accuracy rate calculation process.

[Information acquisition processing]
First, the information acquisition process in the network quality management threshold value calculation apparatus 20 according to the present embodiment will be described with reference to FIG. FIG. 13 is a flowchart illustrating information acquisition processing in the network quality management threshold value calculation apparatus according to the second embodiment.

  This information acquisition process is executed when video communication is experimentally performed between the video distribution device 1 and the receiving terminal 3 under a plurality of communication conditions with different amounts of jitter. At this time, the network quality control device 6 may vary the jitter amount, detect in advance the jitter amount at which the video quality starts to deteriorate, select several jitter amounts before and after that, and set the communication conditions.

  In the information acquisition process of FIG. 13, the network quality management threshold value calculation device 20 first performs a test between the video distribution device 1 and the receiving terminal 3 under a plurality of communication conditions with different jitter amounts by the packet information acquisition unit 21. For the video communication to be performed in an automatic manner, the condition-specific packet information regarding the video communication packet related to the video communication captured by the branching unit 5 is acquired, and the acquired condition-specific packet information is stored in the packet information storage unit 22 (step 200). ).

  Next, the network quality management threshold value calculation device 20 uses the quality evaluation result acquisition unit 26 for experimental video communication when the packet information by condition acquired by the packet information acquisition unit 21 is obtained for each communication condition. The quality evaluation result of the video reproduced by the receiving terminal 3 is acquired, and the acquired quality evaluation result is stored in the quality evaluation result storage unit 27 (step 201).

Further, the network quality management threshold value calculation device 20 acquires various terminal information used for video communication at the receiving terminal 3 by the terminal information acquisition unit 23 (step 202).
Subsequently, the network quality management threshold value calculation device 20 uses the steady value calculation unit 24 to calculate the steady unit time and the number of steady received packets during steady video communication in which the video quality at the receiving terminal 3 is not degraded as a steady value. And it preserve | saves to the steady value memory | storage part 25 (step 203).

  At this time, the steady value calculation unit 24 calculates, as a steady unit time, a frame reproduction required time required to reproduce one frame of video at the receiving terminal 3 from the video frame display rate acquired by the terminal information acquisition unit 23. . Further, out of the condition-specific packet information stored in the packet information storage unit 22, the receiving terminal per unit time based on the steady packet information at the time of steady video communication in which the video quality at the receiving terminal 3 is not degraded. In step 3, the number of stationary reception packets received is calculated.

FIG. 14 is a configuration example of steady values. Among the steady values, the steady unit time can be calculated by the reciprocal of the video frame display rate acquired by the terminal information acquisition unit 23. Regarding the number of steady received packets, among each packet included in the steady packet information, a time interval of a steady unit time length is provided in order from the transfer time of the first packet, and the transfer time is included in the time interval for each of these time intervals. The number of received packets may be calculated by counting the number of packets received and averaging the number of packets in each time interval.
Thereby, the network quality management threshold value calculation apparatus 20 ends a series of information acquisition processing.

[Network quality control threshold determination processing]
Next, with reference to FIG. 15, the network quality management threshold value determination process in the network quality management threshold value calculation apparatus 20 according to the present embodiment will be described. FIG. 15 is a flowchart illustrating network quality management threshold value determination processing in the network quality management threshold value calculation apparatus according to the second embodiment.

  In the network quality management threshold value determination process of FIG. 15, the network quality management threshold value calculation device 20 first preliminarily determines the assumption threshold value generation unit 30 based on the steady unit time and the number of steady received packets read from the steady value storage unit 25. A plurality of sets of the assumed unit time and the assumed allowable number of received packets are generated as assumed threshold values from within the determined verification range (step 210).

  FIG. 16 is a configuration example of an assumed threshold value. Here, as the assumed unit time, the verification range is up to nine times the steady unit time, and nine types of values obtained by the steady unit time × n (n is an integer of 1 to 9) are used. Further, regarding the assumed allowable number of received packets, the verification range is increased from 80% of the number of steady received packets to 100% in increments of 5%, and the steady unit time × (n + x) (x is 80%, 85%, 90%) , 95%, or 100%) are used. As a result, the assumed threshold value is 45 (= 9 × 5).

Next, the network quality management threshold value calculation device 20 starts an assumption threshold loop in which the evaluation accuracy rate calculation unit 31 repeatedly executes steps 212 and 113 described later for each assumption threshold value generated by the assumption threshold value generation unit 30 ( Step 211).
In this assumption threshold loop, the evaluation accuracy rate calculation unit 31 first acquires the assumption threshold selected at the head of the assumption threshold loop from the assumption threshold generation unit 30 (step 212), and the evaluation accuracy rate based on this assumption threshold In order to calculate the evaluation correct answer sequence described later (step 213).

In this way, after calculating the evaluation accuracy rate by the evaluation accuracy rate calculation unit 31 for all the assumption thresholds generated by the assumption threshold generation unit 30, the network quality management threshold value calculation device 20 performs the network quality management threshold value determination unit 32. Thus, the correct answer rates of these assumed threshold numbers are compared (step 214).
Then, the network quality management threshold value determination unit 32 determines the assumed unit time and the assumed allowable received packet number of the assumed threshold number having the highest accuracy rate as the unit time and the allowable received packet number indicating the network quality management threshold value (step 215). ).

  Thereafter, the network quality management threshold value determination unit 32 stores the network quality management threshold value including the unit time and the number of allowable received packets in the network quality management threshold value storage unit 14 of the user experience quality estimation system 10 (step 216). A series of network quality control threshold value determination processing ends.

[Evaluation accuracy rate calculation processing]
Next, an evaluation accuracy rate calculation process in the network quality management threshold value calculation apparatus 20 according to the present embodiment will be described. FIG. 17 is a flowchart showing an evaluation accuracy rate calculation process in the network quality management threshold value calculation apparatus according to the second embodiment.

  In the evaluation accuracy rate calculation processing of FIG. 17, the network quality management threshold value calculation device 20 first executes a communication condition loop in which the evaluation accuracy rate calculation unit 31 repeatedly executes steps 221 to 128 described later for each communication condition. (Step 220).

  In this hypothetical threshold loop, the evaluation accuracy rate calculation unit 31 first reads out the condition-specific packet information corresponding to the communication condition selected at the head of the communication condition loop from the packet information storage unit 22 (step 221), and the evaluation The number of packets is counted for each virtual unit time selected at the start of the accuracy rate calculation process (step 222).

  At this time, the evaluation accuracy rate calculation unit 31 sets time intervals in order for each virtual unit time from the first packet described in the packet information according to the condition, and packets transferred within the time interval for each time interval. The number may be counted. In addition, for each packet described in the condition-specific packet information, a time interval may be set using the transfer time of the packet as a start time.

  After that, the evaluation accuracy rate calculation unit 31 verifies whether there is a time interval in which the counted packets exceed the number of assumed allowable received packets selected at the time of starting the evaluation accuracy rate calculation process. (Step 223).

  As a result of the verification, if there is a time interval exceeding the number of assumed allowable received packets (step 223: YES), the evaluation accuracy rate calculating unit 31 estimates and evaluates that the video quality is deteriorated (step 224), and assumes the allowable allowable received packets. When there are no time intervals exceeding the number (step 223: NO), the evaluation accuracy rate calculation unit 31 estimates and evaluates that there is no deterioration of the video quality (step 225).

After estimating the presence or absence of video quality deterioration in this way, the evaluation accuracy rate calculation unit 31 reads the quality evaluation result corresponding to the communication condition selected in the loop processing from the quality evaluation result storage unit 27 (step 226). ) Compare this quality evaluation result with the estimated evaluation result (step 227).
Here, when both the evaluation results indicate that there is deterioration or no deterioration and both evaluation results match (step 227: YES), the evaluation accuracy rate calculation unit 31 performs the estimation evaluation based on the assumed threshold value. Is the correct answer, the number of correct answers for the estimated evaluation related to the virtual threshold selected at the start of the evaluation correct rate calculation process is counted by 1 (step 228).

  On the other hand, when one of the two evaluations shows deterioration and the other shows no deterioration, and the evaluation results of the two do not match (step 227: NO), the evaluation accuracy rate calculation unit 31 performs the estimation evaluation based on the assumed threshold value. Because the answer is incorrect, the number of correct answers is not counted.

In this way, after performing the process of counting the number of correct answers in accordance with the correct / incorrect answer of the estimated evaluation, the evaluation correct rate calculating unit 31 ends the loop process related to the communication condition selected in Step 220 and proceeds to Step 220. Returning to the next loop processing for the next communication condition.
Further, when the loop processing regarding all communication conditions is completed, the evaluation accuracy rate calculation unit 31 ends a series of evaluation accuracy rate calculation processing.

  FIG. 18 is an example of the calculation result of the evaluation accuracy rate. FIG. 19 is a graph showing a calculation result of the evaluation accuracy rate. Here, when the assumed unit time is 33 ms and the assumed allowable received packet rate is 85%, the evaluation accuracy rate is 100%, which is the largest. Therefore, in this example, unit time = 33 ms and assumed allowable number of received packets = 94 are selected as the network quality management threshold.

[Effect of the second embodiment]
As described above, in the present embodiment, the steady-state value calculation unit 24 uses the steady-state packet information in the steady-state video communication in which the video quality at the receiving terminal 3 is not deteriorated among the condition-specific packet information. The number of stationary received packets received per stationary unit time, which is the frame reproduction required time for one frame, is calculated, and the assumption threshold generation unit 30 performs a predetermined verification based on the stationary unit time and the number of stationary received packets A plurality of sets of assumed unit time and assumed allowable number of received packets are generated as assumption threshold values from within the range.

  Then, the evaluation correct answer rate calculation unit 31 verifies the packet transfer status included in each condition-specific packet information based on the assumed unit time and the assumed allowable number of received packets for each assumed threshold value. By estimating and evaluating the presence / absence of video quality degradation in the network, and comparing the estimated evaluation with the quality evaluation result in the communication condition, the evaluation accuracy rate for the hypothetical threshold is calculated, and the network quality management threshold determining unit 32 Of the estimated correct answer rates of the assumed threshold value, the assumed unit time and the assumed allowable number of received packets for the assumed threshold value with the highest correct answer rate are determined as the network quality control threshold value for the network.

Therefore, since unit time and the number of allowable received packets are used as the network quality control threshold, it is verified whether or not the video communication packet transferred in actual video communication is transferred within the allowable number of received packets. Thus, it is determined whether or not quality degradation has occurred for the video communication.
As a result, it is possible to calculate a network quality management threshold considering the influence on the user experience quality due to a buffer overrun phenomenon in which the packet arrives at the receiving terminal 3 earlier than the scheduled time.

  In the present embodiment, the steady value calculation unit 24 calculates the number of steady received packets for each content received per steady unit time at the receiving terminal 3 for each of a plurality of video communications with different contents, and the steady steady for each content. An average value of the number of received packets may be calculated as the number of stationary received packets, and a network quality control threshold from which dependency on content is eliminated can be calculated.

Further, in the present embodiment, the steady value calculation unit 24 uses the steady unit time consisting of the frame playback required time for one frame at the receiving terminal 3, so when verifying each frame and the packet transfer status The packet delimiter can be almost synchronized.
Therefore, the evaluation accuracy rate calculation unit 31 can estimate and evaluate the presence / absence of video quality degradation under the same conditions as the video quality evaluation at the receiving terminal 3, and can improve the calculation accuracy of the network quality management threshold value. .

Further, in the present embodiment, the assumed threshold value generation unit 30 generates each assumed unit time by individually multiplying the steady unit time by a plurality of integers specified in the verification range, and further, When calculating the number of assumed allowable received packets corresponding to each assumed unit time, the number of stationary received packets is multiplied by the integer used to calculate the assumed unit time, and this multiplication result is multiplied by a plurality of values specified in the verification range. A plurality of assumed unit times are generated for each assumed unit time by individually multiplying the respective multiplication results by the respective ratios, so that each frame and packet for verifying the packet transfer status The break can be almost synchronized.
Therefore, the evaluation accuracy rate calculation unit 31 can estimate and evaluate the presence / absence of video quality degradation under the same conditions as the video quality evaluation at the receiving terminal 3, and can improve the calculation accuracy of the network quality management threshold value. .

[Third Embodiment]
Next, a user experience quality estimation model creation device according to a third embodiment of the present invention will be described with reference to FIG. FIG. 20 is a block diagram illustrating a configuration of a user experience quality estimation model creation device according to the third embodiment.

The user experience quality estimation model creation device 40 as a whole is composed of an information processing communication terminal such as a general server device or workstation, and receives packet information obtained by video communication performed in advance on a trial basis and reception at that time. It has a function of creating a user experience quality estimation model for estimating a user experience quality value corresponding to the packet loss rate measured from any video communication based on the video quality evaluation at the terminal 3.
Note that the video distribution device 1, packet network 2, receiving terminal 3, video display device 4, branching device 5, and network quality control device 6 in FIG. 20 have the same or similar configurations as those in FIGS. Therefore, detailed description thereof is omitted here.

[Configuration of Third Embodiment]
Next, with reference to FIG. 20, the structure of the user experience quality estimation model creation apparatus 40 concerning this Embodiment is demonstrated in detail.
As shown in FIG. 20, the user experience quality estimation model creation device 40 includes, as main functional units, a packet information acquisition unit 41, a packet information storage unit 42, a packet loss rate calculation unit 43, a quality evaluation result acquisition unit 44, A quality evaluation result storage unit 45 and an estimated model creation unit 46 are provided.

For each test video communication performed between the video distribution device 1 and the receiving terminal 3 under a plurality of communication conditions with different packet loss numbers generated by the network quality control device 6, the packet information acquisition unit 41 A function of acquiring packet information related to video communication packets related to the test video communication captured by the branching device 5 and the acquired packet information to the test video communication ID unique to each test video communication to the packet information storage unit 42 And a function to save.
In the packet information, as shown in FIG. 10, the packet number (sequence number) of the packet for each video communication packet that has passed through at least the branching device 5 and transferred from the packet network 2 to the receiving terminal 3 is included. The transfer time is included.

  The packet loss rate calculation unit 43 confirms the packet number (sequence number) of the packet information of each test video communication read from the packet information storage unit 42, and determines the total number of transmitted packets from the packet number difference between the first packet and the last packet. For each test video communication, and also confirms the packet number, detects a discontinuous part (jump number) not recorded as packet information as a lost packet lost in the packet network 2, and these loss It has a function of counting the number of packets for each test video communication and a function of calculating a packet loss rate for each test video communication by dividing the number of packet losses by the total number of transmitted packets.

The quality evaluation result acquisition unit 44 has a function of acquiring the quality evaluation result of the video reproduced by the receiving terminal 3 for each test communication for which the packet information has been acquired by the packet information acquisition unit 41, and the acquired quality evaluation result as the quality evaluation result. And a function of saving in the storage unit 45.
FIG. 21 is a configuration example of the quality evaluation result. Here, a test video communication ID unique to each test video communication and an evaluation value (MOS) quantified by 5-step evaluation or the like for the test video communication are registered as a set.

  The estimation model creation unit 46 uses the packet loss rate of each test video communication obtained by the packet loss rate calculation unit 43 and the quality evaluation result of each test video communication read from the quality evaluation result storage unit 45 to each test video. FIG. 1 shows a function for deriving a correspondence relationship between the packet loss rate and the user experience quality as a user experience quality estimation model by using a test video communication ID unique to communication, and this user experience quality estimation model. And the function of saving in the estimated model storage unit 17 of the user experience quality estimation system 10.

  The user experience quality estimation model creation device 40 includes a general configuration provided in a normal information processing device, such as an arithmetic processing unit, a storage unit, a data communication interface unit, a data input / output interface unit, and an operation input unit. Functional units are provided, and each functional unit described above performs processing operations using these functional units.

Among the functional units of the user experience quality estimation model creation device 40 described above, the packet information acquisition unit 41, the packet loss rate calculation unit 43, the quality evaluation result acquisition unit 44, and the estimation model creation unit 46 are the arithmetic processing unit. Realize it.
The arithmetic processing unit includes a microprocessor such as a CPU and its peripheral circuits, and implements various functional units by reading and executing a program stored in advance in the storage unit.

  Of the functional units of the user experience quality estimation model creation device 40, the packet information storage unit 42 and the quality evaluation result storage unit 45 may be realized by a storage device such as a hard disk or a semiconductor memory. Each of these storage units may be realized by separate storage devices, or any one or a plurality of storage units may be realized by one storage device.

[Operation of Third Embodiment]
Next, the operation of the user experience quality estimation model creation device 40 according to the present embodiment will be described with reference to FIG. FIG. 22 is a flowchart illustrating a user experience quality estimation model creation process of the user experience quality estimation model creation device according to the third embodiment.

  In the user experience quality estimation model creation process of FIG. 22, the user experience quality estimation model creation device 40 first uses the packet information acquisition unit 41 to perform the video distribution device 1 and the receiving terminal under a plurality of communication conditions with different numbers of packet losses. 3, for the test video communication performed on a trial basis with 3, packet information regarding the video communication packet related to the test video communication captured by the branching unit 5 is acquired, and the acquired packet information is stored in the packet information storage unit 42. (Step 300).

Next, in the user experience quality estimation model creation device 40, the packet loss rate calculation unit 43 calculates the packet loss rate for each packet information of each test video communication read from the packet information storage unit 42 (step 301).
Further, the user experience quality estimation model creation device 40 acquires the quality evaluation result of the video reproduced by the receiving terminal 3 for each test communication for which the packet information acquisition unit 41 has acquired the packet information by the quality evaluation result acquisition unit 44. The quality evaluation result is stored in the quality evaluation result storage unit 45 (step 302).

  Subsequently, the user experience quality estimation model creation device 40 uses the estimation model creation unit 46 to read the packet loss rate of each test video communication obtained by the packet loss rate calculation unit 43 and the quality evaluation result storage unit 45. By associating the test video communication quality evaluation results with the test video communication ID unique to each test video communication, the correspondence between the packet loss rate and the user experience quality as shown in FIG. An approximate function expression of the regression curve shown is derived as a user experience quality estimation model (step 303).

  Thereafter, the user experience quality estimation model creation device 40 stores the derived user experience quality estimation model in the estimation model storage unit 17 of the user experience quality estimation system 10 by the estimation model creation unit 46 (step 304). This completes the user experience quality estimation model creation process.

[Effect of the third embodiment]
Thus, in the present embodiment, the packet loss rate calculation unit 43 calculates the packet loss rate from the packet information obtained from each test video communication, and the quality evaluation result acquisition unit 44 calculates the packet loss rate for each test video communication. The quality evaluation result of the video reproduced by the receiving terminal 3 is acquired, and the estimation model creation unit 46 uses the packet loss rate and the quality evaluation result to determine the correspondence between the packet loss rate and the user experience quality. Therefore, it is possible to easily create a user experience quality estimation model for estimating the user experience quality from the packet loss rate that can be measured from the packet network 2.

[Extended embodiment]
The present invention has been described above with reference to the embodiments, but the present invention is not limited to the above embodiments. Various changes that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention.

  In the first embodiment, each functional unit may be realized by one information processing communication terminal, but each functional unit may be realized by a separate information processing communication terminal. For example, the packet information acquisition unit 11, the packet information storage unit 12, the lost packet detection unit 13, the network quality management threshold storage unit 14, and the discarded packet estimation unit 15 may be realized by a quality measuring device connected to the receiving terminal 3. Alternatively, it may be realized by the receiving terminal 3.

  Further, the remaining discarded packet estimation unit 15, the packet loss rate calculation unit 16, the estimation model storage unit 17, the user experience quality estimation unit 18, and the screen display unit 19 are realized by a service quality management device managed by a communication service provider. Then, the user experience quality may be estimated based on the total number of transmitted packets, the number of lost packets, and the number of discarded packets notified from the quality measuring device or the receiving terminal 3 by data communication.

  Further, in the second embodiment, when the steady value calculation unit 24 calculates the steady received packet number, the steady received packet number is calculated as an example from the video communication for which the condition-specific packet information is measured. Although described, the present invention is not limited to this.

For example, for each of a plurality of video communications with different contents, the number of steady received packets for each content received by the receiving terminal 3 per steady unit time is calculated, and the average value of the steady received packets for each content is calculated as the number of steady received packets. Alternatively, it is possible to calculate a network quality control threshold from which the dependency on the content is excluded.
Further, depending on the service provided, instead of the average value, the minimum value of the number of steady received packets for each content may be selected as the number of steady received packets based on the service policy.

  In the third embodiment, the case where the regression function approximate function expression as shown in FIG. 4 is derived as the user experience quality estimation model has been described as an example. However, the present invention is not limited to this. Absent. For example, it may be a conversion table indicating the correspondence between the packet loss rate and the user experience quality value.

In the second embodiment, the calculation method of the network quality management threshold used in the first embodiment has been described. However, the calculation method of the network quality management threshold is limited to the second embodiment. The network quality control threshold value may be calculated by other methods.
In the third embodiment, the method for creating the user experience quality estimation model used in the first embodiment has been described. The method for creating the user experience quality estimation model is described in the third embodiment. The user experience quality estimation model may be created by other methods without limitation.

  DESCRIPTION OF SYMBOLS 1 ... Video distribution apparatus (transmission terminal), 2 ... Packet network, 3 ... Reception terminal, 4 ... Video display apparatus, 5 ... Branching device, 6 ... Network quality control apparatus, 10 ... User experience quality estimation system, 11 ... Packet information Acquisition unit, 12 ... Packet information storage unit, 13 ... Lost packet detection unit, 14 ... Network quality control threshold storage unit, 15 ... Discarded packet estimation unit, 16 ... Packet loss rate calculation unit, 17 ... Estimated model storage unit, 18 ... User experience quality estimation unit, 19 ... screen display unit, 20 ... network quality control threshold value calculation device, 21 ... packet information acquisition unit, 22 ... packet information storage unit, 23 ... terminal information acquisition unit, 24 ... steady value calculation unit, 25 ... steady value storage unit, 26 ... quality evaluation result acquisition unit, 27 ... quality evaluation result storage unit, 30 ... assumption threshold generation unit, 31 ... evaluation accuracy rate calculation unit, 32 ... network quality control threshold determination unit, 33 ... screen display Part, 40 User bodily sensation quality estimation model creation device, 41 ... packet information acquisition unit, 42 ... packet information storage unit, 43 ... packet loss rate calculation unit, 44 ... quality evaluation result acquisition unit, 45 ... quality evaluation result storage unit, 46 ... estimation model Creation department.

Claims (6)

A user experience quality estimation system for estimating a user experience quality perceived by a user with respect to a video received and reproduced by the reception terminal for video communication performed between a transmission terminal and a reception terminal via a packet network,
A packet information storage unit for storing packet information indicating a packet transfer time measured for each packet for video communication transferred from the packet network to the receiving terminal;
A network quality management threshold value storage unit that stores an allowable number of received packets indicating the number of packets that can be received by the receiving terminal per unit time as a network quality management threshold value;
An estimated model storage unit for storing a user experience quality estimation model indicating a relationship between a packet loss rate indicating a packet transfer state of video communication and a user experience quality value obtained by the video communication at that time;
A lost packet detector for detecting lost packets lost in the packet network based on the packet information and counting the number of lost packets;
The number of transfer packets is counted per unit time based on the transfer time information of each packet included in the packet information, and the number of packets exceeding the allowable number of received packets is estimated as the number of discarded packets discarded at the receiving terminal. A discarded packet estimation unit,
Packet loss rate for calculating a packet loss rate by dividing the total of the number of lost packets and the number of discarded packets by the total number of transmitted packets transmitted from the transmitting terminal to the receiving terminal calculated from the packet information A calculation unit;
A user experience quality estimation system comprising: a user experience quality estimation unit that estimates a user experience quality value corresponding to the packet loss rate with reference to the user experience quality estimation model. In the user experience quality estimation system according to claim 1,
A network quality control threshold value calculation device for calculating the network quality control threshold value;
The network quality control threshold value calculation device comprises:
By condition indicating the packet transfer time measured for each video communication packet transferred from the packet network to the receiving terminal for each of the video communications performed under a plurality of communication conditions with different amounts of jitter in the packet network A packet information storage unit for storing packet information;
For each of the communication conditions, a quality evaluation result storage unit that stores a quality evaluation result of the video reproduced by the receiving terminal;
Based on the steady packet information during steady video communication in which the video quality at the receiving terminal is not deteriorated among the packet information by condition, the number of steady received packets received per steady unit time set in advance is calculated. A steady value calculation unit;
Based on the stationary unit time and the stationary received packet number, an assumed threshold generation unit that generates a plurality of sets of assumed unit time and assumed allowable received packet number as a hypothetical threshold from within a predetermined verification range;
For each hypothetical threshold, based on the hypothetical unit time and the hypothetical allowable number of received packets, by verifying the packet transfer status included in each condition-specific packet information, the presence or absence of video quality degradation under the communication conditions An evaluation accuracy rate calculation unit that calculates an evaluation accuracy rate related to the assumed threshold value by performing estimation estimation and comparing the estimation evaluation with the quality evaluation result in the communication condition;
A network quality management threshold value determining unit that determines the assumed unit time and the assumed allowable number of received packets as the network quality control threshold value for the network among the estimated correct answer rates of the respective assumed threshold values. A user experience quality estimation system. In the user experience quality estimation system according to claim 1,
A user experience quality estimation model creating device for creating the user experience quality estimation model;
The user experience quality estimation model creation device,
For each test video communication performed under a plurality of communication conditions with different packet loss numbers in the packet network, a packet loss rate calculating unit that calculates a packet loss rate from packet information obtained from these test video communications,
A quality evaluation result acquisition unit for acquiring a quality evaluation result of the video reproduced by the receiving terminal for each test video communication;
A user experience quality, comprising: an estimation model creation unit that derives the user experience quality estimation model based on a correspondence relationship between the packet loss rate obtained for each test video communication and the quality evaluation result Estimation system. For video communication performed between a transmission terminal and a reception terminal via a packet network, a user used in a user experience quality estimation system that estimates a user experience quality perceived by a user with respect to an image received and reproduced by the reception terminal An experience quality estimation method,
A packet information storage step for storing packet information indicating a packet transfer time measured for each packet for video communication transferred from the packet network to the receiving terminal;
A network quality management threshold storage unit that stores an allowable number of received packets indicating the number of packets that can be received by the receiving terminal per unit time as a network quality management threshold;
An estimation model storage unit for storing a user experience quality estimation model indicating a relationship between a packet loss rate indicating a packet transfer status of video communication and a user experience quality value obtained by the video communication at that time; ,
A lost packet detection unit that detects lost packets lost in the packet network based on the packet information and counts the number of lost packets; and
The discard packet estimation unit counts the number of transfer packets per unit time based on the transfer time information of each packet included in the packet information, and the number of packets exceeding the allowable number of received packets is discarded by the receiving terminal. A discard packet estimation step for estimating the number of discarded packets;
The packet loss rate calculation unit divides the total of the number of lost packets and the number of discarded packets by the total number of transmitted packets transmitted from the transmitting terminal to the receiving terminal calculated from the packet information, thereby obtaining a packet loss A packet loss rate calculating step for calculating a rate;
A user experience quality estimation unit comprising: a user experience quality estimation unit that estimates a user experience quality value corresponding to the packet loss rate with reference to the user experience quality estimation model. . In the user experience quality estimation method according to claim 4,
The network quality control threshold value calculation device further comprises a network quality control threshold value calculation step for calculating the network quality control threshold value,
The network quality control threshold calculation step includes
Packets measured by the packet information storage unit for each video communication packet transferred from the packet network to the receiving terminal for each video communication performed under a plurality of communication conditions with different jitter amounts in the packet network A packet information storage step for storing condition-specific packet information indicating a transfer time;
A quality evaluation result storing step for storing a quality evaluation result of a video reproduced by the receiving terminal for each communication condition;
The stationary value calculation unit receives the stationary unit per preset stationary unit time based on the stationary packet information at the time of stationary video communication in which the video quality at the receiving terminal is not deteriorated among the packet information by condition. A steady value calculating step for calculating the number of received packets;
Assumed threshold generation unit generates a plurality of sets of assumed unit time and assumed allowable received packet number as assumption thresholds from a predetermined verification range based on the steady unit time and the number of stationary received packets. Steps,
The evaluation accuracy rate calculation unit verifies the packet transfer status included in each condition-specific packet information based on the assumed unit time and the assumed allowable number of received packets for each of the assumed thresholds. An evaluation accuracy rate calculation step for calculating an evaluation accuracy rate for the hypothetical threshold by estimating and evaluating the presence or absence of video quality degradation of the image, and comparing the estimation evaluation with the quality evaluation result in the communication conditions;
A network quality management threshold value determining unit determines an assumed unit time and an assumed allowable number of received packets as the network quality management threshold value for the network, with respect to the assumed threshold value with the highest accuracy rate among the evaluation correct answer rates of the respective assumed threshold values. And a management threshold value determination step. A user experience quality estimation method comprising: In the user experience quality estimation method according to claim 4,
The user experience quality estimation model creation device further includes a user experience quality estimation model creation step of creating the user experience quality estimation model,
The user experience quality estimation model creation step includes:
A packet loss rate calculation unit calculates a packet loss rate from packet information obtained from the test video communication for each test video communication performed under a plurality of communication conditions with different packet loss numbers in the packet network. A packet loss rate calculation step;
A quality evaluation result acquisition unit that acquires a quality evaluation result of a video reproduced by the receiving terminal for each test video communication; and
An estimation model creation unit comprising: an estimation model creation step of deriving the user experience quality estimation model based on a correspondence relationship between the packet loss rate obtained for each test video communication and the quality evaluation result. Characteristic user experience quality estimation method.

Images