JP5373704B2 - Multimedia quality estimation apparatus and multimedia quality estimation method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To determine a conversation type and to estimate a multimedia quality while taking the conversation type and utilization environment conditions into account. <P>SOLUTION: A multimedia quality estimation apparatus 2 has: a communication packet collection section 21 for collecting a packet exchanged between communication terminals 1; a voice information collection section 22 for extracting voice information from the packet; a video information collection section 23 for extracting video information from the packet; a delay quality estimation section 24 for estimating a delay quality; a voice quality estimation section 25 for estimating a voice quality; a task determination section 26 for determining a conversation type by extracting the feature amount of a conversation between the communication terminals 1 from the voice information; a video quality estimation section 27 for estimating a video quality; a utilization environment condition input section 28 for receiving utilization environment conditions of the communication terminals 1 from the outside; and a multimedia quality estimation section 29 for estimating a multimedia quality while taking the conversion type and the utilization environment conditions into account. <P>COPYRIGHT: (C)2012,JPO&amp;INPIT

Description

The present invention determines a conversation type representing a type of conversation performed between communication terminals in a video / audio communication service that transmits video and audio over a network, and uses the conversation type to determine the type of conversation in the video / audio communication service. The present invention relates to a multimedia quality estimation apparatus and a multimedia quality estimation method for estimating media quality.

In video distribution services, it is desired to establish a method for appropriately and promptly evaluating the quality of multimedia provided in order to realize the transmission and reception of high-quality video.
Conventionally, video quality evaluation is based on so-called subjective quality evaluation, which measures the quality that a user feels when actually viewing the video. However, subjective quality assessment requires dedicated equipment and enormous time and effort. Therefore, in order to perform video quality evaluation more efficiently, development of a technique for estimating subjective quality from the amount of features physically measured from video (hereinafter referred to as objective quality evaluation technique) is desired.

Conventional objective quality assessment technology has created a model that takes into account the relationship between multimedia quality, audio quality, video quality and delay quality, and the interaction between audio quality, video quality and delay quality. (For example, refer to Patent Document 1). Here, multimedia quality refers to the overall quality of communication services in consideration of audio quality, video quality, and delay quality.
Conventionally, the video / audio communication service is a video conference service. As a method for estimating multimedia quality, the same level of communication is performed in a one-to-one direction. It was assumed that it was being used.

JP-A-2005-244321

  However, the usage state of the video conference service varies depending on the purpose of use. For example, there are conferences for discussions that lead to conclusions, and other conferences that only send unilateral video and audio, such as lectures and lectures. In conferences where discussions are held, it seems that the situation of video movement and conversation is similar at two or more points connected by the video conference service. Even if the movement or utterance of the video is active on the side of the receiver, the movement of the utterance is almost absent on the receiver side.

  Considering the video quality, audio quality, and delay quality, which are the components in the multimedia quality estimation model, the audio quality weight is stronger than the video quality in a meeting with high work performance, and the delay quality weight Also become stronger. In addition, in the case of a lecture / lecture for the purpose of unilateral transmission, it is considered that the weight of video quality becomes stronger than the quality of audio and the weight of delay quality also becomes weaker.

  Furthermore, in addition to the differences in the communication situation, there are differences in the interactive usage environment. That is, when a communication terminal with a high-resolution display is used at one point and a communication terminal with a low-resolution display is used at the other point, weights for video quality and audio quality are different. It is considered that the lower the display resolution, the lower the importance for video quality and the higher the importance for audio quality.

The present invention has been made in view of the above, and an object of the present invention is to provide a conversation type that can determine a conversation type representing a form of conversation performed between communication terminals in a bidirectional video / audio communication service. To provide a determination device and a conversation type determination method.
Another object of the present invention is to provide a multimedia quality estimation apparatus and a multimedia quality estimation method capable of estimating multimedia quality in consideration of conversation type and usage environment conditions.

The present invention determines a conversation type representing a type of conversation performed between communication terminals in a video / audio communication service for transmitting video and audio over a network, and estimates a multimedia quality in the video / audio communication service. A media quality estimation device , a communication packet collecting means for collecting communication packets transmitted and received between communication terminals, a voice information collecting means for extracting voice information from the communication packets collected by the communication packet collecting means, A conversation type determining unit for extracting a conversation feature amount between communication terminals from the voice information extracted by the voice information collecting unit and determining a conversation type from the feature amount; and video information from the communication packet collected by the communication packet collecting unit. Video information collecting means for extracting video, and estimating the video quality from the video information extracted by the video information collecting means Image quality estimating means, sound quality estimating means for estimating sound quality from the sound information extracted by the sound information collecting means, delay quality estimating means for estimating delay quality from the extracted sound information and video information, A conversation type determined by the conversation type determination unit, a use environment condition indicating the capability of the communication terminal, a video quality estimation value calculated by the video quality estimation unit, and an audio quality estimation value calculated by the audio quality estimation unit; Multimedia quality estimating means for estimating multimedia quality, which is the overall quality of the communication service, based on the delay quality estimated value calculated by the delay quality estimating means, and the conversation type determining means includes the voice An exchange frequency calculating means for calculating the frequency of conversation exchange between communication terminals as the feature amount of the conversation from the voice information extracted by the information collecting means; Classification means for determining the conversation type by classifying the frequency of exchange according to a predetermined determination threshold, and the use environment condition indicating the capability of the communication terminal is the resolution or the screen size of the display device of the communication terminal. The multimedia quality estimation means, a coefficient table that pre-stores a coefficient determined in consideration of the interaction between the conversation type and the use environment condition for each conversation type and each use environment condition; The coefficient corresponding to the conversation type determined by the conversation type determination means and the usage environment condition input from the outside is acquired from the coefficient table, the acquired coefficient, the video quality estimated value, the audio quality estimated value, and the delay And multimedia quality calculating means for calculating a multimedia quality estimated value by substituting the quality estimated value into a predetermined quality estimation formula. Is.
In the configuration example of the multimedia quality estimation apparatus of the present invention, the exchange frequency calculating means calculates the frequency of the exchange from the occurrence state of voice packets.

Further, the multimedia quality estimation apparatus of the present invention is characterized in that, instead of the video quality estimation means, the audio quality estimation means, and the delay quality estimation means, a video quality estimation value from a quality information packet collected by the communication packet collection means, Quality information extracting means for extracting a speech quality estimate value and a delay quality estimate value, the multimedia quality estimation means, a conversation type determined by the conversation type determination means, a use environment condition representing the capability of the communication terminal, Based on the video quality estimation value extracted by the quality information extraction unit, the audio quality estimation value extracted by the quality information extraction unit, and the delay quality estimation value extracted by the quality information extraction unit, a comprehensive communication service It is characterized by estimating multimedia quality, which is quality .

In addition, the present invention determines a conversation type representing a conversation type performed between communication terminals in a video / audio communication service for transmitting video and audio over a network, and estimates multimedia quality in the video / audio communication service. A multimedia quality estimation method for collecting communication packets transmitted and received between communication terminals; a voice information collecting step for extracting voice information from the communication packets collected in the communication packet collecting step; From the voice information extracted in the voice information collection step, the feature amount of the conversation between the communication terminals is extracted, the conversation type determination step for determining the conversation type from the feature amount, and the communication packet collected in the communication packet collection step The video information collection step for extracting video information and the video information collection step A video quality estimation step for estimating video quality from the extracted video information, a voice quality estimation step for estimating audio quality from the audio information extracted in the audio information collection step, and a delay quality from the extracted audio information and video information A delay quality estimation step for estimating the conversation type, a conversation type determined in the conversation type determination step, a use environment condition indicating the capability of the communication terminal, a video quality estimation value calculated in the video quality estimation step, and the audio quality estimation A speech quality estimation value calculated in the step, and a multimedia quality estimation step for estimating a multimedia quality that is an overall quality of the communication service based on the delay quality estimation value calculated in the delay quality estimation step, The conversation type determining step includes the feature amount of the conversation from the voice information extracted in the voice information collecting step. The communication frequency calculation step for calculating the frequency of conversation exchange between the communication terminals, and the classification step for determining the conversation type by classifying the calculated frequency of exchange according to a predetermined discrimination threshold, The use environment condition representing the ability of the communication terminal represents the resolution or the screen size of the display device of the communication terminal, and the multimedia quality estimation step takes into account the interaction between the conversation type and the use environment condition. The coefficient corresponding to the conversation type determined in the conversation type determination step and the use environment condition input from the outside is referred to with reference to a coefficient table that stores the determined coefficient in advance for each conversation type and each use environment condition. Obtained from the coefficient table, the obtained coefficient, the video quality estimated value, the audio quality estimated value, and the delay quality estimated value It includes a multimedia quality calculating step of calculating a multimedia quality estimated value by substituting it into the quality estimation formula .
In the configuration example of the multimedia quality estimation method of the present invention, the exchange frequency calculating step calculates the frequency of the exchange from the occurrence state of the voice packet.

Further, the multimedia quality estimation method of the present invention is a video quality estimation value obtained from the quality information packet collected in the communication packet collection step, instead of the video quality estimation step, the audio quality estimation step, and the delay quality estimation step. A quality information extraction step for extracting a speech quality estimate value and a delay quality estimate value, wherein the multimedia quality estimation step includes a conversation type determined in the conversation type determination step, and a use environment condition indicating the capability of the communication terminal; Based on the video quality estimation value extracted in the quality information extraction step, the audio quality estimation value extracted in the quality information extraction step, and the delay quality estimation value extracted in the quality information extraction step, a comprehensive communication service It is characterized by estimating multimedia quality, which is quality .

  According to the present invention, communication packet collecting means for collecting communication packets transmitted and received between communication terminals, voice information collecting means for extracting voice information from communication packets collected by the communication packet collecting means, and voice information collecting means The conversation type can be determined by extracting the feature amount of the conversation between the communication terminals from the extracted voice information and providing the conversation type determination means for determining the conversation type from the feature amount. When the multimedia quality in such a bidirectional video / audio communication service is estimated, the estimation considering the conversation type can be performed.

  In the present invention, video quality estimation means for estimating video quality from video information, audio quality estimation means for estimating audio quality from audio information, and delay quality estimation means for estimating delay quality from audio information and video information And providing a multimedia quality estimation means for estimating the multimedia quality based on the conversation type determined by the conversation type determination device, the use environment condition, the video quality estimation value, the voice quality estimation value, and the delay quality estimation value, Highly accurate subjective quality estimation considering the conversation type and usage environment conditions can be realized, and multimedia quality corresponding to the usage status of the two-way communication service can be estimated with high accuracy. As a result, according to the present invention, it is possible to estimate multimedia quality with high accuracy in both cases of strong interactive conversations such as conferences and free conversations and weak interactive conversations such as lectures and lectures. Can do. In the present invention, even when a communication terminal with a high-resolution display is used at one point and a communication terminal with a low-resolution display is used at the other point, the multimedia quality is highly accurate. Can be estimated.

It is a block diagram which shows an example of the video / audio communication service system using the multimedia quality estimation apparatus which concerns on the 1st Embodiment of this invention. It is a flowchart which shows operation | movement of the multimedia quality estimation apparatus which concerns on the 1st Embodiment of this invention. It is a block diagram which shows the structure of the task judgment part in the multimedia quality estimation apparatus which concerns on the 1st Embodiment of this invention. It is a flowchart which shows operation | movement of the task judgment part in the multimedia quality estimation apparatus which concerns on the 1st Embodiment of this invention. It is a block diagram which shows the structure of the multimedia quality estimation part in the multimedia quality estimation apparatus which concerns on the 1st Embodiment of this invention. It is a flowchart which shows operation | movement of the multimedia quality estimation part in the multimedia quality estimation apparatus which concerns on the 1st Embodiment of this invention. It is a figure which shows the example of the coefficient table in the multimedia quality estimation apparatus which concerns on the 1st Embodiment of this invention. It is a block diagram which shows the example which provides the multimedia quality estimation apparatus in the communication terminal in the 2nd Embodiment of this invention.

[First Embodiment]
Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. FIG. 1 is a block diagram showing an example of a video / audio communication service system using the multimedia quality estimation apparatus according to the first embodiment of the present invention.
The video / audio communication service system performs quality management of a plurality of communication terminals 1, a plurality of multimedia quality estimation devices 2, a network 3 interconnecting the plurality of communication terminals 1, and a video / audio communication service system. And a quality control device 4. Each multimedia quality estimation device 2 is installed between the communication terminal 1 and the network 3.

  The multimedia quality estimation device 2 includes a communication packet collection unit 21 that collects communication packets transmitted and received between the communication terminals 1, an audio information collection unit 22 that extracts audio information from the communication packets, and video information from the communication packets. Between the video information collecting unit 23 to be extracted, the delay quality estimating unit 24 for estimating the delay quality from the audio information and the video information, the audio quality estimating unit 25 for estimating the audio quality from the audio information, and the communication terminal 1 from the audio information Are extracted, and a task determination unit 26 serving as a conversation type determination unit that determines a conversation type from the feature amount, a video quality estimation unit 27 that estimates video quality from video information, and a communication terminal 1 from the outside. A use environment condition input unit 28 that receives a use environment condition representing the ability of the user, and a multimedia quality estimation unit 29 that estimates multimedia quality. The communication packet collection unit 21, the voice information collection unit 22, and the task determination unit 26 constitute a conversation type determination device.

FIG. 2 is a flowchart showing the operation of the multimedia quality estimation apparatus 2. The communication packet collecting unit 21 collects communication packets transmitted and received between the communication terminals 1 via the network 3 (step S1 in FIG. 2).
The voice information collecting unit 22 extracts voice information (voice packet) from the communication packets collected by the communication packet collecting unit 21 (step S2 in FIG. 2).
The video information collection unit 23 extracts video information (video packet) from the communication packets collected by the communication packet collection unit 21 (step S3 in FIG. 2).

  In the present embodiment, “quality” in video quality, audio quality, delay quality and multimedia quality means user experience quality (QoE), which is an average obtained by subjective evaluation experiments. Opinion score (Mean Opinion Score: MOS) or objective quality evaluation value obtained by estimating MOS.

  The delay quality estimation unit 24 calculates the relative delay time between video and audio from the video information extracted by the video information collection unit 23 and the audio information extracted by the audio information collection unit 22, and estimates the delay quality (FIG. 2 step S4).

  The voice quality estimation unit 25 estimates the voice quality based on the voice packet extracted by the voice information collection unit 22 or the voice signal included in the voice packet (step S5 in FIG. 2). As a method for estimating the voice quality, for example, ITU-T (International Telecommunication Union Telecommunication Standardization Sector) recommendation P.I. A speech quality objective evaluation scale PESQ (Perceptual Evaluation of Speech Quality) algorithm according to 862 can be used.

  The task determination unit 26 measures the occurrence status of a voiced section for each communication terminal from the voice information extracted by the voice information collection unit 22, and determines the video / voice communication service system from the occurrence frequency of conversation exchange between the communication terminals 1. The conversation type representing the conversation format of the speaker who uses is determined (step S6 in FIG. 2). Conversation types include conferences, lectures (lectures), free conversations, and data verification.

As the feature amount of the conversation for determining the conversation type, for example, the literature “Ito, Kitawaki,“ Delay Quality Evaluation Method Focusing on Temporal Feature Quantity of Conversation Voice ”, Journal of Acoustical Society of Japan, Vol.43, No.11. , P. 851-857, 1987 ”is used. This exchange frequency R _n is obtained for each conversation the number of voice exchanges between a speaker A using one communication terminal 1 and a speaker B using the other communication terminal 1. The number of exchanges is calculated as follows as a value converted per unit time (for example, 1 minute).

Here, i is the conversation number, R _{nab (i)} is the number of exchanges between the speakers A and B in the conversation (i), T _m is the conversation time length, and t _m is the unit time (1 minute). The number of exchanges is the number of times that speaker A has finished speaking and switched to speaker B's speech, or the number of times speaker B has finished speaking and switched to speaker A's speech, that is, the number of times the speaker has switched. It is.

Then, the task determination unit 26 can determine the conversation type by thresholding the calculated exchange frequency R _n with the determination threshold disclosed in JP-A-2005-148225. Specifically, if the exchange frequency R _n is less than a predetermined value a (for example, 0), the conversation type is determined as “lecture”, and the exchange frequency R _n is a predetermined value a or more and a predetermined value b (a <b ), The conversation type is determined to be “conference”, and if the exchange frequency R _n is not less than the predetermined value b and less than the predetermined value c (b <c), the conversation type is determined to be “free conversation”. If the exchange frequency R _n is greater than or equal to the predetermined value c, the conversation type is determined as “data verification”.

  FIG. 3 is a block diagram illustrating the configuration of the task determination unit 26, and FIG. 4 is a flowchart illustrating the operation of the task determination unit 26. The task determination unit 26 includes a voiced interval detection unit 261 that measures the occurrence of a voiced segment for each communication terminal, an exchange count calculation unit 262 that calculates the number of conversation exchanges between the communication terminals 1, and the communication terminal 1. An exchange frequency calculation unit 263 that calculates the frequency of conversations between them, and a classification unit 264 that determines a conversation type from the frequency of exchanges.

First, the voiced section detection unit 261 measures the occurrence status of the voiced section for each communication terminal from the voice information extracted by the voice information collection unit 22 (step S261 in FIG. 4).
The number of exchanges calculation unit 262 calculates the number of exchanges, which is the number of times the speaker is switched, from the occurrence state of the voiced section (step S262).

Exchanging frequency calculating unit 263, using the exchange number that has been calculated, to calculate the frequency R _n exchange conversations per unit time by the equation (1) (step S263).
Then, the classification unit 264 determines the conversation type by classifying the calculated exchange frequency R _{n according} to the predetermined determination thresholds a, b, and c as described above (step S264). When the determination of the conversation type is completed, the process of the task determination unit 26 ends (YES in step S265).

As a method for measuring the exchange frequency R _n , there is also a method using voice packet information instead of voice signals. That is, ITU-T Recommendation G. 729, G.G. If silence compression technique described in Annex the G.723.1 is used, sent only voice packets, estimated from the in silent section packets are not sent, the frequency R _n interactions from occurrence of the voice packet can do.

  The conversation type can also be estimated by combining audio information and video information. In this case, the task determination unit 26 measures the transition of the temporal occurrence state of the video motion amount from the video information, and further measures the temporal occurrence state of the sound section. In the case of communication with strong bidirectionality, it is assumed that the degree of coincidence between the temporal occurrence state of the video motion amount and the temporal occurrence state of the sound section is high. For this reason, it is possible to estimate the frequency of conversation exchange using the degree of coincidence between the temporal occurrence state of the video motion amount and the temporal occurrence state of the voiced section.

  The video quality estimation unit 27 estimates the video quality based on the video packet extracted by the video information collection unit 23 or the video signal included in the video packet (step S7 in FIG. 2). As a method of estimating the video quality based on the video signal, ITU-T Recommendation J.I. 144, J. et al. 246, J.H. 247, J. et al. There is a method described in H.249. These methods are methods for estimating video quality using media signals or statistical information obtained from media signals.

  Next, the multimedia quality estimation unit 29 determines the conversation type determined by the task determination unit 26, the usage environment condition input from the usage environment condition input unit 28, and the video quality estimation value input from the video quality estimation unit 27. Based on the speech quality estimation value input from the speech quality estimation unit 25 and the delay quality estimation value input from the delay quality estimation unit 24, the multimedia quality is estimated (step S8 in FIG. 2).

FIG. 5 is a block diagram showing the configuration of the multimedia quality estimation unit 29, and FIG. 6 is a flowchart showing the operation of the multimedia quality estimation unit 29.
The multimedia quality estimation unit 29 includes a coefficient table 291 and a multimedia quality calculation unit 292.

  FIG. 7 is a diagram illustrating an example of the coefficient table 291 of the multimedia quality estimation unit 29. The coefficient table 291 is stored in advance by associating the usage environment condition indicating the conversation type and the capability of the communication terminal 1 with the coefficients α, β, γ, δ, ε, ψ, φ, μ of the multimedia quality estimation formula described later. It is what you are doing.

  The multimedia quality calculation unit 292 receives, for example, a usage environment condition from the user of the communication terminal 1 via the usage environment condition input unit 28 (step S291 in FIG. 6), and receives a numerical value representing the conversation type from the task determination unit 26. Then (step S292), the coefficients α, β, γ, δ, ε, ψ, φ, μ corresponding to the input usage environment condition and conversation type are acquired from the coefficient table 291 (step S293).

  The multimedia quality calculation unit 292 receives the video quality estimation value MOSv input from the video quality estimation unit 27 (step S294), and receives the audio quality estimation value MOSa input from the audio quality estimation unit 25 ( In step S295, the delay quality estimation value MOSd input from the delay quality estimation unit 24 is received (step S296), and these quality estimation values and coefficients α, β, γ, δ, ε, ψ, φ, μ Is substituted into the following multimedia quality estimation formula to calculate the multimedia quality estimate value MOSt (step S297), and the calculated multimedia quality estimate value MOSt is output (step S298).

As a multimedia quality estimation method for a two-way communication service, there is a method disclosed in Patent Document 1. In the method disclosed in Patent Document 1, the input to the multimedia quality estimation model is a video quality estimation value MOSv, an audio quality estimation value MOSa, and a delay quality estimation value MOSd. The multimedia quality estimation model is a model that assumes conversation. The multimedia quality estimation formula representing the relationship between the video quality estimated value MOSv, the audio quality estimated value MOSa, the delay quality estimated value MOSd, and the multimedia quality estimated value MOSt is as follows.
MOSt = α · MOSa + β · MOSv + γ · MOSd + δ · MOSa · MOSv
+ Ε · MOSa · MOSd + ψ · MOSv · MOSd
+ Φ · MOSa · MOSv · MOSd + μ (2)

The multimedia quality calculation unit 292 calculates the multimedia quality estimated value MOSt using the equation (2).
Above, the process of the multimedia quality estimation apparatus 2 is complete | finished.

  Here, how to obtain the coefficients α, β, γ, δ, ε, ψ, φ, μ set in advance in the coefficient table 291 will be described. First, a plurality of experimental conditions with different video quality, audio quality, delay quality, usage environment conditions, and conversation type are set, and a subjective evaluation experiment is performed in a laboratory or the like for each of these experimental conditions. Each quality can be controlled by using an experimental system that can control the communication environment of the video / audio communication service. In the subjective evaluation experiment, the subject conducts a conversation that simulates the assumed conversation type, and the video quality evaluation value MOSv, the audio quality evaluation value MOSa, the delay quality evaluation value MOSd, and the multimedia quality evaluation that is the overall quality. The value MOSt is obtained. Then, the optimum coefficients α, β, γ, δ, ε, ψ, φ, μ are obtained for each use environment condition and each conversation type by regression analysis using the data of these evaluation values and Expression (2). What should I do?

  In the example of FIG. 4, the conversation types are classified into four, and numbers 1, 2, 3, and 4 are assigned to, for example, conferences, lectures, free conversations, and data collation, respectively. In the example of FIG. 4, the use environment condition is the resolution of the display device of the communication terminal 1 that receives the video. Here, three stages of QVGA (Quarter Video Graphics Array), VGA (Video Graphics Array) / SD (Standard Definition), and HD (High Definition) are provided. Note that the use environment condition may represent the size of the screen of the display device.

  When considering video quality, audio quality, and delay quality, which are the components of the multimedia quality estimation model, in the case of a conference where the conversation type has a strong work performance, the weight for audio quality becomes stronger than the video quality. The weight of delay quality also increases. In addition, in the case of a lecture / lecture intended for unilateral transmission, it is considered that the weight of the video quality becomes stronger than the sound quality, and the weight of the delay quality becomes weaker.

  Furthermore, in addition to such a difference in conversation type, a difference in usage environment conditions also appears. That is, when a communication terminal with a high-resolution display is used at one point and a communication terminal with a low-resolution display is used at the other point, weights for video quality and audio quality are different. It is considered that the lower the display resolution, the lower the importance for video quality and the higher the importance for audio quality. The coefficient table 291 is derived in consideration of such characteristics.

[Second Embodiment]
In the first embodiment, the multimedia quality estimation device 2 is installed immediately before the communication terminal 1. However, the present invention is not limited to this, and if there is a communication packet collection unit different from the multimedia quality estimation device 2, The multimedia quality estimation apparatus 2 may be arranged at an arbitrary location on the network 3 to perform multimedia quality estimation.
In addition to notifying the communication terminal 1 of the quality estimation result by the multimedia quality estimation device 2, the quality management device 4 also notifies the quality management result to the quality management device 4 via the network 3. You may make it useful for.
Further, the multimedia quality estimation apparatus 2 may be used by being incorporated in the communication terminal 1 as shown in FIG.

[Third Embodiment]
In the first embodiment, the audio information and the video information are collected from the communication packets themselves transmitted and received between the communication terminals 1. However, the quality information packet in which the quality estimation results of the audio information and the video information are mounted. May be collected.

  For example, in the case of a two-way communication service transmitted and received by RTP (Real-time Transport Protocol) packets, RFC 3611 RTCP-XR established by IETF (Internet Engineering Task Force) can be used. The RTCP-XR packet stores network quality information and media quality information estimated from packet quality, and video quality information, audio quality information, and delay quality information can be extracted from this information.

  In the case of the present embodiment, instead of the delay quality estimation unit 24, the audio quality estimation unit 25, and the video quality estimation unit 27, a quality information extraction unit that extracts video quality information, audio quality information, and delay quality information from the RTCP-XR packet. May be provided in the multimedia quality estimation apparatus 2.

  Note that the multimedia quality estimation apparatus 2 according to the first to third embodiments can be realized by a computer having a CPU, a storage device, and an external interface, and a program for controlling these hardware resources. . In such a computer, the program for realizing the conversation type determination method and multimedia quality estimation method of the present invention is recorded in a recording medium such as a flexible disk, a CD-ROM, a DVD-ROM, or a memory card. Provided. The CPU writes the program read from the recording medium into the storage device, and executes the processes described in the first to third embodiments according to the program.

  The present invention can be applied to a technique for estimating multimedia quality in a video / audio communication service.

  DESCRIPTION OF SYMBOLS 1 ... Communication terminal, 2 ... Multimedia quality estimation apparatus, 3 ... Network, 4 ... Quality management apparatus, 21 ... Communication packet collection part, 22 ... Audio | voice information collection part, 23 ... Image | video information collection part, 24 ... Delay quality estimation part 25 ... Audio quality estimation unit, 26 ... Task judgment unit, 27 ... Video quality estimation unit, 28 ... Usage environment condition input unit, 29 ... Multimedia quality estimation unit, 261 ... Sound section detection unit, 262 ... Calculation of number of exchanges Part, 263 ... exchange frequency calculation part, 264 ... classification part, 291 ... coefficient table, 292 ... multimedia quality calculation part.

Claims (6)

Multimedia quality estimation apparatus for determining a conversation type representing a form of conversation between communication terminals in a video / audio communication service for transmitting video and audio over a network and estimating multimedia quality in the video / audio communication service Because
Communication packet collecting means for collecting communication packets transmitted and received between communication terminals;
Voice information collecting means for extracting voice information from the communication packets collected by the communication packet collecting means;
A conversation type determination unit that extracts a feature amount of conversation between communication terminals from the voice information extracted by the voice information collection unit, and determines a conversation type from the feature amount ;
Video information collecting means for extracting video information from the communication packets collected by the communication packet collecting means;
Video quality estimation means for estimating video quality from the video information extracted by the video information collection means;
Voice quality estimating means for estimating voice quality from the voice information extracted by the voice information collecting means;
Delay quality estimating means for estimating delay quality from the extracted audio information and video information;
A conversation type determined by the conversation type determination unit, a use environment condition indicating the capability of the communication terminal, a video quality estimation value calculated by the video quality estimation unit, and an audio quality estimation value calculated by the audio quality estimation unit; Multimedia quality estimation means for estimating multimedia quality, which is the overall quality of the communication service, based on the delay quality estimation value calculated by the delay quality estimation means,
The conversation type determination means includes
An exchange frequency calculating means for calculating the frequency of conversation exchange between communication terminals as the feature amount of the conversation from the voice information extracted by the voice information collecting means;
Classification means for determining the conversation type by classifying the calculated frequency of exchange according to a predetermined determination threshold;
The use environment condition representing the capability of the communication terminal represents the resolution or the screen size of the display device of the communication terminal,
The multimedia quality estimation means includes:
A coefficient table that pre-stores coefficients determined in consideration of the interaction between the conversation type and the use environment condition for each conversation type and each use environment condition;
The coefficient corresponding to the conversation type determined by the conversation type determination means and the usage environment condition input from the outside is acquired from the coefficient table, the acquired coefficient, the video quality estimated value, the audio quality estimated value, and the delay A multimedia quality estimation device, comprising: a multimedia quality calculation means for calculating a multimedia quality estimate by substituting the quality estimate into a predetermined quality estimation formula . The multimedia quality estimation apparatus according to claim 1 , wherein
The multimedia quality estimation apparatus, wherein the exchange frequency calculation means calculates the frequency of the exchange from the occurrence state of voice packets. The multimedia quality estimation apparatus according to claim 1 or 2 ,
Quality of extracting video quality estimation value, audio quality estimation value, and delay quality estimation value from the quality information packet collected by the communication packet collection means instead of the video quality estimation means, the audio quality estimation means, and the delay quality estimation means With information extraction means,
The multimedia quality estimation means includes a conversation type determined by the conversation type determination means, a use environment condition indicating the capability of the communication terminal, an estimated video quality value extracted by the quality information extraction means, and the quality information extraction. Multimedia quality estimation characterized in that multimedia quality, which is a total quality of a communication service, is estimated based on a speech quality estimation value extracted by the means and a delay quality estimation value extracted by the quality information extraction means apparatus. A multimedia quality estimation method for determining a conversation type representing a form of conversation between communication terminals in a video / audio communication service for transmitting video and audio over a network and estimating multimedia quality in the video / audio communication service Because
A communication packet collecting step for collecting communication packets transmitted and received between communication terminals;
A voice information collecting step for extracting voice information from the communication packets collected in the communication packet collecting step;
A conversation type determination step for extracting a conversation feature amount between communication terminals from the voice information extracted in the voice information collection step, and determining a conversation type from the feature amount ;
A video information collecting step of extracting video information from the communication packet collected in the communication packet collecting step;
A video quality estimation step for estimating video quality from the video information extracted in this video information collection step;
A voice quality estimation step of estimating voice quality from the voice information extracted in the voice information collection step;
A delay quality estimation step for estimating a delay quality from the extracted audio information and video information;
The conversation type determined in the conversation type determination step, the usage environment condition indicating the capability of the communication terminal, the video quality estimation value calculated in the video quality estimation step, and the audio quality estimation value calculated in the audio quality estimation step A multimedia quality estimation step for estimating a multimedia quality that is a total quality of the communication service based on the delay quality estimation value calculated in the delay quality estimation step,
The conversation type determination step includes
An exchange frequency calculation step for calculating the frequency of conversation exchange between communication terminals as the feature amount of the conversation from the voice information extracted in the voice information collection step;
A classification step of determining the conversation type by classifying the calculated frequency of exchange according to a predetermined determination threshold;
The use environment condition representing the capability of the communication terminal represents the resolution or the screen size of the display device of the communication terminal,
The multimedia quality estimation step includes:
The conversation type determined in the conversation type determination step and the external are determined by referring to a coefficient table in which coefficients determined in consideration of interaction between the conversation type and the usage environment condition are stored in advance for each conversation type and each usage environment condition. Is obtained from the coefficient table, and the obtained coefficient, the video quality estimation value, the audio quality estimation value, and the delay quality estimation value are substituted into a predetermined quality estimation expression. A multimedia quality estimation method comprising a multimedia quality calculation step of calculating a multimedia quality estimate . The multimedia quality estimation method according to claim 4 , wherein
The multimedia quality estimation method according to claim 1, wherein the exchange frequency calculating step calculates the frequency of the exchange based on a voice packet occurrence state. The multimedia quality estimation method according to claim 4 or 5 ,
Instead of the video quality estimation step, the audio quality estimation step, and the delay quality estimation step, the quality for extracting the video quality estimation value, the audio quality estimation value, and the delay quality estimation value from the quality information packet collected in the communication packet collection step Including an information extraction step,
In the multimedia quality estimation step, the conversation type determined in the conversation type determination step, a use environment condition indicating the capability of the communication terminal, the video quality estimation value extracted in the quality information extraction step, and the quality information extraction A multimedia quality estimation characterized by estimating a multimedia quality, which is a total quality of a communication service, based on the speech quality estimation value extracted in the step and the delay quality estimation value extracted in the quality information extraction step. Method.

Images