JP5491150B2 - Audio quality factor index calculation method, apparatus, and program - Google Patents

Description

  The present invention relates to an audio communication quality estimation technique, and more particularly to a technique for calculating an audio quality factor index indicating an influence of a lost audio packet on the audio quality of a reproduced audio signal.

In providing a stream-type multimedia communication service, it is important to measure the audio quality experienced by the user during service provision from the viewpoint of in-service quality management and user complaint management that manages the communication quality during service provision. is there.
The most accurate method for measuring the user experience quality is a subjective quality evaluation method that actually allows the subject to evaluate the experience quality, but the quality of the service being provided cannot be measured. Therefore, development of a technique for objectively measuring user experience quality is desired.

One objective quality evaluation method is a parametric model that gives a quality factor index as an input and estimates the user experience quality based on the relationship between the quality factor index and the user experience quality. The parametric packet layer model is a model that measures the quality factor index given as the input of this model based on the header information of the packet transmitted by multimedia communication, and is suitable for measuring the user experience quality during service provision .
In the conventional model, the IP packet loss rate is often used as a parameter indicating the influence of IP packet loss due to overflow of the communication network or the reception buffer (see, for example, Patent Document 1).

  FIG. 10 is an explanatory diagram showing a packet storage form of audio data. Here, the relationship between the audio data and the IP packet structure for storing the audio data in the data transmission in the IP / UDP / RTP / MPEG2-TS protocol structure is shown.

  As shown in FIG. 10, in the MPEG-2 system -TS (Transport Stream) protocol, audio data obtained by dividing a frame sequence obtained by encoding an audio (speech) signal is obtained for each fixed number of frames. A PES packet is formed with a PES (Packetized Elementary Stream) header. This PES packet is divided and stored in the payload of each TS packet, and a TS header is attached to each PES packet to form a TS packet having a fixed length of 188 bytes as a whole.

JP 2008-17087 A

  An audio TS packet in which such audio data is stored is stored in an IP packet together with a non-audio TS packet such as a video TS packet including video (video) data. Hereinafter, an IP packet including an audio TS packet is referred to as an audio packet.

  In multimedia communication, since audio data has a smaller data amount per unit time than video data, the number of audio TS packets transmitted per unit time is considerably smaller than that of video TS packets. In actual multimedia communication, at most one audio TS packet is included in an IP packet, and some IP packets do not include an audio TS packet.

  Therefore, even if an IP packet including a TS packet is lost, there is a high probability that the lost IP packet is not an audio packet. Therefore, the audio quality of the reproduced audio signal obtained by receiving and reproducing the audio data of the audio TS packet at the receiving terminal. Often does not affect.

  However, in the above-described conventional quality estimation technique, when calculating the IP packet loss rate of multimedia communication to be evaluated, among the IP packets of the multimedia communication, the calculation target is not the number of audio packet losses but the video packet loss rate. The number of losses including IP packets other than audio packets such as packets is used. Therefore, according to the conventional quality estimation technique, since the IP packet loss rate that does not accurately capture the loss situation regarding the audio packet is used as the quality factor index, the audio quality cannot be accurately estimated. There was a problem.

  Even if one audio packet is lost, the effect varies. When an audio packet is lost, one audio TS packet included therein is lost. At this time, if the frame data is included in the lost audio TS packet, all these frames are affected by the loss.

  As shown in FIG. 10 described above, regarding the effect on the frame when loss X and loss Y occur in the audio TS packet, in the loss X in which one audio TS packet is lost, the frame F1, Although the two frames F2 are affected, the loss Y in which two audio TS packets are lost affects only one frame F3. In particular, for loss Y, only one frame of frame F3 is affected even if any one audio TS packet is lost.

  Thus, the number of lost audio TS packets does not necessarily correspond to the number of frames affected by this, and multiple frames are affected even if only one audio TS packet is lost. There is also. Accordingly, since each audio packet includes one audio TS packet, the influence of one audio packet loss on audio data is not the same.

  For this reason, even if the loss rate of audio packets is used as a quality factor indicator in the conventional quality estimation technique described above, the effect of the loss of audio packets on the audio data is not properly captured. There was a problem that the quality could not be estimated accurately.

  The present invention is for solving such problems, and provides an audio quality factor index calculation technique capable of calculating an audio quality factor index that appropriately captures the effect of loss of audio packets on the audio quality of a reproduced audio signal. The purpose is to do.

In order to achieve such an object, an audio quality factor index calculation method according to the present invention divides a frame sequence obtained by encoding an audio signal into a plurality of audio data, and each of the audio data is divided into audio packets. For multimedia communication that is stored and transmitted, based on the audio packet captured from the multimedia communication, an audio indicating the effect of the loss of the audio packet on the audio quality of the reproduced audio signal received and played back via the multimedia communication an audio quality factor index calculating method of calculating the quality factor index, among the rows of audio packets captured from the receiver of the multimedia communication, for one or more consecutive lost section lost audio packet, the O Loss by multiplying the packet analysis step of extracting Iopaketto number of lost and an average packet time length and the audio packet loss number of audio packets obtained by dividing the capture time of these audio packets by the total number of audio packets captured A loss-affected frame that calculates the packet loss time length of a section and calculates the number of frames affected by the audio data lost in the loss section from the relationship between the packet loss time length and the unit frame time length of the frame. And a loss effect time length calculation step of calculating a loss effect time length in the reproduced audio signal as an audio quality factor index based on the number of loss effect frames.

  At this time, if there are multiple loss intervals in the loss influence frame number calculation step, the number of loss influence frames is calculated for each loss interval, and the loss influence time length calculation step is performed for each loss interval. The loss influence time length may be calculated based on the number of affected frames, and the total loss influence time obtained by accumulating these loss influence time lengths may be calculated as an audio quality factor index.

  At this time, if the start time position of the audio data coincides with the start time position of the frame at a constant period in the loss influence frame number calculation step, the packet loss time length M, the average packet time length U, the period P, and the unit frame From the time length F, the number W of loss-affected frames may be calculated based on W = ceil (M / F) + (M% F) × {1−U / (P × F)}.

  Alternatively, when the start time position of the audio data does not periodically coincide with the start time position of the frame in the loss-affected frame number calculation step, based on the packet loss time length M and the unit frame time length F, W = 1 + M / F Thus, the number W of loss-affected frames may be calculated.

Also, when there are multiple loss intervals in the loss-affected frame number calculation step, an average audio packet loss number is calculated by averaging the number of audio packet losses in these loss intervals, and this average audio packet loss number and average packet time length are calculated. The average packet loss time length is calculated by multiplying the average packet loss time length by calculating the average number of frames affected by the loss from the average packet loss time length and the unit frame time length of the frame. Thus, the total loss influence time obtained by multiplying the average loss influence frame number by the number of loss intervals may be calculated as an audio quality factor index.

  At this time, if the start time position of the audio data coincides with the start time position of the frame at a constant period in the loss influence frame number calculation step, the average packet loss time length M ′, the average packet time length U, the period P, and From the unit frame time length F, the average loss-affected frame number W ′ is calculated based on W ′ = ceil (M ′ / F) + (M ′% F) × {1−U / (P × F)}. It may be.

  Alternatively, if the start time position of the audio data does not periodically coincide with the start time position of the frame in the loss influence frame number calculation step, W ′ = 1 + M ′ is calculated from the average packet loss time length M ′ and the unit frame time length F. The average loss affected frame number W ′ may be calculated based on / F.

Also, the audio quality factor indicator calculating apparatus according to the present invention divides a frame sequence obtained by encoding an audio signal into a plurality of audio data, and stores the audio data in audio packets for transmission. Based on the audio packet captured from this multimedia communication, an audio quality factor index is calculated that indicates the effect of loss of the audio packet on the audio quality of the playback audio signal received and played back via the multimedia communication. a factor index computing device extracts, among the rows of audio packets captured from the receiver of the multimedia communication, for one or more consecutive lost section lost audio packet, the number of the audio packet loss That packet analysis unit and a loss interval by multiplying the average packet time length and the audio packet loss number of audio packets obtained by dividing the captured time required to capture these audio packets by the total number of audio packets captured The number of loss-affected frames is calculated by calculating the number of frames affected by the audio data lost in the loss period from the relationship between the packet loss time length and the unit frame time length of the frame. A calculating unit; and a loss affecting time length calculating unit that calculates a loss affecting time length in the reproduced audio signal as an audio quality factor index based on the number of loss affecting frames.

  The program according to the present invention is a program for causing a computer to execute each step of any one of the audio quality factor index calculation methods described above.

  According to the present invention, it is possible to calculate an audio quality factor index that appropriately captures the influence of audio packet loss on the audio quality of a reproduced audio signal. As a result, the audio quality can be accurately estimated.

It is a block diagram which shows the structure of the audio quality factor parameter | index calculation apparatus concerning 1st Embodiment. It is explanatory drawing which shows IP / UDP / RTP / MPEG2-TS protocol structure. It is explanatory drawing which shows the relationship between IP packet loss and the number of audio packet losses. It is explanatory drawing which shows the relationship between each flame | frame of audio | voice data, and the payload of TS packet. It is explanatory drawing which shows the relationship between packet loss time length and the number of loss influence frames. It is explanatory drawing which shows the relationship between an audio frame and TS packet. It is a flowchart which shows operation | movement of the audio quality factor parameter | index calculation apparatus concerning 1st Embodiment. It is a flowchart which shows operation | movement of the audio quality factor parameter | index calculation apparatus concerning 2nd Embodiment. It is a block diagram which shows the structure of the audio quality estimation apparatus concerning 3rd Embodiment. It is explanatory drawing which shows the packet storage form of audio data.

Next, embodiments of the present invention will be described with reference to the drawings.
[First Embodiment]
First, an audio quality factor index calculating apparatus according to a first embodiment of the present invention will be described with reference to FIG. FIG. 1 is a block diagram illustrating a configuration of an audio quality factor indicator calculating apparatus according to the first embodiment.

  This audio quality factor index calculation device 10 is composed of an information communication processing device such as a server device as a whole, divides a frame sequence obtained by encoding an audio (voice) signal into a plurality of audio data, and converts these audio data into For multimedia communication stored in audio packets and transmitted from the transmission terminal 1 to the reception terminal 2 via the communication network 3, the transmission path from the communication network 3 to the reception terminal 2 in which the multimedia communication is performed Alternatively, based on the audio packet captured from the receiving terminal 2, an audio quality factor index indicating the influence of the loss of the audio packet on the audio quality of the reproduced audio signal received and played back by the receiving terminal 2 via the multimedia communication is calculated. It is a device to do.

  In the present embodiment, the audio quality factor indicator calculating apparatus 10 calculates the number of audio packet losses for a loss section in which one or a plurality of audio packets are continuously lost from a sequence of audio packets captured from multimedia communication. Based on the number of audio packet losses, the number of frames affected by the audio data lost in the loss section of the frame sequence is calculated, and the playback audio signal is calculated based on the number of frames affected by the loss. Is calculated as an audio quality factor index.

[Configuration of First Embodiment]
Next, the configuration of the audio quality factor indicator calculation device 10 according to the present embodiment will be described in detail with reference to FIG.
The audio quality factor index calculation apparatus 10 includes, as main functional units, a communication interface unit (hereinafter referred to as a communication I / F unit) 11, an operation input unit 12, a screen display unit 13, a storage unit 14, and an arithmetic processing unit. 15 is provided.

The communication I / F unit 11 includes a dedicated data communication circuit and has a function of capturing IP packets related to multimedia communication to be evaluated from the transmission path 4 and the receiving terminal 2.
The operation input unit 12 includes an operation input device such as a keyboard and a mouse, and has a function of detecting an operator operation and outputting the operation to the arithmetic processing unit 15.
The screen display unit 13 includes a screen display device such as an LCD, and has a function of displaying various information such as operation menus, setting screens, and calculated audio quality factor indicators on the screen based on the control of the arithmetic processing unit 15. doing.

  The storage unit 14 includes a storage device such as a hard disk or a semiconductor memory. The storage unit 14 includes data used by the arithmetic processing unit 15, for example, IP packet data captured by the communication I / F unit 11, calculated audio quality factor indicators, and audio quality. It has a function of storing various processing information such as intermediate data and programs used in the factor index calculation process.

  The arithmetic processing unit 15 includes a microprocessor such as a CPU and its peripheral circuits, and has a function of realizing various processing units by reading and executing a program in the storage unit 14. As main processing units realized by the arithmetic processing unit 15, there are a packet analysis unit 15A, a loss influence frame number calculation unit 15B, and a loss influence time length calculation unit 15C.

  The packet analysis unit 15A extracts the number of audio packet losses for one or a plurality of loss periods in which audio packets are continuously lost from the sequence of audio packets captured from the multimedia communication by the communication I / F unit 11. It has a function.

  The loss-affected frame number calculation unit 15B calculates a loss-affected frame number of a frame affected by audio data lost in the loss section in the frame sequence based on the audio packet loss number extracted by the packet analysis unit 15A. have.

  The loss affected time length calculation unit 15C has a function of calculating the loss affected time length in the reproduced audio signal as an audio quality factor index based on the number of loss affected frames calculated by the loss affected frame number calculating unit 15B.

[Audio packet identification method]
Next, the principle of the present invention will be described.
First, a method for determining whether an IP packet is an audio packet will be described.
FIG. 2 is an explanatory diagram showing the IP / UDP / RTP / MPEG2-TS protocol structure. From the 13-bit PID (Packet Identifier) described in the header of the TS packet in the IP packet, it is possible to know what kind of media the data included in the payload of the TS packet is. Therefore, it is possible to confirm whether each IP packet is an audio packet by holding the PID for audio that is a quality evaluation target as premise information and confirming the PID of each TS packet.

[Audio packet loss detection method]
Next, an audio packet loss detection method will be described.
Audio packet loss detection is performed based on a 4-bit CC (Continuity Counter) that satisfies continuity in the TS header portion for each medium. By observing this lack of CC continuity, it is possible to detect lost audio packets. In the case of FIG. 2, since the audio packet P3 has been lost, the CC of the audio TS packet becomes “14” of the audio packet P4 after “12” of the audio packet P1. For this reason, it is possible to detect that one audio packet P3 is lost even if there is no information of the lost IP packet.

[Calculation method of audio packet loss count]
Next, a method for calculating the number of audio packet losses will be described.
Based on the IP packets received within a certain time, the number of audio packet losses is calculated as the audio burst length for each loss of audio packets. The number of audio packet loss with respect to each loss is the audio included in each audio packet loss when the loss is counted as one audio packet loss regardless of the continuous length when audio packets are continuously lost. The number of packets. In other words, the number of numbers that are missing when CC continuity is seen becomes the number of audio packet losses for the audio packet loss.

FIG. 3 is an explanatory diagram showing the relationship between IP packet loss and the number of audio packet losses.
In FIG. 3, first, IP packets P11 and P12 are continuously lost, but only IP packet P11 is an audio packet. For this reason, the number L of audio packet losses in this loss section is “1”.

Further, the IP packets P14 and P16 located before and after the normally transferred IP packet P15 are sandwiched. At this time, the IP packets P14 and P16, which are audio packets, are not continuous. However, since the IP packet P15 is not an audio packet, the IP packets 14 and P16 are continuous when viewed as an audio bucket sequence. . For this reason, the audio packet loss number L in this loss interval is “2”.
Further, although the IP packet P18 is lost, since this is not an audio packet, no loss has occurred.

[Calculation method of the number of frames affected by loss]
Next, a method for calculating the number of loss-affected frames will be described.
The number of frames affected by loss is the number of frames affected by each audio packet loss using the number of audio packet losses. FIG. 4 is an explanatory diagram showing the relationship between each frame of audio data and the payload of the TS packet.

  First, the packet loss time length M for the target loss interval T is calculated. The packet loss time length M represents the time length on the audio data corresponding to the loss interval T. Specifically, as shown in the following equation (1), the average packet time length U corresponding to one audio packet is calculated by dividing the IP packet acquisition time t by the total number N of audio packets, and this The packet loss time length M is calculated by multiplying the average packet time length U by the number L of audio packet losses lost in the loss interval T.

If there are 500 audio packets when an IP packet for 10 seconds is acquired, the average packet time length U per audio packet is 20 ms. Therefore, as shown in FIG. 4, when three audio packets are continuously lost in the loss interval T, the packet loss time length M in the loss interval T is 60 ms from the above-described equation (1).
However, the packet loss time length M does not coincide with the loss influence time length V at which the audio data is affected by the loss interval T.

  The reason is that the audio data constituting the frame is encoded data, and the audio signal in the time interval corresponding to the frame is reproduced only after all the data constituting the frame is received. This is because the audio signal cannot be normally reproduced in the time interval corresponding to the frame when a part of data constituting the frame is lost.

  Therefore, a frame that partially overlaps the packet loss time length M is affected by the loss period T, and the time length of the frame is affected. As shown in FIG. 4, it can be seen that the affected loss influence time length V is longer than the packet loss time length M.

  In view of such a discrepancy between the packet loss time length M and the loss influence time length V, in the present invention, first, based on the packet loss time length M, the number of frames affected by the loss interval T, that is, loss influence frames. The number W is calculated, and then the loss influence time length V is calculated based on the loss influence frame number W.

[Method for estimating the number of frames affected by loss]
Next, a method for estimating the expected value of the number W of loss-affected frames will be described.
FIG. 5 is an explanatory diagram showing the relationship between the packet loss time length and the number of frames affected by loss.
The number W of loss-affected frames varies depending on the packet loss time length M and the time position relationship between the packet loss time length M and the frame.

  As shown in FIG. 5, the frame in which the packet loss time length M starts is assumed to be frame 1, and the number of misaligned frames from the start time position of frame 1 to the start time position of packet loss time length M is d (0 ≦ d <1). When the number of misaligned frames d is “0”, that is, when the packet loss time length M and the start time position of the frame 1 coincide with each other, the end time position of the packet loss time length M is counted from the frame 1 as A (A is an integer equal to or greater than 1) The frame A is assumed to be taken, and the number of remaining frames from the end time position of the packet loss time length M to the end time position of the frame A is represented as B (0 ≦ B <1). .

  Accordingly, assuming that the time length of each frame is the unit frame time length F, the time length from the start time position of frame 1 to the start time position of packet loss time length M is d × F. Further, since the time length from the end time position of the packet loss time length M to the end time position of the frame A is B × F, the packet loss time length M is obtained by M = (A−B) F. .

  Here, when the number of misaligned frames d is less than or equal to the number of remaining frames B (0 ≦ d ≦ B), since d × F ≦ B × F, the end time position of the packet loss time length M is the end of the frame A It will coincide with the end time position of the frame A or before the time position. Therefore, the number W of loss-affected frames in this case is W = A.

  On the other hand, when the number of misaligned frames d is larger than the number of remaining frames B (B <d <1), d × F> B × F, so the end time position of the packet loss time length M is the end time of frame A It exists behind the position. Therefore, the number W of loss-affected frames in this case is W = A + 1.

At this time, the state in which the frame having the loss affected frame number W = A is affected by the loss interval T, that is, the probability that 0 ≦ d ≦ B is B, and the frame having the loss affected frame number W = A + 1 is in the loss interval T. The probability of being affected, ie B <d <1, is 1-B.
Therefore, if it is assumed that the relationship between the start time position of the frame and the start time position of the packet loss time length M is random and the probability that the two start time positions overlap is a value close to 0, the loss The expected value of the number W of loss-affected frames affected by the section T is obtained by the following equation (2).

  On the other hand, for every P frames, the start time position of the frame may always overlap the start time position of the audio data in the audio packet. FIG. 6 is an explanatory diagram showing the relationship between audio frames and TS packets.

  For example, as shown in FIG. 6, P audio frames are always stored in the PES packet, and the number of audio TS packets corresponding to the P frames is S. In this case, the start time position of the frame and the start time position of the audio data in the audio packet overlap every S audio packets.

  That is, for each audio packet loss, it can be considered that the start time position of the packet loss time length M overlaps the start time position of the frame with a probability of 1 / S. At this time, it is equal to the case of d = 0 in FIG. 6 described above, and only A frames are affected regardless of the value of B. Further, for the remaining probability of 1-1 / S, the expected value of the number of affected frames is calculated by the above-described equation (2).

  Therefore, the expected value of the number W of loss-affected frames affected by the loss interval T can be obtained by the following equation (3). A is a value obtained by rounding up the value obtained by dividing M by F, and is represented by the round-up function ceil (M / F) in equation (3). 1-B is a value indicating a remainder obtained when M is divided by F, and is expressed as M% F in the equation (3).

[Calculation method of the length of loss impact time]
Next, a method for calculating the loss influence time length V will be described.
The loss influence time length V is calculated by multiplying the loss influence frame number W by the unit frame time length F. The unit frame time length F is a time length corresponding to a coding unit of audio data. In the case of AAC (Advanced Audio Coding), the unit frame time length F [ms] = 1024 / sampling rate [kHz].
Further, as shown in FIG. 3 described above, when a plurality of loss intervals are included in the captured IP packet, the loss influence time length V is calculated for each loss interval, and these are accumulated. Thus, the total loss influence time length R may be calculated.

[Operation of First Embodiment]
Next, with reference to FIG. 7, the operation of the audio quality factor indicator calculating apparatus according to the present embodiment will be described. FIG. 7 is a flowchart showing the operation of the audio quality factor index calculating apparatus according to the first embodiment.
In this embodiment, data transmission conforming to the IP / UDP / RTP / MPEG2-TS packet structure will be described as an example, but data transmission according to other coding schemes and transmission schemes is also supported. It is possible.

  First, the packet analysis unit 15A describes IP packets for the capture time t (ms) captured by the communication I / F unit 11 and stored in the storage unit 14 in the headers of the TS packets in these IP packets. By referring to the 13-bit PID (Packet Identifier), it is determined whether the TS packet includes audio data of interest. Based on the determination result, an IP packet including target audio data is selected as an audio packet, and the total number N of audio packets is counted.

Further, the packet analysis unit 15A detects a lost audio packet based on the continuity of the 4-bit CC (Continuity Counter) value included in the TS header, and counts the number of audio packet losses.
For example, when CC = 7 and 8 are continuously lacking, the audio packet loss number L is 2. At this time, if there are a plurality of loss intervals, the number of audio packet losses L1... Lk is calculated for each loss interval.

  Thereafter, the packet analysis unit 15A notifies the loss time-affected frame number calculation unit 15B of the capture time t, the total number of audio packets N, and the number of audio packet losses L1,.

  The loss-affected frame number calculation unit 15B acquires the capture time t, the total number N of audio packets, and the number of audio packet losses L1... Lk from the packet analysis unit 15A, and calculates the average packet time length based on the following equation (4). U and packet loss time length M1... Mk (ms) in each loss section are calculated. i is an integer of 1 to k.

Thereafter, the loss-affected frame number calculation unit 15B uses the unit frame time length F and the period P that are set in advance in the storage unit 14 based on the following equation (5) or equation (6). The number of loss affected frames W1... Wk in the loss section is calculated and notified to the loss affected time length calculation unit 15C.
At this time, when the period P is P> 0, the number of loss-affected frames W1... Wk in each audio packet loss is calculated based on the equation (5), and when the period P is P = 0, Based on equation (6), the number W1 ... Wk of loss-affected frames in each audio packet loss is calculated.

  Note that the number S of audio TS packets corresponding to P frames introduced in the above-described equation (3) is one period of P frames, and as a result, S is the number of TS packets per period. Yeah. Therefore, S is equal to the value obtained by dividing the one-cycle time length P × F by the average packet time length U = t / N. Equation (5) uses U, P, and F instead of S.

  The loss influence time length calculation unit 15C obtains the loss influence frame number W1... Wk in each loss section from the loss influence frame number calculation unit 15B, and formula (7) for each of the loss influence frame numbers W1. As shown in FIG. 5, the loss influence time lengths V1... Vk in each loss section are calculated by multiplying the unit frame time length F preset in the storage unit.

  Thereafter, the loss influence time length calculation unit 15C accumulates these V1... Vk as shown in Expression (8), so that the loss of the audio packet included in the capture time t gives the audio quality of the reproduced audio signal. The total loss influence time length R is calculated as an audio quality factor index indicating the influence.

[Effect of the first embodiment]
As described above, according to the present embodiment, the number of audio packet losses in a loss section in which one or a plurality of audio packets are continuously lost in a sequence of audio packets captured from multimedia communication by the packet analysis unit 15A. L is extracted, and the loss influence frame number calculation unit 15B calculates the loss influence frame number W of the frames affected by the audio data lost in the loss section in the frame sequence based on the audio packet loss number L. The loss influence time length calculation unit 15C calculates the loss influence time length V in the reproduced audio signal as an audio quality factor index based on the number W of loss influence frames.

Thus, even in multimedia communication including IP packets other than audio packets such as video packets, the loss of audio packets is accurately captured by the number L of audio packet losses.
Further, since the number W of loss-affected frames is calculated from the number L of lost audio packets, the number of audio packets lost in the loss interval does not necessarily correspond to the number of frames affected by the loss interval. Issues specific to packets have also been eliminated.

Further, since the loss influence time length V is calculated from the number W of loss influence frames, the audio based on the real time of the reproduction audio signal, not the packet-based audio quality factor indicators such as the number of IP packets and the loss rate. Quality factor indicators are calculated.
Therefore, according to the present embodiment, it is possible to calculate an audio quality factor index that appropriately captures the influence of loss of audio packets on the audio quality of the reproduced audio signal. As a result, the audio quality can be accurately estimated.

  In the present embodiment, when there are a plurality of loss intervals, the loss influence frame number calculation unit 15B calculates the loss influence frame number W for each loss interval, and the loss influence time length calculation unit 15C calculates the loss interval. Each time, the loss influence time length V is calculated based on the number W of loss influence frames in the loss section, and the total loss influence time R obtained by accumulating these loss influence time lengths V is calculated as an audio quality factor index. Even when a plurality of loss intervals occur, the audio quality factor index can be calculated with high accuracy.

  Further, in the present embodiment, the loss-affected frame number calculation unit 15B calculates the packet loss time length M of the loss section by multiplying the average packet time length U of audio packets by the audio packet loss number L. Since the number W of loss-affected frames is calculated from the packet loss time length M and the unit frame time length F of the frame, the correspondence between the audio packet in the loss period and the affected frame is calculated on a real time basis. The loss effect time length can be calculated more accurately.

  In this embodiment, the loss-affected frame number calculation unit 15B calculates W = ceil (M / F) + () from the packet loss time length M, the average packet time length U, the period P, and the unit frame time length F. Since the number W of loss-affected frames is calculated based on (M% F) × {1-U / (P × F)}, the start time position of the audio data coincides with the start time position of the frame at a constant period. Even in this case, the loss influence time length V can be calculated with extremely high accuracy.

  In the present embodiment, the loss-affected frame number calculation unit 15B calculates the loss-affected frame number W from the packet loss time length M and the unit frame time length F based on W = 1 + M / F. Even when the start time position of the audio data does not periodically coincide with the start time position of the frame, the loss influence time length V can be calculated with extremely high accuracy.

[Second Embodiment]
Next, an audio quality factor index calculating apparatus according to a second embodiment of the present invention will be described.
In the first embodiment, when there are a plurality of loss intervals, a case where the total loss influence time length R is calculated by accumulating the loss influence time length V calculated for each loss interval will be described as an example. did. In the present embodiment, a case will be described in which the total loss affected time length R is calculated using the average value of the number of audio packet losses lost in each loss interval.

In the present embodiment, the loss-affected frame number calculation unit 15B calculates the average audio packet loss number obtained by averaging the audio packet loss numbers in a plurality of loss intervals, and the average audio packet loss number and average packet time length. The average packet loss time length of the loss interval is multiplied by the average packet loss time length of the loss interval, and the average number of frames affected by the loss from the average packet loss time length and the unit frame time length of the frame. And a function of calculating.
The other configuration of the audio quality factor indicator calculation apparatus 10 according to the present embodiment is the same as that of the first embodiment, and a detailed description thereof is omitted here.

[Operation of Second Embodiment]
Next, with reference to FIG. 8, the operation of the audio quality factor indicator calculation apparatus according to the present embodiment will be described. FIG. 8 is a flowchart showing the operation of the audio quality factor index calculating apparatus according to the second embodiment.
In this embodiment, data transmission conforming to the IP / UDP / RTP / MPEG2-TS packet structure will be described as an example, but data transmission according to other coding schemes and transmission schemes is also supported. It is possible.

  First, the packet analysis unit 15A describes IP packets for the capture time t (ms) captured by the communication I / F unit 11 and stored in the storage unit 14 in the headers of the TS packets in these IP packets. By referring to the 13-bit PID (Packet Identifier), it is determined whether the TS packet includes audio data of interest. Based on the determination result, an IP packet including target audio data is selected as an audio packet, and the total number N of audio packets is counted.

Further, the packet analysis unit 15A detects a lost audio packet based on the continuity of the 4-bit CC (Continuity Counter) value included in the TS header, and counts the number of audio packet losses.
For example, when CC = 7 and 8 are continuously lacking, the audio packet loss number L is 2. At this time, if there are a plurality of loss intervals, the number of audio packet losses L1... Lk is calculated for each loss interval.

  Thereafter, the packet analysis unit 15A notifies the loss time-affected frame number calculation unit 15B of the capture time t, the total number of audio packets N, and the number of audio packet losses L1,.

The loss-affected frame number calculation unit 15B acquires the capture time t, the total number N of audio packets, and the number of audio packet losses L1... Lk from the packet analysis unit 15A.
Next, the loss-affected frame number calculation unit 15B calculates an average audio packet loss number L ′ as an average value of the audio packet loss numbers L1... Lk, and calculates the average packet time length U based on the following equation (9). Then, an average packet loss time length M ′ (ms) is calculated as an average value of the packet loss time lengths in each loss section.

Thereafter, the loss-affected frame number calculation unit 15B uses the unit frame time length F and the period P set in the storage unit 14 in advance, based on the following formula (10) or formula (11), The average loss affected frame number W ′ in the loss interval is calculated and notified to the loss affected time length calculation unit 15C.
At this time, when the period P is P> 0, the average loss affected frame number W ′ in each audio packet loss is calculated based on the equation (10), and when the period P is P = 0, Based on Expression (11), the average loss-influencing frame number W ′ for each audio packet loss is calculated.

  Note that the number S of audio TS packets corresponding to P frames introduced in the above-described equation (3) is one period of P frames, and as a result, S is the number of TS packets per period. Yeah. Therefore, S is equal to the value obtained by dividing the one-cycle time length P × F by the average packet time length U = t / N. Equation (10) uses U, P, F instead of S.

  The loss influence time length calculation unit 15C obtains the average loss influence frame number W ′ in each loss section from the loss influence frame number calculation unit 15B, and the equation (12) indicates the average loss influence frame number W ′. As described above, the average loss influence time length V ′ in each loss section is calculated by multiplying the unit frame time length F preset in the storage unit 14.

  Thereafter, the loss affected time length calculation unit 15C multiplies the average loss affected time length V ′ by the number of loss sections k as shown in Expression (13), thereby losing the audio packet included in the capture time t. The total loss influence time length R is calculated as an audio quality factor index indicating the influence of the reproduction audio signal on the audio quality.

[Effect of the second embodiment]
Thus, in the present embodiment, when there are a plurality of loss intervals, the loss influence frame number calculation unit 15B calculates the average loss influence frame number W ′ in these loss intervals, and the loss influence time length calculation unit 15C. Thus, since the total loss influence time R obtained by multiplying the average loss influence frame number W ′ by the number k of loss intervals is calculated as an audio quality factor index, it is possible to simplify the calculation processing for each loss interval. it can.

  In the present embodiment, the loss-affected frame number calculation unit 15B calculates an average audio packet loss number L ′ obtained by averaging the audio packet loss numbers in a plurality of loss intervals, and calculates the average audio packet loss number L ′. By multiplying the average packet time length U, the average packet loss time length M ′ of the loss interval is calculated. From this average packet loss time length M ′ and the frame unit frame time length F, the number of loss-affected frames is averaged. Since the average loss affected frame number W ′ is calculated, the calculation process of the loss affected frame number for each loss section can be greatly simplified.

  In this embodiment, the loss-affected frame number calculation unit 15B calculates W ′ = ceil (M ′ / M) from the average packet loss time length M ′, the average packet time length U, the period P, and the unit frame time length F. Since the average loss-affected frame number W ′ is calculated based on (F) + (M ′% F) × {1−U / (P × F)}, the start time position of the audio data is determined at a constant period. Even when it coincides with the start time position of the frame, the loss influence time length V can be calculated with extremely high accuracy.

  In this embodiment, the loss-affected frame number calculation unit 15B calculates the average loss-affected frame number W ′ from the average packet loss time length M ′ and the unit frame time length F based on W ′ = 1 + M ′ / F. Since the calculation is performed, even if the start time position of the audio data does not periodically coincide with the start time position of the frame, the loss influence time length V can be calculated with extremely high accuracy.

[Third Embodiment]
Next, an audio quality estimation apparatus according to the third embodiment of the present invention will be described with reference to FIG. FIG. 9 is a block diagram showing a configuration of an audio quality estimation apparatus according to the third embodiment.

  In the present embodiment, the total loss effect time length R of the multimedia communication to be evaluated, which is calculated by the audio quality factor index calculation method according to the first or second embodiment described above, is used as the audio quality factor index. Thus, the case where the subjective quality evaluation value Q is estimated as the audio quality of the multimedia communication will be described.

  This audio quality estimation device 20 is composed of an information communication processing device such as a server device as a whole, and uses the total loss affected time length R obtained from the multimedia communication to be evaluated as the audio quality of the multimedia communication. It is an apparatus for estimating the subjective quality evaluation value Q.

  The audio quality estimation apparatus 20 includes, as main functional units, a communication interface unit (hereinafter referred to as a communication I / F unit) 21, an operation input unit 22, a screen display unit 23, a storage unit 24, an arithmetic processing unit 25, an encoding unit. An audio quality value database (hereinafter referred to as encoded audio quality value DB) 26A and a coefficient database (hereinafter referred to as coefficient DB) 26B are provided.

  Among these functional units in the audio quality estimation device 20, the configuration of the communication I / F unit 21, the operation input unit 22, the screen display unit 23, the storage unit 24, and the arithmetic processing unit 25 is the same as the audio quality factor index calculation device described above. 10 communication I / F units 11, operation input unit 12, screen display unit 13, storage unit 14, and arithmetic processing unit 15.

  As main processing units realized by the arithmetic processing unit 25, there are an audio quality factor index calculating unit 25A, an encoded audio quality value acquiring unit 25B, a coefficient acquiring unit 25C, and a subjective quality evaluation value calculating unit 25D.

The audio quality factor index calculation unit 25A includes the packet analysis unit 15A, the loss effect frame number calculation unit 15B, and the loss effect time length calculation unit 15C of the audio quality factor index calculation device 10 described above, and is an evaluation target. It has a function of calculating the total loss effect time length R of multimedia communication. Here, a case where the total loss influence time length R is used as an audio quality factor index as a loss influence time length in a practical situation where a plurality of loss sections occur will be described as an example.
Instead of the audio quality factor index calculation unit 25A, the total loss influence time length R may be acquired from the audio quality factor index calculation device 10 externally connected to the audio quality estimation device 20.

  The encoded audio quality value acquisition unit 25B has a function of acquiring the encoded audio quality value Ie corresponding to the audio bit rate and sampling rate of the multimedia communication to be evaluated from the encoded audio quality value DB 26A. .

  The coefficient acquisition unit 25C has a function of acquiring coefficients n1, n2, and n3 corresponding to codec type information of multimedia communication to be evaluated from the coefficient DB 26B.

  The subjective quality evaluation value calculation unit 25D includes the total loss affected time length R calculated by the audio quality factor index calculation unit 25A, the encoded audio quality value Ie acquired by the encoded audio quality value acquisition unit 25B, and the coefficient acquisition unit 25C. The subjective quality evaluation value Q is calculated as the audio quality of the multimedia communication to be evaluated from the coefficients n1, n2, and n3 acquired in step (1).

  The encoded audio quality value DB 26A contains an encoded audio quality value Ie indicating the audio quality (subjective quality) obtained by the multimedia communication for each combination of the audio bit rate and sampling rate of the multimedia communication to be evaluated. Database. For multimedia communication determined by the combination of the audio bit rate and the sampling rate, a subjective quality evaluation experiment may be performed in advance, and the opinion evaluation result may be registered as the encoded audio quality value Ie. Here, a DMOS (Differential MOS) value is used as the encoded audio quality value Ie, but is not limited to this.

  The coefficient DB 26B is a database in which coefficients n1, n2, and n3 that define the shape of the subjective quality evaluation value Q estimation model are entered for each codec type information. These coefficients n1, n2, and n3 are coefficients that depend on the codec used in multimedia communication. For example, by analyzing the opinion evaluation results obtained in the subjective quality evaluation experiment using a nonlinear least square method, the codec The relationship between the type and the coefficients n1, n2, and n3 may be derived in advance, and the coefficients n1, n2, and n3 may be registered for each codec type.

  The total loss influence time length R and the subjective quality evaluation value Q have a characteristic that the subjective quality evaluation value Q monotonously decreases as the total loss influence time length R increases, and the encoded audio quality value Ie, the index Using the function e () and the coefficients n1, n2, and n3, it is expressed by an estimated model function expression such as the following expression (14).

Therefore, the audio bit rate and sampling rate of multimedia communication to be evaluated and the codec type information are input as known setting information from, for example, the operation input unit 22.
The audio quality factor index calculation unit 25A calculates the total loss effect time length R, the encoded audio quality value acquisition unit 25B acquires the encoded audio quality value Ie, and the coefficient acquisition unit 25C supports the codec type information. The coefficients n1, n2, and n3 to be acquired are acquired.

  Then, in the subjective quality evaluation value calculation unit 25D, the obtained encoded audio quality value Ie, coefficients n1, n2, and n3 and the total loss influence time length R are stored in advance in the storage unit 24 as an estimation model. By applying the above equation (14), the subjective quality evaluation value Q can be calculated as the audio quality of the multimedia communication.

[Extended embodiment]
The present invention has been described above with reference to the embodiments, but the present invention is not limited to the above embodiments. Various changes that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention.

  DESCRIPTION OF SYMBOLS 1 ... Transmission terminal, 2 ... Reception terminal, 3 ... Communication network, 4 ... Transmission path, 10 ... Audio quality factor parameter | index calculation apparatus, 11 ... Communication I / F part, 12 ... Operation input part, 13 ... Screen display part, 14 DESCRIPTION OF SYMBOLS ... Storage part, 15 ... Arithmetic processing part, 15A ... Packet analysis part, 15B ... Loss influence frame number calculation part, 15C ... Loss influence time length calculation part, 20 ... Audio quality estimation apparatus, 21 ... Communication I / F part, 22 ... operation input unit, 23 ... screen display unit, 24 ... storage unit, 25 ... arithmetic processing unit, 25A ... audio quality factor index calculation unit, 25B ... encoded audio quality value acquisition unit, 25C ... coefficient acquisition unit, 25D ... subjective Quality evaluation value calculation unit, 26A ... encoded audio quality value DB, 26B ... coefficient DB, T ... loss interval, L ... number of audio packet losses, N ... total number of audio packets, U ... average packet time Length, P ... Average packet time length, M ... Packet loss time length, F ... Unit frame time length, W ... Loss-affected frame number, V ... Loss-affected time length, L '... Average audio packet loss number, M' ... Average Packet loss time length, W′—average loss-affected frame number, V′—average loss-affected time length, R—total loss-affected time length, n1, n2, n3—coefficient, Ie—encoded audio quality value, Q—subjectivity Quality evaluation value.

Claims (9)

For multimedia communication in which a frame sequence obtained by encoding an audio signal is divided into a plurality of audio data, and each of these audio data is stored and transmitted in an audio packet, based on the audio packet captured from the multimedia communication An audio quality factor index calculation method for calculating an audio quality factor index indicating an effect of loss of the audio packet on the audio quality of a reproduced audio signal received and reproduced via the multimedia communication,
A packet analysis step of extracting the number of lost audio packets for a loss section in which one or a plurality of audio packets are continuously lost in the sequence of the audio packets captured from the receiver of the multimedia communication;
The packet loss time length of the loss interval is calculated by multiplying the average packet time length of audio packets obtained by dividing the capture time of these audio packets by the total number of audio packets captured and the number of audio packet losses. A loss-affected frame number calculating step of calculating a loss-affected frame number of a frame affected by the audio data lost in the loss section from the relationship between the packet loss time length and the unit frame time length of the frame;
An audio quality factor index calculation method comprising: a loss effect time length calculation step of calculating a loss effect time length in the reproduced audio signal as the audio quality factor index based on the number of loss effect frames. The audio quality factor indicator calculation method according to claim 1,
The loss-affected frame number calculating step calculates the number of loss-affected frames for each loss section when there are a plurality of loss sections,
The loss impact time length calculation step calculates, for each loss section, the loss impact time length based on the number of loss impact frames in the loss section, and calculates the total loss impact time obtained by accumulating these loss impact time lengths. An audio quality factor index calculation method characterized by calculating as an audio quality factor index. The audio quality factor index calculation method according to claim 2 ,
In the loss-affected frame number calculating step, when the start time position of the audio data coincides with the start time position of the frame at a constant period, the packet loss time length M, the average packet time length U, the period P, From the unit frame time length F,
W = ceil (M / F) + (M% F) × {1−U / (P × F)}
And calculating the number W of loss-affected frames based on the audio quality factor index calculation method. The audio quality factor index calculation method according to claim 2 ,
When the start time position of the audio data does not periodically coincide with the start time position of the frame, the loss-affected frame number calculation step calculates from the packet loss time length M and the unit frame time length F:
W = 1 + M / F
And calculating the number W of loss-affected frames based on the audio quality factor index calculation method. The audio quality factor indicator calculation method according to claim 1,
The loss-affected frame number calculating step calculates an average audio packet loss number obtained by averaging the audio packet loss numbers in the loss intervals when there are a plurality of loss intervals, and calculates the average audio packet loss number and the average packet. The average packet loss time length is calculated by multiplying the time length, and the average loss-affected frame number in these loss intervals is calculated from the average packet loss time length and the unit frame time length of the frame ,
The audio quality factor index calculation method, wherein the loss impact time length calculation step calculates, as the audio quality factor index, a total loss effect time obtained by multiplying the average loss effect frame number by the number of loss intervals. In the audio quality factor index calculation method according to claim 5 ,
In the loss-affected frame number calculating step, when the start time position of the audio data coincides with the start time position of the frame at a constant period, the average packet loss time length M ′, the average packet time length U, the period From P and the unit frame time length F,
W ′ = ceil (M ′ / F) + (M ′% F) × {1−U / (P × F)}
And calculating the average loss-affected frame number W ′ based on the audio quality factor index calculation method. In the audio quality factor index calculation method according to claim 5 ,
When the start time position of the audio data does not periodically coincide with the start time position of the frame, the loss-affected frame number calculation step calculates the average packet loss time length M ′ and the unit frame time length F from:
W '= 1 + M' / F
And calculating the average loss-affected frame number W ′ based on the audio quality factor index calculation method. For multimedia communication in which a frame sequence obtained by encoding an audio signal is divided into a plurality of audio data, and each of these audio data is stored and transmitted in an audio packet, based on the audio packet captured from the multimedia communication An audio quality factor index calculating device for calculating an audio quality factor index indicating an effect of the loss of the audio packet on the audio quality of the reproduced audio signal received and reproduced via the multimedia communication,
A packet analysis unit that extracts the number of audio packet losses for a loss section in which one or a plurality of the audio packets are continuously lost in the sequence of the audio packets captured from the receiver of the multimedia communication;
The packet loss in the loss interval is obtained by multiplying the average packet time length of audio packets obtained by dividing the capture time required to capture these audio packets by the total number of audio packets captured and the number of audio packet losses. Calculate the time length, and calculate the number of loss-affected frames by calculating the number of frames affected by the audio data lost in the loss section from the relationship between the packet loss time length and the unit frame time length of the frame. And
An audio quality factor index calculating apparatus comprising: a loss effect time length calculation unit that calculates a loss effect time length in the reproduced audio signal as the audio quality factor index based on the number of loss effect frames. The computer program for executing the steps of the audio quality factor index calculating method according to any one of claims 1 to 7.

Images