CN115734043A - Video quality problem determination method, device, storage medium and program product - Google Patents

Video quality problem determination method, device, storage medium and program product Download PDF

Info

Publication number
CN115734043A
CN115734043A CN202110993951.3A CN202110993951A CN115734043A CN 115734043 A CN115734043 A CN 115734043A CN 202110993951 A CN202110993951 A CN 202110993951A CN 115734043 A CN115734043 A CN 115734043A
Authority
CN
China
Prior art keywords
frame
packet
video
parameter
quality
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202110993951.3A
Other languages
Chinese (zh)
Inventor
杨辉
姜军庆
徐斌斌
高鹏
庞宏超
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Huawei Technologies Co Ltd
Original Assignee
Huawei Technologies Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Huawei Technologies Co Ltd filed Critical Huawei Technologies Co Ltd
Priority to CN202110993951.3A priority Critical patent/CN115734043A/en
Priority to PCT/CN2022/078279 priority patent/WO2023024469A1/en
Publication of CN115734043A publication Critical patent/CN115734043A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client 
    • H04N21/63—Control signaling related to video distribution between client, server and network components; Network processes for video distribution between server and clients or between remote clients, e.g. transmitting basic layer and enhancement layers over different transmission paths, setting up a peer-to-peer communication via Internet between remote STB's; Communication protocols; Addressing
    • H04N21/647—Control signaling between network components and server or clients; Network processes for video distribution between server and clients, e.g. controlling the quality of the video stream, by dropping packets, protecting content from unauthorised alteration within the network, monitoring of network load, bridging between two different networks, e.g. between IP and wireless

Abstract

The present application relates to a video quality problem determination method, apparatus, storage medium, and program product, the video quality problem determination method comprising: receiving protocol information on a video playing end and network equipment, determining abnormal frame quality parameters and abnormal packet quality parameters according to the protocol information, and determining the reason and equipment causing video quality abnormality of the video playing end according to the abnormal frame quality parameters and/or the abnormal packet quality parameters. According to the method for determining the video quality problem, the reason causing the video quality problem and the equipment can be rapidly and accurately determined.

Description

Video quality problem determination method, device, storage medium and program product
Technical Field
The present application relates to the field of video quality detection, and in particular, to a method, an apparatus, a storage medium, and a program product for determining a video quality problem.
Background
Compared with 4G, the 5G technology has the advantages of large bandwidth and low time delay, so that technical support can be provided for video return scenes such as high-definition video live broadcast, camera real-time monitoring, unmanned aerial vehicle shooting and the like which need real-time transmission of video streaming media.
Video backhaul is a real-time service scenario based on streaming media. Taking high-definition live video as an example, video frames are obtained after video acquisition is completed through equipment such as a camera, video coding compression is completed on the video frames through an encoder/a knapsack server, the compressed video frames are packaged through a streaming media layer protocol to generate live streaming, the live streaming is transmitted to a video server through network equipment, and picture rendering and display are performed through a video player after decapsulation and decoding.
With the popularization of the application of the video return scene, the requirement of positioning analysis of video quality problems in the scene is generated. Among them, video quality issues include: the problems of jamming, screen splash and the like which can occur in the process of watching videos by users and lead to poor user experience. The problems of blocking, screen splash and the like are caused by the abnormity of the video frame, and a plurality of devices may participate in the process of acquiring, encoding, encapsulating, transmitting, decapsulating and decoding the video frame. In order to improve the user experience, it is necessary to locate a device causing a quality problem of a video and a specific reason why the device causes the quality problem of the video. The existing technology can not rapidly and accurately determine the reason and equipment causing the video quality problem.
Disclosure of Invention
In view of the above, a method, an apparatus, a storage medium, and a program product for determining a video quality problem are provided, according to the method, the apparatus, the storage medium, and the program product for determining a video quality problem of the embodiments of the present application, the determination of the cause and the device causing the video quality problem can be completed quickly and accurately.
In a first aspect, an embodiment of the present application provides a method for determining a video quality problem, where the method includes: receiving protocol information on a video playing end and network equipment, wherein the network equipment is used for transmitting video data to the video playing end, the protocol information indicates a sending mode of a video frame, and the video frame comprises a plurality of data packets; determining an abnormal frame quality parameter and an abnormal packet quality parameter according to the protocol information, wherein the abnormal frame quality parameter represents that the quality of a video frame to which the abnormal frame quality parameter belongs is abnormal, and the abnormal packet quality parameter represents that the quality of a data packet to which the abnormal packet quality parameter belongs is abnormal; and determining the reason and equipment causing the video quality abnormity of the video playing end according to the abnormal frame quality parameters and/or the abnormal packet quality parameters.
According to the video quality problem determining method, the protocol information sent by the video playing end and the protocol information sent by the network equipment for transmitting the video data to the video playing end are received, so that the sending modes of the video frames and a plurality of data packets included in the video frames can be determined according to the protocol information, and further abnormal frame quality parameters and abnormal packet quality parameters can be determined. The abnormal frame quality parameter represents the quality abnormality of the video frame, the abnormal packet quality parameter represents the quality abnormality of the data packet, and the video quality has a correlation with the video frame and the data packet, so that the abnormal frame quality parameter and the abnormal packet quality parameter are correlated with the occurring video quality abnormality, and the reason and the equipment causing the video quality abnormality at the video playing end can be determined according to the abnormal frame quality parameter and the abnormal packet quality parameter. According to the video quality problem determining method, the reason and the equipment which cause video quality abnormity at the video playing end are determined through the combination of the frame-level abnormal frame quality parameters and the packet-level abnormal packet quality parameters, the time cost for completing the determination of the reason and the equipment at one time can be reduced, and the accuracy of the determined reason and the equipment is improved.
According to the method for determining the video quality problem, when some equipment can only obtain the payload head, the protocol information can be extracted according to the payload head, the reason and the equipment which cause the video quality abnormity of the video playing end can be accurately determined, and the application range of the method for determining the video quality problem can be enlarged.
According to the first aspect, in a first possible implementation manner of the method for determining a video quality problem, the protocol information includes streaming media layer protocol information, network layer protocol information, and transport layer protocol information of a packet, where the streaming media layer protocol information includes streaming media layer protocol information of a packet on a video playing end, the network layer protocol information includes network layer protocol information of packets on a network device and on the video playing end, the transport layer protocol information includes transport layer protocol information of packets on the network device and on the video playing end, and determining an abnormal frame quality parameter and an abnormal packet quality parameter according to the protocol information includes: determining at least one video frame and each data packet corresponding to each frame in the at least one video frame according to the streaming media layer protocol information; determining a frame quality parameter of each frame in the at least one video frame according to the streaming media layer protocol information and the network layer protocol information; determining packet quality parameters of each data packet corresponding to each frame in the at least one video frame according to the network layer protocol information and the transmission layer protocol information; and determining the abnormal frame quality parameter according to the frame quality parameter corresponding to each frame in the at least one video frame and the frame quality parameter threshold, and determining the abnormal packet quality parameter according to the packet quality parameter of each data packet corresponding to each frame in the at least one video frame and the packet quality parameter threshold.
In this way, the frame quality parameter and the packet quality parameter can be determined through the combination of different protocol information so as to embody the quality of the video frame and the quality of the data packet under the video frame. Whether the frame quality parameter and the packet quality parameter are abnormal or not can be determined through a frame quality parameter threshold and a packet quality parameter threshold respectively, so that the mode of determining the abnormal frame quality parameter and the abnormal packet quality parameter is more convenient.
According to the first possible implementation manner of the first aspect, in a second possible implementation manner of the video quality problem determining method, the determining the abnormal frame quality parameter according to the frame quality parameter and the frame quality parameter threshold corresponding to each frame in the at least one video frame, and determining the abnormal packet quality parameter according to the packet quality parameter and the packet quality parameter threshold of each data packet corresponding to each frame in the at least one video frame includes: for each frame of the at least one video frame, determining a frame quality parameter of which the value is greater than the frame quality parameter threshold value in the frame quality parameters of the video frame as the abnormal frame quality parameter; and aiming at each frame of the at least one video frame, determining the packet quality parameter with the value larger than the threshold value of the packet quality parameter in the packet quality parameters of the data packet corresponding to the video frame as the abnormal packet quality parameter.
In this way, the frame quality parameter and the packet quality parameter of the anomaly may be determined. The abnormal parameters are determined by comparing the parameter values with the threshold values, so that the determination of the abnormal parameters can be accurately and quickly completed.
According to the first or second possible implementation manner of the first aspect, in a third possible implementation manner of the method for determining a video quality problem, the determining a frame quality parameter of each of the at least one video frame according to the streaming media layer protocol information includes a frame timestamp indicating a video frame to which the data packet belongs, the network layer protocol information of the data packet includes a packet time indicating a time of the data packet, the frame quality parameter includes a frame start packet jitter parameter and a frame end packet jitter parameter, the frame start packet jitter parameter indicates a time delay jitter condition of a first data packet corresponding to the video frame, the frame end packet jitter parameter indicates a time delay jitter condition of a last data packet corresponding to the video frame, and the determining a frame quality parameter of each of the at least one video frame according to the streaming media layer protocol information and the network layer protocol information includes: for each frame of the at least one video frame, determining the arrival time of a first packet and the arrival time of a last packet of the video frame according to the packet time of a data packet corresponding to the video frame, wherein the arrival time of the first packet represents the arrival time of a first data packet corresponding to the video frame, and the arrival time of the last packet represents the arrival time of a last data packet corresponding to the video frame; respectively calculating a head packet interval and a tail packet interval of adjacent video frames according to the arrival time of the head packet and the arrival time of the tail packet, wherein the head packet interval represents the arrival time difference of a first data packet corresponding to the adjacent video frames, and the tail packet interval represents the arrival time difference of a last data packet corresponding to the adjacent video frames; respectively calculating the frame interval of the adjacent video frames according to the frame time stamps; calculating the frame head packet jitter parameter of the current video frame according to the head packet interval, the frame interval and the frame head packet jitter parameter of the previous video frame; and calculating the frame end packet jitter parameter of the current video frame according to the end packet interval, the frame interval and the frame end packet jitter parameter of the previous video frame.
By the method, a frame head packet jitter parameter representing the time delay jitter condition of the first data packet corresponding to the video frame and a frame tail packet jitter parameter representing the time delay jitter condition of the last data packet corresponding to the video frame can be calculated, and the approximation degree of the frame quality parameter of the video frame and the actual jitter condition of the video frame is improved while the quality of the video frame is represented.
According to any one of the first to third possible implementation manners of the first aspect, in a fourth possible implementation manner of the method for determining a video quality problem, the determining a frame quality parameter of each frame in the at least one video frame according to the streaming media layer protocol information of the packet includes a streaming media layer sequence number indicating an order of the packet in the video frame to which the packet belongs, the network layer protocol information of the packet includes a packet time indicating a time of the packet, the frame quality parameter includes a frame packet loss parameter and a frame misordering parameter, the frame packet loss parameter indicates a loss condition of the packet of the video frame, the frame misordering parameter indicates a misordering condition of the packet of the video frame, and the determining a frame quality parameter of each frame in the at least one video frame according to the streaming media layer protocol information and the network layer protocol information includes: determining a frame loss parameter of the video frame according to a stream media layer sequence number of a data packet corresponding to the video frame aiming at each frame of the at least one video frame; and aiming at each frame of the at least one video frame, sequencing the data packets corresponding to the video frame according to the sequence of the packet time of the data packets corresponding to the video frame to obtain a sequencing result corresponding to the video frame, and determining the frame disorder parameter of the video frame according to the sequencing result and the stream media layer sequence number of the data packets corresponding to the video frame.
By the method, the frame packet loss parameter representing the loss condition of the data packet of the video frame and the frame disorder parameter representing the disorder condition of the data packet of the video frame can be calculated, and the flexibility of selecting the frame quality parameter is improved while the quality of the video frame is represented.
According to a fourth possible implementation manner of the first aspect, in a fifth possible implementation manner of the method for determining a video quality problem, the determining, for each of the at least one video frame, a frame loss parameter of the video frame according to a sequence number of a streaming media layer of a data packet corresponding to the video frame includes: for each frame of the at least one video frame, determining the maximum value and the minimum value of the stream media layer sequence number of the data packet corresponding to the video frame and the number of the stream media layer sequence number according to the stream media layer sequence number of the data packet corresponding to the video frame, determining the theoretical value of the number of the data packets corresponding to the video frame according to the maximum value and the minimum value of the stream media layer sequence number, and determining the frame packet loss parameter of the video frame according to the difference between the theoretical value of the number of the data packets corresponding to the video frame and the number of the stream media layer sequence number of the data packet corresponding to the video frame; determining a frame disorder parameter of the video frame according to the sequencing result and the sequence number of the stream media layer of the data packet corresponding to the video frame, including: and when the numerical value of the sequence number of the current data packet in the streaming media layer is greater than the sequence number of the streaming media layer of the data packet before the current data packet in the sequencing result, determining the frame disorder parameter corresponding to the video frame according to the sequencing sequence number of the current data packet in the sequencing result, the sequence number of the current data packet in the streaming media layer, and the minimum value of the sequence number of the streaming media layer of the data packet corresponding to the video frame.
By the method, the frame packet loss parameter can reflect the information of whether the data packet in the video frame loses packets, and can also reflect the information of the number of lost packets when the data packet loses packets; the frame disorder parameter can reflect the information whether the data packets in the video frame are in disorder or not, and can also reflect the disorder depth information of the disordered data packets when the disorder exists, so that the accuracy of determining the quality condition of the video frame according to the frame quality parameter can be improved.
According to any one of the first to fifth possible implementation manners of the first aspect, in a sixth possible implementation manner of the method for determining a video quality problem, a packet quality parameter of each data packet corresponding to each frame in the at least one video frame is determined according to network layer protocol information and transport layer protocol information of the data packet, where the packet quality parameter includes: the data packet processing method comprises one or more of a packet interval parameter representing the time interval between a current data packet and a previous data packet, a packet jitter parameter representing the time delay jitter between the current data packet and the previous data packet, a disorder parameter representing the disorder condition of the data packet, a packet loss parameter representing the packet loss condition of the data packet, a retransmission parameter representing the retransmission condition of the data packet, a time delay parameter representing the time delay condition of the data packet and a corresponding response message, an IP message fragment parameter representing whether the data packet has an IP message fragment, and a checksum error parameter representing whether the checksum of the data packet has an error.
By the method, various packet quality parameters for representing the quality of the data packet can be obtained, the quality of the data packet determined according to the packet quality parameters is closer to the actual quality of the data packet by obtaining the various packet quality parameters, and the subsequent determination of the reason causing the video quality problem and the accuracy of the equipment according to the abnormal packet quality parameters and the abnormal frame quality parameters can be improved.
According to a sixth possible implementation manner of the first aspect, in a seventh possible implementation manner of the video quality problem determining method, the determining, according to the abnormal frame quality parameter and/or the abnormal packet quality parameter, a cause and a device causing video quality abnormality at a video playing end includes: when the frame quality parameters of the video frames and the packet quality parameters of the data packets corresponding to the video frames meet a first condition, determining that the reason causing the video quality abnormity of a video playing end is a video coding problem, and determining that equipment causing the video quality abnormity of the video playing end is equipment used for video coding; wherein the first condition comprises: the frame packet loss parameter of the video frame is abnormal and the packet loss parameter of the data packet corresponding to the video frame is normal, or the frame disorder parameter of the video frame is abnormal and the disorder parameter of the data packet corresponding to the video frame is normal.
Because the protocol information of the data packet is obtained at the network equipment and the video playing end, the frame quality parameter and the packet quality parameter can reflect the abnormal condition in the transmission process, the frame packet loss parameter abnormality indicates that a data packet in the video frame is in a loss state during transmission, and if the packet loss parameter of the data packet in the video frame is normal at the moment, the data packet is not lost in the transmission process, so that the data packet can be determined to be lost before transmission, namely, during video coding; the frame disorder parameter abnormity indicates that a data packet in a video frame is in a disorder state during transmission, and if the disorder parameter of the data packet in the video frame is normal at the moment, the data packet is not in disorder during transmission, so that the data packet can be determined to be in disorder before transmission, namely during video coding; based on the reasons, the video coding problem can be accurately reflected by the first condition that the frame packet loss parameter of the video frame is abnormal and the packet loss parameter of the data packet corresponding to the video frame is normal, or the frame disorder parameter of the video frame is abnormal and the disorder parameter of the data packet corresponding to the video frame is normal.
According to a sixth possible implementation manner or a seventh possible implementation manner of the first aspect, in an eighth possible implementation manner of the video quality problem determining method, the determining, according to the abnormal frame quality parameter and/or the abnormal packet quality parameter, a cause and a device causing video quality abnormality at a video playing end includes: when the frame quality parameters of the video frames and the packet quality parameters of the data packets corresponding to the video frames meet a second condition, determining that the reason causing the video quality abnormity of the video playing end is a terminal parameter problem, and determining that the equipment causing the video quality abnormity of the video playing end is equipment used for video playing; wherein the second condition comprises: any one of the jitter parameter of the first frame packet and the jitter parameter of the last frame packet of the video frame is abnormal, and any one of the IP message fragment parameter, the check sum error parameter of the data packet corresponding to the video frame is abnormal.
Because the protocol information of the data packet is obtained at the network equipment and the video playing end, the frame quality parameter and the packet quality parameter can reflect the abnormal condition in the transmission process, the jitter parameter of the first packet/the jitter parameter of the last packet of the frame is abnormal, which indicates that the time delay jitter of the first/last data packet in the video frame is overlarge during the transmission, if the IP message fragment parameter of the data packet in the video frame is abnormal at the moment, the data packet is fragmented during the transmission process, so that the parameter configuration of the video playing end can be determined to fail to meet the requirement of the transmission of the data packet when the data packet is not fragmented; if the verification and error parameters of the data packet under the video frame are abnormal at this time, the verification of the data packet with the video playing end in the transmission process is not successfully completed, so that the requirement of successful verification of the data packet with the parameter configuration of the video playing end can be determined to be not met; based on the above reasons, the second condition that any one of the jitter parameter of the first packet and the jitter parameter of the last packet of the video frame is abnormal and any one of the fragment parameter, the checksum error parameter of the IP packet of the data packet corresponding to the video frame is abnormal can accurately reflect the parameter problem of the terminal.
According to any one of the sixth possible implementation manner to the eighth possible implementation manner of the first aspect, in a ninth possible implementation manner of the video quality problem determining method, the determining, according to the abnormal frame quality parameter and/or the abnormal packet quality parameter, a cause and a device for causing video quality abnormality at a video playing end includes: when the frame quality parameters of the video frames and the packet quality parameters of the data packets corresponding to the video frames meet a third condition, determining that the reason causing the video quality abnormity at the video playing end is the network transmission problem; wherein the third condition comprises: the frame packet loss parameter of the video frame is abnormal and the packet loss parameter of the data packet corresponding to the video frame is abnormal, or the frame disorder parameter of the video frame is abnormal and the disorder parameter of the data packet corresponding to the video frame is abnormal, or any one of the frame head packet jitter parameter and the frame tail packet jitter parameter of the video frame is abnormal and any one of the packet jitter parameter, the packet interval parameter, the retransmission parameter and the time delay parameter of the data packet corresponding to the video frame is abnormal.
Because the protocol information of the data packet is acquired at the network equipment and the video playing end, the frame quality parameter and the packet quality parameter can reflect the abnormal condition in the transmission process, the frame packet loss parameter abnormality indicates that a data packet in the video frame is in a loss state during transmission, and if the packet loss parameter abnormality of the data packet in the video frame indicates that the data packet is lost in the transmission process, the network transmission problem can be determined; the frame disorder parameter abnormity indicates that a data packet under the video frame is in a disorder state during transmission, and if the disorder parameter of the data packet under the video frame is abnormal at the moment, the data packet is in disorder during transmission, so that the problem of network transmission can be determined; based on the reasons, in a third condition, "the frame packet loss parameter of the video frame is abnormal and the packet loss parameter of the data packet corresponding to the video frame is abnormal, or the frame disorder parameter of the video frame is abnormal and the disorder parameter of the data packet corresponding to the video frame is abnormal", the network transmission problem can be accurately reflected; the abnormal frame head packet jitter parameter/frame tail packet jitter parameter indicates that the time delay jitter of the first/last data packet under the video frame is too large during transmission, and if the packet interval parameter of the data packet under the video frame is abnormal at the moment, the time difference of the two data packets during the transmission process is too large, the problem of network transmission can be determined; if the packet jitter parameter of the data packet under the video frame is abnormal at this time, the delay jitter of the data packet in the transmission process is over large, and the problem of network transmission can be determined; if the retransmission parameters of the data packet under the video frame are abnormal at this time, the data packet is transmitted for multiple times in the transmission process, and the problem of network transmission can be determined; if the time delay parameter of the data packet under the video frame is abnormal at this time, the time consumption of the data packet in the transmission process is too long, and the problem of network transmission can be determined. Based on the above reasons, in the third condition, "any one of the jitter parameter of the first packet of the video frame and the jitter parameter of the last packet of the video frame is abnormal, and any one of the jitter parameter, the packet interval parameter, the retransmission parameter, and the delay parameter of the data packet corresponding to the video frame is abnormal" can accurately reflect the problem of the terminal parameter.
According to a ninth possible implementation manner of the first aspect, in a tenth possible implementation manner of the video quality problem determining method, after determining that a cause of video quality abnormality at a video playing end is a network transmission problem, the determining, according to the abnormal frame quality parameter and/or the abnormal packet quality parameter, a cause and a device of video quality abnormality at the video playing end further includes: determining the network layer sequence number of the data packet corresponding to the abnormal packet quality parameter according to the network layer protocol information of the data packet; determining the packet time, the transmission layer sequence number and the checksum of the data packet corresponding to the abnormal packet quality parameter according to the transmission layer protocol information of the data packet; receiving network topology data on network equipment, and determining a transmission path of each data packet in a network according to the network topology data; finding out data packets of which the packet time, the network layer serial number, the transmission layer serial number and the check sum all meet the conditions in each network device on the transmission path according to the transmission path of the data packet corresponding to the abnormal packet quality parameter in the network; and according to the found packet quality parameters of the data packets meeting the conditions, determining the network equipment causing the video quality abnormity of the video playing end from the network equipment on the transmission path of the data packets.
The packet quality parameter of the data packet on each device can indicate the influence of the device on the transmission of the data packet, and the found data packet can be used for counting the influence of different network devices on the transmission of the data packet in the transmission process of the data packet and determining the specific device causing the video quality abnormity of the video playing end by finding the corresponding data packet in each network device on the transmission path.
According to a tenth possible implementation manner of the first aspect, in an eleventh possible implementation manner of the method for determining a video quality problem, the conditions that the packet time, the network layer sequence number, the transport layer sequence number, and the checksum satisfy include: and the absolute value of the difference between the packet time and the packet time of the data packet corresponding to the abnormal packet quality parameter is smaller than a preset threshold, and the network layer sequence number, the transmission layer sequence number, the checksum and the data packet corresponding to the abnormal packet quality parameter are the same.
In this way, the same packet on different network devices on the transmission path can be found. The method can further determine the packet quality parameter abnormal condition on the upstream equipment, and can accurately determine the specific network equipment causing the video quality abnormality of the video playing end when the reason causing the video quality abnormality of the video playing end is a network transmission problem.
According to a tenth possible implementation manner or an eleventh possible implementation manner of the first aspect, in a twelfth possible implementation manner of the video quality problem determining method, when there are a plurality of data packets corresponding to the abnormal packet quality parameter, there are a plurality of groups of found data packets that satisfy the condition, and according to the found packet quality parameter of the data packet that satisfies the condition, a network device that causes video quality abnormality at a video playing end is determined from among network devices on a transmission path of the data packet, including: according to the packet quality parameters of a plurality of groups of data packets meeting the conditions, determining network equipment which may cause video quality abnormity at a video playing end from each network equipment on the transmission path of the data packets for a plurality of times; counting the network equipment which is determined for many times and is possible to cause the video quality abnormity of the video playing end, wherein the network equipment which has the most occurrence times is used as the network equipment which causes the video quality abnormity of the video playing end.
The packet quality parameters of the plurality of data packets on each network device on the transmission path are determined for multiple times, so that the accuracy of the determined network device causing the video quality abnormity can be improved, and the probability of the video quality abnormity is reduced due to the same problem of the same device after the corresponding solution is executed for the determined network device causing the video quality abnormity.
In a second aspect, an embodiment of the present application provides an apparatus for determining a video quality problem, where the apparatus includes: the information receiving module is used for receiving protocol information on a video playing end and network equipment, the network equipment is used for transmitting video data to the video playing end, the protocol information indicates a sending mode of a video frame, and the video frame comprises a plurality of data packets; a parameter determining module, configured to determine, according to the protocol information, an abnormal frame quality parameter and an abnormal packet quality parameter, where the abnormal frame quality parameter indicates that a video frame to which the abnormal frame quality parameter belongs is abnormal in quality, and the abnormal packet quality parameter indicates that a data packet to which the abnormal packet quality parameter belongs is abnormal in quality; and the abnormal reason determining module is used for determining the reason and the equipment causing the video quality abnormity of the video playing end according to the abnormal frame quality parameters and/or the abnormal packet quality parameters.
According to the second aspect, in a first possible implementation manner of the video quality problem determination apparatus, the protocol information includes streaming media layer protocol information, network layer protocol information, and transport layer protocol information of a packet, where the streaming media layer protocol information includes streaming media layer protocol information of a packet on a video playing end, the network layer protocol information includes network layer protocol information of a packet on a network device and on the video playing end, the transport layer protocol information includes transport layer protocol information of a packet on a network device and on a video playing end, and determining an abnormal frame quality parameter and an abnormal packet quality parameter according to the protocol information includes: determining at least one video frame and each data packet corresponding to each frame in the at least one video frame according to the streaming media layer protocol information; determining a frame quality parameter of each frame in the at least one video frame according to the streaming media layer protocol information and the network layer protocol information; determining packet quality parameters of each data packet corresponding to each frame in the at least one video frame according to the network layer protocol information and the transmission layer protocol information; and determining the abnormal frame quality parameter according to the frame quality parameter corresponding to each frame in the at least one video frame and the frame quality parameter threshold, and determining the abnormal packet quality parameter according to the packet quality parameter of each data packet corresponding to each frame in the at least one video frame and the packet quality parameter threshold.
In a second possible implementation manner of the video quality problem determining apparatus according to the first possible implementation manner of the second aspect, the determining the abnormal frame quality parameter according to the frame quality parameter and the frame quality parameter threshold corresponding to each frame in the at least one video frame, and determining the abnormal packet quality parameter according to the packet quality parameter and the packet quality parameter threshold of each packet corresponding to each frame in the at least one video frame includes: for each frame of the at least one video frame, determining a frame quality parameter of which the value is greater than the frame quality parameter threshold value in the frame quality parameters of the video frame as the abnormal frame quality parameter; and aiming at each frame of the at least one video frame, determining the packet quality parameter with the value larger than the threshold value of the packet quality parameter in the packet quality parameters of the data packet corresponding to the video frame as the abnormal packet quality parameter.
In a third possible implementation manner of the video quality problem determining apparatus according to the first or second possible implementation manner of the second aspect, the determining the frame quality parameter of each of the at least one video frame according to the streaming media layer protocol information of the data packet includes a frame timestamp indicating a video frame to which the data packet belongs, the network layer protocol information of the data packet includes a packet time indicating a time of the data packet, the frame quality parameter includes a frame head packet jitter parameter and a frame tail packet jitter parameter, the frame head packet jitter parameter indicates a delay jitter condition of a first data packet corresponding to the video frame, the frame tail packet jitter parameter indicates a delay jitter condition of a last data packet corresponding to the video frame, and the determining the frame quality parameter of each of the at least one video frame according to the streaming media layer protocol information and the network layer protocol information includes: for each frame of the at least one video frame, determining the arrival time of a first packet and the arrival time of a last packet of the video frame according to the packet time of a data packet corresponding to the video frame, wherein the arrival time of the first packet represents the arrival time of a first data packet corresponding to the video frame, and the arrival time of the last packet represents the arrival time of a last data packet corresponding to the video frame; respectively calculating the first packet interval and the last packet interval of adjacent video frames according to the arrival time of the first packet and the arrival time of the last packet, wherein the first packet interval represents the arrival time difference of the first data packet corresponding to the adjacent video frames, and the last packet interval represents the arrival time difference of the last data packet corresponding to the adjacent video frames; respectively calculating the frame interval of the adjacent video frames according to the frame time stamps; calculating the frame head packet jitter parameter of the current video frame according to the head packet interval, the frame interval and the frame head packet jitter parameter of the previous video frame; and calculating the frame end packet jitter parameter of the current video frame according to the end packet interval, the frame interval and the frame end packet jitter parameter of the previous video frame.
In a fourth possible implementation manner of the video quality problem determination apparatus according to any one of the first to third possible implementation manners of the second aspect, the determining a frame quality parameter of each of the at least one video frame according to the stream media layer protocol information of the packet includes a stream media layer sequence number indicating an order of the packet in the video frame to which the packet belongs, the network layer protocol information of the packet includes a packet time indicating a time of the packet, the frame quality parameter includes a frame loss parameter and a frame misordering parameter, the frame loss parameter indicates a loss condition of the packet of the video frame, the frame misordering parameter indicates a misordering condition of the packet of the video frame, and the determining a frame quality parameter of each of the at least one video frame according to the stream media layer protocol information and the network layer protocol information includes: determining a frame loss parameter of the video frame according to a stream media layer sequence number of a data packet corresponding to the video frame aiming at each frame of the at least one video frame; and for each frame of the at least one video frame, sequencing the data packets corresponding to the video frame according to the sequence of the packet time of the data packets corresponding to the video frame to obtain a sequencing result corresponding to the video frame, and determining a frame disorder parameter of the video frame according to the sequencing result and the sequence number of the streaming media layer of the data packets corresponding to the video frame.
According to a fourth possible implementation manner of the second aspect, in a fifth possible implementation manner of the apparatus for determining a video quality problem, the determining, for each of the at least one video frame, a frame loss parameter of the video frame according to a sequence number of a streaming media layer of a data packet corresponding to the video frame includes: for each frame of the at least one video frame, determining the maximum value and the minimum value of the stream media layer sequence number of the data packet corresponding to the video frame and the number of the stream media layer sequence number according to the stream media layer sequence number of the data packet corresponding to the video frame, determining the theoretical value of the number of the data packets corresponding to the video frame according to the maximum value and the minimum value of the stream media layer sequence number, and determining the frame packet loss parameter of the video frame according to the difference between the theoretical value of the number of the data packets corresponding to the video frame and the number of the stream media layer sequence number of the data packet corresponding to the video frame; determining a frame disorder parameter of the video frame according to the sequencing result and the sequence number of the streaming media layer of the data packet corresponding to the video frame, including: and when the value of the stream media layer sequence number of the current data packet is greater than the stream media layer sequence number of the data packet before the current data packet in the sequencing result, determining the frame disorder parameter corresponding to the video frame according to the sequencing sequence number of the current data packet in the sequencing result, the stream media layer sequence number of the current data packet and the minimum value of the stream media layer sequence number of the data packet corresponding to the video frame.
According to any one of the first to fifth possible implementation manners of the second aspect, in a sixth possible implementation manner of the video quality problem determination apparatus, a packet quality parameter of each data packet corresponding to each frame in the at least one video frame is determined according to network layer protocol information and transport layer protocol information of the data packet, where the packet quality parameter includes: the data packet processing method comprises one or more of a packet interval parameter representing the time interval between a current data packet and a previous data packet, a packet jitter parameter representing the time delay jitter between the current data packet and the previous data packet, a disorder parameter representing the disorder condition of the data packet, a packet loss parameter representing the packet loss condition of the data packet, a retransmission parameter representing the retransmission condition of the data packet, a time delay parameter representing the time delay condition of the data packet and a corresponding response message, an IP message fragment parameter representing whether the data packet has an IP message fragment, and a checksum error parameter representing whether the checksum of the data packet has an error.
According to a sixth possible implementation manner of the second aspect, in a seventh possible implementation manner of the video quality problem determining apparatus, the determining, according to the abnormal frame quality parameter and/or the abnormal packet quality parameter, a cause and a device for causing video quality abnormality at a video playing end includes: when the frame quality parameters of the video frames and the packet quality parameters of the data packets corresponding to the video frames meet a first condition, determining that the reason causing the video quality abnormity of a video playing end is a video coding problem, and determining that equipment causing the video quality abnormity of the video playing end is equipment used for video coding; wherein the first condition comprises: the frame packet loss parameter of the video frame is abnormal and the packet loss parameter of the data packet corresponding to the video frame is normal, or the frame disorder parameter of the video frame is abnormal and the disorder parameter of the data packet corresponding to the video frame is normal.
According to a sixth possible implementation manner or a seventh possible implementation manner of the second aspect, in an eighth possible implementation manner of the video quality problem determining apparatus, the determining, according to the abnormal frame quality parameter and/or the abnormal packet quality parameter, a cause and a device that cause video quality abnormality at a video playing end includes: when the frame quality parameters of the video frames and the packet quality parameters of the data packets corresponding to the video frames meet a second condition, determining that the reason causing the video quality abnormity of the video playing end is a terminal parameter problem, and determining that the equipment causing the video quality abnormity of the video playing end is equipment used for video playing; wherein the second condition comprises: any one of the jitter parameter of the first frame packet and the jitter parameter of the last frame packet of the video frame is abnormal, and any one of the IP message fragment parameter, the checksum error parameter of the data packet corresponding to the video frame is abnormal.
According to any one of the sixth possible implementation manner to the eighth possible implementation manner of the second aspect, in a ninth possible implementation manner of the video quality problem determining apparatus, the determining, according to the abnormal frame quality parameter and/or the abnormal packet quality parameter, a cause and a device for causing video quality abnormality at a video playing end includes: when the frame quality parameters of the video frames and the packet quality parameters of the data packets corresponding to the video frames meet a third condition, determining that the reason causing the video quality abnormity of a video playing end is a network transmission problem; wherein the third condition comprises: the frame packet loss parameter of the video frame is abnormal and the packet loss parameter of the data packet corresponding to the video frame is abnormal, or the frame disorder parameter of the video frame is abnormal and the disorder parameter of the data packet corresponding to the video frame is abnormal, or any one of the frame head packet jitter parameter and the frame tail packet jitter parameter of the video frame is abnormal and any one of the packet jitter parameter, the packet interval parameter, the retransmission parameter and the time delay parameter of the data packet corresponding to the video frame is abnormal.
According to a ninth possible implementation manner of the second aspect, in a tenth possible implementation manner of the video quality problem determining apparatus, after determining that a cause of video quality abnormality at a video playing end is a network transmission problem, the determining, according to the abnormal frame quality parameter and/or the abnormal packet quality parameter, a cause and a device of video quality abnormality at the video playing end further includes: determining the network layer sequence number of the data packet corresponding to the abnormal packet quality parameter according to the network layer protocol information of the data packet; determining the packet time, the transmission layer sequence number and the checksum of the data packet corresponding to the abnormal packet quality parameter according to the transmission layer protocol information of the data packet; receiving network topology data on network equipment, and determining a transmission path of each data packet in a network according to the network topology data; finding out data packets of which the packet time, the network layer serial number, the transmission layer serial number and the check sum all meet the conditions in each network device on the transmission path according to the transmission path of the data packet corresponding to the abnormal packet quality parameter in the network; and according to the found packet quality parameters of the data packets meeting the conditions, determining the network equipment causing the video quality abnormity of the video playing end from the network equipment on the transmission path of the data packets.
In an eleventh possible implementation manner of the video quality problem determination apparatus according to the tenth possible implementation manner of the second aspect, the conditions that the packet time, the network layer sequence number, the transport layer sequence number, and the checksum satisfy include: and the absolute value of the difference between the packet time and the packet time of the data packet corresponding to the abnormal packet quality parameter is smaller than a preset threshold, and the network layer sequence number, the transmission layer sequence number, the checksum and the data packet corresponding to the abnormal packet quality parameter are the same.
According to a tenth possible implementation manner or an eleventh possible implementation manner of the second aspect, in a twelfth possible implementation manner of the video quality problem determining apparatus, when there are a plurality of data packets corresponding to the abnormal packet quality parameter, there are a plurality of groups of found data packets that satisfy the condition, and according to the found packet quality parameter of the data packet that satisfies the condition, a network device that causes video quality abnormality at a video playing end is determined from among network devices on a transmission path of the data packet, including: according to the packet quality parameters of a plurality of groups of data packets meeting the conditions, determining network equipment which may cause video quality abnormity at a video playing end from each network equipment on the transmission path of the data packets for a plurality of times; counting the network equipment which is determined for many times and is possible to cause the video quality abnormity of the video playing end, wherein the network equipment which has the most occurrence times is used as the network equipment which causes the video quality abnormity of the video playing end.
In a third aspect, an embodiment of the present application provides an apparatus for determining a video quality problem, including: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to perform the video quality problem determination method of the first aspect described above or of one or several of many possible implementations of the first aspect.
In a fourth aspect, embodiments of the present application provide a non-transitory computer-readable storage medium, on which computer program instructions are stored, where the computer program instructions, when executed by a processor, implement the video quality problem determination method of the first aspect or one or more of the many possible implementation manners of the first aspect.
In a fifth aspect, embodiments of the present application provide a computer program product, which includes computer readable code or a non-transitory computer readable storage medium carrying computer readable code, when the computer readable code runs in an electronic device, a processor in the electronic device executes the video quality problem determination method of the first aspect or one or more of the many possible implementations of the first aspect.
These and other aspects of the present application will be more readily apparent from the following description of the embodiment(s).
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate exemplary embodiments, features, and aspects of the application and, together with the description, serve to explain the principles of the application.
Fig. 1 illustrates an exemplary application scenario of a video quality problem determination method according to an embodiment of the present application.
Fig. 2 shows an example of identifying a data stream type according to an embodiment of the present application.
Fig. 3 shows an exemplary method schematic diagram of a video quality problem determination method according to an embodiment of the application.
Fig. 4 illustrates a schematic diagram of an exemplary method for determining an abnormal frame quality parameter and/or an abnormal packet quality parameter according to an embodiment of the present application.
Fig. 5 shows an example of a video frame and a method for determining a packet corresponding to the video frame according to an embodiment of the present application.
Fig. 6 shows an example of a way of calculating the first frame packet jitter parameter and the last frame packet jitter parameter according to an embodiment of the present application.
Fig. 7 shows an example of a manner of calculating a frame loss parameter according to an embodiment of the present application.
Fig. 8 shows an example of a manner of calculating a frame misordering parameter according to an embodiment of the present application.
FIG. 9 illustrates an example of a normal distribution probability curve generated according to an embodiment of the present application.
Fig. 10 illustrates an example of implementing abnormal situation determination of parameters according to an embodiment of the present application.
Fig. 11 is an exemplary diagram illustrating correspondence between an abnormal frame quality parameter, an abnormal packet quality parameter, a cause of video quality abnormality at a video player end, and a device according to an embodiment of the present application.
Fig. 12 shows an example of determining a specific network device causing video quality abnormality at a video playing end according to an embodiment of the present application.
Fig. 13 shows an exemplary structural diagram of a video quality problem determination apparatus according to an embodiment of the present application.
Fig. 14 shows an exemplary structural diagram of a video quality problem determination apparatus according to an embodiment of the present application.
Detailed Description
Various exemplary embodiments, features and aspects of the present application will be described in detail below with reference to the accompanying drawings. In the drawings, like reference numbers can indicate functionally identical or similar elements. While the various aspects of the embodiments are presented in drawings, the drawings are not necessarily drawn to scale unless specifically indicated.
The word "exemplary" is used exclusively herein to mean "serving as an example, embodiment, or illustration. Any embodiment described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments.
Furthermore, in the following detailed description, numerous specific details are set forth in order to provide a better understanding of the present application. It will be understood by those skilled in the art that the present application may be practiced without some of these specific details. In some instances, methods, means, elements and circuits that are well known to those skilled in the art have not been described in detail so as not to obscure the present application.
In the prior art, in order to realize the location of the cause of video quality problems caused by video frame abnormalities, such as blocking, screen splash, and the like, a scheme is provided for acquiring a large amount of data based on equipment in the existing network (i.e., the network where the video frame with the abnormal occurrence is located), and then analyzing the acquired data by utilizing the statistical distribution of big data to find out the cause and the equipment causing the video quality problems. The specific implementation manner of the technical scheme is as follows:
user experience data and resource management data are obtained from user equipment (namely video playing equipment) through data acquisition modes such as probe acquisition and equipment reporting, and network topology data are obtained from network equipment through the data acquisition modes reported by the equipment. The user experience data includes a video mean opinion score (vMOS) of the user on the video, a pause duration, a pause frequency, a video quality switching number (for example, a switching number of a video display format between 1080P, 720P, and 360P), and the like. The user experience data is used for representing the quality of experience and impression of a user when the user watches the video, and the video playing quality can be determined according to the user experience data. The asset management data includes network access information of the user. The resource management data is used to characterize a connection relationship between the user device and the network device. The network topology data is used for representing the connection relation between the network devices, and the total number of users connected to each network device can be determined according to the resource management data and the network topology data.
For each network device, the percentage of the total number of users occupied by the number of users with poor video playing quality on the network device may be determined as the quality difference rate of the network device. And counting the quality difference rate of each network device, and determining the distribution condition of the quality difference rate of the network devices in the network according to the connection relation between the network devices.
And determining the quality of experience (QoE) according to the determined quality difference rate of each network device and a preset threshold. The quality of experience includes calculating a variation coefficient of the quality difference rate, a mean value of the quality difference rate, and the like, and determining the problem device according to the quality of experience, for example, determining a degree of deviation between the variation coefficient of the quality difference rate and a preset threshold or a degree of deviation between the mean value of the quality difference rate and a preset threshold, and finding the device causing the large deviation according to the degree of deviation.
Determining whether the network equipment has lower layer network equipment or not according to the connection relation among the network equipment, if not, the network equipment is the network equipment which causes video quality problems and is determined by the scheme in the prior art; if so, the reason and the equipment causing the video quality problem are further confirmed through the experience quality of the lower-layer network equipment.
The statistical distribution scheme using big data proposed by the prior art has certain disadvantages. Firstly, the scheme determines the reason of video quality problem once, the time cost of equipment is high, and the real-time performance of the scheme is poor. Secondly, the scheme needs to perform overall analysis on the data acquired when the video quality problem is generated for multiple times, that is, the reason and the equipment for causing the video quality problem determined in the prior art are also determined when the video quality problem is generated for multiple times, and the reason and the equipment for generating the video quality problem for multiple times may be different. The reason and the equipment which are determined by the prior art and cause the video quality problem may be only one of the reason and the equipment which cause the video quality problem for a plurality of times, and the determined reason and the equipment which cause the video quality problem are not accurate enough. Even if the device is optimized according to the determined cause and device causing the video quality problem, the probability that some same video quality problems occur again in the following cannot be reduced.
In summary, the prior art cannot quickly and accurately determine the reason and the device causing the video quality problem.
In order to solve the technical problem, the present application provides a method, an apparatus, a storage medium, and a program product for determining a video quality problem.
Fig. 1 illustrates an exemplary application scenario of a video quality problem determination method according to an embodiment of the present application.
As shown in fig. 1, the scene may be, for example, a live video scene, and the scene may include a video provider for providing video data, a plurality of network devices for transmitting the video data, and a video player for presenting video pictures for a user. The video providing end can be a camera with video data acquisition capability, or a device capable of receiving, storing and outputting video data. The video providing end can encode the collected or received video data and package the video data into a live streaming form according to a streaming media layer protocol. The encapsulated video data may be transmitted through a network device, which may include, for example, a wireless base station, a bearer network device, a core network device, a switch, etc. in fig. 1. The radio base station may be, for example, a 5G base station (next generation NB, gNB). The core network device may be, for example, a Unified Distributed Gateway (UDG) or the like, and the switch may be, for example, an end of row (EoR) or the like using a certain deployment manner. The video data is finally output to a video playing end, and the video playing end can be a mobile phone, a tablet computer, a notebook computer and the like. On the video playing end, the video data in the form of live streaming can be unpacked, decoded and rendered, so that a user can watch videos through the video playing end.
The application scenario in fig. 1 further includes a terminal device (or server) for determining the reason and device of the video quality problem, where the terminal device (or server) may be different from the video provider, the network device, and the video player. The terminal device (or server) may include the video quality problem determination apparatus according to the embodiment of the present application, and may be configured to execute the video quality problem determination method according to the embodiment of the present application.
The terminal device (or the server) may further include a protocol information collecting device for collecting protocol information from the network device and the video playing terminal, and a topology data collecting device for collecting network topology data from the network device. Alternatively, the video quality problem determination device, the protocol information acquisition device, and the topology data acquisition device may be respectively disposed on a single terminal device or server, or any two of the three devices may be disposed on the same terminal device or server.
Those skilled in the art should understand that the application scenarios of the embodiments of the present application are not limited to the above-described apparatuses. For example, there may be a plurality of video playing terminals connected to the network device, and for the video quality problem of each video playing terminal, the video quality problem determination method according to the embodiment of the present application may be respectively executed on the terminal device (or the server).
In a possible implementation manner, taking the application scenario of fig. 1 as an example, the protocol information collecting device and the topology data collecting device may collect protocol information and network topology data respectively. When the video providing end continuously outputs the live stream, the protocol information acquisition device can continuously acquire the protocol information on the video playing end and the network equipment. If the connection relation of the network equipment is not changed in the process, the topology data acquisition device acquires the network topology data once. According to the protocol information on the video playing end and the network equipment, the reason and the equipment causing the video quality abnormity of the video playing end can be determined by adopting the video quality problem determining method.
An exemplary method for collecting protocol information in the embodiments of the present application is described below.
In one possible implementation, collecting protocol information may be implemented based on existing technology. For example, the protocol information acquisition device first acquires data streams on a video playing terminal and network equipment. The data stream on the video player and the network device may include a video stream (video) and may also include other types of data streams such as a voice stream. The protocol information acquisition device can be combined with a streaming media protocol to identify the acquired data stream and acquire the data packet of which the type is the video stream from the data stream. After the data with the type of the video stream is determined, the protocol information acquisition device can further acquire the stream media layer protocol information, the network layer protocol information and the transport layer protocol information of the data packet from the data with the type of the video stream according to the stream media layer protocol, the network layer protocol, the transport layer protocol and the like, thereby completing the acquisition of the protocol information.
The collection of the protocol information can be realized by deploying a probe, or by carrying out network port packet capturing through equipment such as a computer, or by realizing the functions of user tracking and the like of the network equipment. The protocol information acquisition by the probe is described herein as an example. The probe can have Deep Packet Inspection (DPI) capability, and can automatically recognize different types of data streams and acquire protocol information.
The data packet may include a payload header and a payload, where the payload portion includes the specific content of the data. The payload header includes protocol information indicating a transmission mode of the data packet, and in order to ensure data security, payload encryption may be performed on the data packet when the protocol information is acquired, or only the payload header (packet header) of the data packet may be detected. For example, the stream media layer payload header, the network layer payload header, and the transport layer payload header of the data packet at the video playing end can be detected and the corresponding stream media layer protocol information, network layer protocol information, and transport layer protocol information can be extracted, and the network layer payload header, the transport layer payload header, and the corresponding network layer protocol information, and transport layer protocol information can be extracted for the data packet at other network devices.
For example, for a payload header of a streaming media layer, protocol information extraction may be implemented through identification of an application type and a protocol type. The application type of a certain data stream in the data can be identified by using a situation awareness technology (SA). For example, when a user watches live video at a certain time period by using a certain application program, the video quality abnormality occurs, and a data stream which is related to the video quality abnormality and needs to be analyzed can be obtained from a data stream on a video playing end used by the user for watching the live video according to the user identity information, the time information of a data packet, the type of the application program and/or the like. The default is to get from all data streams on the video player side. For the obtained data stream, the type of the data stream can be identified layer by layer from a network layer, a transmission layer, a stream media layer, an application layer and the like by using a judgment mode specified by the existing protocol, the video stream existing in the data stream is determined and distinguished according to the specific type of the video stream, and the type of the video stream containing the video frame is determined. Fig. 2 shows an example of identifying a data stream type according to an embodiment of the present application.
As shown in fig. 2, the protocols used by the network layer may include Internet Protocol (IP), and the protocols used by the transport layer may include Transmission Control Protocol (TCP), user Datagram Protocol (UDP). Protocols used by the streaming media layer may include real-time streaming protocol (RTSP), real-time messaging protocol (RTMP), real-time transport protocol (RTP), real-time transport control protocol (RTCP), user datagram protocol (UDP-based data transfer protocol (UDT), or secure reliable transport protocol (SRT), and MPEG 2transport stream (MPEG-2transport stream, MPEG 2-TS). The protocol used by the application layer may include the h.264 video standard protocol or the h.265 video standard protocol.
Referring to fig. 2, according to the protocol used by the transport layer, the protocol used by the streaming media layer, and the protocol used by the application layer, the supportable video stream types may include: TCP + RTMP video stream, TCP + RTSP + RTP video stream, UDP + RTP + MPEG2-TS video stream, UDP + SRT/UDT + MPEG2-TS video stream, and UDP + MPEG2-TS video stream. The above protocol may also support control streams, such as the TCP + RTMP control stream, the TCP + RTSP control stream, the UDP + RTCP control stream, etc. of fig. 2, which do not contain video frames.
For different video streams, the protocol information may be obtained from data packets in the video streams. The protocol information may include network layer protocol information, transport layer protocol information, streaming media layer protocol information of packets in the video stream, and a calculated transport layer CHECKSUM (hereinafter referred to as CHECKSUM) derived from a standard calculation method of a CHECKSUM (cheksum).
An exemplary acquisition method of network topology data in the embodiment of the present application is described below.
For example, the network topology data may be collected from the network devices. The network topology data may include a topology connection relationship or a service path of each network device in a network where the network device connected to the video playing terminal is located. Wherein the traffic path characterizes a connection relationship between network devices through which the traffic flow passes. So that the upstream and downstream relationship of the network device can be determined from the network topology data.
After the protocol information and the network topology data are acquired, the protocol information acquisition device and the topology data acquisition device can respectively transmit the acquired information or data to the video quality problem determination device. The video quality problem determination apparatus uses the video quality problem determination method according to the embodiment of the present application. The protocol information and the network topology data can be used for determining the reason and equipment causing the video quality abnormity of the video playing end.
Fig. 3 shows an exemplary method schematic of a video quality problem determination method according to an embodiment of the application.
As shown in fig. 3, in a possible implementation manner, a method for determining a video quality problem according to an embodiment of the present application includes:
the method comprises the steps that S1, protocol information on a video playing end and network equipment is received, the network equipment is used for transmitting video data to the video playing end, the protocol information indicates a sending mode of a video frame, and the video frame comprises a plurality of data packets;
s2, determining abnormal frame quality parameters and abnormal packet quality parameters according to the protocol information, wherein the abnormal frame quality parameters represent the quality abnormality of the video frame to which the abnormal frame quality parameters belong, and the abnormal packet quality parameters represent the quality abnormality of the data packet to which the abnormal packet quality parameters belong;
and S3, determining the reason and equipment causing the video quality abnormity of the video playing end according to the abnormal frame quality parameter and/or the abnormal packet quality parameter.
According to the video quality problem determining method, the protocol information sent by the video playing end and the protocol information sent by the network equipment for transmitting the video data to the video playing end are received, so that the sending modes of the video frames and a plurality of data packets included in the video frames can be determined according to the protocol information, and further abnormal frame quality parameters and abnormal packet quality parameters can be determined. The abnormal frame quality parameters represent the quality abnormality of the video frame, the abnormal packet quality parameters represent the quality abnormality of the data packet, and the video quality, the video frame and the data packet have a correlation relationship, so the abnormal frame quality parameters and the abnormal packet quality parameters are correlated with the occurring video quality abnormality, and the reason and the equipment causing the video quality abnormality at the video playing end can be determined according to the abnormal frame quality parameters and the abnormal packet quality parameters. According to the video quality problem determining method, the reason and the equipment which cause video quality abnormity at the video playing end are determined through the combination of the frame-level abnormal frame quality parameters and the packet-level abnormal packet quality parameters, the time cost for completing the determination of the reason and the equipment at one time can be reduced, and the accuracy of the determined reason and the equipment is improved.
According to the method for determining the video quality problem, when some equipment can only obtain the payload head, the protocol information can be extracted according to the payload head, the reason and the equipment which cause the video quality abnormity of the video playing end can be accurately determined, and the application range of the method for determining the video quality problem can be enlarged.
For example, in step S1, protocol information may be acquired. In step S2, identification of the video frame and extraction of the data packet under the video frame can be realized. For video quality problems such as video screen splash and video stuck, frame quality parameters can be defined to represent the quality of video frames. The frame quality parameters may be multiple, such as a frame head packet jitter parameter, a frame tail packet jitter parameter, a frame packet loss parameter, and a frame out-of-order parameter, which are mentioned below. Different frame quality parameters can be preset with different calculation modes, the value of the frame quality parameter can be calculated according to the protocol information through the preset calculation mode of the frame quality parameter, and the value of the frame quality parameter obtained through calculation is subjected to abnormity detection through the frame quality parameter threshold value, so that the frame quality parameter with abnormal value, namely the abnormal frame quality parameter, is determined. According to the abnormal frame quality parameters, the corresponding abnormal video frame and all data packets under the abnormal video frame can be further determined. Packet quality parameters may be defined to characterize the quality of a data packet. There may be a plurality of packet quality parameters, such as packet interval parameter, packet jitter parameter, packet loss parameter, out-of-order parameter, retransmission parameter, delay parameter, IP packet fragmentation parameter, checksum error parameter, which are mentioned hereinafter. Different calculation modes can be preset for different packet quality parameters, the numerical value of the packet quality parameter can be calculated according to protocol information through the preset calculation mode of the packet quality parameter, and the abnormal detection is carried out on the numerical value of the packet quality parameter obtained through calculation through a packet quality parameter threshold value, so that the packet quality parameter with abnormal numerical value, namely the abnormal packet quality parameter, is determined.
In step S3, the reason and the device causing the video quality abnormality are determined according to whether the frame quality parameter of the abnormality of the abnormal video frame and the packet quality parameter of the abnormality of the data packet under the abnormal video frame satisfy the corresponding condition. The condition may be plural, such as a first condition, a second condition, and a third condition mentioned hereinafter. Wherein, when different conditions are met, the determined causes and equipment causing the video quality abnormity may be different. If the device causing the video quality abnormity can be uniquely determined according to the determined reason causing the video quality abnormity. For example, if it is determined that the cause of the video quality abnormality is a video encoding problem, the only device causing the video quality abnormality may be the device used for video encoding. For another example, it is determined that the cause of the video quality abnormality is a terminal parameter problem, and then it can be uniquely determined that the device causing the video quality abnormality is a device used for video playing, that is, a video playing end. In this case, the video quality problem determination method according to the embodiment of the present application has achieved determination of the cause and the device causing the video quality abnormality. If the equipment causing the video quality abnormity cannot be uniquely determined according to the determined reason causing the video quality abnormity, for example, the reason causing the video quality abnormity is determined to be the network transmission problem, and the equipment causing the video quality abnormity cannot be uniquely determined because a plurality of network equipment can participate in the network transmission process. According to the method and the device, the abnormal data packet can be determined according to the abnormal packet quality parameters, the plurality of data packets of which the protocol information meets the conditions on the plurality of network devices participating in the network transmission process are found according to the protocol information of the abnormal data packet, and the devices causing the video quality abnormality are analyzed and determined according to the found packet quality parameters of the plurality of data packets.
When the video quality problem determination method of the embodiment of the present application is executed to determine the reason and the device that cause video quality abnormality at the video playing end at the present time, the frame quality parameter threshold and the packet quality parameter threshold that are used may be obtained according to AI model determination. The AI model can be trained by taking the frame quality parameters and the packet quality parameters calculated within a period of time before the reason for determining the video quality abnormality of the video playing end and the equipment at the current time are determined as samples, so that the problems that the common samples need to be marked and the data of the abnormal sample modeling is difficult to obtain can be solved, and the complexity of determining the abnormal frame quality parameters and the abnormal packet quality parameters is reduced.
As can be seen from the above description, the abnormal frame quality parameter may be an abnormal parameter among the frame quality parameters, and the abnormal packet quality parameter may be an abnormal parameter among the packet quality parameters. In step S2, according to the protocol information, a frame quality parameter and a packet quality parameter may be determined first, and then an abnormal frame quality parameter and an abnormal packet quality parameter may be determined according to the frame quality parameter and the packet quality parameter.
Fig. 4 illustrates a schematic diagram of an exemplary method for determining an abnormal frame quality parameter and/or an abnormal packet quality parameter according to an embodiment of the present application.
As shown in fig. 4, in a possible implementation manner, the protocol information includes streaming media layer protocol information, network layer protocol information, and transport layer protocol information of the data packet, the streaming media layer protocol information includes streaming media layer protocol information of the data packet on the video playing end, the network layer protocol information includes network layer protocol information of the data packet on the network device and on the video playing end, the transport layer protocol information includes transport layer protocol information of the data packet on the network device and on the video playing end, and step S2 includes:
s21, determining at least one video frame and each data packet corresponding to each frame in the at least one video frame according to the streaming media layer protocol information.
S22, determining the frame quality parameter of each frame in the at least one video frame according to the stream media layer protocol information and the network layer protocol information; the frame quality parameters may include a frame head packet jitter parameter, a frame tail packet jitter parameter, a frame packet loss parameter, and a frame out-of-order parameter.
S23, determining the packet quality parameters of each data packet corresponding to each frame in the at least one video frame according to the network layer protocol information and the transmission layer protocol information; the packet quality parameters may include packet interval parameters, packet jitter parameters, packet loss parameters, out-of-order parameters, retransmission parameters, delay parameters, IP packet fragmentation parameters, checksum error parameters.
S24, determining the abnormal frame quality parameters according to the frame quality parameters and the frame quality parameter threshold values corresponding to the frames in the at least one video frame, and determining the abnormal packet quality parameters according to the packet quality parameters and the packet quality parameter threshold values of the data packets corresponding to the frames in the at least one video frame.
In this way, the frame quality parameter and the packet quality parameter can be determined through the combination of different protocol information so as to embody the quality of the video frame and the quality of the data packet under the video frame. Whether the frame quality parameter and the packet quality parameter are abnormal or not can be determined through a frame quality parameter threshold and a packet quality parameter threshold respectively, so that the mode of determining the abnormal frame quality parameter and the abnormal packet quality parameter is more convenient.
Step S21 may be implemented based on the prior art. For example, in step S21, for the obtained protocol information, identification of a video frame and extraction of a data packet corresponding to the video frame are performed according to different types of video streams (see fig. 2 and the related description above). A common method is to find a packet with a streaming media type in a video stream according to the streaming media layer protocol information of the packet (see step S211 below for example). Extracting the information of the stream media type from the stream media layer protocol information of the found data packets with the stream media type (see step S212 below for example), obtaining a stream media layer sequence number from the stream media layer protocol information of the data packets with the stream media type being video, and then finding the data packets for calculating the obtained packet quality parameters and the frame timestamps of the data packets according to the stream media layer sequence number. All the frame time stamps of the packets used for calculating the obtained packet quality parameter are counted, and the packets having the same frame time stamp are taken as the packets belonging to the same video frame (see step S213 below for an example). In this case, each frame in at least one video frame may include a data packet having the same frame time stamp, and the frame time stamps of the data packets in different video frames may be different, so that each video frame and each data packet corresponding to each video frame can be determined.
Wherein, for the video Stream carrying MPEG2-TS (UDP + RTP + MPEG2-TS video Stream, UDP + SRT/UDT + MPEG2-TS video Stream, UDP + MPEG2-TS video Stream), the type of the Stream media can be determined by the value under Stream type under TS header in the protocol information. The sequence number of the streaming media layer can be determined by the value of Stream PID when the value under Stream type in the protocol information is Video; for Video streams (TCP + RTSP + RTP Video stream, UDP + RTP Video stream) carrying RTP except the Video stream carrying MPEG2-TS, the type of the streaming media can be determined by the value under Payload type in the protocol information, and the SEQUENCE NUMBER of the streaming media layer can be determined by the value of RTP SEQUENCE NUBER when the value under Payload type is Video; for Video streams carrying RTMP (TCP + RTMP Video streams) except the Video stream carrying the MPEG2-TS and the Video stream carrying the RTP, the Type of the streaming media can be determined by the value under the Type ID in the protocol information, and the sequence number of the streaming media layer can be determined by the protocol information in the data packet of which the value under the Type ID is Video.
Fig. 5 shows an example of a video frame and a method for determining a packet corresponding to the video frame according to an embodiment of the present application.
As shown in fig. 5, taking the video stream of type UDP + RTP + MPEG2-TS as an example, the exemplary process of step S21 is as follows:
s211, in the acquired protocol information, the type of the data is determined according to the value of the TS PID field of the TS header. For example, the TS PID field value of TS 1# in fig. 5 is 0, and the data type of the TS transport segment to which the TS header belongs is a Program Association Table (PAT). And after the data type of the TS transmission segment is determined to be a Program association table, determining a data packet with the streaming media type according to the value of a Program Map PID in the TS header. For example, the Program Map PID field value of TS 1# in fig. 5 is 0x0010f. In the protocol information, all TS transport segments including the obtained Program Map PID value (0 x0010 f), for example, TS transport segments corresponding to TS 8# in fig. 5, are packets with streaming media type. The method of step S211 is executed for all TS headers in the streaming media layer protocol information, and all data packets with streaming media type are determined.
S212, from the TS head of the data packet with the determined Stream media type, the Stream media type is determined according to the Stream type of the program mapping table. For example, as described in S211, TS 8# in fig. 5 is a TS header of a packet having a streaming media type. In TS 8#, stream type of a Program Map Table (PMT) is Video, and a streaming media type of a packet to which TS 8# belongs is Video. And after the Stream media type is determined to be the video, determining the Stream media layer sequence number of the data packet of which the Stream media type is the video according to the value of Stream PID of the program mapping table. For example, in TS 8#, the value of the Stream PID field of PMT is 0x0100, which is the Stream media layer sequence number of the packet to which TS 8# belongs. The protocol information includes a packet of the obtained Stream PID value (0 x 0100), that is, a packet of the streaming media type being video. For example, the data packets corresponding to TS 11#, TS 12#, TS 13#, and TS 14# in fig. 5 are data packets whose streaming media type is video. The method of step S212 is performed for the TS headers of all the packets with the streaming media type, and all the packets with the streaming media type being video are found.
S213, combining or encapsulating the data packets of at least one TS transmission segment of the video stream media type to obtain a plurality of data packets. After combining or encapsulating, each data packet includes a plurality of TS transmission segments, for example, as shown in fig. 5, data packets corresponding to TS 11# and TS 12# are combined to obtain one data packet. And combining the data packets corresponding to the TS 13# and the TS 14# to obtain a data packet. The data packets may be randomly combined. These combined or encapsulated packets may be packets used to calculate the derived packet quality parameter. The protocol information includes data of a PES header, a Decoding Time Stamp (DTS) is extracted from the PES header, and a unit of the DTS is converted into a second. The value obtained after the unit conversion is the frame timestamp of the data packet, that is, the frame timestamp of the video frame to which the data packet belongs. Since the video frames are identified by the frame time stamps of the streaming media layer, the packets having the same frame time stamp can be considered as packets belonging to the same video frame. For all the data packets with the streaming media type being video, the method of step S213 is executed, and different video frames are identified and the data packets corresponding to the different video frames are determined.
Step S21, after the identification of different video frames and the determination of corresponding data packets are realized, the data packets with the stream media layer can be further determined, frame time stamps, stream media layer sequence numbers and the like are obtained from the stream media layer of the data packets, and a transmission layer sequence Number, a checksum, an acknowledgement message sequence Number (ACK Number) and the like are obtained from a transmission layer of the data packets; the network layer sequence number, the Fragment flag (DF), the packet time, etc. are obtained from the network layer of the data packet, and these information can be used in the process of calculating the frame quality parameter and the packet quality parameter.
It will be understood by those skilled in the art that the at least one video frame may also include a portion of the entire video frames (frames to which the data packets of which the streaming media type is video belong) identified by step S21. The method can further comprise the step of screening at least one video frame from the identified video frames according to whether the frame time stamps of the video frames are positioned in the same time period or not, or determining at least one video frame from the identified video frames according to the sequence numbers of the stream media layers of data packets in the identified video frames, or screening at least one video frame from the identified video frames according to a preset sampling mechanism. As long as the type of the streaming media that satisfies the data packet in the at least one video frame is a video, the present application does not limit the specific manner of determining the at least one video frame according to the streaming media layer protocol information.
In step S22, for the different video frames identified in step S21, the frame quality parameters of each video frame are obtained according to the streaming media layer protocol information and the network layer protocol information of the data packets corresponding to the different video frames. The frame quality parameters of the video frame may, for example, include one or more of a frame start packet jitter parameter, a frame end packet jitter parameter, a frame packet loss parameter, a frame out-of-order parameter. In combination with the frame quality parameter of the video frame and the packet quality parameter of the data packet corresponding to the video frame, the cause and the device causing the video quality problem may be determined, for example, it is determined that the cause causing the video quality problem is a video encoding problem and the device causing the video quality problem is a device used for video encoding, or it is determined that the cause causing the video quality problem is a video playing problem and the device causing the video quality problem is a device used for video playing, or it is determined that the cause causing the video quality problem is a network transmission problem and the device causing the video quality problem is a network device, and so on. The following describes an exemplary manner of calculating different frame quality parameters.
Fig. 6 shows an example of a way of calculating the first frame packet jitter parameter and the last frame packet jitter parameter according to an embodiment of the present application.
In one possible implementation, the streaming media layer protocol information of the data packet includes a frame timestamp indicating a video frame to which the data packet belongs. The network layer protocol information of the data packet includes a packet time indicating a time of the data packet. The frame quality parameters comprise frame head packet jitter parameters and frame tail packet jitter parameters. The frame head packet jitter parameter represents the time delay jitter condition of the first data packet corresponding to the video frame, the frame tail packet jitter parameter represents the time delay jitter condition of the last data packet corresponding to the video frame,
step S22, comprising:
and determining the arrival time of a first packet and the arrival time of a last packet of the video frame according to the packet time of a data packet corresponding to the video frame aiming at each frame of the at least one video frame. The arrival time of the head packet represents the arrival time of the first data packet corresponding to the video frame, and the arrival time of the tail packet represents the arrival time of the last data packet corresponding to the video frame;
and respectively calculating the head packet interval and the tail packet interval of the adjacent video frames according to the arrival time of the head packet and the arrival time of the tail packet. The head packet interval represents the arrival time difference of the first data packet corresponding to the adjacent video frame, and the tail packet interval represents the arrival time difference of the last data packet corresponding to the adjacent video frame.
Respectively calculating the frame interval of the adjacent video frames according to the frame time stamps;
calculating the frame head packet jitter parameter of the current video frame according to the head packet interval, the frame interval and the frame head packet jitter parameter of the previous video frame;
and calculating the frame end packet jitter parameter of the current video frame according to the end packet interval, the frame interval and the frame end packet jitter parameter of the previous video frame.
By the method, a frame head packet jitter parameter representing the time delay jitter condition of the first data packet corresponding to the video frame and a frame tail packet jitter parameter representing the time delay jitter condition of the last data packet corresponding to the video frame can be obtained through calculation, and the approaching degree of the frame quality parameter of the video frame and the actual jitter condition of the video frame is improved while the quality of the video frame is represented.
For example, the frame head packet jitter parameter and the frame tail packet jitter parameter in the embodiment of the present application may be used to represent the delay jitter of the data packet corresponding to the video frame, and the video frame screen-lost and pause probability is higher at the time corresponding to the data packet with the larger delay jitter.
After the at least one video frame and the data packet corresponding to the at least one video frame are obtained according to the step S21, the frame head packet jitter parameter and the frame tail packet jitter parameter can be calculated in the following manner:
for example, taking fig. 6 as an example, the at least one video frame may include 4 frames, for each video frame 1 to 4, according to the packet time indicating the time of the data packet in the network layer protocol information of the data packet corresponding to the video frame, a data packet with the earliest packet time in the data packets corresponding to the video frame may be named as a first packet, and the packet time of the first packet is the arrival time of the first packet. The data packet with the latest packet time is named as a tail packet, and the packet time of the tail packet is the arrival time of the tail packet. Based on this, the arrival times of the first packets of the video frames 1-4 can be recorded as T1, T2, T3, and T4, respectively, and the arrival times of the last packets of the video frames 1-4 can be recorded as T1', T2', T3', and T4', respectively.
Calculating the head packet interval and the tail packet interval of adjacent video frames according to the head packet arrival time and the tail packet arrival time of each video frame in the video frames 1-4, wherein when n is more than or equal to 2 and less than or equal to 4 and is an integer, the head packet interval Deltan of the video frame n and the video frame n-1 can be calculated by the difference between the head packet arrival time Tn of the video frame n and the head packet arrival time Tn-1 of the video frame n-1, the tail packet interval Deltan 'of the video frame n and the video frame n-1 can be calculated by the difference between the tail packet arrival time Tn' of the video frame n and the tail packet arrival time Tn-1 'of the video frame n-1, and when n =1, the head packet interval Delta1 and the tail packet interval Delta1' of the video frame 1 are equal to 0 by default. Taking fig. 6 as an example, according to the arrival times T1, T2, T3, and T4 of the first packets of the video frames 1-4, the first packet interval Delta1=0 of the video frame 1, the first packet interval Delta2= T2-T1 of the video frame 2 and the video frame 1, the first packet interval Delta3= T3-T2 of the video frame 3 and the video frame 2, and the first packet interval Delta4= T4-T3 of the video frame 4 and the video frame 3 can be obtained. According to the end packet arrival times T1', T2', T3', T4' of the video frames 1-4, the end packet interval Delta1'=0 of the video frame 1, the end packet interval Delta2' = T2'-T1' of the video frame 2 and the video frame 1, the end packet interval Delta3'= T3' -T2 'of the video frame 3 and the video frame 2, and the end packet interval Delta4' = T4'-T3' of the video frame 4 and the video frame 3 can be obtained.
For each of the video frames 1-4, the frame interval of the adjacent video frames can be calculated according to the frame timestamp of the video frame to which the indication packet belongs in the network layer protocol information of the data packet corresponding to the video frame. Taking fig. 6 as an example, the frame timestamps of video frames 1-4 can be recorded as TS1, TS2, TS3, and TS4, respectively. N is greater than or equal to 2 and less than or equal to 4, and is an integer, the frame Interval of video frame n and video frame n-1 can be calculated by the difference between the frame timestamp TSn of video frame n and the frame timestamp TSn-1 of video frame n-1, and when n =1, the frame Interval1 of video frame 1 is equal to 0 by default. In this case, the frame Interval1=0 of the video 1, the frame Interval2= TS2-TS1 of the video frame 2 and the video frame 1, the frame Interval3= TS3-TS2 of the video frame 3 and the video frame 2, and the frame Interval4= TS4-TS3 of the video frame 4 and the video frame 3 can be obtained from the frame time stamps TS1, TS2, TS3, and TS4 of the video frames 1 to 4.
The frame head packet jitter parameter of each video frame in the video frames 1-4 can be calculated according to the head packet interval and the frame interval between each video frame in the video frames 1-4 and the previous video frame, and the frame head packet jitter parameter of the previous video frame. The frame header jitter parameter of video frame 1 is equal to 0 by default. An example of the way to calculate the jitter parameters of the first frame packet of other video frames besides video frame 1 is shown in formula (1):
Jittern＝Jittern-1*15/16+(Deltan-Intervaln)*1/16 (1)
in the formula (1), n is greater than or equal to 2 and less than or equal to 4 and is an integer, jittern represents a frame head packet jitter parameter of a video frame n, 15/16 and 1/16 are preset control parameters, the value of the control parameter can be an optimal parameter value determined by setting different values to compare the calculated frame head packet jitter parameter (or frame end packet jitter parameter) with the real frame head packet jitter (or frame end packet jitter), and the control parameter can be used for representing the proportion of the value of the frame head packet jitter parameter (or frame end packet jitter parameter) of the previous video frame in the value of the frame head packet jitter parameter (or frame end packet jitter parameter) of the current video frame. Taking the first packet interval and the frame interval of the video frames 1-4 in fig. 6 as an example, the frame first packet jitter parameters of the video frames 1-4 can be obtained by combining the formula (1) as follows:
the frame start packet Jitter parameter Jitter1 of video frame 1=0;
the frame start packet Jitter parameter Jitter2 of video frame 2= Jitter1 × 15/16+ (Delta 2-Interval 2) × 1/16;
the frame start packet Jitter parameter Jitter of video frame 3, jitter3= Jitter2 × 15/16+ (Delta 3-Interval 3) × 1/16;
the frame start packet Jitter parameter Jitter4 of video frame 4= Jitter3 × 15/16+ (Delta 4-Interval 4) × 1/16.
According to the end packet interval and the frame interval between each video frame 1-4 and the previous video frame, and the frame end packet jitter parameter of the previous video frame, the frame end packet jitter parameter of each video frame 1-4 can be calculated. The end of frame jitter parameter for video frame 1 is equal to 0 by default. An example of the way to calculate the jitter parameters of the end of frame of other video frames besides video frame 1 is shown in formula (2):
Jittern’＝Jittern-1’*15/16+(Deltan’-Intervaln’)*1/16 (2)
in the formula (2), n is more than or equal to 2 and less than or equal to 4 and is an integer, jittren' represents the frame end packet jitter parameter of the video frame n, and 15/16 and 1/16 are preset control parameters. Taking the end packet interval and the frame interval of the video frames 1-4 in fig. 6 as an example, the frame end packet jitter parameters of the video frames 1-4 can be obtained by combining the formula (2) as follows:
the end-of-frame Jitter parameter Jitter1' =0 for video frame 1;
a frame end packet Jitter parameter Jitter2'= Jitter1' × 15/16+ (Delta 2'-Interval 2')/1/16 of video frame 1;
a frame end packet Jitter parameter Jitter3'= Jitter2' × 15/16+ (Delta 3'-Interval 3')/1/16 of video frame 1;
the end of frame Jitter parameter Jitter4'= Jitter3' × 15/16+ (Delta 4'-Interval 4')/1/16 of video frame 1.
In this way, the calculation of the jitter parameters of the head packet and the jitter parameters of the tail packet can be completed.
Fig. 7 and 8 respectively show examples of a frame loss parameter and a frame misordering parameter according to an embodiment of the present application.
In one possible implementation, the streaming media layer protocol information of the packet includes a streaming media layer sequence number indicating an order of the packet in the video frame to which the packet belongs. The network layer protocol information of the data packet comprises a packet time indicating the time of the data packet, the frame quality parameter comprises a frame packet loss parameter and a frame disorder parameter, the frame packet loss parameter represents the loss condition of the data packet of the video frame, the frame disorder parameter represents the disorder condition of the data packet of the video frame,
step S22, comprising:
determining a frame loss parameter of the video frame according to a stream media layer sequence number of a data packet corresponding to the video frame aiming at each frame of the at least one video frame;
and aiming at each frame of the at least one video frame, sequencing the data packets corresponding to the video frame according to the sequence of the packet time of the data packets corresponding to the video frame to obtain a sequencing result corresponding to the video frame, and determining the frame disorder parameter of the video frame according to the sequencing result and the stream media layer sequence number of the data packets corresponding to the video frame.
By the method, the frame packet loss parameter representing the loss condition of the data packet of the video frame and the frame disorder parameter representing the disorder condition of the data packet of the video frame can be calculated, and the flexibility of selecting the frame quality parameter is improved while the quality of the video frame is represented.
For example, the frame loss can be determined from the perspective of the streaming media layer of the packet, and can be achieved by counting and analyzing the sequence numbers of the streaming media layers, for example, as shown in fig. 7, the sequence numbers S of the streaming media layers of the packets a-i corresponding to the video frame 2 a -S i May be respectively 11, 12, 13, 15, 16, 17, 18, 19, 20, and the sequence number S of the streaming media layer is obtained according to statistics a -S i The frame loss situation can be analyzed and determined. Exemplary implementations of which may be found in reference to the following description.
The frame disorder condition can be determined from the perspective of the stream media layer and the network layer of the data packet, the data packet is sequenced according to the packet time to obtain a sequencing result, and the sequencing result and the sequence number of the stream media layer of the data packet are analyzed and determined in combination. For example, as shown in FIG. 8, the streaming media layer sequence number S of the packet A-K corresponding to video frame 3 A - S K 21, 22, 23, 25, 26, 27, 28, 24, 29, 30, 31, the result of the sorting of the packets a-K corresponding to the video frame 3 may be, for example, P = { A, B, C, D, E, F, G, H, I, J, K }, and the result P of the sorting and the statistical streaming media layer sequence number S are obtained A -S K The frame loss situation can be analyzed and determined. Exemplary implementations of which may be found in reference to the following description.
In a possible implementation manner, determining, for each of the at least one video frame, a frame loss parameter of the video frame according to a sequence number of a streaming media layer of a data packet corresponding to the video frame includes:
and determining the maximum value and the minimum value of the serial number of the stream media layer of the data packet corresponding to the video frame and the number of the serial number of the stream media layer according to the serial number of the stream media layer of the data packet corresponding to the video frame, determining the theoretical value of the number of the data packets corresponding to the video frame according to the maximum value and the minimum value of the serial number of the stream media layer, and determining the frame packet loss parameter of the video frame according to the difference between the theoretical value of the number of the data packets corresponding to the video frame and the number of the serial number of the stream media layer of the data packet corresponding to the video frame.
Determining a frame disorder parameter of the video frame according to the sequencing result and the sequence number of the streaming media layer of the data packet corresponding to the video frame, including:
and when the value of the stream media layer sequence number of the current data packet is greater than the stream media layer sequence number of the data packet before the current data packet in the sequencing result, determining the frame disorder parameter corresponding to the video frame according to the sequencing sequence number of the current data packet in the sequencing result, the stream media layer sequence number of the current data packet and the minimum value of the stream media layer sequence number of the data packet corresponding to the video frame.
By the method, the frame packet loss parameter can reflect the information of whether the data packet in the video frame loses packets, and can also reflect the information of the number of lost packets when the data packet loses packets; the frame disorder parameter can reflect the information whether the data packets in the video frame are in disorder or not, and can also reflect the disorder depth information of the disordered data packets when the disorder exists, so that the accuracy of determining the quality condition of the video frame according to the frame quality parameter can be improved.
For example, as shown in FIG. 7, the streaming media layer sequence number S of the packet a-i corresponding to video frame 2 a - S i 11, 12, 13, 15, 16, 17, 18, 19, and 20, and the maximum value max =20, the minimum value min =11, and the number of the streaming media layer sequence numbers is 9, of the streaming media layer sequence numbers of the packets a-i corresponding to the video frame 2, according to the streaming media layer sequence numbers of the packets a-i corresponding to the video frame 2. According to the maximum value 20 and the minimum value 11 of the sequence number of the streaming media layer of the video frame 2, the theoretical value q = max-min +1=20-11+1=10 of the number of the data packets corresponding to the video frame 2 and the sequence number of the streaming media layer of the data packets corresponding to the video frame can be determinedThe difference of the number 9 is equal to 1, so that there is a packet loss situation in the video frame 2, and the number of packet losses is 1.
As shown in FIG. 8, the streaming media layer sequence number S of the packet A-K corresponding to video frame 3 A -S K 21, 22, 23, 25, 26, 27, 28, 24, 29, 30, 31, respectively, the sequencing result may be, for example, P = { A, B, C, D, E, F, G, H, I, J, K }, according to the streaming media layer sequence number S of the packet a-K corresponding to video frame 3 A -S K It can be determined that the value (24) of the streaming media layer sequence number of packet H is less than the value (28) of the streaming media layer sequence number of packet G preceding packet H in the ordering result P, it can be determined that an out-of-order condition has occurred for video frame 3, and the packet that caused the out-of-order condition is packet H. Streaming media layer sequence number S that can be based on packet H H The value 24 and the minimum value MIN =21 of the sequence number of the streaming media layer of the data packet A-K corresponding to the video frame 3 are calculated to obtain the theoretical value S of the sequence number of the data packet H in the sequencing result P H And the MIN +1=4 calculates the disorder depth of 8-4=4 according to the sequence number 8 of the data packet H determined by the sequence result P and the theoretical value 4 of the sequence number of the data packet H in the sequence result P. When there is only one out-of-order data packet, the out-of-order depth of the out-of-order data packet can be used as the value of the frame out-of-order parameter corresponding to the video frame. When a plurality of data packets causing disorder are available, the sum of the disorder depth of each disordered data packet can be averaged according to the number of the disordered data packets to obtain an average disorder depth, and the average disorder depth is used as the numerical value of the frame disorder parameter corresponding to the video frame.
By the method, the frame packet loss parameter and the frame disorder parameter can be obtained through calculation.
In step S23, for the different video frames identified in step S21, packet quality parameters of the data packet are obtained according to the network layer protocol information and the transport layer protocol information of the data packet corresponding to the different video frames, so as to reflect the quality of the data packet, where the packet quality parameters may include one or more of a packet interval parameter, a packet jitter parameter, a disorder parameter, a packet loss parameter, a retransmission parameter, a delay parameter, an IP packet fragmentation parameter, and a checksum error parameter, for example. This procedure can be implemented based on the prior art, and is exemplarily described below for different packet quality parameters and their calculation, respectively.
In a possible implementation manner, a packet quality parameter of each data packet corresponding to each frame in the at least one video frame is determined according to network layer protocol information and transport layer protocol information of the data packet, where the packet quality parameter includes: one or more of a packet interval parameter indicating a time interval between a current data packet and a previous data packet, a packet jitter parameter indicating a time delay jitter between the current data packet and the previous data packet, a disorder parameter indicating a disorder condition of the data packet, a packet loss parameter indicating a packet loss condition of the data packet, a retransmission parameter indicating a retransmission condition of the data packet, a time delay parameter indicating a time delay condition of the data packet, an IP packet fragmentation parameter indicating whether the data packet has an IP packet fragmentation, and a checksum error parameter indicating whether a checksum of the data packet has an error.
By the method, various packet quality parameters used for representing the quality of the data packet can be obtained, the quality of the data packet determined according to the packet quality parameters is closer to the actual quality of the data packet by obtaining the various packet quality parameters, and the follow-up determination of the reason causing the video quality problem and the accuracy of the equipment according to the abnormal packet quality parameters and the abnormal frame quality parameters can be improved.
The calculation method of each packet quality parameter is described below.
The packet interval parameter may be determined according to the packet time in the network layer protocol information of the data packet, and the calculation method may refer to the above example of the calculation method of the frame interval. For example, assuming that video frame 4 includes 5 packets, for each of packets 1-5, the packet interval parameter may be calculated from the packet time indicating the time of the packet in the network layer protocol information for that packet. When m is greater than or equal to 2 and less than or equal to 5 and is an integer, the packet interval parameter of the data packet m and the data packet m-1 can be calculated by the difference between the packet time of the data packet m and the packet time of the data packet m-1, and when m =1, the packet interval parameter of the data packet 1 is equal to 0 by default.
The packet jitter parameter may be determined according to a packet time in the network layer protocol information of the data packet and a preset control parameter, and the calculation manner may refer to the above example of the calculation manner of the frame head packet jitter parameter or the frame tail packet jitter parameter. For example, assume that the at least one video frame comprises video frames 1-4. For each of the video frames 1-4, a packet interval parameter may be calculated according to a packet time indicating a time of a data packet in network layer protocol information of the data packet corresponding to the video frame. The packet jitter parameter of each data packet can be calculated according to the packet interval parameter of each data packet corresponding to each video frame and the previous data packet, the packet jitter parameter of the previous data packet and the preset control parameter. For example, for video frame 4, video frame 4 may, for example, comprise 5 packets, where 2 ≦ m ≦ 5 and is an integer, the packet jitter parameters for packet m and packet m-1 may be determined by the packet interval parameters for packet m and packet m-1, the packet jitter parameters for packet m-1, and the control parameters (e.g., 15/16 and 1/16).
The out-of-order parameter may be determined according to a network layer sequence number in the network layer protocol information and a transport layer sequence number in the transport layer protocol information of the current data packet and the previous data packet, where a value of the out-of-order parameter may be equal to 1 when the current data packet is an out-of-order data packet, and a value of the out-of-order parameter may be equal to 0 when the current data packet is not an out-of-order data packet. For example, the data packets are sorted according to the packet time, and if the transmission layer sequence number and the network layer sequence number of the current data packet are both reduced compared with the transmission layer sequence number and the network layer sequence number of the previous data packet, it indicates that the current data packet is an out-of-order data packet.
The packet loss parameter may be determined according to a network layer sequence number in the network layer protocol information of the data packet and a transmission layer sequence number, a checksum, and a confirmation packet sequence number in the transmission layer protocol information, where a value of the packet loss parameter may be equal to 1 when the current data packet is a packet to be lost, and may be equal to 0 when the current data packet is not a packet to be lost. For example, calculating a packet loss parameter of a data packet corresponding to a video stream using a TCP protocol, sorting the data packets according to packet time, if the transmission layer sequence number of a current data packet is decreased and the network layer sequence number is increased compared to the transmission layer sequence number and the network layer sequence number of a previous data packet, and before the packet time of the current data packet, a data packet identical to the network layer sequence number, the transmission layer sequence number, the checksum of the current data packet does not appear in a video buffer, but a data packet identical to the acknowledgement packet sequence number of the current data packet appears, indicating that the current data packet is a packet loss data packet. For another example, the packet loss parameter of the data packet corresponding to the video stream using the UDP protocol is calculated, and the network layer sequence number of the data packet may be directly used to determine whether the data packet is lost. The data packets may be sorted according to packet time, and if the network layer sequence number of the current data packet is reduced compared with the network layer sequence number of the previous data packet, it indicates that the current data packet is a packet-lost data packet.
The retransmission parameter may be determined according to a network layer sequence number in the network layer protocol information of the data packet and a transmission layer sequence number, a checksum, and a confirmation packet sequence number in the transmission layer protocol information, where a value of the retransmission parameter may be equal to 1 when the current data packet is a retransmitted data packet, and a value of the retransmission parameter may be equal to 0 when the current data packet is not a retransmitted data packet. For example, calculating retransmission parameters of data packets corresponding to a video stream using a TCP protocol, the data packets may be sorted according to packet time, and if the transport layer sequence number of the current data packet is decreased and the network layer sequence number is increased compared to the transport layer sequence number and the network layer sequence number of the previous data packet, and before the packet time of the current data packet, a data packet identical to the network layer sequence number, the transport layer sequence number, and the checksum of the current data packet appears in the video buffer, it is indicated that the current data packet is a retransmitted data packet. For another example, the retransmission parameter of the data packet corresponding to the video stream using the UDP protocol is calculated, the data packets may be sorted according to the packet time, and if there is a data packet whose network layer sequence number and checksum are the same as those of the current data packet and whose packet time is earlier than that of the current data packet, it indicates that the current data packet is a retransmitted data packet.
The delay parameter may be determined according to a packet time in network layer protocol information of the data packet and a confirmation packet sequence number in transport layer protocol information, for example, the delay parameter of the data packet corresponding to a video stream using a TCP protocol is calculated, a data packet having the same confirmation packet sequence number as the current data packet may be found according to the confirmation packet sequence number of the current data packet, and a value of the delay parameter is determined according to a difference between the packet time of the data packet and a packet time of the found data packet and the current data packet.
The IP packet fragmentation parameter may be determined according to a fragmentation flag in network layer protocol information of the data packet, for example, when the fragmentation flag exists in the network layer protocol information, it represents that the data packet has an IP packet fragmentation, a value of the IP packet fragmentation parameter may be equal to 1, when the fragmentation flag does not exist in the network layer protocol information, it represents that the data packet does not have an IP packet fragmentation, and a value of the IP packet fragmentation parameter may be equal to 0.
The checksum error parameter may be determined according to a checksum in the transport layer protocol information of the data packet and a checksum value calculated according to the prior art, and if the two values are not consistent, it indicates that a checksum error exists in the current data packet. In the presence of a checksum error, the value of the checksum error parameter is equal to 1. In the absence of a checksum error, the value of the checksum error parameter is equal to 0. When the value of the checksum in the transport layer protocol information is an invalid value (the value is 0), the checksum in the transport layer protocol information and the calculated checksum value are not compared.
In this way, a variety of packet quality parameters can be calculated. It will be appreciated by those skilled in the art that the variety of packet quality parameters and the manner of calculation should be more than the examples described above. The embodiments of the present application do not limit the type and calculation method of the packet quality parameter as long as the quality of the data packet can be represented by the packet quality parameter.
In step S24, the frame quality parameter calculated in step S22 and the packet quality parameter calculated in step S23 are determined to be abnormal by combining the frame quality parameter threshold and the packet quality parameter threshold, respectively.
In one possible implementation, step S24 includes:
for each frame of the at least one video frame, determining a frame quality parameter of which the value is greater than the frame quality parameter threshold value in the frame quality parameters of the video frame as the abnormal frame quality parameter;
and aiming at each frame of the at least one video frame, determining the packet quality parameter with the value larger than the threshold value of the packet quality parameter in the packet quality parameters of the data packet corresponding to the video frame as the abnormal packet quality parameter.
In this way, the frame quality parameter and the packet quality parameter of the anomaly may be determined. The abnormal parameters are determined by comparing the parameter values with the threshold values, so that the determination of the abnormal parameters can be accurately and quickly completed.
For example, the frame quality parameter threshold and the packet quality parameter threshold may be determined in real time by using preset fixed values or by using an AI model. An exemplary method for determining frame quality parameter thresholds and packet quality parameter thresholds using an AI model is described below.
For example, for each frame quality parameter, there may be a frame quality parameter threshold; for each type of packet quality parameter, there may be a packet quality parameter threshold. When the AI model determines a frame quality parameter threshold (or a packet quality parameter threshold) corresponding to a frame quality parameter (or a packet quality parameter), the data used may be a plurality of values of the frame quality parameter (or a plurality of values of the packet quality parameter corresponding to a plurality of packets) corresponding to a plurality of video frames, which are calculated within a period of time before the current time is determined. For example, a frame head packet jitter parameter calculated in a period of time before the current time (hereinafter, referred to as a historical frame head packet jitter parameter) is used as a parameter sample of the AI model, and the AI model may generate a normal distribution probability curve P (x) of the historical frame head packet jitter parameter x by fitting based on a statistical condition of values of the historical frame head packet jitter parameter, and calculate a frame quality parameter threshold a for the frame head packet jitter parameter using a cumulative distribution function F (a) (F (a) = P (x < = a)) of the normal distribution probability curve. FIG. 9 illustrates an example of a normal distribution probability curve generated according to an embodiment of the present application. As shown in fig. 9, the histogram is a distribution of values of the jitter parameters of the first packet of the historical frame, and the curve is a normal distribution probability curve P (x) fitted according to the distribution of the jitter parameters of the first packet of the historical frame.
An abnormality threshold is preset, and when the cumulative distribution function F (a) is equal to the abnormality threshold, the value of a may be the corresponding frame quality parameter threshold. If the jitter parameter of the frame head packet calculated after the current time is greater than the threshold, the frame head packet jitter parameter which is abnormal can be determined, that is, the frame quality parameter which is abnormal is determined. Taking the normal distribution probability curve of fig. 9 as an example, the abnormality degree threshold is preset to 75%, for example, and it is calculated from the cumulative distribution function of the normal distribution probability curve, and when the value of the cumulative distribution function F (a) is equal to 75%, the corresponding value of a may be equal to 100, for example, in this case, the normal range of the value of the frame start packet jitter parameter is 0 to 100. Fig. 10 illustrates an example of implementing abnormal situation determination of parameters according to an embodiment of the present application. As shown in fig. 10, the value of a frame header packet jitter parameter that needs to determine an abnormal condition is 240, which is greater than 100, and the frame header packet jitter parameter is an abnormal frame quality parameter, so that the abnormal condition determination of the parameter can be realized.
When the frame quality parameter (or the packet quality parameter) for determining the abnormal condition is obtained through calculation, the frame quality parameter threshold (or the packet quality parameter threshold) obtained by the AI model can be used to determine whether the frame quality parameter (or the packet quality parameter) for determining the abnormal condition is the abnormal frame quality parameter (or the abnormal packet quality parameter). Further, the parameter samples of the frame quality parameter (or abnormal packet quality parameter) that have been determined to be abnormal in the parameter samples may also be clipped, so that the frame quality parameter threshold (or packet quality parameter threshold) is also updated as the parameter samples change.
For example, the above-described method is adopted for determining the abnormal condition of the frame quality parameter, and the determined abnormal condition of the frame quality parameter can be shown in the following table 1.
TABLE 1
Figure BDA0003233203920000241
As shown in table 1, the value of 1 for the jitter abnormal condition of the first packet of the frame, the jitter abnormal condition of the last packet of the frame, the packet loss abnormal condition of the frame, and the frame disorder abnormal condition represents that the corresponding frame quality parameter of the frame is an abnormal frame quality parameter, and the frame is also an abnormal frame. As can be seen from table 1, the video frame 1 has no frame quality parameter abnormality, the frame head packet jitter parameter abnormality and the frame disorder parameter abnormality of the video frame 2, the frame head packet jitter parameter abnormality and the frame tail packet jitter parameter abnormality and the frame packet loss parameter abnormality of the video frame 3, and the video frame 4 has no frame quality parameter abnormality.
The above-described method is used to determine the abnormal condition of the packet quality parameter, and the determined abnormal condition of the packet quality parameter may be shown in table 2 below, for example.
TABLE 2
Figure BDA0003233203920000251
In table 2, the gNB, UDG, eoR respectively indicate a 5G base station, a unified distributed gateway, and a switch using a certain deployment method. The value of the packet loss abnormal condition, the disorder abnormal condition, the packet jitter abnormal condition, the packet interval abnormal condition, the retransmission abnormal condition, the delay abnormal condition, the IP packet fragmentation abnormal condition, and the checksum error abnormal condition is 1, which represents that the packet quality parameter is an abnormal packet quality parameter, and the packet is also an abnormal packet. Then, as can be seen from table 2, on the 5G base station gNB, the data packets 1, 3, 4, 5 have no packet quality parameter abnormality, and the packet jitter parameter of the data packet 2 is abnormal; on the unified distributed gateway UDG, packet jitter parameters of a data packet 1 are abnormal, packet loss parameters and time delay parameters of a data packet 2 are abnormal, and packet loss parameters and packet jitter parameters of a data packet 3 are abnormal; on the switch EoR using a certain deployment mode, the packet jitter parameter of the data packet 1 is abnormal, the packet loss parameter of the data packet 2 is abnormal, and the packet loss parameter and the packet jitter parameter of the data packet 3 are abnormal.
After the abnormal frame quality parameters and the abnormal packet quality parameters are obtained in step S2, in step S3, the reason and the device causing the video quality abnormality at the video playing end can be determined according to the relationship between the abnormal frame quality parameters and the abnormal packet quality parameters and the preset conditions. Fig. 11 is an exemplary diagram illustrating correspondence between an abnormal frame quality parameter, an abnormal packet quality parameter, a cause of video quality abnormality at a video player end, and a device according to an embodiment of the present application.
As shown in fig. 11, the packet quality parameter corresponding to the frame quality parameter can be found by using the correspondence relationship between the video frame and the data packet, so that the data packet to which the packet quality parameter belongs is the data packet under the video frame to which the frame quality parameter belongs. The frame quality parameters and their corresponding packet quality parameters generally have a certain correlation. For example, when a packet loss parameter of a certain data packet is abnormal, the situation of the frame packet loss parameter of the video frame to which the data packet belongs can be checked. When the disorder parameter of a certain data packet is abnormal, the condition of the frame disorder parameter of the video frame to which the data packet belongs can be checked. When the packet interval parameter or the packet jitter parameter of a certain data packet is abnormal, the data packet may not arrive according to the expected decoding time, so that the video frame to which the data packet belongs may be greatly jittered, and the conditions of the frame head packet jitter parameter and the frame tail packet jitter parameter of the video frame to which the data packet belongs may be checked. When the retransmission parameter or the delay parameter of a certain data packet is abnormal, the transmission of the data packet may be slowed down, so that the video frame to which the data packet belongs may be greatly jittered, and the conditions of the frame head packet jitter parameter and the frame tail packet jitter parameter of the video frame to which the data packet belongs may be checked. The IP packet fragmentation parameter and checksum error parameter of a packet are also associated with the jitter of the video frame to which the packet belongs, and thus. When the IP message fragment parameter or the check sum error parameter of a certain data packet is abnormal, the conditions of the frame head packet jitter parameter and the frame tail packet jitter parameter of the video frame to which the data packet belongs can be checked.
As can be seen from the above description, there may be multiple abnormal frame quality parameters and abnormal packet quality parameters, and conditions having a corresponding relationship with the reason and the device causing the video quality abnormality at the video playing end may be set first, and then it is determined whether the abnormal frame quality parameters and the abnormal packet quality parameters satisfy the corresponding conditions, so as to determine the reason and the device causing the video quality abnormality at the video playing end.
The following describes exemplary conditions that may be set, and causes and devices that cause video quality abnormality at the video player end determined according to the conditions, with reference to fig. 11.
In one possible implementation, step S3 includes:
when the frame quality parameters of the video frames and the packet quality parameters of the data packets corresponding to the video frames meet a first condition, determining that the reason causing the video quality abnormity of a video playing end is a video coding problem, and determining that equipment causing the video quality abnormity of the video playing end is equipment used for video coding;
wherein the first condition comprises:
the frame packet loss parameter of the video frame is abnormal and the packet loss parameter of the data packet corresponding to the video frame is normal, or
The frame disorder parameter of the video frame is abnormal, and the disorder parameter of the data packet corresponding to the video frame is normal.
Because the protocol information of the data packet is obtained at the network equipment and the video playing end, the frame quality parameter and the packet quality parameter can reflect the abnormal condition in the transmission process, the frame packet loss parameter abnormality indicates that a data packet in the video frame is in a loss state during transmission, and if the packet loss parameter of the data packet in the video frame is normal at the moment, the data packet is not lost in the transmission process, so that the data packet can be determined to be lost before transmission, namely, during video coding; the frame disorder parameter abnormity indicates that a data packet in a video frame is in a disorder state during transmission, and if the disorder parameter of the data packet in the video frame is normal at the moment, the data packet is not in disorder during transmission, so that the data packet can be determined to be in disorder before transmission, namely during video coding; based on the above reasons, the first condition, that is, the frame packet loss parameter of the video frame is abnormal and the packet loss parameter of the data packet corresponding to the video frame is normal, or the frame disorder parameter of the video frame is abnormal and the disorder parameter of the data packet corresponding to the video frame is normal, can accurately reflect the video coding problem.
For example, as shown in fig. 11, if a frame packet loss parameter in the frame quality parameters is abnormal and a packet loss parameter in the packet quality parameters is normal, or a frame disorder parameter in the frame quality parameters is abnormal and a disorder parameter in the packet quality parameters is normal, it may be determined that the abnormal frame quality parameter and the abnormal packet quality parameter satisfy a first condition, and it is further determined that a cause causing the video quality problem occurs in the video encoding process, the video encoding process is dominated by a device used for video encoding, and there is only one device used for video encoding, so that it may be uniquely determined that the device causing the video encoding problem is a device used for video encoding.
In practical application, as long as it can be determined that the frame quality parameter and the packet quality parameter satisfy any one of the first conditions, the reason and the device causing the video quality abnormality at the video playing end can be directly determined, and it is not necessary to determine whether the frame quality parameter and the packet quality parameter satisfy the other remaining possibility in the first conditions. Or, it may be set to determine whether each of the frame quality parameter and the packet quality parameter meets the first condition, and after determining that at least one of the frame quality parameter and the packet quality parameter meets the first condition, determine the reason and the device causing the video quality abnormality at the video playing end. It should be understood by those skilled in the art that both the above two setting manners may be used to determine the cause and the device causing the video quality abnormality at the video playing end, and the application is not limited thereto.
In one possible implementation, step S3 includes:
when the frame quality parameters of the video frames and the packet quality parameters of the data packets corresponding to the video frames meet a second condition, determining that the reason causing the video quality abnormity of the video playing end is a terminal parameter problem, and determining that the equipment causing the video quality abnormity of the video playing end is equipment used for video playing;
wherein the second condition comprises:
any one of the jitter parameter of the first frame packet and the jitter parameter of the last frame packet of the video frame is abnormal, and any one of the IP message fragment parameter, the checksum error parameter of the data packet corresponding to the video frame is abnormal.
Because the protocol information of the data packet is obtained at the network device and the video playing end, the frame quality parameter and the packet quality parameter can reflect the abnormal situation in the transmission process, the frame head packet jitter parameter/frame tail packet jitter parameter abnormity indicates that the time delay jitter of the first/last data packet in the video frame is overlarge during transmission, if the IP message fragment parameter of the data packet in the video frame is abnormal at the moment, the data packet is subjected to fragmentation in the transmission process, and therefore the parameter configuration of the video playing end can be determined not to meet the requirement of the data packet transmission when the data packet is not fragmented; if the verification and error parameters of the data packet under the video frame are abnormal at this time, the verification of the data packet with the video playing end in the transmission process is not successfully completed, so that the requirement of successful verification of the data packet with the parameter configuration of the video playing end can be determined to be not met; based on the above reasons, the second condition that any one of the jitter parameter of the first packet and the jitter parameter of the last packet of the video frame is abnormal and any one of the fragment parameter, the checksum error parameter of the IP packet of the data packet corresponding to the video frame is abnormal can accurately reflect the problem of the terminal parameter.
For example, as shown in fig. 11, if the jitter parameter of the first packet in the frame quality parameter/the jitter parameter of the last packet in the frame quality parameter is abnormal and the fragment parameter/checksum error parameter of the IP packet in the packet quality parameter is normal, it may be determined that the abnormal frame quality parameter and the abnormal packet quality parameter satisfy the second condition, and it is further determined that the cause of the video quality problem occurs on the play parameter in the video playing process, the video playing process is dominated by the device used for video playing, and there is only one device (i.e., video playing end) used for video playing, so that it may be uniquely determined that the device causing the terminal parameter problem is the device used for video playing.
In practical applications, as long as it can be determined that the frame quality parameter and the packet quality parameter satisfy any one of the second conditions, the reason and the device causing the video quality abnormality at the video playing end can be directly determined, and it is not necessary to determine whether the frame quality parameter and the packet quality parameter satisfy the remaining possible conditions in the second conditions. Or, it may be configured to determine whether each of the frame quality parameter and the packet quality parameter meets the second condition, and after determining that at least one of the frame quality parameter and the packet quality parameter meets the second condition, determine the reason and the device causing the video quality abnormality at the video playing end. It should be understood by those skilled in the art that both of the above-mentioned two setting manners can be used for determining the cause and the device causing the video quality abnormality at the video playing end, and the application is not limited thereto.
In one possible implementation, step S3 includes:
when the frame quality parameters of the video frames and the packet quality parameters of the data packets corresponding to the video frames meet a third condition, determining that the reason causing the video quality abnormity of a video playing end is a network transmission problem;
wherein the third condition comprises:
the frame packet loss parameter of the video frame is abnormal and the packet loss parameter of the data packet corresponding to the video frame is abnormal, or
The frame disorder parameter of the video frame is abnormal and the disorder parameter of the data packet corresponding to the video frame is abnormal, or
Any one of the jitter parameter of the first frame packet and the jitter parameter of the last frame packet of the video frame is abnormal, and any one of the jitter parameter, the packet interval parameter, the retransmission parameter and the delay parameter of the data packet corresponding to the video frame is abnormal.
Because the protocol information of the data packets is acquired at the network equipment and the video playing end, the frame quality parameter and the packet quality parameter can reflect the abnormal condition in the transmission process, the frame packet loss parameter abnormity indicates that a data packet is lost in the video frame during transmission, and if the packet loss parameter abnormity of the data packet in the video frame at the moment indicates that the data packet is lost in the transmission process, the problem of network transmission can be determined; the frame disorder parameter abnormity indicates that a data packet in the video frame is in a disorder state during transmission, and if the disorder parameter of the data packet in the video frame is abnormal at the moment, the data packet is in disorder during transmission, so that the problem of network transmission can be determined; based on the above reasons, in the third condition, "the frame packet loss parameter of the video frame is abnormal and the packet loss parameter of the data packet corresponding to the video frame is abnormal, or the frame disorder parameter of the video frame is abnormal and the disorder parameter of the data packet corresponding to the video frame is abnormal", the network transmission problem can be accurately reflected; the abnormal frame head packet jitter parameter/frame tail packet jitter parameter indicates that the time delay jitter of the first/last data packet under the video frame is too large during transmission, and if the packet interval parameter of the data packet under the video frame is abnormal at the moment, the time difference of the two data packets during the transmission process is too large, the problem of network transmission can be determined; if the packet jitter parameter of the data packet under the video frame is abnormal at this time, it indicates that the delay jitter of the data packet in the transmission process is too large, and it can be determined that the network transmission has problems; if the retransmission parameters of the data packet under the video frame are abnormal at this time, the data packet is transmitted for multiple times in the transmission process, and the problem of network transmission can be determined; if the time delay parameter of the data packet under the video frame is abnormal at this time, the time consumption of the data packet in the transmission process is too long, and the problem of network transmission can be determined. Based on the above reasons, in the third condition, "any one of the jitter parameter of the first packet of the video frame and the jitter parameter of the last packet of the video frame is abnormal, and any one of the jitter parameter, the packet interval parameter, the retransmission parameter, and the delay parameter of the data packet corresponding to the video frame is abnormal" can accurately reflect the problem of the terminal parameter.
For example, as shown in fig. 11, if a frame packet loss parameter in the frame quality parameters is abnormal and a packet loss parameter in the packet quality parameters is abnormal, or a frame disorder parameter in the frame quality parameters is abnormal and a disorder parameter in the packet quality parameters is abnormal, or a frame head packet jitter parameter/a frame tail packet jitter parameter in the frame quality parameters is abnormal and a packet jitter parameter/a packet interval parameter/a retransmission parameter/a time delay parameter in the packet quality parameters is abnormal, it may be determined that the abnormal frame quality parameter and the abnormal packet quality parameter satisfy a third condition, and it is determined that a cause causing the video quality abnormality occurs in a network transmission process, and it is a network device causing the video quality abnormality.
Table 3 shows an example of determining that the cause of the video quality problem is a network transmission problem according to an embodiment of the present application.
TABLE 3
Figure BDA0003233203920000281
Referring to table 3, the frame timestamp identifies only one frame. The packet information to be checked corresponding to one frame may include packet time, network layer sequence number, transport layer sequence number, checksum, and packet quality parameter abnormal condition. For the abnormal packet quality parameters, the result of the video quality problem obtained by analysis can be given in the column of analysis result according to the incidence relation between the frame quality parameters and the packet quality parameters. For the example of table 3, since the frame packet loss parameter of the video frame 1 is abnormal, the packet loss parameter of the data packet 2 corresponding to the video frame 1 is abnormal, and the packet loss parameter of the data packet 3 corresponding to the video frame 1 is abnormal, the third condition is satisfied, and it can be determined that the cause of the video quality problem is the network transmission problem.
In practical applications, as long as it can be determined that the frame quality parameter and the packet quality parameter satisfy any one of the third conditions, the reason causing the video quality abnormality at the video playing end can be directly determined without determining whether the frame quality parameter and the packet quality parameter satisfy the remaining possible conditions in the third conditions. Alternatively, it may be configured to determine whether each of the frame quality parameter and the packet quality parameter meets the third condition, and after determining that at least one of the frame quality parameter and the packet quality parameter meets the third condition, determine the reason causing the video quality abnormality at the video playing end. It should be understood by those skilled in the art that both of the above-mentioned two setting manners can be used for determining the cause of the video quality abnormality at the video playing end, and the application is not limited thereto.
Since a plurality of network devices may be used in the network transmission process, the specific device causing the problem of the terminal parameter can be further determined.
In a possible implementation manner, after determining that the cause of the video quality abnormality at the video playing end is a network transmission problem, step S3 further includes:
determining the network layer sequence number of the data packet corresponding to the abnormal packet quality parameter according to the network layer protocol information of the data packet;
determining the packet time, the transmission layer sequence number and the checksum of the data packet corresponding to the abnormal packet quality parameter according to the transmission layer protocol information of the data packet;
receiving network topology data on network equipment, and determining a transmission path of each data packet in a network according to the network topology data;
finding out data packets of which the packet time, the network layer serial number, the transmission layer serial number and the check sum all meet the conditions in each network device on the transmission path according to the transmission path of the data packet corresponding to the abnormal packet quality parameter in the network;
and according to the found packet quality parameters of the data packets meeting the conditions, determining the network equipment causing the video quality abnormity of the video playing end from the network equipment on the transmission path of the data packets.
The packet quality parameter of the data packet on each device can indicate the influence of the device on the transmission of the data packet, and the found data packet can be used for counting the influence of different network devices on the transmission of the data packet in the transmission process of the data packet and determining the specific device causing the video quality abnormity of the video playing end by finding the corresponding data packet in each network device on the transmission path.
For example, after determining that the frame quality parameter and the packet quality parameter meet the third condition, the network layer sequence number of the data packet corresponding to the packet quality parameter may be determined according to the network layer protocol information, and the packet time, the transmission layer sequence number, and the checksum of the data packet corresponding to the packet quality parameter may be determined according to the transmission layer protocol information. According to the network topology data acquired by the topology data acquisition device, the transmission path of the data packet corresponding to the abnormal packet quality parameter on the network equipment can be acquired. For example, referring to table 3, the abnormal packet quality parameter may be, for example, a packet loss parameter of the data packet 2 and a packet loss parameter of the data packet 3, and according to the network topology data, it may be determined that a transmission path of the data packet 2 and the data packet 3 may be, for example, gNB → UDG → EoR.
According to the transmission path of the data packet corresponding to the abnormal packet quality parameter in the network, the data packet of which the packet time, the network layer serial number, the transmission layer serial number and the check sum all meet the conditions in each network device on the transmission path can be found, and the network device causing the video quality abnormality of the video playing end is determined by comparing the abnormal packet quality parameters of the data packet meeting the conditions. For example, referring to table 3, a data packet satisfying the condition can be found from the packet time (8.
In one possible implementation manner, the conditions that the packet time, the network layer sequence number, the transport layer sequence number, and the checksum satisfy include:
and the absolute value of the difference between the packet time and the packet time of the data packet corresponding to the abnormal packet quality parameter is smaller than a preset threshold, and the network layer sequence number, the transmission layer sequence number, the checksum and the data packet corresponding to the abnormal packet quality parameter are the same.
In this way, the same packet on different network devices on the transmission path can be found. The method can further determine the packet quality parameter abnormal condition on the upstream equipment, and can accurately determine the specific network equipment causing the video quality abnormality of the video playing end when the reason causing the video quality abnormality of the video playing end is a network transmission problem.
For example, in connection with table 2 and table 3, from the network layer sequence number (48160), the transport layer sequence number (3480724955), and the checksum (1231221433) of packet 2 in table 3, a packet satisfying the network layer sequence number equal to 48160, the transport layer sequence number equal to 3480724955, and the checksum equal to 1231221433 may be found in table 2, it may be further determined whether an absolute value of a difference between a packet time of the found packet and a packet time (12.008) of packet 2 in table 3 (8.
From the network layer sequence number (48162), the transport layer sequence number (3480726183), and the checksum (2134423523) of packet 3 in table 3, a packet satisfying the network layer sequence number equal to 48162, the transport layer sequence number equal to 3480726183, and the checksum equal to 2134423523 may be found in table 2, it may be found whether an absolute value of a difference between the packet time of the found packet and the packet time (8.
According to the found packet quality parameters of the data packets meeting the conditions, the specific network equipment causing the video quality abnormity of the video playing end can be determined. Fig. 12 shows an example of determining a specific network device causing video quality abnormality at a video player according to an embodiment of the present application.
For example, as shown in fig. 12, A, B, C are gNB, UDG and EoR, respectively, and taking the data packet on the gNB, UDG and EoR (hereinafter referred to as the data packet on the A, B, C device) which satisfies the condition found for the data packet 2 of table 3 as an example, the transmission path of the data packet may be gNB → UDG → EoR according to the network topology data. The abnormal packet quality parameter of the data packet 2 in table 3 is a packet loss parameter, and when the packet loss parameters of the data packet on the A, B, C device are all abnormal, it can be determined that the packet loss is introduced by the device a (gNB), which further causes the video quality abnormality; B. when the packet loss parameter of the data packet on the device a is normal, it can be determined that the packet loss is introduced by the device B (UDG), which further causes the video quality to be abnormal; when the packet loss parameter of the data packet on the C device is abnormal and the packet loss parameter of the data packet on the A, B device is normal, it can be determined that the packet loss is introduced by the C device (EoR), which further causes the video quality abnormality.
Theoretically, according to the transmission path of gNB → UDG → EoR, if the packet loss parameter of the device a is abnormal, the packet loss parameter of the B, C device is definitely abnormal; if the packet loss parameter of the B device is abnormal, the packet loss parameter of the C device is necessarily abnormal. However, when protocol information is collected by the probe, a collection error may occur, so that an error may occur in the protocol information, and further, an error may occur in a packet quality parameter determined according to the protocol information. For example, when the packet loss parameter of the data packet on the A, B device is abnormal, and when the packet loss parameter of the data packet on the C device is normal, it can be considered that an error occurs when protocol information is collected from the C device, and it can be determined that the packet loss is introduced by the a device, which further causes video quality abnormality; A. when the packet loss parameter of the data packet on the device C is abnormal and the packet loss parameter of the data packet on the device B is normal, it can be considered that an error occurs when protocol information is collected from the device B, and it can be determined that the packet loss is introduced by the device a, which further causes the video quality abnormality.
A. There may be other examples of the packet loss parameter of the data packet on the B, C device, for example, the packet loss parameters of the data packet on the A, B, C device are all normal, the packet loss parameter of the data packet on the a device is abnormal, the packet loss parameter of the data packet on the B, C device is normal, the packet loss parameter of the data packet on the B device is abnormal, and the packet loss parameter of the data packet on the A, C device is normal, and since the occurrence probability of these examples is extremely low, detailed description is not given here.
By the method, different conditions of protocol information acquisition errors on the equipment and data packet loss in the transmission process can be distinguished, and the accuracy of determining the equipment causing video quality abnormity at the video playing end is improved.
After comparative analysis using the above method, comparative results as shown in table 4 were obtained.
TABLE 4
Figure BDA0003233203920000311
See table 4, where each data packet (packet 2, packet 3) with an abnormal packet quality parameter is uniquely identified by a network layer sequence number, a transport layer sequence number, and a checksum. The abnormal packet quality parameter is the packet quality parameter which needs to be compared with the found data packet. The packet loss condition represents an abnormal condition of an abnormal packet quality parameter which needs to be compared with the data packet on the device, wherein 0 represents that the packet quality parameter is normal, and 1 represents that the packet quality parameter is abnormal. In conjunction with fig. 11, it can be determined that the network device that caused the anomaly of the data packet 2 and the data packet 3 of the reference table 3, and thus the anomaly of the video quality, is UDG.
In the example in table 4, the determined network devices causing video quality abnormality are the same for different packets, and in other application scenarios, due to error, there may be a plurality of network devices causing video quality abnormality determined for different packets. In this case, the determined network devices causing video quality abnormality may be further screened to obtain one network device having the highest possibility of causing video quality abnormality.
In a possible implementation manner, when there are a plurality of data packets corresponding to the abnormal packet quality parameter, there are a plurality of groups of data packets found to satisfy the condition,
according to the found packet quality parameters of the data packets meeting the conditions, determining the network equipment causing the video quality abnormity of the video playing end from the network equipment on the transmission path of the data packets, wherein the network equipment comprises:
according to the packet quality parameters of a plurality of groups of data packets meeting the conditions, determining network equipment which may cause video quality abnormity at a video playing end from each network equipment on the transmission path of the data packets for a plurality of times;
counting the network equipment which is determined for many times and is possible to cause the video quality abnormity of the video playing end, wherein the network equipment which has the most occurrence times is used as the network equipment which causes the video quality abnormity of the video playing end.
The accuracy of the determined network equipment causing the video quality abnormity can be improved by determining the packet quality parameters of the plurality of data packets on each network equipment on the transmission path for a plurality of times, so that the probability of the video quality abnormity is reduced due to the same problem of the same equipment after the corresponding solution is executed for the determined network equipment causing the video quality abnormity.
For example, the network device with the highest probability of causing video quality abnormality is determined according to statistics of the occurrence times of all devices in the column of "devices causing abnormality" under all abnormal video frames. In combination with table 4, packet loss parameters caused by UDG are abnormal for 2 times, and packet loss parameters of other devices are abnormal for 0 time, so that the network device with the highest possibility of causing video quality abnormality is UDG. If, in another example, the UDG-induced packet loss parameter is abnormal for 2 times, the EoR-induced packet loss parameter is abnormal for 5 times, and the gNB-induced packet loss parameter is abnormal for 0 time, the network device with the highest probability of causing video quality abnormality is EoR.
Fig. 13 shows an exemplary structural diagram of a video quality problem determination apparatus according to an embodiment of the present application.
As shown in fig. 13, in a possible implementation manner, the present application further proposes a video quality problem determination apparatus, including: an information receiving module 130, configured to receive protocol information on a video playing end and a network device, where the network device is configured to transmit video data to the video playing end, and the protocol information indicates a sending mode of a video frame, where the video frame includes a plurality of data packets; a parameter determining module 131, configured to determine, according to the protocol information, an abnormal frame quality parameter and an abnormal packet quality parameter, where the abnormal frame quality parameter indicates that a quality of a video frame to which the abnormal frame quality parameter belongs is abnormal, and the abnormal packet quality parameter indicates that a quality of a data packet to which the abnormal packet quality parameter belongs is abnormal; an abnormal cause determining module 132, configured to determine a cause and a device causing video quality abnormality at the video playing end according to the abnormal frame quality parameter and/or the abnormal packet quality parameter.
In one possible implementation manner, the protocol information includes streaming media layer protocol information, network layer protocol information, and transport layer protocol information of a data packet, where the streaming media layer protocol information includes streaming media layer protocol information of a data packet on a video playing end, the network layer protocol information includes network layer protocol information of a data packet on a network device and on the video playing end, the transport layer protocol information includes transport layer protocol information of a data packet on a network device and on the video playing end, and determining an abnormal frame quality parameter and an abnormal packet quality parameter according to the protocol information includes: determining at least one video frame and each data packet corresponding to each frame in the at least one video frame according to the streaming media layer protocol information; determining a frame quality parameter of each frame in the at least one video frame according to the streaming media layer protocol information and the network layer protocol information; determining packet quality parameters of each data packet corresponding to each frame in the at least one video frame according to the network layer protocol information and the transmission layer protocol information; and determining the abnormal frame quality parameter according to the frame quality parameter corresponding to each frame in the at least one video frame and the frame quality parameter threshold, and determining the abnormal packet quality parameter according to the packet quality parameter of each data packet corresponding to each frame in the at least one video frame and the packet quality parameter threshold.
In a possible implementation manner, the determining the abnormal frame quality parameter according to the frame quality parameter and the frame quality parameter threshold corresponding to each frame in the at least one video frame, and determining the abnormal packet quality parameter according to the packet quality parameter and the packet quality parameter threshold corresponding to each data packet in each frame in the at least one video frame includes: for each frame of the at least one video frame, determining a frame quality parameter of which the numerical value is greater than the frame quality parameter threshold value in the frame quality parameters of the video frame as the abnormal frame quality parameter; and aiming at each frame of the at least one video frame, determining the packet quality parameter with the value larger than the threshold value of the packet quality parameter in the packet quality parameters of the data packet corresponding to the video frame as the abnormal packet quality parameter.
In one possible implementation, the determining, according to the streaming media layer protocol information and the network layer protocol information, a frame quality parameter of each frame in the at least one video frame includes: for each frame of the at least one video frame, determining the arrival time of a first packet and the arrival time of a last packet of the video frame according to the packet time of a data packet corresponding to the video frame, wherein the arrival time of the first packet represents the arrival time of a first data packet corresponding to the video frame, and the arrival time of the last packet represents the arrival time of a last data packet corresponding to the video frame; respectively calculating a head packet interval and a tail packet interval of adjacent video frames according to the arrival time of the head packet and the arrival time of the tail packet, wherein the head packet interval represents the arrival time difference of a first data packet corresponding to the adjacent video frames, and the tail packet interval represents the arrival time difference of a last data packet corresponding to the adjacent video frames; respectively calculating the frame interval of the adjacent video frames according to the frame time stamps; calculating the frame head packet jitter parameter of the current video frame according to the head packet interval, the frame interval and the frame head packet jitter parameter of the previous video frame; and calculating the frame end packet jitter parameter of the current video frame according to the end packet interval, the frame interval and the frame end packet jitter parameter of the previous video frame.
In one possible implementation, the determining, according to the streaming media layer protocol information and the network layer protocol information, a frame quality parameter of each frame in the at least one video frame includes: determining a frame loss parameter of the video frame according to a stream media layer sequence number of a data packet corresponding to the video frame aiming at each frame of the at least one video frame; and aiming at each frame of the at least one video frame, sequencing the data packets corresponding to the video frame according to the sequence of the packet time of the data packets corresponding to the video frame to obtain a sequencing result corresponding to the video frame, and determining the frame disorder parameter of the video frame according to the sequencing result and the stream media layer sequence number of the data packets corresponding to the video frame.
In a possible implementation manner, the determining, for each of the at least one video frame, a frame loss parameter of the video frame according to a sequence number of a streaming media layer of a data packet corresponding to the video frame includes: for each frame of the at least one video frame, determining the maximum value and the minimum value of the stream media layer sequence number of the data packet corresponding to the video frame and the number of the stream media layer sequence number according to the stream media layer sequence number of the data packet corresponding to the video frame, determining the theoretical value of the number of the data packets corresponding to the video frame according to the maximum value and the minimum value of the stream media layer sequence number, and determining the frame packet loss parameter of the video frame according to the difference between the theoretical value of the number of the data packets corresponding to the video frame and the number of the stream media layer sequence number of the data packet corresponding to the video frame; determining a frame disorder parameter of the video frame according to the sequencing result and the sequence number of the streaming media layer of the data packet corresponding to the video frame, including: and when the numerical value of the sequence number of the current data packet in the streaming media layer is greater than the sequence number of the streaming media layer of the data packet before the current data packet in the sequencing result, determining the frame disorder parameter corresponding to the video frame according to the sequencing sequence number of the current data packet in the sequencing result, the sequence number of the current data packet in the streaming media layer, and the minimum value of the sequence number of the streaming media layer of the data packet corresponding to the video frame.
In a possible implementation manner, a packet quality parameter of each data packet corresponding to each frame in the at least one video frame is determined according to network layer protocol information and transport layer protocol information of the data packet, where the packet quality parameter includes: the data packet processing method comprises one or more of a packet interval parameter representing the time interval between a current data packet and a previous data packet, a packet jitter parameter representing the time delay jitter between the current data packet and the previous data packet, a disorder parameter representing the disorder condition of the data packet, a packet loss parameter representing the packet loss condition of the data packet, a retransmission parameter representing the retransmission condition of the data packet, a time delay parameter representing the time delay condition of the data packet and a corresponding response message, an IP message fragment parameter representing whether the data packet has an IP message fragment, and a checksum error parameter representing whether the checksum of the data packet has an error.
In a possible implementation manner, the determining, according to the abnormal frame quality parameter and/or the abnormal packet quality parameter, a cause and a device causing video quality abnormality at a video playing end includes: when the frame quality parameters of the video frames and the packet quality parameters of the data packets corresponding to the video frames meet a first condition, determining that the reason causing the video quality abnormity of a video playing end is a video coding problem, and determining that equipment causing the video quality abnormity of the video playing end is equipment used for video coding; wherein the first condition comprises: the frame packet loss parameter of the video frame is abnormal and the packet loss parameter of the data packet corresponding to the video frame is normal, or the frame disorder parameter of the video frame is abnormal and the disorder parameter of the data packet corresponding to the video frame is normal.
In a possible implementation manner, the determining, according to the abnormal frame quality parameter and/or the abnormal packet quality parameter, a cause and a device causing video quality abnormality at a video playing end includes: when the frame quality parameters of the video frames and the packet quality parameters of the data packets corresponding to the video frames meet a second condition, determining that the reason causing the video quality abnormity of the video playing end is a terminal parameter problem, and determining that the equipment causing the video quality abnormity of the video playing end is equipment used for video playing; wherein the second condition comprises: any one of the jitter parameter of the first frame packet and the jitter parameter of the last frame packet of the video frame is abnormal, and any one of the IP message fragment parameter, the checksum error parameter of the data packet corresponding to the video frame is abnormal.
In a possible implementation manner, the determining, according to the abnormal frame quality parameter and/or the abnormal packet quality parameter, a cause and a device causing video quality abnormality at a video playing end includes: when the frame quality parameters of the video frames and the packet quality parameters of the data packets corresponding to the video frames meet a third condition, determining that the reason causing the video quality abnormity of a video playing end is a network transmission problem; wherein the third condition comprises: the frame packet loss parameter of the video frame is abnormal and the packet loss parameter of the data packet corresponding to the video frame is abnormal, or the frame disorder parameter of the video frame is abnormal and the disorder parameter of the data packet corresponding to the video frame is abnormal, or any one of the frame head packet jitter parameter and the frame tail packet jitter parameter of the video frame is abnormal and any one of the packet jitter parameter, the packet interval parameter, the retransmission parameter and the time delay parameter of the data packet corresponding to the video frame is abnormal.
In a possible implementation manner, after determining that the cause of the video quality abnormality at the video playing end is a network transmission problem, the determining, according to the abnormal frame quality parameter and/or the abnormal packet quality parameter, the cause of the video quality abnormality at the video playing end and the device thereof further includes: determining the network layer sequence number of the data packet corresponding to the abnormal packet quality parameter according to the network layer protocol information of the data packet; determining the packet time, the transmission layer sequence number and the checksum of the data packet corresponding to the abnormal packet quality parameter according to the transmission layer protocol information of the data packet; receiving network topology data on network equipment, and determining a transmission path of each data packet in a network according to the network topology data; finding out data packets of which the packet time, the network layer serial number, the transmission layer serial number and the check sum all meet the conditions in each network device on the transmission path according to the transmission path of the data packet corresponding to the abnormal packet quality parameter in the network; and according to the found packet quality parameters of the data packets meeting the conditions, determining the network equipment causing the video quality abnormity of the video playing end from the network equipment on the transmission path of the data packets.
In one possible implementation, the conditions that the packet time, the network layer sequence number, the transport layer sequence number, and the checksum satisfy include: and the absolute value of the difference between the packet time and the packet time of the data packet corresponding to the abnormal packet quality parameter is smaller than a preset threshold, and the network layer sequence number, the transmission layer sequence number, the checksum and the data packet corresponding to the abnormal packet quality parameter are the same.
In a possible implementation manner, when there are a plurality of data packets corresponding to the abnormal packet quality parameter, there are a plurality of groups of found data packets meeting the condition, and according to the found packet quality parameter of the data packet meeting the condition, determining, from each network device on the transmission path of the data packet, a network device that causes the video quality abnormality at the video playing end, including: according to the packet quality parameters of a plurality of groups of data packets meeting the conditions, determining network equipment which may cause video quality abnormity at a video playing end from each network equipment on the transmission path of the data packets for a plurality of times; counting the network equipment which is determined for many times and is possible to cause the video quality abnormity of the video playing end, wherein the network equipment which has the most occurrence times is used as the network equipment which causes the video quality abnormity of the video playing end.
An embodiment of the present application provides a video quality problem determination apparatus, including: a processor and a memory for storing processor-executable instructions; wherein the processor is configured to implement the above method when executing the instructions.
Embodiments of the present application provide a non-transitory computer readable storage medium having stored thereon computer program instructions which, when executed by a processor, implement the above-described method.
Embodiments of the present application provide a computer program product comprising computer readable code, or a non-transitory computer readable storage medium carrying computer readable code, which when run in a processor of an electronic device, the processor in the electronic device performs the above method.
Fig. 14 shows an exemplary structural diagram of a video quality problem determination apparatus according to an embodiment of the present application.
As shown in fig. 14, the video quality problem determination apparatus may include at least one of a mobile phone, a foldable electronic device, a tablet computer, a desktop computer, a laptop computer, a handheld computer, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, a cellular phone, a Personal Digital Assistant (PDA), an Augmented Reality (AR) device, a Virtual Reality (VR) device, an Artificial Intelligence (AI) device, a wearable device, a vehicle-mounted device, an intelligent home device, or a smart city device, a server device. The embodiment of the present application does not particularly limit the specific type of the video quality problem determination apparatus.
The video quality problem determination apparatus may include a processor 110, a memory 121, and a communication module 160. It is to be understood that the illustrated structure of the embodiments of the present application does not constitute a specific limitation to the video quality problem determination apparatus. In other embodiments of the present application, the video quality problem determination apparatus may include more or fewer components than shown, or combine certain components, or split certain components, or a different arrangement of components. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.
Processor 110 may include one or more processing units, such as: the processor 110 may include an Application Processor (AP), a modem processor, a Graphics Processing Unit (GPU), an Image Signal Processor (ISP), a controller, a video codec, a Digital Signal Processor (DSP), a baseband processor, and/or a neural-Network Processing Unit (NPU), etc. The different processing units may be separate devices or may be integrated into one or more processors.
The processor 110 may generate operation control signals according to the instruction operation code and the timing signal, so as to complete the control of instruction fetching and instruction execution.
A memory may also be provided in processor 110 for storing instructions and data. In some embodiments, the memory in the processor 110 may be a cache memory. The memory may store instructions or data that have been used or used more frequently by the processor 110. If the processor 110 needs to use the instruction or data, it can be called directly from the memory. Avoiding repeated accesses reduces the latency of the processor 110, thereby increasing the efficiency of the system.
Memory 121 may be used to store computer-executable program code, which includes instructions. The memory 121 may include a program storage area and a data storage area. The storage program area may store an operating system, an application program required for at least one function (e.g., an abnormal frame quality parameter determination function, an abnormal packet quality parameter determination function), and the like. The storage data area may store data (such as frame quality parameters, packet quality parameters, etc.) created during use of the video quality problem determination apparatus, and the like. Further, the memory 121 may include a high-speed random access memory, and may further include a nonvolatile memory, such as at least one magnetic disk storage device, a flash memory device, a universal flash memory (UFS), and the like. The processor 110 executes various functional methods of the video quality problem determination apparatus or the above-described video quality problem determination method by executing instructions stored in the memory 121 and/or instructions stored in a memory provided in the processor.
The communication module 160 may be configured to receive streaming data (e.g., protocol information and network topology data in the embodiment of the present application) from other apparatuses or devices (e.g., protocol information acquisition apparatus and topology data acquisition apparatus in the embodiment of the present application) through wireless communication/wired communication, and output the streaming data (e.g., information of the apparatus and the reason causing the video quality problem determined in the embodiment of the present application) to the other apparatuses or devices. For example, solutions for wireless communication including WLAN (such as Wi-Fi network), bluetooth (BT), global Navigation Satellite System (GNSS), frequency Modulation (FM), near Field Communication (NFC), infrared (IR), and the like may be provided.
The computer readable storage medium may be a tangible device that can hold and store the instructions for use by the instruction execution device. The computer readable storage medium may be, for example, but not limited to, an electronic memory device, a magnetic memory device, an optical memory device, an electromagnetic memory device, a semiconductor memory device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: a portable computer diskette, a hard disk, a Random Access Memory (RAM), a Read-Only Memory (ROM), an erasable Programmable Read-Only Memory (EPROM or flash Memory), a Static Random Access Memory (SRAM), a portable Compact Disc Read-Only Memory (CD-ROM), a Digital Versatile Disc (DVD), a Memory stick, a floppy disk, a mechanical coding device, such as a punch card or in-groove bump structure having instructions stored thereon, and any suitable combination of the foregoing.
The computer readable program instructions or code described herein may be downloaded to the respective computing/processing device from a computer readable storage medium, or to an external computer or external storage device via a network, such as the internet, a local area network, a wide area network, and/or a wireless network. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers and/or edge servers. The network adapter card or network interface in each computing/processing device receives computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in a computer-readable storage medium in the respective computing/processing device.
The computer program instructions for carrying out operations of the present application may be assembler instructions, instruction Set Architecture (ISA) instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The computer readable program instructions may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer through any type of Network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the internet using an internet service provider). In some embodiments, the electronic circuitry can execute computer-readable program instructions to implement aspects of this application by personalizing, with state information of the computer-readable program instructions, an electronic circuit such as a Programmable Logic circuit, a Field-Programmable Gate Array (FPGA), or a Programmable Logic Array (PLA).
Various aspects of the present application are described herein with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the application. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer-readable program instructions.
These computer-readable program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks. These computer-readable program instructions may also be stored in a computer-readable storage medium that can direct a computer, programmable data processing apparatus, and/or other devices to function in a particular manner, such that the computer-readable medium storing the instructions comprises an article of manufacture including instructions which implement the function/act specified in the flowchart and/or block diagram block or blocks.
The computer readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce a computer implemented process such that the instructions which execute on the computer, other programmable apparatus or other devices implement the functions/acts specified in the flowchart and/or block diagram block or blocks.
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of apparatus, systems, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved.
It is also noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by hardware (e.g., an electronic Circuit or an ASIC (Application Specific Integrated Circuit)) for performing the corresponding functions or acts, or combinations of hardware and software, such as firmware.
While the invention has been described in connection with various embodiments, other variations to the disclosed embodiments can be understood and effected by those skilled in the art in practicing the claimed invention, from a review of the drawings, the disclosure, and the appended claims. In the claims, the word "comprising" does not exclude other elements or steps, and the word "a" or "an" does not exclude a plurality. A single processor or other unit may fulfill the functions of several items recited in the claims. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.
Having described embodiments of the present application, the foregoing description is intended to be exemplary, not exhaustive, and not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen in order to best explain the principles of the embodiments, the practical application, or improvements made to the technology in the marketplace, or to enable others of ordinary skill in the art to understand the embodiments disclosed herein.

Claims (15)

1. A method for video quality problem determination, the method comprising:
receiving protocol information on a video playing end and network equipment, wherein the network equipment is used for transmitting video data to the video playing end, the protocol information indicates a sending mode of a video frame, and the video frame comprises a plurality of data packets;
determining an abnormal frame quality parameter and an abnormal packet quality parameter according to the protocol information, wherein the abnormal frame quality parameter represents that the quality of a video frame to which the abnormal frame quality parameter belongs is abnormal, and the abnormal packet quality parameter represents that the quality of a data packet to which the abnormal packet quality parameter belongs is abnormal;
and determining the reason and equipment causing the video quality abnormity of the video playing end according to the abnormal frame quality parameters and/or the abnormal packet quality parameters.
2. The method of claim 1, wherein the protocol information comprises streaming media layer protocol information, network layer protocol information, and transport layer protocol information for the packets, wherein the streaming media layer protocol information comprises streaming media layer protocol information for the packets at the video playback end, wherein the network layer protocol information comprises network layer protocol information for the packets at the network device and at the video playback end, wherein the transport layer protocol information comprises transport layer protocol information for the packets at the network device and at the video playback end,
the determining of the abnormal frame quality parameter and the abnormal packet quality parameter according to the protocol information includes:
determining at least one video frame and each data packet corresponding to each frame in the at least one video frame according to the streaming media layer protocol information;
determining a frame quality parameter of each frame in the at least one video frame according to the streaming media layer protocol information and the network layer protocol information;
determining packet quality parameters of each data packet corresponding to each frame in the at least one video frame according to the network layer protocol information and the transmission layer protocol information;
and determining the abnormal frame quality parameter according to the frame quality parameter corresponding to each frame in the at least one video frame and the frame quality parameter threshold, and determining the abnormal packet quality parameter according to the packet quality parameter of each data packet corresponding to each frame in the at least one video frame and the packet quality parameter threshold.
3. The method of claim 2, wherein the determining the abnormal frame quality parameter according to the frame quality parameter and the frame quality parameter threshold corresponding to each frame of the at least one video frame, and the determining the abnormal packet quality parameter according to the packet quality parameter and the packet quality parameter threshold corresponding to each data packet of each frame of the at least one video frame comprises:
for each frame of the at least one video frame, determining a frame quality parameter of which the value is greater than the frame quality parameter threshold value in the frame quality parameters of the video frame as the abnormal frame quality parameter;
and determining the packet quality parameters with the numerical values larger than the packet quality parameter threshold value in the packet quality parameters of the data packets corresponding to the video frame as the abnormal packet quality parameters aiming at each frame of the at least one video frame.
4. The method of claim 2 or 3, wherein the streaming media layer protocol information of the data packet includes a frame timestamp indicating a video frame to which the data packet belongs, the network layer protocol information of the data packet includes a packet time indicating a time of the data packet, the frame quality parameter includes a frame head packet jitter parameter and a frame end packet jitter parameter, the frame head packet jitter parameter represents a delay jitter condition of a first data packet corresponding to the video frame, the frame end packet jitter parameter represents a delay jitter condition of a last data packet corresponding to the video frame,
determining a frame quality parameter of each frame in the at least one video frame according to the streaming media layer protocol information and the network layer protocol information, including:
for each frame of the at least one video frame, determining the arrival time of a first packet and the arrival time of a last packet of the video frame according to the packet time of a data packet corresponding to the video frame, wherein the arrival time of the first packet represents the arrival time of a first data packet corresponding to the video frame, and the arrival time of the last packet represents the arrival time of a last data packet corresponding to the video frame;
respectively calculating a head packet interval and a tail packet interval of adjacent video frames according to the arrival time of the head packet and the arrival time of the tail packet, wherein the head packet interval represents the arrival time difference of a first data packet corresponding to the adjacent video frames, and the tail packet interval represents the arrival time difference of a last data packet corresponding to the adjacent video frames;
respectively calculating the frame interval of the adjacent video frames according to the frame time stamps;
calculating the frame head packet jitter parameter of the current video frame according to the head packet interval, the frame interval and the frame head packet jitter parameter of the previous video frame;
and calculating the frame end packet jitter parameter of the current video frame according to the end packet interval, the frame interval and the frame end packet jitter parameter of the previous video frame.
5. The method according to any of claims 2-4, wherein the streaming media layer protocol information of the data packet comprises a streaming media layer sequence number indicating an order of the data packet in the video frame to which the data packet belongs, the network layer protocol information of the data packet comprises a packet time indicating a time of the data packet, the frame quality parameter comprises a frame loss parameter representing a loss condition of the data packet of the video frame and a frame misordering parameter representing a misordering condition of the data packet of the video frame,
determining a frame quality parameter of each frame in the at least one video frame according to the streaming media layer protocol information and the network layer protocol information, including:
for each frame of the at least one video frame, determining a frame packet loss parameter of the video frame according to a stream media layer sequence number of a data packet corresponding to the video frame;
and aiming at each frame of the at least one video frame, sequencing the data packets corresponding to the video frame according to the sequence of the packet time of the data packets corresponding to the video frame to obtain a sequencing result corresponding to the video frame, and determining the frame disorder parameter of the video frame according to the sequencing result and the stream media layer sequence number of the data packets corresponding to the video frame.
6. The method of claim 5, wherein the determining, for each of the at least one video frame, a frame loss parameter of the video frame according to a streaming media layer sequence number of a data packet corresponding to the video frame comprises:
for each frame of the at least one video frame, determining the maximum value and the minimum value of the stream media layer sequence number of the data packet corresponding to the video frame and the number of the stream media layer sequence number according to the stream media layer sequence number of the data packet corresponding to the video frame, determining the theoretical value of the number of the data packets corresponding to the video frame according to the maximum value and the minimum value of the stream media layer sequence number, and determining the frame packet loss parameter of the video frame according to the difference between the theoretical value of the number of the data packets corresponding to the video frame and the number of the stream media layer sequence number of the data packet corresponding to the video frame;
determining a frame disorder parameter of the video frame according to the sequencing result and the sequence number of the stream media layer of the data packet corresponding to the video frame, including:
and when the value of the stream media layer sequence number of the current data packet is greater than the stream media layer sequence number of the data packet before the current data packet in the sequencing result, determining the frame disorder parameter corresponding to the video frame according to the sequencing sequence number of the current data packet in the sequencing result, the stream media layer sequence number of the current data packet and the minimum value of the stream media layer sequence number of the data packet corresponding to the video frame.
7. The method according to claims 2-6, wherein the packet quality parameter of each data packet corresponding to each frame of the at least one video frame is determined according to the network layer protocol information and the transport layer protocol information of the data packet, and the packet quality parameter comprises: the data packet processing method comprises one or more of a packet interval parameter representing the time interval between a current data packet and a previous data packet, a packet jitter parameter representing the time delay jitter between the current data packet and the previous data packet, a disorder parameter representing the disorder condition of the data packet, a packet loss parameter representing the packet loss condition of the data packet, a retransmission parameter representing the retransmission condition of the data packet, a time delay parameter representing the time delay condition of the data packet and a corresponding response message, an IP message fragment parameter representing whether the data packet has an IP message fragment, and a checksum error parameter representing whether the checksum of the data packet has an error.
8. The method according to claim 7, wherein the determining the cause and equipment causing video quality abnormality at a video playing end according to the abnormal frame quality parameter and/or the abnormal packet quality parameter comprises:
when the frame quality parameters of the video frames and the packet quality parameters of the data packets corresponding to the video frames meet a first condition, determining that the reason causing the video quality abnormity of a video playing end is a video coding problem, and determining that equipment causing the video quality abnormity of the video playing end is equipment used for video coding;
wherein the first condition comprises:
the frame packet loss parameter of the video frame is abnormal and the packet loss parameter of the data packet corresponding to the video frame is normal, or
The frame disorder parameters of the video frames are abnormal, and the disorder parameters of the data packets corresponding to the video frames are normal.
9. The method according to claim 7 or 8, wherein the determining the cause and equipment causing video quality abnormality at a video playing end according to the abnormal frame quality parameter and/or the abnormal packet quality parameter comprises:
when the frame quality parameters of the video frames and the packet quality parameters of the data packets corresponding to the video frames meet a second condition, determining that the reason causing the video quality abnormity of the video playing end is a terminal parameter problem, and determining that the equipment causing the video quality abnormity of the video playing end is equipment used for video playing;
wherein the second condition comprises:
any one of the jitter parameter of the first frame packet and the jitter parameter of the last frame packet of the video frame is abnormal, and any one of the IP message fragment parameter, the check sum error parameter of the data packet corresponding to the video frame is abnormal.
10. The method according to any one of claims 7-9, wherein the determining the cause and equipment causing video quality abnormality at a video playing end according to the abnormal frame quality parameter and/or the abnormal packet quality parameter comprises:
when the frame quality parameters of the video frames and the packet quality parameters of the data packets corresponding to the video frames meet a third condition, determining that the reason causing the video quality abnormity of a video playing end is a network transmission problem;
wherein the third condition comprises:
the frame packet loss parameter of the video frame is abnormal and the packet loss parameter of the data packet corresponding to the video frame is abnormal, or
The frame disorder parameter of the video frame is abnormal and the disorder parameter of the data packet corresponding to the video frame is abnormal, or
Any one of the jitter parameter of the first frame packet and the jitter parameter of the last frame packet of the video frame is abnormal, and any one of the jitter parameter, the packet interval parameter, the retransmission parameter and the time delay parameter of the data packet corresponding to the video frame is abnormal.
11. The method according to claim 10, wherein after determining that the cause of video quality abnormality at the video playing side is a network transmission problem, said determining the cause and device of video quality abnormality at the video playing side according to the abnormal frame quality parameter and/or the abnormal packet quality parameter further comprises:
determining a network layer sequence number of the data packet corresponding to the abnormal packet quality parameter according to the network layer protocol information of the data packet;
determining the packet time, the transmission layer sequence number and the checksum of the data packet corresponding to the abnormal packet quality parameter according to the transmission layer protocol information of the data packet;
receiving network topology data on network equipment, and determining a transmission path of each data packet in a network according to the network topology data;
finding out data packets of which the packet time, the network layer serial number, the transmission layer serial number and the check sum all meet the conditions in each network device on the transmission path according to the transmission path of the data packet corresponding to the abnormal packet quality parameter in the network;
and according to the found packet quality parameters of the data packets meeting the conditions, determining the network equipment causing the video quality abnormity of the video playing end from the network equipment on the transmission path of the data packets.
12. An apparatus for video quality problem determination, the apparatus comprising:
the information receiving module is used for receiving protocol information on a video playing end and network equipment, the network equipment is used for transmitting video data to the video playing end, the protocol information indicates a sending mode of a video frame, and the video frame comprises a plurality of data packets;
a parameter determining module, configured to determine, according to the protocol information, an abnormal frame quality parameter and an abnormal packet quality parameter, where the abnormal frame quality parameter indicates that the quality of a video frame to which the abnormal frame quality parameter belongs is abnormal, and the abnormal packet quality parameter indicates that the quality of a data packet to which the abnormal packet quality parameter belongs is abnormal;
and the abnormal reason determining module is used for determining the reason and the equipment causing the video quality abnormity of the video playing end according to the abnormal frame quality parameters and/or the abnormal packet quality parameters.
13. A video quality problem determination apparatus, comprising:
a processor;
a memory for storing processor-executable instructions;
wherein the processor is configured to implement the method of any one of claims 1-11 when executing the instructions.
14. A non-transitory computer readable storage medium having stored thereon computer program instructions, wherein the computer program instructions, when executed by a processor, implement the method of any one of claims 1-11.
15. A computer program product comprising computer readable code or a non-transitory computer readable storage medium carrying computer readable code, wherein when the computer readable code is run in an electronic device, a processor in the electronic device performs the method of any of claims 1-11.
CN202110993951.3A 2021-08-27 2021-08-27 Video quality problem determination method, device, storage medium and program product Pending CN115734043A (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN202110993951.3A CN115734043A (en) 2021-08-27 2021-08-27 Video quality problem determination method, device, storage medium and program product
PCT/CN2022/078279 WO2023024469A1 (en) 2021-08-27 2022-02-28 Video quality problem determination method and apparatus, storage medium, and program product

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110993951.3A CN115734043A (en) 2021-08-27 2021-08-27 Video quality problem determination method, device, storage medium and program product

Publications (1)

Publication Number Publication Date
CN115734043A true CN115734043A (en) 2023-03-03

Family

ID=85290196

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110993951.3A Pending CN115734043A (en) 2021-08-27 2021-08-27 Video quality problem determination method, device, storage medium and program product

Country Status (2)

Country Link
CN (1) CN115734043A (en)
WO (1) WO2023024469A1 (en)

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN100459528C (en) * 2005-07-14 2009-02-04 华为技术有限公司 Method for inspecting Qos in telecommunication network
CN101909060B (en) * 2010-08-05 2012-11-21 浙江工业大学 Qos control method suitable for real-time streaming media transmission of mobile videos
CN103024440B (en) * 2013-01-11 2014-12-31 德科仕通信(上海)有限公司 Video quality evaluation and fault diagnosis method aiming at HAS (HTTP Adaptive Streaming)
CN108156056B (en) * 2017-12-28 2021-07-09 华为技术有限公司 Network quality measuring method and device
CN111953961A (en) * 2020-06-30 2020-11-17 武汉烽火众智数字技术有限责任公司 Method and device for analyzing video playing problems based on national standard
CN111918055B (en) * 2020-06-30 2022-06-24 上海艾策通讯科技股份有限公司 Video quality abnormity positioning method and device, storage medium and electronic equipment

Also Published As

Publication number Publication date
WO2023024469A1 (en) 2023-03-02

Similar Documents

Publication Publication Date Title
US9253063B2 (en) Bi-directional video compression for real-time video streams during transport in a packet switched network
US9585062B2 (en) System and method for implementation of dynamic encoding rates for mobile devices
KR100651486B1 (en) Apparatus and Method for transporting MPEG contents through Internet Protocol Network
US20130091528A1 (en) Video reproduction system, receive terminal, home gateway device, and quality control system
US20140146877A1 (en) Method for dynamically adapting video image parameters for facilitating subsequent applications
CN1938972A (en) Methods and apparatuses for measuring transmission quality of multimedia data
JP6743192B2 (en) Video service quality evaluation method and apparatus
CN108307202B (en) Real-time video transcoding sending method and device and user terminal
US20110187926A1 (en) Apparatus and method for correcting jitter
US9729903B2 (en) Data transmitting device and method
CN109194974B (en) Media low-delay communication method and system for network video live broadcast
CN112565224B (en) Video processing method and device
CN109600341B (en) Instant messaging detection method, equipment and computer storage medium
WO2016068760A1 (en) Video stream synchronization
EP3491784B1 (en) Estimation of losses in a video stream
CN112954433A (en) Video processing method and device, electronic equipment and storage medium
KR102392888B1 (en) Method and Apparatus for Improving Packet Loss Recovery
US10402415B2 (en) Intelligently distributed stage data mining system
WO2023217188A1 (en) Livestream data transmission method, apparatus and system, device and medium
CN115734043A (en) Video quality problem determination method, device, storage medium and program product
CN104904170A (en) Method and apparatus for efficiently prioritizing elements in a video stream for low-bandwidth transmission
CN112688824B (en) RTP packet loss detection method, device, equipment and computer readable storage medium
CN110545447B (en) Audio and video synchronization method and device
US10772010B2 (en) Transmission apparatus, method for controlling transmission, and recording medium
JP6888689B2 (en) Transmitter, receiver and video distribution method

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication