WO2023024469A1 - Procédé et appareil de détermination de problème de qualité vidéo, support de stockage et produit-programme - Google Patents

Procédé et appareil de détermination de problème de qualité vidéo, support de stockage et produit-programme Download PDF

Info

Publication number
WO2023024469A1
WO2023024469A1 PCT/CN2022/078279 CN2022078279W WO2023024469A1 WO 2023024469 A1 WO2023024469 A1 WO 2023024469A1 CN 2022078279 W CN2022078279 W CN 2022078279W WO 2023024469 A1 WO2023024469 A1 WO 2023024469A1
Authority
WO
WIPO (PCT)
Prior art keywords
frame
packet
video
parameter
abnormal
Prior art date
Application number
PCT/CN2022/078279
Other languages
English (en)
Chinese (zh)
Inventor
杨辉
姜军庆
徐斌斌
高鹏
庞宏超
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Publication of WO2023024469A1 publication Critical patent/WO2023024469A1/fr

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client 
    • H04N21/63—Control signaling related to video distribution between client, server and network components; Network processes for video distribution between server and clients or between remote clients, e.g. transmitting basic layer and enhancement layers over different transmission paths, setting up a peer-to-peer communication via Internet between remote STB's; Communication protocols; Addressing
    • H04N21/647—Control signaling between network components and server or clients; Network processes for video distribution between server and clients, e.g. controlling the quality of the video stream, by dropping packets, protecting content from unauthorised alteration within the network, monitoring of network load, bridging between two different networks, e.g. between IP and wireless

Definitions

  • the present application relates to the field of video quality detection, and in particular to a video quality problem determination method, device, storage medium and program product.
  • 5G technology Compared with 4G, 5G technology has the advantages of large bandwidth and low latency, so it can provide technical support for high-definition video live broadcast, real-time camera monitoring, drone shooting and other video return scenarios that require real-time transmission of video streaming media.
  • Video backhaul is a real-time business scenario based on streaming media.
  • video frames are obtained after video capture by cameras and other equipment, and video frames are encoded and compressed by encoders/backpack servers, etc.
  • compressed video frames are encapsulated by streaming media layer protocols to generate live streams, live streams It is transmitted to the video server through the network equipment, and after decapsulation and decoding, the video player renders and displays the picture.
  • video quality problems include: freezing, blurred screen and other problems that may occur when users watch videos and lead to poor user experience.
  • the above-mentioned problems such as freezing and blurring are caused by the abnormality of the video frame, and there may be multiple devices involved in the process of video frame collection, encoding, packaging, transmission, decapsulation, and decoding.
  • it is necessary to locate the device that causes the video quality problem and the specific reason why the device causes the video quality problem.
  • Existing technologies are still unable to quickly and accurately complete the determination of the causes and equipment that lead to video quality problems.
  • a method, device, storage medium, and program product for determining video quality problems are proposed. According to the method, device, storage medium, and program product for determining video quality problems in the embodiments of the present application, it is possible to quickly and accurately complete the video quality problem. Causes of quality problems and determination of equipment.
  • the embodiments of the present application provide a method for determining video quality problems, the method comprising: receiving protocol information on the video player and the network device, and the network device is used to transmit video data to the video player , the protocol information indicates the transmission mode of the video frame, and the video frame includes a plurality of data packets; an abnormal frame quality parameter and an abnormal packet quality parameter are determined according to the protocol information, and the abnormal frame quality parameter indicates the The quality of the video frame to which the abnormal frame quality parameter belongs is abnormal, and the abnormal packet quality parameter indicates that the quality of the data packet to which the abnormal packet quality parameter belongs is abnormal; according to the abnormal frame quality parameter and/or the abnormal Determine the cause and device of the abnormal video quality at the video playback end.
  • the video frame and the content included in the video frame can be determined according to the protocol information.
  • the transmission mode of multiple data packets can further determine abnormal frame quality parameters and abnormal packet quality parameters.
  • the abnormal frame quality parameter indicates that the quality of the video frame to which it belongs is abnormal
  • the abnormal packet quality parameter indicates that the quality of the data packet to which it belongs is abnormal
  • the video quality is associated with the video frame and the data packet
  • the abnormal frame quality parameter and the abnormal The packet quality parameter is associated with the abnormal video quality, so that according to the abnormal frame quality parameter and the abnormal packet quality parameter, the cause and equipment that cause the abnormal video quality of the video player can be determined.
  • the video quality problem determination method of the present application jointly determines the cause and equipment that cause the video quality abnormality at the video playback end through abnormal frame quality parameters at the frame level and abnormal packet quality parameters at the packet level, which can reduce the cost of completing the determination of the cause and equipment once. Time cost, and improve the accuracy of determining the cause and equipment.
  • the protocol information can also be extracted according to the payload header, and the cause and device that cause the video quality abnormality at the video playback end can be accurately determined. Expand the applicability of methods for determining video quality issues.
  • the protocol information includes streaming media layer protocol information, network layer protocol information, and transport layer protocol information of data packets
  • the stream The media layer protocol information includes the streaming media layer protocol information of the data packet on the video playback end
  • the network layer protocol information includes the network layer protocol information of the data packet on the network device and the video playback end
  • the transport layer protocol information includes The transport layer protocol information of the data packet on the network device and on the video playback end
  • said determining abnormal frame quality parameters and abnormal packet quality parameters according to the protocol information includes: determining at least one according to the streaming media layer protocol information The video frame and each data packet corresponding to each frame in the at least one video frame; according to the streaming media layer protocol information and the network layer protocol information, determine the frame quality parameters of each frame in the at least one video frame; according to The network layer protocol information and the transport layer protocol information determine the packet quality parameters of each data packet corresponding to each frame in the at least one video frame; according to the frame quality parameters corresponding to each frame
  • the frame quality parameter and the packet quality parameter can be determined through the combination of different protocol information, so as to reflect the quality of the video frame and the quality of the data packets under the video frame. Whether the frame quality parameter and the packet quality parameter are abnormal can be determined by the frame quality parameter threshold and the packet quality parameter threshold respectively, making it more convenient to determine the abnormal frame quality parameter and the abnormal packet quality parameter.
  • the second possible implementation of the method for determining video quality problems according to the frame quality parameters corresponding to each frame in the at least one video frame and the frame A quality parameter threshold, determining the abnormal frame quality parameter, determining the abnormal packet quality parameter according to the packet quality parameter and the packet quality parameter threshold of each data packet corresponding to each frame in the at least one video frame, including: For each frame of the at least one video frame, among the frame quality parameters of the video frame, the frame quality parameter whose value is greater than the frame quality parameter threshold is determined as the abnormal frame quality parameter; for the at least one video frame For each frame, among the packet quality parameters of the data packets corresponding to the video frame, the packet quality parameter whose value is greater than the packet quality parameter threshold is determined as the abnormal packet quality parameter.
  • abnormal frame quality parameters and packet quality parameters can be determined.
  • Abnormal parameters are determined by comparing parameter values with threshold values, and the determination of abnormal parameters can be completed accurately and quickly.
  • the streaming media layer protocol information of the data packet includes The frame time stamp of the video frame
  • the network layer protocol information of the data packet includes the packet time indicating the time of the data packet
  • the frame quality parameter includes a frame header jitter parameter and a frame tail packet jitter parameter
  • the frame header packet The jitter parameter represents the delay jitter situation of the first data packet corresponding to the video frame
  • the frame tail packet jitter parameter represents the delay jitter situation of the last data packet corresponding to the video frame
  • determining the frame quality parameters of each frame in the at least one video frame includes: for each frame of the at least one video frame, determining the frame quality parameter according to the packet time of the data packet corresponding to the video frame
  • the first packet arrival time and the last packet arrival time of the video frame, the first packet arrival time represents the arrival time of the first data packet corresponding to the video frame
  • the first frame packet jitter parameter representing the delay jitter of the first data packet corresponding to the video frame and the frame end packet jitter representing the delay jitter of the last data packet corresponding to the video frame can be calculated parameter, while characterizing the quality of the video frame, it improves the closeness between the frame quality parameter of the video frame and the actual jitter of the video frame.
  • the stream of data packets includes a stream media layer sequence number indicating the sequence of the data packet in the video frame to which it belongs, the network layer protocol information of the data packet includes a packet time indicating the time of the data packet, and the frame quality parameter includes a frame loss Packet parameter and frame out-of-sequence parameter, described frame packet loss parameter represents the loss situation of the data packet of video frame, and described frame out-of-sequence parameter represents the out-of-sequence situation of the data packet of video frame, and described according to described streaming media layer protocol Information and the network layer protocol information, determining the frame quality parameters of each frame in the at least one video frame includes: for each frame of the at least one video frame, according to the streaming media layer of the data packet corresponding to the video frame The serial number determines the frame packet loss parameter of the video frame; for each frame of the at least one video frame, sorts the
  • the frame packet loss parameter representing the loss of data packets of the video frame and the frame disorder parameter representing the disorder of the data packets of the video frame can be calculated, which improves the quality of the video frame while characterizing the quality of the video frame. Flexibility in choice of frame quality parameters.
  • determining the frame loss parameter of the video frame includes: for each frame of the at least one video frame, according to the streaming media layer sequence number of the data packet corresponding to the video frame, determining The maximum value of the streaming media layer sequence number of the data packet corresponding to this video frame, the minimum value and the number of streaming media layer sequence numbers, according to the maximum value and minimum value of the streaming media layer sequence number, determine the corresponding data of this video frame
  • the theoretical value of the number of packets according to the difference between the theoretical value of the number of data packets corresponding to the video frame and the number of streaming media layer serial numbers of the data packets corresponding to the video frame, determine the frame packet loss parameter of the video frame ; According to the sequence number of the streaming media layer of the data packet corresponding to the sorting result and the video frame, determine the frame disorder parameter of the
  • the frame packet loss parameter can reflect the information of whether the data packet in the video frame is lost or not, and when there is a packet loss, it can also reflect the information of the number of the packet loss; the frame disorder parameter can reflect the information in the video frame
  • the information about whether the data packets are out of order can also reflect the information about the out-of-order depth of the out-of-order data packets when there is out-of-order, so that the accuracy of determining the quality of video frames according to the frame quality parameters can be improved.
  • the at least one video frame The packet quality parameters of each data packet corresponding to each frame in the frame are determined according to the network layer protocol information and the transport layer protocol information of the data packet, and the packet quality parameters include: the packet representing the time interval between the current data packet and the previous data packet The interval parameter, the packet jitter parameter indicating the delay jitter between the current data packet and the previous data packet, the out-of-order parameter indicating the disorder of the data packet, the packet loss parameter indicating the packet loss, and the packet retransmission The retransmission parameter, the delay parameter indicating the delay of the data packet and the corresponding response message, the IP packet fragmentation parameter indicating whether there is IP packet fragmentation in the data packet, and whether the checksum of the data packet is One or more of the wrong checksum parameters occurred.
  • the abnormal frame quality parameter and/or the abnormal packet quality Parameter determination causes the cause and equipment of abnormal video quality at the video playback end, including: for any video frame, when the frame quality parameter of the video frame and the packet quality parameter of the data packet corresponding to the video frame meet the first condition, determine The cause of the abnormal video quality at the video playback end is a video encoding problem, and it is determined that the device causing the abnormal video quality at the video playback end is a device used for video encoding; wherein, the first condition includes: the frame loss parameter of the video frame is abnormal And the packet loss parameter of the data packet corresponding to the video frame is normal, or the frame disorder parameter of the video frame is abnormal and the disorder parameter of the data packet corresponding to the video frame is normal.
  • the frame quality parameter and the packet quality parameter can reflect the abnormal situation in the transmission process, and the frame packet loss parameter abnormality indicates that there is a data packet under the video frame being transmitted If the packet loss parameter of the data packet under the video frame is normal at this time, it means that no data packet is lost during transmission, so it can be determined that the data packet is lost before transmission, that is, during video encoding; frame disorder Abnormal parameters indicate that there are data packets under the video frame that are out of order during transmission.
  • the out-of-order parameters of the data packets under the video frame are normal at this time, it means that the data packets are not out of order during transmission, so it can be determined that the data packets are in the out-of-order state.
  • the first condition is that the frame packet loss parameter of the video frame is abnormal and the packet loss parameter of the data packet corresponding to the video frame is normal, or the video frame
  • the out-of-sequence parameter of the frame is abnormal and the out-of-order parameter of the data packet corresponding to the video frame is normal, which can accurately reflect the video encoding problem.
  • the abnormal packet quality parameter determines the cause and equipment that cause the abnormal video quality of the video player, including: for any video frame, the frame quality parameter of the video frame and the packet quality parameter of the data packet corresponding to the video frame meet the second condition, determine that the cause of the abnormal video quality at the video player end is a terminal parameter problem, and determine that the device causing the abnormal video quality at the video player end is the device used for video playback; wherein, the second condition includes: the frame of the video frame Any parameter in the first packet jitter parameter and the frame tail packet jitter parameter is abnormal, and any parameter in the IP packet fragmentation parameter and the checksum error parameter of the data packet corresponding to the video frame is abnormal.
  • the frame quality parameter and the packet quality parameter can reflect the abnormal situation in the transmission process, and the abnormality of the first frame packet jitter parameter/frame end packet jitter parameter indicates that the video The delay jitter of the first/last data packet in the frame is too large during transmission. If the IP packet fragmentation parameter of the data packet in the video frame is abnormal at this time, it means that the data packet has experienced fragmentation during transmission.
  • the parameter configuration of the video player fails to meet the requirements of data packet transmission without fragmentation; if the checksum error parameter of the data packet under the video frame is abnormal at this time, it means that the data packet failed to The verification with the video playback end is successfully completed, so it can be determined that the parameter configuration of the video playback end fails to meet the requirements for successful verification with the data packet; based on the above reasons, the second condition, that is, the frame header jitter parameters of the video frame and the frame Any parameter in the tail packet jitter parameter is abnormal and any parameter in the IP packet fragmentation parameter and the checksum error parameter of the data packet corresponding to the video frame is abnormal, which can accurately reflect the terminal parameter problem.
  • the The frame quality parameter of the video frame and/or the packet quality parameter of the abnormality determine the cause and equipment that cause the abnormal video quality of the video player, including: for any video frame, the frame quality parameter of the video frame and the corresponding video frame
  • the third condition includes: the frame packet loss parameter of the video frame is abnormal and the video frame
  • the packet loss parameter of the data packet corresponding to the frame is abnormal, or the frame out-of-order parameter of the video frame is abnormal and the out-of-order parameter of the data packet corresponding to the video frame is abnormal, or the frame header jitter parameter of the video frame and the frame Any parameter in the last packet jitter parameter is abnormal and any parameter in the packet jitter parameter, packet interval parameter, retrans
  • the frame quality parameter and the packet quality parameter can reflect the abnormal situation in the transmission process, and the frame packet loss parameter abnormality indicates that there is a data packet under the video frame being transmitted If the packet loss parameter of the data packet under the video frame is abnormal at this time, it means that there is a data packet lost during transmission, so it can be determined that there is a problem with the network transmission; the abnormal frame disorder parameter indicates that there is a data packet under the video frame It is in an out-of-order state during transmission.
  • the out-of-order parameters of the data packets under the video frame are abnormal at this time, it means that some data packets are in out-of-order during transmission, so it can be determined that there is a problem in network transmission; based on the above reasons, in the third condition "The frame packet loss parameter of the video frame is abnormal and the packet loss parameter of the data packet corresponding to the video frame is abnormal, or the frame out-of-order parameter of the video frame is abnormal and the out-of-order data packet corresponding to the video frame Abnormal parameter", which can accurately reflect the network transmission problem; the abnormality of the frame first packet jitter parameter/frame end packet jitter parameter indicates that the delay jitter of the first/last data packet under the video frame is too large during transmission.
  • the packet interval parameter of the data packet under the frame is abnormal, it means that the time difference between the two data packets in the transmission process is too large, and it can be determined that there is a problem in the network transmission; if the packet jitter parameter of the data packet under the video frame is abnormal at this time, it means that the data packet If the delay and jitter in the transmission process is too large, it can be determined that there is a problem in network transmission; if the retransmission parameters of the data packet under the video frame are abnormal at this time, it means that the data packet has undergone multiple transmissions during the transmission process, and it can be determined that there is a network transmission problem.
  • the delay parameter of the data packet under the video frame is abnormal at this time, it means that the transmission time of the data packet is too long during the transmission process, and it can be determined that there is a problem with the network transmission.
  • the third condition "any parameter in the frame header jitter parameter and the frame tail packet jitter parameter of the video frame is abnormal and the packet jitter parameter, the packet interval parameter, and the packet interval parameter of the data packet corresponding to the video frame are Any one of the retransmission parameters and delay parameters is abnormal" can accurately reflect the terminal parameter problem.
  • the The abnormal frame quality parameter and/or the abnormal packet quality parameter determine the cause and equipment that cause the abnormal video quality of the video player, and also include: according to the network layer protocol information of the data packet, determine the abnormal packet quality parameter The network layer sequence number of the corresponding data packet; according to the transport layer protocol information of the data packet, determine the packet time, transport layer sequence number, and checksum of the data packet corresponding to the abnormal packet quality parameter; receive the network layer on the network device Topology data, determine the transmission path of each data packet in the network according to the network topology data; according to the transmission path of the data packet corresponding to the abnormal packet quality parameter described in the network, find out the packet time, The data packet whose network layer serial number, transport layer serial number, and checksum all meet the conditions; according to the found packet quality parameters of the data packet that meets the conditions, determine from each network
  • the packet quality parameter of the data packet on each device can indicate the impact of the device on the transmission of the data packet, and by finding the corresponding data packets in each network device on the transmission path, the statistics can be calculated using the found data packets In the process of packet transmission, the impact of different network devices on the transmission of data packets, and determine the specific device that causes the abnormal video quality of the video player.
  • the packet time, network layer sequence number, transport layer sequence number, checksum The conditions to be met include: the absolute value of the difference between the packet time and the packet time of the data packet corresponding to the abnormal packet quality parameter is less than a preset threshold, and the network layer sequence number, the transport layer sequence number, the checksum and the The network layer sequence number, transport layer sequence number, and checksum of the data packets corresponding to the abnormal packet quality parameters are all the same.
  • the same data packet on different network devices along the transmission path can be found.
  • the cause of the abnormality of the packet quality parameters on different devices is the current device or the upstream device of the current device.
  • the cause of the abnormal video quality at the video playback end is a network transmission problem, and the specific network device that causes the abnormal video quality at the video playback end.
  • the twelfth possible implementation of the method for determining video quality problems how many data packets correspond to the abnormal packet quality parameter
  • the packet quality parameters of the data packets that meet the conditions found from each network device on the transmission path of the data packets, determine the network that causes the abnormal video quality of the video playback end.
  • Equipment including: according to the packet quality parameters of multiple sets of data packets that meet the conditions, from each network device on the transmission path of the data packet, repeatedly determine the network equipment that may cause abnormal video quality at the video playback end; count the multiple determinations Among the network devices that may cause abnormal video quality at the video playback end, the network device that appears most frequently is the network device that causes abnormal video quality at the video playback end.
  • the accuracy of the determined network device that causes abnormal video quality can be improved, so that the determined network that causes abnormal video quality After the device implements the corresponding solution, the probability of abnormal video quality caused by the same problem on the same device is reduced.
  • an embodiment of the present application provides an apparatus for determining a video quality problem, the apparatus including: an information receiving module, configured to receive protocol information on a video player and a network device, and the network device is used to convert the video The data is transmitted to the video player, and the protocol information indicates the sending mode of the video frame, and the video frame includes a plurality of data packets; the parameter determination module is used to determine abnormal frame quality parameters and abnormal packet quality according to the protocol information Parameters, the abnormal frame quality parameter represents the abnormal quality of the video frame to which the abnormal frame quality parameter belongs, and the abnormal packet quality parameter represents the abnormal quality of the data packet to which the abnormal packet quality parameter belongs; abnormal reason The determining module is configured to determine the cause and equipment for abnormal video quality at the video playback terminal according to the abnormal frame quality parameter and/or the abnormal packet quality parameter.
  • the protocol information includes streaming media layer protocol information, network layer protocol information, and transport layer protocol information of data packets
  • the stream The media layer protocol information includes the streaming media layer protocol information of the data packet on the video playback end
  • the network layer protocol information includes the network layer protocol information of the data packet on the network device and the video playback end
  • the transport layer protocol information includes The transport layer protocol information of the data packet on the network device and on the video playback end
  • said determining abnormal frame quality parameters and abnormal packet quality parameters according to the protocol information includes: determining at least one according to the streaming media layer protocol information The video frame and each data packet corresponding to each frame in the at least one video frame; according to the streaming media layer protocol information and the network layer protocol information, determine the frame quality parameters of each frame in the at least one video frame; according to The network layer protocol information and the transport layer protocol information determine the packet quality parameters of each data packet corresponding to each frame in the at least one video frame; according to the frame quality parameters corresponding to each frame
  • the second possible implementation manner of the apparatus for determining video quality problems according to the frame quality parameters corresponding to each frame in the at least one video frame and the frame A quality parameter threshold, determining the abnormal frame quality parameter, determining the abnormal packet quality parameter according to the packet quality parameter and the packet quality parameter threshold of each data packet corresponding to each frame in the at least one video frame, including: For each frame of the at least one video frame, among the frame quality parameters of the video frame, the frame quality parameter whose value is greater than the frame quality parameter threshold is determined as the abnormal frame quality parameter; for the at least one video frame For each frame, among the packet quality parameters of the data packets corresponding to the video frame, the packet quality parameter whose value is greater than the packet quality parameter threshold is determined as the abnormal packet quality parameter.
  • the streaming media layer protocol information of the data packet includes The frame time stamp of the video frame
  • the network layer protocol information of the data packet includes the packet time indicating the time of the data packet
  • the frame quality parameter includes a frame header jitter parameter and a frame tail packet jitter parameter
  • the frame header packet The jitter parameter represents the delay jitter situation of the first data packet corresponding to the video frame
  • the frame tail packet jitter parameter represents the delay jitter situation of the last data packet corresponding to the video frame
  • determining the frame quality parameters of each frame in the at least one video frame includes: for each frame of the at least one video frame, determining the frame quality parameter according to the packet time of the data packet corresponding to the video frame
  • the first packet arrival time and the last packet arrival time of the video frame, the first packet arrival time represents the arrival time of the first data packet corresponding to the video frame
  • the stream of data packets includes a stream media layer sequence number indicating the sequence of the data packet in the video frame to which it belongs, the network layer protocol information of the data packet includes a packet time indicating the time of the data packet, and the frame quality parameter includes a frame loss Packet parameter and frame out-of-sequence parameter, described frame packet loss parameter represents the loss situation of the data packet of video frame, and described frame out-of-sequence parameter represents the out-of-sequence situation of the data packet of video frame, and described according to described streaming media layer protocol Information and the network layer protocol information, determining the frame quality parameters of each frame in the at least one video frame includes: for each frame of the at least one video frame, according to the streaming media layer of the data packet corresponding to the video frame The serial number determines the frame packet loss parameter of the video frame; for each frame of the at least one video frame, sorts the
  • determining the frame loss parameter of the video frame includes: for each frame of the at least one video frame, according to the streaming media layer sequence number of the data packet corresponding to the video frame, determining The maximum value of the streaming media layer sequence number of the data packet corresponding to this video frame, the minimum value and the number of streaming media layer sequence numbers, according to the maximum value and minimum value of the streaming media layer sequence number, determine the corresponding data of this video frame
  • the theoretical value of the number of packets according to the difference between the theoretical value of the number of data packets corresponding to the video frame and the number of streaming media layer serial numbers of the data packets corresponding to the video frame, determine the frame packet loss parameter of the video frame ; According to the sequence number of the streaming media layer of the data packet corresponding to the sorting result and the video frame, determine the frame disorder parameter of the video frame
  • the at least one video frame The packet quality parameters of each data packet corresponding to each frame in the frame are determined according to the network layer protocol information and the transport layer protocol information of the data packet, and the packet quality parameters include: the packet representing the time interval between the current data packet and the previous data packet The interval parameter, the packet jitter parameter indicating the delay jitter between the current data packet and the previous data packet, the out-of-order parameter indicating the disorder of the data packet, the packet loss parameter indicating the packet loss, and the packet retransmission Retransmission parameters, delay parameters indicating the delay of data packets and corresponding response messages, IP packet fragmentation parameters indicating whether there is IP packet fragmentation in data packets, indicating whether there is an error in the checksum of data packets One or more of the checksum error parameters.
  • the abnormal frame quality parameter and/or the abnormal packet quality Parameter determination causes the cause and equipment of abnormal video quality at the video playback end, including: for any video frame, when the frame quality parameter of the video frame and the packet quality parameter of the data packet corresponding to the video frame meet the first condition, determine The cause of the abnormal video quality at the video playback end is a video encoding problem, and it is determined that the device causing the abnormal video quality at the video playback end is a device used for video encoding; wherein, the first condition includes: the frame loss parameter of the video frame is abnormal And the packet loss parameter of the data packet corresponding to the video frame is normal, or the frame disorder parameter of the video frame is abnormal and the disorder parameter of the data packet corresponding to the video frame is normal.
  • the abnormal packet quality parameter determines the cause and equipment that cause the abnormal video quality of the video player, including: for any video frame, the frame quality parameter of the video frame and the packet quality parameter of the data packet corresponding to the video frame meet the second condition, determine that the cause of the abnormal video quality at the video player end is a terminal parameter problem, and determine that the device causing the abnormal video quality at the video player end is the device used for video playback; wherein, the second condition includes: the frame of the video frame Any parameter in the first packet jitter parameter and the frame tail packet jitter parameter is abnormal, and any parameter in the IP packet fragmentation parameter and the checksum error parameter of the data packet corresponding to the video frame is abnormal.
  • the The frame quality parameter of the video frame and/or the packet quality parameter of the abnormality determine the cause and equipment that cause the abnormal video quality of the video player, including: for any video frame, the frame quality parameter of the video frame and the corresponding video frame
  • the third condition includes: the frame packet loss parameter of the video frame is abnormal and the video frame
  • the packet loss parameter of the data packet corresponding to the frame is abnormal, or the frame out-of-order parameter of the video frame is abnormal and the out-of-order parameter of the data packet corresponding to the video frame is abnormal, or the frame header jitter parameter of the video frame and the frame Any parameter in the last packet jitter parameter is abnormal and any parameter in the packet jitter parameter, packet interval parameter, re
  • the The abnormal frame quality parameter and/or the abnormal packet quality parameter determine the cause and equipment that cause the abnormal video quality of the video player, and also include: according to the network layer protocol information of the data packet, determine the abnormal packet quality parameter The network layer sequence number of the corresponding data packet; according to the transport layer protocol information of the data packet, determine the packet time, transport layer sequence number, and checksum of the data packet corresponding to the abnormal packet quality parameter; receive the network layer on the network device Topology data, determine the transmission path of each data packet in the network according to the network topology data; according to the transmission path of the data packet corresponding to the abnormal packet quality parameter described in the network, find out the packet time, The data packet whose network layer serial number, transport layer serial number, and checksum all meet the conditions; according to the found packet quality parameters of the data packet that meets the conditions, determine
  • the packet time, network layer sequence number, transport layer sequence number, checksum The conditions to be met include: the absolute value of the difference between the packet time and the packet time of the data packet corresponding to the abnormal packet quality parameter is less than a preset threshold, and the network layer sequence number, the transport layer sequence number, the checksum and the The network layer sequence number, transport layer sequence number, and checksum of the data packets corresponding to the abnormal packet quality parameters are all the same.
  • the apparatus for determining video quality problems how many data packets correspond to the abnormal packet quality parameter
  • the packet quality parameters of the data packets that meet the conditions found from each network device on the transmission path of the data packets, determine the network that causes the abnormal video quality of the video playback end.
  • Equipment including: according to the packet quality parameters of multiple sets of data packets that meet the conditions, from each network device on the transmission path of the data packet, repeatedly determine the network equipment that may cause abnormal video quality at the video playback end; count the multiple determinations Among the network devices that may cause abnormal video quality at the video playback end, the network device that appears most frequently is the network device that causes abnormal video quality at the video playback end.
  • an embodiment of the present application provides an apparatus for determining a video quality problem, including: a processor; a memory for storing processor-executable instructions; wherein, the processor is configured to execute the above-mentioned first aspect or the first aspect A method for determining a video quality problem in one or several possible implementation manners of one aspect.
  • the embodiments of the present application provide a non-volatile computer-readable storage medium, on which computer program instructions are stored, and when the computer program instructions are executed by a processor, the above-mentioned first aspect or multiple aspects of the first aspect are implemented.
  • the embodiments of the present application provide a computer program product, including computer readable code, or a non-volatile computer readable storage medium bearing computer readable code, when the computer readable code is stored in an electronic
  • the processor in the electronic device executes the method for determining a video quality problem in the first aspect or one or more of the multiple possible implementations of the first aspect.
  • Fig. 1 shows an exemplary application scenario of a method for determining a video quality problem according to an embodiment of the present application.
  • FIG. 2 shows an example of identifying data stream types in this embodiment of the present application.
  • Fig. 3 shows an exemplary method schematic diagram of a method for determining a video quality problem according to an embodiment of the present application.
  • Fig. 4 shows a schematic diagram of an exemplary method for determining an abnormal frame quality parameter and/or an abnormal packet quality parameter according to an embodiment of the present application.
  • Fig. 5 shows an example of a method for determining a video frame and a data packet corresponding to the video frame according to an embodiment of the present application.
  • FIG. 6 shows an example of a calculation method of a packet jitter parameter at the beginning of a frame and a packet jitter parameter at the end of a frame according to an embodiment of the present application.
  • FIG. 7 shows an example of a calculation manner of a frame packet loss parameter according to an embodiment of the present application.
  • FIG. 8 shows an example of a calculation manner of a frame out-of-sequence parameter according to an embodiment of the present application.
  • FIG. 9 shows an example of a normal distribution probability curve generated according to an embodiment of the present application.
  • Fig. 10 shows an example of realizing abnormal condition determination of parameters according to an embodiment of the present application.
  • Fig. 11 shows an exemplary schematic diagram of the correspondence between abnormal frame quality parameters, abnormal packet quality parameters, causes of abnormal video quality at the video player end, and devices according to an embodiment of the present application.
  • FIG. 12 shows an example of determining a specific network device that causes abnormal video quality at a video player end according to an embodiment of the present application.
  • Fig. 13 shows an exemplary structural diagram of an apparatus for determining a video quality problem according to an embodiment of the present application.
  • Fig. 14 shows an exemplary structural diagram of an apparatus for determining a video quality problem according to an embodiment of the present application.
  • user experience data and asset management data are obtained from user devices (ie, video playback devices), and network topology data is obtained from network devices through data collection methods reported by devices.
  • the user experience data includes the average video mean opinion score (vMOS), freeze duration, freeze frequency, and video quality switching times (such as switching between 1080P, 720P, and 360P video display formats). times), etc.
  • the user experience data is used to represent the quality of the user's experience and perception when watching a video, and the video playback quality can be determined according to the user experience data.
  • Asset management data includes users' network access information. Asset management data is used to represent the connection relationship between user equipment and network equipment.
  • Network topology data is used to represent the connection relationship between network devices. According to the asset management data and network topology data, the total number of users connected to each network device can be determined.
  • the percentage of the number of users with poor video playback quality in the total number of users on the network device may be determined as the poor quality rate of the network device.
  • the poor quality rate of each network device is counted, and the distribution of the poor quality rate of network devices in the network can be determined according to the connection relationship between network devices.
  • Quality of experience is determined according to the determined poor quality rate of each network device and a preset threshold.
  • the quality of experience includes calculating the coefficient of variation of the poor quality rate, the average value of the poor quality rate, etc., and determining the problem equipment through the quality of experience, such as determining the degree of skewed distribution between the coefficient of variation of the poor quality rate and the preset threshold or the mean value of the poor quality rate. The degree of skewed distribution with the preset threshold, and find the equipment that causes a large deviation through the degree of skewed distribution.
  • the network equipment determines whether the network equipment has a lower-level network equipment, if not, then the network equipment is the network equipment that causes the video quality problem determined by the solution of the prior art; if so, through the lower-level network equipment Quality of Experience further identifies the cause and device causing video quality issues.
  • the statistical distribution scheme using big data proposed in the prior art still has certain shortcomings.
  • the solution determines the cause of the video quality problem and the time cost of the equipment is high, and the real-time performance of the solution is poor.
  • this solution requires an overall analysis of the data collected when video quality problems occur multiple times. That is to say, the causes and equipment that lead to video quality problems determined by the existing technology are also determined for multiple video quality problems. , while multiple video quality issues may have different causes and devices.
  • the cause and equipment of the video quality problem determined by the prior art may be only one of the causes and equipment of the video quality problem many times, and the determined cause and equipment of the video quality problem are not accurate enough. Optimizing the device based on the identified cause of the video quality issue and optimizing the device will not reduce the chances of some of the same video quality issues recurring in the future.
  • this application provides a video quality problem determination method, device, storage medium, and program product.
  • the video quality problem determination method in the embodiment of the application can quickly and accurately determine the cause and effect of the video quality problem. equipment.
  • Fig. 1 shows an exemplary application scenario of a method for determining a video quality problem according to an embodiment of the present application.
  • the scene may be, for example, a live video scene, which may include a video provider for providing video data, a plurality of network devices for transmitting video data, and a video server for displaying video images to users. player side.
  • the video provider may be a camera capable of collecting video data, or a device capable of receiving, storing and outputting video data.
  • the video provider can encode the collected or received video data, and encapsulate it into a live streaming form according to the streaming media layer protocol.
  • the encapsulated video data may be transmitted through network equipment, and the network equipment may include, for example, the wireless base station, bearer network equipment, core network equipment, switch, etc. in FIG. 1 .
  • the wireless base station may be, for example, a 5G base station (next generation NB, gNB).
  • the core network device may be, for example, a unified distributed gateway (unified distributed gateway, UDG), etc.
  • the switch may, for example, be a switch (end of row, EoR) using a certain deployment method.
  • the video data is finally output to a video player, which can be, for example, a mobile phone, a tablet computer, a notebook, and the like.
  • the video data in the form of live streaming can be decapsulated, decoded, and rendered, so that users can watch the video through the video player end.
  • the application scenario in FIG. 1 also includes a terminal device (or server) for determining the cause of the video quality problem and the device, and the terminal device (or server) may be different from the above-mentioned video provider, network device, and video player.
  • the terminal device (or server) may include the apparatus for determining a video quality problem in the embodiment of the present application, which can be used to execute the method for determining a video quality problem in the embodiment of the present application.
  • the terminal device may also include a protocol information collection device for collecting protocol information from the network device and the video player, and a topology data collection device for collecting network topology data from the network device.
  • a protocol information collection device for collecting protocol information from the network device and the video player
  • a topology data collection device for collecting network topology data from the network device.
  • the device for determining video quality problems, the device for collecting protocol information, and the device for collecting topology data can also be respectively set on a single terminal device or server, or any two of the three can be set together on the same terminal device or server.
  • the application scenarios of the embodiments of the present application are not limited to the foregoing devices.
  • the protocol information collection device and the topology data collection device may collect protocol information and network topology data respectively.
  • the protocol information collection device can continuously collect the protocol information on the video playback terminal and network equipment. If there is no change in the connection relationship of the network equipment during this process, the topology data collection device only needs to collect the network topology data once.
  • using the method for determining the video quality problem in the embodiment of the present application can determine the cause of the abnormal video quality of the video player and the device.
  • the device for collecting protocol information firstly obtains data streams on the video playback terminal and network equipment.
  • the data streams on the video playback terminal and the network device may include video streams (video), and may also include other types of data streams such as voice streams.
  • the protocol information collection device can identify the collected data stream in combination with the streaming media protocol, and obtain data packets of the video stream type from the data stream. After determining the data whose type is video stream, the protocol information acquisition device can further obtain the streaming media layer protocol information and network layer protocol information of the data packet from the data whose type is video stream according to the streaming media layer protocol, network layer protocol, transport layer protocol, etc. Information, transport layer protocol information, so as to complete the collection of protocol information.
  • the collection of protocol information can be realized by deploying probes, or through network port packet capture through computers and other equipment, or through functions such as user tracking that come with network equipment.
  • This section uses probes to collect protocol information as an example.
  • the probe can have deep packet inspection (deep packet inspection, DPI) capability, and can automatically realize the identification of different types of data flows and the acquisition of protocol information.
  • DPI deep packet inspection
  • a data packet may include a payload header (packet header) and a payload, wherein the payload part includes the specific content of the data.
  • the payload header includes protocol information indicating the transmission mode of the data packet. In order to ensure data security, when obtaining the protocol information, the payload of the data packet can be encrypted, or only the payload header (packet header) of the data packet can be detected.
  • the streaming media layer payload header, network layer payload header, and transport layer payload header can be detected and the corresponding streaming media layer protocol information, network layer protocol information, and transport layer protocol information can be extracted
  • the network layer payload header and the transport layer payload header can be detected and the corresponding network layer protocol information and transport layer protocol information can be extracted.
  • protocol information extraction can be realized by identifying the application type and protocol type.
  • the application type of a certain data flow in the data can be identified by using the service awareness (SA) technology. For example, if a user experiences abnormal video quality when using an application to watch a live video at a certain time period, the user can watch the live video from the user according to the user identity information, the time information of the data packet, and/or the type of application, etc. Obtain data streams related to video quality exceptions that need to be analyzed from the data streams on the video playback end. By default, it is obtained from all data streams on the video player.
  • SA service awareness
  • the judgment method specified in the existing protocol can be used to identify the data stream type layer by layer from the network layer, transport layer, streaming media layer and application layer, and determine the video stream existing in the data stream Furthermore, specific types of video streams are distinguished to determine the type of video streams containing video frames.
  • FIG. 2 shows an example of identifying data stream types in this embodiment of the present application.
  • the protocol used by the network layer can include Internet Protocol (Internet Protocol, IP), and the protocol used by the transport layer can include Transmission Control Protocol (transmission control protocol, TCP) and user datagram protocol (user datagram). protocol, UDP).
  • the protocols used by the streaming media layer may include real-time streaming protocol (real time streaming protocol, RTSP), real-time messaging protocol (real time messaging protocol, RTMP), real-time transport protocol (real-time transport protocol, RTP), real-time transmission control protocol (real-time transport control protocol, RTCP), user datagram protocol-based data transfer protocol (UDP-based data transfer protocol, UDT) or secure reliable transport protocol (secure reliable transport, SRT), and MPEG2 transport stream (MPEG -2 transport stream, MPEG2-TS).
  • the protocol used by the application layer may include the H.264 video standard protocol or the H.265 video standard protocol.
  • the supported video stream types can include: TCP+RTMP video stream, TCP+RTSP+RTP video stream, UDP+ RTP video stream, UDP+RTP+MPEG2-TS video stream, UDP+SRT/UDT+MPEG2-TS video stream, and UDP+MPEG2-TS video stream.
  • the above protocols can also support control streams, such as TCP+RTMP control streams, TCP+RTSP control streams, UDP+RTCP control streams, etc. in FIG. 2 , and the control streams do not contain video frames.
  • the protocol information can be obtained from the data packets under the video stream.
  • the protocol information may include the network layer protocol information, the transport layer protocol information, the streaming media layer protocol information of the data packets in the video stream, and the calculated transport layer checksum obtained according to the standard calculation method of checksum (CHECKSUM). and (hereinafter referred to as checksum).
  • network topology data can be collected from network devices.
  • the network topology data may include the topology connection relationship or service path of each network device in the network where the network device connected to the video playback terminal is located.
  • the service path represents the connection relationship between network devices that the service flow passes through. It makes it possible to determine the upstream and downstream relationship of network devices according to the network topology data.
  • the protocol information collecting device and the topology data collecting device can respectively transmit the collected information or data to the video quality problem determining device.
  • the device for determining a video quality problem uses the method for determining a video quality problem according to an embodiment of the present application. Protocol information and network topology data can be used to determine the cause and equipment that cause abnormal video quality at the video playback end.
  • Fig. 3 shows an exemplary method schematic diagram of a method for determining a video quality problem according to an embodiment of the present application.
  • the method for determining a video quality problem includes:
  • the network device is used to transmit video data to the video player, the protocol information indicates a sending method of a video frame, and the video frame includes a plurality of data packets;
  • the abnormal frame quality parameter indicates that the quality of the video frame to which the abnormal frame quality parameter belongs is abnormal
  • the abnormal packet quality parameter Indicating that the quality of the data packet to which the abnormal packet quality parameter belongs is abnormal
  • the video frame and the content included in the video frame can be determined according to the protocol information.
  • the transmission mode of multiple data packets can further determine abnormal frame quality parameters and abnormal packet quality parameters.
  • the abnormal frame quality parameter indicates that the quality of the video frame to which it belongs is abnormal
  • the abnormal packet quality parameter indicates that the quality of the data packet to which it belongs is abnormal
  • the video quality is associated with the video frame and the data packet
  • the abnormal frame quality parameter and the abnormal The packet quality parameter is associated with the abnormal video quality, so that according to the abnormal frame quality parameter and the abnormal packet quality parameter, the cause and equipment that cause the abnormal video quality of the video player can be determined.
  • the video quality problem determination method of the present application jointly determines the cause and equipment that cause the video quality abnormality at the video playback end through abnormal frame quality parameters at the frame level and abnormal packet quality parameters at the packet level, which can reduce the cost of completing the determination of the cause and equipment once. Time cost, and improve the accuracy of determining the cause and equipment.
  • the protocol information can also be extracted according to the payload header, and the cause and device that cause the video quality abnormality at the video playback end can be accurately determined. Expand the applicability of methods for determining video quality issues.
  • step S1 protocol information can be obtained.
  • step S2 the recognition of the video frame and the extraction of the data packets under the video frame can be realized.
  • frame quality parameters can be defined to characterize the quality of video frames.
  • Different frame quality parameters can be preset with different calculation methods. Through the preset calculation method of frame quality parameters, the value of frame quality parameters can be calculated according to the protocol information, and the calculated value of frame quality parameters can be obtained through the frame quality parameter threshold value Anomaly detection is performed to determine frame quality parameters with abnormal values, that is, abnormal frame quality parameters.
  • Packet quality parameters can be defined to characterize the quality of data packets.
  • packet quality parameters such as packet interval parameters, packet jitter parameters, packet loss parameters, out-of-sequence parameters, retransmission parameters, delay parameters, IP packet fragmentation parameters, and checksum errors mentioned below parameter.
  • Different packet quality parameters can be preset with different calculation methods. Through the preset calculation method of packet quality parameters, the value of the packet quality parameter can be calculated according to the protocol information, and the value of the packet quality parameter calculated by the packet quality parameter threshold value Anomaly detection is performed to determine packet quality parameters with abnormal values, that is, abnormal packet quality parameters.
  • step S3 according to whether the abnormal frame quality parameters of the abnormal video frame and the abnormal packet quality parameters of the data packets under the abnormal video frame meet the corresponding conditions, determine the cause and equipment causing the abnormal video quality.
  • the only device that can be determined to cause abnormal video quality is the device used for video playback, that is, the video playback terminal.
  • the video quality problem determination method has been realized to determine the cause and equipment that lead to abnormal video quality. If the device that causes the abnormal video quality cannot be uniquely determined based on the determined cause of the abnormal video quality, for example, it is determined that the cause of the abnormal video quality is a network transmission problem. Since multiple network devices may participate in the network transmission process, it cannot be uniquely determined. Devices that cause abnormal video quality.
  • This application can determine the abnormal data packets according to the abnormal packet quality parameters, and according to the protocol information of the abnormal data packets, find multiple data packets whose protocol information satisfies the conditions on multiple network devices participating in the network transmission process, and use the found multiple The packet quality parameter of the data packet is analyzed to determine the device causing the abnormal video quality.
  • the frame quality parameter threshold and packet quality parameter threshold used may be determined according to the AI model.
  • the AI model can be trained by using the frame quality parameters and packet quality parameters calculated for a period of time before determining the current cause of abnormal video quality at the video playback end and the device as samples, which can solve common samples that need to be marked and abnormal samples
  • the modeled data inaccessibility problem reduces the complexity of determining abnormal frame quality parameters and abnormal packet quality parameters.
  • the abnormal frame quality parameter may be an abnormal parameter among the frame quality parameters
  • the abnormal packet quality parameter may be an abnormal parameter among the packet quality parameters.
  • the frame quality parameter and the packet quality parameter may be determined first according to the protocol information, and then the abnormal frame quality parameter and the abnormal packet quality parameter are determined according to the frame quality parameter and the packet quality parameter.
  • Fig. 4 shows a schematic diagram of an exemplary method for determining an abnormal frame quality parameter and/or an abnormal packet quality parameter according to an embodiment of the present application.
  • the protocol information includes streaming media layer protocol information, network layer protocol information and transport layer protocol information of the data packet
  • the streaming media layer protocol information includes the video player
  • the streaming media layer protocol information of the data packet on the network layer protocol information includes the network layer protocol information of the data packet on the network device and the video player end
  • the transport layer protocol information includes the network layer protocol information on the network device and the video player end
  • the transport layer protocol information of the data packet, step S2 includes:
  • the frame quality parameters may include frame header jitter parameters, frame tail packet jitter parameters, Frame loss parameters, frame disorder parameters.
  • the packet quality parameters may include packet interval parameters, packet jitter parameters, Packet loss parameters, out-of-sequence parameters, retransmission parameters, delay parameters, IP packet fragmentation parameters, and checksum error parameters.
  • the frame quality parameter and the packet quality parameter can be determined through the combination of different protocol information, so as to reflect the quality of the video frame and the quality of the data packets under the video frame. Whether the frame quality parameter and the packet quality parameter are abnormal can be determined by the frame quality parameter threshold and the packet quality parameter threshold respectively, making it more convenient to determine the abnormal frame quality parameter and the abnormal packet quality parameter.
  • Step S21 can be implemented based on existing technologies. For example, in step S21, for the acquired protocol information, video frame identification and data packets corresponding to the video frame are extracted according to different types of video streams (see above FIG. 2 and related descriptions). A common method is to find a data packet with a streaming media type in the video stream according to the streaming media layer protocol information of the data packet (see step S211 below for an example).
  • Extract the information of the streaming media type from the streaming media layer protocol information of the data packet with the streaming media type found (see step S212 below for an example), and obtain in the streaming media layer protocol information of the data packet of the video streaming media type
  • the serial number of the streaming media layer and then according to the serial number of the streaming media layer, the data packets used to calculate and obtain the packet quality parameter and the frame time stamps of these data packets are found.
  • Count the frame time stamps of all data packets used to calculate and obtain the packet quality parameter, and use the packets with the same frame time stamp as the data packets belonging to the same video frame (see step S213 below for an example).
  • each frame in at least one video frame may include data packets with the same frame time stamp, and the frame time stamps of the data packets in different video frames may be different, so that each video frame and each video frame corresponding to each frame may be determined. data pack.
  • the streaming media type can be passed in the protocol information
  • the value under the Stream type under the TS header is determined.
  • Streaming media layer sequence number can be determined by the value of Stream PID when the value under Stream type in the protocol information is Video (video); for the video stream (TCP+RTSP+TCP+RTSP+ RTP video stream, UDP+RTP video stream), the streaming media type can be determined by the value under Payload type in the protocol information, and the serial number of the streaming media layer can be determined by the value of RTP SEQUENCE NUMBER when the value under Payload type is Video (video) Determine; for video streams carrying RTMP (TCP+RTMP video streams) other than video streams carrying MPEG2-TS and video streams carrying RTP, the streaming media type can be determined by the value under Type ID in the protocol information, streaming media The layer sequence number can be determined by the protocol information in the data packet whose value under Type ID is Video (video).
  • Fig. 5 shows an example of a method for determining a video frame and a data packet corresponding to the video frame according to an embodiment of the present application.
  • step S21 As shown in Figure 5, taking the video stream whose type is UDP+RTP+MPEG2-TS as an example, the exemplary process of step S21 is as follows:
  • the TS PID field value of TS 1# in FIG. 5 is 0, indicating that the data type of the TS transmission segment to which the TS header belongs is a program association table (program association table, PAT).
  • PAT program association table
  • the Program Map PID field value of TS 1# in Figure 5 is 0x0010f.
  • all TS transmission segments including the obtained Program Map PID value (0x0010f), such as the TS transmission segment corresponding to TS 8# in Figure 5, are data packets with streaming media type.
  • the method in the above step S211 is executed to determine all data packets with the streaming media type.
  • S212 From the TS header of the determined data packet with the streaming media type, first determine the streaming media type according to the Stream type in the program mapping table. For example, it can be known from the description of S211 that TS 8# in Fig. 5 is the TS header of the data packet with streaming media type.
  • the Stream type of the program mapping table (Program Map Table, PMT) is Video, indicating that the streaming media type of the packet to which TS 8# belongs is video.
  • PMT Program Map Table
  • the value of the Stream PID field of PMT is 0x0100, which is the serial number of the streaming media layer of the data packet to which TS 8# belongs.
  • the protocol information includes the data packet with the obtained Stream PID value (0x0100), that is, the data packet whose streaming media type is video.
  • the data packets corresponding to TS 11#, TS 12#, TS 13#, and TS 14# in Figure 5 are data packets whose streaming media type is video.
  • the method in the above step S212 is executed to find all the data packets with the streaming media type being video.
  • each data packet includes a plurality of TS transmission segments, for example, as shown in Figure 5, the data packets corresponding to TS 11# and TS 12# are combined to obtain a data packet. Combine the data packets corresponding to TS 13# and TS 14# to get a data packet. Wherein, the combination manner of the data packets may be random combination. These combined or encapsulated data packets may be used to calculate and obtain packet quality parameters.
  • the protocol information contains the data of the packetized elementary stream (Packetized Elementary Stream, PES) header, and the decoding time stamp (decode time stamp, DTS) is extracted from the PES header, and the unit of DTS is converted into seconds.
  • the value obtained after unit conversion is the frame time stamp of the data packet, that is, the frame time stamp of the video frame to which the data packet belongs. Since the identifier of the video frame is the frame time stamp of the streaming media layer, it can be considered that the data packets with the same frame time stamp belong to the same video frame.
  • the method of step S213 above is executed to identify different video frames and determine the data packets corresponding to different video frames.
  • the data packet with the streaming media layer can be further determined, and the frame time stamp and the streaming media layer serial number etc. are obtained from the streaming media layer of the data packet, and the The transport layer obtains the transport layer sequence number, checksum, confirmation message sequence number (ACK Number), etc.; obtains the network layer sequence number, fragmentation flag (Don't Fragment, DF) and packet time from the network layer of the data packet etc.
  • the transport layer obtains the transport layer sequence number, checksum, confirmation message sequence number (ACK Number), etc.; obtains the network layer sequence number, fragmentation flag (Don't Fragment, DF) and packet time from the network layer of the data packet etc.
  • At least one video frame may also include a part of all video frames identified in step S21 (frames to which data packets whose streaming media type is video belong). It can be further set to filter out at least one video frame from the identified video frames according to whether the frame timestamp of the video frame is in the same time period, or to obtain at least one video frame from the identified video frame according to the streaming layer sequence number of the data packet in the identified video frame
  • the frame determines at least one video frame, or selects at least one video frame from identified video frames according to a preset sampling mechanism. As long as the streaming media type of the data packet in at least one video frame is video, the present application does not limit the specific manner of determining at least one video frame according to the streaming media layer protocol information.
  • the frame quality parameters of each video frame are obtained according to the streaming media layer protocol information and the network layer protocol information of the data packets corresponding to the different video frames.
  • the frame quality parameters of the video frame may, for example, include one or more of a frame header packet jitter parameter, a frame tail packet jitter parameter, a frame packet loss parameter, and a frame disorder parameter.
  • the cause and equipment that cause the video quality problem can be determined, for example, it is determined that the cause of the video quality problem is a video encoding problem, and at the same time, it is determined The device used for video encoding, or, it is determined that the cause of the video quality problem is a video playback problem, and at the same time, it is determined that the device causing the video quality problem is the device used for video playback, or, it is determined that the cause of the video quality problem is network transmission problem while identifying the device causing the video quality issue as a network device, etc. Exemplary descriptions are given below for the calculation manners of different frame quality parameters.
  • FIG. 6 shows an example of a calculation method of a packet jitter parameter at the beginning of a frame and a packet jitter parameter at the end of a frame according to an embodiment of the present application.
  • the streaming media layer protocol information of the data packet includes a frame time stamp indicating a video frame to which the data packet belongs.
  • the network layer protocol information of the data packet includes a packet time indicating the time of the data packet.
  • the frame quality parameters include frame header jitter parameters and frame tail packet jitter parameters.
  • the first packet jitter parameter of the frame represents the delay jitter situation of the first data packet corresponding to the video frame
  • the frame tail packet jitter parameter represents the delay jitter situation of the last data packet corresponding to the video frame
  • Step S22 including:
  • the first packet arrival time represents the arrival time of the first data packet corresponding to the video frame
  • the tail packet arrival time represents the arrival time of the last data packet corresponding to the video frame
  • the first packet interval and the last packet interval of adjacent video frames are respectively calculated.
  • the first packet interval represents the arrival time difference of the first data packet corresponding to adjacent video frames
  • the last packet interval represents the arrival time difference of the last data packet corresponding to adjacent video frames.
  • the first packet interval, the frame interval, and the first frame packet jitter parameter of the previous video frame calculate the first frame packet jitter parameter of the current video frame
  • the trailing packet interval, the frame interval, and the frame trailing packet jitter parameter of the previous video frame calculate the frame trailing packet jitter parameter of the current video frame.
  • the first frame packet jitter parameter representing the delay jitter of the first data packet corresponding to the video frame and the frame end packet jitter representing the delay jitter of the last data packet corresponding to the video frame can be calculated parameter, while characterizing the quality of the video frame, it improves the closeness between the frame quality parameter of the video frame and the actual jitter of the video frame.
  • the frame header jitter parameter and frame tail packet jitter parameter in the embodiment of the present application can be used to characterize the delay jitter of the data packet corresponding to the video frame. The higher the probability of picture blurring and freezing.
  • the first frame packet jitter parameter and the frame end packet jitter parameter can be calculated in the following manner:
  • At least one video frame may include 4 frames, for each video frame in video frames 1-4, according to the network layer protocol information of the data packet corresponding to the video frame, the For the packet time of time, among the data packets corresponding to the video frame, the data packet with the earliest packet time can be named as the first packet, and the packet time of the first packet is the arrival time of the first packet.
  • a data packet with the latest packet time is named as the tail packet, and the packet time of the tail packet is the arrival time of the tail packet.
  • the first packet and the first packet arrival of each video frame in at least one video frame can be determined Time, Tail Packet, Tail Packet Arrival Time. Based on this, the arrival time of the first packet of video frames 1-4 can be recorded as T1, T2, T3, T4 respectively, and the arrival time of the last packet of video frames 1-4 can be recorded as T1', T2', T3', T4'.
  • the first packet arrival time and the last packet arrival time of each video frame in video frames 1-4 calculate the first packet interval and the last packet interval of adjacent video frames, wherein, when 2 ⁇ n ⁇ 4 and it is an integer, the video frame
  • the first packet interval Deltan of n and video frame n-1 can be calculated by the difference between the first packet arrival time Tn of video frame n and the first packet arrival time Tn-1 of video frame n-1, video frame n and video frame n-
  • the tail packet interval Delta' of 1 can be calculated by the difference between the tail packet arrival time Tn' of video frame n and the tail packet arrival time Tn-1' of video frame n-1.
  • the first packet of video frame 1 The interval Delta1 and the tail packet interval Delta1' are equal to 0 by default.
  • the first packet interval Delta3 between video frame 3 and video frame 2 T3-T2
  • the first packet interval Delta4 between video frame 4 and video frame 3 T4-T3.
  • the tail packet interval Delta1' 0 of video frame 1
  • the tail packet interval Delta2' of video frame 2 and video frame 1 T2'-T1'
  • Delta3' T3'-T2' between video frame 3 and video frame 2
  • Delta4' T4'-T3' between video frame 4 and video frame 3.
  • the frame interval of adjacent video frames can be calculated.
  • the frame timestamps of video frames 1-4 can be recorded as TS1, TS2, TS3, and TS4 respectively.
  • the frame interval between video frame n and video frame n-1 Intervaln can be calculated by the difference between the frame time stamp TSn of video frame n and the frame time stamp TSn-1 of video frame n ⁇ 1.
  • the frame interval Interval1 0 of video 1
  • the frame interval Interval2 TS2-TS1 of video frame 2 and video frame 1
  • the video frame 1-4 can be calculated Frame header packet jitter parameters for each video frame.
  • the frame header jitter parameter of video frame 1 is equal to 0 by default. Except for video frame 1, an example of the calculation method of the frame header jitter parameters of other video frames is shown in formula (1):
  • Jittern Jittern-1*15/16+(Deltan-Intervaln)*1/16 (1)
  • Jittern represents the frame header jitter parameter of the video frame n
  • 15/16 and 1/16 are preset control parameters
  • the values of the control parameters can be different by setting The optimal parameter value determined after comparing the calculated frame first packet jitter parameter (or frame tail packet jitter parameter) with the real frame first packet jitter (or frame tail packet jitter), the control parameter can be used to represent the previous The ratio of the value of the first packet jitter parameter (or frame end packet jitter parameter) of a video frame to the value of the frame header packet jitter parameter (or frame end packet jitter parameter) of the current video frame.
  • the frame first packet jitter parameters of video frames 1-4 can be obtained as follows:
  • the frame header jitter parameter Jitter1 0 of video frame 1;
  • the frame first packet jitter parameter Jitter2 Jitter1*15/16+(Delta2-Interval2)*1/16 of video frame 2;
  • the frame first packet jitter parameter Jitter3 Jitter2*15/16+(Delta3-Interval3)*1/16 of video frame 3;
  • the video frame 1-4 can be calculated End-of-frame packet jitter parameters for each video frame.
  • the frame end packet jitter parameter of video frame 1 is equal to 0 by default. Except video frame 1, an example of the calculation method of the frame end packet jitter parameter of other video frames is shown in formula (2):
  • Jittern’ Jittern-1’*15/16+(Deltan’-Intervaln’)*1/16 (2)
  • the frame tail packet jitter parameter Jitter1' 0 of video frame 1;
  • the frame end packet jitter parameter Jitter2' Jitter1'*15/16+(Delta2'-Interval2')*1/16 of the frame tail of video frame 1;
  • the frame end packet jitter parameter Jitter3' Jitter2'*15/16+(Delta3'-Interval3')*1/16 of the frame tail of video frame 1;
  • the frame end packet jitter parameter of video frame 1 Jitter4' Jitter3'*15/16+(Delta4'-Interval4')*1/16.
  • FIG. 7 and FIG. 8 respectively show examples of calculation manners of a frame packet loss parameter and a frame disorder parameter according to an embodiment of the present application.
  • the streaming media layer protocol information of the data packet includes a streaming media layer sequence number indicating a sequence of the data packet in the video frame to which it belongs.
  • the network layer protocol information of the data packet includes the packet time indicating the time of the data packet, and the frame quality parameter includes a frame loss parameter and a frame out-of-sequence parameter, and the frame loss parameter indicates the loss situation of the data packet of the video frame , the frame out-of-sequence parameter represents the out-of-order situation of the data packets of the video frame,
  • Step S22 including:
  • the frame packet loss parameter representing the loss of data packets of the video frame and the frame disorder parameter representing the disorder of the data packets of the video frame can be calculated, which improves the quality of the video frame while characterizing the quality of the video frame. Flexibility in choice of frame quality parameters.
  • the frame loss situation can be determined from the perspective of the streaming media layer of the data packet, and it can be realized by counting the sequence numbers of the streaming media layer and analyzing them.
  • the media layer serial numbers S a -S i can be 11, 12, 13, 15, 16, 17, 18, 19, 20 respectively, and the stream media layer serial numbers S a -S i obtained according to the statistics can be analyzed to determine the frame loss Condition.
  • Frame out-of-sequence can be determined from the perspective of the streaming media layer and the network layer of the data packet.
  • the sorting result is obtained by sorting the data packets according to the packet time, and combined with the sorting result and the stream media layer sequence number of the statistical data packet to analyze and determine.
  • the streaming media layer sequence numbers SA - SK of the data packet AK corresponding to the video frame 3 are 21, 22, 23, 25, 26, 27, 28, 24, 29, 30, 31 respectively
  • the serial numbers S A -SK of the media layer can analyze and determine the packet loss of the frame. For an exemplary implementation thereof, reference may be made to the description below.
  • determining the frame packet loss parameter of the video frame according to the streaming media layer sequence number of the data packet corresponding to the video frame includes:
  • the streaming media layer sequence number of the data packet corresponding to the sorting result and the video frame determine the frame out-of-sequence parameter of the video frame, including:
  • the current data The minimum value of the streaming layer sequence number of the packet and the streaming layer sequence number of the data packet corresponding to the video frame determines the frame disorder parameter corresponding to the video frame.
  • the frame packet loss parameter can reflect the information of whether the data packet in the video frame is lost or not, and when there is a packet loss, it can also reflect the information of the number of the packet loss; the frame disorder parameter can reflect the information in the video frame
  • the information about whether the data packets are out of order can also reflect the information about the out-of-order depth of the out-of-order data packets when there is out-of-order, so that the accuracy of determining the quality of video frames according to the frame quality parameters can be improved.
  • the streaming layer sequence numbers S a -S i of the data packets ai corresponding to video frame 2 are 11, 12, 13, 15, 16, 17, 18, 19, 20 respectively, according to
  • the streaming media layer sequence numbers SA - SK of the data packet AK corresponding to the video frame 3 are 21, 22, 23, 25, 26, 27, 28, 24, 29, 30, 31 respectively, sorted
  • the out-of-order depth of the out-of-order data packet can be used as the value of the frame out-of-order parameter corresponding to the video frame.
  • the sum of the out-of-order depth of each out-of-order data packet can be averaged according to the number of out-of-order data packets to obtain the average out-of-order depth, and the average out-of-order depth is used as The value of the frame out-of-order parameter corresponding to this video frame.
  • the frame packet loss parameter and the frame disorder parameter can be calculated.
  • the packet quality parameters of the data packets are obtained to reflect the data packet quality
  • the packet quality parameters may include, for example, one or more of packet interval parameters, packet jitter parameters, out-of-sequence parameters, packet loss parameters, retransmission parameters, delay parameters, IP packet fragmentation parameters, and checksum error parameters.
  • the packet quality parameter of each data packet corresponding to each frame in the at least one video frame is determined according to the network layer protocol information and the transport layer protocol information of the data packet, and the packet quality parameter includes : The packet interval parameter that represents the time interval between the current data packet and the previous data packet, the packet jitter parameter that represents the delay jitter between the current data packet and the previous data packet, the out-of-order parameter that represents the disordered state of the data packet, and the data packet The packet loss parameter for packet loss, the retransmission parameter for data packet retransmission, the delay parameter for data packet delay, the IP packet fragmentation parameter for IP packet fragmentation, and the If the checksum of the packet is incorrect one or more of the checksum error parameters.
  • the packet jitter parameter can be determined according to the packet time in the network layer protocol information of the data packet and the preset control parameters, and its calculation method can refer to the example of the calculation method of the first frame packet jitter parameter or the frame end packet jitter parameter above.
  • the packet time indicating the time of the data packet can calculate the packet interval parameter.
  • the packet jitter parameter of the previous data packet and the preset control parameter the packet jitter parameter of each data packet can be calculated.
  • video frame 4 may include 5 data packets, for example, when 2 ⁇ m ⁇ 5 and is an integer, the packet jitter parameters of data packet m and data packet m-1 can be passed through data packet m and data packet m
  • the packet interval parameter of -1, the packet jitter parameter of data packet m-1, and the control parameters are determined.
  • the out-of-order parameter can be determined according to the network layer serial number in the network layer protocol information of the current data packet and the previous data packet and the transport layer serial number in the transport layer protocol information.
  • the value of the parameter can be equal to 1, and when the current data packet is not an out-of-order data packet, the value of the out-of-order parameter can be equal to 0.
  • the data packets are sorted according to the packet time. If the transport layer sequence number and the network layer sequence number of the current data packet are reduced compared with the transport layer sequence number and the network layer sequence number of the previous data packet, it indicates that the current data packet Packets are out-of-order packets.
  • the packet loss parameter can be determined according to the network layer serial number in the network layer protocol information of the data packet and the transport layer serial number, checksum, and confirmation message serial number in the transport layer protocol information.
  • the current data packet is a lost data packet
  • the value of the packet loss parameter may be equal to 1
  • the value of the packet loss parameter may be equal to 0.
  • the data packets can be sorted according to the packet time.
  • the current data packet If compared with the transport layer sequence number and the network layer sequence number of the previous data packet, the current data packet The serial number of the transport layer decreases, the serial number of the network layer increases, and before the packet time of the current data packet, in the video buffer, the network layer serial number, the serial number of the transport layer, and the checksum of the current data packet have not appeared The same data packet, but a data packet with the same sequence number as the confirmation message of the current data packet has appeared, which indicates that the current data packet is a lost data packet.
  • the network layer sequence number of the data packet can be directly used to determine whether the packet is lost.
  • the data packets can be sorted according to the packet time, and if the network layer sequence number of the current data packet decreases compared with the network layer sequence number of the previous data packet, it indicates that the current data packet is a lost data packet.
  • the retransmission parameters can be determined according to the network layer sequence number in the network layer protocol information of the data packet and the transport layer sequence number, checksum, and confirmation message sequence number in the transport layer protocol information.
  • the current data packet is a retransmitted data packet
  • the value of the retransmission parameter may be equal to 1
  • the value of the retransmission parameter may be equal to 0.
  • the data packets can be sorted according to the packet time, if compared with the transport layer sequence number and the network layer sequence number of the previous data packet, the current data packet.
  • the serial number of the transport layer decreases, the serial number of the network layer increases, and before the packet time of the current data packet, in the video buffer, the same as the network layer serial number, transport layer serial number, and checksum of the current data packet appear , it indicates that the current data packet is a retransmitted data packet.
  • Another example is to calculate the retransmission parameters of the data packets corresponding to the video stream using the UDP protocol, and the data packets can be sorted according to the packet time. If the network layer sequence number and checksum of the current data packet are the same, and the packet time A data packet earlier than the packet time of the current data packet indicates that the current data packet is a retransmitted data packet.
  • the time delay parameter can be determined according to the packet time in the network layer protocol information of the data packet and the confirmation message sequence number in the transport layer protocol information. For example, the time delay parameter of the data packet corresponding to the video stream using the TCP protocol can be calculated according The confirmation message sequence number of the current data packet Find the data packet with the same confirmation message sequence number as the current data packet, and determine the delay according to the packet time of the data packet and the difference between the packet time of the found data packet and the current data packet The numeric value of the parameter.
  • the IP packet fragmentation parameter can be determined according to the fragmentation flag in the network layer protocol information of the data packet. For example, when there is a fragmentation flag in the network layer protocol information, it means that the data packet has an IP The value of the fragment parameter can be equal to 1. When there is no fragment flag in the network layer protocol information, it means that there is no IP packet fragmentation in the data packet, and the value of the IP packet fragment parameter can be equal to 0.
  • the checksum error parameter can be determined according to the checksum in the transport layer protocol information of the data packet and the checksum value calculated according to the prior art. If the two values are inconsistent, it means that the current data packet has a checksum error.
  • the value of the checksum error parameter is equal to 1 when there is a checksum error.
  • the value of the checksum error parameter is equal to zero when there are no checksum errors. Wherein, when the value of the checksum in the transport layer protocol information is an invalid value (the value is 0), the checksum in the transport layer protocol information and the calculated checksum value are not compared.
  • step S24 for the frame quality parameter calculated in step S22 and the packet quality parameter calculated in step S23, abnormal determination of the parameters is performed in combination with the frame quality parameter threshold and the packet quality parameter threshold respectively.
  • step S24 includes:
  • the frame quality parameter whose value is greater than the frame quality parameter threshold is determined as the abnormal frame quality parameter
  • the packet quality parameter whose value is greater than the packet quality parameter threshold is determined as the abnormal packet quality parameter.
  • abnormal frame quality parameters and packet quality parameters can be determined.
  • Abnormal parameters are determined by comparing parameter values with threshold values, and the determination of abnormal parameters can be completed accurately and quickly.
  • the frame quality parameter threshold and the packet quality parameter threshold can be determined in real time by using a preset fixed value or by using an AI model.
  • An exemplary method for determining the threshold of the frame quality parameter and the threshold of the packet quality parameter by using the AI model is introduced below.
  • each frame quality parameter may correspond to a frame quality parameter threshold; and each packet quality parameter may correspond to a packet quality parameter threshold.
  • the AI model determines a frame quality parameter threshold (or packet quality parameter threshold) corresponding to a frame quality parameter (or packet quality parameter)
  • the data used can be calculated during a period of time before the current moment, and multiple video Multiple values of this type of frame quality parameter corresponding to frames (or multiple values of this type of packet quality parameter corresponding to multiple data packets).
  • FIG. 9 shows an example of a normal distribution probability curve generated according to an embodiment of the present application. As shown in FIG. 9 , the histogram shows the numerical distribution of the jitter parameter of the first packet of the historical frame, and the curve is a normal distribution probability curve P(x) fitted according to the distribution of the jitter parameter of the first packet of the historical frame.
  • An abnormality threshold is set in advance, and when the cumulative distribution function F(a) is equal to the abnormality threshold, the value of a can be the corresponding frame quality parameter threshold. If the frame header packet jitter parameter calculated after the current moment is greater than the threshold, it can be determined to be an abnormal frame header packet jitter parameter, that is, to be determined to be an abnormal frame quality parameter.
  • the abnormality threshold is preset as 75%, and calculated according to the cumulative distribution function of the normal distribution probability curve, when the value of the cumulative distribution function F(a) is equal to 75%, The corresponding value of a may be equal to 100, for example. In this case, the normal range of the value of the frame header jitter parameter is 0-100.
  • Fig. 10 shows an example of realizing abnormal condition determination of parameters according to an embodiment of the present application.
  • the value of a frame header jitter parameter that needs to be determined to be abnormal is 240, and the value is greater than 100.
  • the frame header jitter parameter is an abnormal frame quality parameter, so that the abnormality of the parameter can be determined.
  • the frame quality parameter (or packet quality parameter threshold) obtained by the AI model can be used to determine the frame quality parameter (or packet quality parameter) that needs to be determined to be abnormal. or packet quality parameter) is an abnormal frame quality parameter (or abnormal packet quality parameter). Further, the parameter samples that have been determined to be abnormal frame quality parameters (or abnormal packet quality parameters) in the parameter samples can also be cropped, so that the frame quality parameter threshold (or packet quality parameter threshold) is also updated as the parameter samples change .
  • the method described above is used to determine the abnormality of the frame quality parameter, and the determined abnormality of the frame quality parameter can be shown in Table 1 below, for example.
  • the value of 1 for abnormal jitter at the beginning of the frame, abnormal jitter at the end of the frame, abnormal packet loss, and abnormal frame disorder indicates that the corresponding frame quality parameter of the frame is an abnormal frame quality parameter, this frame is also an abnormal frame.
  • video frame 1 has no frame quality parameter abnormality
  • the first frame packet jitter parameter of video frame 2 is abnormal
  • the frame out-of-sequence parameter is abnormal
  • the frame first packet jitter parameter of video frame 3 is abnormal
  • the frame tail packet jitter parameter is abnormal
  • the frame loss parameter is abnormal
  • the frame quality parameter of video frame 4 is abnormal.
  • the method described above is used to determine the abnormal situation of the packet quality parameter, and the determined abnormal situation of the packet quality parameter can be shown in Table 2 below, for example.
  • gNB, UDG, and EoR represent 5G base stations, unified distributed gateways, and switches using a certain deployment method, respectively.
  • abnormal packet loss, out-of-sequence abnormality, packet jitter abnormality, packet interval abnormality, retransmission abnormality, delay abnormality, IP packet fragmentation abnormality, and checksum error abnormality have a value of 1 Indicates that the package quality parameter is an abnormal package quality parameter, and the package is also an abnormal package.
  • the packet quality parameters of data packets 1, 3, 4, and 5 are not abnormal, and the packet jitter parameters of data packet 2 are abnormal; on the unified distributed gateway UDG, the packet jitter parameters of data packet 1 are abnormal , the packet loss parameter and delay parameter of data packet 2 are abnormal, and the packet loss parameter and packet jitter parameter of data packet 3 are abnormal; The packet loss parameter is abnormal, and the packet loss parameter and packet jitter parameter of packet 3 are abnormal.
  • step S3 After the abnormal frame quality parameters and abnormal packet quality parameters are obtained in step S2, in step S3, it can be determined according to the relationship between the abnormal frame quality parameters, the abnormal packet quality parameters and the preset conditions, which causes the abnormal video quality of the video playback terminal. Causes and equipment.
  • Fig. 11 shows an exemplary schematic diagram of the correspondence between abnormal frame quality parameters, abnormal packet quality parameters, causes of abnormal video quality at the video player end, and devices according to an embodiment of the present application.
  • the packet quality parameter corresponding to the frame quality parameter can be found by using the correspondence between the video frame and the data packet, so that the data packet to which the packet quality parameter belongs is the data packet under the video frame to which the frame quality parameter belongs.
  • the frame quality parameter and its corresponding packet quality parameter usually have a certain correlation. For example, when the packet loss parameter of a certain data packet is abnormal, the situation of the frame packet loss parameter of the video frame to which the data packet belongs can be checked. When the out-of-sequence parameter of a data packet is abnormal, you can check the frame out-of-sequence parameter of the video frame to which the data packet belongs.
  • the packet interval parameter or packet jitter parameter of a certain data packet When the packet interval parameter or packet jitter parameter of a certain data packet is abnormal, it may cause the data packet to fail to arrive at the expected time of decoding, which may cause the video frame to which the data packet belongs to have a large jitter. You can view the video frame to which the data packet belongs The case of the first frame packet jitter parameter and the frame end packet jitter parameter.
  • the sending of the data packet may be slowed down, which may cause the video frame to which the data packet belongs to shake greatly. You can view the video frame to which the data packet belongs. The situation of the jitter parameters of the first packet of the frame and the jitter parameters of the last packet of the frame.
  • the IP packet fragmentation parameter and the checksum error parameter of the data packet are also associated with the jitter of the video frame to which the data packet belongs, therefore.
  • the IP packet fragmentation parameters or checksum error parameters of a data packet are abnormal, you can check the frame header jitter parameters and frame tail packet jitter parameters of the video frame to which the data packet belongs.
  • step S3 includes:
  • the cause of the abnormal video quality at the video playback end is a video encoding problem
  • the The device with abnormal video quality at the video playback end is the device used for video encoding
  • the first condition includes:
  • the frame packet loss parameter of the video frame is abnormal and the packet loss parameter of the data packet corresponding to the video frame is normal, or
  • the frame out-of-order parameter of the video frame is abnormal and the out-of-order parameter of the data packet corresponding to the video frame is normal.
  • the frame quality parameter and the packet quality parameter can reflect the abnormal situation in the transmission process, and the frame packet loss parameter abnormality indicates that there is a data packet under the video frame being transmitted If the packet loss parameter of the data packet under the video frame is normal at this time, it means that no data packet is lost during transmission, so it can be determined that the data packet is lost before transmission, that is, during video encoding; frame disorder Abnormal parameters indicate that there are data packets under the video frame that are out of order during transmission.
  • the out-of-order parameters of the data packets under the video frame are normal at this time, it means that the data packets are not out of order during transmission, so it can be determined that the data packets are in the out-of-order state.
  • the first condition is that the frame packet loss parameter of the video frame is abnormal and the packet loss parameter of the data packet corresponding to the video frame is normal, or the video frame
  • the out-of-sequence parameter of the frame is abnormal and the out-of-order parameter of the data packet corresponding to the video frame is normal, which can accurately reflect the video encoding problem.
  • the frame loss parameter in the frame quality parameter is abnormal and the packet loss parameter in the packet quality parameter is normal, or the frame disorder parameter in the frame quality parameter is abnormal and the disorder parameter in the packet quality parameter is normal , it can be determined that the abnormal frame quality parameter and the abnormal packet quality parameter meet the first condition, and then determine that the cause of the video quality problem occurs during the video encoding process, the video encoding process is dominated by the equipment used for video encoding, and the video encoding There is one and only one device used, so the only device that can be sure to cause the video encoding problem is the device used for video encoding.
  • the cause and device that cause the abnormal video quality at the video playback end can be directly determined without having to determine the frame quality
  • the cause and device that cause the abnormal video quality at the video playback end can be directly determined without having to determine the frame quality
  • the parameter and the packet quality parameter satisfy the remaining ones in the first condition.
  • it may also be set to determine whether the frame quality parameter and the packet quality parameter satisfy each possibility in the first condition, and after determining that the frame quality parameter and the packet quality parameter satisfy at least one possibility in the first condition, then determine the result Reasons and equipment for abnormal video quality at the video playback end.
  • both of the above two setting methods can be used to determine the cause and equipment that cause the abnormal video quality of the video playback terminal, and this application does not limit this.
  • step S3 includes:
  • the cause of the abnormal video quality at the video playback end is a terminal parameter problem, and it is determined that the cause
  • the device with abnormal video quality at the video playback end is the device used for video playback;
  • the second condition includes:
  • Any parameter in the frame header jitter parameter and frame tail packet jitter parameter of the video frame is abnormal and any parameter in the IP packet fragmentation parameter and checksum error parameter of the data packet corresponding to the video frame is abnormal .
  • the frame quality parameter and the packet quality parameter can reflect the abnormal situation in the transmission process, and the abnormality of the first frame packet jitter parameter/frame end packet jitter parameter indicates that the video The delay jitter of the first/last data packet in the frame is too large during transmission. If the IP packet fragmentation parameter of the data packet in the video frame is abnormal at this time, it means that the data packet has experienced fragmentation during transmission.
  • the parameter configuration of the video playback terminal fails to meet the requirements for data packets to be transmitted without fragmentation;
  • the video playback end successfully completes the verification, so it can be determined that the parameter configuration of the video playback end fails to meet the requirements for successful verification of the data packet;
  • the second condition that is, the first frame packet jitter parameter and the end of the frame of the video frame Any parameter in the packet jitter parameters is abnormal and any parameter in the IP packet fragmentation parameter and the checksum error parameter of the data packet corresponding to the video frame is abnormal, which can accurately reflect the terminal parameter problem.
  • the frame first packet jitter parameter/frame end packet jitter parameter in the frame quality parameter is abnormal and the IP packet fragmentation parameter/checksum error parameter in the packet quality parameter is normal, it can be determined that the abnormal The frame quality parameter and the abnormal packet quality parameter meet the second condition, and then it is determined that the cause of the video quality problem appears on the playback parameters during the video playback process, the video playback process is dominated by the device used for video playback, and the video playback There is only one device used (that is, the video playback terminal), so it can be uniquely determined that the device that causes the terminal parameter problem is the device used for video playback.
  • the cause and device that cause the abnormal video quality at the video playback end can be directly determined without having to determine the frame quality parameters and packet quality parameters satisfy the remaining possible cases in the second condition.
  • it may also be set to determine whether the frame quality parameter and the packet quality parameter satisfy each possibility in the second condition, and after determining that the frame quality parameter and the packet quality parameter satisfy at least one possibility in the second condition, then determine the result Reasons and equipment for abnormal video quality at the video playback end.
  • step S3 includes:
  • the third condition includes:
  • the frame packet loss parameter of the video frame is abnormal and the packet loss parameter of the data packet corresponding to the video frame is abnormal, or
  • the frame out-of-order parameter of the video frame is abnormal and the out-of-order parameter of the data packet corresponding to the video frame is abnormal, or
  • Any one of the frame header jitter parameters and frame tail packet jitter parameters of the video frame is abnormal and any of the packet jitter parameters, packet interval parameters, retransmission parameters, and delay parameters of the data packets corresponding to the video frame A parameter exception.
  • the frame quality parameter and the packet quality parameter can reflect the abnormal situation in the transmission process, and the frame packet loss parameter abnormality indicates that there is a data packet under the video frame being transmitted If the packet loss parameter of the data packet under the video frame is abnormal at this time, it means that there is a data packet lost during transmission, so it can be determined that there is a problem with the network transmission; the abnormal frame disorder parameter indicates that there is a data packet under the video frame It is in an out-of-order state during transmission.
  • the out-of-order parameters of the data packets under the video frame are abnormal at this time, it means that some data packets are in out-of-order during transmission, so it can be determined that there is a problem in network transmission; based on the above reasons, in the third condition "The frame packet loss parameter of the video frame is abnormal and the packet loss parameter of the data packet corresponding to the video frame is abnormal, or the frame out-of-order parameter of the video frame is abnormal and the out-of-order data packet corresponding to the video frame Abnormal parameter", which can accurately reflect the network transmission problem; the abnormality of the frame first packet jitter parameter/frame end packet jitter parameter indicates that the delay jitter of the first/last data packet under the video frame is too large during transmission.
  • the packet interval parameter of the data packet under the frame is abnormal, it means that the time difference between the two data packets in the transmission process is too large, and it can be determined that there is a problem in the network transmission; if the packet jitter parameter of the data packet under the video frame is abnormal at this time, it means that the data packet If the delay and jitter in the transmission process is too large, it can be determined that there is a problem in network transmission; if the retransmission parameters of the data packet under the video frame are abnormal at this time, it means that the data packet has undergone multiple transmissions during the transmission process, and it can be determined that there is a network transmission problem.
  • the delay parameter of the data packet under the video frame is abnormal at this time, it means that the transmission time of the data packet is too long during the transmission process, and it can be determined that there is a problem with the network transmission.
  • the third condition "any parameter in the frame header jitter parameter and the frame tail packet jitter parameter of the video frame is abnormal and the packet jitter parameter, the packet interval parameter, and the packet interval parameter of the data packet corresponding to the video frame are Any one of the retransmission parameters and delay parameters is abnormal" can accurately reflect the terminal parameter problem.
  • the frame loss parameter in the frame quality parameter is abnormal and the packet loss parameter in the packet quality parameter is abnormal, or the frame disorder parameter in the frame quality parameter is abnormal and the disorder parameter in the packet quality parameter is abnormal , or if the frame first packet jitter parameter/frame end packet jitter parameter in the frame quality parameter is abnormal and the packet jitter parameter/packet interval parameter/retransmission parameter/delay parameter in the packet quality parameter is abnormal, you can determine the abnormal frame quality parameter and all The aforementioned abnormal packet quality parameters meet the third condition, and then it is determined that the cause of the abnormal video quality occurs during network transmission, and the abnormal video quality is caused by network equipment.
  • Table 3 shows an example of determining that the cause of the video quality problem is a network transmission problem according to the embodiment of the present application.
  • a frame timestamp identifies a unique frame.
  • the packet information that needs to be viewed corresponding to a frame may include packet time, network layer sequence number, transport layer sequence number, checksum, and abnormality of packet quality parameters.
  • For abnormal packet quality parameters according to the association relationship between frame quality parameters and packet quality parameters, in the "Analysis Result" column, the analysis results that lead to video quality problems can be given.
  • the frame packet loss parameter of video frame 1 is abnormal
  • the packet loss parameter of data packet 2 corresponding to video frame 1 is abnormal
  • the packet loss parameter of data packet 3 corresponding to video frame 1 is abnormal, therefore satisfying the third conditions, it can be determined that the cause of the video quality problem is a network transmission problem.
  • the cause of the abnormal video quality at the video playback end can be directly determined without having to determine the frame quality parameter and Whether the packet quality parameter satisfies the remaining possibilities in the third condition.
  • it may also be set to determine whether the frame quality parameter and the packet quality parameter satisfy each possibility in the third condition, and after determining that the frame quality parameter and the packet quality parameter satisfy at least one possibility in the third condition, then determine the result The cause of abnormal video quality on the video player side.
  • the network transmission process may use multiple network devices, it is also possible to further determine the specific device causing the terminal parameter problem.
  • step S3 further includes:
  • the network layer protocol information of the data packet determine the network layer serial number of the data packet corresponding to the abnormal packet quality parameter
  • the transport layer protocol information of the data packet determine the packet time, transport layer sequence number, and checksum of the data packet corresponding to the abnormal packet quality parameter
  • the transmission path of the data packet corresponding to the abnormal packet quality parameter described in the network find the data packet that satisfies the conditions of packet time, network layer serial number, transport layer serial number, and checksum in each network device on the transmission path ;
  • the network equipment causing the abnormal video quality of the video playback terminal is determined from among the network equipments on the transmission path of the data packets.
  • the packet quality parameter of the data packet on each device can indicate the impact of the device on the transmission of the data packet, and by finding the corresponding data packets in each network device on the transmission path, the statistics can be calculated using the found data packets During the transmission of the packet, the impact of different network devices on the transmission of the data packet, and determine the specific device that causes the abnormal video quality of the video player.
  • the network layer sequence number of the data packet corresponding to the abnormal packet quality parameter can be determined, and according to the transport layer
  • the protocol information can determine the packet time, transport layer sequence number, and checksum of the data packet corresponding to the abnormal packet quality parameter.
  • the transmission path of the data packet corresponding to the abnormal packet quality parameter on the network device can be obtained.
  • the abnormal packet quality parameter can be, for example, the packet loss parameter of data packet 2 and the packet loss parameter of data packet 3.
  • it can be determined, for example, that the transmission path of data packet 2 and data packet 3 can be gNB ⁇ UDG ⁇ EoR.
  • the transmission path of the data packet corresponding to the abnormal packet quality parameter in the network you can find the data packet that meets the conditions of the packet time, network layer serial number, transport layer serial number, and checksum in each network device on the transmission path.
  • the network device that causes the abnormal video quality of the video playback terminal is determined. For example, see Table 3, based on packet time (8:14:12.008), network layer sequence number (48160), transport layer sequence number (3480724955), checksum (1231221433) of packet 2, and packet 3 Packet time (8:14:12.019), network layer sequence number (48162), transport layer sequence number (3480726183), checksum (2134423523), and find a packet that meets the conditions.
  • the packet time, the network layer serial number, the transport layer serial number, and the checksum satisfying conditions include:
  • the absolute value of the difference between the packet time and the packet time of the data packet corresponding to the abnormal packet quality parameter is less than a preset threshold, and the network layer sequence number, the transport layer sequence number, the checksum and the abnormal packet quality parameter.
  • the network layer sequence number, transport layer sequence number, and checksum of the corresponding data packets are all the same.
  • the same data packet on different network devices along the transmission path can be found.
  • the cause of the abnormality of the packet quality parameters on different devices is the current device or the upstream device of the current device.
  • the cause of the abnormal video quality at the video playback end is a network transmission problem, and the specific network device that causes the abnormal video quality at the video playback end.
  • the network layer serial number (48162), transport layer serial number (3480726183), and checksum (2134423523) of data packet 3 in Table 3 it can be found in Table 2 that the network layer serial number is equal to 48162, and the transport layer serial number is equal to 3480726183, the data packet whose checksum is equal to 2134423523, you can find the data packet 5 on the gNB, the data packet 3 on the UDG and the data packet 3 on the EoR that meet the requirements, and then further determine the packet time of the found data packet and Table 3 Whether the absolute value of the difference between the packet time (8:14:12.019) of data packet 3 is less than the preset threshold, for example, the value of the preset threshold can be equal to 0.01, in this case, the data packet 5 on the gNB, the The absolute value of the packet time difference between data packet 3 and data packet 3 on EoR and data packet 3 in Table 3 is less than the preset threshold, data packet 5 on gNB, data packet 3 on UDG and data packet on E
  • the specific network device causing the abnormal video quality at the video playback end can be determined.
  • FIG. 12 shows an example of determining a specific network device that causes abnormal video quality at a video player end according to an embodiment of the present application.
  • A, B, and C are gNB, UDG, and EoR respectively, and the data packets on gNB, UDG, and EoR that meet the conditions found for data packet 2 in Table 3 (hereinafter referred to as A , B, and C devices) as an example, according to the network topology data, it can be determined that the transmission path of the data packet is gNB ⁇ UDG ⁇ EoR.
  • the abnormal packet quality parameter of data packet 2 in Table 3 is the packet loss parameter.
  • the packet loss parameters of the data packets on devices A, B, and C are all abnormal, it can be determined that the packet loss introduced by device A (gNB) leads to The video quality is abnormal; the packet loss parameters of data packets on B and C are abnormal, and when the packet loss parameters of data packets on device A are normal, it can be determined that device B (UDG) introduces packet loss, which leads to abnormal video quality; device C The packet loss parameters of the above data packets are abnormal, and when the packet loss parameters of the data packets on devices A and B are normal, it can be determined that device C (EoR) introduces packet loss, which leads to abnormal video quality.
  • the packet loss parameters of device A are abnormal, the packet loss parameters of devices B and C must be abnormal; if the packet loss parameters of device B are abnormal, the packet loss parameters of device C must be abnormal. Must be abnormal.
  • collection errors may occur, causing errors in the protocol information, which in turn may cause errors in the packet quality parameters determined according to the protocol information. Therefore, according to different network devices found on the transmission path When determining the network device that causes the video quality problem of the data packets that meet the conditions, the influence of the protocol information collection error can be further excluded.
  • the packet loss parameters of data packets on devices A and B are abnormal, and the packet loss parameters of data packets on device C are normal, it can be considered that there is an error when collecting protocol information from device C, and it can be determined that device A has introduced packet loss, and then The video quality is abnormal; the packet loss parameters on devices A and C are abnormal, and when the packet loss parameters on device B are normal, it can be considered that there is an error when collecting protocol information from device B, and it can be determined that device A introduced Packet loss, resulting in abnormal video quality.
  • packet loss parameters on devices A, B, and C There are other examples of packet loss parameters on devices A, B, and C.
  • the packet loss parameters on devices A, B, and C are all normal, and the packet loss parameters on device A are abnormal.
  • Examples of normal packet loss parameters on devices B and C, abnormal packet loss parameters on device B, and normal packet loss parameters on devices A and C, because these examples have a very low probability of occurrence It will not be described in detail here.
  • each data packet (packet 2, packet 3) with abnormal packet quality parameters is uniquely identified by a network layer sequence number, a transport layer sequence number and a checksum.
  • the abnormal packet quality parameter is a packet quality parameter that needs to be compared with the found data packets.
  • the packet loss represents the abnormality of the abnormal packet quality parameters that need to be compared on the device. 0 means that the packet quality parameters are normal, and 1 means that the packet quality parameters are abnormal. Combining with FIG. 11 , it can be determined that the network device that causes the abnormality of data packet 2 and data packet 3 in Table 3 and then causes abnormal video quality is UDG.
  • the determined network devices that cause abnormal video quality are the same, and in other application scenarios, due to errors, it may also occur There are many cases of network devices with abnormal video quality.
  • the determined multiple network devices that cause abnormal video quality may be further screened to obtain a network device that is most likely to cause abnormal video quality.
  • the network device that causes the abnormal video quality of the video player including:
  • the network device with the most occurrences is counted as the network device that causes abnormal video quality at the video playback end.
  • the accuracy of the determined network device that causes abnormal video quality can be improved, so that the determined network that causes abnormal video quality After the device implements the corresponding solution, the probability of abnormal video quality caused by the same problem on the same device is reduced.
  • the network device most likely to cause abnormal video quality is determined.
  • Table 4 there are 2 times of abnormal packet loss parameters caused by UDG, and 0 times of abnormal packet loss parameters caused by other devices, so the network device most likely to cause abnormal video quality is UDG. If in another example, there are 2 packet loss parameter exceptions caused by UDG, 5 packet loss parameter exceptions caused by EoR, and 0 packet loss parameter exceptions caused by gNB, then the network with the greatest possibility of video quality abnormality The device is EoR.
  • Fig. 13 shows an exemplary structural diagram of an apparatus for determining a video quality problem according to an embodiment of the present application.
  • the present application also proposes an apparatus for determining video quality problems, the apparatus including: an information receiving module 130, configured to receive protocol information on the video playback terminal and network equipment , the network device is used to transmit the video data to the video player, the protocol information indicates the sending mode of the video frame, and the video frame includes a plurality of data packets; the parameter determination module 131 is used to determine according to the protocol information An abnormal frame quality parameter and an abnormal packet quality parameter, the abnormal frame quality parameter indicating that the quality of the video frame to which the abnormal frame quality parameter belongs is abnormal, and the abnormal packet quality parameter indicating the abnormal packet quality parameter The quality of the associated data packet is abnormal; the abnormal cause determination module 132 is used to determine the cause and equipment that cause the abnormal video quality of the video playback terminal according to the abnormal frame quality parameter and/or the abnormal packet quality parameter.
  • an information receiving module 130 configured to receive protocol information on the video playback terminal and network equipment , the network device is used to transmit the video data to the video player, the protocol information indicates the sending mode of the video frame, and the video frame
  • the protocol information includes streaming media layer protocol information, network layer protocol information, and transport layer protocol information of the data packet
  • the streaming media layer protocol information includes the flow of the data packet on the video playback end.
  • Media layer protocol information, the network layer protocol information includes the network layer protocol information of the data packet on the network device and the video playback end
  • the transport layer protocol information includes the transport layer of the data packet on the network device and the video playback end Protocol information
  • the determination of abnormal frame quality parameters and abnormal packet quality parameters according to the protocol information includes: according to the streaming media layer protocol information, determining at least one video frame and the frame corresponding to each frame in the at least one video frame Each data packet; according to the streaming media layer protocol information and the network layer protocol information, determine the frame quality parameters of each frame in the at least one video frame; according to the network layer protocol information and the transport layer protocol information, Determining the packet quality parameter of each data packet corresponding to each frame in the at least one video frame; determining the abnormal frame quality parameter according to the frame quality parameter corresponding to each frame in the at least one video
  • the abnormal frame quality parameter is determined according to the frame quality parameter corresponding to each frame in the at least one video frame and the frame quality parameter threshold, and the abnormal frame quality parameter is determined according to each frame quality parameter in the at least one video frame.
  • the packet quality parameter of each data packet corresponding to the frame and the packet quality parameter threshold value, determining the abnormal packet quality parameter includes: for each frame of the at least one video frame, among the frame quality parameters of the video frame, the value A frame quality parameter greater than the frame quality parameter threshold is determined as the abnormal frame quality parameter; for each frame of the at least one video frame, among the packet quality parameters of the data packet corresponding to the video frame, the value is greater than The packet quality parameter of the packet quality parameter threshold is determined as the abnormal packet quality parameter.
  • the streaming media layer protocol information of the data packet includes a frame time stamp indicating the video frame to which the data packet belongs
  • the network layer protocol information of the data packet includes a packet time indicating the time of the data packet
  • the frame quality parameter includes a frame header jitter parameter and a frame tail packet jitter parameter
  • the frame header jitter parameter represents the delay jitter of the first data packet corresponding to the video frame
  • the frame tail packet jitter parameter represents The delay and jitter situation of the last data packet corresponding to the video frame, according to the streaming media layer protocol information and the network layer protocol information, determine the frame quality parameters of each frame in the at least one video frame, including: for For each frame of the at least one video frame, the arrival time of the first packet and the arrival time of the last packet of the video frame are determined according to the packet time of the data packet corresponding to the video frame, and the arrival time of the first packet represents the first packet corresponding to the video frame.
  • the arrival time of a data packet, the arrival time of the last packet corresponding to the video frame is represented by the tail packet arrival time; according to the arrival time of the first packet and the arrival time of the tail packet, the first packet interval and the first packet interval of adjacent video frames are calculated respectively.
  • the tail packet interval, the first packet interval represents the arrival time difference of the first data packet corresponding to the adjacent video frame, and the tail packet interval represents the arrival time difference of the last data packet corresponding to the adjacent video frame; according to the frame time stamp, respectively calculate the frame interval of adjacent video frames; according to the first packet interval, frame interval, and the frame first packet jitter parameter of the previous video frame, calculate the frame first packet jitter parameter of the current video frame; according to the tail packet interval, frame interval, and the frame end packet jitter parameter of the previous video frame, and calculate the frame end packet jitter parameter of the current video frame.
  • the streaming media layer protocol information of the data packet includes a streaming media layer sequence number indicating the sequence of the data packet in the video frame to which it belongs
  • the network layer protocol information of the data packet includes indicating data
  • the packet time of the time of packet, described frame quality parameter comprises frame packet loss parameter and frame disorder parameter
  • described frame packet loss parameter represents the loss situation of the data packet of video frame
  • described frame disorder parameter represents the data of video frame Out-of-sequence situation of the packet
  • determining the frame packet loss parameter of the video frame according to the streaming media layer sequence number of the data packet corresponding to the video frame includes: For each frame of the at least one video frame, according to the streaming media layer serial number of the data packet corresponding to the video frame, determine the maximum value, the minimum value and the streaming media layer serial number of the data packet corresponding to the video frame Layer sequence number, according to the maximum value and the minimum value of the streaming media layer sequence number, determine the theoretical value of the data packet number corresponding to the video frame, according to the theoretical value of the data packet number corresponding to the video frame and the The difference of the streaming media layer sequence number of the data packet corresponding to the video frame determines the frame packet loss parameter of the video frame; according to the streaming media layer serial number of the data packet corresponding to the sorting result and the video frame, determine The frame out-of-sequence parameters of the video frame include: when the numerical value of the streaming media layer serial number of the current data packet is greater than the streaming media layer
  • the packet quality parameter of each data packet corresponding to each frame in the at least one video frame is determined according to the network layer protocol information and the transport layer protocol information of the data packet, and the packet quality parameter includes : The packet interval parameter that represents the time interval between the current data packet and the previous data packet, the packet jitter parameter that represents the delay jitter between the current data packet and the previous data packet, the out-of-order parameter that represents the disordered state of the data packet, and the data packet Packet loss parameter for packet loss, retransmission parameter for data packet retransmission, delay parameter for data packet and corresponding response message delay, IP packet fragmentation for data packet One or more of the text fragmentation parameter and the checksum error parameter indicating whether the checksum of the data packet is wrong.
  • the determining the cause and device for abnormal video quality at the video playback end according to the abnormal frame quality parameter and/or the abnormal packet quality parameter includes: for any video frame, When the frame quality parameter of the video frame and the packet quality parameter of the data packet corresponding to the video frame meet the first condition, it is determined that the cause of the abnormal video quality at the video playback end is a video encoding problem, and it is determined that the video quality at the video playback end is abnormal
  • the device is a device used for video encoding; wherein the first condition includes: the frame loss parameter of the video frame is abnormal and the packet loss parameter of the data packet corresponding to the video frame is normal, or the frame loss parameter of the video frame
  • the out-of-order parameter is abnormal and the out-of-order parameter of the data packet corresponding to the video frame is normal.
  • the determining the cause and device for abnormal video quality at the video playback end according to the abnormal frame quality parameter and/or the abnormal packet quality parameter includes: for any video frame, When the frame quality parameter of the video frame and the packet quality parameter of the data packet corresponding to the video frame meet the second condition, it is determined that the cause of the abnormal video quality at the video playback end is a terminal parameter problem, and it is determined that the video quality at the video playback end is abnormal
  • the device is a device used for video playback; wherein, the second condition includes: any parameter in the first frame packet jitter parameter and the frame end packet jitter parameter of the video frame is abnormal and the data packet corresponding to the video frame Any of the IP packet fragmentation parameters and checksum error parameters is abnormal.
  • the determining the cause and device for abnormal video quality at the video playback end according to the abnormal frame quality parameter and/or the abnormal packet quality parameter includes: for any video frame, When the frame quality parameter of the video frame and the packet quality parameter of the data packet corresponding to the video frame meet the third condition, it is determined that the cause of the abnormal video quality at the video playback end is a network transmission problem; wherein the third condition includes : The frame packet loss parameter of the video frame is abnormal and the packet loss parameter of the data packet corresponding to the video frame is abnormal, or the frame out-of-order parameter of the video frame is abnormal and the out-of-order parameter of the data packet corresponding to the video frame is abnormal Abnormal, or any parameter in the frame first packet jitter parameter and frame end packet jitter parameter of the video frame is abnormal and the packet jitter parameter, packet interval parameter, retransmission parameter, and delay parameter of the data packet corresponding to the video frame Any parameter in is abnormal.
  • the reason and equipment of abnormal video quality also include: according to the network layer protocol information of the data packet, determine the network layer sequence number of the data packet corresponding to the abnormal packet quality parameter; according to the transport layer protocol information of the data packet, determine Packet time, transport layer sequence number, and checksum of the data packet corresponding to the abnormal packet quality parameter; receive network topology data on the network device, and determine the transmission path of each data packet in the network according to the network topology data; The transmission path of the data packet corresponding to the abnormal packet quality parameter is found in each network device on the transmission path, and the packet time, the network layer serial number, the transport layer serial number, and the checksum all meet the conditions; according to the found The packet quality parameters of the data packets satisfying the conditions are determined from the network devices on the transmission path of the data packets to determine the network device that causes the
  • the conditions for satisfying the packet time, the network layer sequence number, the transport layer sequence number, and the checksum include: the difference between the packet time and the packet time of the data packet corresponding to the abnormal packet quality parameter The absolute value of the difference is less than the preset threshold, and the network layer sequence number, transport layer sequence number, checksum and the network layer sequence number, transport layer sequence number, checksum of the data packet corresponding to the abnormal packet quality parameter are the same.
  • the network device that causes the abnormal video quality of the video player including: according to the packet quality parameters of multiple sets of data packets that meet the conditions, from each network device on the transmission path of the data packet Among the network devices that may cause abnormal video quality at the video playback end, the network device that may cause abnormal video quality at the video playback end is determined multiple times; among the network devices that may cause abnormal video quality at the video playback end, the network device with the most occurrences is counted as the network device that causes abnormal video quality at the video playback end. network equipment.
  • An embodiment of the present application provides an apparatus for determining a video quality problem, including: a processor and a memory for storing instructions executable by the processor; wherein the processor is configured to implement the above method when executing the instructions.
  • An embodiment of the present application provides a non-volatile computer-readable storage medium, on which computer program instructions are stored, and when the computer program instructions are executed by a processor, the foregoing method is realized.
  • An embodiment of the present application provides a computer program product, including computer-readable codes, or a non-volatile computer-readable storage medium bearing computer-readable codes, when the computer-readable codes are stored in a processor of an electronic device When running in the electronic device, the processor in the electronic device executes the above method.
  • Fig. 14 shows an exemplary structural diagram of an apparatus for determining a video quality problem according to an embodiment of the present application.
  • the device for determining video quality problems may include a mobile phone, a foldable electronic device, a tablet computer, a desktop computer, a laptop computer, a handheld computer, a notebook computer, an ultra-mobile personal computer (ultra-mobile personal computer, UMPC) ), netbooks, cellular phones, personal digital assistants (personal digital assistant, PDA), augmented reality (augmented reality, AR) devices, virtual reality (virtual reality, VR) devices, artificial intelligence (artificial intelligence, AI) devices, wearable At least one of portable devices, vehicle-mounted devices, smart home devices, smart city devices, and server devices.
  • PDA personal digital assistant
  • augmented reality augmented reality, AR
  • VR virtual reality
  • AI artificial intelligence
  • the embodiment of the present application does not specifically limit the specific type of the apparatus for determining a video quality problem.
  • the apparatus for determining a video quality problem may include a processor 110 , a memory 121 , and a communication module 160 . It can be understood that the structure shown in the embodiment of the present application does not constitute a specific limitation on the apparatus for determining a video quality problem. In some other embodiments of the present application, the device for determining a video quality problem may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange different components. The illustrated components can be realized in hardware, software or a combination of software and hardware.
  • the processor 110 may include one or more processing units, for example: the processor 110 may include an application processor (application processor, AP), a modem processor, a graphics processing unit (graphics processing unit, GPU), an image signal processor (image signal processor, ISP), controller, video codec, digital signal processor (digital signal processor, DSP), baseband processor, and/or neural network processor (neural-network processing unit, NPU), etc. Wherein, different processing units may be independent devices, or may be integrated in one or more processors.
  • application processor application processor, AP
  • modem processor graphics processing unit
  • GPU graphics processing unit
  • image signal processor image signal processor
  • ISP image signal processor
  • controller video codec
  • digital signal processor digital signal processor
  • baseband processor baseband processor
  • neural network processor neural-network processing unit
  • the processor 110 can generate an operation control signal according to the instruction opcode and the timing signal, and complete the control of fetching and executing the instruction.
  • a memory may also be provided in the processor 110 for storing instructions and data.
  • the memory in processor 110 may be a cache memory.
  • the memory may store instructions or data used by the processor 110 or used frequently. If the processor 110 needs to use the instruction or data, it can be called directly from the memory. Repeated access is avoided, and the waiting time of the processor 110 is reduced, thus improving the efficiency of the system.
  • the memory 121 may be used to store computer-executable program code including instructions.
  • the memory 121 may include an area for storing programs and an area for storing data.
  • the stored program area can store an operating system, an application program required by at least one function (such as an abnormal frame quality parameter determination function, an abnormal packet quality parameter determination function) and the like.
  • the storage data area can store data (such as frame quality parameters, packet quality parameters, etc.) created during the use of the device for determining video quality problems.
  • the memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one magnetic disk storage device, flash memory device, universal flash storage (universal flash storage, UFS) and the like.
  • the processor 110 executes the instructions stored in the memory 121 and/or the instructions stored in the memory provided in the processor to execute various function methods of the device for determining video quality problems or the above-mentioned method for determining video quality problems.
  • the communication module 160 can be used to receive streaming data (such as the protocol information collection device in the embodiment of the application) from other devices or equipment (such as the protocol information collection device and the topology data collection device in the embodiment of the application) through wireless communication/wired communication. information, network topology data), and output streaming data to other devices or devices (such as the cause of the video quality problem and device information determined in the embodiment of this application).
  • streaming data such as the protocol information collection device in the embodiment of the application
  • it can provide WLAN (such as Wi-Fi network), Bluetooth (Bluetooth, BT), global navigation satellite system (global navigation satellite system, GNSS), frequency modulation (frequency modulation, FM), near field communication technology (near field communication) , NFC), infrared technology (infrared, IR) and other wireless communication solutions.
  • a computer readable storage medium may be a tangible device that can retain and store instructions for use by an instruction execution device.
  • a computer readable storage medium may be, for example, but is not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing.
  • Non-exhaustive list of computer-readable storage media include: portable computer disk, hard disk, random access memory (Random Access Memory, RAM), read only memory (Read Only Memory, ROM), erasable Electrically Programmable Read-Only-Memory (EPROM or flash memory), Static Random-Access Memory (Static Random-Access Memory, SRAM), Portable Compression Disk Read-Only Memory (Compact Disc Read-Only Memory, CD -ROM), Digital Video Disc (DVD), memory sticks, floppy disks, mechanically encoded devices such as punched cards or raised structures in grooves with instructions stored thereon, and any suitable combination of the foregoing .
  • RAM Random Access Memory
  • ROM read only memory
  • EPROM or flash memory erasable Electrically Programmable Read-Only-Memory
  • Static Random-Access Memory SRAM
  • Portable Compression Disk Read-Only Memory Compact Disc Read-Only Memory
  • CD -ROM Compact Disc Read-Only Memory
  • DVD Digital Video Disc
  • Computer readable program instructions or codes described herein may be downloaded from a computer readable storage medium to a respective computing/processing device, or downloaded to an external computer or external storage device over a network, such as the Internet, local area network, wide area network, and/or wireless network.
  • the network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and/or edge servers.
  • a network adapter card or a network interface in each computing/processing device receives computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in a computer-readable storage medium in each computing/processing device .
  • Computer program instructions for performing the operations of the present application may be assembly instructions, instruction set architecture (Instruction Set Architecture, ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or in one or more source or object code written in any combination of programming languages, including object-oriented programming languages—such as Smalltalk, C++, etc., and conventional procedural programming languages—such as the “C” language or similar programming languages.
  • Computer readable program instructions may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer, or entirely on the remote computer or server implement.
  • the remote computer can be connected to the user's computer through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or it can be connected to an external computer such as use an Internet service provider to connect via the Internet).
  • electronic circuits such as programmable logic circuits, field-programmable gate arrays (Field-Programmable Gate Array, FPGA) or programmable logic arrays (Programmable Logic Array, PLA), the electronic circuit can execute computer-readable program instructions, thereby realizing various aspects of the present application.
  • These computer-readable program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine such that when executed by the processor of the computer or other programmable data processing apparatus , producing an apparatus for realizing the functions/actions specified in one or more blocks in the flowchart and/or block diagram.
  • These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause computers, programmable data processing devices and/or other devices to work in a specific way, so that the computer-readable medium storing instructions includes An article of manufacture comprising instructions for implementing various aspects of the functions/acts specified in one or more blocks in flowcharts and/or block diagrams.
  • each block in a flowchart or block diagram may represent a module, a portion of a program segment, or an instruction that includes one or more Executable instructions.
  • the functions noted in the block may occur out of the order noted in the figures. For example, two blocks in succession may, in fact, be executed substantially concurrently, or they may sometimes be executed in the reverse order, depending upon the functionality involved.
  • each block in the block diagrams and/or flowcharts, and combinations of blocks in the block diagrams and/or flowcharts can be implemented with hardware (such as circuits or ASIC (Application Specific Integrated Circuit, application-specific integrated circuit)), or can be implemented with a combination of hardware and software, such as firmware.
  • hardware such as circuits or ASIC (Application Specific Integrated Circuit, application-specific integrated circuit)
  • firmware such as firmware

Abstract

La présente demande concerne un procédé et un appareil de détermination de problème de qualité vidéo, un support de stockage et un produit-programme. Le procédé de détermination de problème de qualité vidéo comprend : la réception d'informations de protocole sur un lecteur vidéo et sur un dispositif de réseau ; la détermination d'un paramètre de qualité de trame anormal et d'un paramètre de qualité de paquet anormal selon les informations de protocole ; et la détermination, selon le paramètre de qualité de trame anormal et/ou le paramètre de qualité de paquet anormal, d'une cause et d'un dispositif entraînant une qualité vidéo anormale du lecteur vidéo. Selon le procédé de détermination de problème de qualité vidéo dans les modes de réalisation de la présente demande, la cause et le dispositif entraînant un problème de qualité vidéo peuvent être déterminés de manière rapide et précise.
PCT/CN2022/078279 2021-08-27 2022-02-28 Procédé et appareil de détermination de problème de qualité vidéo, support de stockage et produit-programme WO2023024469A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202110993951.3A CN115734043A (zh) 2021-08-27 2021-08-27 视频质量问题确定方法、装置、存储介质及程序产品
CN202110993951.3 2021-08-27

Publications (1)

Publication Number Publication Date
WO2023024469A1 true WO2023024469A1 (fr) 2023-03-02

Family

ID=85290196

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/078279 WO2023024469A1 (fr) 2021-08-27 2022-02-28 Procédé et appareil de détermination de problème de qualité vidéo, support de stockage et produit-programme

Country Status (2)

Country Link
CN (1) CN115734043A (fr)
WO (1) WO2023024469A1 (fr)

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2007006194A1 (fr) * 2005-07-14 2007-01-18 Huawei Technologies Co., Ltd. Procédé et dispositif de détection de la qualité de service dans un réseau de communication
CN101909060A (zh) * 2010-08-05 2010-12-08 浙江工业大学 一种适用于移动视频实时流媒体传输的Qos控制方法
CN103024440A (zh) * 2013-01-11 2013-04-03 德科仕通信(上海)有限公司 针对has的视频质量评价和故障诊断方法
CN108156056A (zh) * 2017-12-28 2018-06-12 华为技术有限公司 网络质量测量方法及其装置
CN111918055A (zh) * 2020-06-30 2020-11-10 上海艾策通讯科技股份有限公司 视频质量异常定位方法、装置、存储介质及电子设备
CN111953961A (zh) * 2020-06-30 2020-11-17 武汉烽火众智数字技术有限责任公司 一种基于国标视频播放问题分析方法及装置

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2007006194A1 (fr) * 2005-07-14 2007-01-18 Huawei Technologies Co., Ltd. Procédé et dispositif de détection de la qualité de service dans un réseau de communication
CN101909060A (zh) * 2010-08-05 2010-12-08 浙江工业大学 一种适用于移动视频实时流媒体传输的Qos控制方法
CN103024440A (zh) * 2013-01-11 2013-04-03 德科仕通信(上海)有限公司 针对has的视频质量评价和故障诊断方法
CN108156056A (zh) * 2017-12-28 2018-06-12 华为技术有限公司 网络质量测量方法及其装置
CN111918055A (zh) * 2020-06-30 2020-11-10 上海艾策通讯科技股份有限公司 视频质量异常定位方法、装置、存储介质及电子设备
CN111953961A (zh) * 2020-06-30 2020-11-17 武汉烽火众智数字技术有限责任公司 一种基于国标视频播放问题分析方法及装置

Also Published As

Publication number Publication date
CN115734043A (zh) 2023-03-03

Similar Documents

Publication Publication Date Title
US11665384B2 (en) Method and apparatus for transmitting media data in multimedia transport system
US10063606B2 (en) Systems and methods for using client-side video buffer occupancy for enhanced quality of experience in a communication network
US9253063B2 (en) Bi-directional video compression for real-time video streams during transport in a packet switched network
US9237108B2 (en) Jitter buffer emulation for RTP streams in passive network monitoring systems
US11070327B2 (en) Method and apparatus for re-transmitting MMT packet and method and apparatus for requesting MMT packet re-transmission
US8081635B2 (en) Reconstruction of errored media streams in a communication system
US8873590B2 (en) Apparatus and method for correcting jitter
CN108989078B (zh) 一种视联网中节点设备故障检测方法和装置
CN111385221B (zh) 一种数据处理方法和通信设备
US9729903B2 (en) Data transmitting device and method
CN112601072A (zh) 视频业务质量评估的方法及装置
EP3491784B1 (fr) Estimation des pertes dans un flux vidéo
WO2016068760A1 (fr) Synchronisation de flux vidéo
CN112954433A (zh) 视频处理方法、装置、电子设备及存储介质
WO2023024469A1 (fr) Procédé et appareil de détermination de problème de qualité vidéo, support de stockage et produit-programme
US10116415B2 (en) Transmission device, receiving device, transmission method, and receiving method
US20230328308A1 (en) Synchronization of multiple content streams
JP2013121016A (ja) 品質劣化判定装置及び方法
CN112688824B (zh) Rtp丢包检测方法、装置、设备及计算机可读存储介质
WO2013098812A1 (fr) Transport sur système et procédé
CN108200481B (zh) 一种rtp-ps流处理方法、装置、设备及存储介质
CN114125493B (zh) 一种流媒体的分布式存储方法、装置和设备
WO2019082861A1 (fr) Dispositif de transmission, dispositif de réception et procédé de distribution de vidéo
CN115225542A (zh) 视频信息处理方法、装置、电子设备和存储介质
CN117014696A (zh) 基于rtp的音视频传输方法、装置、电子设备及存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22859826

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE