WO2021159329A1 - 流媒体网络时延确定方法、装置、计算机设备、可读存储介质和远程驾驶系统 - Google Patents

流媒体网络时延确定方法、装置、计算机设备、可读存储介质和远程驾驶系统 Download PDF

Info

Publication number
WO2021159329A1
WO2021159329A1 PCT/CN2020/074924 CN2020074924W WO2021159329A1 WO 2021159329 A1 WO2021159329 A1 WO 2021159329A1 CN 2020074924 W CN2020074924 W CN 2020074924W WO 2021159329 A1 WO2021159329 A1 WO 2021159329A1
Authority
WO
WIPO (PCT)
Prior art keywords
time
sending
ntp
rtp
current frame
Prior art date
Application number
PCT/CN2020/074924
Other languages
English (en)
French (fr)
Inventor
杨超
Original Assignee
深圳元戎启行科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳元戎启行科技有限公司 filed Critical 深圳元戎启行科技有限公司
Priority to CN202080003164.3A priority Critical patent/CN113519146B/zh
Priority to PCT/CN2020/074924 priority patent/WO2021159329A1/zh
Publication of WO2021159329A1 publication Critical patent/WO2021159329A1/zh

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L43/00—Arrangements for monitoring or testing data switching networks
    • H04L43/10—Active monitoring, e.g. heartbeat, ping or trace-route
    • H04L43/106—Active monitoring, e.g. heartbeat, ping or trace-route using time related information in packets, e.g. by adding timestamps
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L43/00—Arrangements for monitoring or testing data switching networks
    • H04L43/06—Generation of reports
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L43/00—Arrangements for monitoring or testing data switching networks
    • H04L43/08—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
    • H04L43/0852—Delays
    • H04L43/0858—One way delays
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/60—Network streaming of media packets
    • H04L65/61—Network streaming of media packets for supporting one-way streaming services, e.g. Internet radio
    • H04L65/612—Network streaming of media packets for supporting one-way streaming services, e.g. Internet radio for unicast
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/60—Network streaming of media packets
    • H04L65/65—Network streaming protocols, e.g. real-time transport protocol [RTP] or real-time control protocol [RTCP]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/60—Network streaming of media packets
    • H04L65/70—Media network packetisation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/80—Responding to QoS
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04J—MULTIPLEX COMMUNICATION
    • H04J3/00—Time-division multiplex systems
    • H04J3/02—Details
    • H04J3/06—Synchronising arrangements
    • H04J3/0635—Clock or time synchronisation in a network
    • H04J3/0638—Clock or time synchronisation among nodes; Internode synchronisation
    • H04J3/0658—Clock or time synchronisation among packet nodes
    • H04J3/0661—Clock or time synchronisation among packet nodes using timestamps
    • H04J3/0667—Bidirectional timestamps, e.g. NTP or PTP for compensation of clock drift and for compensation of propagation delays
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L43/00—Arrangements for monitoring or testing data switching networks
    • H04L43/16—Threshold monitoring

Definitions

  • This application relates to the field of communication technology, and in particular to a method, device, computer equipment, readable storage medium, and remote driving system for determining the delay of a streaming media network.
  • RTP Real-time Transport Protocol
  • RTCP Real-time Transport Control Protocol, Real-time Transport Control Protocol
  • the RTP protocol data packet contains a time-related field, namely Timestamp.
  • the RTCP protocol datagram contains two time-related fields, NTP timestamp and RTP timestamp. These three time-related fields are used to calculate the position of a frame of audio or image in the entire media stream, or to achieve synchronization of multiple streams.
  • NTP timestamp time-related fields
  • RTP timestamp time-related fields
  • These three time-related fields are used to calculate the position of a frame of audio or image in the entire media stream, or to achieve synchronization of multiple streams.
  • the embodiments of the present application provide a method, a device, a computer device, a readable storage medium, and a remote driving system for determining a streaming media network delay, which can determine the network delay in streaming media transmission.
  • a method for determining the delay of a streaming media network comprising:
  • the network delay of the current frame is determined according to the sending NTP time and the receiving NTP time.
  • a device for determining the delay of a streaming media network comprising:
  • a sending RTP time acquiring module configured to acquire the sending RTP time of the current frame of streaming media, where the sending RTP time refers to the sending time of the current frame acquired according to the RTP protocol data packet;
  • the report RTP time acquisition module is configured to acquire the report RTP time reported by any sender sent before the RTP time, and the report RTP time refers to the RTP time reported by the sender acquired according to the RTCP protocol;
  • the sending NTP time determining module is configured to determine the sending NTP time of the current frame according to the sending RTP time and the report RTP time.
  • the sending NTP time refers to the sending NTP time corresponding to the sending RTP time, and sending the NTP time of the current frame;
  • a receiving NTP time determining module configured to obtain the receiving NTP time of the current frame, where the receiving NTP time refers to the NTP time of receiving the current frame;
  • the delay determining module is configured to determine the network delay of the current frame according to the sending NTP time and the receiving NTP time.
  • a computer-readable storage medium has a computer program stored thereon, and when the computer program is executed by a processor, the steps of the above-mentioned method for determining the delay of a streaming media network are realized.
  • a computer device includes a memory and a processor, and computer-readable instructions are stored in the memory.
  • the processor executes the above-mentioned streaming media network delay determination method.
  • a remote driving system includes:
  • An unmanned vehicle is equipped with an image acquisition device to collect streaming media
  • An NTP server which is in communication connection with the image acquisition device
  • the remote control device is in communication connection with the image acquisition device and the NTP server.
  • the remote control device is used to obtain the sending RTP time of the current frame of the streaming media.
  • the sending RTP time refers to all the data obtained according to the RTP protocol data packet The sending time of the current frame; obtaining the reported RTP time reported by any sender sent before the sending RTP time, where the reported RTP time refers to the RTP time reported by the sender obtained according to the RTCP protocol; according to the Sending the RTP time and the report RTP time, determining the sending NTP time of the current frame, the sending NTP time refers to the NTP time of sending the current frame corresponding to the sending RTP time; acquiring the current frame
  • the receiving NTP time is the NTP time of receiving the current frame; the network delay of the current frame is determined according to the sending NTP time and the receiving NTP time.
  • the foregoing method, device, computer equipment, readable storage medium and remote driving system for determining the delay of a streaming media network are obtained by obtaining the sending RTP time of the current frame of the streaming media, and obtaining the report of any sender report sent before the sending RTP time RTP time.
  • the sending NTP time of the current frame is determined according to the sending RTP time and the report RTP time, and then the network delay can be determined according to the sending NTP time and the receiving NTP time.
  • the method, device, computer equipment, readable storage medium, and remote driving system for determining the delay of the streaming media network provided by the embodiments of the present application can realize the determination of the delay of the streaming media network and have strong practicability.
  • the sending NTP time of the streaming media can be accurately determined, so that the sending time and the receiving time are based on the same Time synchronization protocol, the determined network delay is more accurate.
  • FIG. 1 is a schematic diagram of an application scenario of a method for determining a delay of a streaming media network provided by an embodiment of the application;
  • FIG. 2 is a schematic flowchart of a method for determining a delay of a streaming media network according to an embodiment of the application
  • FIG. 3 is a schematic flowchart of a method for determining a delay of a streaming media network according to an embodiment of the application
  • FIG. 4 is a schematic flowchart of a method for determining a delay of a streaming media network according to an embodiment of the application
  • FIG. 5 is a schematic flowchart of a method for determining a delay of a streaming media network according to an embodiment of the application
  • FIG. 6 is a schematic flowchart of a method for determining a delay of a streaming media network according to an embodiment of the application
  • FIG. 7 is a schematic diagram of a framework of an apparatus for determining a delay of a streaming media network according to an embodiment of the application
  • Fig. 8 is a block diagram of the structure of a computer device provided by an embodiment of the present application.
  • the method for determining the delay of a streaming media network can be applied to the application environment shown in FIG.
  • the RTP protocol and the RTCP protocol are adopted between the receiving ends 104 to realize the transmission of streaming media data.
  • Both the sending end 102 and the receiving end 104 are in communication connection with an NTP (Network Time Protocol, Network Time Protocol) server 106 through a network.
  • the NTP server uses the NTP protocol to synchronize the time of the sender 102 and the receiver 104.
  • the sending end 102 may be an image acquisition device or the like.
  • the receiving end 104 may be, but is not limited to, various personal computers, notebook computers, smart phones, tablet computers, and portable wearable devices.
  • the NTP server 106 can be implemented as an independent server or a server cluster composed of multiple servers.
  • the method for determining the delay of a streaming media network can be applied in remote control.
  • the receiving end remotely receives the video data or audio data sent by the sending end. Due to the requirements of transmission quality or real-time requirements, it needs to be implemented. Determination of network delay.
  • the method provided in this embodiment can be applied to a remote unmanned driving system.
  • the remote unmanned driving system includes an unmanned vehicle, an image acquisition device installed in the unmanned vehicle, an NTP server 106, and a remote control device.
  • the image acquisition device is the aforementioned sending end 102
  • the remote control device is the aforementioned receiving end 104.
  • the remote control equipment communicates with unmanned and timely vehicles and image acquisition devices.
  • an embodiment of the present application provides a method for determining the delay of a streaming media network.
  • the method is applied to the receiving end in FIG. 1 as an example for description.
  • the method includes:
  • the sending RTP time refers to the sending time of the current frame obtained according to the RTP protocol data packet.
  • Streaming media can be video data, audio data, or data that contains both video and audio.
  • the current frame of streaming media refers to the video frame or audio frame being transmitted at the current moment.
  • the sender includes its own clock source, which provides time information to the data packets of the RTP protocol and RTCP.
  • the sending RTP time refers to the sending time of the current frame data determined by the sending end's own clock source.
  • the sending RTP time can be determined by the time stamp of the current frame data. At the beginning of the session, the time stamp is initialized to an initial value.
  • the unit of RTP time can be base. For example, base can be 9000, so 9000 represents 1s.
  • the sender periodically sends RTCP information to the receiver, and the RTCP information includes a sender report (Sender Rrport, SR).
  • Each sender report carries two time-related fields: NTP timestamp (timestamp) and RTP timestamp (timestamp).
  • the report RTP time refers to the RTP timestamp carried in the sender's report, which is used to characterize the sending time of the sender's report under the sender's own clock source.
  • the selection of the sender report can be any sender report before sending the RTP time. Of course, if the sending RTP time corresponds to a sender report, and at the moment of sending the current frame data, there is just one sender report to send, then send The party report may also be the sender report corresponding to the sending RTP time.
  • S30 Determine the sending NTP time of the current frame according to the sent RTP time and the reported RTP time, where the reported RTP time refers to the RTP time reported by the sender obtained according to the RTCP protocol.
  • a corresponding NTP time can be determined.
  • the sending RTP time and the reporting RTP time are both the time under the sender's own clock source. Therefore, according to the relationship between the sending RTP time and the reporting RTP time, the sending NTP time of the current frame can be determined.
  • S40 Acquire the receiving NTP time of the current frame, where the receiving NTP time refers to the NTP time of receiving the current frame.
  • the receiving NTP time is used to characterize the NTP time when the receiving end receives the current frame data.
  • the receiving end communicates with the NTP server, and the NTP server uses the NTP protocol to synchronize the time of the receiving end, and the time in the receiving end is NTP time. Get the time when the current frame arrives at the receiving end directly, and get the receiving NTP time.
  • Send RTP time comes from the timestamp field in the RTP protocol data packet
  • Sending NTP time NTP time corresponding to the RTP data packet calculated based on the sending RTP time and reporting RTP time;
  • Report RTP time comes from the RTP timestamp field in the report packet of the RTCP protocol sender
  • Receiving NTP time The current NTP time obtained when the data receiving end receives the RTP packet.
  • S50 Determine the network delay of the current frame according to the sending NTP time and the receiving NTP time.
  • the difference between the received NTP time and the sent NTP time can represent the transmission delay of the current frame data during network transmission, that is, the network delay of the current frame.
  • the NTP time is used for the sending time and the receiving time of the current frame data. Therefore, the calculated network delay is an absolute delay with high accuracy.
  • This embodiment provides a method for determining the network delay of the current frame.
  • steps S10-S50 are repeated to determine the network delay of each frame of streaming media, so that the network delay of streaming media can be determined in real time.
  • the report RTP time is obtained by obtaining the sending RTP time of the current frame of the streaming media, and obtaining the RTP time reported by any sender sent before the sending RTP time.
  • the sending NTP time of the current frame is determined according to the sending RTP time and the report RTP time, and then the network delay can be determined according to the sending NTP time and the receiving NTP time.
  • the method provided in the embodiment of the present application can realize the determination of the delay of the streaming media network, and has strong practicability.
  • the sending NTP time of the streaming media can be accurately determined, so that the sending time and the receiving time are based on the same Time synchronization protocol, the determined network delay is more accurate.
  • this embodiment relates to a possible implementation manner of determining the sending NTP time of the current frame according to the sending RTP time and the reporting RTP time.
  • S30 includes:
  • S310 Determine the time difference between the sending RTP time and the reporting RTP time, and obtain the relative time difference
  • S320 Obtain the reported NTP time corresponding to the reported RTP time, where the reported NTP time refers to the NTP time reported by the sender;
  • S330 Determine to send the NTP time according to the reported NTP time and the relative time difference.
  • the relative time difference is used to characterize the relative difference between the sending RTP time and the reporting RTP time.
  • the reported NTP time refers to the NTP time reported by the sender.
  • the reported NTP time comes from the NTP timestamp field in the RTCP protocol sender report packet.
  • each sender report carries two time-related fields, one is the NTP timestamp and the other is the RTP time, and both times are used to characterize the sending time of the sender's report. Then, the NTP time reported by the sender and the RTP time are uniquely corresponding, and the NTP time reported by the sender can be determined according to the correspondence between the two.
  • the NTP time corresponding to the sending RTP time of the current frame can be determined.
  • the relative time difference is obtained by determining the time difference between the sending RTP time and the reporting RTP time.
  • the relative time difference is obtained by the difference between two RTP times under the same clock source, so it is very accurate.
  • the NTP time and RTP time in the sender’s report are also uniquely corresponding, so the reported NTP time is highly accurate. Therefore, the transmission NTP time determined by the relative time difference and the reported NTP time is highly accurate, thereby determining the network The delay accuracy is high.
  • determining the implementation manner of sending the NTP time according to the reported NTP time and the relative time difference includes the following steps, that is, S330 includes:
  • S331 Convert the relative time difference to NTP time to obtain the NTP time difference
  • S332 Calculate the sum of the reported NTP time and the NTP time difference to obtain the sent NTP time.
  • the sending RTP time is directly different from the reporting RTP time, and the time difference under the RTP protocol is obtained. Convert this time difference to the time difference under the NTP protocol and sum it with the reported NTP time to get the sent NTP time.
  • the relative time difference is converted into the NTP time difference, which can retain the calculation accuracy and improve the accuracy of determining the NTP time difference, thereby further improving the accuracy of determining the sending NTP time and improving the accuracy of determining the delay of the streaming media network.
  • S20 includes:
  • the time synchronization between the sending end and the receiving end of the NTP server can be performed periodically, and time synchronization is performed at a certain time interval to improve the accuracy of time synchronization. In this way, the closer to the sending RTP time, the more accurate the reported NTP time corresponding to the reported RTP time.
  • the NTP time of the sender which is further determined by the RTP time reported by the closest sender, has high accuracy.
  • S50 specifically includes:
  • network delay receiving NTP time-sending NTP time.
  • this embodiment relates to a possible implementation of acquiring the sending RTP time of the current frame of the streaming media and acquiring the receiving NTP time.
  • S10 includes:
  • S120 Obtain the sending RTP time according to the time stamp of the first data packet.
  • the corresponding S40 includes:
  • the sending end When the sending end sends each frame of streaming media data to the receiving end, it packs the data of each frame into several data packets and transmits them to the receiving end in turn.
  • the receiving end receives the data packets in turn, and after all the data packets are received, the data packets are spliced to form a complete video or audio.
  • Each data packet carries a time stamp. The time of the first data packet among multiple data packets is used as the sending RTP time, and the time when all data packets are received is used as the receiving NTP time, so that the time of sending and receiving data packets one by one is also added to the network delay calculation , Further improve the accuracy of network delay calculation.
  • the method further includes:
  • the network delay of the current frame is greater than the preset threshold, it means that the transmission network delay of the current frame of streaming media does not meet the requirements, and an alarm message can be output to remind the operator, or Further control the remote control target to stop working. In this way, the intelligence of remote control can be improved, and the security of remote control targets can be improved.
  • the remote control target is an unmanned vehicle
  • the unmanned vehicle if the network delay of the current frame is greater than a preset threshold, the unmanned vehicle is controlled to stop.
  • a preset threshold When remotely controlling the driving of an unmanned vehicle, it is necessary to judge the road condition information in real time based on the image information collected by the image acquisition device. If the network delay is too large, it will cause the judgment to be untimely and control the driving state or direction of the unmanned vehicle. If it is not in time, there will be greater danger. Therefore, by judging the network delay of each frame of streaming media in real time, when the network delay is greater than the preset threshold, timely control of unmanned vehicles to stop will greatly improve remote control of unmanned vehicles. The safety of driving a vehicle.
  • Streaming media includes one of video and audio.
  • the streaming media includes video and audio.
  • the video and audio are separately packaged and transmitted.
  • the video and audio are separately determined for network delay.
  • an embodiment of the present application provides a streaming media network delay determination device 10, the device includes: a sending RTP time acquiring module 100, a report RTP time acquiring module 200, a sending NTP time determining module 300, and a receiving NTP The time determination module 400 and the time delay determination module 500. in,
  • the sending RTP time acquiring module 100 is configured to acquire the sending RTP time of the current frame of streaming media, where the sending RTP time refers to the sending time of the current frame acquired according to the RTP protocol data packet;
  • the report RTP time acquisition module 200 is configured to acquire the report RTP time reported by any sender sent before the sending RTP time, and the report RTP time refers to the RTP time reported by the sender acquired according to the RTCP protocol;
  • the sending NTP time determination module 300 is configured to determine the sending NTP time of the current frame according to the sending RTP time and the report RTP time.
  • the sending NTP time refers to the sending NTP time corresponding to the sending RTP time. State the NTP time of the current frame;
  • the receiving NTP time determining module 400 is configured to obtain the receiving NTP time of the current frame, where the receiving NTP time refers to the NTP time of receiving the current frame;
  • the delay determining module 500 is configured to determine the network delay of the current frame according to the sending NTP time and the receiving NTP time.
  • the report RTP time acquisition module 200 includes a relative time difference determination unit, a report NTP time determination unit, and a sending NTP time determination unit.
  • the relative time difference determining unit is configured to determine the time difference between the sent RTP time and the reported RTP time to obtain the relative time difference;
  • the report NTP time determining unit is configured to obtain the reported NTP time corresponding to the reported RTP time, and the reported NTP time Refers to the NTP time reported by the sender;
  • the sending NTP time determining unit is configured to determine the sending NTP time according to the reported NTP time and the relative time difference.
  • the sending NTP time determining unit is used to convert the relative time difference to an NTP time to obtain the NTP time difference; calculate the sum of the reported NTP time and the NTP time difference to obtain the sending NTP time.
  • the sending NTP time determination unit is used to shift the relative time difference to the left by 32 bits to obtain the obtained NTP time difference.
  • the transmission time determination unit for NTP product relative time difference and the calculated 232, to obtain the NTP time difference In one embodiment, the transmission time determination unit for NTP product relative time difference and the calculated 232, to obtain the NTP time difference.
  • the report RTP time acquisition module 200 is configured to acquire the RTP time reported by the sender closest to the sent RTP time before the sent RTP time to obtain the reported RTP time.
  • the delay determining module 500 is used to calculate the difference between the receiving NTP time and the sending NTP time to obtain the network delay of the current frame.
  • the sending RTP time acquiring module 100 is used to acquire the time stamp of the first data packet in the data packet of the current frame; according to the time stamp of the first data packet, the sending RTP time is obtained .
  • the receiving NTP time determining module 400 obtains the NTP time when all the data packets of the current frame are received to obtain the receiving NTP time.
  • the streaming media network delay determining device 10 further includes an alarm information output module 600 for controlling the remote control target to stop working if the network delay of the current frame is greater than a preset threshold. .
  • the remote control target is an unmanned vehicle
  • the streaming media network delay determining device 10 further includes a control module 700 for controlling the unmanned vehicle to park.
  • the streaming media includes at least one of video and audio.
  • the division of the modules in the streaming media network delay determining device 10 is only for illustration. In other embodiments, the streaming media network delay determining device 10 may be divided into different modules as needed to complete the streaming. The media network delay determines all or part of the functions of the apparatus 10.
  • the various modules in the apparatus 10 for determining the delay of a streaming media network may be implemented in whole or in part by software, hardware, and a combination thereof.
  • the above-mentioned modules may be embedded in the form of hardware or independent of the processor in the computer equipment, or may be stored in the memory of the computer equipment in the form of software, so that the processor can call and execute the corresponding operations of the above-mentioned modules.
  • the computer equipment includes a processor, a memory, and a display screen connected through a system bus.
  • the processor is used to provide calculation and control capabilities to support the operation of the entire computer equipment.
  • the memory is used to store data, programs, and/or instruction codes, etc., and at least one computer program is stored on the memory, and the computer program can be executed by the processor to implement the streaming media network suitable for computer equipment provided in the embodiments of the present application. Delay determining method.
  • the memory may include non-volatile storage media such as magnetic disks, optical disks, read-only memory (Read-Only Memory, ROM), or random-access-memory (Random-Access-Memory, RAM).
  • the memory includes a non-volatile storage medium and internal memory.
  • the non-volatile storage medium stores an operating system, a database, and a computer program.
  • the database stores data related to the method for determining the delay of a streaming media network provided by the above embodiments, for example, information such as the name of each process or application can be stored.
  • the computer program can be executed by a processor to implement a streaming media network delay determination method provided by each embodiment of the present application.
  • the internal memory provides a cached operating environment for the operating system, database and computer program in the non-volatile storage medium.
  • the display screen can be a touch screen, such as a capacitive screen or an electronic screen, which is used to display the interface information of the application corresponding to the foreground process. It can also be used to detect touch operations on the display screen and generate corresponding instructions, such as performing front-end and back-end operations. Application switching instructions, etc.
  • FIG. 8 is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied.
  • the specific computer device may Including more or fewer parts than shown in the figure, or combining some parts, or having a different arrangement of parts.
  • the computer device also includes a network interface connected through a system bus.
  • the network interface may be an Ethernet card or a wireless network card, etc., for communicating with external computer equipment, for example, for communicating with a server.
  • the network delay of the current frame is determined according to the sending NTP time and the receiving NTP time.
  • the processor further implements the following steps when executing the computer program: determining the time difference between the sending RTP time and the reporting RTP time to obtain the relative time difference; acquiring the reporting NTP time corresponding to the reporting RTP time,
  • the reported NTP time refers to the NTP time reported by the sender; the sending NTP time is determined according to the difference between the reported NTP time and the relative time.
  • the processor further implements the following steps when executing the computer program: converting the relative time difference to NTP time to obtain the NTP time difference; calculating the sum of the reported NTP time and the NTP time difference to obtain the sending NTP time.
  • the processor further implements the following steps when executing the computer program: shifting the relative time difference to the left by 32 bits to obtain the obtained NTP time difference.
  • the processor further implements the following steps when executing the computer program: calculating the product of the relative time difference and 2 32 to obtain the NTP time difference.
  • the processor further implements the following steps when executing the computer program: obtaining the RTP time reported by the sender closest to the sending RTP time before the sending RTP time, and obtaining the reported RTP time.
  • the processor further implements the following step when executing the computer program: calculating the difference between the receiving NTP time and the sending NTP time to obtain the network delay of the current frame.
  • the processor further implements the following steps when executing the computer program: acquiring the time stamp of the first data packet in the data packet of the current frame; obtaining the time stamp of the first data packet according to the time stamp of the first data packet Send RTP time.
  • the processor further implements the following steps when executing the computer program: obtaining the NTP time when all the data packets of the current frame are received, and obtaining the received NTP time.
  • the processor further implements the following steps when executing the computer program: if the network delay of the current frame is greater than a preset threshold, output an alarm message.
  • the processor further implements the following step when executing the computer program: if the network delay of the current frame is greater than the preset threshold, control the remote control target to stop working.
  • the remote control target is an unmanned vehicle
  • the processor further implements the following steps when executing the computer program: controlling the unmanned vehicle to park.
  • the streaming media includes at least one of video and audio.
  • the embodiment of the present application also provides a computer-readable storage medium.
  • One or more non-volatile computer-readable storage media containing computer-executable instructions when the computer-executable instructions are executed by one or more processors, cause the processors to perform the following steps of the streaming media network delay determination method:
  • the network delay of the current frame is determined according to the sending NTP time and the receiving NTP time.
  • the following steps are further implemented: determining the time difference between the sending RTP time and the reporting RTP time to obtain the relative time difference; obtaining the reporting NTP time corresponding to the reporting RTP time, so The reported NTP time refers to the NTP time reported by the sender; the sending NTP time is determined according to the difference between the reported NTP time and the relative time.
  • the following steps are further implemented: the relative time difference is converted to NTP time to obtain the NTP time difference; the sum of the reported NTP time and the NTP time difference is calculated to obtain the transmission NTP time.
  • the following steps are further implemented: shift the relative time difference to the left by 32 bits to obtain the obtained NTP time difference.
  • the following steps are further implemented: calculating the product of the relative time difference and 2 32 to obtain the NTP time difference.
  • the following steps are further implemented: before the sending RTP time, the RTP time reported by the sender closest to the sending RTP time is obtained, and the reported RTP time is obtained.
  • the following steps are further implemented: calculating the difference between the receiving NTP time and the sending NTP time to obtain the network delay of the current frame.
  • the following steps are also implemented: obtaining the time stamp of the first data packet in the data packet of the current frame; and obtaining all data packets according to the time stamp of the first data packet The sending RTP time.
  • the following steps are further implemented: obtaining the NTP time when all the data packets of the current frame are received, and obtaining the receiving NTP time.
  • the following steps are further implemented: if the network delay of the current frame is greater than a preset threshold, output alarm information.
  • the following steps are further implemented: if the network delay of the current frame is greater than the preset threshold, the remote control target is controlled to stop working.
  • the remote control target is an unmanned vehicle
  • the computer program when executed by the processor, the following steps are further implemented: controlling the unmanned vehicle to park.
  • the streaming media includes at least one of video and audio.
  • An embodiment of the present application also provides a remote driving system, which includes an unmanned vehicle, an NTP server, and a remote control device.
  • the unmanned vehicle is equipped with an image acquisition device for acquiring the sending RTP time of the current frame of the streaming media.
  • the sending RTP time refers to the sending time of the current frame acquired according to the RTP protocol data packet; acquiring the sending RTP time
  • the previously sent report RTP time reported by any sender the reported RTP time refers to the RTP time reported by the sender obtained according to the RTCP protocol; the RTP time is determined according to the sent RTP time and the reported RTP time
  • the sending NTP time of the current frame the sending NTP time refers to the sending NTP time corresponding to the sending RTP time, the sending NTP time of the current frame; acquiring the receiving NTP time of the current frame, the receiving NTP time refers to receiving The NTP time of the current frame; the network delay of the current frame is determined according to the sending NTP time and the receiving NTP time.
  • the remote control device is further configured to control the unmanned vehicle to stop if the network delay of the current frame is greater than a preset threshold.
  • the above remote control device includes a memory and a processor, and computer-readable instructions are stored in the memory. step.
  • Non-volatile memory may include read only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory.
  • Volatile memory may include random access memory (RAM), which acts as external cache memory.
  • RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous Link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).
  • SRAM static RAM
  • DRAM dynamic RAM
  • SDRAM synchronous DRAM
  • DDR SDRAM double data rate SDRAM
  • ESDRAM enhanced SDRAM
  • SLDRAM synchronous Link (Synchlink) DRAM
  • Rambus direct RAM
  • DRAM direct memory bus dynamic RAM
  • RDRAM memory bus dynamic RAM

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Health & Medical Sciences (AREA)
  • Cardiology (AREA)
  • General Health & Medical Sciences (AREA)
  • Environmental & Geological Engineering (AREA)
  • Data Exchanges In Wide-Area Networks (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)

Abstract

一种流媒体网络时延确定方法、装置、计算机设备、可读存储介质和远程驾驶系统，流媒体网络时延确定方法包括：获取流媒体当前帧的发送RTP时间；获取所述发送RTP时间之前发送的任一个发送方报告的报告RTP时间；根据所述发送RTP时间与所述报告RTP时间，确定所述当前帧的发送NTP时间；获取所述当前帧的接收NTP时间；根据所述发送NTP时间和所述接收NTP时间确定所述当前帧的网络时延。本申请提供的流媒体网络时延确定方法实用性强。

Description

流媒体网络时延确定方法、装置、计算机设备、可读存储介质和远程驾驶系统 技术领域
本申请涉及通信技术领域，特别是涉及流媒体网络时延确定方法、装置、计算机设备、可读存储介质和远程驾驶系统。
背景技术
RTP(Real-time Transport Protocol，实时传输协议)和RTCP(Real-time Transport Control Protocol，实时传输控制协议)是两种网络传输协议，广泛应用于流媒体的传输。
RTP协议数据包中包含一个时间相关字段，即Timestamp。而RTCP协议数据报中包含两个时间相关字段，为NTP时间戳和RTP时间戳。这三种时间相关字段，用于计算一帧音频或图像在整个媒体流中的位置，或者用于实现多码流的同步。然而，实际应用中，实时确定媒体流网络时延也非常重要，例如：通过视频的网络时延情况实时了解视频传输质量；又例如，在远程控制无人驾驶车辆时，需要通过视频传输的网络时延判断远程操作的安全性。
相关技术中，并没有规定计算流媒体网络时延的方法。
发明内容
本申请实施例提供一种流媒体网络时延确定方法、装置、计算机设备、可读存储介质和远程驾驶系统，可以确定出流媒体传输中的网络时延。
一种流媒体网络时延确定方法，所述方法包括：
获取流媒体当前帧的发送RTP时间，所述发送RTP时间是指根据RTP协议数据包获取的所述当前帧的发送时间；
获取所述发送RTP时间之前发送的任一个发送方报告的报告RTP时间，所述报告RTP时间是指根据RTCP协议获取的所述发送方报告的RTP时间；
根据所述发送RTP时间与所述报告RTP时间，确定所述当前帧的发送NTP时间，所述发送NTP时间是指与所述发送RTP时间对应的，发送所述 当前帧的NTP时间；
获取所述当前帧的接收NTP时间，所述接收NTP时间是指接收所述当前帧的NTP时间；
根据所述发送NTP时间和所述接收NTP时间确定所述当前帧的网络时延。
一种流媒体网络时延确定装置，所述装置包括：
发送RTP时间获取模块，用于获取流媒体当前帧的发送RTP时间，所述发送RTP时间是指根据RTP协议数据包获取的所述当前帧的发送时间；
报告RTP时间获取模块，用于获取所述发送RTP时间之前发送的任一个发送方报告的报告RTP时间，所述报告RTP时间是指根据RTCP协议获取的所述发送方报告的RTP时间；
发送NTP时间确定模块，用于根据所述发送RTP时间与所述报告RTP时间，确定所述当前帧的发送NTP时间，所述发送NTP时间是指与所述发送RTP时间对应的，发送所述当前帧的NTP时间；
接收NTP时间确定模块，用于获取所述当前帧的接收NTP时间，所述接收NTP时间是指接收所述当前帧的NTP时间；
时延确定模块，用于根据所述发送NTP时间和所述接收NTP时间确定所述当前帧的网络时延。
一种计算机可读存储介质，其上存储有计算机程序，所述计算机程序被处理器执行时实现上述的流媒体网络时延确定方法的步骤。
一种计算机设备，包括存储器及处理器，所述存储器中储存有计算机可读指令，所述指令被所述处理器执行时，使得所述处理器执行上述的流媒体网络时延确定方法。
一种远程驾驶系统，包括：
无人驾驶车辆，设置有图像采集装置，用于采集流媒体；
NTP服务器，与所述图像采集装置通信连接；
远程控制设备，与所述图像采集装置及所述NTP服务器通信连接，所述远程控制设备用于获取流媒体当前帧的发送RTP时间，所述发送RTP时间是指根据RTP协议数据包获取的所述当前帧的发送时间；获取所述发送RTP时间之前发送的任一个发送方报告的报告RTP时间，所述报告RTP时间是指根 据RTCP协议获取的所述发送方报告的RTP时间；根据所述发送RTP时间与所述报告RTP时间，确定所述当前帧的发送NTP时间，所述发送NTP时间是指与所述发送RTP时间对应的，发送所述当前帧的NTP时间；获取所述当前帧的接收NTP时间，所述接收NTP时间是指接收所述当前帧的NTP时间；根据所述发送NTP时间和所述接收NTP时间确定所述当前帧的网络时延。
上述流媒体网络时延确定方法、装置、计算机设备、可读存储介质和远程驾驶系统，通过获取流媒体当前帧的发送RTP时间，并获取此发送RTP时间之前发送的任一个发送方报告的报告RTP时间。根据发送RTP时间和报告RTP时间确定出当前帧的发送NTP时间，进而可以根据发送NTP时间和接收NTP时间确定出网络时延。本申请实施例提供的所述流媒体网络时延确定方法、装置、计算机设备、可读存储介质和远程驾驶系统能够实现流媒体网络时延的确定，实用性强。同时，基于RTP与RTCP协议之间的关系，根据流媒体的发送RTP时间和发送方报告的RTP时间，能够准确的确定出流媒体的发送NTP时间，从而使得发送时间和接收时间均是基于同一时间同步协议，确定出的网络时延更加准确。
附图说明
图1为本申请一个实施例提供的流媒体网络时延确定方法的应用场景示意图；
图2为本申请一个实施例提供的流媒体网络时延确定方法流程示意图；
图3为本申请一个实施例提供的流媒体网络时延确定的方法流程示意图；
图4为本申请一个实施例提供的流媒体网络时延确定方法流程示意图；
图5为本申请一个实施例提供的流媒体网络时延确定方法流程示意图；
图6为本申请一个实施例提供的流媒体网络时延确定方法流程示意图；
图7为本申请一个实施例提供的流媒体网络时延确定装置的框架示意图；
图8为与本申请实施例提供的计算机设备结构的框图。
具体实施方式
为了便于理解本发明，下面将参照相关附图对本发明进行更全面的描述。附图中给出了本发明的较佳实施例。但是，本发明可以以许多不同的形式来实现，并不限于本文所描述的实施例。相反地，提供这些实施例的目的是使对本发明的公开内容的理解更加透彻全面。
请参见图1，本申请实施例提供的流媒体网络时延确定方法可以应用于如图1所示的应用环境中，其中，发送端102和接收端104通过网络通信连接，且发射端102和接收端104之间采用RTP协议和RTCP协议实现流媒体数据的传输。发送端102和接收端104均通过网络与NTP(Network Time Protocol，网络时间协议)服务器106通信连接。NTP服务器采用NTP协议对发送端102和接收端104进行时间同步。其中，发送端102可以为图像采集设备等。接收端104可以但不限于是各种个人计算机、笔记本电脑、智能手机、平板电脑和便携式可穿戴设备。NTP服务器106可以用独立的服务器或者是多个服务器组成的服务器集群来实现。
在一些实施例中，流媒体网络时延确定方法可以应用于远程控制中，接收端远程接收发送端发送的视频数据或音频数据，出于对传输质量的要求，或者实时性的要求，需要进行网络时延的确定。
在一些具体的实施例中，本实施例提供的方法可以应用于远程无人驾驶系统。远程无人驾驶系统包括无人驾驶车辆、设置于无人驾驶车辆的图像采集装置、NTP服务器106和远程控制设备。其中，图像采集装置即为上述发送端102，远程控制设备即为上述接收端104。远程控制设备与无人及时车辆及图像采集装置通信连接。
请参见图2，本申请一个实施例提供一种流媒体网络时延确定方法。本实施例以所述方法应用于图1中的接收端为例进行说明。所述方法包括：
S10，获取流媒体当前帧的发送RTP时间，发送RTP时间是指根据RTP协议数据包获取的当前帧的发送时间。
流媒体可以是视频数据，可以是音频数据，也可以是同时包含视频和音频的数据。流媒体的当前帧是指当前时刻正在传输的视频帧或音频帧。发送端内部包括一自身的时钟源，向RTP协议和RTCP的数据包提供时间信息。发送RTP时间是指发送端自身的时钟源确定的当前帧数据的发送时刻。发送RTP时间可以通过当前帧数据的时间戳确定，在会话开始时，该时间戳被初始化 成为一个初始值。RTP时间的单位可以为base，例如，base可以为9000，则，9000代表1s。
S20，获取发送RTP时间之前发送的任一个发送方报告的报告RTP时间，报告RTP时间是指根据RTCP协议获取的发送方报告的RTP时间。
发送端定期向接收端发送RTCP信息，RTCP信息中包括发送方报告(Sender Rrport，SR)。每个发送方报告中携带两个时间相关字段：NTP timestamp(时间戳)和RTP timestamp(时间戳)。报告RTP时间是指发送方报告携带的RTP时间戳，用于表征该发送方报告在发送端自身时钟源下的发送时刻。发送方报告的选取可以是发送RTP时间之前任一个发送方报告，当然，如果发送RTP时间刚好对应一个发送方报告，及在当前帧数据的发送时刻，刚好有一个发送方报告发送，那么，发送方报告也可以是该发送RTP时间对应的发送方报告。
S30，根据发送RTP时间与报告RTP时间，确定当前帧的发送NTP时间，所述报告RTP时间是指根据RTCP协议获取的所述发送方报告的RTP时间。
根据报告RTP时间，可以确定出一个对应的NTP时间。而发送RTP时间与报告RTP时间均为发送端的自身时钟源下的时间。所以，根据发送RTP时间与报告RTP时间的关系，可以确定出当前帧的发送NTP时间。
S40，获取当前帧的接收NTP时间，所述接收NTP时间是指接收所述当前帧的NTP时间。
接收NTP时间用于表征接收端接收当前帧数据时的NTP时刻。接收端与NTP服务器通信连接，NTP服务器采用NTP协议对接收端进行时间同步，接收端中的时间均为NTP时间。直接获取当前帧到达接收端的时刻，即得到接收NTP时间。
为了更好的区分发送RTP时间、发送NTP时间、报告RTP时间和接收NTP时间，以下统一对各个时间的来源进行说明：
发送RTP时间：来源于RTP协议数据包里的timestamp字段；
发送NTP时间：根据发送RTP时间和报告RTP时间计算得到的，RTP数据包对应的NTP时间；
报告RTP时间：来源于RTCP协议发送方报告数据包里的RTP timestamp字段；
接收NTP时间：数据接收端收到RTP数据包时获取的当前NTP时间。
S50，根据发送NTP时间和接收NTP时间确定当前帧的网络时延。
接收NTP时间与发送NTP时间的差值，即可表征当前帧数据在网络传输过程中传输时延，即当前帧的网络时延。对于当前帧数据的发送时间和接收时间都采用NTP时间，因此，计算得到的网络时延为绝对时延，精确度高。
本实施例提供的是当前帧网络时延的确定方法，使用时，重复步骤S10-S50，可以确定每一帧流媒体的网络时延，从而可以实时确定出流媒体的网络时延。
本实施例中，通过获取流媒体当前帧的发送RTP时间，并获取此发送RTP时间之前发送的任一个发送方报告的RTP时间，得到报告RTP时间。根据发送RTP时间和报告RTP时间确定出当前帧的发送NTP时间，进而可以根据发送NTP时间和接收NTP时间确定出网络时延。本申请实施例提供的所述方法能够实现流媒体网络时延的确定，实用性强。同时，基于RTP与RTCP协议之间的关系，根据流媒体的发送RTP时间和发送方报告的RTP时间，能够准确的确定出流媒体的发送NTP时间，从而使得发送时间和接收时间均是基于同一时间同步协议，确定出的网络时延更加准确。
请参见图3，本实施例涉及根据发送RTP时间与报告RTP时间，确定当前帧的发送NTP时间的一种可能的实现方式。在一个实施例中，S30包括：
S310，确定发送RTP时间与报告RTP时间的时间差，得到相对时间差；
S320，获取报告RTP时间对应的报告NTP时间，报告NTP时间是指发送方报告的NTP时间；
S330，根据报告NTP时间与相对时间差，确定发送NTP时间。
相对时间差用于表征发送RTP时间与报告RTP时间的相对差。报告NTP时间是指发送方报告的NTP时间，报告NTP时间来源于RTCP协议发送方报告数据包里的NTP timestamp字段。如上所述，每个发送方报告携带两个时间相关字段，一个为NTP时间戳，一个为RTP时间，两个时间均用于表征发送方报告的发送时间。那么，发送方报告的NTP时间与RTP时间是唯一对应的，根据二者的对应关系，可以确定出发送方报告的NTP时间。报告NTP时间和相对时间差已知，那么可以确定出当前帧的发送RTP时间对应的NTP时间。本实施例中，通过确定发送RTP时间与报告RTP时间的时间差，得到 相对时间差，相对时间差是同一时钟源下的两个RTP时间做差获得，因此非常准确。同时，发送方报告中的NTP时间和RTP时间也是唯一对应，因此得到的报告NTP时间准确性高，所以，通过相对时间差和报告NTP时间确定出的发送NTP时间准确性高，从而确定出的网络时延准确性高。
请参见图4，在一些实施例中，根据报告NTP时间与相对时间差，确定发送NTP时间的实现方式，包括如下步骤，即S330包括：
S331，将相对时间差转换为NTP时间，得到NTP时间差；
S332，计算报告NTP时间与NTP时间差的和，得到发送NTP时间。
发送RTP时间与报告RTP时间直接作差，得到的RTP协议下的时间差。将此时间差转换为NTP协议下的时间差，并与报告NTP时间求和，即可得到发送NTP时间。
将相对时间差转换为NTP时间差的方式有多种，在一个实施例中，可以通过将相对时间差左移32位，得到NTP时间差，即：NTP时间差＝(发送RTP时间-报告RTP时间)<<32/base。在另一些实施例中，还可以通过计算相对时间差与2 32的乘积，得到NTP时间差，即：NTP时间差＝(发送RTP时间-报告RTP时间)*2 32/base。通过这样的方式将相对时间差转换为NTP时间差，能够保留运算精度，提高NTP时间差确定的准确性，从而进一步提高发送NTP时间确定的准确性，提高流媒体网络时延确定的准确性。
本实施例涉及获取发送RTP时间之前发送的任一个发送方报告的报告RTP时间的一种可能的实现方式，在一个实施例中，S20包括：
获取发送RTP时间之前，与发送RTP时间最接近的一个发送方报告的RTP时间，得到报告RTP时间。
NTP服务器对发送端和接收端的时间同步，可以是周期性进行，隔一定时间进行一次时间同步，以提高时间同步的准确性。如此，与发送RTP时间越接近，报告RTP时间对应的报告NTP时间就越准确。通过最接近的一个发送方报告的RTP时间进一步确定的发送方NTP时间准确性高。
本实施例涉及根据发送NTP时间和接收NTP时间确定当前帧的网络时延的一种可能的实现方式，在一个实施例中，S50具体包括：
计算接收NTP时间与发送NTP时间的差，得到当前帧的网络时延。
即：网络时延＝接收NTP时间-发送NTP时间。
请参见图5，本实施例涉及获取流媒体当前帧的发送RTP时间，以及获取接收NTP时间的一种可能的实现方式，S10包括：
S110，获取当前帧的数据包中第一个数据包的时间戳；
S120，根据第一个数据包的时间戳，得到发送RTP时间。
对应的S40包括：
获取接收完当前帧所有数据包时的NTP时间，得到接收NTP时间。
发送端将每一帧流媒体数据发送至接收端时，会将每一帧的数据打包为若干个数据包，并依次传输至接收端。接收端依次接收数据包，接收完所有数据包后，再将数据包进行拼接，形成完整的视频或音频。每个数据包中携带时间戳。将多个数据包中的第一个数据包的时间作为发送RTP时间，并将接收完所有数据包时的时间作为接收NTP时间，这样将数据包逐个发送和接收的时间也加入网络延时计算中，进一步提高了网络延时计算的准确性。
请参见图6，在一个实施例中，所述方法还进一步包括：
S60，若当前帧的网络时延大于预设阈值，则输出报警信息；
S70，若当前帧的网络时延大于预设阈值，控制远程控制目标停止工作。
以本实施例提供的方法应用于远程控制为例，若当前帧的网络时延大于预设阈值，则说明流媒体当前帧的传输网络时延不满足要求，可以输出报警信息提示操作人员，或者进一步控制远程控制目标停止工作。这样可以提高远程控制的智能性，且能够提高远程控制目标的安全性。
在一些实施例中，远程控制目标为无人驾驶车辆，若当前帧的网络时延大于预设阈值，控制无人驾驶车辆停车。远程控制无人驾驶车辆行驶时，需要实时根据图像采集设备采集的图像信息判断路况信息等，若网络延迟过大，则会造成判断的不及时，以及对无人驾驶车辆行驶状态或方向控制的不及时，会存在较大的危险，因此通过实时判断流媒体每一帧的网络时延，在网络时延大于预设阈值时，及时控制无人驾驶车辆靠边停车，会大大提高远程控制无人驾驶车辆行进的安全性。
流媒体包括视频和音频中的一种。在一个实施例中，流媒体即包括视频，还包括音频，则传输时，将视频和音频分别进行打包传输，参照上述实施例提供的方法，对视频和音频分别进行网络时延的确定。
应该理解的是，虽然流程图中的各个步骤按照箭头的指示依次显示，但 是这些步骤并不是必然按照箭头指示的顺序依次执行。除非本文中有明确的说明，这些步骤的执行并没有严格的顺序限制，这些步骤可以以其它的顺序执行。而且，图中的至少一部分步骤可以包括多个子步骤或者多个阶段，这些子步骤或者阶段并不必然是在同一时刻执行完成，而是可以在不同的时刻执行，这些子步骤或者阶段的执行顺序也不必然是依次进行，而是可以与其它步骤或者其它步骤的子步骤或者阶段的至少一部分轮流或者交替地执行。
请参见图7，本申请一个实施例提供一种流媒体网络时延确定装置10，所述装置包括：发送RTP时间获取模块100、报告RTP时间获取模块200、发送NTP时间确定模块300、接收NTP时间确定模块400和时延确定模块500。其中，
发送RTP时间获取模块100，用于获取流媒体当前帧的发送RTP时间，所述发送RTP时间是指根据RTP协议数据包获取的所述当前帧的发送时间；
报告RTP时间获取模块200，用于获取所述发送RTP时间之前发送的任一个发送方报告的报告RTP时间，所述报告RTP时间是指根据RTCP协议获取的所述发送方报告的RTP时间；
发送NTP时间确定模块300，用于根据所述发送RTP时间与所述报告RTP时间，确定所述当前帧的发送NTP时间，所述发送NTP时间是指与所述发送RTP时间对应的，发送所述当前帧的NTP时间；
接收NTP时间确定模块400，用于获取所述当前帧的接收NTP时间，所述接收NTP时间是指接收所述当前帧的NTP时间；
时延确定模块500，用于根据所述发送NTP时间和所述接收NTP时间确定所述当前帧的网络时延。
在一个实施例中，报告RTP时间获取模块200包括相对时间差确定单元、报告NTP时间确定单元和发送NTP时间确定单元。相对时间差确定单元用于确定所述发送RTP时间与所述报告RTP时间的时间差，得到相对时间差；报告NTP时间确定单元，用于获取所述报告RTP时间对应的报告NTP时间，所述报告NTP时间是指所述发送方报告的NTP时间；发送NTP时间确定单元，用于根据所述报告NTP时间与所述相对时间差，确定所述发送NTP时间。
在一个实施例中，发送NTP时间确定单元用于将所述相对时间差转换为 NTP时间，得到NTP时间差；计算所述报告NTP时间与所述NTP时间差的和，得到所述发送NTP时间。
在一个实施例中，发送NTP时间确定单元用于将所述相对时间差左移32位，得到所述得到NTP时间差。
在一个实施例中，发送NTP时间确定单元用于计算所述相对时间差与2 32的乘积，得到所述NTP时间差。
在一个实施例中，报告RTP时间获取模块200用于获取所述发送RTP时间之前，与所述发送RTP时间最接近的一个发送方报告的RTP时间，得到所述报告RTP时间。
在一个实施例中，时延确定模块500用于计算所述接收NTP时间与所述发送NTP时间的差，得到所述当前帧的网络时延。
在一个实施例中，发送RTP时间获取模块100用于获取所述当前帧的数据包中第一个数据包的时间戳；根据所述第一个数据包的时间戳，得到所述发送RTP时间。
在一个实施例中，接收NTP时间确定模块400获取接收完所述当前帧的所有的数据包时的NTP时间，得到所述接收NTP时间。
请继续参见图7，在一个实施例中，流媒体网络延时确定装置10还包括报警信息输出模块600，用于若所述当前帧的网络时延大于预设阈值，控制远程控制目标停止工作。
在一个实施例中，所述远程控制目标为无人驾驶车辆，流媒体网络延时确定装置10还包括控制模块700，用于控制所述无人驾驶车辆停车。
在一个实施例中，所述流媒体包括视频和音频中的至少一种。
上述实施例提供的所述流媒体网络时延确定装置10，其实现原理和技术效果与上述方法实施例类似，在此不再赘述。
上述流媒体网络时延确定装置10中各个模块的划分仅用于举例说明，在其他实施例中，可将所述流媒体网络时延确定装置10按照需要划分为不同的模块，以完成上述流媒体网络时延确定装置10的全部或部分功能。
上述流媒体网络时延确定装置10中的各个模块可全部或部分通过软件、硬件及其组合来实现。上述各模块可以硬件形式内嵌于或独立于计算机设备中的处理器中，也可以以软件形式存储于计算机设备中的存储器中，以便于 处理器调用执行以上各个模块对应的操作。
请参见图8所示，在一个实施例中，提供了一种计算机设备的内部结构示意图。该计算机设备包括通过系统总线连接的处理器、存储器和显示屏。其中，该处理器用于提供计算和控制能力，支撑整个计算机设备的运行。存储器用于存储数据、程序、和/或指令代码等，存储器上存储至少一个计算机程序，该计算机程序可被处理器执行，以实现本申请实施例中提供的适用于计算机设备的流媒体网络时延确定方法。存储器可包括磁碟、光盘、只读存储记忆体(Read-Only Memory，ROM)等非易失性存储介质，或随机存储记忆体(Random-Access-Memory，RAM)等。例如，在一个实施例中，存储器包括非易失性存储介质及内存储器。非易失性存储介质存储有操作系统、数据库和计算机程序。该数据库中存储有用于实现以上各个实施例所提供的一种流媒体网络时延确定方法相关的数据，比如可存储有每个进程或应用的名称等信息。该计算机程序可被处理器所执行，以用于实现本申请各个实施例所提供的一种流媒体网络时延确定方法。内存储器为非易失性存储介质中的操作系统、数据库和计算机程序提供高速缓存的运行环境。显示屏可以是触摸屏，比如为电容屏或电子屏，用于显示前台进程对应的应用的界面信息，还可以被用于检测作用于该显示屏的触摸操作，生成相应的指令，比如进行前后台应用的切换指令等。
本领域技术人员可以理解，图8中示出的结构，仅仅是与本申请方案相关的部分结构的框图，并不构成对本申请方案所应用于其上的计算机设备的限定，具体的计算机设备可以包括比图中所示更多或更少的部件，或者组合某些部件，或者具有不同的部件布置。如该计算机设备还包括通过系统总线连接的网络接口，网络接口可以是以太网卡或无线网卡等，用于与外部的计算机设备进行通信，比如可用于同服务器进行通信。
在本申请实施例中，该计算机设备所包括的处理器执行存储在存储器上的计算机程序时实现以下步骤：
获取流媒体当前帧的发送RTP时间，所述发送RTP时间是指根据RTP协议数据包获取的所述当前帧的发送时间；
获取所述发送RTP时间之前发送的任一个发送方报告的报告RTP时间，所述报告RTP时间是指根据RTCP协议获取的所述发送方报告的RTP时间；
根据所述发送RTP时间与所述报告RTP时间，确定所述当前帧的发送NTP时间，所述发送NTP时间是指与所述发送RTP时间对应的，发送所述当前帧的NTP时间；
获取接收所述当前帧的接收NTP时间，所述接收NTP时间是指接收所述当前帧的NTP时间；
根据所述发送NTP时间和所述接收NTP时间确定所述当前帧的网络时延。
在一个实施例中，处理器执行计算机程序时还实现以下步骤：确定所述发送RTP时间与所述报告RTP时间的时间差，得到相对时间差；获取所述报告RTP时间对应的报告NTP时间，所述报告NTP时间是指所述发送方报告的NTP时间；根据所述报告NTP时间与所述相对时间差，确定所述发送NTP时间。
在一个实施例中，处理器执行计算机程序时还实现以下步骤：将所述相对时间差转换为NTP时间，得到NTP时间差；计算所述报告NTP时间与所述NTP时间差的和，得到所述发送NTP时间。
在一个实施例中，处理器执行计算机程序时还实现以下步骤：将所述相对时间差左移32位，得到所述得到NTP时间差。
在一个实施例中，处理器执行计算机程序时还实现以下步骤：计算所述相对时间差与2 32的乘积，得到所述NTP时间差。
在一个实施例中，处理器执行计算机程序时还实现以下步骤：获取所述发送RTP时间之前，与所述发送RTP时间最接近的一个发送方报告的RTP时间，得到所述报告RTP时间。
在一个实施例中，处理器执行计算机程序时还实现以下步骤：计算所述接收NTP时间与所述发送NTP时间的差，得到所述当前帧的网络时延。
在一个实施例中，处理器执行计算机程序时还实现以下步骤：获取所述当前帧的数据包中第一个数据包的时间戳；根据所述第一个数据包的时间戳，得到所述发送RTP时间。
在一个实施例中，处理器执行计算机程序时还实现以下步骤：获取接收完所述当前帧的所有的数据包时的NTP时间，得到所述接收NTP时间。
在一个实施例中，处理器执行计算机程序时还实现以下步骤：若所述当 前帧的网络时延大于预设阈值，则输出报警信息。
在一个实施例中，处理器执行计算机程序时还实现以下步骤：若所述当前帧的网络时延大于所述预设阈值，控制远程控制目标停止工作。
在一个实施例中，所述远程控制目标为无人驾驶车辆，处理器执行计算机程序时还实现以下步骤：控制所述无人驾驶车辆停车。
在一个实施例中，所述流媒体包括视频和音频中的至少一种。
上述实施例提供的计算机设备的处理器执行计算机程序时实现步骤，其实现原理和技术效果与上述方法实施例类似，在此不再赘述。
本申请实施例还提供了一种计算机可读存储介质。一个或多个包含计算机可执行指令的非易失性计算机可读存储介质，当计算机可执行指令被一个或多个处理器执行时，使得处理器执行以下流媒体网络时延确定方法的步骤：
获取流媒体当前帧的发送RTP时间，所述发送RTP时间是指根据RTP协议数据包获取的所述当前帧的发送时间；
获取所述发送RTP时间之前发送的任一个发送方报告的报告RTP时间，所述报告RTP时间是指根据RTCP协议获取的所述发送方报告的RTP时间；
根据所述发送RTP时间与所述报告RTP时间，确定所述当前帧的发送NTP时间，所述发送NTP时间是指与所述发送RTP时间对应的，发送所述当前帧的NTP时间；
获取所述当前帧的接收NTP时间，所述接收NTP时间是指接收所述当前帧的NTP时间；
根据所述发送NTP时间和所述接收NTP时间确定所述当前帧的网络时延。
在一个实施例中，计算机程序被处理器执行时还实现以下步骤：确定所述发送RTP时间与所述报告RTP时间的时间差，得到相对时间差；获取所述报告RTP时间对应的报告NTP时间，所述报告NTP时间是指所述发送方报告的NTP时间；根据所述报告NTP时间与所述相对时间差，确定所述发送NTP时间。
在一个实施例中，计算机程序被处理器执行时还实现以下步骤：将所述相对时间差转换为NTP时间，得到NTP时间差；计算所述报告NTP时间与所述NTP时间差的和，得到所述发送NTP时间。
在一个实施例中，计算机程序被处理器执行时还实现以下步骤：将所述相对时间差左移32位，得到所述得到NTP时间差。
在一个实施例中，计算机程序被处理器执行时还实现以下步骤：计算所述相对时间差与2 32的乘积，得到所述NTP时间差。
在一个实施例中，计算机程序被处理器执行时还实现以下步骤：获取所述发送RTP时间之前，与所述发送RTP时间最接近的一个发送方报告的RTP时间，得到所述报告RTP时间。
在一个实施例中，计算机程序被处理器执行时还实现以下步骤：计算所述接收NTP时间与所述发送NTP时间的差，得到所述当前帧的网络时延。
在一个实施例中，计算机程序被处理器执行时还实现以下步骤：获取所述当前帧的数据包中第一个数据包的时间戳；根据所述第一个数据包的时间戳，得到所述发送RTP时间。
在一个实施例中，计算机程序被处理器执行时还实现以下步骤：获取接收完所述当前帧的所有的数据包时的NTP时间，得到所述接收NTP时间。
在一个实施例中，计算机程序被处理器执行时还实现以下步骤：若所述当前帧的网络时延大于预设阈值，则输出报警信息。
在一个实施例中，计算机程序被处理器执行时还实现以下步骤：若所述当前帧的网络时延大于所述预设阈值，控制远程控制目标停止工作。
在一个实施例中，所述远程控制目标为无人驾驶车辆，计算机程序被处理器执行时还实现以下步骤：控制所述无人驾驶车辆停车。
在一个实施例中，所述流媒体包括视频和音频中的至少一种。
上述实施例提供的计算机可读存储介质，其实现原理和技术效果与上述方法实施例类似，在此不再赘述。
本申请一个实施例还提供一种远程驾驶系统，其包括无人驾驶车辆、NTP服务器和远程控制设备。无人驾驶车辆设置有图像采集装置，用于获取流媒体当前帧的发送RTP时间，所述发送RTP时间是指根据RTP协议数据包获取的所述当前帧的发送时间；获取所述发送RTP时间之前发送的任一个发送方报告的报告RTP时间，所述报告RTP时间是指根据RTCP协议获取的所述发送方报告的RTP时间；根据所述发送RTP时间与所述报告RTP时间，确定所述当前帧的发送NTP时间，所述发送NTP时间是指与所述发送RTP时 间对应的，发送所述当前帧的NTP时间；获取所述当前帧的接收NTP时间，所述接收NTP时间是指接收所述当前帧的NTP时间；根据所述发送NTP时间和所述接收NTP时间确定所述当前帧的网络时延。
在一个实施例中，所述远程控制设备还用于若所述当前帧的网络时延大于预设阈值，控制所述无人驾驶车辆停车。
需要说明的是，上述远程控制设备包括存储器及处理器，存储器中存储有计算机可读指令，指令被处理器执行时，使得处理器执行上述任一实施例中的流媒体网络时延确定方法的步骤。
上述实施例提供的远程驾驶系统，其实现原理和技术效果与上述方法实施例类似，在此不再赘述。
本申请所使用的对存储器、存储、数据库或其它介质的任何引用可包括非易失性和/或易失性存储器。合适的非易失性存储器可包括只读存储器(ROM)、可编程ROM(PROM)、电可编程ROM(EPROM)、电可擦除可编程ROM(EEPROM)或闪存。易失性存储器可包括随机存取存储器(RAM)，它用作外部高速缓冲存储器。作为说明而非局限，RAM以多种形式可得，诸如静态RAM(SRAM)、动态RAM(DRAM)、同步DRAM(SDRAM)、双数据率SDRAM(DDR SDRAM)、增强型SDRAM(ESDRAM)、同步链路(Synchlink)DRAM(SLDRAM)、存储器总线(Rambus)直接RAM(RDRAM)、直接存储器总线动态RAM(DRDRAM)、以及存储器总线动态RAM(RDRAM)。
以上所述实施例的各技术特征可以进行任意的组合，为使描述简洁，未对上述实施例中的各个技术特征所有可能的组合都进行描述，然而，只要这些技术特征的组合不存在矛盾，都应当认为是本说明书记载的范围。
以上所述实施例仅表达了本发明的几种实施方式，其描述较为具体和详细，但并不能因此而理解为对发明专利范围的限制。应当指出的是，对于本领域的普通技术人员来说，在不脱离本发明构思的前提下，还可以做出若干变形和改进，这些都属于本发明的保护范围。因此，本发明专利的保护范围应以所附权利要求为准。

Claims (20)

  1. 一种流媒体网络时延确定方法，其特征在于，所述方法包括：
    获取流媒体当前帧的发送RTP时间，所述发送RTP时间是指根据RTP协议数据包获取的所述当前帧的发送时间；
    获取所述发送RTP时间之前发送的任一个发送方报告的报告RTP时间，所述报告RTP时间是指根据RTCP协议获取的所述发送方报告的RTP时间；
    根据所述发送RTP时间与所述报告RTP时间，确定所述当前帧的发送NTP时间，所述发送NTP时间是指与所述发送RTP时间对应的，发送所述当前帧的NTP时间；
    获取所述当前帧的接收NTP时间，所述接收NTP时间是指接收所述当前帧的NTP时间；
    根据所述发送NTP时间和所述接收NTP时间确定所述当前帧的网络时延。
  2. 根据权利要求1所述的方法，其特征在于，所述根据所述发送RTP时间与所述报告RTP时间，确定所述当前帧的发送NTP时间，包括：
    确定所述发送RTP时间与所述报告RTP时间的时间差，得到相对时间差；
    获取所述报告RTP时间对应的报告NTP时间，所述报告NTP时间是指所述发送方报告的NTP时间；
    根据所述报告NTP时间与所述相对时间差，确定所述发送NTP时间。
  3. 根据权利要求2所述的方法，其特征在于，所述根据所述报告NTP时间与所述相对时间差，确定所述发送NTP时间，包括：
    将所述相对时间差转换为NTP时间，得到NTP时间差；
    计算所述报告NTP时间与所述NTP时间差的和，得到所述发送NTP时间。
  4. 根据权利要求3所述的方法，其特征在于，所述将所述相对时间差转换为NTP时间，得到NTP时间差，包括：
    将所述相对时间差左移32位，得到所述得到NTP时间差。
  5. 根据权利要求3所述的方法，其特征在于，所述将所述相对时间差转换为NTP时间，得到NTP时间差，包括：
    计算所述相对时间差与2 32的乘积，得到所述NTP时间差。
  6. 根据权利要求1所述的方法，其特征在于，所述获取所述发送RTP时间之前发送的任一个发送方报告的报告RTP时间，包括：
    获取所述发送RTP时间之前，与所述发送RTP时间最接近的一个发送方报告的RTP时间，得到所述报告RTP时间。
  7. 根据权利要求1所述的方法，其特征在于，所述根据所述发送NTP时间和所述接收NTP时间确定所述当前帧的网络时延，包括：
    计算所述接收NTP时间与所述发送NTP时间的差，得到所述当前帧的网络时延。
  8. 根据权利要求1所述的方法，其特征在于，所述获取流媒体当前帧的发送RTP时间，包括：
    获取所述当前帧的数据包中第一个数据包的时间戳；
    根据所述第一个数据包的时间戳，得到所述发送RTP时间。
  9. 根据权利要求8所述的方法，其特征在于，所述获取所述当前帧的接收NTP时间，包括：
    获取接收完所述当前帧的所有的数据包时的NTP时间，得到所述接收NTP时间。
  10. 根据权利要求1所述的方法，其特征在于，所述方法还包括：
    若所述当前帧的网络时延大于预设阈值，则输出报警信息。
  11. 根据权利要求10所述的方法，其特征在于，所述方法还包括：
    若所述当前帧的网络时延大于所述预设阈值，控制远程控制目标停止工作。
  12. 根据权利要求11所述的方法，其特征在于，所述远程控制目标为无人驾驶车辆，所述控制远程控制的目标停止工作，包括：
    控制所述无人驾驶车辆停车。
  13. 根据权利要求1至12任一项所述的方法，其特征在于，所 述流媒体包括视频和音频中的至少一种。
  14. 一种流媒体网络时延确定装置，其特征在于，所述装置包括：
    发送RTP时间获取模块，用于获取流媒体当前帧的发送RTP时间，所述发送RTP时间是指根据RTP协议数据包获取的所述当前帧的发送时间；
    报告RTP时间获取模块，用于获取所述发送RTP时间之前发送的任一个发送方报告的报告RTP时间，所述报告RTP时间是指根据RTCP协议获取的所述发送方报告的RTP时间；
    发送NTP时间确定模块，用于根据所述发送RTP时间与所述报告RTP时间，确定所述当前帧的发送NTP时间，所述发送NTP时间是指与所述发送RTP时间对应的，发送所述当前帧的NTP时间；
    接收NTP时间确定模块，用于获取所述当前帧的接收NTP时间，所述接收NTP时间是指接收所述当前帧的NTP时间；
    时延确定模块，用于根据所述发送NTP时间和所述接收NTP时间确定所述当前帧的网络时延。
  15. 根据权利要求14所述的装置，其特征在于，所述报告RTP时间获取模块包括：
    相对时间差确定单元，用于确定所述发送RTP时间与所述报告RTP时间的时间差，得到相对时间差；
    报告NTP时间确定单元，用于获取所述报告RTP时间对应的报告NTP时间，所述报告NTP时间是指所述发送方报告的NTP时间；
    发送NTP时间确定单元，用于根据所述报告NTP时间与所述相对时间差，确定所述发送NTP时间。
  16. 根据权利要求14所述的装置，其特征在于，还包括：
    控制模块，用于若所述当前帧的网络时延大于预设阈值，控制远程控制目标停止工作。
  17. 一种计算机可读存储介质，其上存储有计算机程序，其特征在于，所述计算机程序被处理器执行时实现如权利要求1至13中任一项所述的流媒体网络时延确定方法的步骤。
  18. 一种计算机设备，包括存储器及处理器，其特征在于，所述 存储器中储存有计算机可读指令，所述指令被所述处理器执行时，使得所述处理器执行如权利要求1至13中任一项所述的流媒体网络时延确定方法。
  19. 一种远程驾驶系统，其特征在于，包括：
    无人驾驶车辆，设置有图像采集装置，用于采集流媒体；
    NTP服务器，与所述图像采集装置通信连接；
    远程控制设备，与所述图像采集装置及所述NTP服务器通信连接，所述远程控制设备用于获取流媒体当前帧的发送RTP时间，所述发送RTP时间是指根据RTP协议数据包获取的所述当前帧的发送时间；获取所述发送RTP时间之前发送的任一个发送方报告的报告RTP时间，所述报告RTP时间是指根据RTCP协议获取的所述发送方报告的RTP时间；根据所述发送RTP时间与所述报告RTP时间，确定所述当前帧的发送NTP时间，所述发送NTP时间是指与所述发送RTP时间对应的，发送所述当前帧的NTP时间；获取所述当前帧的接收NTP时间，所述接收NTP时间是指接收所述当前帧的NTP时间；根据所述发送NTP时间和所述接收NTP时间确定所述当前帧的网络时延。
  20. 根据权利要求19所述的远程驾驶系统，其特征在于，所述远程控制设备还用于若所述当前帧的网络时延大于预设阈值，控制所述无人驾驶车辆停车。
PCT/CN2020/074924 2020-02-12 2020-02-12 流媒体网络时延确定方法、装置、计算机设备、可读存储介质和远程驾驶系统 WO2021159329A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN202080003164.3A CN113519146B (zh) 2020-02-12 2020-02-12 流媒体网络时延确定方法、装置、计算机设备、可读存储介质和远程驾驶系统
PCT/CN2020/074924 WO2021159329A1 (zh) 2020-02-12 2020-02-12 流媒体网络时延确定方法、装置、计算机设备、可读存储介质和远程驾驶系统

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2020/074924 WO2021159329A1 (zh) 2020-02-12 2020-02-12 流媒体网络时延确定方法、装置、计算机设备、可读存储介质和远程驾驶系统

Publications (1)

Publication Number Publication Date
WO2021159329A1 true WO2021159329A1 (zh) 2021-08-19

Family

ID=77291923

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/074924 WO2021159329A1 (zh) 2020-02-12 2020-02-12 流媒体网络时延确定方法、装置、计算机设备、可读存储介质和远程驾驶系统

Country Status (2)

Country Link
CN (1) CN113519146B (zh)
WO (1) WO2021159329A1 (zh)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113904959A (zh) * 2021-11-02 2022-01-07 广州小鹏自动驾驶科技有限公司 一种时延分析方法、装置、车辆、存储介质
CN113965488A (zh) * 2021-09-10 2022-01-21 北京百度网讯科技有限公司 数据包的延迟获取方法及设备、电子设备与存储介质
CN115174979A (zh) * 2022-06-20 2022-10-11 阿里巴巴（中国）有限公司 流媒体传输网络及传输控制方法、装置、设备及存储介质
CN115333961A (zh) * 2022-06-30 2022-11-11 北京邮电大学 基于深度强化学习的无线通信网络管控方法及相关设备

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1540893A (zh) * 1998-12-21 2004-10-27 卓联半导体V.N.有限公司 为硬件rtcp统计收集应用提供存储和带宽节省的方法和数据结构
WO2005006621A1 (en) * 2003-07-04 2005-01-20 National University Of Ireland, Galway System and method for determining clock skew in a packet-based telephony session
CN1738437A (zh) * 2004-06-22 2006-02-22 Lg电子株式会社 同步移动通信终端的视频/音频数据
CN101282482A (zh) * 2008-05-04 2008-10-08 中兴通讯股份有限公司 视频数据与音频数据同步播放的装置、系统和方法
CN102202070A (zh) * 2010-03-23 2011-09-28 冯思雅 流媒体快速流间同步机制
JP5074834B2 (ja) * 2007-06-29 2012-11-14 沖電気工業株式会社 音声・映像同期方法、音声・映像同期システム及び音声・映像受信端末
CN109017757A (zh) * 2018-08-22 2018-12-18 吉林大学 汽车远程代驾方法及系统

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2008167351A (ja) * 2007-01-04 2008-07-17 Toshiba Corp 端末装置
CN101123611B (zh) * 2007-09-25 2012-05-23 中兴通讯股份有限公司 一种流媒体数据的发送方法
CN107453936A (zh) * 2016-06-01 2017-12-08 中兴通讯股份有限公司 一种诊断语音时延的方法和网关设备
WO2018170852A1 (zh) * 2017-03-23 2018-09-27 华为技术有限公司 多设备间唇音同步方法及设备
CN109189567B (zh) * 2018-08-30 2021-10-08 百度在线网络技术（北京）有限公司 时延计算方法、装置、设备及计算机可读存储介质
CN109151611B (zh) * 2018-09-21 2020-09-08 深圳市璇玑实验室有限公司 基于rtp/rtcp的hevc实时视频传输控制方法
CN109286813B (zh) * 2018-11-14 2020-06-05 北京奇艺世纪科技有限公司 一种视频通信质量检测方法和装置

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1540893A (zh) * 1998-12-21 2004-10-27 卓联半导体V.N.有限公司 为硬件rtcp统计收集应用提供存储和带宽节省的方法和数据结构
WO2005006621A1 (en) * 2003-07-04 2005-01-20 National University Of Ireland, Galway System and method for determining clock skew in a packet-based telephony session
CN1738437A (zh) * 2004-06-22 2006-02-22 Lg电子株式会社 同步移动通信终端的视频/音频数据
JP5074834B2 (ja) * 2007-06-29 2012-11-14 沖電気工業株式会社 音声・映像同期方法、音声・映像同期システム及び音声・映像受信端末
CN101282482A (zh) * 2008-05-04 2008-10-08 中兴通讯股份有限公司 视频数据与音频数据同步播放的装置、系统和方法
CN102202070A (zh) * 2010-03-23 2011-09-28 冯思雅 流媒体快速流间同步机制
CN109017757A (zh) * 2018-08-22 2018-12-18 吉林大学 汽车远程代驾方法及系统

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
JUNG TAE-JUN; SEO KWANG-DEOK: "A Client-driven media synchromization mechanism for RTP packet-based video streaming", JOURNAL OF REAL-TIME IMAGE PROCESSING, vol. 12, 7 April 2015 (2015-04-07), pages 455 - 464, XP036018716, ISSN: 1861-8200, DOI: 10.1007/s11554-015-0497-3 *
REN YANZHEN , YU ZHANWU , HU RUIMIN: "Real-Time Data Transport and Synchronization Scheme Based on RTP /RTCP", COMPUTER ENGINEERING AND APPLICATIONS, no. 10, 1 April 2003 (2003-04-01), pages 144 - 147, XP055835941, ISSN: 1002-8331 *
ZHOU XU-YAN;LI XIAN-ZHE;TAN BIN;ZHU BING;XIAO YING: "RESEARCH OF AUDIO AND VIDEO SYNCHRONIZATION ALGORITHM BASED ON RTP/RTCP PROTOCOL", JOURNAL OF JINGGANGSHAN UNIVERSITY (NATURAL SCIENCE), vol. 36, no. 2, 15 March 2015 (2015-03-15), pages 38 - 41+73, XP055835947, ISSN: 1674-8085, DOI: 10.3969/j.issn.1674-8085.2015.02.010 *

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113965488A (zh) * 2021-09-10 2022-01-21 北京百度网讯科技有限公司 数据包的延迟获取方法及设备、电子设备与存储介质
CN113904959A (zh) * 2021-11-02 2022-01-07 广州小鹏自动驾驶科技有限公司 一种时延分析方法、装置、车辆、存储介质
CN115174979A (zh) * 2022-06-20 2022-10-11 阿里巴巴（中国）有限公司 流媒体传输网络及传输控制方法、装置、设备及存储介质
CN115174979B (zh) * 2022-06-20 2023-12-29 阿里巴巴（中国）有限公司 流媒体传输网络及传输控制方法、装置、设备及存储介质
CN115333961A (zh) * 2022-06-30 2022-11-11 北京邮电大学 基于深度强化学习的无线通信网络管控方法及相关设备
CN115333961B (zh) * 2022-06-30 2023-10-13 北京邮电大学 基于深度强化学习的无线通信网络管控方法及相关设备

Also Published As

Publication number Publication date
CN113519146B (zh) 2023-06-23
CN113519146A (zh) 2021-10-19

Similar Documents

Publication Publication Date Title
WO2021159329A1 (zh) 流媒体网络时延确定方法、装置、计算机设备、可读存储介质和远程驾驶系统
TWI733132B (zh) 基於使用低速指紋之內插來建立及使用時間映射以幫助促進訊框準確性內容修正
CN103701658B (zh) 一种视联网的网络状况测试方法和装置
US10097874B2 (en) System and method for monitoring media stream buffers of media output devices in order to synchronize media content output
TW202007180A (zh) 動態控制指紋辨識速率以促進媒體內容的時間準確性修訂
CN102546081B (zh) 丢包检测方法、系统和媒体客户端
US20160006526A1 (en) Systems and methods of network clock comparison
US20090110132A1 (en) System and method for re-synchronization of a pss session to an mbms session
WO2018170852A1 (zh) 多设备间唇音同步方法及设备
US9031679B2 (en) Methods, systems, and computer readable media for utilizing a plurality of pre-encoded payloads to generate a packet stream transmission
CN109687927B (zh) 一种确定时间戳的方法、通信设备和通信系统
BR112015015813B1 (pt) Método e aparelho para ajustar a qualidade de vídeo com base em ambiente de rede
CN101808167A (zh) 一种流程跟踪方法以及装置和系统
WO2021159332A1 (zh) 图像采集触发方法、装置、计算机设备、可读存储介质和监控设备
CN112565224A (zh) 一种视频处理方法及装置
CN112579820B (zh) 时间跳变的录像数据处理方法、装置、介质及电子设备
CN108027717B (zh) 切换无线显示模式并刷新处于不稳定状态的图形处理单元
WO2018188365A1 (zh) 同步播放方法、装置和系统
WO2024124797A1 (zh) 无线网络节点同步方法、系统、设备及可读存储介质
CN116614199A (zh) 时钟同步方法、装置、从时钟设备、时钟同步系统和介质
JP2003179662A5 (zh)
WO2023201822A1 (zh) 多相机同步校正方法、装置及存储介质
CN113747208B (zh) 音视频数据同步方法、数据同步方法、服务端及接收端
CN107786292A (zh) 一种协议栈网络时间的测量方法及装置
CN110138487B (zh) 一种时间噪声传递特性测量方法和网络节点

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20918283

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 16.01.2023)

122 Ep: pct application non-entry in european phase

Ref document number: 20918283

Country of ref document: EP

Kind code of ref document: A1