CN117014696A

CN117014696A - RTP-based audio and video transmission method and device, electronic equipment and storage medium

Info

Publication number: CN117014696A
Application number: CN202310955430.8A
Authority: CN
Inventors: 王磊
Original assignee: Unicloud Technology Co Ltd
Current assignee: Unicloud Technology Co Ltd
Priority date: 2023-07-31
Filing date: 2023-07-31
Publication date: 2023-11-07

Abstract

The application provides an RTP-based audio and video transmission method, an RTP-based audio and video transmission device, electronic equipment and a storage medium, wherein the RTP-based audio and video transmission method comprises the following steps: receiving RTP data packets acquired by camera equipment; detecting the packet loss rate of RTP data packets within a period of time based on a packet loss rate data statistics strategy, and inserting the packet loss rate data into a real-time audio/video stream in the form of SEI (solid state interface) in an H.264 or H.265 code stream, wherein the packet loss rate data statistics strategy is detected by utilizing sequence number field characteristics specified in an RTP protocol header; and extracting the packet loss rate SEI information in the audio and video stream to play and present. The RTP-based audio and video transmission method can grasp the packet loss process in the network process of the camera equipment in real time, can acquire the packet loss rate data in the RTP video transmission process in real time, and can display the current network packet loss rate data in a proper mode in time along with the playing picture of the real-time video in the playing process of the player.

Description

RTP-based audio and video transmission method and device, electronic equipment and storage medium

Technical Field

The application belongs to the field of audio and video transmission, and particularly relates to an audio and video transmission method, an audio and video transmission device, electronic equipment and a storage medium based on RTP.

Background

RTP is a very mature transmission protocol applied in the field of audio and video transmission, and is widely applied to scenes such as video monitoring, video live broadcast, video conference and the like. In the video monitoring scene based on GB/T28181, if the signal of the camera adopts an RTP over UDP mode to remotely transmit the audio and video stream to a remote streaming media server across the Internet, because of inherent unreliability of UDP, any network jitter can cause data packet delay and even packet loss due to jitter of a transmission network in an Internet environment, so that partial frame data of the audio and video stream is lost, and finally, the phenomenon of picture blocking and screen display caused by decoding errors of a player occurs.

In order to grasp the packet loss condition in the network transmission process of the camera in real time, in a video monitoring system, a method is needed to acquire the packet loss rate data in the RTP video transmission process in real time, and the method can be used for displaying the current network packet loss rate data in a proper manner in time along with a playing picture of the real-time video in the playing process of a player so as to quickly locate and find out related network problems.

Disclosure of Invention

In view of the above, the present application aims to provide an audio/video transmission method, an apparatus, an electronic device and a storage medium based on RTP, by collecting packet loss rate data in the RTP video transmission process in real time and finally presenting the data on a player, so that people can timely and accurately know the packet loss condition of an audio/video stream in the cross-network transmission process, and timely make relevant optimization measures to improve the service quality.

In order to achieve the above purpose, the technical scheme of the application is realized as follows:

in a first aspect, the present application provides an audio/video transmission method based on RTP, the method comprising:

receiving RTP data packets acquired by camera equipment;

detecting the packet loss rate of RTP data packets within a period of time based on a packet loss rate data statistics strategy, and inserting the packet loss rate data into a real-time audio/video stream in the form of SEI (solid state interface) in an H.264 or H.265 code stream, wherein the packet loss rate data statistics strategy is detected by utilizing sequence number field characteristics specified in an RTP protocol header;

and extracting the packet loss rate SEI information in the audio and video stream to play and present.

Further, the detecting the packet loss rate of the RTP audio/video stream data packet in a period of time based on the packet loss rate data statistics policy inserts the packet loss rate data into the real-time audio/video stream in the form of SEI in the h.264 or h.265 code stream, where the packet loss rate data statistics policy is detected by using sequence number field characteristics specified in the RTP protocol header, and includes:

establishing a circular queue about the receiving state of the RTP data packet, and setting a corresponding RTP data packet receiving state mark in the circular queue by utilizing a sequence number field in the received RTP data packet;

according to the statistical time interval condition, counting the packet loss rate of RTP data packets in a period of time at regular time;

and inserting the packet loss rate data into the real-time audio and video stream in the form of SEI in the H.264 or H.265 code stream, and packaging the data into a live broadcast protocol to be forwarded to the player.

Further, according to the statistical time interval condition, the counting the packet loss rate of the RTP packet in a period of time at regular time includes:

setting a statistical window in a circulation queue, setting the size of the statistical window as w, and setting a statistical interval as [ max (p_start, p_end-w), p_end ];

scanning the number of data packets with bit of 1 in the statistical interval, and taking the data packets as molecules, wherein the number of the data packets comprises the number of the received data packets;

calculating total data packet number from max (p_start, p_end-w) to p_end, and taking the total data packet number as a denominator, wherein the total data packet number comprises the sum of the received data packet number and the data packet number with packet loss;

and calculating the number of the data packets/the number of the total data packets to obtain the packet loss rate in the statistical interval.

Further, the method further comprises the following steps:

setting a timer and setting a timer time interval;

and executing the packet loss rate data of the statistical circulation queue in the statistical interval according to the set timer time interval.

Further, the extracting the packet loss rate SEI information in the audio and video stream for playing and presenting includes:

the player determines a video code stream according to a video coding format indicated in a video encapsulation format specific field, continuously identifies NALU units in the video code stream, and judges whether the NALU units are SEI NALU units according to NALU type identification;

and in response to the SEI NALU unit, judging whether the SEI data is packet loss rate data according to the SEI type in the SEI data packet and the custom global UUID value, and extracting corresponding packet loss rate data from the SEI NALU unit and displaying the data to a player.

The second aspect, based on the same inventive concept, further provides an audio and video transmission device based on RTP, the device comprises:

the data packet receiving module is configured to receive RTP data packets acquired by the camera equipment;

the system comprises a packet loss rate data statistics module, a packet loss rate data processing module and a packet loss rate data processing module, wherein the packet loss rate data statistics module is configured to detect the packet loss rate of RTP data packets in a period of time based on a packet loss rate data statistics strategy, and insert the packet loss rate data into a real-time audio/video stream in the form of SEI (solid state imaging) in an H.264 or H.265 code stream, wherein the packet loss rate data statistics strategy is detected by utilizing sequence number field characteristics specified in an RTP protocol header;

and the playing and presenting module is configured to extract the packet loss rate SEI information in the audio and video stream for playing and presenting.

In a third aspect, based on the same inventive concept, the present application further provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor implements the RTP-based audio/video transmission method according to the first aspect when executing the program.

In a fourth aspect, based on the same inventive concept, the present application also provides a non-transitory computer readable storage medium, wherein the non-transitory computer readable storage medium stores computer instructions for causing the computer to perform the RTP based audio and video transmission method according to the first aspect.

Compared with the prior art, the RTP-based audio and video transmission method, the RTP-based audio and video transmission device, the electronic equipment and the storage medium have the following beneficial effects:

according to the RTP-based audio and video transmission method, device, electronic equipment and storage medium, the RTP data packet acquired by the camera equipment is received, the packet loss rate of the RTP data packet in a period of time is detected based on a packet loss rate data statistics strategy, and the packet loss rate data is inserted into a real-time audio and video stream in an SEI (solid-state interface) form in an H.264 or H.265 code stream, wherein the packet loss rate data statistics strategy is detected by utilizing sequence number field characteristics specified in an RTP protocol header; extracting the packet loss rate SEI information in the audio and video stream for playing and presenting; the application can grasp the packet loss condition of the camera equipment in the process of transmitting the audio and video data across the network in real time, can acquire the packet loss rate data in the process of transmitting the RTP video in real time, and can improve the service quality by acquiring the packet loss rate data in the process of transmitting the RTP video in real time and finally displaying the packet loss rate data on the player so as to enable people to know the packet loss condition of the audio and video stream in the process of transmitting the audio and video data across the network in real time and make relevant optimization measures in real time.

Drawings

The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this specification, illustrate embodiments of the application and together with the description serve to explain the application. In the drawings:

fig. 1 is a flowchart of an audio/video transmission method based on RTP according to an embodiment of the application;

fig. 2 is a schematic diagram of a video monitoring system based on RTP transmission of video monitoring signals according to an embodiment of the application;

FIG. 3 is a schematic diagram of a circular queue according to an embodiment of the present application;

fig. 4 is a schematic structural diagram of an RTP-based audio/video transmission apparatus according to an embodiment of the application;

fig. 5 is a schematic diagram of a hardware structure of an electronic device according to an embodiment of the present application.

Detailed Description

The present application will be further described in detail below with reference to specific embodiments and with reference to the accompanying drawings, in order to make the objects, technical solutions and advantages of the present application more apparent.

It should be noted that unless otherwise defined, technical or scientific terms used in the embodiments of the present application should be given the ordinary meaning as understood by one of ordinary skill in the art to which the present application belongs. The terms "first," "second," and the like, as used in embodiments of the present application, do not denote any order, quantity, or importance, but rather are used to distinguish one element from another. The word "comprising" or "comprises", and the like, means that elements or items preceding the word are included in the element or item listed after the word and equivalents thereof, but does not exclude other elements or items. The terms "connected" or "connected," and the like, are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "upper", "lower", "left", "right", etc. are used merely to indicate relative positional relationships, which may also be changed when the absolute position of the object to be described is changed.

Embodiments of the present application are described in detail below with reference to the accompanying drawings.

Referring to fig. 1 and 2, an audio/video transmission method based on RTP according to an embodiment of the application includes the following steps:

step S101, an RTP data packet acquired by the camera equipment is received.

Specifically, in this embodiment, RTP packets are collected by a camera device (a camera, etc.) and an audio/video stream is remotely transmitted to a streaming media server across the internet by using an RTP over UDP method.

Step S102, detecting the packet loss rate of RTP data packets within a period of time based on a packet loss rate data statistics strategy, and inserting the packet loss rate data into a real-time audio/video stream in the form of SEI in an H.264 or H.265 code stream, wherein the packet loss rate data statistics strategy is detected by utilizing sequence number field characteristics specified in an RTP protocol header.

Specifically, in this embodiment, the processing procedure for the streaming media server is specifically as follows:

1. and (5) collecting packet loss rate data.

As shown in fig. 3, according to the specification of the sequence number of the RTP protocol, the field is 16 bits, the value range is 0-65535, and each RTP packet monotonically increases in sequence. Therefore, a ring buffer of RTP packet receiving status is first established, which contains 65536 bits (i.e., 8192 bytes), each bit corresponding to the receiving status of RTP packets of sequence number of the same sequence number.

The RTP packet receiving status circular queue (ring buffer, which is explained by the ring buffer below) is provided with two pointers, one is a p_start pointer, which represents the queue head, the other is a p_end pointer, which represents the queue tail, and adopts a tail insertion mode, both pointers initialize bit 0, and point to bit bits corresponding to pkt with sequence number of 0.

Because of the ring-shaped queue, the sequence of the queue pointers needs to be specified. Assuming that the head of the queue is the front and the tail of the queue is the back, the two queue element pointers p1, p2 define the following sequential relationship:

comparison of p1 and p2 is agreed as follows:

assuming that the pointer value of p1 is i1 and the pointer value of p2 is i2, then,

if i2> i1, and i2-i1<32768, then p2 gt p1 (i.e., p1 is in front of p 2);

if i2< i1, and i1-i2> = 36788, p2 gt p1 (i.e. p1 is before p 2);

if i2=i1, then p1 eq p2 (i.e., p1 and p2 point to the same position);

otherwise p2 lt p1 (i.e. p2 is in front of p 1).

The comparison operation according to the above pointer is specified, for example, as follows:

if p1=5, p2=32772, then p2 gt p1 is because 32772>5, and 32772-5< 32768.

If p1=5, p2= 32773, then p2 lt p1 because 32773>5, and 32773-5=32768.

If p1=65535, p2=1, then p2 gt p1 is because 1<65535, and 65535-1> =32768.

If p1=65536, p2=32767, then p2 gt p1 is because 32767<65535, and 65535-32767> =32768.

According to the above comparison method, when an RTP packet with a sequence number n is received, the following describes a procedure of setting a bit corresponding to a ring buffer of an RTP packet reception state:

let us assume the pointer value of p_end, the pointer value of p_start, the pointer value of istart, n, the pointer value of in.

If n & gtp_end, bit position 0 between the [ p_end, n ] intervals is set, and bit of the position in is set to 1. If n ge p_start, then it is stated that the ring buffer queue tail has passed over head (indicating that the ring buffer is full), p_start needs to be moved to the position of (n+2)% 65536 (% operation indicates modulo). Finally, p_end is set to (n+1)% 65536.

If nle p_end and n ge p_start, i.e. the bit to be set is exactly between p_start and p_end, the bit of the set position in is 1.

Otherwise, the bit to be set is before p_start, the packet is considered as an illegal packet, and the packet is directly discarded and is not processed.

2. Statistics of packet loss rate data

With the continuous reception of the RTP data packet, the state of the RTP data packet reception state ring buffer area is continuously updated, and then the actual packet loss rate can be calculated. Because the whole ring buffer can count the latest 65536 RTP packet states, if 65536 is taken as a denominator to calculate the packet loss rate, the statistical time span is larger, and the change of the packet loss rate cannot timely reflect the actual condition of the network at the time.

To solve this problem, in this embodiment, a statistical window is set for the ring buffer, and the window is set to have a size w, and then the statistical interval is [ max (p_start, p_end-w), p_end ]. The size of the statistical window may be set according to the service requirement, or may dynamically correspond to a time window calculated in seconds according to the actual video code stream.

Hereinafter, the statistical window is assumed to be w, and the statistical interval is [ max (p_start, p_end-w), p_end ]. Then the packet loss rate in the window interval can be obtained by first scanning the number of packets with bit 1 (only the received packet) in this statistics window interval as a numerator and named received_packets, then calculating max (p_start, p_end-w), and the total number of packets to p_end (including the sum of the number of packets that have been received and the number of packets that may have lost) as a denominator and named packet_in_window, and then calculating the received_packets/packets_in_window.

3. Transmitting packet loss rate data

After the RTP push session is established, the streaming media server starts a timer (a timer time interval can be set according to the real-time requirement of the service, for example, 1 s) for the session, when the timer is triggered, the RTP packet receiving state annular buffer corresponding to the RTP push session is used for carrying out packet loss rate data statistics once in a statistics window, and then the packet loss rate data is packaged into SEI NALU of the video stream H.264/H.265 and sent out.

For h.264, the encapsulation is as follows:

for h.265, it is encapsulated in the following format:

and step S103, extracting the packet loss rate SEI information in the audio and video stream for playing and presenting.

Specifically, after receiving the video stream, the player determines whether h.264 or h.265 is encoded according to the video encoding format indicated in the specific field of the corresponding video encapsulation format, then continuously identifies NALU units in the code stream, identifies whether the NALU units are SEI NALU units according to NALU types, judges whether the packet loss rate data is lost according to the SEI types in the SEI data packets and the custom global UUID value if the packet loss rate data is the SEI NALU units, and finally extracts the corresponding packet loss rate data from the SEI NALU and displays the corresponding packet loss rate data to the player.

In this embodiment, since the packet loss rate data is naturally interleaved in the original video stream, on one hand, the player can display the packet loss rate data when playing the live broadcast signal in real time; on the one hand, if the real-time live broadcast signal is recorded, the packet loss rate stored in the SEI form is still stored in the recorded video file; therefore, the situation of the current packet loss rate can still be seen when the video is back seen, so that the current network condition can be traced back conveniently, and the reason of the video playing flower screen blocking can be known; finally, if the historical packet loss rate data needs to be subjected to warehouse-in analysis in the future, the recorded file can be conveniently and rapidly scanned, video decoding is not needed, only the packet loss rate data in SEI (solid phase interface) is extracted, the accurate time point at the time is mapped according to the dts timestamp information in the video stream, and the packet loss rate time sequence data is warehoused in a data warehouse.

The method effectively avoids the problems that an independent channel is required to be reconstructed to transmit the packet loss rate data, a special place is required to store the packet loss rate time sequence, and the packet loss rate data is required to be obtained continuously through a relevant interface when the video is played. By adopting the mode, the method is very troublesome, has higher requirements on the system, easily causes compressive concurrency force on the query interface under the condition of large concurrency quantity, and has access bottleneck. The method of the application utilizes the transmission channel of the video stream, almost does not need to add any extra overhead, and can smoothly transmit the packet loss rate data by adding a few bytes of data in the video stream.

The audio and video transmission method based on RTP is mainly applied to a video monitoring system based on RTP transmission video monitoring signals, and detects the packet loss rate of RTP data packets in a period of time based on a packet loss rate data statistics policy by receiving RTP data packets acquired by camera equipment, and inserts the packet loss rate data into a real-time audio and video stream in the form of SEI in an H.264 or H.265 code stream, wherein the packet loss rate data statistics policy is detected by utilizing sequence number field characteristics specified in an RTP protocol header; the method and the device extract the packet loss rate SEI information in the audio and video stream to play and present, can grasp the packet loss condition of the camera equipment in the process of audio and video data crossing the network in real time, and can acquire the packet loss rate data in the process of RTP video transmission in real time. The method is characterized in that the packet loss rate data in the RTP video transmission process are collected in real time and finally presented on a player, so that people can timely and accurately know the packet loss condition of the audio and video stream in the cross-network transmission process, and timely make relevant optimization measures to improve the service quality.

Based on the above embodiment, the present embodiment further provides another idea, that is, the streaming media server uses the sequence number field of the RTP packet to count the packet loss rate, and then time stamps the packet loss rate data and sends the data to a special database or a cache server, and the player takes out from the database or the cache server and displays the data.

It should be noted that the foregoing describes some embodiments of the present application. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims may be performed in a different order than in the embodiments described above and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

Based on the same inventive concept, the embodiment of the application also provides an audio/video transmission device based on RTP, which corresponds to the method of any embodiment.

As shown in fig. 4, the audio/video transmission device based on RTP includes:

a data packet receiving module 11 configured to receive an RTP data packet acquired by the image pickup apparatus;

the packet loss rate data statistics module 12 is configured to detect a packet loss rate of an RTP data packet within a period of time based on a packet loss rate data statistics policy, and insert the packet loss rate data into a real-time audio/video stream in the form of an SEI in an h.264 or h.265 code stream, where the packet loss rate data statistics policy is detected by using sequence number field characteristics specified in an RTP protocol header;

and the playing and presenting module 13 is configured to extract the packet loss rate SEI information in the audio and video stream for playing and presenting.

For convenience of description, the above devices are described as being functionally divided into various modules, respectively. Of course, the functions of each module may be implemented in the same piece or pieces of software and/or hardware when implementing an embodiment of the present application.

The device of the foregoing embodiment is configured to implement the corresponding RTP-based audio/video transmission method in any of the foregoing embodiments, and has the beneficial effects of the corresponding method embodiment, which is not described herein.

Based on the same inventive concept, corresponding to the method of any embodiment, the embodiment of the application further provides an electronic device, which comprises a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the processor executes the program to realize the RTP-based audio/video transmission method according to any embodiment.

Fig. 5 shows a more specific hardware architecture of an electronic device according to this embodiment, where the device may include: a processor 1010, a memory 1020, an input/output interface 1030, a communication interface 1040, and a bus 1050. Wherein processor 1010, memory 1020, input/output interface 1030, and communication interface 1040 implement communication connections therebetween within the device via a bus 1050.

The processor 1010 may be implemented by a general-purpose CPU (Central Processing Unit ), microprocessor, application specific integrated circuit (Application Specific Integrated Circuit, ASIC), or one or more integrated circuits, etc. for executing relevant programs to implement the technical solutions provided in the embodiments of the present disclosure.

The Memory 1020 may be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory ), static storage device, dynamic storage device, or the like. Memory 1020 may store an operating system and other application programs, and when the embodiments of the present specification are implemented in software or firmware, the associated program code is stored in memory 1020 and executed by processor 1010.

The input/output interface 1030 is used to connect with an input/output module for inputting and outputting information. The input/output module may be configured as a component in a device (not shown) or may be external to the device to provide corresponding functionality. Wherein the input devices may include a keyboard, mouse, touch screen, microphone, various types of sensors, etc., and the output devices may include a display, speaker, vibrator, indicator lights, etc.

Communication interface 1040 is used to connect communication modules (not shown) to enable communication interactions of the present device with other devices. The communication module may implement communication through a wired manner (such as USB, network cable, etc.), or may implement communication through a wireless manner (such as mobile network, WIFI, bluetooth, etc.).

Bus 1050 includes a path for transferring information between components of the device (e.g., processor 1010, memory 1020, input/output interface 1030, and communication interface 1040).

It should be noted that although the above-described device only shows processor 1010, memory 1020, input/output interface 1030, communication interface 1040, and bus 1050, in an implementation, the device may include other components necessary to achieve proper operation. Furthermore, it will be understood by those skilled in the art that the above-described apparatus may include only the components necessary to implement the embodiments of the present description, and not all the components shown in the drawings.

The electronic device of the foregoing embodiment is configured to implement the corresponding RTP-based audio/video transmission method in any of the foregoing embodiments, and has the beneficial effects of the corresponding method embodiment, which is not described herein.

Based on the same inventive concept, the present application also provides a non-transitory computer readable storage medium corresponding to the method of any of the above embodiments, where the non-transitory computer readable storage medium stores computer instructions for causing the computer to execute the RTP-based audio/video transmission method according to any of the above embodiments.

The computer readable media of the present embodiments, including both permanent and non-permanent, removable and non-removable media, may be used to implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of storage media for a computer include, but are not limited to, phase change memory (PRAM), static Random Access Memory (SRAM), dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), read Only Memory (ROM), electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium, which can be used to store information that can be accessed by a computing device.

The computer instructions stored in the storage medium of the foregoing embodiments are used to make the computer execute the RTP-based audio/video transmission method according to any one of the foregoing embodiments, and have the beneficial effects of the corresponding method embodiments, which are not described herein.

Those of ordinary skill in the art will appreciate that: the discussion of any of the embodiments above is merely exemplary and is not intended to suggest that the scope of the application (including the claims) is limited to these examples; the technical features of the above embodiments or in the different embodiments may also be combined within the idea of the application, the steps may be implemented in any order, and there are many other variations of the different aspects of the embodiments of the application as described above, which are not provided in detail for the sake of brevity.

Additionally, well-known power/ground connections to Integrated Circuit (IC) chips and other components may or may not be shown within the provided figures, in order to simplify the illustration and discussion, and so as not to obscure the embodiments of the present application. Furthermore, the devices may be shown in block diagram form in order to avoid obscuring the embodiments of the present application, and also in view of the fact that specifics with respect to implementation of such block diagram devices are highly dependent upon the platform within which the embodiments of the present application are to be implemented (i.e., such specifics should be well within purview of one skilled in the art). Where specific details (e.g., circuits) are set forth in order to describe example embodiments of the application, it should be apparent to one skilled in the art that embodiments of the application can be practiced without, or with variation of, these specific details. Accordingly, the description is to be regarded as illustrative in nature and not as restrictive.

While the application has been described in conjunction with specific embodiments thereof, many alternatives, modifications, and variations of those embodiments will be apparent to those skilled in the art in light of the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may use the embodiments discussed.

The present embodiments are intended to embrace all such alternatives, modifications and variances which fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, and the like, which are within the spirit and principles of the embodiments of the application, are intended to be included within the scope of the application.

Claims

1. An audio and video transmission method based on RTP, which is characterized by comprising the following steps:

receiving RTP data packets acquired by camera equipment;

2. The RTP-based audio and video transmission method according to claim 1, wherein the packet loss rate data statistics policy detects a packet loss rate of an RTP audio and video stream packet in a period of time, inserts packet loss rate data into a real-time audio and video stream in an SEI form in an h.264 or h.265 code stream, and wherein the packet loss rate data statistics policy is detected by using a sequence number field characteristic specified in an RTP protocol header, and includes:

3. The RTP-based audio/video transmission method according to claim 2, wherein the timing statistics of the packet loss rate of the RTP packet in a period of time according to a statistical time interval condition comprises:

4. The RTP-based audio and video transmission method according to claim 3, further comprising:

setting a timer and setting a timer time interval;

5. The RTP-based audio and video transmission method according to claim 1, wherein the extracting the packet loss rate SEI information in the audio and video stream for playing and presenting includes:

the player determines a video code stream according to a video coding format indicated in a specific field of the video packaging format, continuously identifies NALU units in the video code stream, and judges whether the NALU units are SEINALU units according to NALU type identification;

and in response to the SEINALU unit, judging whether the SEINALU unit is the packet loss rate data according to the SEI type in the SEI data packet and the custom global UUID value, and extracting the corresponding packet loss rate data from the SEINALU unit and displaying the data to the player.

6. An RTP-based audio and video transmission apparatus, the apparatus comprising:

7. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the RTP-based audio-video transmission method according to any one of claims 1-5 when executing the program.

8. A non-transitory computer readable storage medium, wherein the non-transitory computer readable storage medium stores computer instructions for causing the computer to perform the RTP-based audio-video transmission method of any one of claims 1-5.