CN113645192A - RTP data packet processing method and device - Google Patents
RTP data packet processing method and device Download PDFInfo
- Publication number
- CN113645192A CN113645192A CN202110807200.8A CN202110807200A CN113645192A CN 113645192 A CN113645192 A CN 113645192A CN 202110807200 A CN202110807200 A CN 202110807200A CN 113645192 A CN113645192 A CN 113645192A
- Authority
- CN
- China
- Prior art keywords
- packet
- rtp
- sequence number
- data packet
- image data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003672 processing method Methods 0.000 title claims description 19
- 238000000034 method Methods 0.000 claims abstract description 53
- 238000004458 analytical method Methods 0.000 claims description 8
- 230000008569 process Effects 0.000 abstract description 17
- 238000010586 diagram Methods 0.000 description 20
- 230000005540 biological transmission Effects 0.000 description 13
- 238000004590 computer program Methods 0.000 description 9
- 230000006870 function Effects 0.000 description 6
- 238000012163 sequencing technique Methods 0.000 description 6
- 230000002159 abnormal effect Effects 0.000 description 4
- 238000004364 calculation method Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 230000008901 benefit Effects 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 238000003491 array Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000002457 bidirectional effect Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 239000000126 substance Substances 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
- H04L65/60—Network streaming of media packets
- H04L65/65—Network streaming protocols, e.g. real-time transport protocol [RTP] or real-time control protocol [RTCP]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
- H04L69/22—Parsing or analysis of headers
Abstract
The application discloses a method and a device for processing an RTP data packet. The method comprises the following steps: receiving RTP data packets one by one; analyzing each RTP data packet to obtain a packet sequence number, a frame sequence number and image data of each RTP data packet; determining a corresponding storage position of the image data of each RTP data packet in a preset buffer area according to the packet sequence number and the frame sequence number of each RTP data packet, and storing the image data to the corresponding storage position, wherein the preset buffer area can store the image data of at least two frames of images; and combining the image data stored in the preset buffer area in the dimensionality of the image frame and then sending the image data to a decoder for decoding. The method only creates a buffer zone once in the whole process, and avoids the adoption of a mode of applying for a memory every time an RTP data packet is received; the storage position of the image data in the buffer area can be directly determined every time one RTP data packet is analyzed, the mode that the image data are completely received and then processed is avoided, and the high efficiency and the low time delay of the image processing process are ensured.
Description
Technical Field
The present application relates to the field of image data processing technologies, and in particular, to a method and an apparatus for processing an RTP packet.
Background
In the video image transmission process, the common practice is to packetize, depacketize, and packetize based on the RTP Protocol (Real-time Transport Protocol), and transmit and receive based on the UDP Protocol (User Datagram Protocol).
Commonly used types of image frames are IDR frame (Instantaneous Decoding Refresh), I frame (intra coded frame), P frame (inter prediction coded frame), and B frame (bidirectional prediction coded frame). In the case of a high resolution original image, such as 4K resolution, the size of these image frames may be larger than 1500 bytes of MTU (Maximum Transmission Unit) of network Transmission. Therefore, in order to transmit the image frames through the network, the image frames need to be packetized and transmitted, and the RTP protocol is used at this time. According to the RTP protocol, a sending end divides an image into a plurality of small data packets, namely RTP data packets, each RTP data packet is ensured to be smaller than an MTU (maximum Transmission Unit), then network sending is carried out through a UDP (user Datagram protocol), and at a receiving end, the RTP data packets corresponding to a frame of received image are analyzed and packaged to form a frame of complete image data. Each complete frame of data received is immediately submitted to the decoder for decoding.
However, the unreliable network transmission of the UDP protocol may cause problems such as packet loss and disorder of RTP packets, and for this reason, some schemes are provided in the prior art to handle problems such as disorder and packet loss during the transmission of RTP packets, but these schemes have low efficiency and cannot meet the requirement of low delay of the screen.
Disclosure of Invention
In view of this, a main objective of the present application is to provide a method and a device for processing an RTP packet, which are used to solve the technical problems of disorder and packet loss of the RTP packet during transmission.
According to a first aspect of the present application, there is provided an RTP packet processing method, including:
receiving RTP data packets one by one;
analyzing each RTP data packet to obtain a packet sequence number, a frame sequence number and image data of each RTP data packet;
determining a corresponding storage position of the image data of each RTP data packet in a preset buffer area according to the packet sequence number and the frame sequence number of each RTP data packet, and storing the image data to the corresponding storage position, wherein the preset buffer area can store the image data of at least two frames of images;
and combining the image data stored in the preset buffer area in the dimensionality of the image frame and then sending the image data to a decoder for decoding.
Optionally, the determining, according to the packet sequence number and the frame sequence number of each RTP data packet, a storage location of the image data of each RTP data packet in the preset buffer includes:
determining whether the packet sequence number of an RTP data packet before a currently received RTP data packet has the maximum value of the packet sequence number or not, and whether the packet sequence number of the currently received RTP data packet accords with a preset packet sequence number arrangement rule or not;
if not, determining the storage position of the image data of the currently received RTP data packet by using a first position determination strategy;
if yes, the storage position of the image data of the RTP data packet received currently is determined by the second position determination strategy.
Optionally, the determining, by using the first location determination policy, a storage location of image data of a currently received RTP packet includes:
determining the packet sequence number of the first RTP data packet in the preset buffer area;
and determining the storage position of the image data of the currently received RTP data packet according to the packet sequence number of the first RTP data packet, the packet sequence number of the currently received RTP data packet and the byte length of the image data.
Optionally, the determining, by using the second location determination policy, a storage location of image data of a currently received RTP packet includes:
setting a new number for the RTP data packet received currently according to an increasing arrangement rule;
and determining the storage position of the image data of the currently received RTP data packet according to the packet sequence number of the first RTP data packet, the new number of the currently received RTP data packet and the byte length of the image data.
Optionally, the determining, according to the packet sequence number and the frame sequence number of each RTP data packet, a storage location of the image data of each RTP data packet in the preset buffer includes:
determining a current image frame to which the currently received RTP data packet belongs according to the frame sequence number of the currently received RTP data packet;
determining whether the packet sequence number of the currently received RTP data packet conforms to the sequential incremental arrangement rule of the current image frame;
if not, determining the packet sequence number of the lost RTP data packet, recording the packet sequence number of the lost RTP data packet into a preset packet loss list, and deleting the corresponding packet sequence number from the preset packet loss list when the packet sequence number of the RTP data packet recorded in the preset packet loss list is subsequently received.
Optionally, the sending, after combining the image data stored in the preset buffer in the dimensionality of the image frame, the image data to a decoder for decoding includes:
determining whether the RTP data packet of the current image frame is received completely according to the preset packet loss list;
if the receiving is finished, combining the image data of the RTP data packet of the current image frame and then sending the combined image data to a decoder for decoding;
if the receiving is not finished, whether the image data of the RTP data packet of the current image frame is combined or not is determined after the preset time, and then the combined image data is sent to a decoder for decoding.
Optionally, the determining, according to the preset packet loss list, whether the RTP packet of the current image frame is received completely includes:
determining whether the currently received RTP data packet is the last RTP data packet in the current image frame according to the analysis result of the currently received RTP data packet;
if yes, determining whether a packet sequence number of an RTP data packet in the current image frame exists in the preset packet loss list;
if yes, determining that the RTP data packet of the current image frame is not received completely;
if not, determining that the RTP data packet of the current image frame is received completely.
Optionally, after combining the image data of the RTP packets of the current image frame and sending the combined image data to a decoder for decoding, the method further includes:
clearing all storage positions occupied by image data of an RTP data packet of a current image frame in a preset buffer zone;
and moving the image data of the RTP data packet of the next image frame to the forefront of the preset buffer area and re-determining the packet sequence number of the first RTP data packet in the preset buffer area.
Optionally, the determining whether to combine the image data of the RTP data packet of the current image frame after the preset time and send the combined image data to a decoder for decoding includes:
determining whether an RTP data packet lost by a current image frame can be received after a preset time;
if so, combining the image data of the RTP data packet of the current image frame and then sending the combined image data to a decoder for decoding;
if not, performing packet loss processing on the RTP data packet of the current image frame, wherein the packet loss processing comprises the following modes:
the method comprises the steps that an RTP data packet lost by a current image frame is stored in a corresponding storage position in a preset buffer area, and image data of the RTP data packet of the current image frame are combined and then sent to a decoder for decoding; and/or the presence of a gas in the gas,
discarding the current image frame, not receiving the RTP data packet of the subsequent image frame in the current GOP group of pictures to which the current image frame belongs, and informing the sending end to resend the new GOP group of pictures.
According to a second aspect of the present application, there is provided an RTP packet processing apparatus including:
a receiving unit, configured to receive RTP packets one by one;
the analysis unit is used for analyzing each RTP data packet to obtain a packet sequence number, a frame sequence number and image data of each RTP data packet;
a determining unit, configured to determine, according to a packet sequence number and a frame sequence number of each RTP data packet, a corresponding storage location of image data of each RTP data packet in a preset buffer area, and store the image data to the corresponding storage location, where the preset buffer area is capable of storing image data of at least two frames of images;
and the sending unit is used for combining the image data stored in the preset buffer area in the dimensionality of the image frame and then sending the image data to a decoder for decoding.
In accordance with a third aspect of the present application, there is provided an electronic device comprising: a processor, a memory storing computer-executable instructions,
the executable instructions, when executed by the processor, implement the RTP packet processing method.
According to a fourth aspect of the present application, there is provided a computer-readable storage medium storing one or more programs which, when executed by a processor, implement the foregoing RTP packet processing method.
The beneficial effect of this application is: the RTP data packet processing method of the embodiment of the application firstly receives RTP data packets one by one; then, analyzing each RTP data packet to obtain a packet sequence number, a frame sequence number and image data of each RTP data packet; then according to the packet sequence number and the frame sequence number of each RTP data packet, determining a corresponding storage position of the image data of each RTP data packet in a preset buffer area, and storing the image data to the corresponding storage position, wherein the preset buffer area can store the image data of at least two frames of images; and finally, combining the image data stored in the preset buffer area in the dimensionality of the image frame and then sending the image data to a decoder for decoding. The RTP data packet processing method of the embodiment of the application only creates a buffer zone once in the whole process, and avoids adopting a mode of applying for a memory every time an RTP data packet is received; in addition, the storage position of the image data in the buffer area is directly determined in the receiving and analyzing process of each RTP data packet, the mode of sequencing after all the RTP data packets are received is avoided, and the high efficiency and the low time delay of the picture in the processing process are ensured.
Drawings
Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are only for purposes of illustrating the preferred embodiments and are not to be construed as limiting the application. Also, like reference numerals are used to refer to like parts throughout the drawings. In the drawings:
fig. 1 is a schematic diagram of a protocol format of an RTP packet in the prior art;
fig. 2 is a schematic flowchart of an RTP packet processing method according to an embodiment of the present application;
FIG. 3 is a diagram illustrating an RTP extension header format according to an embodiment of the present application;
fig. 4 is a schematic diagram illustrating a storage location of image data of an RTP packet in a predetermined buffer according to an embodiment of the present application;
FIG. 5 is a diagram illustrating a structure of a GOP group of pictures in the prior art;
fig. 6 is a block diagram of an RTP packet processing apparatus according to an embodiment of the present application;
fig. 7 is a schematic structural diagram of an electronic device in an embodiment of the present application.
Detailed Description
Exemplary embodiments of the present application will be described in more detail below with reference to the accompanying drawings. These embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art. While exemplary embodiments of the present application are shown in the drawings, it should be understood that the present application may be embodied in various forms and should not be limited to the embodiments set forth herein.
To facilitate understanding of the embodiments of the present application, as shown in fig. 1, a schematic diagram of a protocol format of RTP data packets in the prior art is provided, and as can be seen from fig. 1, each RTP data packet has a corresponding sequence number (serial number), the sequence numbers have continuity between multiple frames, and are arranged in sequence from small to large, that is, the sequence number of a first RTP data packet of a first frame image is from 0 to n, the sequence number of a first RTP data packet of a second frame image is from n +1, and according to a header format of RTP, the sequence number has only two bytes, so that the sequence number ranges from 0 to 65535, and 0 to 65535 are used cyclically, that is, if the sequence number of one RTP data packet is 65535, the sequence number of a next generated RTP data packet is 0.
Due to the unreliable network transmission of the UDP protocol, the problems of packet loss, disorder and the like of the RTP data packet can be caused. The packet loss means that data is lost in the transmission process and does not reach the designated destination address. The disorder refers to that at a receiving end, an RTP data packet with a large sequence number is received first, and then a data packet with a small sequence number is received (except the case that the former is 65535, and the latter is 0), the disorder can be specifically divided into intra-frame disorder and inter-frame disorder, and the intra-frame disorder refers to the disorder of the RTP data packet in one frame; the inter-frame disorder refers to the disorder of the RTP data packets between two or more frames, that is, all the RTP data packets of the previous frame are not completely received, and the RTP data packets of the next frame are already received.
Due to the problems of packet loss and disorder, the image data received by the receiving end is different from the original image data sent by the sending end, so that the decoder cannot correctly decode the image data, and finally the problem of screen splash of the picture occurs.
In this regard, some solutions are provided in the prior art, such as:
1) and (3) sequencing after all receiving: when receiving an RTP data packet, applying for a memory, after receiving all RTP data packets of the current image frame, sequencing, and copying the image data of each RTP data packet to a buffer area after sequencing.
2) Storing and sorting a linked list: that is, each RTP packet corresponds to a node in a linked list, and when receiving an RTP packet, the sequence number of the RTP packet is determined from the first node to determine the position of the current RTP packet, and the image data of each RTP packet is copied to a buffer after the sequence.
However, the first method has low efficiency because it needs to apply for a memory every time an RTP packet is received, and the second method still has low efficiency because it needs to continuously traverse the data in the linked list, and cannot meet the requirement of low delay.
Based on this, fig. 2 shows a schematic flow chart of an RTP packet processing method according to an embodiment of the present application, and referring to fig. 2, the RTP packet processing method according to the embodiment of the present application includes the following steps S210 to S240:
step S210, receiving RTP packets one by one.
The RTP packet processing method according to the embodiment of the present application may be an operation executed by a data receiving end, and when processing an RTP packet, the data receiving end needs to receive the RTP packet first.
Step S220, parsing each RTP packet to obtain a packet sequence number, a frame sequence number, and image data of each RTP packet.
Each time an RTP packet is received, the RTP packet can be parsed, so that a packet sequence number, a frame index and image data corresponding to the RTP packet are obtained. The standard RTP data packets all need to have a sequence number field, and a sending end needs to set the value of the sequence number field for each RTP data packet when sending the RTP data packets, wherein the value is recycled from 0 to 65535. The frame sequence number, frame index, is used to indicate the image frame to which the RTP packet belongs, and when the transmitting end sends the RTP packet, the transmitting end sets the frame index for each RTP packet, and then places the frame index in the extension header portion of the RTP packet.
Each RTP requires a fixed 12-byte header, the extension header needs to accommodate a frame index of 4 bytes, and the defined by profile and length fields of the extension header also need 4 bytes, so the total length of the RTP protocol field is 12+4+ 4-20 bytes. The network MTU is 1500 bytes, and in order to enable each RTP packet to be normally transmitted in the network, the length of the image data portion in each RTP packet is set to 1400 bytes, and in summary, the total length of one RTP packet may be 1400+ 20-1420 bytes. As shown in fig. 3, a schematic diagram of an RTP extension header format according to an embodiment of the present application is provided.
Step S230, determining a corresponding storage location of the image data of each RTP data packet in a preset buffer according to the packet sequence number and the frame sequence number of each RTP data packet, and storing the image data to the corresponding storage location, where the preset buffer can store the image data of at least two frames of images.
In order to store the image data of the RTP data packet, in the embodiment of the present application, a buffer is created in advance in the receiving end for storing the image data of the RTP data packet. Since the delay of the RTP packet transmission does not usually exceed one frame, the size of the buffer can be set to a size enough to store the image data of two frames of images.
Each RTP data packet contains image data, the packet sequence number and the frame sequence number of the RTP data packet contained in each RTP data packet and the image data can be obtained by analyzing each RTP data packet, then the storage position of the image data in the buffer zone can be determined according to the packet sequence number and the frame sequence number of the RTP data packet, and finally the image data is placed at the corresponding position of the buffer zone.
Step S240, combining the image data stored in the preset buffer in the dimensionality of the image frame, and sending the combined image data to a decoder for decoding.
As described above, one frame of image is split into a plurality of RTP packets before transmission, so in order to restore the image data in the RTP packets to the original image, all the image data included in one frame of image need to be combined together and sent to a decoder for decoding in the dimension of the image frame, so as to obtain a complete original image.
The RTP data packet processing method of the embodiment of the application only creates a buffer zone once in the whole process, and avoids adopting a mode of applying for a memory every time an RTP data packet is received; in addition, the storage position of the image data in the buffer area is directly determined in the receiving and analyzing process of each RTP data packet, the mode of sequencing after all the RTP data packets are received is avoided, and the high efficiency and the low time delay of the picture in the processing process are ensured.
In one embodiment of the present application, it is determined whether a packet sequence number of an RTP data packet before a currently received RTP data packet has a maximum value of a packet sequence number, and whether the packet sequence number of the currently received RTP data packet conforms to a preset packet sequence number arrangement rule; if not, determining the storage position of the image data of the currently received RTP data packet by using a first position determination strategy; if yes, the storage position of the image data of the RTP data packet received currently is determined by the second position determination strategy.
When determining the storage position of the image data corresponding to the preset buffer area, the embodiment of the application may face the following situations:
1) normal conditions are as follows: this is the case that the packet sequence numbers of RTP packets in a frame are sequentially arranged in an increasing order, for example, the packet sequence numbers are sequentially arranged in an increasing order within the ranges of 200, 201, 202, … …, and 300;
2) special cases are as follows: this case means that the packet sequence numbers of RTP packets in a frame are not all arranged in an incremental manner, specifically, the packet sequence number of a middle RTP packet in a frame reaches a maximum value of 65535, and then the packet sequence number of the next RTP packet in the frame is 0;
3) abnormal conditions are as follows: this case refers to the case of out-of-order or packet loss, and as the packet numbers of the RTP packets received in the previous frame are 201, 202, 204, and 205 … … in this order, when the RTP packet of the current frame is received, the RTP packet with the frame number of 203 is received, the frame-crossing out-of-order occurs.
For the above special cases, if the packet number of the RTP data packet before the currently received RTP data packet has reached the maximum value 65535 of the packet number, the RTP data packet continuously sent by the sending end will be renumbered from 0, so that if the packet number of the currently received RTP data packet at this time conforms to the preset packet number arrangement rule, the storage location of the image data of the currently received RTP data packet may be determined by using the second location determination policy defined in advance in the embodiment of the present application. The preset packet sequence number arrangement rule may be understood as a series of packet sequence numbers of RTP packets that should be obtained by renumbering RTP packets from 0 after the packet sequence number of the RTP packet has reached the maximum value 65535 of the packet sequence number, for example, the following arrangement forms may be considered as the case meeting the preset packet sequence number arrangement rule:
1) … …, 65535, 0, 1, 2, 3 (current), … …;
2) … …, 65535, 0, 1, 3, 2 (current), … ….
For the above normal situation and abnormal situation, if the packet sequence number of the RTP data packet does not reach the maximum value of the packet sequence number before the currently received RTP data packet, or the packet sequence number of the currently received RTP data packet does not conform to the preset packet sequence number arrangement rule, the storage location of the image data of the currently received RTP data packet may be determined by using the first location determination policy defined in advance in the embodiment of the present application. For example, the following arrangement is possible:
1) normal conditions are as follows: 0. 1, 2, 3 (current), … …, 65535;
2) abnormal conditions are as follows: … …, 65533, 65535, 0, 65534 (current), 1, 2, … ….
In one embodiment of the present application, determining a storage location of image data of a currently received RTP packet using a first location determination policy comprises: determining the packet sequence number of a first RTP data packet in a preset buffer area; and determining the storage position of the image data of the currently received RTP data packet according to the packet sequence number of the first RTP data packet, the packet sequence number of the currently received RTP data packet and the byte length of the image data.
In the embodiment of the present application, when determining the storage location of the image data of the currently received RTP packet by using the first location determining policy, the packet sequence number first _ sn of the first RTP packet stored in the current preset buffer, that is, the packet sequence number of the RTP packet located at the forefront of the preset buffer, may be determined first, and then the storage location of the image data of the currently received RTP packet may be calculated according to the packet sequence number first _ sn of the first RTP packet, the packet sequence number of the currently received RTP packet, and the byte length of the image data, which is as 1400, where specifically, the following form may be expressed:
storage position (sequence number-first _ sn) 1400, (1)
For example, assuming that the packet sequence number first _ sn of the first RTP packet in the current preset buffer is 200, if the packet sequence number of the currently received RTP packet is 201, the storage location of the image data of the RTP packet in the buffer is (201-.
If the packet sequence numbers of the RTP data packets conform to the normal condition of the above embodiment, that is, the packet sequence numbers of the RTP data packets in a frame are sequentially arranged in an increasing order, then according to the above logic, the specific storage location of the image data of each RTP data packet in the buffer area can be sequentially obtained. If the packet sequence number of the RTP packet is abnormal, that is, the frame crossing disorder occurs, the specific storage location of the image data of the RTP packet in the buffer can still be calculated according to the logic.
For example, the packet numbers of the RTP packets received in the previous frame are 201, 202, and 204 in sequence, and the storage positions of the image data of the RTP packets with the packet numbers 201, 202, and 204 in the buffer can be calculated by the above formula (1), and at this time, the storage position of the image data of the RTP packet with the packet number 203 in the buffer is empty. When receiving the RTP packet of the current frame, if the RTP packet with the packet sequence number 203 belonging to the previous frame image is received, the storage position of the image data of the RTP packet with the packet sequence number 203 in the buffer area can still be calculated according to the above formula (1). That is to say, the situation of disorder of the packet serial numbers does not affect the embodiment of the present application to directly determine the storage location by using the above formula (1), as long as the above special situation does not occur.
In one embodiment of the present application, determining a storage location of image data of a currently received RTP packet using the second location determination policy comprises: setting a new number for the RTP data packet received currently according to an increasing arrangement rule; and determining the storage position of the image data of the currently received RTP data packet according to the packet sequence number of the first RTP data packet, the new sequence number of the currently received RTP data packet and the byte length of the image data.
As described above, in the embodiment of the present application, when determining the storage location of the image data of the currently received RTP packet by using the second location determination policy, the situation is mainly that the packet sequence number of a certain RTP packet in the middle of a frame reaches the maximum value 65535, if the storage location of the RTP packet received subsequently is determined directly according to the above formula (1), a calculation error will occur, therefore, the second position determination policy adopted in this embodiment of the present application may be to continue to increment the packet sequence number of the RTP packet according to the increment rule, that is, although the packet sequence number of the next RTP packet is 0, it is recorded as 65536 during the processing, and then it is recorded as 65537, 65538 … …, by analogy, assuming that the packet sequence number of the subsequent RTP packet in the current frame is n, the new number recorded in the processing process can be represented as n +65535+1 until all the RTP packets in the frame image are received.
If the renumbering process does not have the conditions of disorder, packet loss and the like, the storage position of the image data of the RTP data packet of the frame is calculated according to the calculation method of the formula (1); if the problems of disorder and packet loss occur, the packet sequence number of the RTP packet of the frame can still be calculated according to the calculation method of the above formula (1).
For example, assuming that the packet sequence number of a certain RTP packet in the middle of a frame has reached the maximum value of 65535, and the packet sequence number of the currently received RTP packet is 5, the RTP packet may be renumbered as 5+65535+1 as 65541, and then the storage location of the image data of the RTP packet is calculated as (65541-first _ sn) × 1400 according to the formula (1).
In an embodiment of the present application, determining, according to the packet number and the frame number of each RTP packet, a storage location of image data of each RTP packet in a preset buffer includes: determining a current image frame to which the currently received RTP data packet belongs according to the frame sequence number of the currently received RTP data packet; determining whether the packet sequence number of the currently received RTP data packet conforms to the sequential incremental arrangement rule of the current image frame; if not, determining the packet sequence number of the lost RTP data packet, recording the packet sequence number of the lost RTP data packet into a preset packet loss list, and deleting the corresponding packet sequence number from the preset packet loss list when the packet sequence number of the RTP data packet recorded in the preset packet loss list is subsequently received.
As described above, in order to determine whether an RTP packet of a frame of image is received completely, in the embodiment of the present application, a packet loss list may be established in advance for storing information such as a packet sequence number of a lost RTP packet, when determining whether a packet loss occurs, the embodiment may first determine which image frame the currently received RTP packet belongs to according to a frame sequence number of the currently received RTP packet, then determine whether a packet sequence number of the RTP packet conforms to a sequential increasing arrangement rule of the current image frame, and if not, may further determine the packet sequence number of the lost RTP packet, and further may record the packet sequence number of the lost RTP packet in a preset packet loss list.
For example, suppose that an RTP packet with sequence number 5 is currently received, and a next RTP packet with sequence number 8 is received, and this indicates that packet misordering or packet loss occurs, so that sequence number 6 and sequence number 7 may be first added to a preset packet loss list, and if an RTP packet with a corresponding packet number is subsequently received, the corresponding record may be deleted from the preset packet loss list, for example, if a packet with sequence number 6 is subsequently received, the record with sequence number 6 may be deleted from the preset packet loss list.
In an embodiment of the present application, the sending the image data stored in the preset buffer to a decoder for decoding after combining the dimensions of the image frames includes: determining whether the RTP data packet of the current image frame is received completely according to a preset packet loss list; if the receiving is finished, combining the image data of the RTP data packet of the current image frame and then sending the combined image data to a decoder for decoding; if the receiving is not finished, whether the image data of the RTP data packet of the current image frame is combined or not is determined after the preset time, and then the combined image data is sent to a decoder for decoding.
The condition that each frame of image is lost is recorded in the preset packet loss list, and the record is continuously updated according to the subsequently received RTP data packets, so that the embodiment of the application can determine whether the RTP data packets of the current image frame are completely received according to the information recorded in the preset packet loss list, if the packet sequence number of the RTP data packet of the current image frame still exists in the preset packet loss list, the RTP data packet of the current image frame is not completely received, the judgment can be performed after waiting for a certain time, and if the packet sequence number of the RTP data packet of the current image frame does not exist in the preset packet loss list, the RTP data packet of the current image frame is completely received, the image data of the RTP data packet of the current image frame can be combined and then sent to a decoder for decoding.
In an embodiment of the present application, determining whether receiving of an RTP packet of a current image frame is completed according to a preset packet loss list includes: determining whether the currently received RTP data packet is the last RTP data packet in the current image frame according to the analysis result of the currently received RTP data packet; if yes, determining whether a packet sequence number of an RTP data packet in the current image frame exists in a preset packet loss list; if yes, determining that the RTP data packet of the current image frame is not received completely; if not, determining that the RTP data packet of the current image frame is received completely.
According to the embodiment of the application, when determining whether the RTP data packet of the current image frame is completely received according to the preset packet loss list, an analysis result of the currently received RTP data packet may be obtained first, and according to the existing RTP protocol, an identifier indicating whether the current RTP data packet is a header packet, a middle packet, or an end packet of a frame image may be recorded in a header format of the RTP data packet, so that whether the currently received RTP data packet is a last RTP data packet of the current image frame may be determined according to the analysis result, if so, whether a packet sequence number of the RTP data packet in the current image frame exists in the preset packet loss list may be further determined, if so, it is determined that the RTP data packet of the current image frame is not completely received, and if not, it is determined that all the RTP data packets of the current image frame are completely received.
In one embodiment of the present application, after combining the image data of the RTP packets of the current image frame and sending the combined image data to a decoder for decoding, the method further includes: clearing all storage positions occupied by image data of an RTP data packet of a current image frame in a preset buffer zone; and moving the image data of the RTP data packet of the next image frame to the forefront of the preset buffer area and re-determining the packet sequence number of the first RTP data packet in the preset buffer area.
When it is determined that the RTP data packets of one frame of image are completely received, the image data of the RTP data packet of the frame of image is sent to a decoder for decoding, so that the image data of the frame of image is completely transmitted, the positions occupied by the image data of the RTP data packet of the frame of image in the preset buffer area can be completely emptied, and the image data of the RTP data packet of the next image frame is moved to the front of the preset buffer area, so as to continuously store the image data of the subsequent image frame.
Since the embodiment of the present application depends on the first packet sequence number, i.e. first _ sn, of the first RTP packet stored at the forefront in the entire preset buffer when determining the storage location, after moving the image data of the RTP packet of the next image frame to the forefront of the preset buffer, the packet sequence number of the first RTP packet in the preset buffer should also change, and therefore, the value of first _ sn should be updated synchronously.
In one embodiment of the present application, determining whether to combine image data of RTP packets of a current image frame and send the combined image data to a decoder for decoding after a preset time includes: determining whether an RTP data packet lost by a current image frame can be received after a preset time; if so, combining the image data of the RTP data packet of the current image frame and then sending the combined image data to a decoder for decoding; if not, performing packet loss processing on the RTP data packet of the current image frame, wherein the packet loss processing comprises the following modes: the method comprises the steps that an RTP data packet lost by a current image frame is stored in a corresponding storage position in a preset buffer area, and image data of the RTP data packet of the current image frame are combined and then sent to a decoder for decoding; and/or discarding the current image frame, not receiving the RTP data packet of the subsequent image frame in the current GOP group of pictures to which the current image frame belongs any more, and informing the sending end to resend the new GOP group of pictures.
When the currently received RTP packet is the last RTP packet of the current image frame, if the packet sequence number of the RTP packet of the current image frame still exists in the preset packet loss list at this time, it indicates that the RTP packet of the current image frame is not completely received, at this time, a preset time may be waited to see whether the lost RTP packet can be received, where the size of the preset time may be defined according to a specific requirement, for example, the preset time may be a time when 10 RTP packets are received, or may be a specific period of time, such as 100ms, and the like, and is not specifically limited herein.
If the RTP data packet lost by the current image frame can be received after the preset time, the image data of the RTP data packet of the current image frame can be combined and then sent to a decoder for decoding, and if the RTP data packet of the current image frame can not be completely received, the RTP data packet of the current image frame can be subjected to packet loss processing.
The packet loss processing method provided by the embodiment of the present application may include the following two methods:
1) and submitting the image data of the received RTP data packet to a decoder, wherein the corresponding position of the RTP data packet which is not received is empty in a preset buffer area. As shown in fig. 4, a schematic diagram of a storage location of image data of an RTP packet in a preset buffer according to an embodiment of the present application is provided, and assuming that a 10 th RTP packet of a current image frame is lost, image data of 9 th and 11 th RTP packets still leave the storage location of image data of the 10 th RTP packet when stored in the buffer, so that the processing is more friendly to a decoding process of a decoder, but a problem of screen blurring of a picture due to the loss of the image data is caused;
2) discarding the current image frame, and not receiving the subsequent image frame in the current GOP group to which the current image frame belongs, and simultaneously informing the sending end to immediately start sending a new GOP group, namely immediately sending a new IDR frame.
After dropping one frame in a group of GOP pictures, since decoding of the following frame will be failed, in order to avoid this problem, the above-mentioned mode 2) can drop all the following frames and inform the transmitting end to immediately transmit a new group of GOP pictures.
GOP (group of pictures) refers to a group of encoded image frames, the number of frames can be Set, the first frame is generally a unique IDR frame or I frame in a GOP, and the last frame is some image related information, such as SPS (Sequence parameter Set), PPS (Picture parameter Set), etc., and these are Set according to the related rules of the encoder. An IDR frame or I frame can be decoded independently of other frames, P frame decoding requires reference to previous I or P frames, and B frames require dependency on previous or subsequent frames. As shown in fig. 5, a schematic diagram of a GOP group picture structure in the prior art is provided.
In summary, the RTP packet processing method of the present application at least achieves the following technical effects:
1) only one buffer area is created in the whole RTP data packet processing flow, and the mode of applying for one memory every time one RTP data packet is received is avoided, so that the method is more efficient and faster;
2) the position of the image data in the buffer area is directly determined in the receiving and analyzing processes of each RTP data packet, the mode of sequencing after all RTP data packets are received is avoided, and the high efficiency and the low time delay of the image processing process are ensured;
3) the problem that a linked list storage ordering method needs to be traversed continuously is solved, and the processing efficiency is improved;
4) on the whole, in two dimensions of space complexity and time complexity, smaller processing overhead is achieved, and a scheme for effectively and feasibly processing the problems of disorder, packet loss and the like in the network transmission process of the RTP data packet is provided.
The RTP packet processing method and the RTP packet processing device belong to the same technical concept, and the embodiment of the application further provides the RTP packet processing device. Fig. 6 shows a block diagram of an RTP packet processing apparatus according to an embodiment of the present application, and referring to fig. 6, the RTP packet processing apparatus 600 includes: a receiving unit 610, a parsing unit 620, a determining unit 630, and a transmitting unit 640. Wherein the content of the first and second substances,
a receiving unit 610, configured to receive RTP packets one by one;
the parsing unit 620 is configured to parse each RTP data packet to obtain a packet sequence number, a frame sequence number, and image data of each RTP data packet;
a determining unit 630, configured to determine, according to the packet sequence number and the frame sequence number of each RTP data packet, a corresponding storage location of the image data of each RTP data packet in a preset buffer area, and store the image data to the corresponding storage location, where the preset buffer area is capable of storing the image data of at least two frames of images;
the sending unit 640 is configured to combine the image data stored in the preset buffer in the dimensionality of the image frame and send the combined image data to a decoder for decoding.
In an embodiment of the present application, the determining unit 630 is specifically configured to: determining whether the packet sequence number of an RTP data packet before a currently received RTP data packet has the maximum value of the packet sequence number or not, and whether the packet sequence number of the currently received RTP data packet accords with a preset packet sequence number arrangement rule or not; if not, determining the storage position of the image data of the currently received RTP data packet by using a first position determination strategy; if yes, the storage position of the image data of the RTP data packet received currently is determined by the second position determination strategy.
In an embodiment of the present application, the determining unit 630 is specifically configured to: determining the packet sequence number of a first RTP data packet in a preset buffer area; and determining the storage position of the image data of the currently received RTP data packet according to the packet sequence number of the first RTP data packet, the packet sequence number of the currently received RTP data packet and the byte length of the image data.
In an embodiment of the present application, the determining unit 630 is specifically configured to: setting a new number for the RTP data packet received currently according to an increasing arrangement rule; and determining the storage position of the image data of the currently received RTP data packet according to the packet sequence number of the first RTP data packet, the new sequence number of the currently received RTP data packet and the byte length of the image data.
In an embodiment of the present application, the determining unit 630 is specifically configured to: determining a current image frame to which the currently received RTP data packet belongs according to the frame sequence number of the currently received RTP data packet; determining whether the packet sequence number of the currently received RTP data packet conforms to the sequential incremental arrangement rule of the current image frame; if not, determining the packet sequence number of the lost RTP data packet, recording the packet sequence number of the lost RTP data packet into a preset packet loss list, and deleting the corresponding packet sequence number from the preset packet loss list when the packet sequence number of the RTP data packet recorded in the preset packet loss list is subsequently received.
In an embodiment of the present application, the sending unit 640 is specifically configured to: determining whether the RTP data packet of the current image frame is received completely according to a preset packet loss list; if the receiving is finished, combining the image data of the RTP data packet of the current image frame and then sending the combined image data to a decoder for decoding; if the receiving is not finished, whether the image data of the RTP data packet of the current image frame is combined or not is determined after the preset time, and then the combined image data is sent to a decoder for decoding.
In an embodiment of the present application, the sending unit 640 is specifically configured to: determining whether the currently received RTP data packet is the last RTP data packet in the current image frame according to the analysis result of the currently received RTP data packet; if yes, determining whether a packet sequence number of an RTP data packet in the current image frame exists in a preset packet loss list; if yes, determining that the RTP data packet of the current image frame is not received completely; if not, determining that the RTP data packet of the current image frame is received completely.
In one embodiment of the present application, the apparatus further comprises: the clearing unit is used for clearing all storage positions occupied by the image data of the RTP data packet of the current image frame in the preset buffer area; and the moving unit is used for moving the image data of the RTP data packet of the next image frame to the forefront of the preset buffer area and re-determining the packet sequence number of the first RTP data packet in the preset buffer area.
In an embodiment of the present application, the sending unit 640 is specifically configured to: determining whether an RTP data packet lost by a current image frame can be received after a preset time; if so, combining the image data of the RTP data packet of the current image frame and then sending the combined image data to a decoder for decoding; if not, performing packet loss processing on the RTP data packet of the current image frame, wherein the packet loss processing comprises the following modes: the method comprises the steps that an RTP data packet lost by a current image frame is stored in a corresponding storage position in a preset buffer area, and image data of the RTP data packet of the current image frame are combined and then sent to a decoder for decoding; and/or discarding the current image frame, not receiving the RTP data packet of the subsequent image frame in the current GOP group of pictures to which the current image frame belongs any more, and informing the sending end to resend the new GOP group of pictures.
It should be noted that:
fig. 7 illustrates a schematic structural diagram of an electronic device. Referring to fig. 7, at a hardware level, the electronic device includes a memory and a processor, and optionally further includes an interface module, a communication module, and the like. The Memory may include a Memory, such as a Random-Access Memory (RAM), and may also include a non-volatile Memory, such as at least one disk Memory. Of course, the electronic device may also include hardware required for other services.
The processor, the interface module, the communication module, and the memory may be connected to each other via an internal bus, which may be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, an EISA (Extended Industry Standard Architecture) bus, or the like. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one double-headed arrow is shown in FIG. 7, but this does not indicate only one bus or one type of bus.
A memory for storing computer executable instructions. The memory provides computer executable instructions to the processor through the internal bus.
A processor executing computer executable instructions stored in the memory and specifically configured to perform the following operations:
receiving RTP data packets one by one;
analyzing each RTP data packet to obtain a packet sequence number, a frame sequence number and image data of each RTP data packet;
determining a corresponding storage position of the image data of each RTP data packet in a preset buffer area according to the packet sequence number and the frame sequence number of each RTP data packet, and storing the image data to the corresponding storage position, wherein the preset buffer area can store the image data of at least two frames of images;
and combining the image data stored in the preset buffer area in the dimensionality of the image frame and then sending the image data to a decoder for decoding.
The functions performed by the RTP packet processing apparatus according to the embodiment shown in fig. 6 of the present application may be implemented in or by a processor. The processor may be an integrated circuit chip having signal processing capabilities. In implementation, the steps of the above method may be performed by integrated logic circuits of hardware in a processor or instructions in the form of software. The Processor may be a general-purpose Processor, including a Central Processing Unit (CPU), a Network Processor (NP), and the like; but also Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs) or other Programmable logic devices, discrete Gate or transistor logic devices, discrete hardware components. The various methods, steps, and logic blocks disclosed in the embodiments of the present application may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of the method disclosed in connection with the embodiments of the present application may be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software modules in the decoding processor. The software module may be located in ram, flash memory, rom, prom, or eprom, registers, etc. storage media as is well known in the art. The storage medium is located in a memory, and a processor reads information in the memory and completes the steps of the method in combination with hardware of the processor.
The electronic device may further execute the steps executed by the RTP packet processing method in fig. 2, and implement the functions of the RTP packet processing method in the embodiment shown in fig. 2, which are not described herein again in this embodiment of the present application.
An embodiment of the present application further provides a computer-readable storage medium, where the computer-readable storage medium stores one or more programs, and when the one or more programs are executed by a processor, the one or more programs implement the foregoing RTP packet processing method, and are specifically configured to perform:
receiving RTP data packets one by one;
analyzing each RTP data packet to obtain a packet sequence number, a frame sequence number and image data of each RTP data packet;
determining a corresponding storage position of the image data of each RTP data packet in a preset buffer area according to the packet sequence number and the frame sequence number of each RTP data packet, and storing the image data to the corresponding storage position, wherein the preset buffer area can store the image data of at least two frames of images;
and combining the image data stored in the preset buffer area in the dimensionality of the image frame and then sending the image data to a decoder for decoding.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.
The present application is described in terms of flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
In a typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.
The memory may include forms of volatile memory in a computer readable medium, Random Access Memory (RAM) and/or non-volatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of a computer-readable medium.
Computer-readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), Digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, a computer readable medium does not include a transitory computer readable medium such as a modulated data signal and a carrier wave.
It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in the process, method, article, or apparatus that comprises the element.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) characterized by computer-usable program code.
The above are merely examples of the present application and are not intended to limit the present application. Various modifications and changes may occur to those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.
Claims (10)
1. An RTP packet processing method, comprising:
receiving RTP data packets one by one;
analyzing each RTP data packet to obtain a packet sequence number, a frame sequence number and image data of each RTP data packet;
determining a corresponding storage position of the image data of each RTP data packet in a preset buffer area according to the packet sequence number and the frame sequence number of each RTP data packet, and storing the image data to the corresponding storage position, wherein the preset buffer area can store the image data of at least two frames of images;
and combining the image data stored in the preset buffer area in the dimensionality of the image frame and then sending the image data to a decoder for decoding.
2. The method of claim 1, wherein the determining, according to the packet sequence number and the frame sequence number of each RTP packet, a corresponding storage location of the image data of each RTP packet in a preset buffer comprises:
determining whether the packet sequence number of an RTP data packet before a currently received RTP data packet has the maximum value of the packet sequence number or not, and whether the packet sequence number of the currently received RTP data packet accords with a preset packet sequence number arrangement rule or not;
if not, determining the storage position of the image data of the currently received RTP data packet by using a first position determination strategy;
if yes, the storage position of the image data of the RTP data packet received currently is determined by the second position determination strategy.
3. The method of claim 2, wherein determining the storage location of the image data of the currently received RTP packet using the first location determination policy comprises:
determining the packet sequence number of the first RTP data packet in the preset buffer area;
and determining the storage position of the image data of the currently received RTP data packet according to the packet sequence number of the first RTP data packet, the packet sequence number of the currently received RTP data packet and the byte length of the image data.
4. The method of claim 2, wherein determining the storage location of the image data of the currently received RTP packet using the second location determination policy comprises:
setting a new number for the RTP data packet received currently according to an increasing arrangement rule;
and determining the storage position of the image data of the currently received RTP data packet according to the packet sequence number of the first RTP data packet, the new number of the currently received RTP data packet and the byte length of the image data.
5. The method of claim 1, wherein the determining, according to the packet sequence number and the frame sequence number of each RTP packet, a corresponding storage location of the image data of each RTP packet in a preset buffer comprises:
determining a current image frame to which the currently received RTP data packet belongs according to the frame sequence number of the currently received RTP data packet;
determining whether the packet sequence number of the currently received RTP data packet conforms to the sequential incremental arrangement rule of the current image frame;
if not, determining the packet sequence number of the lost RTP data packet, recording the packet sequence number of the lost RTP data packet into a preset packet loss list, and deleting the corresponding packet sequence number from the preset packet loss list when the packet sequence number of the RTP data packet recorded in the preset packet loss list is subsequently received.
6. The method of claim 5, wherein the sending the image data stored in the preset buffer to a decoder for decoding after combining dimensions of image frames comprises:
determining whether the RTP data packet of the current image frame is received completely according to the preset packet loss list;
if the receiving is finished, combining the image data of the RTP data packet of the current image frame and then sending the combined image data to a decoder for decoding;
if the receiving is not finished, whether the image data of the RTP data packet of the current image frame is combined or not is determined after the preset time, and then the combined image data is sent to a decoder for decoding.
7. The method of claim 6, wherein the determining whether the RTP packet of the current image frame is completely received according to the preset packet loss list comprises:
determining whether the currently received RTP data packet is the last RTP data packet in the current image frame according to the analysis result of the currently received RTP data packet;
if yes, determining whether a packet sequence number of an RTP data packet in the current image frame exists in the preset packet loss list;
if yes, determining that the RTP data packet of the current image frame is not received completely;
if not, determining that the RTP data packet of the current image frame is received completely.
8. The method of claim 6, wherein after combining the image data of the RTP packets of the current image frame and sending the combined image data to a decoder for decoding, the method further comprises:
clearing all storage positions occupied by image data of an RTP data packet of a current image frame in a preset buffer zone;
and moving the image data of the RTP data packet of the next image frame to the forefront of the preset buffer area and re-determining the packet sequence number of the first RTP data packet in the preset buffer area.
9. The method of claim 6, wherein the determining whether to combine the image data of the RTP packets of the current image frame after the preset time and send the combined image data to a decoder for decoding comprises:
determining whether an RTP data packet lost by a current image frame can be received after a preset time;
if so, combining the image data of the RTP data packet of the current image frame and then sending the combined image data to a decoder for decoding;
if not, performing packet loss processing on the RTP data packet of the current image frame, wherein the packet loss processing comprises the following modes:
the method comprises the steps that an RTP data packet lost by a current image frame is stored in a corresponding storage position in a preset buffer area, and image data of the RTP data packet of the current image frame are combined and then sent to a decoder for decoding; and/or the presence of a gas in the gas,
discarding the current image frame, not receiving the RTP data packet of the subsequent image frame in the current GOP group of pictures to which the current image frame belongs, and informing the sending end to resend the new GOP group of pictures.
10. An RTP packet processing apparatus, comprising:
a receiving unit, configured to receive RTP packets one by one;
the analysis unit is used for analyzing each RTP data packet to obtain a packet sequence number, a frame sequence number and image data of each RTP data packet;
a determining unit, configured to determine, according to a packet sequence number and a frame sequence number of each RTP data packet, a corresponding storage location of image data of each RTP data packet in a preset buffer area, and store the image data to the corresponding storage location, where the preset buffer area is capable of storing image data of at least two frames of images;
and the sending unit is used for combining the image data stored in the preset buffer area in the dimensionality of the image frame and then sending the image data to a decoder for decoding.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110807200.8A CN113645192A (en) | 2021-07-16 | 2021-07-16 | RTP data packet processing method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110807200.8A CN113645192A (en) | 2021-07-16 | 2021-07-16 | RTP data packet processing method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113645192A true CN113645192A (en) | 2021-11-12 |
Family
ID=78417631
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110807200.8A Pending CN113645192A (en) | 2021-07-16 | 2021-07-16 | RTP data packet processing method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113645192A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117097822A (en) * | 2023-10-19 | 2023-11-21 | 北京浩瀚深度信息技术股份有限公司 | Method, system and storage medium for stream type recombination network data package |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1981492A (en) * | 2004-05-12 | 2007-06-13 | 诺基亚公司 | Buffer level signaling for rate adaptation in multimedia streaming |
| CN103096183A (en) * | 2013-02-05 | 2013-05-08 | 清华大学 | Efficient streaming media transmission method |
| CN103281528A (en) * | 2013-04-10 | 2013-09-04 | 深圳康佳通信科技有限公司 | Streaming media decoding method, system and mobile terminal |
| US20140355616A1 (en) * | 2013-05-31 | 2014-12-04 | Qualcomm Incorporated | Single network abstraction layer unit packets with decoding order number for video coding |
| CN105052149A (en) * | 2013-03-29 | 2015-11-11 | 高通股份有限公司 | Improved rtp payload format designs |
| CN108696771A (en) * | 2017-04-11 | 2018-10-23 | 上海谦问万答吧云计算科技有限公司 | A kind of video broadcasting method and device |
| CN109150876A (en) * | 2018-08-20 | 2019-01-04 | 深圳市昊源科技有限公司 | A kind of the QOS method, apparatus and system of video wireless transmission |
| CN109714649A (en) * | 2017-10-26 | 2019-05-03 | 北京航天长峰科技工业集团有限公司 | The method of RTP-OVER-UDP packet loss and the treatment mechanism that reorders |
-
2021
- 2021-07-16 CN CN202110807200.8A patent/CN113645192A/en active Pending
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1981492A (en) * | 2004-05-12 | 2007-06-13 | 诺基亚公司 | Buffer level signaling for rate adaptation in multimedia streaming |
| CN103096183A (en) * | 2013-02-05 | 2013-05-08 | 清华大学 | Efficient streaming media transmission method |
| CN105052149A (en) * | 2013-03-29 | 2015-11-11 | 高通股份有限公司 | Improved rtp payload format designs |
| CN103281528A (en) * | 2013-04-10 | 2013-09-04 | 深圳康佳通信科技有限公司 | Streaming media decoding method, system and mobile terminal |
| US20140355616A1 (en) * | 2013-05-31 | 2014-12-04 | Qualcomm Incorporated | Single network abstraction layer unit packets with decoding order number for video coding |
| CN108696771A (en) * | 2017-04-11 | 2018-10-23 | 上海谦问万答吧云计算科技有限公司 | A kind of video broadcasting method and device |
| CN109714649A (en) * | 2017-10-26 | 2019-05-03 | 北京航天长峰科技工业集团有限公司 | The method of RTP-OVER-UDP packet loss and the treatment mechanism that reorders |
| CN109150876A (en) * | 2018-08-20 | 2019-01-04 | 深圳市昊源科技有限公司 | A kind of the QOS method, apparatus and system of video wireless transmission |
Non-Patent Citations (2)
| Title |
|---|
| 王薇等: "一种多媒体数据可靠传输策略", 《电讯技术》 * |
| 王薇等: "一种多媒体数据可靠传输策略", 《电讯技术》, no. 05, 28 October 2004 (2004-10-28) * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117097822A (en) * | 2023-10-19 | 2023-11-21 | 北京浩瀚深度信息技术股份有限公司 | Method, system and storage medium for stream type recombination network data package |
| CN117097822B (en) * | 2023-10-19 | 2024-01-19 | 北京浩瀚深度信息技术股份有限公司 | Method, system and storage medium for stream type recombination network data package |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| AU2018236689B2 (en) | Video data stream concept | |
| US11323136B2 (en) | Method and apparatus for processing a received sequence of data packets by removing unsuitable error correction packets from the sequence | |
| RU2310290C2 (en) | Method for buffering images for predicting and displaying standards | |
| JP2019165455A (en) | Network device and error handling method | |
| EP3142381B1 (en) | Network video playing method and device | |
| WO2016053526A1 (en) | Systems and methods for signaling request acceleration in the transport layer | |
| CN113365066B (en) | Video data transmission method and device | |
| US8018931B2 (en) | Communication apparatus and integrated circuit for communication | |
| WO2023226915A1 (en) | Video transmission method and system, device, and storage medium | |
| CN110876066A (en) | Adaptive forward error correction method, apparatus, medium and device | |
| CN111010603A (en) | Video caching and forwarding processing method and device | |
| CN113645192A (en) | RTP data packet processing method and device | |
| US20190089966A1 (en) | Efficient frame loss recovery and reconstruction in dyadic hierarchy based coding | |
| CN108093258A (en) | Coding/decoding method, computer installation and the computer readable storage medium of bit stream data | |
| CN110574378B (en) | Method and apparatus for media content asset change | |
| CN114615549B (en) | Streaming media seek method, client, storage medium and mobile device | |
| CN115942000B (en) | H.264 format video stream transcoding method, device, equipment and medium | |
| CN107454432B (en) | Data sending method and device | |
| US20180343207A1 (en) | Streaming Media Data Transmission Method and Apparatus | |
| CN115866300A (en) | TS (transport stream) time synchronization information insertion method, device, equipment and readable storage medium | |
| WO2005101846A1 (en) | Method and apparatus for buffer management | |
| CN117478958A (en) | Video playing method, device, electronic equipment and storage medium | |
| CN111555842A (en) | Method and device for transmitting data frame | |
| KR100713623B1 (en) | The system and method for processing network abstraction layer in the coupled frame of general processor and video processor | |
| CN114095760A (en) | Data transmission method and data transmission device thereof |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |