CN113141520B

CN113141520B - Video stream transmission method and device, electronic equipment and storage medium

Info

Publication number: CN113141520B
Application number: CN202010053329.XA
Authority: CN
Inventors: 郑建峰
Original assignee: Beijing Dajia Internet Information Technology Co Ltd
Current assignee: Beijing Dajia Internet Information Technology Co Ltd
Priority date: 2020-01-17
Filing date: 2020-01-17
Publication date: 2022-04-01
Anticipated expiration: 2040-01-17
Also published as: CN113141520A

Abstract

The disclosure relates to a video streaming method, a video streaming device, an electronic device and a storage medium, and relates to the field of video streaming, wherein the video streaming method and the device are used for reducing bandwidth resources occupied by stuffing bytes. The video streaming transmission method comprises the following steps: acquiring an N frame Packed Elementary Stream (PES) data packet and an N +1 frame PES data packet, wherein the N frame PES data packet and the N +1 frame PES data packet are adjacent PES data packets of a video stream to be transmitted, and N is a positive integer; the nth frame PES packet includes an end payload; and according to the length of the packet header of the (N + 1) th frame PES data packet, the length of the ending payload and the length of all the payloads of the (N + 1) th frame PES data packet, packaging the packet header, the ending payload and the starting payload or all the payloads in sequence to obtain a transport stream TS data packet, and sending the TS data packet.

Description

Video stream transmission method and device, electronic equipment and storage medium

Technical Field

The present disclosure relates to the field of video streaming, and in particular, to a video streaming method and apparatus, an electronic device, and a storage medium.

Background

In internet communication, video streams are often transmitted, and a compression standard (MPEG 2) based on digital storage media moving pictures and voice is a video stream format commonly used in internet communication.

Fig. 1 is a diagram illustrating a structure of a Transport Stream (TS) packet of an MPEG2 according to an exemplary embodiment. As shown in fig. 1, in the MPEG2 TS packet process, an Elementary Stream (ES) packet is added to a packet header of a Packetized Elementary Stream (PES) to obtain a PES packet, where the ES packet serves as a payload of the PES packet. And dividing a PES packet into a plurality of sub-packets according to 184 bytes (the length of a payload in the TS packet specified by the protocol), and adding a 4-byte TS packet header to each sub-packet to obtain a plurality of TS packets with fixed lengths of 188 bytes. When the length of a PES packet is not an integer multiple of 184 bytes, i.e. the length a of the last sub-packet is less than 184 bytes, padding bytes (stuffing bytes) with the length of 184-a bytes are inserted when the last sub-packet is encapsulated, so as to obtain a TS packet with a fixed length of 188 bytes.

Assuming that the length values of the PES packets are randomly distributed, the average number of stuffing bytes inserted when transmitting video frames is 92 bytes/frame. For a video stream with a frame rate of 30 frames/second (30fps), the average inserted stuff bytes for 1 second is 92 bytes/frame 30 frames/second 2760 bytes/second 22080 bits/second (i.e. 22 kbps). That is, in order to transmit a 30fps video stream, no matter how large the bitrate of the video itself is, the bandwidth resource of 22kbps is additionally occupied on average.

Disclosure of Invention

The present disclosure provides a video streaming method, apparatus, electronic device and storage medium, which are used to reduce bandwidth resources occupied by stuff bytes. The technical scheme of the disclosure is as follows:

according to a first aspect of embodiments of the present disclosure, there is provided a video streaming method, including:

acquiring an N frame Packed Elementary Stream (PES) data packet and an N +1 frame PES data packet, wherein the N frame PES data packet and the N +1 frame PES data packet are adjacent PES data packets of a video stream to be transmitted, and N is a positive integer; the N frame PES packet includes an end payload, the end payload referring to an end portion of the N frame PES packet not divided by a preset length;

according to the length of the packet header of the (N + 1) th frame PES data packet, the length of the ending payload and the length of all the payloads of the (N + 1) th frame PES data packet, encapsulating the packet header, the ending payload and the start payload or all the payloads in sequence to obtain a transport stream TS data packet, wherein the start payload refers to the payload of the starting part of the (N + 1) th frame PES data packet, and the length of the start payload is equal to the preset length minus the length of the ending payload and the length of the packet header of the (N + 1) th frame PES data packet;

and transmitting the TS data packet.

Optionally, the encapsulating, according to the length of the packet header of the N +1 th frame PES packet, the length of the end payload, and the length of all payloads of the N +1 th frame PES packet, according to the sequence of the packet header, the end payload, and the start payload or all the payloads, to obtain a transport stream TS packet, includes:

and if the sum of the length of the packet header, the length of the ending payload and the length of all the payloads is greater than the preset length, encapsulating the packet header, the ending payload and the starting payload in sequence to obtain the TS data packet.

and if the sum of the length of the packet header, the length of the ending effective load and the length of all the effective loads is equal to the preset length, encapsulating the packet header, the ending effective load and all the effective loads according to the sequence to obtain the TS data packet.

and if the sum of the length of the packet header, the length of the ending payload and the length of all the payloads is less than the preset length, encapsulating according to the sequence of a padding byte, the packet header, the ending payload and all the payloads to obtain the TS data packet, wherein the length of the padding byte is equal to the preset length minus the length of the packet header, the length of the ending payload and the length of all the payloads.

Optionally, the TS packet further includes first indication information and second indication information; the first indication information is used for indicating that the TS data packet merges two adjacent frame PES data packets, and the second indication information is used for indicating the starting position of the (N + 1) th frame PES data packet in the TS data packet.

According to a second aspect of embodiments of the present disclosure, there is provided a video streaming apparatus including:

the processing module is used for acquiring an Nth frame of Packaged Elementary Stream (PES) data packet and an N +1 th frame of PES data packet, wherein the Nth frame of PES data packet and the N +1 th frame of PES data packet are adjacent PES data packets of a video stream to be transmitted, and N is a positive integer; the N frame PES packet includes an end payload, the end payload referring to an end portion of the N frame PES packet not divided by a preset length;

the processing module is further configured to encapsulate, according to the length of the packet header of the N +1 th frame PES packet, the length of the end payload, and the length of all payloads of the N +1 th frame PES packet, the packet header, the end payload, and a start payload or the length of all payloads, to obtain a transport stream TS packet, where the start payload refers to a payload of a start portion of the N +1 th frame PES packet, and the length of the start payload is equal to the preset length minus the length of the end payload and the length of the packet header of the N +1 th frame PES packet;

and the transceiving module is used for sending the TS data packet.

Optionally, the processing module is specifically configured to:

According to a third aspect of the embodiments of the present disclosure, there is provided an electronic apparatus including:

a processor;

a memory for storing the processor-executable instructions;

wherein the processor is configured to execute the instructions to implement the video streaming method according to the first aspect.

According to a fourth aspect of embodiments of the present disclosure, there is provided a storage medium having instructions that, when executed by a processor of an electronic device, enable the electronic device to perform the video streaming method according to the first aspect.

According to a fifth aspect of embodiments of the present disclosure, there is provided a computer program product which, when run on a computing device, causes the computing device to perform the video streaming method according to the first aspect, or causes the computing device to implement the functionality of the video streaming apparatus according to the second aspect.

The technical scheme provided by the embodiment of the disclosure at least brings the following beneficial effects:

according to the video stream transmission method, the video stream transmission device, the electronic equipment and the storage medium, the data packets of two adjacent frames are combined, namely the ending payload of the PES data packet of the N frame and the starting payload or all payloads of the PES data packets of the (N + 1) th frame are combined in the same TS data packet, and the part originally used for transmitting the stuffing bytes is used for transmitting the payloads, so that the bandwidth resources occupied by the stuffing bytes are reduced.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the disclosure and are not to be construed as limiting the disclosure.

Fig. 1 is a schematic diagram illustrating an MPEG2 TS packet structure in accordance with an exemplary embodiment.

Fig. 2 is an architecture diagram illustrating a communication system in accordance with an exemplary embodiment.

FIG. 3 is a schematic diagram illustrating a PES packet combined in a front-to-back comparison in accordance with an exemplary embodiment.

Fig. 4 is a flow diagram illustrating a method of video streaming according to an example embodiment.

Fig. 5 is a diagram illustrating a structure of a TS packet according to an exemplary embodiment.

Fig. 6 is a flow diagram illustrating another video streaming method according to an example embodiment.

Fig. 7 is a diagram illustrating a structure of a TS packet according to an exemplary embodiment.

Fig. 8 is a schematic diagram illustrating another structure of a TS packet according to an example embodiment.

Fig. 9 is a block diagram illustrating a video streaming apparatus according to an example embodiment.

Fig. 10 is a schematic structural diagram of an electronic device according to an exemplary embodiment.

Fig. 11 is a schematic structural diagram of an electronic device according to an exemplary embodiment.

Detailed Description

In order to make the technical solutions of the present disclosure better understood by those of ordinary skill in the art, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings.

It should be noted that the terms "first," "second," and the like in the description and claims of the present disclosure and in the above-described drawings are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the disclosure described herein are capable of operation in sequences other than those illustrated or otherwise described herein. The implementations described in the exemplary embodiments below are not intended to represent all implementations consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present disclosure, as detailed in the appended claims.

Fig. 2 is an architecture diagram illustrating a communication system in accordance with an exemplary embodiment. As shown in fig. 2, the communication system includes: electronic equipment 201, application server 202. Optionally, the communication system may further include a base station 203 and a core network device 204.

The application server 202 and the core network device 204 are both connected to the internet.

The electronic device 201 may include a cell phone, tablet, computer, personal computer, smart watch, and the like. The electronic device 201 (e.g., a computer) may be directly connected to the internet, or the electronic device 201 (e.g., a cell phone) may be connected to the internet through the base station 203 and the core network device 204.

APP, Internet web page, etc. may be run and displayed on the electronic device 201. The application server 202 may provide services, content to the user through APP, Internet web pages, etc. on the electronic device. Specifically, the electronic device 201 (for example, a mobile phone) may be installed with a video playback APP, and the application server 202 may send a video stream to the video playback APP of the electronic device 201 through the internet, the core network device 204, and the base station 203, so as to be displayed on the video playback APP. Alternatively, the electronic device 201 may also send the video stream to the application server 202 through the base station 203, the core network device 204, and the internet, i.e., upload the video stream to the application server 202.

As mentioned above, MPEG2 is a video stream format commonly used in internet communications. The MPEG2 TS packet can be obtained by encapsulating the ES packet and PES packet layer. The related concepts involved are described below:

in the disclosed embodiment, the MPEG2 standard, namely the ISO/IEC13818 standard, is a second generation audio and video lossy compression standard established by the MPEG organization, and its formal name is "compression standard based on digital storage media moving images and voice". The MPEG2 TS standard is an audio-visual transport stream standard defined by the first part of the MPEG2 standard (ISO/IEC 13818-1 or ITU-T rec.h.222.0). The TS packets referred to in this disclosure refer to MPEG2 TS packets.

ES refers to a video compression stream or audio stream that is not encapsulated by MPEG2 TS, such as a video compression stream defined by the second part of the MPEG2 standard (ISO/IEC 13818-2 or ITU-T rec.h.262), or a h.264 video compression stream defined by the ITU-T rec.h.264 standard.

PES is a data encapsulation structure defined in the MPEG2-TS standard.

As mentioned above, the TS packets are fixed in size, and before merging, after the PES packets are split and encapsulated into TS packets, stuffing bytes may be inserted into the last TS packet, which may occupy additional bandwidth. According to the video stream transmission method provided by the disclosure, in two adjacent frames of PES (packetized elementary stream) data packets, the tail part of the payload of the previous PES data packet and the whole payload or the starting payload of the next PES data packet are combined and transmitted in one TS data packet, so that padding bytes are not required to be inserted, and the bandwidth is saved.

FIG. 3 is a schematic diagram illustrating a PES packet combined in a front-to-back comparison in accordance with an exemplary embodiment. As shown in a of fig. 3, before the PES packets are merged, stuffing bytes are added to the TS packet in which the end portion of the nth frame PES packet is located, and stuffing bytes are also added to the TS packet in which the end portion of the N +1 th frame PES packet is located. As shown in B of fig. 3, after the PES packets of two adjacent frames are merged, the end payload of the N-th frame PES packet and the start payload or all of the payloads of the N + 1-th frame PES packet are merged into the same TS packet, and the portion originally used for transmitting the stuff bytes is used for transmitting the payloads, thereby saving bandwidth resources.

Specifically, fig. 4 is a flowchart illustrating a video streaming method according to an exemplary embodiment. As shown in fig. 4, the video streaming method includes S401 to S403:

s401, the sending end obtains the N frame PES data packet and the (N + 1) frame PES data packet.

The N frame PES data packet and the N +1 frame PES data packet are adjacent PES data packets of a video stream to be transmitted, and N is a positive integer.

That is, the nth frame PES packet is encapsulated by the nth frame ES packet, and the N +1 th frame PES packet is encapsulated by the N +1 th frame ES packet. The present disclosure takes the ES packet, PES packet, and TS packet as an example, but is not intended to be limited thereto, and is also applicable to other packets that employ a similar two-layer encapsulation structure.

The nth frame PES packet includes an end payload, which refers to an end portion of the nth frame PES packet that is not divisible by a preset length. The predetermined length is 184 bytes.

S402, the sending end encapsulates the packet head, the ending payload, the starting payload or all the payloads according to the length of the packet head of the (N + 1) th frame PES data packet, the length of the ending payload and the length of all the payloads of the (N + 1) th frame PES data packet, so as to obtain a TS data packet.

Wherein, the start payload refers to the payload of the start part of the (N + 1) th frame PES packet, and the length of the start payload is equal to the preset length minus the length of the end payload and the length of the packet header of the (N + 1) th frame PES packet.

The TS data packet also comprises first indication information and second indication information.

The first indication information is used to indicate that the TS packet is merged with the two-frame PES packet, that is, in order to distinguish from the prior art that one TS packet only encapsulates one frame PES packet, the TS packet merged with the two-frame PES packet is indicated in a special manner, so that the receiving end can parse the TS packet in a corresponding manner.

The second indication information is used to indicate the start position of the N +1 th frame PES packet in the TS packet, and since two frame PES packets are combined in one TS packet, it is indicated to the receiving end which byte data in one TS packet belongs to the N-th frame PES packet and which byte data belongs to the N +1 th frame PES packet.

Fig. 5 is a diagram illustrating a structure of a TS packet according to an exemplary embodiment. Illustratively, the generated TS packets include:

a first part: a packet header (TS packet header) of the TS packet, where the packet header of the TS packet occupies 4 bytes, and a payload start indication (payload _ unit _ start _ indicator) field in the packet header of the TS packet is first indication information, and a value may be set to 1, which is used to indicate that the TS packet merges two packets, that is, indicate that the TS packet includes header data of a new PES packet.

A second part: an adaptation field (adaptation _ field) field, i.e., a field in which padding bytes are located, and a first byte thereof is used to indicate the length of the adaptation field. For bytes where padding bytes are not needed, the first byte of the adaptation field takes the value 0.

And a third part: and the packet head of the (N + 1) th frame PES packet. Taking the packet header (PES packet header) of the (N + 1) th frame PES packet as an example, the PES packet length (PES _ packet _ length) field in the PES packet header may be set to 0, indicating that the length of the PES packet is not limited, i.e., the end position of the PES packet is not determined according to the value of the PES packet length field. A PES packet header data length (PES _ header _ data _ length) field in the packet header is used to indicate the length of the packet header of the PES packet, i.e., to determine the end position of the packet header of the N +1 th frame PES packet.

The fourth part: end payload of nth frame PES packet.

The fifth part is that: the start payload or the entire payload of the N +1 th frame PES packet.

It should be noted that a start code word (StartCode) is included between the second part and the third part to indicate the start position of the N +1 th frame PES packet, i.e., the start position of the packet header of the N +1 th frame PES packet. The start codeword (StartCode) occupies three bytes, and takes the value of "0 x 000001" (16 systems).

A sync word (SyncWord) is included between the fourth part and the fifth part, and in this disclosure, the sync word (SyncWord) is second indication information for indicating a start position of the N +1 th frame PES packet. A sync word (SyncWord) occupies three bytes, taking the value "0 x 000001" (16 systems).

The payload start indication (payload _ unit _ start _ indicator) field, adaptation field (adaptation field), PES packet length (PES _ packet _ length) field, and PES packet header data length (PES _ header _ data _ length) field referred to in this disclosure are all fields defined within the standard ITU-T rec.h.222.0.

S403, the sending end sends the TS data packet to the receiving end.

The sending end may be the application server 202, and the receiving end may be the electronic device 201, that is, the video streaming method may be performed by the application server 202, and is used to send the data packet to the electronic device 201. Alternatively, the sending end may be the electronic device 201, and the receiving end may be the application server 202, that is, the video streaming method may be executed by the electronic device 201, and is used to send the data packet to the application server 202.

Step S402 will be described below with reference to fig. 6. Fig. 6 is a flow diagram illustrating another video streaming method according to an example embodiment. The video streaming method inputs the ES data stream and outputs the MPEG2 TS data stream conforming to ISO/IEC 13818-1 standard. As shown in fig. 6, the video streaming method includes:

s601, initializing an array and a variable.

A variable payload _ len is defined for indicating a preset length of a payload of a TS packet.

A variable PES _ header _ len is defined to indicate the length of the header of the nth frame PES packet or the N +1 th frame PES packet, and for the nth frame PES packet and the N +1 th frame PES packet being PES packets, the PES _ header _ len indicates the length of the header of the PES packet.

An array of original _ data [ ] with a length of at least the maximum number of bytes of the ES packet may be allocated for buffering the ES packet, i.e. before encapsulating an ES packet to obtain a PES packet, the ES packet is stored in the array of original _ data [ ]. Accordingly, a variable origi _ data _ len is defined for indicating the effective data length in the array origi _ data [ ]. The initialization value of original _ data _ len is 0.

Since the nth frame PES packet or the (N + 1) th frame PES packet is encapsulated by the ES packet, when the nth frame ES packet is encapsulated, original _ data [ ] stores the nth frame ES packet, and original _ data _ len indicates the length of the payload of the nth frame PES packet. When encapsulating the (N + 1) th frame ES packet, the (N + 1) th frame ES packet is stored with the original _ data [ ], and the length of the payload of the (N + 1) th frame ES packet is indicated by the original _ data _ len.

An array pending _ data [ ] of at least 184 bytes in length may be allocated for buffering the end payload of the nth frame PES packet.

Accordingly, a variable pending _ data _ len is defined for indicating the valid data length in the array pending _ data [ ], i.e., indicating the length of the end payload of the N-th frame PES packet, and the initialization value of pending _ data _ len is 0.

If the length of the PES packet of the nth frame is an integer multiple of the preset length (184 bytes), the data in the pending _ data [ ] is empty, and the value of the pending _ data _ len is 0.

A variable data _ left _ len is defined to indicate the remaining unpackaged valid data length in the array original _ data [ ], with the initialization value of data _ left _ len being equal to original _ data _ len.

S602, splitting the Nth frame PES data packet according to the preset length to obtain M sub-data packets, if the length of the Nth frame PES data packet can be divided by the preset length, respectively encapsulating the M sub-data packets (adding the packet headers of the TS data packets) to obtain M TS data packets, and executing the steps S603-S604; if the data cannot be divided evenly, the first M-1 sub-packets are encapsulated (the header of the TS packet is added) to obtain M-1 TS packets, and steps S605-S607 are performed.

The length of the N-th frame PES packet can be divided by the preset length, that is, the N-th frame PES packet can be divided into sub-packets with the preset length, and when the N-th frame PES packet is packaged into a TS packet, the N-th frame PES packet does not need to be combined with the (N + 1) -th frame PES packet, and padding bytes do not need to be added.

The ES data packet of the Nth frame is firstly stored into an array original _ data [ ].

Fig. 7 is a diagram illustrating a structure of a TS packet according to an exemplary embodiment. For example, as shown in fig. 7, for a first sub-packet, including a PES packet header of an nth frame PES packet and the foremost payload _ len-PES _ header _ len bytes of data in an array original _ data [ ], the first sub-packet is encapsulated (a packet header of the TS packet is added) to obtain a first TS packet, and the payload _ len-PES _ header _ len is subtracted (payload _ len-PES _ header _ len).

For the case of being divisible as shown in fig. 7, for each sub-packet from the second sub-packet to the mth sub-packet, the remaining payload _ len bytes in the array original _ data [ ] are sequentially corresponded, for example, the second sub-packet corresponds to the first payload _ len bytes after the first payload _ len-pes _ header _ len bytes are removed, the third sub-packet corresponds to the remaining second payload _ len bytes, and so on. And respectively encapsulating the second sub data packet to the Mth sub data packet (adding a packet head of the TS data packet) to obtain a second TS data packet to the Mth TS data packet, and subtracting the payload _ len from the data _ left _ len in each encapsulation.

Fig. 8 is a schematic diagram illustrating another structure of a TS packet according to an example embodiment. In the case of the case shown in fig. 8, the processing method of adding the header of the TS packet to the first M-1 sub-packets to obtain M-1 TS packets, and the specific example of the processing method of adding the header of the TS packet to the M sub-packets to obtain M TS packets are not repeated here.

It should be noted that, each TS packet generated may be buffered for waiting time and sent to the receiving end or directly sent to the receiving end.

After the step is executed, for the case of being able to be divided completely, the data in the array pending _ data [ ] is empty, and the value of pending _ data _ len is 0. For the case of non-integer division, the mth sub-packet (i.e. the last remaining byte data in the array original _ data [ ], i.e. the end payload of the nth frame PES packet) is stored into the array pending _ data [ ], the value of pending _ data _ len is the length of the mth sub-packet (i.e. the length of the end payload of the nth frame PES packet), and pending _ data _ len is greater than 0. Regardless of whether the integer division is available, the (N + 1) th frame ES packet is stored in an array original _ data [ ], where the original _ data _ len takes the length of the (N + 1) th frame ES packet, i.e., the length of the payload of the (N + 1) th frame PES packet. The processing of the (N + 1) th frame ES packet is started next.

S603, if the sum of the length PES _ header _ len of the header of the N +1 th frame PES packet and the length original _ data _ len of the payload of the N +1 th frame PES packet is greater than the preset length payload _ len, the step S602 is performed with the N +1 th frame PES packet as the nth frame PES packet.

As shown in a in fig. 7, when the N-th frame PES packet does not need to be merged with the N + 1-th frame PES packet, if the N + 1-th frame PES packet is long in length and needs to be split, the N + 1-th frame is used as a new N-th frame PES packet, and the same splitting, determining and encapsulating processes are performed.

Illustratively, for a first sub-packet after splitting the N +1 th frame PES packet, the first sub-packet includes a PES packet header of the N +1 th frame PES packet and the foremost payload _ len-PES _ header _ len byte data in the array original _ data [ ], the first sub-packet is encapsulated (the packet header of the TS packet is added) to obtain a first TS packet, and the payload _ len-PES _ header _ len is subtracted (payload _ len-PES _ header _ len). The processing method of the other sub-packets after the N +1 th frame PES packet is split is the same as the processing method of the nth frame PES packet in step S602, and is not repeated here.

S604, if the sum of the length PES _ header _ len of the packet header of the (N + 1) th frame PES data packet and the length original _ data _ len of the payload of the (N + 1) th frame PES data packet is less than or equal to the preset length payload _ len, encapsulating the (N + 1) th frame PES data packet (adding the packet header of the TS data packet) to obtain the TS data packet.

For example, as shown in B and C in fig. 7, when the nth frame PES packet does not need to be merged with the (N + 1) th frame PES packet, if the length of the (N + 1) th frame PES packet is short and does not need to be split, the (N + 1) th frame PES packet may be encapsulated to obtain a TS packet.

It should be noted that, as shown in B in fig. 7, for the length PES _ header _ len of the header of the N +1 th frame PES packet and the length original _ data _ len of the payload of the N +1 th frame PES packet, the sum of which is equal to the preset length payload _ len, the generated TS packet does not include padding bytes.

As shown in C in fig. 7, for the sum of the length PES _ header _ len of the header of the N +1 th frame PES packet and the length original _ data _ len of the payload of the N +1 th frame PES packet being smaller than the preset length payload _ len, the generated TS packet includes a stuff byte having a length of (payload _ len-PES _ header _ len-original _ data _ len).

At this time, values of variables pending _ data _ len and data _ left _ len are both 0.

S605, if the sum of the length PES _ header _ len of the packet header of the (N + 1) th frame PES data packet and the length pending _ data _ len of the ending payload of the N frame PES data packet is greater than or equal to the preset length payload _ len, encapsulating the ending payload of the N frame PES data packet (adding a TS packet header) to obtain a TS data packet. Otherwise, step S606 or S607 is executed.

As shown in a of fig. 8, since the merging of the nth frame PES packet and the N +1 th frame PES packet must be carried out with the header of the N +1 th frame PES packet, if the payload of the TS packet is removed from the end payload of the nth frame PES packet, and the remaining portion cannot accommodate or can only accommodate the header of the N +1 th frame PES packet, the merging cannot be carried out, and only padding bytes can be added.

S606, if the sum of the length PES _ header _ len of the packet header of the (N + 1) th frame PES packet, the length pending _ data _ len of the end payload of the N N th frame PES packet, and the length original _ data _ len of the payload of the (N + 1) th frame PES packet is greater than the preset length payload _ len, then packaging (adding the packet header of the TS packet) is performed according to the sequence of the packet header of the (N + 1) th frame PES packet, the end payload of the N th frame PES packet, and the start payload of the (N + 1) th frame PES packet, and the TS packet is obtained.

As shown in B in fig. 8, when the N +1 th frame PES packet is long, the N +1 th frame is split into a plurality of sub-packets, and the TS packet is obtained by encapsulating the packet header of the N +1 th frame PES packet, the end payload of the N th frame PES packet, and the first sub-packet (the start payload of the N +1 th frame PES packet).

Wherein, the length of the packet header of the (N + 1) th frame PES data packet is PES _ header _ len. The end payload of the nth frame PES packet is byte data of length pending _ data _ len in the array pending _ data [ ]. The start payload of the (N + 1) th frame PES packet is byte data with the foremost length (payload _ len-PES _ header _ len-pending _ data _ len) in the array original _ data [ ].

After the step is executed, the data in the array pending _ data [ ] is empty, and the value of pending _ data _ len is 0. data _ left _ len minus (payload _ len-pes _ header _ len-pending _ data _ len).

It should be noted that, for the processing procedure of the remaining subpackets of the N +1 th frame, reference is made to the processing procedure of the second and subsequent subpackets of the PES packet of the nth frame in step S602, which is not repeated here.

S607, if the sum of the length PES _ header _ len of the packet header of the (N + 1) th frame PES packet, the length pending _ data _ len of the end payload of the N N th frame PES packet, and the length original _ data _ len of the payload of the (N + 1) th frame PES packet is equal to the preset length payload _ len, then encapsulating (adding the packet header of the TS packet) the packet header of the (N + 1) th frame PES packet, the end payload of the N N th frame PES packet, and the whole payload of the (N + 1) th frame PES packet to obtain the TS packet.

That is, when the N +1 th frame PES packet is short, the TS packet is obtained by directly encapsulating the packet header of the N +1 th frame PES packet, the end payload of the N th frame PES packet, and all the payloads of the N +1 th frame PES packet without splitting the N +1 th frame PES packet.

Wherein, the length of the packet header of the (N + 1) th frame PES data packet is PES _ header _ len. The end payload of the nth frame PES packet is byte data of length pending _ data _ len in the array pending _ data [ ]. The entire payload of the (N + 1) th frame PES packet is all byte data in the array original _ data [ ].

Note that, as shown in C in fig. 8, the TS packet generated at this time does not include the stuff bytes.

S608, if the sum of the length PES _ header _ len of the packet header of the (N + 1) th frame PES packet, the length pending _ data _ len of the end payload of the N +1 th frame PES packet, and the length original _ data _ len of the payload of the (N + 1) th frame PES packet is smaller than the preset length payload _ len, then packaging (adding the packet header of the TS packet) is performed according to the sequence of the padding bytes, the packet header of the (N + 1) th frame PES packet, the end payload of the N +1 th frame PES packet, and all the payloads of the (N + 1) th frame PES packet, and the TS packet is obtained.

As shown in D in fig. 8, for the sum of the length PES _ header _ len of the header of the N +1 th frame PES packet, the length pending _ data _ len of the end payload of the nth frame PES packet, and the length original _ data _ len of the payload of the N +1 th frame PES packet, which is smaller than the preset length payload _ len, the generated TS packet includes a stuff byte having a length equal to the preset length payload _ len minus the length PES _ header _ len of the header of the N +1 th frame PES packet, the length pending _ data _ len of the end payload of the nth frame PES packet, and the length original _ data _ len of the entire payload of the N +1 th frame PES packet, that is, payload _ len _ header _ len _ data _ length.

After step S607 or S608 is completed, the data in the array pending _ data [ ] is empty, and the value of pending _ data _ len is 0. data _ left _ len takes a value of 0.

Steps S606, S607, and S608 correspond to a specific embodiment of step S402, and the generated TS packet includes the first indication information and the second indication information. The TS packet generated for the other steps in fig. 6 does not include the first indication information and the second indication information.

According to the video stream transmission method provided by the disclosure, data packets of two adjacent frames are merged, namely, an ending payload of an Nth frame PES data packet and a starting payload or all payloads of an N +1 th frame PES data packet are merged in the same TS data packet, and a part originally used for transmitting stuffing bytes is used for transmitting the payloads, so that bandwidth resources occupied by the stuffing bytes are reduced.

Fig. 9 is a block diagram illustrating a video streaming apparatus according to an example embodiment. As shown in fig. 9, the video streaming apparatus 900 includes a processing module 901 and a transceiver module 902.

A processing module 901 configured to perform obtaining an nth frame Packetized Elementary Stream (PES) data packet and an N +1 th frame PES data packet, where the nth frame PES data packet and the N +1 th frame PES data packet are adjacent PES data packets of a video stream to be transmitted, and N is a positive integer; the nth frame PES packet includes an end payload, which refers to an end portion of the nth frame PES packet that is not divisible by a preset length.

The processing module 901 is further configured to perform encapsulation according to the length of the packet header of the N +1 th frame PES packet, the length of the end payload, and the length of all the payloads of the N +1 th frame PES packet, in the order of the packet header, the end payload, and the start payload or all the payloads, to obtain a transport stream TS packet, where the start payload refers to the payload of the start portion of the N +1 th frame PES packet, and the length of the start payload is equal to the preset length minus the length of the end payload and the length of the packet header of the N +1 th frame PES packet.

A transceiving module 902 configured to perform transmitting the TS packet.

Optionally, the processing module 901 is specifically configured to execute:

With regard to the apparatus in the above-described embodiment, the specific manner in which each module performs the operation has been described in detail in the embodiment related to the method, and will not be elaborated here.

Fig. 10 is a schematic diagram illustrating a configuration of a server according to an example embodiment. The server may be the application server 202 described above. As shown in fig. 10, the server 100 includes: a processor 101, a communication interface 102, volatile memory 103, and non-volatile memory 104.

The processor 101 may be a chip. For example, the integrated circuit may be a Field Programmable Gate Array (FPGA), an Application Specific Integrated Circuit (ASIC), a system on chip (SoC), a Central Processing Unit (CPU), a Network Processor (NP), a digital signal processing circuit (DSP), a Microcontroller (MCU), a Programmable Logic Device (PLD), or other integrated chips.

The communication interface 102 is used for the sms server 100 to communicate with other communication devices, for example, the communication interface 102 may include an ethernet interface, a wired or wireless network interface, and the like.

Volatile memory 103 may include Random Access Memory (RAM) including dynamic RAM and/or static RAM, on-chip and/or off-chip cache memory, and the like.

Non-volatile memory 104, which may be embedded and/or removable, may include, for example, read-only memory, flash memory, magnetic storage devices, such as hard disks, floppy disk drives, magnetic tape, etc., optical disk drives and/or media, non-volatile random access memory (NVRAM), and so forth.

Similar to the volatile memory 103, the non-volatile memory 104 may include a cache area for temporary storage of data. At least a portion of the volatile and/or nonvolatile memory may be embedded in the processor 101. The memory may store one or more software programs, instructions, blocks of information, data, etc. that when invoked by the processor 101 may be used to perform the functions of the server 100 of the present disclosure, such as performing the video streaming methods described above.

Fig. 11 is a schematic structural diagram of an electronic device according to an exemplary embodiment. As shown in fig. 11, the electronic device 11 may be a mobile phone. Taking a mobile phone as an example, the electronic device 11 may include: radio Frequency (RF) circuit 110, memory 120, input unit 130, display unit 140, sensor 150, audio circuit 160, wireless fidelity (Wi-Fi) module 170, processor 180, bluetooth module 181, and power supply 190.

The RF circuit 110 may be used for receiving and transmitting signals during information transmission and reception or during a call, and may receive downlink data of a base station and then send the downlink data to the processor 180 for processing; the uplink data may be transmitted to the base station. Typically, the RF circuitry includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier, a duplexer, and the like.

Memory 120 may be used to store instructions executable by processor 180. The processor 180 executes instructions stored in the memory 120 to perform various functions of the electronic device 11 and data processing, such as performing the video streaming method described above. The memory 120 may include high speed random access memory and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid state storage device. The memory 120 stores an operating system enabling the electronic device 11 to operate, such as developed by apple Inc

Operating System, developed by Google

Open source operating system, developed by Microsoft corporation

An operating system, etc. The memory 120 may store an operating system and various application programs, and may also store codes for performing the methods of the embodiments of the present application.

The input unit 130 (e.g., a touch screen) may be used to receive input numeric or character information, generate signal inputs related to user settings and function control of the electronic device 11. Specifically, the input unit 130 may include a touch screen 131 disposed on the front surface of the electronic device 11 and configured to collect a touch operation of the user thereon or nearby.

The display unit 140 (i.e., a display screen) may be used to display information input by or provided to the user and a Graphical User Interface (GUI) of various menus of the electronic apparatus 11. The display unit 140 may include a display screen 141 disposed on the front surface of the electronic device 11. The display screen 141 may be configured in the form of a liquid crystal display, a light emitting diode, or the like. The display unit 140 may be used to display various graphical user interfaces described herein. The touch screen 131 may cover the display screen 141, or the touch screen 131 and the display screen 141 may be integrated to implement input and output functions of the electronic device 11, and after the integration, the touch screen may be referred to as a touch display screen for short.

The electronic device 11 may further comprise at least one sensor 150, such as a light sensor, a motion sensor. The electronic device 11 may also be provided with other sensors such as a gyroscope, barometer, hygrometer, thermometer, infrared sensor, and the like.

The audio circuitry 160, speaker 161, microphone 162 may provide an audio interface between a user and the electronic device 11. The audio circuit 160 may transmit the electrical signal converted from the received audio data to the speaker 161, and convert the electrical signal into a sound signal for output by the speaker 161; on the other hand, the microphone 162 converts the collected sound signal into an electrical signal, converts the electrical signal into audio data after being received by the audio circuit 160, and outputs the audio data to the RF circuit 110 to be transmitted to, for example, another terminal or outputs the audio data to the memory 120 for further processing.

Wi-Fi belongs to short-range wireless transmission technology, and the electronic device 11 can help the user send and receive e-mails, browse webpages, access streaming media and the like through the Wi-Fi module 170, and provides wireless broadband Internet access for the user.

The processor 180 is a control center of the electronic device 11, connects various parts of the entire terminal using various interfaces and lines, and performs various functions of the electronic device 11 and processes data by running or executing software programs stored in the memory 120 and calling data stored in the memory 120. Processor 180 may refer to one or more processors herein, and processor 180 may include one or more processing units; the processor 180 may also integrate an application processor, which mainly handles operating systems, user interfaces, applications, etc., and a baseband processor, which mainly handles wireless communications. It will be appreciated that the baseband processor described above may not be integrated into the processor 180. In the present application, the processor 180 may run an operating system, an application program, a user interface display, a touch response, and the communication method described in the embodiments of the present application.

And the bluetooth module 181 is configured to perform information interaction with other bluetooth devices having a bluetooth module through a bluetooth protocol. For example, the electronic device 11 may establish a bluetooth connection with a wearable electronic device (e.g., a smart watch) that is also equipped with a bluetooth module through the bluetooth module 181, so as to perform data interaction.

The electronic device 11 also includes a power supply 190 (such as a battery) to power the various components. The power supply may be logically coupled to the processor 180 through a power management system to manage charging, discharging, and power consumption functions through the power management system.

In an exemplary embodiment, there is also provided a storage medium, such as a memory, comprising instructions executable by the processor 101 of the server 100 to perform the video streaming method described above. Alternatively, the storage medium may be a non-transitory computer readable storage medium, which may be, for example, a ROM, a Random Access Memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like.

In an exemplary embodiment, there is also provided a computer program product containing instructions which, when run on a computing device, cause the computing device to perform the video streaming method provided above, or to carry out the functionality of the video streaming apparatus provided above.

Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. This application is intended to cover any variations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.

It will be understood that the present disclosure is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A video streaming method, comprising:

and transmitting the TS data packet.

2. The video streaming method according to claim 1, wherein the encapsulating according to the length of the packet header of the N +1 th frame PES packet, the length of the end payload, and the length of the entire payload of the N +1 th frame PES packet in the order of the packet header, the end payload, and the start payload or the entire payload, obtains a transport stream TS packet, and includes:

3. The video streaming method according to claim 1, wherein the encapsulating according to the length of the packet header of the N +1 th frame PES packet, the length of the end payload, and the length of the entire payload of the N +1 th frame PES packet in the order of the packet header, the end payload, and the start payload or the entire payload, obtains a transport stream TS packet, and includes:

4. The video streaming method according to claim 1, wherein the encapsulating according to the length of the packet header of the N +1 th frame PES packet, the length of the end payload, and the length of the entire payload of the N +1 th frame PES packet in the order of the packet header, the end payload, and the start payload or the entire payload, obtains a transport stream TS packet, and includes:

5. The video streaming method according to any one of claims 1 to 4, wherein the TS packet further includes first indication information and second indication information; the first indication information is used for indicating that the TS data packet merges two adjacent frame PES data packets, and the second indication information is used for indicating the starting position of the (N + 1) th frame PES data packet in the TS data packet.

6. A video streaming apparatus, comprising:

and the transceiving module is used for sending the TS data packet.

7. The video streaming apparatus according to claim 6, wherein the processing module is specifically configured to:

8. The video streaming apparatus according to claim 6, wherein the processing module is specifically configured to:

9. The video streaming apparatus according to claim 6, wherein the processing module is specifically configured to:

10. The video streaming apparatus according to any one of claims 6 to 9, wherein the TS packet further includes first indication information and second indication information; the first indication information is used for indicating that the TS data packet merges two adjacent frame PES data packets, and the second indication information is used for indicating the starting position of the (N + 1) th frame PES data packet in the TS data packet.

11. An electronic device, comprising:

a processor;

a memory for storing the processor-executable instructions;

wherein the processor is configured to execute the instructions to implement the video streaming method of any of claims 1-5.

12. A storage medium having instructions that, when executed by a processor of an electronic device, enable the electronic device to perform the video streaming method of any of claims 1-5.