CN115802094A - Video message processing method and device, computer equipment and readable medium - Google Patents

Video message processing method and device, computer equipment and readable medium Download PDF

Info

Publication number
CN115802094A
CN115802094A CN202111062123.4A CN202111062123A CN115802094A CN 115802094 A CN115802094 A CN 115802094A CN 202111062123 A CN202111062123 A CN 202111062123A CN 115802094 A CN115802094 A CN 115802094A
Authority
CN
China
Prior art keywords
video
frame
message
discarded
packet
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202111062123.4A
Other languages
Chinese (zh)
Inventor
陈道伟
张海军
熊钢
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
ZTE Corp
Original Assignee
ZTE Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by ZTE Corp filed Critical ZTE Corp
Priority to CN202111062123.4A priority Critical patent/CN115802094A/en
Priority to PCT/CN2022/117440 priority patent/WO2023036163A1/en
Publication of CN115802094A publication Critical patent/CN115802094A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L47/00—Traffic control in data switching networks
    • H04L47/10—Flow control; Congestion control
    • H04L47/32—Flow control; Congestion control by discarding or delaying data units, e.g. packets or frames
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/238—Interfacing the downstream path of the transmission network, e.g. adapting the transmission rate of a video stream to network bandwidth; Processing of multiplex streams
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/433—Content storage operation, e.g. storage operation in response to a pause request, caching operations
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/436—Interfacing a local distribution network, e.g. communicating with another STB or one or more peripheral devices inside the home
    • H04N21/4363—Adapting the video stream to a specific local network, e.g. a Bluetooth® network
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/438—Interfacing the downstream path of the transmission network originating from a server, e.g. retrieving encoded video stream packets from an IP network
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client 
    • H04N21/63—Control signaling related to video distribution between client, server and network components; Network processes for video distribution between server and clients or between remote clients, e.g. transmitting basic layer and enhancement layers over different transmission paths, setting up a peer-to-peer communication via Internet between remote STB's; Communication protocols; Addressing
    • H04N21/647—Control signaling between network components and server or clients; Network processes for video distribution between server and clients, e.g. controlling the quality of the video stream, by dropping packets, protecting content from unauthorised alteration within the network, monitoring of network load, bridging between two different networks, e.g. between IP and wireless

Landscapes

  • Engineering & Computer Science (AREA)
  • Signal Processing (AREA)
  • Multimedia (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Computer Security & Cryptography (AREA)
  • Data Exchanges In Wide-Area Networks (AREA)

Abstract

The utility model provides a video message processing method, which comprises the steps that in the process of sending the video message cached in a buffer area to a video terminal, in response to the satisfaction of a preset video message discarding condition, the video message to be discarded is determined, wherein the video message carries a video frame type identifier; determining a video frame type corresponding to a video frame type identifier of a video message to be discarded, and determining an associated video message of the video message to be discarded in a buffer area according to a preset message association strategy and the video frame type; and discarding the video message to be discarded and the associated video message. Determining the associated message of the video message to be discarded according to the type of the video frame, and performing different discarding treatments on the video messages of different video frame types, thereby avoiding disordered packet loss, ensuring the transmission of key video frames, reducing the transmission of video frames which cannot be decoded, improving the transmission efficiency of a video network, and ensuring the video transmission quality. The present disclosure also provides a video message processing apparatus, a computer device and a readable medium.

Description

Video message processing method and device, computer equipment and readable medium
Technical Field
The present disclosure relates to the field of video processing technologies, and in particular, to a video packet processing method, an apparatus, a computer device, and a readable medium.
Background
With the technical development of videos and networks, the demand of videos on bandwidth is increasing. In general, a set top box and a video terminal such as a VR (Virtual Reality) helmet have great convenience in accessing a network through a wireless technology (e.g., a wireless local area network). However, a wireless network is easily interfered by a wireless environment, so that the access bandwidth of a video terminal fluctuates up and down. When the access bandwidth of the video terminal cannot meet the video transmission requirement, the video message is partially discarded. As shown in fig. 1, a typical video encoding sequence GOP (Group of pictures) typically contains I frames, P frames and B frames, wherein the I frames are generated by intra-frame compression, the P frames are generated by inter-frame compression of the I frames or P frames adjacent to the P frames, and the B frames are generated by inter-frame compression of the previous I or P frames and the following P frames.
Dropped video packets are typically dropped from before the queue, dropped from after the queue, or aged by arrival time. On one hand, the video message discarding mode can cause video continuous frame dropping to cause video blocking, and on the other hand, due to the fact that association exists between video message frames and frames, the transmitted video frames can not be decoded correctly and can be discarded by a video terminal due to the fact that key frames are discarded.
Disclosure of Invention
The disclosure provides a video message processing method, a video message processing device, computer equipment and a readable medium.
In a first aspect, an embodiment of the present disclosure provides a method for processing a video packet, where the method includes:
in the process of sending the video message cached in the buffer area to the video terminal, determining the video message to be discarded in response to the satisfaction of a preset video message discarding condition, wherein the video message carries a video frame type identifier;
determining a video frame type corresponding to the video frame type identifier of the video message to be discarded, and determining an associated video message of the video message to be discarded in the buffer area according to a preset message association strategy and the video frame type;
and discarding the video message to be discarded and the associated video message.
In some embodiments, the video packet dropping condition is a video packet first dropping condition, and the meeting of the video packet first dropping condition includes: and the sending of the video message cached in the buffer area fails.
In some embodiments, the video packet that failed to be sent is the video packet to be discarded.
In some embodiments, the determining, in the buffer area, the video packet to be discarded according to a preset packet association policy and the video frame type includes:
determining an associated frame corresponding to the video frame type according to a preset message associated first strategy;
and determining the associated video message corresponding to the associated frame in the buffer area.
In some embodiments, the video frame types include: the message association first strategy comprises the following steps:
in a video coding sequence image group, the related frames of the I frame are a P frame and a B frame between the current I frame and the next I frame; the related frames of the P frame are the P frame and the B frame between the current P frame and the next I frame; b-frames are unassociated frames.
In some embodiments, the video packet dropping condition is a video packet second dropping condition, and the meeting of the video packet second dropping condition includes: the effective bandwidth between the video message processing device and the video terminal is smaller than the preset bandwidth.
In some embodiments, the determining, in the buffer according to the preset packet association policy and the video frame type, the video packet to be discarded includes:
determining an associated frame corresponding to the video frame type according to a preset message associated second strategy;
and determining the associated video message corresponding to the associated frame in the buffer area.
In some embodiments, the video frame types include: the message association second strategy comprises the following steps:
in a video coding sequence image group, the related frames of the I frame are a P frame and a B frame between the current I frame and the next I frame; the associated frames of the P frames comprise a first associated frame and a second associated frame, the first associated frame is a P frame and a B frame between the current P frame and the next I frame, the second associated frame is a B frame between the current P frame and the previous P frame, and if no other P frame exists before the current P frame, the second associated frame is a B frame between the current P frame and the previous I frame; b-frames are unrelated frames.
In some embodiments, the determining a video packet to be discarded includes:
determining the number of messages to be discarded within a preset time length according to the effective bandwidth and the preset bandwidth;
determining the type of a video frame to be discarded;
determining a video message to be discarded in the buffer area according to the video frame type identifier of the video frame type to be discarded;
the discarding the to-be-discarded video packet and the associated video packet includes:
and discarding the video message to be discarded and the associated video message according to the number of the messages to be discarded within the preset time length.
In some embodiments, the determining a video frame type to be dropped comprises:
and determining the type of the video frame to be discarded according to the preset priority corresponding to the type of the video frame.
In some embodiments, after discarding the to-be-discarded video packet and the associated video packet, the method further comprises:
and sending the video message cached currently in the buffer area to a video terminal.
In another aspect, an embodiment of the present disclosure further provides a video packet processing apparatus, including a to-be-discarded video packet determining module, a relevant video packet determining module, and a packet discarding module, where the to-be-discarded video packet determining module is configured to determine, in a process of sending a video packet cached in a buffer to a video terminal, a to-be-discarded video packet in response to that a preset video packet discarding condition is met, where a video frame type identifier is carried in the video packet;
the associated video message determining module is used for determining a video frame type corresponding to the video frame type identifier of the video message to be discarded, and determining the associated video message of the video message to be discarded in the buffer area according to a preset message association strategy and the video frame type;
the message discarding module is used for discarding the video message to be discarded and the associated video message.
In another aspect, an embodiment of the present disclosure further provides a computer device, including: one or more processors; a storage device having one or more programs stored thereon; the one or more programs, when executed by the one or more processors, cause the one or more processors to implement the video message processing method as previously described.
In yet another aspect, the disclosed embodiments also provide a computer readable medium, on which a computer program is stored, where the program is executed to implement the video message processing method as described above.
In the video message processing method provided by the embodiment of the disclosure, in the process of sending the video message cached in the buffer area to the video terminal, in response to the satisfaction of the preset video message discarding condition, the video message to be discarded is determined, wherein the video message carries the video frame type identifier; determining a video frame type corresponding to a video frame type identifier of a video message to be discarded, and determining an associated video message of the video message to be discarded in a buffer area according to a preset message association strategy and the video frame type; and discarding the video message to be discarded and the associated video message. According to the embodiment of the disclosure, the associated message of the video message to be discarded can be determined according to the type of the video frame, and different discarding processes are performed on the video messages of different video frame types, so that disordered packet loss is avoided, transmission of key video frames is ensured, and transmission of video frames which cannot be decoded can be reduced, thereby improving the transmission efficiency of a video network and ensuring the video transmission quality.
Drawings
FIG. 1 is a schematic representation of a group of pictures of a video encoding sequence;
FIG. 2 is a system architecture diagram of an embodiment of the present disclosure;
fig. 3 is a first flowchart illustrating a video packet processing method according to an embodiment of the present disclosure;
fig. 4 is a schematic diagram of a video frame type identifier map WMM-AC-VI according to an embodiment of the present disclosure;
fig. 5 is a schematic flow chart illustrating determining a related video packet in a passive video packet discarding scheme according to an embodiment of the present disclosure;
fig. 6 is a schematic diagram of a first policy associated with a packet according to an embodiment of the present disclosure;
fig. 7 is a schematic flow chart illustrating a process of determining a relevant video packet in a scheme for actively discarding a video packet according to the embodiment of the present disclosure;
fig. 8 is a schematic diagram of a second policy associated with a packet according to the embodiment of the present disclosure;
fig. 9 is a schematic flow chart illustrating a process of determining a video packet to be discarded in the scheme of actively discarding a video packet according to the present disclosure;
fig. 10 is a first schematic structural diagram of a video packet processing apparatus according to an embodiment of the present disclosure;
fig. 11 is a schematic structural diagram of a video packet processing apparatus according to a second embodiment of the present disclosure.
Detailed Description
Example embodiments will be described more fully hereinafter with reference to the accompanying drawings, but which may be embodied in different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.
As used herein, the term "and/or" includes any and all combinations of one or more of the associated listed items.
The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises," "comprising," "includes," and/or "made from" \8230; \8230 ";" made from ";" specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
Embodiments described herein may be described with reference to plan and/or cross-sectional views in idealized representations of the present disclosure. Accordingly, the example illustrations may be modified in accordance with manufacturing techniques and/or tolerances. Accordingly, the embodiments are not limited to the embodiments shown in the drawings, but include modifications of configurations formed based on a manufacturing process. Thus, the regions illustrated in the figures have schematic properties, and the shapes of the regions shown in the figures illustrate specific shapes of regions of elements, but are not intended to be limiting.
Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the relevant art and the present disclosure, and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.
The embodiment of the disclosure provides a video message processing method, which is applied to a video network shown in fig. 2. As shown in fig. 2, the video network includes a video generation device, a video message processing device, and a video terminal, the video generation device is connected to the video message processing device through a network, and the video terminal is usually connected to the video message processing device through a wireless network. The video generating device may be a media server or a Content Distribution Network (CDN) server, and is configured to perform video coding to generate a video packet, and mark different video frame type identifiers for the video packet according to different video frame types; the video message processing device is a video network transmission device for forwarding video streams, and can be a home gateway or a wireless router and other devices; the video terminal is used for decoding the video stream and displaying the video, and can be a set top box or other equipment. In the embodiment of the disclosure, the video message is transmitted between the video message processing device and the video terminal in a Wi-Fi mode.
With reference to fig. 2 and fig. 3, a video packet processing method provided in the embodiment of the present disclosure includes the following steps:
and step 31, in the process of sending the video message cached in the buffer area to the video terminal, determining the video message to be discarded in response to the satisfaction of the preset video message discarding condition, wherein the video message carries the video frame type identifier.
The video generating device encodes according to a video source to generate a video message, marks a video frame type identifier for the video message according to the video frame type, and sends the video message carrying the video frame type identifier to the video message processing device. The video message processing device receives the video message sent by the video generating device in real time, and buffers the video message in a sending buffer area of a Wi-Fi (wireless fidelity) to wait for sending.
The video frame type identifier may be marked on an IP (Internet Protocol) layer or a TCP (Transmission Control Protocol)/UDP (User Datagram Protocol) layer of the video packet, and in the embodiment of the present disclosure, as shown in fig. 4, the example of marking the video frame type identifier on the IP layer is described. For example, for a video message of an I frame, the video frame type identifier may be marked as frame type identifier-I; for the video message of the P frame, the video frame type identifier can be marked as a frame type identifier-P; for a video message with B frames, the video frame type identifier may be marked as frame type identifier-B. In some embodiments, the video frame type identifier of the IP layer may be a DSCP (Differentiated Services Code Point) priority. Under the condition that the video message processing device is connected with the video terminal by adopting Wi-Fi, different types of video frame differential transmission can be realized by utilizing different Wi-Fi Multimedia-Access category (WMM-AC) transmission mechanisms of the Wi-Fi. When the video message processing device forwards the video message to the video terminal through Wi-Fi, the video message is usually mapped into Wi-Fi WMM-AC-VI. Generally, the video packet processing apparatus cannot recognize the video packet, and in the embodiment of the present disclosure, since the video generating apparatus already marks different types of video frames by the video frame type identifier when the video packet is generated by encoding, the video packet processing apparatus does not need to recognize the video packet of the application layer, and can distinguish the different types of video frames by only recognizing the video frame type identifier (generally, the video packet processing apparatus can recognize the video frame type identifier by hardware), thereby speeding up the recognition of the video packet.
In this step, the video packet processing apparatus determines that the video packet is to be discarded, when the video packet discarding condition is satisfied. In the embodiment of the present disclosure, there are two cases of passively discarding video packets and actively discarding video packets, where under the different cases, the video packet discarding conditions are different, and the video packets to be discarded and the determination methods are also different, and the two cases are described in detail later. It should be noted that, when the video packet discarding condition is not satisfied, the video packet buffered in the buffer is continuously sent to the video terminal, and for example, the video packet may be sent in the order of sending the queue in the buffer.
And step 32, determining the video frame type corresponding to the video frame type identifier of the video message to be discarded, and determining the associated video message of the video message to be discarded in the buffer area according to the preset message association policy and the video frame type.
In this step, the video packet processing apparatus determines the video frame type of the video packet to be discarded by identifying the video frame type identifier in the video packet to be discarded, and finds the associated video packet of the video packet to be discarded in the buffer area by using different packet association strategies according to different situations of passively discarding the video packet or actively discarding the packet.
And step 33, discarding the video message to be discarded and the associated video message.
In this step, the video message processing apparatus discards the video message to be discarded and the video message associated therewith in the buffer, and the discarded video message is no longer sent to the video terminal.
In the video message processing method provided by the embodiment of the disclosure, in the process of sending the video message cached in the buffer area to the video terminal, in response to the satisfaction of the preset video message discarding condition, the video message to be discarded is determined, wherein the video message carries the video frame type identifier; determining a video frame type corresponding to a video frame type identifier of a video message to be discarded, and determining an associated video message of the video message to be discarded in a buffer area according to a preset message association strategy and the video frame type; and discarding the video message to be discarded and the associated video message. According to the embodiment of the disclosure, the associated message of the video message to be discarded can be determined according to the type of the video frame, and different discarding processes are performed on the video messages of different video frame types, so that disordered packet loss is avoided, transmission of key video frames is ensured, and transmission of video frames which cannot be decoded can be reduced, thereby improving the transmission efficiency of a video network and ensuring the video transmission quality.
The following describes a scheme for passively discarding video packets according to an embodiment of the present disclosure with reference to fig. 5 and fig. 6.
In the process of forwarding the video message to the video terminal, the video message processing device triggers the passive discarding of the video message due to the failure of video message transmission caused by wireless interference and the like. The video message processing device judges the video frame type of the video message according to the video frame type identification of the video message, thereby determining the associated video message in the Wi-Fi video sending buffer area according to the association relation of the video frame, and discarding the video message with failed transmission and the associated video message.
In some embodiments, the video packet dropping condition is a video packet first dropping condition, and the meeting of the video packet first dropping condition includes: and the sending of the video message cached in the buffer area fails. The first discarding condition of the video message is a triggering condition for passively discarding the video message, and once the video message is failed to be sent to the video terminal, the video message is triggered to be passively discarded. In this case, the video packet that fails to be sent is the video packet to be discarded.
In some embodiments, the packet association policy is a packet association first policy, and the packet association first policy is an association policy for passively discarding video packets. As shown in fig. 5, the determining, in a buffer area, a video packet associated with the to-be-discarded video packet according to a preset packet association policy and a video frame type (i.e., step 32) includes the following steps:
step 321, determining an associated frame corresponding to the video frame type according to a preset message associated first policy.
In this step, the video message processing apparatus determines a corresponding associated frame according to a video frame type of the video message to be discarded, based on the message associated first policy. In the first strategy of message association, association frames of different video frame types are different, that is, association relations for performing video message association discarding are different for different video frame types. The following describes the first policy associated with the packet in detail with reference to fig. 6.
Step 322, determining the associated video packet corresponding to the associated frame in the buffer.
In this step, the video message processing device determines, according to the video frame type identifier corresponding to the video frame type of the associated frame, the video messages corresponding to the video frame type identifier in the buffer area, where the video messages are the associated video messages of the video message to be discarded.
The video frame types include: the message association first strategy comprises the following steps: in a video coding sequence image group, the related frames of the I frame are a P frame and a B frame between the current I frame and the next I frame; the related frames of the P frame are the P frame and the B frame between the current P frame and the next I frame; b-frames are unassociated frames.
As shown in fig. 6, a video coding sequence GOP starts with an I-frame and ends with an I-frame, and includes a plurality of P-frames and B-frames (no other I-frames) between the starting I-frame and the ending I-frame. The association relationship of the I frame is association relationship 1, which specifically includes: the associated frames of the I-frame are P-frames and B-frames between the current I-frame and the next I-frame. That is, if the type of the video frame of the video packet to be dropped is an I frame (i.e., the current I frame, such as the 1 st video frame), the associated frames of the I frame are all P frames and B frames between the 1 st video frame and the 11 th video frame, i.e., the 2 nd to 10 th video frames. The association relationship of the P frame is association relationship 2, which specifically includes: the associated frames of the P-frame are the P-frame and the B-frame between the current P-frame and the next I-frame. That is, if the video frame type of the video packet to be discarded is a P frame (i.e., a current P frame, such as a 3 rd video frame), the associated frames of the P frame are all P frames and B frames between the 3 rd video frame and the 11 th video frame, i.e., 4 th to 10 th video frames; if the video frame type of the video packet to be discarded is a P frame (i.e., the current P frame, such as the 6 th video frame), the associated frames of the P frame are all P frames and B frames between the 6 th video frame and the 11 th video frame, i.e., the 7 th to 10 th video frames. The association relationship of the B frame is an association relationship 3, which specifically includes: b-frames are unrelated frames. That is, if the video frame type of the video packet to be discarded is a B frame (for example, the 4 th video frame), the B frame has no associated frame, that is, there is no associated video packet.
The following describes a scheme for actively discarding a video packet according to an embodiment of the present disclosure with reference to fig. 7, fig. 8, and fig. 9.
In the process that the video message processing device forwards the video message to the video terminal, the effective bandwidth between the video message processing device and the video terminal is reduced due to wireless interference, air interface competition occupation and the like, so that the transmission requirement of the video message cannot be completely met, the video message processing device can estimate the effective bandwidth in real time, and judge whether to trigger active discarding of the video message according to the bandwidth requirement and effective bandwidth estimation for transmitting the video message.
In some embodiments, the video packet discarding condition is a video packet second discarding condition, and the meeting of the video packet second discarding condition includes: the effective bandwidth between the video message processing device and the video terminal is smaller than the preset bandwidth. The second discarding condition of the video message is a triggering condition for actively discarding the video message, and once the message processing device judges that the effective bandwidth between the video message processing device and the video terminal is smaller than the preset bandwidth, the message processing device indicates that the effective bandwidth between the current video message processing device and the video terminal cannot meet the requirement of transmitting the video message, and correspondingly triggers the video message to be actively discarded. It should be noted that the determination of whether the second discarding condition of the video packet is satisfied may be performed according to a preset period.
In some embodiments, the preset packet association policy is a packet association second policy, and the packet association second policy is an association policy for actively discarding the video packet. As shown in fig. 7, the determining, in the buffer area, the associated video packet of the video packet to be discarded according to the preset packet association policy and the video frame type (i.e., step 32) includes the following steps:
step 321', determining an associated frame corresponding to the video frame type according to the preset message associated second policy.
In this step, the video packet processing apparatus determines a corresponding associated frame according to the video frame type of the video packet to be discarded, based on the packet associated second policy. In the second strategy of message association, association frames of different video frame types are different, that is, association relations for performing video message association discarding are different for different video frame types. Subsequently, the second policy associated with the message is described in detail with reference to fig. 8.
Step 322', the associated video packet corresponding to the associated frame is determined in the buffer.
In this step, the video message processing device determines, according to the video frame type identifier corresponding to the video frame type of the associated frame, the video messages corresponding to the video frame type identifier in the buffer area, where the video messages are the associated video messages of the video message to be discarded.
The video frame types include: the message association second strategy comprises the following steps: in a video coding sequence image group, the related frames of the I frame are a P frame and a B frame between the current I frame and the next I frame; the related frames of the P frames comprise a first related frame and a second related frame, the first related frame is a P frame and a B frame between the current P frame and the next I frame, the second related frame is a B frame between the current P frame and the previous P frame, and if no other P frame exists before the current P frame, the second related frame is a B frame between the current P frame and the previous I frame; b-frames are unassociated frames.
As shown in fig. 8, a video coding sequence GOP starts with an I-frame and ends with an I-frame, and includes a plurality of P-frames and B-frames (no other I-frame) between the starting I-frame and the ending I-frame. The association relationship of the I frame is association relationship 1, which specifically includes: the associated frames of the I-frame are P-frames and B-frames between the current I-frame and the next I-frame. That is, if the type of the video frame of the video packet to be dropped is an I frame (i.e., the current I frame, such as the 1 st video frame), the associated frames of the I frame are all P frames and B frames between the 1 st video frame and the 11 th video frame, i.e., the 2 nd to 10 th video frames. The association relationship of the P frame is association relationship 2, which specifically includes: the associated frames of the P frames comprise a first associated frame and a second associated frame, the first associated frame is an associated frame after the current P frame, and the second associated frame is an associated frame before the current P frame. That is, if the video frame type of the video packet to be discarded is a P frame (i.e., a current P frame, such as a 3 rd video frame), the first associated frame of the P frame is all P frames and B frames between the 3 rd video frame and the 11 th video frame, i.e., 4 th to 10 th video frames; the second associated frame of the P frame is a B frame between the 3 rd video frame and the 1 st video frame, i.e. the 2 nd video frame. If the type of the video frame of the video message to be discarded is a P frame (namely, the current P frame, such as the 6 th video frame), the first associated frame of the P frame is all P frames and B frames between the 6 th video frame and the 11 th video frame, namely, the 7 th to 10 th video frames; the second associated frame of the P-frames is a B-frame, i.e., 4-5 video frames, between the 6 th video frame and the 3 rd video frame (i.e., the P-frame immediately preceding the current P-frame). The association relationship of the B frame is an association relationship 3, which specifically includes: b-frames are unrelated frames. That is, if the video frame type of the video packet to be discarded is a B frame (for example, the 4 th video frame), the B frame has no associated frame, that is, there is no associated video packet.
In some embodiments, as shown in fig. 9, the determining a video packet to be discarded (i.e. step 31) includes the following steps:
and 311, determining the number of messages to be discarded in a preset time length according to the effective bandwidth and the preset bandwidth.
In this step, the video packet processing apparatus determines the number of packets to be discarded within a preset time period according to the effective bandwidth and the preset bandwidth between the video packet processing apparatus and the video terminal, and the specific implementation manner of determining the number of packets to be discarded within the preset time period according to the effective bandwidth and the preset bandwidth may be implemented by using the existing scheme, which is not described herein again.
Step 312, determine the type of video frame to be dropped.
In some embodiments, the video frame type to be dropped may be determined according to a preset priority (i.e., a first priority) corresponding to the video frame type, where the priority is defined according to the packet association second policy. In the second strategy of message association, the B frame has no associated frame, so the video message with the video frame type of the B frame is preferentially and actively discarded. The associated frame of the P frame is more than that of the I frame, therefore, the video message with the video frame type of the P frame is compared with the video message with the video frame type of the I frame, and the video message with the video frame type of the P frame is preferentially and actively discarded. That is to say, in the process of determining the type of the video frame to be discarded, the purpose of discarding as few as possible and ensuring the video messages of the key video frames (such as I frames) as possible is to ensure the accuracy of video decoding. Therefore, in the first priority, the priority of the B frame is higher than that of the P frame, and the priority of the P frame is higher than that of the I frame.
Step 313, determining the video message to be discarded in the buffer according to the video frame type identifier of the video frame type to be discarded.
In this step, the video message processing apparatus determines the video message corresponding to the video frame type identifier in the buffer area, where the video message is the video message to be discarded.
Correspondingly, in the scheme of actively discarding the video packets, the discarding of the to-be-discarded video packets and the associated video packets (i.e. step 33) includes the following steps: and discarding the video messages to be discarded and the associated video messages according to the number of the messages to be discarded within a preset time length.
In the scheme of actively discarding the video message, the video message with the video frame type of B frame is preferentially taken as the video message to be discarded, if the number of the video messages with the video frame type of B frame is not enough (less than the number of the video messages to be discarded within the preset duration determined in step 311), the video message with the video frame type of P frame is preferentially taken as the video message to be discarded, and if the number of the video messages with the video frame type of P frame and the number of the associated video messages thereof are also not enough (namely, the sum of the number of the video messages with the video frame type of P frame, the associated video messages of the video messages with the video frame type of P frame, and the video messages with the video frame type of B frame is still less than the number of the video messages to be discarded within the preset duration determined in step 311), the video message with the video frame type of I frame is taken as the video message to be discarded. In this step, the upper limit of the number of discarded video packets is the number of packets to be discarded within the preset duration determined in step 311.
In some embodiments, after discarding the pending video packets and associated video packets (i.e., step 33), the video packet processing method further comprises the steps of: and sending the video message cached currently in the buffer area to the video terminal, wherein the video message cached currently in the buffer area is the residual video message in the buffer area after the video message is discarded.
In some embodiments, the video message processing method may further include the steps of: and configuring retransmission times or retransmission duration for the video message according to the importance priority (second priority) corresponding to the video frame type of the video message cached in the buffer area.
Because the interdependence relationships of the I frame, the P frame and the B frame in the video message are different, and the importance degrees of the video frames of all types are the I frame, the P frame and the B frame from high to low in sequence, in order to ensure that the video frame with high importance degree obtains lower packet loss rate in the Wi-Fi air interface transmission process, the second priority of the I frame is greater than the second priority of the P frame and the second priority of the B frame is greater than the second priority of the P frame. Correspondingly, the higher the second priority is, the more the retransmission times of the video message are, the longer the retransmission time is.
According to the embodiment of the disclosure, when the video generating device performs video coding to generate a video message, different video frame type identifiers are marked according to different video frame types, so that the video message processing device can identify different video frame types through the video frame type identifiers, and different discarding processing schemes are adopted, thereby realizing video network transmission optimization and reducing video blocking.
Based on the same technical concept, an embodiment of the present disclosure further provides a video packet processing apparatus, as shown in fig. 10, where the video packet processing apparatus includes a to-be-discarded video packet determining module 101, an associated video packet determining module 102, and a packet discarding module 103, and the to-be-discarded video packet determining module 101 is configured to determine, in a process of sending a video packet cached in a buffer to a video terminal, a to-be-discarded video packet in response to a preset video packet discarding condition being satisfied, where the video packet carries a video frame type identifier.
The associated video packet determining module 102 is configured to determine a video frame type corresponding to the video frame type identifier of the video packet to be discarded, and determine an associated video packet of the video packet to be discarded in the buffer according to a preset packet association policy and the video frame type.
The packet discarding module 103 is configured to discard the video packet to be discarded and the associated video packet.
In some embodiments, the video packet dropping condition is a video packet first dropping condition, and the meeting of the video packet first dropping condition includes: and the sending of the video message cached in the buffer area fails.
In some embodiments, the video packet that fails to be sent is the to-be-discarded video packet.
In some embodiments, the packet association policy is a packet association first policy, and the associated video packet determining module 102 is configured to determine, according to a preset packet association first policy, an associated frame corresponding to the video frame type; and determining the associated video message corresponding to the associated frame in the buffer area.
In some embodiments, the video frame types include: the message association first strategy comprises the following steps: in a video coding sequence image group, the related frames of the I frame are a P frame and a B frame between the current I frame and the next I frame; the related frames of the P frame are the P frame and the B frame between the current P frame and the next I frame; b-frames are unassociated frames.
In some embodiments, the video packet dropping condition is a video packet second dropping condition, and the meeting of the video packet second dropping condition includes: the effective bandwidth between the video message processing device and the video terminal is smaller than the preset bandwidth.
In some embodiments, the preset packet association policy is a packet association second policy, and the associated video packet determining module 102 is configured to determine, according to the preset packet association second policy, an associated frame corresponding to the video frame type; and determining the associated video message corresponding to the associated frame in the buffer area.
In some embodiments, the video frame types include: the message association second strategy comprises the following steps: in a video coding sequence image group, the related frames of the I frame are a P frame and a B frame between the current I frame and the next I frame; the associated frames of the P frames comprise a first associated frame and a second associated frame, the first associated frame is a P frame and a B frame between the current P frame and the next I frame, the second associated frame is a B frame between the current P frame and the previous P frame, and if no other P frame exists before the current P frame, the second associated frame is a B frame between the current P frame and the previous I frame; b-frames are unassociated frames.
In some embodiments, the to-be-discarded video packet determining module 101 is configured to determine, according to the effective bandwidth and the preset bandwidth, a number of to-be-discarded packets within a preset time duration; determining the type of a video frame to be discarded; and determining the video message to be discarded in the buffer area according to the video frame type identifier of the video frame type to be discarded.
The packet discarding module 103 is configured to discard the video packet to be discarded and the associated video packet according to the number of the packets to be discarded within the preset time length.
In some embodiments, the to-be-discarded video packet determining module 101 is configured to determine the type of the to-be-discarded video frame according to a preset priority corresponding to the type of the video frame.
In some embodiments, as shown in fig. 11, the video packet processing apparatus further includes a sending module 104, where the sending module 104 is configured to send the video packet currently cached in the buffer to the video terminal after the packet discarding module 103 discards the to-be-discarded video packet and the associated video packet.
An embodiment of the present disclosure further provides a computer device, including: one or more processors and storage; the storage device stores one or more programs, and when the one or more programs are executed by the one or more processors, the one or more processors are enabled to implement the video message processing method provided in the foregoing embodiments.
The disclosed embodiments also provide a computer readable medium, on which a computer program is stored, where the computer program, when executed, implements the video message processing method provided in the foregoing embodiments.
It will be understood by those of ordinary skill in the art that all or some of the steps of the methods, functional modules/units in the apparatus, disclosed above may be implemented as software, firmware, hardware, and suitable combinations thereof. In a hardware implementation, the division between functional modules/units mentioned in the above description does not necessarily correspond to the division of physical components; for example, one physical component may have multiple functions, or one function or step may be performed by several physical components in cooperation. Some or all of the physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application specific integrated circuit. Such software may be distributed on computer readable media, which may include computer storage media (or non-transitory media) and communication media (or transitory media). The term computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data, as is well known to those of ordinary skill in the art. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital Versatile Disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can accessed by a computer. In addition, communication media typically embodies computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media as known to those skilled in the art.
Example embodiments have been disclosed herein, and although specific terms are employed, they are used and should be interpreted in a generic and descriptive sense only and not for purposes of limitation. In some instances, features, characteristics and/or elements described in connection with a particular embodiment may be used alone or in combination with features, characteristics and/or elements described in connection with other embodiments, unless expressly stated otherwise, as would be apparent to one skilled in the art. It will therefore be understood by those skilled in the art that various changes in form and details may be made therein without departing from the scope of the invention encompassed by the appended claims.

Claims (14)

1. A method for processing video packets, the method comprising:
in the process of sending the video message cached in the buffer area to the video terminal, determining the video message to be discarded in response to the satisfaction of a preset video message discarding condition, wherein the video message carries a video frame type identifier;
determining a video frame type corresponding to the video frame type identifier of the video message to be discarded, and determining an associated video message of the video message to be discarded in the buffer area according to a preset message association strategy and the video frame type;
and discarding the video message to be discarded and the associated video message.
2. The method of claim 1, wherein the video packet dropping condition is a video packet first dropping condition, and wherein the meeting the video packet first dropping condition comprises: and the sending of the video message cached in the buffer area fails.
3. The method of claim 2, wherein the video packet that failed to be sent is the video packet to be discarded.
4. The method of claim 2, wherein the packet association policy is a packet association first policy, and wherein determining the video packet associated with the video packet to be discarded in the buffer according to a preset packet association policy and the video frame type comprises:
determining an associated frame corresponding to the video frame type according to a preset message associated first strategy;
and determining the associated video message corresponding to the associated frame in the buffer area.
5. The method of claim 4, wherein the video frame types comprise: the message association first strategy comprises the following steps:
in a video coding sequence image group, the related frames of the I frame are a P frame and a B frame between the current I frame and the next I frame; the related frames of the P frame are the P frame and the B frame between the current P frame and the next I frame; b-frames are unassociated frames.
6. The method of claim 1, wherein the video packet dropping condition is a video packet second dropping condition, and wherein the video packet second dropping condition is satisfied and comprises: the effective bandwidth between the video message processing device and the video terminal is smaller than the preset bandwidth.
7. The method of claim 6, wherein the predetermined packet association policy is a packet association second policy, and wherein determining the video packet associated with the video packet to be discarded in the buffer according to the predetermined packet association policy and the video frame type comprises:
determining an associated frame corresponding to the video frame type according to a preset message associated second strategy;
and determining the associated video message corresponding to the associated frame in the buffer area.
8. The method of claim 7, wherein the video frame types comprise: the message association second strategy comprises the following steps:
in a video coding sequence image group, the related frames of the I frame are a P frame and a B frame between the current I frame and the next I frame; the associated frames of the P frames comprise a first associated frame and a second associated frame, the first associated frame is a P frame and a B frame between the current P frame and the next I frame, the second associated frame is a B frame between the current P frame and the previous P frame, and if no other P frame exists before the current P frame, the second associated frame is a B frame between the current P frame and the previous I frame; b-frames are unrelated frames.
9. The method of claim 6, wherein the determining a video packet to discard comprises:
determining the number of messages to be discarded within a preset time length according to the effective bandwidth and the preset bandwidth;
determining the type of a video frame to be discarded;
determining a video message to be discarded in the buffer area according to the video frame type identifier of the video frame type to be discarded;
the discarding the to-be-discarded video packet and the associated video packet includes:
and discarding the video message to be discarded and the associated video message according to the number of the messages to be discarded within the preset time length.
10. The method of claim 9, wherein the determining a video frame type to discard comprises:
and determining the type of the video frame to be discarded according to the preset priority corresponding to the type of the video frame.
11. The method of any of claims 1-10, wherein after discarding the pending discard video packet and the associated video packet, the method further comprises:
and sending the video message cached currently in the buffer area to a video terminal.
12. A video message processing device is characterized by comprising a to-be-discarded video message determining module, a related video message determining module and a message discarding module, wherein the to-be-discarded video message determining module is used for determining to-be-discarded video messages in response to the satisfaction of preset video message discarding conditions in the process of sending the video messages cached in a buffer area to a video terminal, and the video messages carry video frame type identifiers;
the associated video message determining module is used for determining a video frame type corresponding to the video frame type identifier of the video message to be discarded, and determining the associated video message of the video message to be discarded in the buffer area according to a preset message association strategy and the video frame type;
the message discarding module is used for discarding the video message to be discarded and the associated video message.
13. A computer device, comprising:
one or more processors;
a storage device having one or more programs stored thereon;
the one or more programs, when executed by the one or more processors, cause the one or more processors to implement the video message processing method of any of claims 1-11.
14. A computer-readable medium, on which a computer program is stored, wherein said program, when executed, implements the video message processing method according to any of claims 1-11.
CN202111062123.4A 2021-09-10 2021-09-10 Video message processing method and device, computer equipment and readable medium Pending CN115802094A (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN202111062123.4A CN115802094A (en) 2021-09-10 2021-09-10 Video message processing method and device, computer equipment and readable medium
PCT/CN2022/117440 WO2023036163A1 (en) 2021-09-10 2022-09-07 Video packet processing method and apparatus, computer device, and computer readable storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111062123.4A CN115802094A (en) 2021-09-10 2021-09-10 Video message processing method and device, computer equipment and readable medium

Publications (1)

Publication Number Publication Date
CN115802094A true CN115802094A (en) 2023-03-14

Family

ID=85417129

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111062123.4A Pending CN115802094A (en) 2021-09-10 2021-09-10 Video message processing method and device, computer equipment and readable medium

Country Status (2)

Country Link
CN (1) CN115802094A (en)
WO (1) WO2023036163A1 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20220295352A1 (en) * 2021-03-09 2022-09-15 Arris Enterprises Llc PROPAGATION OF QUALITY OF SERVICE (QoS) MANAGEMENT THROUGH A HETEROGENOUS NETWORK

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2024197958A1 (en) * 2023-03-31 2024-10-03 Nec Corporation Methods, devices and computer readable medium for packet discard

Family Cites Families (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2009005193A (en) * 2007-06-22 2009-01-08 Panasonic Corp Communication terminal
CN101360058A (en) * 2008-09-08 2009-02-04 华为技术有限公司 Method and apparatus for cache overflow control
US8270425B2 (en) * 2009-12-29 2012-09-18 Symbol Technologies, Inc. Method and system for multicast video streaming over a wireless local area network (WLAN)
US8443097B2 (en) * 2010-04-12 2013-05-14 Alcatel Lucent Queue management unit and method for streaming video packets in a wireless network
CA2786200C (en) * 2011-09-23 2015-04-21 Cygnus Broadband, Inc. Systems and methods for prioritization of data for intelligent discard in a communication network
CN104378602B (en) * 2014-11-26 2017-06-23 福建星网锐捷网络有限公司 Video transmission method and device
CN106331835B (en) * 2015-06-26 2019-06-07 成都鼎桥通信技术有限公司 A kind of dynamic adjusting data receives the method and video decoding apparatus of caching
CN105871802A (en) * 2015-12-07 2016-08-17 乐视云计算有限公司 Method, device and system for monitoring transmission of streaming media file
CN107222427A (en) * 2016-03-22 2017-09-29 华为技术有限公司 The method and relevant device of a kind of Message processing

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20220295352A1 (en) * 2021-03-09 2022-09-15 Arris Enterprises Llc PROPAGATION OF QUALITY OF SERVICE (QoS) MANAGEMENT THROUGH A HETEROGENOUS NETWORK
US11991560B2 (en) * 2021-03-09 2024-05-21 Arris Enterprises Llc Propagation of Quality of Service (QoS) management through a heterogenous network

Also Published As

Publication number Publication date
WO2023036163A1 (en) 2023-03-16

Similar Documents

Publication Publication Date Title
US10652580B2 (en) Video data processing method and apparatus
EP2520089B1 (en) Method and system for multicast video streaming over a wireless local area network (wlan)
CN100592670C (en) System and method for controlling self-adaptive forward error dynamically in IPTV network
WO2023036163A1 (en) Video packet processing method and apparatus, computer device, and computer readable storage medium
US9525874B2 (en) Transmitting apparatus and transmission method
US11172267B2 (en) Method and system for scheduling resources for streaming video services in mobile communication networks
US20080170528A1 (en) Method of multicasting base and enhancement layers of a video stream
CN110138632B (en) Data transmission method and device
US10230651B2 (en) Effective intra-frame refresh in multimedia communications over packet networks
CN110474721B (en) Video data transmission method, device and computer readable storage medium
CN111629280B (en) Packet loss processing method and device and readable storage medium
CN110166804B (en) Method, device, communication system and computer readable storage medium for implementing video service
Lu et al. Video streaming over 802.11 WLAN with content-aware adaptive retry
CN111093083A (en) Data transmission method and device
CN111245733B (en) Data transmission method and device
WO2017000836A1 (en) Message transmission method and device
CN109862400B (en) Streaming media transmission method, device and system
CN102223218A (en) Method and equipment for inhibiting media message retransmission
Kim et al. UDP-based extremely low latency streaming
Chen et al. Multi-stages hybrid ARQ with conditional frame skipping and reference frame selecting scheme for real-time video transport over wireless LAN
WO2010054551A1 (en) Method, device and system for video stream transmission
Chilamkurti et al. Concurrent multipath transmission with forward error correction mechanism to overcome burst packet losses for delay-sensitive video streaming in wireless home networks
CN110691036A (en) Data transmission method and device based on video network
CN116112127B (en) Data transmission method, related equipment and storage medium
CN110830160A (en) Data packet transmission method and device

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination