WO2023284445A1 - 视频流的处理方法、装置、设备、存储介质及程序产品 - Google Patents

视频流的处理方法、装置、设备、存储介质及程序产品 Download PDF

Info

Publication number
WO2023284445A1
WO2023284445A1 PCT/CN2022/097539 CN2022097539W WO2023284445A1 WO 2023284445 A1 WO2023284445 A1 WO 2023284445A1 CN 2022097539 W CN2022097539 W CN 2022097539W WO 2023284445 A1 WO2023284445 A1 WO 2023284445A1
Authority
WO
WIPO (PCT)
Prior art keywords
time
frame
frame group
image frame
video stream
Prior art date
Application number
PCT/CN2022/097539
Other languages
English (en)
French (fr)
Inventor
毛峻岭
Original Assignee
腾讯科技（深圳）有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 腾讯科技（深圳）有限公司 filed Critical 腾讯科技（深圳）有限公司
Publication of WO2023284445A1 publication Critical patent/WO2023284445A1/zh
Priority to US18/140,490 priority Critical patent/US20230262273A1/en

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/236—Assembling of a multiplex stream, e.g. transport stream, by combining a video stream with other content or additional data, e.g. inserting a URL [Uniform Resource Locator] into a video stream, multiplexing software data into a video stream; Remultiplexing of multiplex streams; Insertion of stuffing bits into the multiplex stream, e.g. to obtain a constant bit-rate; Assembling of a packetised elementary stream
    • H04N21/2365—Multiplexing of several video streams
    • H04N21/23655—Statistical multiplexing, e.g. by controlling the encoder to alter its bitrate to optimize the bandwidth utilization
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
    • H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/70—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by syntax aspects related to video coding, e.g. related to compression standards
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client 
    • H04N21/61—Network physical structure; Signal processing
    • H04N21/6106—Network physical structure; Signal processing specially adapted to the downstream path of the transmission network
    • H04N21/6131—Network physical structure; Signal processing specially adapted to the downstream path of the transmission network involving transmission via a mobile phone network
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client 
    • H04N21/61—Network physical structure; Signal processing
    • H04N21/6156—Network physical structure; Signal processing specially adapted to the upstream path of the transmission network
    • H04N21/6181—Network physical structure; Signal processing specially adapted to the upstream path of the transmission network involving transmission via a mobile phone network
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client 
    • H04N21/63—Control signaling related to video distribution between client, server and network components; Network processes for video distribution between server and clients or between remote clients, e.g. transmitting basic layer and enhancement layers over different transmission paths, setting up a peer-to-peer communication via Internet between remote STB's; Communication protocols; Addressing
    • H04N21/647—Control signaling between network components and server or clients; Network processes for video distribution between server and clients, e.g. controlling the quality of the video stream, by dropping packets, protecting content from unauthorised alteration within the network, monitoring of network load, bridging between two different networks, e.g. between IP and wireless
    • H04N21/64723—Monitoring of network processes or resources, e.g. monitoring of network load
    • H04N21/64738—Monitoring network characteristics, e.g. bandwidth, congestion level

Definitions

  • the present application relates to the field of computer technology, and in particular to a video stream processing method, device, equipment, computer-readable storage medium, and computer program product.
  • the Group of Pictures (GoP) in the video stream is a group of continuous pictures in the video stream, and the GoP is the frame group in video coding, that is, the coded image frame group.
  • the first frame after encoding is an I frame.
  • audio and video real-time communication is mainly carried out in public network scenarios.
  • the video stream is encoded periodically according to the set GoP duration, that is, the first frame of each GoP period is encoded.
  • the I frame is encoded to obtain an I frame. If there are multiple terminals in the network for real-time audio and video communication, when multiple terminals collide in the uplink transmission of I frames, due to the limited network transmission resources, the transmission of I frames will take more time and increase the time required for video stream transmission. delay.
  • Embodiments of the present application propose a video stream processing method, device, device, computer-readable storage medium, and computer program product, which can reduce the time delay of video stream transmission.
  • An embodiment of the present application provides a method for processing a video stream, executed by a first terminal, including:
  • the first time is the time period from storing the I frame of the first coded image frame group until sending the I frame to the network node
  • the second Time is the network request protection time
  • the I frame of the second encoded image frame set is transmitted to the network node.
  • the embodiment of the present application also provides a device for processing video streams, the device comprising:
  • the first processing module is configured to acquire the first coded image frame group in the video stream
  • the second processing module is configured to determine the first time and the second time corresponding to the first coded image frame group, the first time is from storing the I frame of the first coded image frame group until sending the I frame to the network
  • the time period of the node, the second time is the network request protection time
  • the third processing module is configured to determine at least one second coded image in the video stream according to the first time and the second time when a conflict occurs with at least one second terminal during transmission of the I frame to the network node At the starting moment of the frame group, the second encoded image frame group is after the first encoded image frame group;
  • the fourth processing module is configured to transmit the I frame of the second encoded image frame group to the network node based on the start time of the second encoded image frame group.
  • the embodiment of the present application also provides an electronic device, including: a processor, a memory, and a bus;
  • a bus configured to connect the processor and the memory
  • a memory configured to store operation instructions
  • the processor is configured to execute the above video stream processing method provided by the embodiment of the present application by invoking an operation instruction.
  • the embodiment of the present application also provides a computer-readable storage medium, which stores a computer program.
  • the computer program is executed by a processor, the above video stream processing method provided in the embodiment of the present application is realized.
  • the embodiment of the present application also provides a computer program product, including a computer program or instruction, and when the computer program or instruction is executed by a processor, implements the video stream processing method provided in the embodiment of the present application.
  • the first terminal collides with at least one second terminal in the uplink transmission of the I frame of the first coded image frame group, according to the first time and the second time, determine the video stream after the first coded image frame group at least one start moment of the second coded picture frame group, and based on the determined start moment of the second coded picture frame group, transmitting the I frame of the second coded picture frame group to the network node, so that due to the first time is the time period from storing the I frame of the first coded image frame group to sending the I frame to the network node, and the second time is the network request protection time, so that even if the first terminal sends the I frame of the first coded image frame group to delay , since the second encoded image frame group is determined according to the first time and the second time, it is ensured that the starting moment of at least one second encoded image frame group following the first encoded image frame group can change with the first time and dynamic changes, thereby preventing the first terminal from colliding with the second terminal in the process of transmitting the I frame of the
  • FIG. 1 is a schematic diagram of a system architecture provided by an embodiment of the present application.
  • FIG. 2 is a schematic flowchart of a video stream processing method provided by an embodiment of the present application
  • FIG. 3 is a schematic diagram of the processing of the video stream provided by the embodiment of the present application.
  • FIG. 4 is a schematic diagram of the processing of the video stream provided by the embodiment of the present application.
  • FIG. 5 is a schematic flowchart of another video stream processing method provided by the embodiment of the present application.
  • FIG. 6 is a schematic structural diagram of a device for processing video streams provided by an embodiment of the present application.
  • FIG. 7 is a schematic structural diagram of an electronic device provided by an embodiment of the present application.
  • the embodiment of the present application is a video stream processing method provided for audio and video processing, such as a video stream processing method provided for audio and video processing in the field of cloud technology, and the video stream processing method may involve cloud technology Various fields, such as cloud computing and cloud services in cloud technology.
  • An I frame is an intra picture, which is formed by encoding according to the image content of the current frame in the video, and its size is usually much larger than that of an inter-frame encoded frame.
  • the I frame is usually the first frame of each GOP, which is moderately compressed and used as a reference point for random access, and can be regarded as an image.
  • An I frame can be regarded as a compressed product of an image.
  • a GoP In real-time video coding, in order to reduce the codec delay, in a GoP, it is usually composed of I frame + forward-referenced inter-frame coding frame.
  • the first frame is set as an I frame, and the subsequent frames are all P frames.
  • the P frame is a forward predictive coding frame (predictive-frame), which compresses the coded image with the amount of transmitted data by fully compressing the time redundant information lower than the previous coded frame in the image sequence, and the P frame is also called a predictive frame.
  • the P frame adopts inter-frame coding, and its decoding needs to rely on the previous frame (I frame) to proceed.
  • the I frame Since the I frame is relatively large, its transmission delay and delay jitter are also relatively long, which becomes one of the important factors affecting the delay of real-time video communication.
  • uplink and downlink wireless transmission resources will be configured using different frames. Only uplink data transmission is allowed on the uplink frame, for example, the terminal sends uplink data to the base station; only downlink data transmission is allowed on the downlink frame, for example The base station sends downlink data to the terminal.
  • the frame configuration of a common 5G network is 3D1U, where D represents a downlink frame and U represents an uplink frame.
  • D represents a downlink frame
  • U represents an uplink frame.
  • D represents a downlink frame
  • U represents an uplink frame.
  • Uplink frames are transmitted; the size of the uplink data that can be transmitted in one U frame is related to base station configuration, terminal configuration, signal strength, etc., and the size of uplink data that can be transmitted in one U frame is usually about 125KB; 1080P video stream The size of the I frame is about 80KB, and the encoding parameters can be set according to the actual situation; when multiple terminals in the network transmit video streams at the same time, and multiple terminals simultaneously transmit I frames in the uplink to form a collision (conflict), some of the multiple terminals It is necessary to delay the transmission of the U frame of the next 5ms period, thus generating additional network delay.
  • the first terminal receives the first encoded image frame group in the video stream; the first terminal determines the first encoded image The first time and the second time corresponding to the frame group; when the first terminal collides with at least one second terminal in the process of transmitting the I frame to the network node, the first terminal then according to the first time and the second time Time, determine the start moment of at least one second encoded image frame group in the video stream, and based on the start moment of the second encoded image frame group, after storing the I frame of the second encoded image frame group in the first terminal, the The stored I frame of the second coded image frame group is sent to the network node, which can prevent the first terminal from colliding with at least one second terminal during the uplink transmission of the I frame group of the second coded image frame group, reducing the video flow rate. transmission delay and alleviate delay jitter.
  • FIG. 1 shows a schematic structural diagram of a video stream processing system applicable to the embodiment of the present application. It can be understood that the video stream processing method provided in the embodiment of the present application can be applied to But it is not limited to the application scenario shown in FIG. 1 .
  • the video stream processing system in this example may include but not limited to a first terminal 101 , multiple second terminals 102 , a network node 103 , and a control platform 104 .
  • Network node 103 may communicate with control platform 104 over a network.
  • Video stream transmission is performed between multiple terminals and the network node 103 , where the multiple terminals include a first terminal 101 and multiple second terminals 102 .
  • the first terminal 101 and the plurality of second terminals 102 respectively collect video streams through respective video stream collection devices, where the video stream collection devices may be cameras.
  • the first terminal 101 and the plurality of second terminals 102 respectively store the I frames of the coded image frame group in the collected video stream into their respective buffers, wherein the buffers may be network buffers of modules/chips in the terminals.
  • the first terminal 101 and at least one second terminal 102 collide (collision) in the uplink transmission of the I frame of a coded image frame group the first terminal 101 will determine the starting moment of the subsequent coded image frame group, and Based on the start time of the subsequent encoded image frame group, sending the I frame of the subsequent encoded image frame group to the network node 103 can prevent the first terminal 101 from communicating with the second terminal 101 in the uplink transmission of the I frame of the subsequent encoded image frame group.
  • Terminal 102 has a conflict.
  • the multiple terminals respectively send the video streams to the network node 103 , and the network node 103 transmits the video streams back to the control platform 104 .
  • the control platform 104 remotely controls multiple terminals respectively according to the video stream.
  • Each of the multiple terminals can work independently in a distributed manner, and each terminal does not need to coordinate and exchange information with each other; the network node 103 can allocate resources supporting uplink transmission of coded image frame groups.
  • the first terminal 101 or the second terminal 102 can be a smart phone (such as an Android phone, an iOS phone, etc.), a tablet computer, a notebook computer, a digital broadcast receiver, a MID (Mobile Internet Devices, a mobile Internet device), a PDA (personal digital assistants), desktop computers, vehicle terminals (such as vehicle navigation terminals), smart speakers, smart watches, unmanned vehicles, etc.
  • the network node 103 may be a base station or the like.
  • the control platform 104 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, and can also provide cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication Cloud servers or server clusters for basic cloud computing services such as middleware services, domain name services, security services, content delivery network (Content Delivery Network, CDN), and big data and artificial intelligence platforms.
  • the above-mentioned network may include but not limited to: wired network, wireless network, wherein, the wired network includes: local area network, metropolitan area network and wide area network, and the wireless network includes: bluetooth, Wi-Fi and other networks that realize wireless communication. It can also be determined based on actual application scenario requirements, which is not limited here.
  • FIG. 2 shows a schematic flowchart of a video stream processing method provided by an embodiment of the present application, wherein the method can be executed by any electronic device, such as a first terminal, as an optional implementation The method can be executed by the first terminal.
  • the first terminal will be used as an example to execute the method.
  • the video stream processing method provided by the embodiment of the present application includes the following steps:
  • the first terminal acquires a first coded image frame group in a video stream.
  • the first terminal can obtain the video stream in various ways, for example, by receiving the video stream transmitted by other terminals, or obtain the video stream locally, or obtain the video stream from the server to a video stream (such as a live video stream).
  • the first terminal can also obtain the video stream through video capture.
  • the first terminal can be a terminal with a video capture device (such as a camera), and the video stream is collected through the video capture device.
  • the first Terminals can be unmanned trucks, remotely driven vehicles, cranes, cranes, etc.
  • a self-driving truck collects a video stream of the road surface through a camera, and caches the video stream to a storage device on the driverless truck.
  • the video stream includes multiple GoPs, and the first coded picture frame group is a GoP in the video stream.
  • the first time is the time period from storing the I frame of the first coded image frame group to sending the I frame to the network node, that is, from the time when the I frame of the first coded image frame group is stored in the first From the terminal to the time period until the stored I frame of the first coded image frame group is sent to the network node, it can be understood that the starting moment of the first time is when the I frame of the first coded image frame group is stored in the second
  • the end point of the first time is the time point when the I frame of the first coded image frame group is sent to the network node; the second time is the network request protection time.
  • the first terminal determines the first time and the second time of the first coded image frame group, and the first time is from the time when the I frame of the first coded image frame group is stored in the buffer memory of the first terminal until the A time period during which the I frame of the first coded image frame group stored in the cache is sent to the network node.
  • the cache of the first terminal may be a network cache.
  • the network cache refers to the network cache of the 5G CPE or 5G module inserted into the first terminal.
  • a time period in a 5G network is 5 ms, and a time period includes 3 downlink frames D, 1 uplink frame U, and 1 special subframe S, and the time of each downlink frame D
  • the lengths are all 1ms, the time length of each uplink frame U is 1ms, and the time length of each special subframe S is 1ms.
  • the first terminal stores the I frame of the first coded image frame group into the cache of the first terminal as the start time (T0+1) of the first time, and stores the I frame of the first coded image frame group in the buffer
  • the end time sent to the network node is used as the end time of the first time (T0+15); wherein, T0 is the start time point when the second terminal stores the I frame of the coded image frame group of the second terminal into the cache, for example, T0 can The value is 1000ms.
  • the time period from the start moment of the first time to the end moment of the first time includes: the time when the first terminal stores the I frame of the first coded image frame group into the cache of the first terminal, the time when the first terminal The time when a terminal sends a scheduling request to the network node, the time when the first terminal sends the I frame of the first coded image frame group in the cache to the network node, and the like.
  • determining the second time of the first coded image frame group includes:
  • the second time is determined according to the acquired network scheduling request time and the preset protection time.
  • determining the second time of the first coded image frame group includes steps A1-A2:
  • step A1 the minimum time period from when the network node receives the uplink scheduling request sent by the terminal to when the network node allocates an uplink resource time slot for the uplink scheduling request is determined as the network scheduling request time.
  • the network scheduling request time is related to 5G network settings, for example, the network scheduling request time is usually 5ms (milliseconds).
  • Step A2 according to the network scheduling request time and the preset protection time, determine the second time, that is, the network request protection time.
  • the preset protection time is the buffer sending window protection time.
  • the cache sending window protection time refers to the redundant time reserved for protecting the network cache sending processing. Due to the configuration of the 5G uplink time slot, the uploadable time of the I frame is discrete, so a certain buffer sending window protection time can be reserved before the uplink time slot is planned to be sent, in order to ensure that the data will not be sent in advance to the previous uplink time slot , the buffer sending window protection time value should be lower than the shortest time interval between uplink time slots in two 5G frames. For example, the value range of the preset protection time is 2ms-5ms. The preset protection time is usually as high as possible, so as to better reduce the fluctuation of I frame delay when GoP is reset.
  • the sum of the network scheduling request time and the preset protection time is calculated, and the sum value is used as the network request protection time.
  • the second coded picture frame group is the coded picture frame group after the first coded picture frame group in the video stream.
  • the start time of the second coded image frame group is the start time point of storing the I frame of the second coded image frame group.
  • the starting moment of the second coded image frame group is the time point when the I frame of the second coded image frame group is stored in the cache of the first terminal, and the time point may be that the I frame is stored in the first terminal The start time point of the cache.
  • the time threshold T1 is the time required for I frame transmission in a non-collision state (non-collision state).
  • the subsequent coded picture frame group is a coded picture frame group following a second coded picture frame group adjacent to the first coded picture frame group.
  • the start time of the subsequent coded image frame group is the start time point of storing the I frame of the subsequent coded image frame group into the buffer memory of the first terminal.
  • the start time of the subsequent coded picture frame group can be determined based on the start time of the second coded picture frame group adjacent to the first coded picture frame group and the GoP period (coded picture frame group period).
  • the sum of the start time of the second coded picture frame group adjacent to the first coded picture frame group and the integer multiple of the GoP cycle is used as the start time of the subsequent coded picture frame group;
  • the starting moment of the second coded image frame group adjacent to the group is T, then, the starting moment of the third second coded image frame group after the first coded image frame group is: the same as the first coded image frame group
  • the corresponding start time of the second coded picture frame group is T0+N
  • the GoP period is M
  • the starting moment is T0+N+M
  • T0 is the start time point when the second terminal stores the I frame of the coded image frame group of the second terminal into the cache
  • N and M are positive integers; for example, N is 6ms
  • the value of M is 500ms, that is, T0+N+M is T0+506ms, when the value of T0 is 1000ms, then T0+506ms is 1506ms, and the starting time corresponding to the coded image frame group 1 is 1506ms.
  • the next coded picture frame group adjacent to the coded picture frame group 1 is the coded picture frame group 2, and so on, and the next coded picture frame group adjacent to the coded picture frame group Q-1 is the coded picture frame group Q, where Q is a positive integer, then the starting time corresponding to the coded image frame group Q is T0+N+Q ⁇ M; for example, the value of N is 6ms, the value of M is 500ms, and the value of Q is 20, then the coded image frame group The starting time corresponding to 20 is T0+6+20 ⁇ 500, that is, T0+10006ms. When the value of T0 is 1000ms, then T0+10006ms is 11006ms, and the starting time corresponding to the coded image frame group 20 is 11006ms.
  • the preset time threshold is determined by at least one of the network scheduling request time and the uplink frame waiting time corresponding to the I frame.
  • the time threshold is obtained by summing the network scheduling request time and the uplink frame waiting time corresponding to the I frame, wherein the uplink frame waiting time corresponding to the I frame is the uplink frame waiting time occupied by the actual transmission of the I frame time.
  • the network scheduling request time is 5ms
  • the uplink frame waiting time is: the number of occupied uplink frames ⁇ 5ms.
  • the cache has a waiting window period
  • T1 setting can also be added with a decision protection interval, the time required for I frame transmission in the non-collision state ⁇ T1 ⁇ the time consumed for I frame transmission in the collision state.
  • the network scheduling request time is the minimum time period from when the network node receives the uplink scheduling request sent by the terminal to when the network node allocates an uplink resource time slot for the uplink scheduling request.
  • the uplink frame waiting time corresponding to the I frame is the uplink frame waiting time required for the actual transmission of the I frame.
  • the judgment guard interval refers to the redundant time used to protect the inaccurate judgment of the 5G uplink time slot used for I frame upload due to the jitter of the cache clearing time; due to the configuration of the 5G uplink time slot, the time for the I frame to be uploaded is discrete, Therefore, the decision protection window can be set according to the time interval of uplink time slots, that is, the decision protection interval, and the value of the decision protection window is lower than the shortest time interval of uplink time slots in two 5G frames.
  • the decision guard interval may take a frame interval of 1 ms; T1 may take a value of 11 ms.
  • the first terminal may determine that a collision occurs with at least one second terminal during transmission of the I frame to the network node in the following manner:
  • the first terminal compares the first time with the preset time threshold, and judges whether the preset condition is met; when the first time is greater than the preset time threshold and the preset condition is met, determine to transmit the I to the network node A collision occurs with at least one second terminal during the frame.
  • the preset conditions include at least one of the following:
  • the upstream transmission of the I frame of the first coded image frame group is the first upstream transmission of the I frame of the video stream;
  • the fourth time of a plurality of consecutive third encoded image frame groups in the video stream is greater than the time threshold, the fourth time is the uplink transmission duration of the I frame of the third encoded image frame group, and the uplink transmission duration is, from the third
  • the I frame of the coded image frame group is stored in the first terminal until the time period when the stored I frame of the third coded image frame group is sent to the network node; multiple third coded image frame groups are the coded picture frame group preceding the picture frame group;
  • the first time is greater than the sum of the time threshold and the frame period of the video stream.
  • the first terminal such as a video device, sets the GoP cycle duration as an integer multiple of the 5G network frame cycle, for example, the 5G network frame cycle can be 5ms; when the first terminal performs I frame uplink transmission, the I frame is sent to Enter the network cache of the 5G module/chip of the first terminal, and the first terminal monitors changes in the cache.
  • the first time T2 is greater than the preset time threshold T1 and satisfies the preset condition, then according to the first time T2 and the second time T3, determine the start moment corresponding to the second coded image frame group in the video stream.
  • the preset conditions include at least one of the following:
  • Continuous multiple times (for example, can be set to 2-3 times, the more the total number of terminals, the more times can be set) I frame transmission, send the I frame in the buffer, each time the buffer is cleared and timed out, for example, T2 is greater than T1; it indicates There is a continuous collision (collision) between the first terminal and at least one second terminal, which is not accidental;
  • T2 is greater than T1+1 frame period, for example, 1 frame period may be 5ms.
  • the starting moment of the second encoded image frame group after the first encoded image frame group can be determined as follows:
  • the start moment of the first time the 3rd time and the preset coded picture frame group period, determine the start moment of the second coded picture frame group, the start moment of the first time is the I of the first coded picture frame group The start time of frame storage;
  • the third time is determined by the first time and the second time, and the third time is the I frame sending adjustment time.
  • determining the starting moment of the second encoded image frame group after the first encoded image frame group includes steps B1-B2:
  • step B1 a third time is obtained according to the first time and the second time, and the third time is an I frame transmission adjustment time.
  • the third time is obtained, including:
  • T2 can take a value of 14ms
  • T3 can take a value of 9ms
  • T4 can take a value of 5ms.
  • determining the third time includes:
  • the first time and the second time determine the third time, including:
  • the third time is m times the frame period of the video stream, where m is a positive integer.
  • the third time T4 is determined according to the frame period T7, the first time T2 and the second time T3 of the video stream; for example, the difference between the first time T2 and the second time T3 is calculated to determine the first time T4 A value is (T2-T3); according to the m times of the frame period of the video stream and the first value (T2-T3), the second value n is obtained, and when the second value n is the smallest non-negative integer, the third time is determined T4 is m ⁇ T7, where m is such that m ⁇ T7 ⁇ T2+T3 is the smallest non-negative integer n.
  • T4 m ⁇ T7, wherein, m is such that m ⁇ T7 ⁇ T2+T3 is the smallest non-negative integer n.
  • T7 can take a value of 16.6ms
  • T2 can take a value of 14ms
  • T3 can take a value of 9ms
  • the first value is T2–T3, that is, the first value is 5ms
  • m ⁇ T7–T2+T3 16.6m-5
  • Step B2 according to the start time of the first time, the third time and the preset coded picture frame group period, determine the start time of the second coded picture frame group, the start time of the first time is the first coded picture frame group period The start time point when the I frame of the frame group is stored in the cache of the first terminal.
  • the preset coded image frame group period is a GoP period, and one GoP period is the distance between two adjacent I frames.
  • the third time T4 and the preset coded picture frame group period of 500 ms determine the corresponding start time of the second coded picture frame group, where T0 is the first
  • T0 is the first
  • N is a positive integer ;
  • the value of N is 1, and the value of T4 is 5ms, then the starting time corresponding to the second coded image frame group is T0+506ms.
  • the stored I frame of the second encoded image frame group Frames are sent to network nodes.
  • the first terminal based on the start time of the second coded picture frame group, stores the I frame of the second coded picture frame group into the cache of the first terminal, and then stores the second coded picture frame in the cache
  • the I frame of the frame group is sent to the network node, so as to prevent the first terminal from colliding with at least one second terminal in the uplink transmission of the I frame.
  • a time period in a 5G network is 5 ms, and a time period includes 3 downlink frames D, 1 uplink frame U, and 1 special subframe S, and the time of each downlink frame D
  • the lengths are all 1ms, the time length of each uplink frame U is 1ms, and the time length of each special subframe S is 1ms.
  • the first terminal Based on the start time (T0+506ms) of the second encoded image frame group, the first terminal stores the I frame of the second encoded image frame group in the buffer, and then stores the I frame of the second encoded image frame group in the buffer sent to the base station, and the first terminal finishes sending the I frame at time (T0+515ms), so that the first terminal can avoid conflict with the second terminal in the uplink transmission of the I frame.
  • the start time point at which the second terminal stores the I frame of the coded image frame group of the second terminal into the cache is T0+500ms.
  • the first terminal sends the starting moment (T0+506ms) of the second coded image frame group to the base station, to instruct the base station to ), allocating resources supporting uplink transmission of the second coded image frame group.
  • the I frame of the second encoded image frame group is obtained after the start moment of the second encoded image frame group
  • the I frame of the second encoded image frame group is stored in the first terminal, and The I frame of the second coded image frame group is sent to the network node.
  • the I frame of the second encoded image frame group is obtained after the start moment of the second encoded image frame group
  • the I frame of the second encoded image frame group is stored in the network cache of the first terminal .
  • the first terminal encodes and obtains the I frame of the second encoded image frame group before the start moment of the second encoded image frame group, therefore, the first terminal is at the beginning of the second encoded image frame group I frame buffering is performed at all times.
  • the first terminal encodes and obtains the I frame of the second encoded image frame group after the start moment of the second encoded image frame group, therefore, the first terminal I frame buffering is performed after the start moment of the second coded image frame group.
  • the I frame of the second coded image frame group is obtained at the corresponding starting moment of the second coded image frame group, and the obtained first coded image frame group is The I frame of the second coded image frame group is stored in the first terminal.
  • the I frame of the second coded picture frame group is obtained at the corresponding start moment of the second coded picture frame group, and the obtained first coded picture frame group is The I frame of the second coded image frame group is stored in the network cache of the first terminal.
  • the first time when the first time is greater than the preset time threshold and meets the preset condition, it indicates that when the first terminal sends the I frame of the first coded image frame group, the first terminal sends the I frame of the first coded image frame group A conflict occurs with at least one second terminal in the uplink transmission of the I frame; based on the determined starting moment of the second coded image frame group, after storing the I frame of the second coded image frame group into the cache of the first terminal, the stored The I frame of the second coded image frame group in the cache is sent to the network node, which can prevent the first terminal from colliding with at least one second terminal in the uplink transmission of the I frame group of the second coded image frame group, reducing the video flow rate. transmission delay and alleviate delay jitter.
  • the method for processing the video stream provided by the embodiment of the present application will be described below in conjunction with an application scenario of unmanned driving.
  • the video stream processing method provided by the embodiment of the present application is applied to an unmanned vehicle, that is, the above-mentioned first terminal is an unmanned vehicle or is installed on the unmanned vehicle.
  • two terminal scenarios are taken as an example in a 5G network.
  • the two terminals include a first terminal and a second terminal, where the first terminal represents an unmanned vehicle 1, and the second terminal Denotes an unmanned vehicle 2 .
  • a time period in a 5G network is 5ms, and a time period includes 3 downlink frames D, 1 uplink frame U, and 1 special subframe S.
  • the time length of each downlink frame D is 1ms
  • U is 1 ms
  • the time length of each special subframe S is 1 ms.
  • the first I frame uplink transmission of the first terminal collides with the second terminal as shown in FIG. 3 , causing the I frame transmission of the first terminal to be delayed by 5 ms.
  • the start time point when the second terminal stores the I frame of the coded image frame group of the second terminal into the buffer is T0, and the first terminal stores the I frame of the first coded image frame group into the cache of the first terminal as the first terminal.
  • the start time (T0+1ms) of a time, the end time when the first terminal sends the I frame of the first coded image frame group in the cache to the base station is taken as the end time (T0+15ms) of the first time.
  • a time period in a 5G network is 5 ms, and a time period includes 3 downlink frames D, 1 uplink frame U, and 1 special subframe S, and the time length of each downlink frame D is 1ms, the duration of each uplink frame U is 1ms, and the duration of each special subframe S is 1ms.
  • the buffer clearing time (first time) T2 of the first terminal is 14ms, and T2 is greater than the T1 that the first terminal is set, and wherein, T1 is 11ms, then the first terminal starts the I frame uplink transmission adjustment process;
  • the second time T3 is 9ms,
  • the third time T4 is 5ms, then the I frame sending start time of the reset GoP cycle of the first terminal is T0+6ms, when the GoP cycle duration is 500ms, the I frame in the next GoP cycle of the first terminal (the second The I frame in the coded image frame group) is sent at T0+506ms, that is, the start time corresponding to the second coded image frame group of the first terminal is T0+506ms.
  • the first terminal and the second terminal avoid the I frame collision during the uplink transmission of the I frame in the coded image frame group of the subsequent GoP period.
  • the uplink data transmission is mainly the return of the video stream, and the video stream data has a high degree of periodicity, so the video stream processing method provided in the embodiment of the present application can avoid I-frame uplink transmission collisions between multiple terminals, thereby reducing the Latency and jitter in video transmission.
  • the scheduling priority of the video stream terminal can be set to the highest in the network configuration, and the video stream processing method provided by this embodiment of the present application can still work effectively.
  • multiple terminals collect soil samples in a harsh environment, and the terminals may be artificial intelligence devices, such as robots.
  • Multiple robots collect video streams of the surrounding environment through their own cameras, and multiple robots store the I frames of the coded image frame groups in the captured video streams into their respective caches.
  • Robot No. 1 among the multiple robots determines The first time and the second time corresponding to the first coded image frame group, the first time is from the I frame of the first coded image frame group is stored in the cache of the first terminal until the first coded image frame group in the cache is stored The time period when the I frame is sent to the base station, and the second time is the network request protection time; when the first time is greater than the preset time threshold and meets the preset condition, it indicates that the No.
  • robot No. 1 robot sends the I frame of the first coded image frame group , robot No. 1 conflicts with other robots in the uplink transmission of the I frame of the first coded image frame group; then robot No. 1 will determine that the second coded image frame group in the video stream corresponds to The starting moment of the second encoded image frame group is after the first encoded image frame group, and the corresponding starting moment of the second encoded image frame group is the start time point when the I frame of the second encoded image frame group is stored in the cache; Based on the starting moment corresponding to the second coded image frame group, after the No.
  • the base station allocates resources supporting the uplink transmission of the coded image frame group, so that robot No. 1 avoids conflicts with other robots in the uplink transmission of the I frame.
  • Multiple robots respectively send video streams to the same base station, and the base station transmits the video streams back to the remote control platform through the network.
  • the control platform remotely controls multiple robots, which can make the robots avoid obstacles and quickly search for the target soil.
  • Each of the multiple robots can work independently in a distributed manner, and each robot does not need to coordinate with each other and perform information interaction.
  • FIG. 5 shows a schematic flowchart of another video stream processing method provided by the embodiment of the present application, wherein the method can be executed by any electronic device, such as the first terminal, as an optional In the implementation manner, description will be made by taking the first terminal as an example for executing the method.
  • the video stream processing method provided by the embodiment of the present application includes the following steps:
  • step S201 the first terminal collects video streams through a camera.
  • the first terminal may be an unmanned vehicle, and the content in the video stream is related to the road surface on which the unmanned vehicle is driving.
  • Step S202 the first terminal obtains the first coded image frame group in the video stream.
  • Step S203 the first terminal determines the first time and the second time of the first coded image frame group.
  • the first time is a time period from when the first terminal stores the I frame of the first coded image frame group to when the I frame is sent to the network node, and the second time is the network request protection time.
  • Step S204 when the first time is greater than the preset time threshold and meets the preset condition, the first terminal determines the starting moment of the second coded image frame group in the video stream according to the first time and the second time.
  • the second encoded image frame group is received after the first encoded image frame group, and the start moment of the second encoded image frame group is to store the I frame of the second encoded image frame group into the first The start time of the terminal's cache.
  • the first time when the first time is greater than the preset time threshold and the preset condition is met, it indicates that the first terminal communicates with at least one second Terminal conflict.
  • Step S205 based on the start time of the second coded image frame group, after the first terminal stores the I frame of the second coded image frame group in the cache, sends the I frame of the second coded image frame group stored in the cache to base station.
  • the first terminal sends the video stream and the I frame of the second encoded image frame group included in the video stream and stored in the buffer to the base station, which can avoid the uplink transmission of the I frame with multiple second terminals. A conflict occurred.
  • Step S206 the base station transmits the video stream back to the control platform.
  • Step S207 the control platform remotely controls the first terminal according to the video stream.
  • control platform remotely controls the unmanned vehicle according to the relevant video stream of the road where the unmanned vehicle is driving, thereby enhancing the driving safety of the unmanned vehicle.
  • the collision of multi-terminal video stream I-frame transmission in the 5G private network scenario is reduced, and the delay and delay jitter of video stream transmission are alleviated.
  • multiple terminals under the 5G private network can refresh the time position of the I frame uplink transmission of each terminal without negotiation, avoiding the gap between multiple terminals. Collision of I frame uplink transmission.
  • the video stream processing method provided in the embodiment of the present application includes the following steps:
  • the first terminal determines a first time T2.
  • the network cache size is read every 1 ms cycle, and when the network cache size is reduced to 0, the elapsed time is recorded
  • the cycle number is T2, and the unit is ms.
  • T4 is determined in any of the following two ways:
  • T2 may take a value of 14ms
  • T3 may take a value of 9ms
  • T4 may take a value of 5ms.
  • T7 can take a value of 16.6ms
  • T2 can take a value of 14ms
  • T3 can take a value of 9ms
  • m ⁇ T7–T2+T3 16.6m-5; when m is 1, it makes 16.6m-5
  • n is 12
  • the start moment of the reset GoP period may be the start moment of the second coded image frame group in the video stream.
  • the first terminal notifies the video receiving end of the start time of the reset GoP period.
  • the video receiving end may be a base station.
  • the start moment of the reset GoP period may be the start moment corresponding to the second encoded image frame group in the video stream, for example, the start moment corresponding to the second encoded image frame group in the video stream is T0+506ms.
  • the start moment of the reset GoP period may be the start moment corresponding to the second coded image frame group in the video stream.
  • the operation is to disconnect the video stream, wait for the T4 video stream reconstruction time, and re-establish the video stream; according to the GoP frame number, if it is the first frame, output the I frame encoding identifier, and perform I frame encoding on the current frame to obtain the I frame; after encoding Determine whether the current moment is later than the reset start moment of the I frame+n ⁇ GoP period, that is, the start moment corresponding to the second coded image frame group in the video stream.
  • the current moment for example, T0+505ms
  • the reset I frame start moment+n ⁇ GoP period T0+506ms.
  • the current moment for example, T0+507ms
  • the reset I frame start moment+n ⁇ GoP period T0+506ms.
  • t2 on the premise that the video stream is not disconnected, for the current frame arrival time t2, if t2 is located in the interval [reset I frame start time + n ⁇ GoP cycle - I frame encoding time, reset I frame start time+n ⁇ GoP period-I frame encoding time+frame period], output the I frame encoding flag, and perform I frame encoding on the current frame to obtain the I frame; after encoding, judge whether the current time is later than the reset I frame start time + n ⁇ GoP cycle, n is the current GoP cycle number, if the current time is later than the reset I frame start time + n ⁇ GoP cycle, then output the I frame transmission flag, and carry out I frame data send.
  • the embodiment of the present application also provides a video stream processing device, which can be installed in the first terminal.
  • the structural diagram of the device is shown in Figure 6.
  • the video stream processing device 40 includes the first A processing module 401 , a second processing module 402 , a third processing module 403 and a fourth processing module 404 .
  • the first processing module 401 is configured to acquire the first coded image frame group in the video stream
  • the second processing module 402 is configured to determine the first time and the second time corresponding to the first encoded image frame group, the first time is from storing the I frame of the first encoded image frame group until storing the stored I frame
  • the time period sent to the network node, the second time is the network request protection time
  • the third processing module 403 is configured to determine at least one second code in the video stream according to the first time and the second time when a conflict occurs with at least one second terminal during the transmission of the I frame to the network node At the starting moment of the image frame group, the second encoded image frame group is after the first encoded image frame group;
  • the fourth processing module 404 is configured to transmit the I frame of the second encoded image frame group to the network node based on the start time of the second encoded image frame group.
  • the second processing module 402 is further configured to:
  • the second time is determined according to the acquired network scheduling request time and the preset protection time.
  • the third processing module 403 is further configured to:
  • the third time and the preset coded picture frame group period determine the start moment of the second coded picture frame group, the start time of the first time is, the first coded picture frame group The start time point when the I frame is stored in the first terminal;
  • the third time is determined by the first time and the second time, and the third time is the I frame sending adjustment time.
  • the third processing module 403 is further configured to:
  • the third processing module 403 is further configured to:
  • the third time is m times the frame period of the video stream, where m is a positive integer.
  • the fourth processing module 404 is further configured to:
  • the I frame of the second coded picture frame group is obtained, then the I frame of the second coded picture frame group is stored in the first terminal, and the second coded picture frame group The I frame is sent to the network node.
  • the fourth processing module 404 is further configured to:
  • the preset time threshold is determined by at least one of the network scheduling request time and the uplink frame waiting time corresponding to the I frame.
  • the preset conditions include at least one of the following:
  • the upstream transmission of the I frame of the first coded image frame group is the first upstream transmission of the I frame of the video stream;
  • the fourth time of multiple consecutive third coded image frame groups in the video stream is greater than the time threshold, and the fourth time is the uplink transmission duration of the I frame of the third coded image frame group, that is, the time of the third coded image frame group
  • the I frame is stored in the first terminal until the time period during which the stored I frame of the third encoded image frame group is sent to the network node, and the plurality of third encoded image frame groups are before the first encoded image frame group;
  • the first time is greater than the sum of the time threshold and the frame period of the video stream.
  • the first terminal collides with at least one second terminal in the uplink transmission of the I frame of the first coded image frame group; according to the first time and the second time, determine that the video stream is after the first coded image frame group
  • the start time of at least one second coded picture frame group and based on the determined start time of the second coded picture frame group, transmit the I frame of the second coded picture frame group to the network node, so, due to the first
  • the time is the time period from storing the I frame of the first coded image frame group to sending the I frame to the network node
  • the second time is the network request protection time, so that even if the first terminal is scheduled by the base station to carry out the I frame Since the second coded image frame group is determined according to the first time and the second time, it is ensured that the starting moment of at least one second coded image frame group following the first coded image frame group can follow the first time
  • the change of the dynamic change thereby avoiding the conflict between the first terminal and the second terminal in the process of transmitting the I
  • the embodiment of the present application also provides an electronic device, the structural diagram of which is shown in Figure 7, the electronic device 9000 includes at least one processor 9001, memory 9002 and bus 9003, at least one The processors 9001 are electrically connected to the memory 9002; the memory 9002 is configured to store at least one computer-executable instruction, and the processor 9001 is configured to execute the at least one computer-executable instruction, thereby executing the video as provided in the embodiment of the present application The method of processing the stream.
  • the processor 9001 may be a Field-Programmable Gate Array (Field-Programmable Gate Array, FPGA) or other devices with logic processing capabilities, such as a Microcontroller Unit (Microcontroller Unit, MCU), a Central Processing Unit (Central Processing Unit) Process Unit, CPU).
  • FPGA Field-Programmable Gate Array
  • MCU Microcontroller Unit
  • CPU Central Processing Unit
  • the embodiment of the present application also provides a computer-readable storage medium, which stores a computer program, and the computer program is used to realize the steps of the video stream processing method provided in the embodiment of the present application when executed by a processor .
  • the embodiment of the present application also provides a computer program product, including a computer program or instruction, and when the computer program or instruction is executed by a processor, implements the video stream processing method provided in the embodiment of the present application.
  • the computer-readable storage medium includes but is not limited to any type of disk (including floppy disk, hard disk, optical disk, CD-ROM, and magneto-optical disk), read-only memory (Read-Only Memory, ROM), random storage (Random Access Memory, RAM), Erasable Programmable Read-Only Memory (EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Flash Memory, Magnetic card or ray card. That is, a readable storage medium includes any medium that stores or transmits information in a form readable by a device (eg, a computer).
  • the embodiment of the present application also provides a computer program product containing instructions, which, when run on a computer device, causes the computer device to execute the video stream processing method provided in the embodiment of the present application.
  • each block in these structural diagrams and/or block diagrams and/or flow diagrams and combinations of blocks in these structural diagrams and/or block diagrams and/or flow diagrams can be realized by computer programs.
  • these computer program products can be provided to general-purpose computers, professional computers, or processors of other programmable data processing methods for implementation, so that the computer or processors of other programmable data processing methods can execute the present invention.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Computer Security & Cryptography (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

本申请实施例提供了一种视频流的处理方法、装置、设备、计算机可读存储介质及计算机程序产品，其中，方法包括：获取视频流中的第一编码图像帧组；确定第一编码图像帧组对应的第一时间和第二时间，第一时间为，从存储第一编码图像帧组的I帧、至将I帧发送给网络节点的时间段，第二时间为网络请求保护时间；当向网络节点传输I帧的过程中与至少一个第二终端发生冲突时，根据第一时间和第二时间，确定视频流中至少一个第二编码图像帧组的起始时刻，所述第二编码图像帧组在所述第一编码图像帧组之后；基于所述第二编码图像帧组的起始时刻，向网络节点传输所述第二编码图像帧组的I帧。

Description

视频流的处理方法、装置、设备、存储介质及程序产品
相关申请的交叉引用
本申请基于申请号为202110802637.2、申请日为2021年07月15日的中国专利申请提出，并要求该中国专利申请的优先权，该中国专利申请的全部内容在此引入本申请作为参考。
技术领域
本申请涉及计算机技术领域，尤其涉及一种视频流的处理方法、装置、设备、计算机可读存储介质及计算机程序产品。
背景技术
视频流中的图像组(Group of Pictures，GoP)是视频流中一组连续的画面，GoP为视频编码中的帧组，即编码图像帧组。通常在一个GoP中，编码后的第一个帧为I帧。
相关技术中，音视频实时通信主要是在公网场景下进行，在视频流开启后，按照设置的GoP时长，周期性地对视频流进行编码，即对每个GoP周期的第一个帧进行I帧编码，得到I帧。网络中如果存在多个终端进行音视频实时通信，当多个终端在I帧的上行传输中发生冲突，由于网络传输资源有限，从而导致I帧传输需要占用更多的时间，增加视频流传输的时延。
发明内容
本申请实施例提出一种视频流的处理方法、装置、设备、计算机可读存储介质及计算机程序产品，能够降低视频流传输的时延。
本申请实施例提供了一种视频流的处理方法，由第一终端执行，包括：
获取视频流中的第一编码图像帧组；
确定第一编码图像帧组对应的第一时间和第二时间，第一时间为，从存储第一编码图像帧组的I帧、直至将所述I帧发送给网络节点的时间段，第二时间为网络请求保护时间；
当向所述网络节点传输所述I帧的过程中与至少一个第二终端发生冲突时，根据第一时间和第二时间，确定视频流中至少一个第二编码图像帧组的起始时刻，所述第二编码图像帧组在第一编码图像帧组之后；
基于第二编码图像帧组的起始时刻，向所述网络节点传输第二编码图像帧组的I帧。
本申请实施例还提供了一种视频流的处理装置，该装置包括：
第一处理模块，配置为获取视频流中的第一编码图像帧组；
第二处理模块，配置为确定第一编码图像帧组对应的第一时间和第二时间，第一时间为，从存储第一编码图像帧组的I帧、直至将所述I帧发送给网络节点的时间段，第二时间为网络请求保护时间；
第三处理模块，配置为当向所述网络节点传输所述I帧的过程中与至少一个第二终端发生冲突时，根据第一时间和第二时间，确定视频流中至少一个第二编码图像帧组的起始时刻，所述第二编码图像帧组在第一编码图像帧组之后；
第四处理模块，配置为基于第二编码图像帧组的起始时刻，向所述网络节点传输第二编码图像帧组的I帧。
本申请实施例还提供了一种电子设备，包括：处理器、存储器和总线；
总线，配置为连接处理器和存储器；
存储器，配置为存储操作指令；
处理器，配置为通过调用操作指令，执行本申请实施例提供的上述视频流的处理方法。
本申请实施例还提供了一种计算机可读存储介质，存储有计算机程序，计算机程序被处理器执行时，实现本申请实施例提供的上述视频流的处理方法。
本申请实施例还提供了一种计算机程序产品，包括计算机程序或指令，所述计算机程序或指令被处理器执行时，实现本申请实施例提供的视频流的处理方法。
本申请实施例提供的技术方案，至少具有如下有益效果：
当第一终端在第一编码图像帧组的I帧的上行传输中与至少一个第二终端发生冲突时，根据第一时间和第二时间，确定视频流中在第一编码图像帧组之后的至少一个第二编码图像帧组的起始时刻，并基于确定的第二编码图像帧组的起始时刻，向所述网络节点传输第二编码图像帧组的I帧，如此，由于第一时间是从存储第一编码图像帧组的I帧至将I帧发送给网络节点的时间段，第二时间是网络请求保护时间，使得即使第一终端发送第一编码图像帧组的I帧延后，由于第二编码图像帧组是根据第一时间和第二时间确定的，保证了第一编码图像帧组后面的至少一个第二编码图像帧组的起始时刻，能够随第一时间的变化而动态变化，进而避免第一终端在向所述网络节点传输第二编码图像帧组的I帧的过程中与第二终端发生冲突，降低了视频流传输的时延，并缓解了时延抖动。
附图说明
图1为本申请实施例提供的系统架构的示意图；
图2为本申请实施例提供的一种视频流的处理方法的流程示意图；
图3为本申请实施例提供的视频流的处理示意图；
图4为本申请实施例提供的视频流的处理示意图；
图5为本申请实施例提供的另一种视频流的处理方法的流程示意图；
图6为本申请实施例提供的一种视频流的处理装置的结构示意图；
图7为本申请实施例提供的一种电子设备的结构示意图。
具体实施方式
为使得本申请的发明目的、特征、优点能够更加的明显和易懂，下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本申请一部分实施例，而非全部实施例。基于本申请中的实施例，本领域技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本申请保护的范围。
下面详细描述本申请的实施例，该实施例的示例在附图中示出，其中自始至终相同或类似的标号表示相同或类似的元件或具有相同或类似功能的元件。下面通过参考附图描述的实施例是示例性的，仅用于解释本申请，而不能解释为对本申请的限制。
本技术领域技术人员可以理解，除非特意声明，这里使用的单数形式“一”、“一个”、“所述”和“该”也可包括复数形式。应该理解的是，本申请的说明书中使用的措辞“包括”是指存在特征、整数、步骤、操作、元件和/或组件，但是并不排除存在或添加一个或多个其他特征、整数、步骤、操作、元件、组件和/或它们的组。应该理解，当我们称元件被“连接”或“耦接”到另一元件时，它可以直接连接或耦接到其他元件，或者也可以存在中间元件。此外，这里使用的“连接”或“耦接”可以包括无线连接或无线耦接。这里使用的措辞“和/或”包括一个或更多个相关联的列出项的全部或任一单元和全部组合。
本申请实施例是针对音视频处理提供的一种视频流的处理方法，如可以是针对云技术领域中的音视频处理所提供的视频流的处理方法，该视频流的处理方法可以涉及云技术的多种领域，例如云技术中的云计算、云服务等。
为了更好的理解及说明本申请实施例的方案，下面对本申请实施例中所涉及到的一些技术用语进行简单说明。
(1)实时视频编码中的I帧和GoP
I帧是帧内编码帧(intra picture)，根据视频中当前帧的图像内容进行编码形成，通常大小会比采用帧间编码的帧要大很多。I帧通常是每个GOP的第一个帧，经过适度地压缩，作为随机访问的参考点，可以当成图像。I帧可以看成是一个图像经过压缩后的产物。
在实时视频编码中，为降低编解码时延，在一个GoP中，通常采用 I帧+前向参考的帧间编码帧构成，当采用H.264编码器，面向实时视频编码场景，通常在一个GoP中设置首帧为I帧，后续的帧均为P帧。P帧为前向预测编码帧(predictive-frame)，通过充分将低于图像序列中前面已编码帧的时间冗余信息来压缩传输数据量的编码图像，P帧也叫预测帧。
P帧采用帧间编码，其解码需要依赖前面帧(I帧)才能进行。
由于I帧较大，其传输的时延和时延抖动也相对较长，成为影响实时视频通信时延的重要因素之一。
(2)第五代移动通信技术(5th Generation Mobile Communication Technology，5G)网络的上行资源调度
在5G网络中，上下行无线传输资源会利用不同的帧来进行配置，上行帧上只能允许上行数据的传输，例如终端发送上行数据到基站；下行帧上只能允许下行数据的传输，例如基站发送下行数据到终端。
常见的5G网络的帧配置为3D1U，其中，D表示下行帧，U表示上行帧。例如，一个时间周期为5ms，则有3个下行帧D、1个上行帧U以及1个特殊子帧S，每个帧时间长度均为1ms；多个终端的上行数据会被基站调度到可以进行上行传输的帧上进行传输，当多个终端同时上行传输的数据量较大时，且当前5ms周期内的上行帧无法满足，则多个终端中的部分终端会被调度到后面5ms周期的上行帧进行传输；1个U帧的可以传输的上行数据的大小与基站配置、终端配置、信号强度等相关，1个U帧的可以传输的上行数据的大小通常在125KB左右；1080P视频流的I帧大小在80KB左右，可根据实际情况进行编码参数设置；当网络内多个终端同时传输视频流，且多个终端同时在上行传输I帧形成碰撞(冲突)，多个终端中的部分终端就需要延后到后面5ms周期的U帧进行传输，从而产生了额外的网络时延。
本申请实施例提供的方案涉及云技术，本申请提供的实施例可以相互结合，对于相同或相似的概念或过程可能在某些实施例中不再赘述。下面将结合附图，对本申请的实施例进行描述。
本申请实施例所提供的方案可以适用于云技术领域中任意需要音视频处理的应用场景，通过该方案第一终端接收视频流中的第一编码图像帧组；第一终端确定第一编码图像帧组对应的第一时间和第二时间；当第一终端向所述网络节点传输所述I帧的过程中与至少一个第二终端发生冲突时，第一终端则根据第一时间和第二时间，确定视频流中至少一个第二编码图像帧组的起始时刻，并基于第二编码图像帧组的起始时刻，将第二编码图像帧组的I帧存入第一终端后，将所存入的第二编码图像帧组的I帧发送给网络节点，可以避免第一终端在第二编码图像帧组的I帧的上行传输中与至少一个第二终端发生冲突，降低了视频流传输的时延，并缓解了时延抖动。
为了更好的理解本申请实施例提供的方案，下面结合一个应用场景对该方案进行说明。
在一个实施例中，图1中示出了本申请实施例所适用的一种视频流的处理系统的结构示意图，可以理解的是，本申请实施例所提供的视频流的处理方法可以适用于但不限于应用于如图1所示的应用场景中。
本示例中，如图1所示，该示例中的视频流的处理系统可以包括但不限于第一终端101、多个第二终端102、网络节点103、控制平台104。网络节点103可以通过网络与控制平台104通信。多个终端与网络节点103之间进行视频流传输，多个终端包括第一终端101和多个第二终端102。第一终端101和多个第二终端102分别通过各自的视频流采集设备采集到视频流，其中，视频流采集设备可以是摄像头。第一终端101和多个第二终端102分别将各自采集到的视频流中编码图像帧组的I帧存入到各自的缓存，其中，缓存可以是终端中模组/芯片的网络缓存。当第一终端101和至少一个第二终端102在一个编码图像帧组的I帧的上行传输中发生碰撞(冲突)时，第一终端101会确定后续的编码图像帧组的起始时刻，并基于后续的编码图像帧组的起始时刻，将后续的编码图像帧组的I帧发送给网络节点103，可以避免第一终端101在后续编码图像帧组的I帧的上行传输中与第二终端102发生冲突。
多个终端分别将视频流发送给网络节点103，网络节点103将视频流回传到控制平台104。控制平台104根据视频流，远程分别对多个终端进行控制。多个终端中的每个终端可以分布式地自行工作，每个终端之间可以不需要相互协调和进行信息交互；网络节点103可以对支持编码图像帧组进行上行传输的资源进行分配。
可理解，上述仅为一种示例，本实施例在此不作限定。
其中，第一终端101或第二终端102可以是智能手机(如Android手机、iOS手机等)、平板电脑、笔记本电脑、数字广播接收器、MID(Mobile Internet Devices，移动互联网设备)、PDA(个人数字助理)、台式计算机、车载终端(例如车载导航终端)、智能音箱、智能手表、无人驾驶的车辆等。网络节点103可以是基站等。控制平台104可以是独立的物理服务器，也可以是多个物理服务器构成的服务器集群或者分布式系统，还可以是提供云服务、云数据库、云计算、云函数、云存储、网络服务、云通信、中间件服务、域名服务、安全服务、内容分发网络(Content Delivery Network，CDN)、以及大数据和人工智能平台等基础云计算服务的云服务器或服务器集群。上述网络可以包括但不限于：有线网络，无线网络，其中，该有线网络包括：局域网、城域网和广域网，该无线网络包括：蓝牙、Wi-Fi及其他实现无线通信的网络。也可基于实际应用场景需求确定，在此不作限定。
参见图2，图2示出了本申请实施例提供的一种视频流的处理方法的 流程示意图，其中，该方法可以由任一电子设备执行，如可以是第一终端，作为一可选实施方式，该方法可以由第一终端执行，为了描述方便，在下文的一些可选实施例的描述中，将以第一终端作为该方法执行主体为例进行说明。如图2所示，本申请实施例提供的视频流的处理方法包括如下步骤：
S101，第一终端获取视频流中的第一编码图像帧组。
在一些实施例中，第一终端可以通过各种可以获取到视频流的方式获取到视频流，例如，可以通过接收其它终端传输的视频流，或者从本地获取到视频流，或者从服务器处获取到视频流(如直播视频流)。
在另一些实施例中，第一终端还可以通过视频采集获取到视频流，例如，第一终端可以是带视频采集设备(如摄像头)的终端，通过视频采集设备进行视频流采集，例如第一终端可以是无人驾驶的货车、远程驾驶的车辆、吊机、天车等。
举例说明，无人驾驶的货车通过摄像头收集有关路面的视频流，并将视频流缓存到无人驾驶的货车上的存储设备。视频流包括多个GoP，第一编码图像帧组为视频流中的一个GoP。
S102，确定第一编码图像帧组对应的第一时间和第二时间。
这里，第一时间为，从存储第一编码图像帧组的I帧至将所述I帧发送给网络节点的时间段，也即，从第一编码图像帧组的I帧被存入第一终端、直至将所存入的第一编码图像帧组的I帧发送给网络节点的时间段，可以理解的，第一时间的起始时刻为第一编码图像帧组的I帧被存入第一终端的时间点，第一时间的终点时刻为，将第一编码图像帧组的I帧发送给网络节点的时间点；第二时间为网络请求保护时间。
在一个实施例中，第一终端确定第一编码图像帧组的第一时间和第二时间，第一时间为从第一编码图像帧组的I帧被存入第一终端的缓存、直至将所存入缓存中的第一编码图像帧组的I帧发送给网络节点的时间段。
在一个实施例中，第一终端的缓存可以是网络缓存。例如，当第一终端插入5G用户驻地设备(Customer-Premises Equipment，CPE)或者5G模组，则网络缓存是指第一终端插入的5G CPE或者5G模组的网络缓存。
举例说明，如图3所示，在5G网络中的一个时间周期为5ms，一个时间周期包括3个下行帧D、1个上行帧U以及1个特殊子帧S，每一个下行帧D的时间长度均为1ms，每一个上行帧U的时间长度均为1ms，每一个特殊子帧S的时间长度均为1ms。第一终端将第一编码图像帧组的I帧存入第一终端的缓存的开始时刻作为第一时间的起始时刻(T0+1)，将缓存中的第一编码图像帧组的I帧发送给网络节点的结束时刻作为第一时间的终点时刻(T0+15)；其中，T0为第二终端将第二终端的编码图 像帧组的I帧存入缓存的开始时间点，例如T0可以取值为1000ms。
在一个实施例中，从第一时间的起始时刻到第一时间的终点时刻的时间段包括：第一终端将第一编码图像帧组的I帧存入第一终端的缓存的时间、第一终端向网络节点发送调度请求的时间、第一终端将缓存中的第一编码图像帧组的I帧发送给网络节点的时间等。
在一个实施例中，确定第一编码图像帧组的第二时间，包括：
根据获取的网络调度请求时间和预设的保护时间，确定第二时间。
在一个实施例中，确定第一编码图像帧组的第二时间，包括步骤A1-A2：
步骤A1，将从网络节点接收到终端发送的上行调度请求、直至网络节点为上行调度请求分配上行资源时隙的最小时间段，确定为网络调度请求时间。
在一个实施例中，网络调度请求时间与5G网络设置有关，例如，网络调度请求时间通常为5ms(毫秒)。
步骤A2，根据网络调度请求时间和预设的保护时间，确定第二时间，即网络请求保护时间。
在一个实施例中，预设的保护时间为缓存发送窗口保护时间。缓存发送窗口保护时间，是指用于保护网络缓存发送处理预留的冗余时间。由于5G上行时隙的配置，I帧可上传的时间是离散的，所以可在计划发送上行时隙前预留一定的缓冲发送窗口保护时间，为保证数据不会提前至前一个上行时隙发送，缓冲发送窗口保护时间取值应低于两个5G帧中上行时隙的最短时间间隔。例如，预设的保护时间取值范围为2ms-5ms。预设的保护时间通常尽量取高值，从而可以更好的降低GoP重置时I帧延迟的波动。
在一个实施例中，计算网络调度请求时间和预设的保护时间的和，将和值作为网络请求保护时间。
S103，当向网络节点传输I帧的过程中与至少一个第二终端发生冲突时，根据第一时间和第二时间，确定视频流中至少一个第二编码图像帧组的起始时刻。
这里，第二编码图像帧组为，视频流中在第一编码图像帧组之后的编码图像帧组，在实际应用中，视频流中第一编码图像帧组之后存在多个(至少两个)连续的第二编码图像帧组，第二编码图像帧组的起始时刻为第二编码图像帧组的I帧存入的开始时间点。
当向网络节点传输I帧的过程中与至少一个第二终端发生冲突时，根据第一时间和第二时间，确定与第一编码图像帧组相邻的第二编码图像帧组的起始时刻、以及位于该相邻的第二编码图像帧组之后的后续编码图像帧组的起始时刻。
在一个实施例中，第二编码图像帧组的起始时刻为将第二编码图像 帧组的I帧存入第一终端的缓存的时间点，该时间点可以为I帧存入第一终端的缓存的开始时间点。
在一个实施例中，时间阈值T1为非碰撞状态(非冲突状态)下I帧传输需要消耗的时间。后续编码图像帧组是在与第一编码图像帧组相邻的第二编码图像帧组之后的编码图像帧组。
在一个实施例中，后续编码图像帧组的起始时刻为，将后续编码图像帧组的I帧存入第一终端的缓存的开始时间点。后续编码图像帧组的起始时刻可基于与第一编码图像帧组相邻的第二编码图像帧组的起始时刻与GoP周期(编码图像帧组周期)所确定，在一个实施例中，将与第一编码图像帧组相邻的第二编码图像帧组的起始时刻与GoP周期的整数倍的和作为后续编码图像帧组的起始时刻；例如，设定与第一编码图像帧组相邻的第二编码图像帧组的起始时刻为T，则，第一编码图像帧组之后的第三个第二编码图像帧组的起始时刻为：与第一编码图像帧组相邻的第二编码图像帧组的起始时刻与两个GoP周期的和。
举例说明，第二编码图像帧组对应的起始时刻为T0+N，GoP周期为M，则与第二编码图像帧组相邻的下一个编码图像帧组(编码图像帧组1)对应的起始时刻为T0+N+M，其中，T0为第二终端将第二终端的编码图像帧组的I帧存入缓存的开始时间点，N，M为正整数；例如，N取值为6ms，M取值为500ms，即T0+N+M为T0+506ms，当T0取值为1000ms，则T0+506ms为1506ms，编码图像帧组1对应的起始时刻为1506ms。与编码图像帧组1相邻的下一个编码图像帧组为编码图像帧组2，依次类推，与编码图像帧组Q-1相邻的下一个编码图像帧组为编码图像帧组Q，其中Q为正整数，则编码图像帧组Q对应的起始时刻为T0+N+Q×M；例如，N取值为6ms，M取值为500ms，Q取值为20，则编码图像帧组20对应的起始时刻为T0+6+20×500，即T0+10006ms，当T0取值为1000ms，则T0+10006ms为11006ms，编码图像帧组20对应的起始时刻为11006ms。
在一个实施例中，预设的时间阈值由网络调度请求时间和I帧对应的上行帧等待时间中的至少一项来确定。
在一个实施例中，将网络调度请求时间与I帧对应的上行帧等待时间之间求和，得到时间阈值，其中，I帧对应的上行帧等待时间为I帧实际传输需要占用的上行帧等待时间。
举例说明，在3D1U帧配置下，网络调度请求时间为5ms，上行帧等待时间为：占用上行帧数×5ms。
在一个实施例中，缓存有等待窗口期，T1设置还可加上判决保护间隔，非碰撞状态下I帧传输需要消耗的时间<T1<碰撞状态下I帧传输需要消耗的时间。计算网络调度请求时间与I帧对应的上行帧等待时间之间的和，得到非碰撞状态下I帧传输需要消耗的时间。计算非碰撞状态下I帧传输需要消耗的时间与判决保护间隔之间的和，得到碰撞状态下I帧传 输需要消耗的时间。其中，网络调度请求时间为从网络节点接收到终端发送的上行调度请求、直至网络节点为上行调度请求分配上行资源时隙的最小时间段。I帧对应的上行帧等待时间为I帧实际传输需要占用的上行帧等待时间。判决保护间隔，是指用于保护因为缓存清空时间抖动造成对I帧上传所用5G上行时隙判断不准确的冗余时间；由于5G上行时隙的配置，I帧可上传的时间是离散的，所以可根据上行时隙时间间隔设置判决保护窗口，即判决保护间隔，判决保护窗口取值低于两个5G帧中上行时隙的最短时间间隔。
举例说明，判决保护间隔可以取值为一个帧间隔1ms；T1可以取值为11ms。
在一些实施例中，第一终端可通过如下方式确定向网络节点传输I帧的过程中与至少一个第二终端发生冲突：
第一终端比较第一时间和预设时间阈值，并判断预设条件是否得到满足；当第一时间大于预设时间阈值，且预设条件得到满足时，确定向所述网络节点传输所述I帧的过程中与至少一个第二终端发生冲突。
其中，预设条件包括以下至少一项：
第一编码图像帧组的I帧上行传输为视频流的首次I帧上行传输；
视频流中的连续多个第三编码图像帧组的第四时间均大于时间阈值，第四时间为，第三编码图像帧组的I帧的上行传输时长，该上行传输时长为，从第三编码图像帧组的I帧被存入第一终端、直至将所存入的第三编码图像帧组的I帧发送给网络节点的时间段；多个第三编码图像帧组为在第一编码图像帧组之前的编码图像帧组；
第一时间大于时间阈值与视频流的帧周期的和。
在一个实施例中，第一终端，例如视频设备，设置GoP周期时长为5G网络帧周期的整数倍，例如5G网络帧周期可以为5ms；第一终端进行I帧上行传输时，将I帧送入第一终端的5G模组/芯片的网络缓存，第一终端监测缓存的变化。当第一时间T2大于预设的时间阈值T1且满足预设条件，则根据第一时间T2和第二时间T3，确定视频流中的第二编码图像帧组对应的起始时刻。其中，预设条件包括以下至少一项：
首次I帧上行传输；
连续多次(例如，可以设置为2-3次，终端总数越多，次数可以设置越多)I帧传输，发送缓存中的I帧，每次缓存都清空超时，例如T2大于T1；则表明第一终端与至少一个第二终端之间是连续的碰撞(冲突)，并非偶然；
T2大于T1+1个帧周期，例如，1个帧周期可以为5ms。
在一个实施例中，根据第一时间和第二时间，可通过如下方式确定第一编码图像帧组之后的第二编码图像帧组的起始时刻：
根据第一时间的起始时刻、第三时间和预设的编码图像帧组周期， 确定第二编码图像帧组的起始时刻，第一时间的起始时刻为第一编码图像帧组的I帧存入的开始时间点；
其中，第三时间由第一时间和第二时间确定，第三时间为I帧发送调整时间。
在一个实施例中，根据第一时间和第二时间，确定第一编码图像帧组之后的第二编码图像帧组的起始时刻，包括步骤B1-B2：
步骤B1，根据第一时间和所述第二时间，得到第三时间，第三时间为I帧发送调整时间。
在一个实施例中，根据第一时间和第二时间，得到第三时间，包括：
计算第一时间和第二时间之间的差值，得到第三时间。
举例说明，计算第一时间T2和第二时间T3之间的差值，得到第三时间T4；其中，T2可以取值为14ms，T3可以取值为9ms，则T4取值为5ms。
在一个实施例中，根据第一时间和第二时间，确定第三时间，包括：
根据视频流的帧周期、第一时间和第二时间，确定第三时间，第三时间为帧周期的整数倍；
根据视频流的帧周期、第一时间和第二时间，确定第三时间，包括：
计算第一时间和第二时间之间的差值，得到第一数值；
根据视频流的帧周期的m倍和第一数值，得到第二数值；
当第二数值为最小非负整数，则确定第三时间为视频流的帧周期的m倍，其中m为正整数。
在一个实施例中，根据视频流的帧周期T7、第一时间T2和第二时间T3，确定第三时间T4；例如，计算第一时间T2和第二时间T3之间的差值，确定第一数值为(T2-T3)；根据视频流的帧周期的m倍和第一数值(T2-T3)，得到第二数值n，当第二数值n为最小非负整数，则确定第三时间T4为m×T7，其中，m是使得m×T7–T2+T3为最小非负整数n。
举例说明，T4＝m×T7，其中，m是使得m×T7–T2+T3为最小非负整数n。T7可以取值为16.6ms，T2可以取值为14ms，T3可以取值为9ms，则第一数值为T2–T3，即第一数值为5ms，m×T7–T2+T3＝16.6m-5；当m为1时，使得16.6m-5为最小非负整数12，即第二数值n为12；T4＝m×T7＝1×T7＝16.6ms。
步骤B2，根据第一时间的起始时刻、第三时间和预设的编码图像帧组周期，确定第二编码图像帧组的起始时刻，第一时间的起始时刻为将第一编码图像帧组的I帧存入第一终端的缓存的开始时间点。
在一个实施例中，预设的编码图像帧组周期为GoP周期，一个GoP周期为两个相邻I帧之间的距离。
举例说明，根据第一时间的起始时刻(T0+1)、第三时间T4和预设的编码图像帧组周期500ms，确定第二编码图像帧组对应的起始时刻， 其中，T0为第二终端将第二终端的编码图像帧组的I帧存入缓存的开始时间点，第二编码图像帧组对应的起始时刻为(T0+1)+T4+N×500，N为正整数；当N取值为1，T4取值为5ms，则第二编码图像帧组对应的起始时刻为T0+506ms。
S104，基于第二编码图像帧组的起始时刻，向所述网络节点传输第二编码图像帧组的I帧。
在实际应用中，第一终端基于第二编码图像帧组的起始时刻，将第二编码图像帧组的I帧存入第一终端后，将所存入的第二编码图像帧组的I帧发送给网络节点。
在一个实施例中，第一终端基于第二编码图像帧组的起始时刻，将第二编码图像帧组的I帧存入第一终端的缓存后，将存入缓存中的第二编码图像帧组的I帧发送给网络节点，以用于避免第一终端与至少一个第二终端在I帧的上行传输中发生冲突。
举例说明，如图4所示，在5G网络中的一个时间周期为5ms，一个时间周期包括3个下行帧D、1个上行帧U以及1个特殊子帧S，每一个下行帧D的时间长度均为1ms，每一个上行帧U的时间长度均为1ms，每一个特殊子帧S的时间长度均为1ms。第一终端基于第二编码图像帧组的起始时刻(T0+506ms)，将第二编码图像帧组的I帧存入缓存后，将存入缓存中的第二编码图像帧组的I帧发送给基站，第一终端在时刻(T0+515ms)完成I帧发送，从而第一终端可以避免与第二终端在I帧的上行传输中发生冲突。其中，第二终端将第二终端的编码图像帧组的I帧存入缓存的开始时间点为T0+500ms。
举例说明，如图4所示，第一终端将第二编码图像帧组的起始时刻(T0+506ms)发送给基站，以指示基站根据第二编码图像帧组的起始时刻(T0+506ms)，对支持第二编码图像帧组进行上行传输的资源进行分配。
在一个实施例中，当在第二编码图像帧组的起始时刻之后，得到第二编码图像帧组的I帧，则将第二编码图像帧组的I帧存入第一终端，并将第二编码图像帧组的I帧发送给网络节点。
在一个实施例中，当在第二编码图像帧组的起始时刻之后，得到第二编码图像帧组的I帧，则将第二编码图像帧组的I帧存入第一终端的网络缓存。
在一个实施例中，第一终端在第二编码图像帧组的起始时刻之前，编码并得到第二编码图像帧组的I帧，因此，第一终端在第二编码图像帧组的起始时刻进行I帧缓存。
在一个实施例中，在一种异常情况下，第一终端在第二编码图像帧组的起始时刻之后，才编码并得到第二编码图像帧组的I帧，因此，第一终端在第二编码图像帧组的起始时刻之后进行I帧缓存。
在一个实施例中，通过重置视频流或调整第一编码图像帧组长度， 使得在第二编码图像帧组对应的起始时刻得到第二编码图像帧组的I帧，并将得到的第二编码图像帧组的I帧存入第一终端。
在一个实施例中，通过重置视频流或调整第一编码图像帧组长度，使得在第二编码图像帧组对应的起始时刻得到第二编码图像帧组的I帧，并将得到的第二编码图像帧组的I帧存入第一终端的网络缓存。
本申请实施例中，当第一时间大于预设的时间阈值且满足预设条件，则表明第一终端发送第一编码图像帧组的I帧时，第一终端在第一编码图像帧组的I帧的上行传输中与至少一个第二终端发生冲突；基于确定的第二编码图像帧组的起始时刻，将第二编码图像帧组的I帧存入第一终端的缓存后，将存入缓存中的第二编码图像帧组的I帧发送给网络节点，可以避免第一终端在第二编码图像帧组的I帧的上行传输中与至少一个第二终端发生冲突，降低了视频流传输的时延，并缓解了时延抖动。
为了更好的理解本申请实施例所提供的方法，下面结合无人驾驶的应用场景，对本申请实施例的提供的视频流的处理方法进行说明。这里，本申请实施例所提供的视频流的处理方法应用于无人驾驶车辆，也即上述提到的第一终端为无人驾驶车辆或设置于该无人驾驶车辆。
举例说明，如图3所示，在5G网络中以2个终端场景为例进行示意，2个终端包括第一终端和第二终端，其中，第一终端表示无人驾驶车辆1，第二终端表示无人驾驶车辆2。在5G网络中的一个时间周期为5ms，一个时间周期包括3个下行帧D、1个上行帧U以及1个特殊子帧S，每一个下行帧D的时间长度均为1ms，每一个上行帧U的时间长度均为1ms，每一个特殊子帧S的时间长度均为1ms。第一终端首次I帧上行传输与第二终端发生如图3所示的碰撞，导致了第一终端的I帧发送延后了5ms。第二终端将第二终端的编码图像帧组的I帧存入缓存的开始时间点为T0，第一终端将第一编码图像帧组的I帧存入第一终端的缓存的开始时刻作为第一时间的起始时刻(T0+1ms)，第一终端将缓存中的第一编码图像帧组的I帧发送给基站的结束时刻作为第一时间的终点时刻(T0+15ms)。如图4所示，在5G网络中的一个时间周期为5ms，一个时间周期包括3个下行帧D、1个上行帧U以及1个特殊子帧S，每一个下行帧D的时间长度均为1ms，每一个上行帧U的时间长度均为1ms，每一个特殊子帧S的时间长度均为1ms。第一终端的缓存清空时间(第一时间)T2为14ms，T2大于第一终端设置的T1，其中，T1为11ms，则第一终端启动I帧上行传输调整过程；第二时间T3为9ms，第三时间T4为5ms，则第一终端的重置的GoP周期的I帧发送起始时间为T0+6ms，当GoP周期时长为500ms，第一终端的下个GoP周期中I帧(第二编码图像帧组中的I帧)发送时刻为T0+506ms，即第一终端的第二编码图像帧组对应的起始时刻为T0+506ms。第一终端和第二终端在后续GoP周期的编码图像帧组中的I帧上行传输中避开了I帧碰撞。
本申请实施例中，在5G专网中实时远程控制的场景，该场景中会有多个行业设备(终端)需要进行远程控制。通常行业设备会有摄像头等视频采集设备实时采集视频，视频被编码后，会通过5G专用网络(例如基站)进行回传。操控端(例如控制平台)接收到5G专用网络回传视频后，操控端会根据视频情况，对行业设备进行远程控制。数据上行传输主要是视频流的回传，视频流数据具有高度的周期性规律，因此本申请实施例提供的视频流的处理方法可以避免多个终端之间的I帧上行传输碰撞，从而降低了视频传输的时延和时延抖动。当5G专网中存在其他大带宽的上行数据业务，可在网络配置中，将视频流终端的调度优先级设为最高，该本申请实施例提供的种视频流的处理方法仍可有效工作。
在一个实施例中，多个终端在恶劣的环境下采集土壤标本，终端可以是人工智能设备，例如机器人。多个机器人通过自带的摄像头采集周围环境的视频流，多个机器人分别将各自采集到的视频流中编码图像帧组的I帧存入到各自的缓存，多个机器人中的1号机器人确定第一编码图像帧组对应的第一时间和第二时间，第一时间为从第一编码图像帧组的I帧存入第一终端的缓存、直至将缓存中的第一编码图像帧组的I帧发送给基站的时间段，第二时间为网络请求保护时间；当第一时间大于预设的时间阈值且满足预设条件，则表明1号机器人发送第一编码图像帧组的I帧时，1号机器人在第一编码图像帧组的I帧的上行传输中与其他机器人发生冲突；则1号机器人会根据第一时间和第二时间，确定视频流中的第二编码图像帧组对应的起始时刻，第二编码图像帧组是在第一编码图像帧组之后，第二编码图像帧组对应的起始时刻为第二编码图像帧组的I帧存入缓存的开始时间点；基于第二编码图像帧组对应的起始时刻，1号机器人将第二编码图像帧组的I帧存入缓存后，将存入缓存中的第二编码图像帧组的I帧发送给基站，此时，基站对支持编码图像帧组进行上行传输的资源进行分配，1号机器人从而避免与其他机器人在I帧的上行传输中发生冲突。多个机器人分别将视频流发送给同一个基站，该基站通过网络将视频流回传到远端的控制平台。控制平台根据视频流，远程分别对多个机器人进行控制，可以使机器人避开障碍物、快速搜寻到目标土壤等。多个机器人中的每个机器人可以分布式地自行工作，每个机器人之间可以不需要相互协调和进行信息交互。
参见图5，图5示出了本申请实施例提供的另一种视频流的处理方法的流程示意图，其中，该方法可以由任一电子设备执行，如可以是第一终端，作为一可选实施方式，将以第一终端作为该方法执行主体为例进行说明。如图5所示，本申请实施例提供的视频流的处理方法包括如下步骤：
步骤S201，第一终端通过摄像头采集视频流。
在一个实施例中，第一终端可以为无人驾驶车辆，视频流中的内容 与无人驾驶车辆正在行驶的路面有关。
步骤S202，第一终端得到视频流中的第一编码图像帧组。
步骤S203，第一终端确定第一编码图像帧组的第一时间和第二时间。
在一个实施例中，第一时间为，从第一终端存储第一编码图像帧组的I帧、直至将该I帧发送给网络节点的时间段，第二时间为网络请求保护时间。
步骤S204，当第一时间大于预设的时间阈值且满足预设条件，则第一终端根据第一时间和第二时间，确定视频流中的第二编码图像帧组的起始时刻。
在一个实施例中，第二编码图像帧组是在第一编码图像帧组之后接收到的，第二编码图像帧组的起始时刻为将第二编码图像帧组的I帧存入第一终端的缓存的开始时间点。
在一个实施例中，当第一时间大于预设的时间阈值、且预设条件得到满足时，则表明第一终端在第一编码图像帧组的I帧的上行传输中，与至少一个第二终端发生冲突。
这里，预设条件所包括的内容可参见前述实施例中的说明，此处不再赘述。
步骤S205，基于第二编码图像帧组的起始时刻，第一终端将第二编码图像帧组的I帧存入缓存后，将存入缓存中的第二编码图像帧组的I帧发送给基站。
在一个实施例中，第一终端将视频流以及视频流包括的存入缓存中的第二编码图像帧组的I帧发送给基站，可以避免与多个第二终端在I帧的上行传输中发生冲突。
步骤S206，基站将视频流回传到控制平台。
步骤S207，控制平台根据视频流，远程对第一终端进行控制。
在一个实施例中，控制平台根据无人驾驶车辆正在行驶的路面的有关视频流，对无人驾驶车辆进行远程控制，从而加强了无人驾驶车辆行驶的安全性。
本申请实施例中，实现了减少5G专网场景下多终端视频流I帧传输的碰撞，缓解视频流传输的时延和时延抖动。通过利用对网络传输缓存的感知和基站资源调度的特性，5G专网下多个终端可以在不需要协商的情况下，刷新各终端的I帧上行传输的时间位置，避免了多个终端之间I帧上行传输的碰撞。
继续对本申请实施例提供的视频流的处理方法进行说明，在一些实施例中，本申请实施例提供的视频流的处理方法包括如下步骤：
S301，第一终端确定第一时间T2。
在一个实施例中，当第一终端调用5G模组/芯片的发送数据接口发送I帧数据时，每隔1ms周期读取一次网络缓存大小，当网络缓存大小 减少到0时，记录经过的时间周期数为T2，单位为ms。
S302，根据第一时间T2和第二时间T3，确定第三时间T4。
在一个实施例中，通过以下两种方式中的任意一种确定T4:
方式1：计算第一时间T2和第二时间T3之间的差值，得到第三时间T4，即T4＝T2–T3；其中，T3为配置参数。例如，T2可以取值为14ms，T3可以取值为9ms，则T4取值为5ms。
方式2：根据视频流的帧周期T7、第一时间T2和第二时间T3，确定第三时间T4；例如，T4＝m×T7，其中，m是使得m×T7–T2+T3为最小非负整数n。
举例说明，T7可以取值为16.6ms，T2可以取值为14ms，T3可以取值为9ms，则m×T7–T2+T3＝16.6m-5；当m为1时，使得16.6m-5为最小非负整数12，即n为12；T4＝m×T7＝1×T7＝16.6ms。
S303，基于第三时间T4，确定重置的GoP周期的起始时刻。
在一个实施例中，重置的GoP周期的起始时刻可以为视频流中的第二编码图像帧组的起始时刻。
S304，第一终端将重置的GoP周期的起始时刻通知视频接收端。
在一个实施例中，视频接收端可以为基站。重置的GoP周期的起始时刻可以为视频流中的第二编码图像帧组对应的起始时刻，例如，频流中的第二编码图像帧组对应的起始时刻为T0+506ms。
S305，基于重置的GoP周期的起始时刻，进行后续的视频帧编码和传输。
在一个实施例中，重置的GoP周期的起始时刻可以为视频流中的第二编码图像帧组对应的起始时刻。
在一个实施例中，重置输入视频流，使得视频流的I帧的到达延后T4，例如T4＝5ms。操作为断开视频流，等待T4视频流重建时间，重新建立视频流；按照GoP帧序号，如果是首帧，则输出I帧编码标识，对当前帧进行I帧编码，得到I帧；编码后判断当前时刻，是否晚于重置的I帧起始时刻+n×GoP周期，即视频流中的第二编码图像帧组对应的起始时刻。例如，重置的I帧起始时刻+n×GoP周期＝(T0+6)+1×500＝T0+506ms，其中，n为1，n为当前的GoP周期编号；如果当前时刻是晚于重置的I帧起始时刻+n×GoP周期，则输出I帧发送标识，进行I帧数据发送。正常情况：当前时刻(例如，T0+505ms)早于重置的I帧起始时刻+n×GoP周期，例如，重置的I帧起始时刻+n×GoP周期＝T0+506ms。异常情况：当前时刻(例如，T0+507ms)晚于重置的I帧起始时刻+n×GoP周期，例如，重置的I帧起始时刻+n×GoP周期＝T0+506ms。
在一个实施例中，在没有断开视频流的前提下，对于当前帧到达时刻t2，如果t2位于区间[重置的I帧起始时刻+n×GoP周期–I帧编码时间， 重置的I帧起始时刻+n×GoP周期–I帧编码时间+帧周期]，则输出I帧编码标识，对当前帧进行I帧编码，得到I帧；编码后判断当前时刻是否晚于重置的I帧起始时刻+n×GoP周期，n为当前的GoP周期编号，如果当前时刻是晚于重置的I帧起始时刻+n×GoP周期，则输出I帧发送标识，进行I帧数据发送。
基于相同的发明构思，本申请实施例还提供了一种视频流的处理装置，该装置可设置于第一终端，该装置的结构示意图如图6所示，视频流的处理装置40，包括第一处理模块401、第二处理模块402、第三处理模块403和第四处理模块404。
第一处理模块401，配置为获取视频流中的第一编码图像帧组；
第二处理模块402，配置为确定第一编码图像帧组对应的第一时间和第二时间，第一时间为，从存储第一编码图像帧组的I帧、直至将所存入的I帧发送给网络节点的时间段，第二时间为网络请求保护时间；
第三处理模块403，配置为当向所述网络节点传输所述I帧的过程中与至少一个第二终端发生冲突时，根据第一时间和第二时间，确定视频流中至少一个第二编码图像帧组的起始时刻，第二编码图像帧组在第一编码图像帧组之后；
第四处理模块404，配置为基于第二编码图像帧组的起始时刻，向所述网络节点传输第二编码图像帧组的I帧。
在一个实施例中，第二处理模块402，还配置为：
根据获取的网络调度请求时间和预设的保护时间，确定第二时间。
在一个实施例中，第三处理模块403，还配置为：
根据第一时间的起始时刻、第三时间和预设的编码图像帧组周期，确定第二编码图像帧组的起始时刻，第一时间的起始时刻为，第一编码图像帧组的I帧存入第一终端的开始时间点；
其中，第三时间由第一时间和第二时间确定，第三时间为I帧发送调整时间。
在一个实施例中，第三处理模块403，还配置为：
根据视频流的帧周期、第一时间和第二时间，确定第三时间，第三时间为所述帧周期的整数倍；
在一个实施例中，第三处理模块403，还配置为：
计算第一时间和第二时间之间的差值，得到第一数值；
根据视频流的帧周期的m倍和第一数值，得到第二数值；
当第二数值为最小非负整数，则确定第三时间为视频流的帧周期的m倍，其中m为正整数。
在一个实施例中，第四处理模块404，还配置为：
当在第二编码图像帧组的起始时刻之后，得到第二编码图像帧组的I帧，则将第二编码图像帧组的I帧存入第一终端，并将第二编码图像帧组 的I帧发送给网络节点。
在一个实施例中，第四处理模块404，还配置为：
重置视频流或调整第一编码图像帧组长度，并在第二编码图像帧组对应的起始时刻得到第二编码图像帧组的I帧，将得到的第二编码图像帧组的I帧存入第一终端。
在一个实施例中，预设的时间阈值由网络调度请求时间和I帧对应的上行帧等待时间中的至少一项来确定。
在一个实施例中，预设条件包括以下至少一项：
第一编码图像帧组的I帧上行传输为，视频流的首次I帧上行传输；
视频流中连续多个第三编码图像帧组的第四时间均大于时间阈值，第四时间为，所述第三编码图像帧组的I帧的上行传输时长，即第三编码图像帧组的I帧被存入第一终端、直至将所存入的第三编码图像帧组的I帧发送给网络节点的时间段，多个第三编码图像帧组在第一编码图像帧组之前；
第一时间大于时间阈值与视频流的帧周期的和。
应用本申请实施例，至少具有如下有益效果：
当第一终端在第一编码图像帧组的I帧的上行传输中与至少一个第二终端发生冲突时；，根据第一时间和第二时间，确定视频流中在第一编码图像帧组之后的至少一个第二编码图像帧组的起始时刻，并基于确定的第二编码图像帧组的起始时刻，向所述网络节点传输第二编码图像帧组的I帧，如此，由于第一时间是从存储第一编码图像帧组的I帧至将I帧发送给网络节点的时间段，第二时间是网络请求保护时间，使得即使第一终端被基站调度到后面的时间周期进行I帧的传输，由于第二编码图像帧组是根据第一时间和第二时间确定的，保证了第一编码图像帧组后面的至少一个第二编码图像帧组的起始时刻，能够随第一时间的变化而动态变化，进而避免第一终端向所述网络节点传输第二编码图像帧组的I帧的过程中与第二终端发生冲突，降低了视频流传输的时延，并缓解了时延抖动。
基于相同的发明构思，本申请实施例还提供了一种电子设备，该电子设备的结构示意图如图7所示，该电子设备9000包括至少一个处理器9001、存储器9002和总线9003，至少一个处理器9001均与存储器9002电连接；存储器9002被配置用于存储有至少一个计算机可执行指令，处理器9001被配置用于执行该至少一个计算机可执行指令，从而执行如本申请实施例提供的视频流的处理方法。
在一些实施例中，处理器9001可以是现场可编程门阵列(Field－Programmable Gate Array，FPGA)或者其它具有逻辑处理能力的器件，如微控制单元(Microcontroller Unit，MCU)、中央处理器(Central Process Unit，CPU)。
基于相同的发明构思，本申请实施例还提供了一种计算机可读存储介质，存储有计算机程序，该计算机程序用于被处理器执行时实现本申请实施例提供的视频流的处理方法的步骤。
本申请实施例还提供了一种计算机程序产品，包括计算机程序或指令，所述计算机程序或指令被处理器执行时，实现本申请实施例提供的视频流的处理方法。
本申请实施例提供的计算机可读存储介质包括但不限于任何类型的盘(包括软盘、硬盘、光盘、CD-ROM、和磁光盘)、只读存储器(Read-Only Memory，ROM)、随即存储器(Random Access Memory，RAM)、可擦写可编程只读存储器(Erasable Programmable Read-Only Memory，EPROM)、电可擦可编程只读存储器(Electrically Erasable Programmable Read-Only Memory，EEPROM)、闪存、磁性卡片或光线卡片。也就是，可读存储介质包括由设备(例如，计算机)以能够读的形式存储或传输信息的任何介质。
本申请实施例还提供了一种包含指令的计算机程序产品，当其在计算机设备上运行时，使得计算机设备执行本申请实施例所提供的视频流的处理方法。
本技术领域技术人员可以理解，可以用计算机程序来实现这些结构图和/或框图和/或流图中的每个框以及这些结构图和/或框图和/或流图中的框的组合。本技术领域技术人员可以理解，可以将这些计算机程序产品提供给通用计算机、专业计算机或其他可编程数据处理方法的处理器来实现，从而通过计算机或其他可编程数据处理方法的处理器来执行本申请公开的结构图和/或框图和/或流图的框或多个框中指定的方案。
以上所述仅是本申请的部分实施方式，应当指出，对于本技术领域的普通技术人员来说，在不脱离本申请原理的前提下，还可以做出若干改进和润饰，这些改进和润饰也应视为本申请的保护范围。

Claims (14)

  1. 一种视频流的处理方法，由第一终端执行，所述方法包括：
    获取视频流中的第一编码图像帧组；
    确定所述第一编码图像帧组对应的第一时间和第二时间，所述第一时间为，从存储所述第一编码图像帧组的I帧至将所述I帧发送给网络节点的时间段，所述第二时间为网络请求保护时间；
    当向所述网络节点传输所述I帧的过程中与至少一个第二终端发生冲突时，根据所述第一时间和所述第二时间，确定所述视频流中至少一个第二编码图像帧组的起始时刻，所述第二编码图像帧组在所述第一编码图像帧组之后；
    基于所述第二编码图像帧组的起始时刻，向所述网络节点传输所述第二编码图像帧组的I帧。
  2. 根据权利要求1所述的方法，其中，所述确定所述第一编码图像帧组对应的第二时间，包括：
    根据获取的网络调度请求时间和预设的保护时间，确定所述第二时间。
  3. 根据权利要求1所述的方法，其中，所述根据所述第一时间和所述第二时间，确定所述视频流中至少一个第二编码图像帧组的起始时刻，包括：
    根据所述第一时间的起始时刻、第三时间和预设的编码图像帧组周期，确定所述第二编码图像帧组的起始时刻，所述第一时间的起始时刻为，所述第一编码图像帧组的I帧存入所述第一终端的时间点；
    其中，所述第三时间为I帧发送调整时间。
  4. 根据权利要求3所述的方法，其中，所述方法还包括：
    根据所述视频流的帧周期、所述第一时间和所述第二时间，确定第 三时间；
    其中，所述第三时间为所述帧周期的整数倍。
  5. 根据权利要求4所述的方法，其中，所述根据所述视频流的帧周期、所述第一时间和所述第二时间，确定第三时间，包括：
    计算所述第一时间和所述第二时间之间的差值，得到第一数值；
    根据所述视频流的帧周期的m倍和所述第一数值，得到第二数值；
    当所述第二数值为最小非负整数，则将所述视频流的帧周期的m倍确定为所述第三时间，其中m为正整数。
  6. 根据权利要求1所述的方法，其中，所述基于所述第二编码图像帧组的起始时刻，向所述网络节点传输所述第二编码图像帧组的I帧，包括：
    在所述第二编码图像帧组的起始时刻之后，得到所述第二编码图像帧组的I帧，则将所述第二编码图像帧组的I帧存入所述第一终端，并将所述第二编码图像帧组的I帧发送给所述网络节点。
  7. 根据权利要求1所述的方法，其中，所述向所述网络节点传输所述第二编码图像帧组的I帧之前，所述方法还包括：
    重置视频流或调整所述第一编码图像帧组长度；
    在所述第二编码图像帧组对应的起始时刻，得到所述第二编码图像帧组的I帧；
    将得到的所述第二编码图像帧组的I帧存入所述第一终端。
  8. 根据权利要求2所述的方法，其中，所述预设的时间阈值由以下至少之一决定：
    所述网络调度请求时间和I帧对应的上行帧等待时间。
  9. 根据权利要求1所述的方法，其中，所述方法还包括：
    当所述第一时间大于预设时间阈值，且预设条件得到满足时，确定 向所述网络节点传输所述I帧的过程中与至少一个第二终端发生冲突。
  10. 根据权利要求9所述的方法，其中，所述预设条件包括以下至少一项：
    所述第一编码图像帧组的I帧上行传输为，所述视频流的首次I帧上行传输；
    所述视频流中连续多个第三编码图像帧组的第四时间均大于所述时间阈值，所述第四时间为，所述第三编码图像帧组的I帧的上行传输时长，所述多个第三编码图像帧组在所述第一编码图像帧组之前；
    所述第一时间大于所述时间阈值与所述视频流的帧周期的和。
  11. 一种视频流的处理装置，所述装置包括：
    第一处理模块，配置为获取视频流中的第一编码图像帧组；
    第二处理模块，配置为确定所述第一编码图像帧组对应的第一时间和第二时间，所述第一时间为，从存储所述第一编码图像帧组的I帧、直至将所述I帧发送给网络节点的时间段，所述第二时间为网络请求保护时间；
    第三处理模块，配置为当向所述网络节点传输所述I帧的过程中与至少一个第二终端发生冲突时，则根据所述第一时间和所述第二时间，确定所述视频流中至少一个第二编码图像帧组的起始时刻，所述第二编码图像帧组在所述第一编码图像帧组之后；
    第四处理模块，配置为基于所述第二编码图像帧组的起始时刻，向所述网络节点传输所述第二编码图像帧组的I帧。
  12. 一种电子设备，所述电子设备包括：处理器及存储器；其中，
    所述存储器，配置为存储计算机程序；
    所述处理器，配置为通过调用所述计算机程序，执行如权利要求1-10中任一项所述的视频流的处理方法。
  13. 一种计算机可读存储介质，存储有计算机程序，所述计算机程序配置为被处理器执行时，实现如权利要求1-10中任一项所述的视频流的处理方法。
  14. 一种计算机程序产品，包括计算机程序或指令，所述计算机程序或指令被处理器执行时，实现权利要求1至10任一项所述的视频流的处理方法。
PCT/CN2022/097539 2021-07-15 2022-06-08 视频流的处理方法、装置、设备、存储介质及程序产品 WO2023284445A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US18/140,490 US20230262273A1 (en) 2021-07-15 2023-04-27 Video stream processing method, apparatus, device, storage medium, and program product

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202110802637.2 2021-07-15
CN202110802637.2A CN113259717B (zh) 2021-07-15 2021-07-15 视频流的处理方法、装置、设备及计算机可读存储介质

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US18/140,490 Continuation US20230262273A1 (en) 2021-07-15 2023-04-27 Video stream processing method, apparatus, device, storage medium, and program product

Publications (1)

Publication Number Publication Date
WO2023284445A1 true WO2023284445A1 (zh) 2023-01-19

Family

ID=77180413

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/097539 WO2023284445A1 (zh) 2021-07-15 2022-06-08 视频流的处理方法、装置、设备、存储介质及程序产品

Country Status (3)

Country Link
US (1) US20230262273A1 (zh)
CN (1) CN113259717B (zh)
WO (1) WO2023284445A1 (zh)

Families Citing this family (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113259717B (zh) * 2021-07-15 2021-09-14 腾讯科技（深圳）有限公司 视频流的处理方法、装置、设备及计算机可读存储介质
CN115834894A (zh) * 2021-09-15 2023-03-21 中兴通讯股份有限公司 数据传输方法、接收端、数据源和计算机可读存储介质
CN114125341B (zh) * 2021-10-29 2024-01-23 北京达佳互联信息技术有限公司 视频处理方法、装置、电子设备、存储介质及产品
CN114390291B (zh) * 2022-01-17 2023-12-22 广州市奥威亚电子科技有限公司 视频编码流传输方法及装置
CN116634144A (zh) * 2022-02-10 2023-08-22 华为技术有限公司 视频传输方法及装置
CN116939286A (zh) * 2022-03-31 2023-10-24 中兴通讯股份有限公司 视频数据的处理方法、装置、边缘计算网关及存储介质
CN115334307B (zh) * 2022-10-11 2023-02-10 浙江大华技术股份有限公司 一种数据传输方法、前端设备、视频采集系统和介质

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20140177733A1 (en) * 2012-12-20 2014-06-26 Hulu, LLC Keyframe Alignment for Encoding Video at Multiple Bitrates
CN104735470A (zh) * 2015-02-11 2015-06-24 海信集团有限公司 一种流媒体数据传输方法及装置
CN104754339A (zh) * 2013-12-26 2015-07-01 成都鼎桥通信技术有限公司 I帧调整方法、装置及系统
CN112203100A (zh) * 2020-09-03 2021-01-08 中国移动通信集团广东有限公司 降低上下行带宽需求的传输方法及系统
CN113068001A (zh) * 2019-12-16 2021-07-02 浙江宇视科技有限公司 基于级联摄像机的数据处理方法、装置、设备和介质
CN113259717A (zh) * 2021-07-15 2021-08-13 腾讯科技（深圳）有限公司 视频流的处理方法、装置、设备及计算机可读存储介质

Family Cites Families (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8134558B1 (en) * 2007-12-06 2012-03-13 Adobe Systems Incorporated Systems and methods for editing of a computer-generated animation across a plurality of keyframe pairs
CN102209237B (zh) * 2011-05-26 2013-05-29 浙江宇视科技有限公司 降低点播实况媒体流时i帧重叠的方法及视频管理服务器
US20170180764A1 (en) * 2014-04-03 2017-06-22 Carrier Corporation Time lapse recording video systems
US20150312651A1 (en) * 2014-04-28 2015-10-29 Honeywell International Inc. System and method of optimized network traffic in video surveillance system
EP3214839B1 (en) * 2014-11-28 2020-01-01 Huawei Technologies Co. Ltd. Method and device for transmitting multiple video streams
CN107251564B (zh) * 2015-03-05 2018-08-28 三菱电机株式会社 数据处理装置及数据处理方法
KR101827694B1 (ko) * 2016-11-07 2018-02-09 주식회사 아이디스 이벤트 스트림 지연에 강인한 멀티스트림 녹화 기능을 구비한 네트워크 영상 기록 장치
CN111741276B (zh) * 2020-08-28 2020-12-04 浙江大华技术股份有限公司 一种视频设备发送关键帧的方法、视频设备和存储装置
CN111787292B (zh) * 2020-09-04 2020-12-04 浙江大华技术股份有限公司 基于多个网络摄像机的数据传输方法及相关装置
CN112399141B (zh) * 2020-10-16 2022-03-25 浙江大华技术股份有限公司 基于多个前端视频设备的数据传输方法及相关装置

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20140177733A1 (en) * 2012-12-20 2014-06-26 Hulu, LLC Keyframe Alignment for Encoding Video at Multiple Bitrates
CN104754339A (zh) * 2013-12-26 2015-07-01 成都鼎桥通信技术有限公司 I帧调整方法、装置及系统
CN104735470A (zh) * 2015-02-11 2015-06-24 海信集团有限公司 一种流媒体数据传输方法及装置
CN113068001A (zh) * 2019-12-16 2021-07-02 浙江宇视科技有限公司 基于级联摄像机的数据处理方法、装置、设备和介质
CN112203100A (zh) * 2020-09-03 2021-01-08 中国移动通信集团广东有限公司 降低上下行带宽需求的传输方法及系统
CN113259717A (zh) * 2021-07-15 2021-08-13 腾讯科技（深圳）有限公司 视频流的处理方法、装置、设备及计算机可读存储介质

Also Published As

Publication number Publication date
US20230262273A1 (en) 2023-08-17
CN113259717A (zh) 2021-08-13
CN113259717B (zh) 2021-09-14

Similar Documents

Publication Publication Date Title
WO2023284445A1 (zh) 视频流的处理方法、装置、设备、存储介质及程序产品
Xie et al. Poi360: Panoramic mobile video telephony over lte cellular networks
US9042444B2 (en) System and method for transmission of data signals over a wireless network
US8629893B2 (en) Video switching without instantaneous decoder refresh-frames
CN104427286B (zh) 一种进行视频通话的方法和系统
CN104768081A (zh) 一种实现流量控制的丢包重传方法
CN103780907A (zh) 一种视频数据流量整形的方法和装置
JP2014241493A (ja) 送信装置、送信方法及びプログラム
CN102946361A (zh) 基于交换机缓存分配的流量控制方法及系统
WO2023045551A1 (zh) 摄像头控制方法、系统、电子设备和存储介质
CN108924485B (zh) 客户端实时视频流中断处理方法及系统、监控系统
WO2023130838A1 (zh) 数据传输方法、相关设备、计算机可读存储介质及计算机程序产品
WO2023036258A1 (zh) 基于无人机的视频直播方法、系统、设备及存储介质
CN113068001A (zh) 基于级联摄像机的数据处理方法、装置、设备和介质
CN102138336B (zh) 动态图像数据的分配方法
CN103269433A (zh) 视频数据传输方法和视频数据传输系统
CN108605114A (zh) 控制方法、装置及网络控制器
WO2022160728A1 (zh) 业务流错峰调度方法及相关设备
CN112203050B (zh) 一种视频续传的方法及装置
WO2024021999A1 (zh) 同步方法、系统及电子设备
CN111629278B (zh) 图像传输方法、装置、可移动平台、服务器及存储介质
CN111432384A (zh) 用于具有录音功能设备的大数据量音频蓝牙实时传输方法
Bankov et al. Approach to real-time communications in Wi-Fi networks
CN112261354B (zh) 基于多个网络摄像机的数据传输方法及相关装置
CN103533056A (zh) 一种面向智能手机的p2p移动流媒体系统及其工作方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22841085

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 21/05/2024)