WO2023036258A1 - 基于无人机的视频直播方法、系统、设备及存储介质 - Google Patents

基于无人机的视频直播方法、系统、设备及存储介质 Download PDF

Info

Publication number
WO2023036258A1
WO2023036258A1 PCT/CN2022/117900 CN2022117900W WO2023036258A1 WO 2023036258 A1 WO2023036258 A1 WO 2023036258A1 CN 2022117900 W CN2022117900 W CN 2022117900W WO 2023036258 A1 WO2023036258 A1 WO 2023036258A1
Authority
WO
WIPO (PCT)
Prior art keywords
data
video stream
video
webrtc
unmanned aerial
Prior art date
Application number
PCT/CN2022/117900
Other languages
English (en)
French (fr)
Inventor
冯银华
冷杰
Original Assignee
深圳市道通智能航空技术股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳市道通智能航空技术股份有限公司 filed Critical 深圳市道通智能航空技术股份有限公司
Publication of WO2023036258A1 publication Critical patent/WO2023036258A1/zh

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/21—Server components or server architectures
    • H04N21/218—Source of audio or video content, e.g. local disk arrays
    • H04N21/2187—Live feed
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/02—Protocols based on web technology, e.g. hypertext transfer protocol [HTTP]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/10—Protocols in which an application is distributed across nodes in the network
    • H04L67/104—Peer-to-peer [P2P] networks
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/231—Content storage operation, e.g. caching movies for short term storage, replicating data over plural servers, prioritizing data for deletion
    • H04N21/23106—Content storage operation, e.g. caching movies for short term storage, replicating data over plural servers, prioritizing data for deletion involving caching operations
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
    • H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
    • H04N21/234363—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements by altering the spatial resolution, e.g. for clients with a lower screen resolution
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/44004—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving video buffer management, e.g. video decoder buffer or video display buffer

Definitions

  • the present invention relates to the technical field of UAV data transmission, in particular to a UAV-based video live broadcast method, system, device and storage medium.
  • the present invention provides a UAV-based video live broadcast method, system, device and storage medium to perform UAV-based video live broadcast with the help of the WebRtc protocol, which reduces live broadcast delay and avoids stuck situations.
  • the present invention provides a method for live video broadcasting based on a drone, the method comprising:
  • the video frame is sent to the preset server through the WebRtc transmission channel, so that the preset server performs live display according to the video frame.
  • the original video stream data includes H264 data
  • the encoding and decoding of the original video stream data to obtain output buffer data includes:
  • the codec codec s the H264 data to generate ByteBuffer video stream data and writes it into the codec output buffer area.
  • the converting the output buffer data into video frames supported by the WebRtc protocol includes:
  • VideoFrame data Perform decoding and data conversion based on the ByteBuffer video stream data to obtain VideoFrame data.
  • the preset server before sending the video frame to the preset server through the WebRtc transmission channel, it also includes:
  • determining the current resolution according to the first rule according to the network signal strength includes:
  • the current resolution is 1080x720p.
  • the drone before obtaining the raw video stream data sent by the drone, it also includes:
  • the WebRtc transmission channel includes an RTCPeerConnection channel and/or an RTCDataChannel channel.
  • the present invention provides a two-dimensional code-based page guidance device, including:
  • the data receiving module is used to obtain the original video stream data sent by the drone;
  • a codec module configured to codec the original video stream data to obtain output buffer data
  • a data conversion module configured to convert the output buffer data into video frames supported by the WebRtc protocol
  • the WebRtc transmission module is configured to send the video frame to a preset server through a WebRtc transmission channel, so that the preset server performs live display according to the video frame.
  • the present invention provides an electronic device, including a memory and a processor, the memory stores a computer program that can run on the processor, and when the processor executes the computer program, any one of the present invention is implemented.
  • the UAV-based video live broadcast method provided in the embodiment.
  • the present invention provides a computer-readable storage medium, the storage medium stores a computer program, the computer program includes program instructions, and when the program instructions are executed, the program instructions provided in any embodiment of the present invention are implemented.
  • UAV-based video live broadcast method UAV-based video live broadcast method.
  • the UAV-based video live broadcast method provided by the embodiment of the present invention is applied to a terminal that maintains communication with the UAV.
  • the original video stream data sent by the UAV is obtained, and then the original video stream
  • the data is encoded and decoded to obtain the output buffer data, and the output buffer data is converted to the video frame supported by the WebRtc protocol, and finally the video frame is sent to the preset server through the WebRtc transmission channel, and the preset server performs live display according to the video frame.
  • FIG. 1 is a flow chart of a UAV-based video live broadcast method provided by Embodiment 1 of the present invention
  • Fig. 2 is a sub-flowchart of the UAV-based video live broadcast method provided by Embodiment 1 of the present invention
  • Fig. 3 is a sub-flowchart of the UAV-based video live broadcast method provided by Embodiment 1 of the present invention.
  • Fig. 4 is a flow chart of the UAV-based video live broadcast method provided by Embodiment 2 of the present invention.
  • Fig. 5 is a sub-flowchart of the UAV-based video live broadcast method provided by Embodiment 2 of the present invention.
  • Fig. 6 is a flow chart of the UAV-based video live broadcast method provided by Embodiment 2 of the present invention.
  • FIG. 7 is a schematic structural diagram of a UAV-based video live broadcast system provided by Embodiment 3 of the present invention.
  • FIG. 8 is a schematic structural diagram of an electronic device provided by Embodiment 4 of the present invention.
  • first”, “second”, etc. may be used herein to describe various directions, actions, steps or elements, etc., but these directions, actions, steps or elements are not limited by these terms. These terms are only used to distinguish a first direction, action, step or element from another direction, action, step or element.
  • a first use case could be termed a second use case, and, similarly, a second use case could be termed a first use case, without departing from the scope of the present invention.
  • Both the first use case and the second use case are use cases, but they are not the same use case.
  • the terms “first”, “second”, etc. should not be interpreted as indicating or implying relative importance or implying the number of indicated technical features.
  • features defined as “first” and “second” may explicitly or implicitly include a combination of one or more features.
  • “plurality” means at least two, such as two, three, etc., unless otherwise specifically defined. It should be noted that when a part is said to be “fixed” to another part, it can be directly on the other part or there can be an intermediate part. When a section is said to be “connected” to another section, it may be directly connected to the other section or there may be intervening sections at the same time.
  • the terms “vertical”, “horizontal”, “left”, “right” and similar expressions are for the purpose of illustration only and do not represent the only embodiment.
  • this embodiment provides a method for live video broadcasting based on drones, which can be applied to drone systems, the system includes drones, terminals and preset servers, wherein: the drone is based on Remote control equipment or self-contained program control device control, unmanned aircraft with mission load; the terminal is an electronic device connected to the remote control equipment, which maintains communication with the drone through the remote control equipment, and is used between the drone and the preset server Data transmission relay, usually using mobile phones, tablets and other devices; the preset server maintains a communication connection with the terminal.
  • the UAV-based video live broadcast method provided in this embodiment can be specifically executed by a UAV or a terminal, or can be completed by interaction between a terminal and a UAV.
  • a terminal is taken as an example for specific description. As shown in Figure 1, the method includes the following steps:
  • the UAV is provided with a flight control module and an image acquisition module, wherein the flight control module is used to monitor and control the flight and tasks of the UAV, and the image acquisition module is used to perform image capture during the flight of the UAV. Acquisition, the original video stream data is the data collected by the image acquisition module and transmitted to the terminal.
  • the image acquisition module is collected and connected to the remote control device through a dedicated communication link (usually a radio communication link established by the antenna of the remote control device and the drone), and the data collected by the image acquisition module is first transmitted by wireless
  • the man-machine is transmitted to the remote control device, and then transmitted to the terminal by the remote control device.
  • the original video stream data finally obtained by the terminal is usually H264 data.
  • the original video stream data can also use other data formats, which are not limited here .
  • the output buffer data is a special encapsulation of the original video stream data by the multimedia codec, so that it can be further converted into special data that can be transmitted through the WebRtc transmission channel (that is, the video frame supported by the WebRtc protocol referred to below).
  • the WebRtc protocol is used as the data transmission protocol between the terminal and the preset server. It can make real-time communication a standard function, and there will be no problem of live broadcast freezing due to a problem with one message causing all messages to go wrong.
  • the WebRtc protocol directly processes and transmits the pictures collected by the camera when it is applied, but in this embodiment, the pictures collected by the drone need to be transmitted to the terminal through a dedicated communication link, and the terminal obtains the The original video stream data cannot be directly transmitted through the WebRtc protocol. Therefore, in this embodiment, the data in the original video needs to be encoded and decoded to obtain data conforming to the WebRtc protocol. Specifically, the encoding and decoding of the original video stream data in step S120 is actually converting it into high-efficiency processing byte encapsulation data, that is, output buffer data, because the underlying video stream data is processed on the original video stream data.
  • the terminal when the terminal is a mobile phone or tablet device using Android, taking the original video stream data including H264 data as an example, the encoding and decoding process is specifically shown in Figure 2, including steps S121-122:
  • the codec adopts the codec provided by MediaCodec.
  • MediaCodec is a part of the Android low-level multimedia architecture. It is usually used in conjunction with MediaExtractor, MediaMuxer, and AudioTrack, and can codec such as H.264, H.265, AAC, 3gp, etc. Common audio and video formats.
  • the codec encodes and decodes the H264 data to generate ByteBuffer video stream data, and writes it into the codec output buffer area.
  • MediaCodec uses a set of input/output buffers to process data synchronously or asynchronously in the process of encoding and decoding: first, the terminal writes the data to be encoded and decoded (that is, the original Video stream data) and submit it to the codec, generate ByteBuffer video stream data after the codec is processed, transfer the ByteBuffer video stream data to the encoding output buffer area, and take back the client's ownership of the input buffer area at the same time; Then, the terminal reads the coded data from the acquired codec output buffer area for processing (not the content in step S122), and after the processing is completed, the codec takes back the client's ownership of the output buffer area.
  • the terminal writes the data to be encoded and decoded (that is, the original Video stream data) and submit it to the codec, generate ByteBuffer video stream data after the codec is processed, transfer the ByteBuffer video stream data to the encoding output buffer area, and take back the client's ownership of the input
  • the video frame is a VideoFrame data object supported by the WebRtc protocol, which can be pushed through the streaming interface provided by WebRtc.
  • the output buffer data obtained in step S120 can only realize efficient data processing, and still cannot be directly transmitted through the WebRtc protocol, so it is further converted through step S130: the terminal continuously reads the output buffer data from the codec output buffer area for Convert to obtain video frames that can be transmitted through the WebRtc protocol.
  • the conversion process of outputting buffered data is also step S130, as shown in FIG. 3 , including steps S131-132:
  • the ByteBuffer video stream data is directly written into the codec output buffer area after generation.
  • the terminal continuously reads the ByteBuffer video stream data from the codec output buffer area, and sets the video stream.
  • the information such as the width, height and time stamp of each frame of image is decoded, and encapsulated to form VideoFrame data.
  • S140 Send the video frame to a preset server through a WebRtc transmission channel, so that the preset server performs live display according to the video frame.
  • the WebRtc transmission channel is the data path between two devices connected by the streaming interface provided by WebRtc.
  • the streaming interface provided by WebRtc connects the terminal and the preset server.
  • the WebRtc transmission channel is actually realized by means of network signals. For example 4 networks.
  • the terminal can transmit the video frame to the preset server through the WebRtc transmission channel, and the preset server displays the video frame in the form of live video after receiving the video frame.
  • the WebRtc transmission channel includes an RTCPeerConnection channel and/or an RTCDataChannel channel.
  • RTCPeerConnection channel is a data channel built by RTCPeerConnetion API, which can realize efficient and stable data flow communication
  • RTCDataChannel is a data channel built by RTCDataChannel API, which can realize point-to-point exchange data with low latency and high throughput.
  • the UAV-based video live broadcast method provided in this embodiment is applied to a terminal that maintains communication with the UAV, first obtains the original video stream data sent by the UAV, and then encodes and decodes the original video stream data to obtain output buffer data , perform data conversion on the output buffer data to obtain video frames supported by the WebRtc protocol, and finally send the video frames to the preset server through the WebRtc transmission channel, and the preset server performs live broadcast display according to the video frames.
  • This method is used in the live broadcast process of drones
  • the real-time live broadcast based on the WebRtc protocol is realized with the help of the terminal, which greatly reduces the delay of the live broadcast picture.
  • the delay of the live broadcast picture is less than 200ms, while the UAV live broadcast delay of the conventional technology is more than 1s, and in this embodiment, there is no need to install a WebRtc conversion device and a network communication device on the UAV, which reduces the hardware cost of the UAV, and the computing performance of the terminal is also far away. It exceeds the computing power requirements for data conversion in the process of UAV live broadcast, and the underlying architecture of data conversion based on the terminal is mature and easy to implement, with low development costs.
  • the existing UAV system can be transformed only with the help of software, with a wide range of application scenarios.
  • Embodiment 2 provides a UAV-based video live broadcast method, which can be implemented on the basis of Embodiment 1. Part of the content in Embodiment 1 is specifically supplemented or exemplified, for example, according to the network signal strength
  • the process of performing resolution adjustment specifically includes:
  • the UAV-based live video method provided in this embodiment includes:
  • the network signal strength indicates the network signal strength of the terminal currently used to connect to the preset server.
  • the terminal can currently use 4G network and 5G network, but the terminal and the preset server are connected through the 4G network, then the network signal strength indicates the terminal's current 4G signal strength.
  • the current resolution indicates the resolution of the live image in the video frame, which is a resolution value adaptively adjusted according to the network signal strength to ensure that the data transmission speed will not affect the live broadcast delay. It is understandable that when the network signal strength is low, the data transmission speed is slow, and the data volume of live images with high resolution is bound to be larger than that of live images with low resolution, and the untimely data transmission will inevitably lead to live broadcast higher latency. Therefore, in this embodiment, a first rule for adjusting the current resolution according to the network signal strength is set, which can automatically select a suitable resolution value as the current resolution according to the network signal strength.
  • step S250 as shown in FIG. 5 specifically includes steps S251-253:
  • Steps S251-253 are only a specific example of determining the current resolution.
  • the first signal strength is a specific signal strength value, such as -67dBm, which can be set according to actual needs without limitation.
  • Figure 5 only shows the process of adjusting between two resolution values according to the first signal strength, but in practical applications, the second signal strength and the third signal strength can also be set.
  • the current resolution is 1080p (1920x1080p) and 720p ( In addition to 1080x720p), resolution values such as 360p and 4k can also be set.
  • 720p is usually the lowest resolution.
  • Video stream resolution is a parameter of a video frame that determines the resolution of the image that the video frame can generate. After determining the current resolution, the terminal adjusts the video stream resolution of the video frame according to the current resolution, which is actually adjusting the size of the final transmitted data.
  • S270 Send the video frame to a preset server through a WebRtc transmission channel, so that the preset server performs live display according to the video frame.
  • FIG. 6 it also includes a process of adjusting the code rate of the original video stream data according to the network signal strength:
  • the network signal strength indicates the network signal strength currently used by the terminal to connect to the preset server.
  • the second rule indicates a rule for increasing or decreasing the code rate of the video stream according to the increase and decrease of the network signal strength.
  • the code rate of the video stream increases with the increase of the network signal strength.
  • the second rule can be a proportional relational expression associated with network signal strength and video stream bit rate, indicating that the two (network signal strength and video stream bit rate) are proportional to each other.
  • the video stream bit rate is used to indicate the image clarity during image transmission between the UAV and the terminal. It changes with the change of the network signal strength and is a constantly fluctuating value, which determines the data volume of the original video stream data. .
  • the bit rate of the video stream during image transmission between the UAV and the terminal is further automatically adjusted according to the network signal strength, so as to reduce the overall live broadcast delay.
  • the terminal After the terminal determines the currently suitable video stream bit rate according to the network signal strength, it sends the information to the drone, and the drone then transmits the original video stream data to the terminal according to the video stream bit rate.
  • This embodiment further provides an adjustment process to the video stream resolution of the video frame according to the network signal strength, and a video stream bit rate adjustment process to the original video stream data, and the video stream resolution, the video stream bit rate and the network signal
  • the strength is adapted to avoid the live broadcast delay caused by the mismatch between the amount of data and the speed of data transmission.
  • FIG. 7 is a schematic structural diagram of a UAV-based video live broadcast system 300 provided in Embodiment 3 of the present invention. As shown in FIG. 7 , the system 300 includes:
  • the data receiving module 310 is used to obtain the original video stream data sent by the drone;
  • a codec module 320 configured to codec the original video stream data to obtain output buffer data
  • a data conversion module 330 configured to convert the output buffer data into video frames supported by the WebRtc protocol
  • the WebRtc transmission module 340 is configured to send the video frame to a preset server through a WebRtc transmission channel, so that the preset server performs live display according to the video frame.
  • the original video stream data includes H264 data
  • the codec module 320 is specifically used for:
  • the codec codec s the H264 data to generate ByteBuffer video stream data and writes it into the codec output buffer area.
  • the data conversion module 330 is specifically used to:
  • VideoFrame data Perform decoding and data conversion based on the ByteBuffer video stream data to obtain VideoFrame data.
  • a network detection module for detecting network signal strength
  • a resolution confirmation module configured to determine the current resolution according to the first rule according to the network signal strength
  • a video frame adjustment module configured to adjust the video stream resolution of the video frame according to the current resolution.
  • the resolution confirmation module is specifically used to:
  • the current resolution is 1080x720p.
  • a network detection module for detecting network signal strength
  • a code rate confirmation module configured to determine the video stream code rate according to the second rule according to the network signal strength
  • the code rate adjustment module is configured to send the video stream code rate to the UAV, so that the UAV sends the original video stream data based on the video stream code rate.
  • the WebRtc transmission channel includes an RTCPeerConnection channel and/or an RTCDataChannel channel.
  • This embodiment provides a UAV-based video live broadcast system, which is applied to a terminal that maintains communication with the UAV, first obtains the original video stream data sent by the UAV, and then encodes and decodes the original video stream data to obtain the output Buffer data, perform data conversion on the output buffer data to obtain video frames supported by the WebRtc protocol, and finally send the video frames to the preset server through the WebRtc transmission channel, and the preset server will perform live broadcast display according to the video frames.
  • This method is used in drone live broadcast
  • the real-time live broadcast based on the WebRtc protocol was realized with the help of the terminal, which greatly reduced the delay of the live broadcast screen, and there was no need to install a WebRtc conversion device and network communication device on the UAV, which reduced the hardware cost of the UAV and the computing performance of the terminal.
  • the existing UAV system can be transformed only with the help of software, with a wide range of application scenarios .
  • FIG. 8 is a schematic structural diagram of an electronic device 400 that can implement a UAV-based video live broadcast method provided by Embodiment 4 of the present invention.
  • the device includes a memory 410, a processor 420, and a processor in the device
  • the number of 420 can be one or more.
  • one processor 420 is taken as an example; the memory 410 and the processor 420 in the device can be connected through a bus or in other ways.
  • a bus connection is taken as an example.
  • the memory 410 can be used to store software programs, computer-executable programs and modules, such as the program instructions/modules corresponding to the UAV-based video live broadcast method in the embodiment of the present invention (for example, based on wireless
  • the processor 420 runs the software programs, instructions and modules stored in the memory 410 to execute various functional applications and data processing of the second page guidance module based on the two-dimensional code, that is, to realize the above-mentioned UAV-based Live video method.
  • the processor 420 is used to run the computer-executable program stored in the memory 410 to implement the following steps: step S110, obtain the original video stream data sent by the drone; step S120, process the original video stream data Perform encoding and decoding to obtain output buffer data; step S130, convert the output buffer data into video frames supported by the WebRtc protocol; step S140, send the video frames to the preset server through the WebRtc transmission channel, so that the preset The server performs live display according to the video frame.
  • the UAV-based video live broadcast device provided by the embodiment of the present invention is not limited to the above-mentioned method operation, and can also perform the UAV-based video broadcasting device provided by any embodiment of the present invention. Related operations in the live method.
  • the memory 410 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system and an application program required by at least one function; the data storage area may store data created according to the use of the terminal, and the like.
  • the memory 410 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid-state storage devices.
  • the memory 410 may further include memory located remotely from the processor 420, and these remote memories may be connected to the device through a network. Examples of the aforementioned networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.
  • Embodiment 5 of the present invention also provides a storage medium including computer-executable instructions.
  • Live video methods include:
  • the video frame is sent to the preset server through the WebRtc transmission channel, so that the preset server performs live display according to the video frame.
  • the computer-executable instructions are not limited to the method operations described above, and can also execute the UAV-based drone provided by any embodiment of the present invention. Related operations in the live video method.
  • the present invention can be realized by means of software and necessary general-purpose hardware, and of course can also be realized by hardware, but in many cases the former is a better implementation Way.
  • the essence of the technical solution of the present invention or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as a floppy disk of a computer , read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), flash memory (FLASH), hard disk or optical disc, etc., including a number of instructions to make an electronic device (which can be a personal computer) , equipment, or network equipment, etc.) execute the methods described in various embodiments of the present invention.
  • a computer-readable storage medium such as a floppy disk of a computer , read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), flash memory (FLASH), hard disk or optical disc, etc
  • the units and modules included are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be realized; in addition, each function
  • the specific names of the units are only for the convenience of distinguishing each other, and are not used to limit the protection scope of the present invention.

Landscapes

  • Engineering & Computer Science (AREA)
  • Signal Processing (AREA)
  • Multimedia (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Databases & Information Systems (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)

Abstract

本发明实施例公开了一种基于无人机的视频直播方法、系统、设备及存储介质。该方法包括：获取无人机发送的原始视频流数据；对原始视频流数据进行编解码得到输出缓冲数据；将输出缓冲数据转换为WebRtc协议支持的视频帧；将视频帧通过WebRtc传输通道发送至预设服务器，以使预设服务器根据视频帧进行直播显示。本发明实施例借助终端实现以WebRtc传输通道进行数据传输，降低了直播画面的延迟，并且无人机上无需设置WebRtc转换装置和网络通信装置，降低了无人机的硬件成本，并且基于终端进行的数据转换方案其底层架构成熟易于实现，开发成本低，仅借助软件上的改造即可对现有无人机系统进行改造，应用场景广泛。

Description

基于无人机的视频直播方法、系统、设备及存储介质
本申请要求于2021年9月10日提交中国专利局、申请号为202111061182X、申请名称为“基于无人机的视频直播方法、系统、设备及存储介质”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本发明涉及无人机数据传输技术领域，尤其涉及一种基于无人机的视频直播方法、系统、设备及存储介质。
背景技术
随着无人机技术的发展，其应用场景越来越广泛，在电力巡线、交通救援以及海关边防等场景中发挥了巨大作用，而应用场景的扩展对无人机的性能提出了更高的要求，例如针对需要无人机实时传输直播画面的任务，由于无人机的作业特性和作业环境，其直播延迟往往很高，从而影响了任务的完成。
目前的无人机系统，在需要直播的应用场景中，主要的直播方式包括两种：一种是将用于查看直播的客户端和无人机置于同一个网络中，但是这种情况下只有该网络内的客户端才能进行观看直播，并且其通常采用hls、hlv等协议的视频传输方式实时性差、时延高；另一种是将无人机拍摄的原始视频发送至安装有直播软件的终端设备，借助终端设备实现直播，但是直播软件在直播时通常使用TCP作为传输层协议，一旦出现丢包后续的所有包都会被缓冲，造成直播卡顿、延迟，而无人机飞行需要实时控制时容易出现事故。因此需要一种针 对无人机的新型直播方式，以确保无人机直播画面的低延时、不易卡顿。
发明内容
有鉴于此，本发明提供了一种基于无人机的视频直播方法、系统、设备及存储介质，以借助WebRtc协议进行基于无人机的视频直播，降低了直播延迟，避免了卡顿情况。
为解决上述技术问题，本发明采用以下技术方案：
第一方面，本发明提供了一种基于无人机的视频直播方法，该方法包括：
获取无人机发送的原始视频流数据；
对所述原始视频流数据进行编解码得到输出缓冲数据；
将所述输出缓冲数据转换为WebRtc协议支持的视频帧；
将所述视频帧通过WebRtc传输通道发送至预设服务器，以使所述预设服务器根据所述视频帧进行直播显示。
可选的，在一些实施例中，所述原始视频流数据包括H264数据，所述对所述原始视频流数据进行编解码得到输出缓冲数据，包括：
将所述H264数据写入编解码器输入缓存区并提交至编解码器；
编解码器将所述H264数据进行编解码生成ByteBuffer视频流数据并写入编解码输出缓存区。
可选的，在一些实施例中，所述将所述输出缓冲数据转换为WebRtc协议支持的视频帧，包括：
自所述编解码输出缓存区读取所述ByteBuffer视频流数据；
基于所述ByteBuffer视频流数据进行解码和数据转换得到VideoFrame数据。
可选的，在一些实施例中，所述将所述视频帧通过WebRtc传输通道发送至预设服务器之前，还包括：
检测网络信号强度；
根据所述网络信号强度按照第一规则确定当前分辨率；
根据所述当前分辨率调整所述视频帧的视频流分辨率。
可选的，在一些实施例中，根据所述网络信号强度按照第一规则确定当前分辨率包括：
判断所述网络信号强度是否大于等于第一信号强度；
若大于等于第一信号强度，则确定当前分辨率为1920x1080p；
若小于第一信号强度，则确定当前分辨率为1080x720p。
可选的，在一些实施例中，所述获取无人机发送的原始视频流数据之前，还包括：
检测网络信号强度；
根据所述网络信号强度按照第二规则确定视频流码率；
将所述视频流码率发送至无人机，以使无人机基于所述视频流码率发送原始视频流数据。
可选的，在一些实施例中，所述WebRtc传输通道包括RTCPeerConnection通道和/或RTCDataChannel通道。
第二方面，本发明提供了一种基于二维码的页面引导装置，包括：
数据接收模块，用于获取无人机发送的原始视频流数据；
编解码模块，用于对所述原始视频流数据进行编解码得到输出缓冲数据；
数据转换模块，用于将所述输出缓冲数据转换为WebRtc协议支持的视频帧；
WebRtc传输模块，用于将所述视频帧通过WebRtc传输通道发送至预设服务器，以使所述预设服务器根据所述视频帧进行直播显示。
第三方面，本发明提供了一种电子设备，包括存储器和处理器，所述存储器上存储有可在处理器运行的计算机程序，所述处理器执行所述计算机程序时实现如本发明任一实施例提供的的基于无人机的视频直播方法。
第四方面，本发明提供了一种计算机可读存储介质，所述存储介质存储有计算机程序，所述计算机程序包括程序指令，所述程序指令当被执行时实现如本发明任一实施例提供的的基于无人机的视频直播方法。
同现有技术相比，本发明实施例提供的基于无人机的视频直播方法，应用于与无人机保持通信的终端，先获取无人机发送的原始视频流数据，再对原始视频流数据进行编解码得到输出缓冲数据，对输出缓冲数据进行数据转换得到WebRtc协议支持的视频帧，最终将视频帧通过WebRtc传输通道发送至预设服务器，由预设服务器根据视频帧进行直播显示，该方法在无人机直播过程中借助终端实现了基于WebRtc协议的实时直播，大大降低了直播画面的延迟，并且无人机上无需设置WebRtc转换装置和网络通信装置，降低了无人机的硬件成本，终端的运算性能也远远超出无人机直播过程中数据转换的算力需求，并且基于终端进行的数据转换底层架构成熟易于实现，开发成本低，仅借助软件即可对现有无人机系统进行改造，应用场景广泛。
附图说明
为了更清楚地说明本申请实施例或现有技术中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作简单的介绍，显而易见的，下面描述中的附图仅仅是本申请的部分实施例，对于本领域普通技术人员来讲，在不付 出创造性劳动的前提下，还可以根据提供的附图获得其他的附图。
图1是本发明实施例一提供的一种基于无人机的视频直播方法的流程图；
图2是本发明实施例一提供的基于无人机的视频直播方法的子流程图；
图3是本发明实施例一提供的基于无人机的视频直播方法的子流程图；
图4是本发明实施例二提供的基于无人机的视频直播方法的流程图；
图5是本发明实施例二提供的基于无人机的视频直播方法的子流程图；
图6是本发明实施例二提供的基于无人机的视频直播方法的流程图；
图7是本发明实施例三提供的一种基于无人机的视频直播系统的结构示意图；
图8是本发明实施例四提供的一种电子设备的结构示意图。
具体实施方式
下面结合本申请实施例中的附图，对本申请实施中的技术方案进行清楚、完整的描述。可以理解的是，此处所描述的具体实施例仅仅是本申请一部分实施例，而不是全部的实施例，仅用于解释本申请，而非对本申请的限定。另外还需要说明的是，基于本申请中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本申请保护的范围。
除非另有定义，本文所使用的所有的技术和科学术语与属于本发明的技术领域的技术人员通常理解的含义相同。本文中在本发明的说明书中使用的术语只是为了描述具体的实施方式的目的，不是旨在于限制本发明。本文所使用的术语“和/或”包括一个或多个相关的所列项目的任意的和所有的组合。
此外，术语“第一”、“第二”等可在本文中用于描述各种方向、动作、步 骤或元件等，但这些方向、动作、步骤或元件不受这些术语限制。这些术语仅用于将第一个方向、动作、步骤或元件与另一个方向、动作、步骤或元件区分。举例来说，在不脱离本发明的范围的情况下，可以将第一用例称为第二用例，且类似地，可将第二用例称为第一用例。第一用例和第二用例两者都是用例，但其不是同一用例。术语“第一”、“第二”等而不能理解为指示或暗示相对重要性或者隐含指明所指示的技术特征的数量。由此，限定有“第一”、“第二”的特征可以明示或者隐含地包括一个或者多个特征的组合。在本发明的描述中，“多个”的含义是至少两个，例如两个，三个等，除非另有明确具体的限定。需要说明的是，当一个部被称为“固定于”另一个部，它可以直接在另一个部上也可以存在居中的部。当一个部被认为是“连接”到另一个部，它可以是直接连接到另一个部或者可能同时存在居中部。本文所使用的术语“垂直的”、“水平的”、“左”、“右”以及类似的表述，只是为了说明的目的，并不表示是唯一的实施方式。
在更加详细地讨论示例性实施例之前应当提到的是，一些示例性实施例被描述成作为流程图描绘的处理或方法。虽然流程图将各步骤描述成顺序的处理，但是其中的许多步骤可以被并行地、并发地或者同时实施。此外，各步骤的顺序可以被重新安排。当其操作完成时处理可以被终止，但是还可以具有未包括在附图中的附加步骤。处理可以对应于方法、函数、规程、子例程、子程序等等。
实施例一
参见图1，本实施例提供了一种基于无人机的视频直播方法，该方法可以应用于无人机系统，该系统包括无人机、终端和预设服务器，其中：无人机为 由遥控设备或自备程序控制装置操纵，带任务载荷的不载人航空器；终端为与遥控设备连接的电子设备，通过遥控设备与无人机保持通信，用于无人机和预设服务器之间的数据传输中继，通常采用手机、平板等设备；预设服务器与所述终端保持通信连接。本实施例提供的基于无人机的视频直播方法具体可以由无人机或终端执行，也可以由终端和无人机之间交互完成，本实施例中以终端为例进行具体说明。如图1所示，该方法包括以下步骤：
S110、获取无人机发送的原始视频流数据。
本实施例中，无人机上设置有飞控模块和图像采集模块，其中飞控模块用于对无人机飞行和任务进行监控和操纵，图像采集模块用于在无人机飞行过程中进行图像采集，原始视频流数据即由图像采集模块采集并传输至终端的数据。
具体的，本实施例中，图像采集模块采集与遥控设备通过专用通信链路(通常为通过遥控设备的天线和无人机建立的无线电通信链路)连接，图像采集模块采集的数据先由无人机传输至遥控设备，再由遥控设备传输至终端，终端最终获取的原始视频流数据通常为H264数据，在一些替代实施例中，原始视频流数据也能够采用其他数据格式，此处不作限制。
S120、对所述原始视频流数据进行编解码得到输出缓冲数据。
输出缓冲数据是由多媒体编解码器对原始视频流数据进行的特殊封装，以便后续将其进一步转换成能够通过WebRtc传输通道传输的特殊数据(即后文所指的WebRtc协议支持的视频帧)。与现有技术不同，本实施例中在进行基于无人机的视频直播时，采用WebRtc协议作为终端和预设服务器之间的数据传输协议，其不需要安装任何插件，通过简单的JavaScript API就可以使得实时通信变成一种标准功能，并且不会出现因为一条消息出问题引发所有消息出问题导 致直播卡顿的问题。但是，现有技术中WebRtc协议在应用时都是直接对摄像头采集的画面进行处理和传输的，而本实施例中无人机采集的画面需要通过专用通信链路传输到终端，终端获取的是原始视频流数据，其并不能直接通过WebRtc协议传输，因此，本实施例中需要对原始视频里数据进行编解码等处理得到符合WebRtc协议的数据。具体的，步骤S120中对原始视频流数据的编解码实际是将其转换为高效率的处理字节封装类数据，也即输出缓冲数据，这是因为在对原始视频流数据进行底层的视频流编码使其符合WebRtc协议时，处理的是字节，如果单纯借助字节数组来进行操作的话，首先效率有很大的问题，其次是数组最大的问题就是越界报异常等信息，因此使用了高效率的处理字节的封装类。并且由于后续仍要对该数据进行进一步处理，因此编解码器处理完成后将输出缓冲数据放在输出缓存区，以待后续使用。
具体的，在一些实施例中，终端为采用Android的手机或平板设备时，以所述原始视频流数据包括H264数据为例，其编解码过程具体如图2所示，包括步骤S121-122：
S121、将所述H264数据写入编解码器输入缓存区并提交至编解码器。
本实施例中编解码器采用MediaCodec提供的编解码器，MediaCodec是Android低层多媒体架构的一部分，通常与MediaExtractor、MediaMuxer、AudioTrack结合使用，能够编解码诸如H.264、H.265、AAC、3gp等常见的音视频格式。
S122、编解码器将所述H264数据进行编解码生成ByteBuffer视频流数据并写入编解码输出缓存区。
具体来说，MediaCodec在编解码的过程中使用了一组输入/输出缓存区来 同步或异步处理数据：首先，终端向获取到的编解码输入缓存区写入要编解码的数据(也即原始视频流数据)并将其提交给编解码器，待编解码器处理完毕后生成ByteBuffer视频流数据，将ByteBuffer视频流数据转存到编码输出缓存区，同时收回客户端对输入缓存区的所有权；然后，终端从获取到编解码输出缓存区读取编码好的数据进行处理(非步骤S122中的内容)，待处理完毕后编解码器收回客户端对输出缓存区的所有权。
S130、将所述输出缓冲数据转换为WebRtc协议支持的视频帧。
视频帧为WebRtc协议支持的VideoFrame数据对象，其能够通过WebRtc提供的推流接口进行推流。步骤S120中得到的输出缓冲数据仅能实现高效的数据处理，依然不能通过WebRtc协议直接进行传输，因此通过步骤S130对其进行进一步转换：终端不断从编解码输出缓存区中读取输出缓冲数据进行转换，得到能够通过WebRtc协议传输的视频帧。
具体的，继续以上述采用Android的手机或平板设备作为终端的示例为例，在一些实施例中，输出缓冲数据的转换过程也即步骤S130,如图3所示包括步骤S131-132：
S131、自所述编解码输出缓存区读取所述ByteBuffer视频流数据。
S132、基于所述ByteBuffer视频流数据进行解码和数据转换得到VideoFrame数据。
前文已经提及ByteBuffer视频流数据在生成后直接写入编解码输出缓存区，无人机系统在进行直播显示时，终端从编解码输出缓存区不断读取ByteBuffer视频流数据，并设置好视频流解码出来每一帧图像的宽、高和时间戳等信息，封装形成VideoFrame数据。
S140、将所述视频帧通过WebRtc传输通道发送至预设服务器，以使所述预设服务器根据所述视频帧进行直播显示。
WebRtc传输通道为WebRtc提供的推流接口连接的两个设备间的数据通路，本实施中由WebRtc提供的推流接口连接的是终端和预设服务器，WebRtc传输通道实际是借助网络信号实现的，例如4网络。具体的，在得到WebRtc协议支持的视频帧之后，终端即可通过WebRtc传输通道将视频帧传输至预设服务器，预设服务器接收视频帧后将其以视频直播的形式进行显示。
在一些具体实施例中，所述WebRtc传输通道包括RTCPeerConnection通道和/或RTCDataChannel通道。RTCPeerConnection通道是由RTCPeerConnetion API搭建的数据通路，能够实现高效和稳定的数据流通信；RTCDataChannel是由RTCDataChannel API搭建的数据通路，能够实现低延时和高吞吐量的点对点的交换数据。
本实施例提供的基于无人机的视频直播方法，应用于与无人机保持通信的终端，先获取无人机发送的原始视频流数据，再对原始视频流数据进行编解码得到输出缓冲数据，对输出缓冲数据进行数据转换得到WebRtc协议支持的视频帧，最终将视频帧通过WebRtc传输通道发送至预设服务器，由预设服务器根据视频帧进行直播显示，该方法在无人机直播过程中借助终端实现了基于WebRtc协议的实时直播，大大降低了直播画面的延迟，在实际应用中，基于本实施例提供的基于无人机的视频直播方法实现了通过公网预览时，直播画面延迟小于200ms，而常规技术的无人机直播延迟都在1s以上，并且本实施例中无人机上无需设置WebRtc转换装置和网络通信装置，降低了无人机的硬件成本，终端的运算性能也远远超出无人机直播过程中数据转换的算力需求，并且基于终端进 行的数据转换底层架构成熟易于实现，开发成本低，仅借助软件即可对现有无人机系统进行改造，应用场景广泛。
实施例二
实施例二提供了一种基于无人机的视频直播方法，其可以在实施例一的基础上实现，对实施例一中的部分内容进行了具体的补充或举例，例如提供了根据网络信号强度进行分辨率调整的过程，具体包括：
如图4所示，本实施例提供的基于无人机的视频直播方法包括：
S210、获取无人机发送的原始视频流数据。
S220、对所述原始视频流数据进行编解码得到输出缓冲数据。
S230、将所述输出缓冲数据转换为WebRtc协议支持的视频帧。
S240、检测网络信号强度。
网络信号强度表示终端当前用于连接预设服务器的网络信号强度，例如终端当前能够使用4G网络和5G网络，但是终端与预设服务器是通过4G网络连接的，那么网络信号强度表示的是终端当前的4G信号强度。
S250、根据所述网络信号强度按照第一规则确定当前分辨率。
当前分辨率表示视频帧中直播图像的分辨率，是一个根据网络信号强度自适应调整的分辨率值，用于确保数据传输速度不会影响直播延迟。可以理解的是，当网络信号强度低时，其数据传输速度慢，而分辨率高的直播图像其数据量必然比分辨率低的直播图像数据量大，则数据传输不及时必然会导致直播出现更高的延迟。因此，本实施例中设置用于根据网络信号强度调整当前分辨率的第一规则，其能够根据网络信号强度自动选择与之适应的分辨率值作为当前分辨率。
在一个具体实施例中，如图5所示步骤S250具体包括步骤S251-253：
S251、判断所述网络信号强度是否大于等于第一信号强度。
S252、若大于等于第一信号强度，则确定当前分辨率为1920x1080p。
S253、若小于第一信号强度，则确定当前分辨率为1080x720p。
步骤S251-253仅为一确定当前分辨率的具体举例，第一信号强度为一个具体的信号强度值，例如-67dBm，根据实际需求自行设置此处不作限制。图5仅给出了根据第一信号强度在两种分辨率值间调整的过程，而实际应用中还可以设置第二信号强度、第三信号强度，当前分辨率除了1080p(1920x1080p)和720p(1080x720p)之外，还可以设置360p和4k等分辨率值，当然在一些实施例中，为了确保显示清晰度，通常720p即为最低清晰度。
S260、根据所述当前分辨率调整所述视频帧的视频流分辨率。
视频流分辨率是视频帧的一项参数，用于确定视频帧能够生成的图像的分辨率。在确定了当前分辨率之后，终端根据当前分辨率对视频帧的视频流分辨率进行调整，实际就是在调整最终传输的数据量大小。
S270、将所述视频帧通过WebRtc传输通道发送至预设服务器，以使所述预设服务器根据所述视频帧进行直播显示。
更具体的，在一些实施例中，如图6所示，还包括根据网络信号强度对原始视频流数据进行码率调整的过程：
S201、检测网络信号强度。
与步骤S240类似的，网络信号强度表示终端当前用于连接预设服务器的网络信号强度。
S202、根据所述网络信号强度按照第二规则确定视频流码率。
第二规则表示根据网络信号强度的升高和降低对应增加或减小视频流码率的规则，本实施例中视频流码率随着网路信号强的增大而增大。在一些具体实施例中，第二规则能够是一个以网络信号强度和视频流码率为关联量的正比例关系式，表示二者(网络信号强度和视频流码率)成正比关系。视频流码率用于表示无人机与终端间进行图传时的图像清晰度，随着网络信号强度的变化而变化，是一个不断波动的值，其决定了原始视频流数据的数据量大小。可以理解的是，原始视频流数据的数据量越大，终端在处理转换时耗费的时间也就越多，而当网络信号强度比较低时，其在转化成视频帧时还需要调整分辨率，进一步增加了处理时间。因此本实施中进一步根据网络信号强度自动调整无人机与终端间进行图传时的视频流码率，以降低整体的直播延迟。
S203、将所述视频流码率发送至无人机，以使无人机基于所述视频流码率发送原始视频流数据。
终端在根据网络信号强度确定了当前适合的视频流码率后，将该信息发送至无人机，无人机随后按照视频流码率向终端传输原始视频流数据。
S204、获取无人机发送的原始视频流数据。
S205、对所述原始视频流数据进行编解码得到输出缓冲数据。
S206、将所述输出缓冲数据转换为WebRtc协议支持的视频帧。
S207、将所述视频帧通过WebRtc传输通道发送至预设服务器，以使所述预设服务器根据所述视频帧进行直播显示。
本实施例进一步提供了根据网络信号强度对视频帧的视频流分辨率的调整过程，以及对原始视频流数据的视频流码率调整过程，是的视频流分辨率、视频流码率与网络信号强度相适应，避免了数据量和数据传输速度不匹配导致的 直播延迟。
实施例三
图7为本发明实施例三提供的一种基于无人机的视频直播系统300的结构示意图，如图7所述，该系统300包括：
数据接收模块310，用于获取无人机发送的原始视频流数据；
编解码模块320，用于对所述原始视频流数据进行编解码得到输出缓冲数据；
数据转换模块330，用于将所述输出缓冲数据转换为WebRtc协议支持的视频帧；
WebRtc传输模块340，用于将所述视频帧通过WebRtc传输通道发送至预设服务器，以使所述预设服务器根据所述视频帧进行直播显示。
可选的，在一些实施例中，所述原始视频流数据包括H264数据，编解码模块320具体用于：
将所述H264数据写入编解码器输入缓存区并提交至编解码器；
编解码器将所述H264数据进行编解码生成ByteBuffer视频流数据并写入编解码输出缓存区。
可选的，在一些实施例中，数据转换模块330具体用于：
自所述编解码输出缓存区读取所述ByteBuffer视频流数据；
基于所述ByteBuffer视频流数据进行解码和数据转换得到VideoFrame数据。
可选的，在一些实施例中，还包括：
网络检测模块，用于检测网络信号强度；
分辨率确认模块，用于根据所述网络信号强度按照第一规则确定当前分辨率；
视频帧调整模块，用于根据所述当前分辨率调整所述视频帧的视频流分辨率。
可选的，在一些实施例中，分辨率确认模块具体用于：
判断所述网络信号强度是否大于等于第一信号强度；
若大于等于第一信号强度，则确定当前分辨率为1920x1080p；
若小于第一信号强度，则确定当前分辨率为1080x720p。
可选的，在一些实施例中，包括：
网络检测模块，用于检测网络信号强度；
码率确认模块，用于根据所述网络信号强度按照第二规则确定视频流码率；
码率调整模块，用于将所述视频流码率发送至无人机，以使无人机基于所述视频流码率发送原始视频流数据。
可选的，在一些实施例中，所述WebRtc传输通道包括RTCPeerConnection通道和/或RTCDataChannel通道。
本实施例提供了一种基于无人机的视频直播系统，应用于与无人机保持通信的终端，先获取无人机发送的原始视频流数据，再对原始视频流数据进行编解码得到输出缓冲数据，对输出缓冲数据进行数据转换得到WebRtc协议支持的视频帧，最终将视频帧通过WebRtc传输通道发送至预设服务器，由预设服务器根据视频帧进行直播显示，该方法在无人机直播过程中借助终端实现了基于WebRtc协议的实时直播，大大降低了直播画面的延迟，并且无人机上无需设置WebRtc转换装置和网络通信装置，降低了无人机的硬件成本，终端的运算性能 也远远超出无人机直播过程中数据转换的算力需求，并且基于终端进行的数据转换底层架构成熟易于实现，开发成本低，仅借助软件即可对现有无人机系统进行改造，应用场景广泛。
实施例四
图8为本发明实施例四提供的一种可以实现基于无人机的视频直播方法的电子设备400的结构示意图，如图8所示，该设备包括存储器410、处理器420，设备中处理器420的数量可以是一个或多个，图8中以一个处理器420为例；设备中的存储器410、处理器420可以通过总线或其他方式连接，图8中以通过总线连接为例。
存储器410作为一种计算机可读存储介质，可用于存储软件程序、计算机可执行程序以及模块，如本发明实施例中的基于无人机的视频直播方法对应的程序指令/模块(例如，基于无人机的视频直播系统中的数据接收模块310、编解码模块320、数据转换模块330和WebRtc传输模块340)。处理器420通过运行存储在存储器410中的软件程序、指令以及模块，从而执行基于二维码的页面引导第二页面引导模块的各种功能应用以及数据处理，即实现上述的基于无人机的视频直播方法。
其中，所述处理器420用于运行存储在存储器410中的计算机可执行程序，以实现如下步骤：步骤S110、获取无人机发送的原始视频流数据；步骤S120、对所述原始视频流数据进行编解码得到输出缓冲数据；步骤S130、将所述输出缓冲数据转换为WebRtc协议支持的视频帧；步骤S140、将所述视频帧通过WebRtc传输通道发送至预设服务器，以使所述预设服务器根据所述视频帧进行 直播显示。
当然,本发明实施例所提供的一种基于无人机的视频直播装置,该装置不限于如上所述的方法操作,还可以执行本发明实施例任意实施例所提供的基于无人机的视频直播方法中的相关操作。
存储器410可主要包括存储程序区和存储数据区，其中，存储程序区可存储操作系统、至少一个功能所需的应用程序；存储数据区可存储根据终端的使用所创建的数据等。此外，存储器410可以包括高速随机存取存储器，还可以包括非易失性存储器，例如至少一个磁盘存储器件、闪存器件、或其他非易失性固态存储器件。在一些实例中，存储器410可进一步包括相对于处理器420远程设置的存储器，这些远程存储器可以通过网络连接至设备。上述网络的实例包括但不限于互联网、企业内部网、局域网、移动通信网及其组合。
实施例五
本发明实施例五还提供一种包括计算机可执行指令的存储介质，所述计算机可执行指令在由计算机处理器执行时用于执行一种基于无人机的视频直播方法，该基于无人机的视频直播方法包括：
获取无人机发送的原始视频流数据；
对所述原始视频流数据进行编解码得到输出缓冲数据；
将所述输出缓冲数据转换为WebRtc协议支持的视频帧；
将所述视频帧通过WebRtc传输通道发送至预设服务器，以使所述预设服务器根据所述视频帧进行直播显示。
当然，本发明实施例所提供的一种包括计算机可执行指令的存储介质,其计 算机可执行指令不限于如上所述的方法操作，还可以执行本发明任意实施例所提供的基于无人机的视频直播方法中的相关操作。
通过以上关于实施方式的描述，所述领域的技术人员可以清楚地了解到，本发明可借助软件及必需的通用硬件来实现，当然也可以通过硬件实现，但很多情况下前者是更佳的实施方式。基于这样的理解，本发明的技术方案本质上或者说对现有技术做出贡献的部分可以以软件产品的形式体现出来，该计算机软件产品可以存储在计算机可读存储介质中，如计算机的软盘、只读存储器(Read-Only Memory,ROM)、随机存取存储器(Random Access Memory,RAM)、闪存(FLASH)、硬盘或光盘等，包括若干指令用以使得一台电子设备(可以是个人计算机，设备，或者网络设备等)执行本发明各个实施例所述的方法。
值得注意的是，上述授权系统的实施例中，所包括的各个单元和模块只是按照功能逻辑进行划分的，但并不局限于上述的划分，只要能够实现相应的功能即可；另外，各功能单元的具体名称也只是为了便于相互区分，并不用于限制本发明的保护范围。
注意，上述仅为本发明的较佳实施例及所运用技术原理。本领域技术人员会理解，本发明不限于这里所述的特定实施例，对本领域技术人员来说能够进行各种明显的变化、重新调整和替代而不会脱离本发明的保护范围。因此，虽然通过以上实施例对本发明进行了较为详细的说明，但是本发明不仅仅限于以上实施例，在不脱离本发明构思的情况下，还可以包括更多其他等效实施例，而本发明的范围由所附的权利要求范围决定。

Claims (10)

  1. 一种基于无人机的视频直播方法，其特征在于，包括：
    获取无人机发送的原始视频流数据；
    对所述原始视频流数据进行编解码得到输出缓冲数据；
    将所述输出缓冲数据转换为WebRtc协议支持的视频帧；
    将所述视频帧通过WebRtc传输通道发送至预设服务器，以使所述预设服务器根据所述视频帧进行直播显示。
  2. 根据权利要求1所述的基于无人机的视频直播方法，其特征在于，所述原始视频流数据包括H264数据，所述对所述原始视频流数据进行编解码得到输出缓冲数据，包括：
    将所述H264数据写入编解码器输入缓存区并提交至编解码器；
    编解码器将所述H264数据进行编解码生成ByteBuffer视频流数据并写入编解码输出缓存区。
  3. 根据权利要求2所述的基于无人机的视频直播方法，其特征在于，所述将所述输出缓冲数据转换为WebRtc协议支持的视频帧，包括：
    自所述编解码输出缓存区读取所述ByteBuffer视频流数据；
    基于所述ByteBuffer视频流数据进行解码和数据转换得到VideoFrame数据。
  4. 根据权利要求1所述的基于无人机的视频直播方法，其特征在于，所述将所述视频帧通过WebRtc传输通道发送至预设服务器之前，还包括：
    检测网络信号强度；
    根据所述网络信号强度按照第一规则确定当前分辨率；
    根据所述当前分辨率调整所述视频帧的视频流分辨率。
  5. 根据权利要求4所述的基于无人机的视频直播方法，其特征在于，根据所述网络信号强度按照第一规则确定当前分辨率包括：
    判断所述网络信号强度是否大于等于第一信号强度；
    若大于等于第一信号强度，则确定当前分辨率为1920x1080p；
    若小于第一信号强度，则确定当前分辨率为1080x720p。
  6. 根据权利要求1所述的基于无人机的视频直播方法，其特征在于，所述获取无人机发送的原始视频流数据之前，还包括：
    检测网络信号强度；
    根据所述网络信号强度按照第二规则确定视频流码率；
    将所述视频流码率发送至无人机，以使无人机基于所述视频流码率发送原始视频流数据。
  7. 根据权利要求2所述的基于无人机的视频直播方法，其特征在于：
    所述WebRtc传输通道包括RTCPeerConnection通道和/或RTCDataChannel通道。
  8. 一种基于无人机的视频直播系统，其特征在于，包括：
    数据接收模块，用于获取无人机发送的原始视频流数据；
    编解码模块，用于对所述原始视频流数据进行编解码得到输出缓冲数据；
    数据转换模块，用于将所述输出缓冲数据转换为WebRtc协议支持的视频帧；
    WebRtc传输模块，用于将所述视频帧通过WebRtc传输通道发送至预设服务器，以使所述预设服务器根据所述视频帧进行直播显示。
  9. 一种电子设备，其特征在于，包括存储器和处理器，所述存储器上存储有可在处理器运行的计算机程序，所述处理器执行所述计算机程序时实现如权 利要求1-7任意一项所述的基于无人机的视频直播方法。
  10. 一种计算机可读存储介质，其特征在于，所述存储介质存储有计算机程序，所述计算机程序包括程序指令，所述程序指令当被执行时，实现如权利要求1-7任意一项所述的基于无人机的视频直播方法。
PCT/CN2022/117900 2021-09-10 2022-09-08 基于无人机的视频直播方法、系统、设备及存储介质 WO2023036258A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202111061182.XA CN113747191A (zh) 2021-09-10 2021-09-10 基于无人机的视频直播方法、系统、设备及存储介质
CN202111061182.X 2021-09-10

Publications (1)

Publication Number Publication Date
WO2023036258A1 true WO2023036258A1 (zh) 2023-03-16

Family

ID=78738017

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/117900 WO2023036258A1 (zh) 2021-09-10 2022-09-08 基于无人机的视频直播方法、系统、设备及存储介质

Country Status (2)

Country Link
CN (1) CN113747191A (zh)
WO (1) WO2023036258A1 (zh)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113747191A (zh) * 2021-09-10 2021-12-03 深圳市道通智能航空技术股份有限公司 基于无人机的视频直播方法、系统、设备及存储介质
CN115348422A (zh) * 2022-08-12 2022-11-15 亿航智能设备（广州）有限公司 无人机图传方法、系统及计算机可读存储介质

Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104244108A (zh) * 2014-09-24 2014-12-24 上海网达软件股份有限公司 一种直播方法及系统
CN104253814A (zh) * 2014-09-12 2014-12-31 清华大学 一种流媒体处理方法、服务器及浏览器
US20150135236A1 (en) * 2011-01-11 2015-05-14 Manolo Fabio Rivera Advanced wireless iptv set top box
CN106209949A (zh) * 2015-05-07 2016-12-07 北京大学 基于WebRTC的交互式直播方法及装置
CN108206833A (zh) * 2018-01-11 2018-06-26 福建星网锐捷通讯股份有限公司 音视频数据传输方法及其系统
CN109587510A (zh) * 2018-12-10 2019-04-05 广州虎牙科技有限公司 一种直播方法、装置、设备和存储介质
CN109862386A (zh) * 2019-03-27 2019-06-07 北京奇艺世纪科技有限公司 直播数据传输方法及装置
CN110830777A (zh) * 2019-11-29 2020-02-21 公安部第一研究所 基于WebRTC协议的跨浏览器预览GB/T28181标准监控视频的方法
CN113747191A (zh) * 2021-09-10 2021-12-03 深圳市道通智能航空技术股份有限公司 基于无人机的视频直播方法、系统、设备及存储介质

Family Cites Families (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105744231A (zh) * 2016-03-18 2016-07-06 普宙飞行器科技（深圳）有限公司 无人机移动直播方法及无人机移动直播系统
EP3349394B1 (en) * 2017-01-16 2021-11-10 Vertigo Media, Inc. System, method, and application for exchanging content in a social network environment
CN110785978B (zh) * 2017-06-20 2022-03-15 瑞典爱立信有限公司 用于直播上行链路自适应流传输的设备及方法
WO2019075581A1 (en) * 2017-10-19 2019-04-25 Lazar Entertainment Inc. SYSTEMS AND METHODS FOR LIVE MULTIMEDIA STREAM BROADCAST
US11151229B1 (en) * 2020-04-10 2021-10-19 Avila Technology, LLC Secure messaging service with digital rights management using blockchain technology
CN111901630A (zh) * 2020-06-17 2020-11-06 视联动力信息技术股份有限公司 一种数据传输方法、装置、终端设备和存储介质
CN112788050A (zh) * 2021-01-25 2021-05-11 山东云缦智能科技有限公司 基于内容分发网络实现低延时直播的系统及方法
CN113099204B (zh) * 2021-04-13 2022-12-13 北京航空航天大学青岛研究院 一种基于vr头戴显示设备的远程实景增强现实方法
CN113158116A (zh) * 2021-04-20 2021-07-23 联通(广东)产业互联网有限公司 基于移动互联网的无人机控制平台

Patent Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150135236A1 (en) * 2011-01-11 2015-05-14 Manolo Fabio Rivera Advanced wireless iptv set top box
CN104253814A (zh) * 2014-09-12 2014-12-31 清华大学 一种流媒体处理方法、服务器及浏览器
CN104244108A (zh) * 2014-09-24 2014-12-24 上海网达软件股份有限公司 一种直播方法及系统
CN106209949A (zh) * 2015-05-07 2016-12-07 北京大学 基于WebRTC的交互式直播方法及装置
CN108206833A (zh) * 2018-01-11 2018-06-26 福建星网锐捷通讯股份有限公司 音视频数据传输方法及其系统
CN109587510A (zh) * 2018-12-10 2019-04-05 广州虎牙科技有限公司 一种直播方法、装置、设备和存储介质
CN109862386A (zh) * 2019-03-27 2019-06-07 北京奇艺世纪科技有限公司 直播数据传输方法及装置
CN110830777A (zh) * 2019-11-29 2020-02-21 公安部第一研究所 基于WebRTC协议的跨浏览器预览GB/T28181标准监控视频的方法
CN113747191A (zh) * 2021-09-10 2021-12-03 深圳市道通智能航空技术股份有限公司 基于无人机的视频直播方法、系统、设备及存储介质

Also Published As

Publication number Publication date
CN113747191A (zh) 2021-12-03

Similar Documents

Publication Publication Date Title
WO2023036258A1 (zh) 基于无人机的视频直播方法、系统、设备及存储介质
US9894393B2 (en) Video encoding for reduced streaming latency
CN103248964B (zh) 基于rtp/rtcp的车载视频传输系统
CN103813213A (zh) 基于移动云计算的实时视频分享平台和方法
WO2018058575A1 (zh) 一种飞行图像数据的处理方法、系统及地面端设备
CN112601096B (zh) 视频解码方法、装置、设备及可读存储介质
WO2018072337A1 (zh) 用于虚拟现实的无线传输方法、装置、终端和头显设备
WO2024041239A1 (zh) 一种沉浸媒体的数据处理方法、装置、设备、存储介质及程序产品
CN110958431A (zh) 多路视频压缩后传系统及方法
CN114928754B (zh) 用于实景三维数据的数据处理方法及电子设备
WO2023040825A1 (zh) 媒体信息的传输方法、计算设备及存储介质
WO2023216798A1 (zh) 音视频的转码装置、方法、设备、介质及产品
CN112887679A (zh) 无损视频远程采集方法和系统
CN104639979A (zh) 视频分享方法及系统
CN103428493A (zh) 轻型化高清视频无线传输系统
US11032345B2 (en) Client side data stream processing
CN217240799U (zh) 一种数据传输处理装置及车辆
CN102638726A (zh) 一种基于太赫兹无线通信的多媒体流传输方法及系统
CN111212285A (zh) 硬件视频编码系统和硬件视频编码系统的控制方法
WO2018054349A1 (zh) 数据发送方法、数据接收方法及其装置和系统
CN115278323A (zh) 一种显示设备、智能设备和数据处理方法
CN201528404U (zh) 视频编解码系统及其视频编码器、视频解码器
WO2019233296A1 (zh) 设备的远程控制
KR100899666B1 (ko) 분산 다중 스트리밍 전송 장치
CN205092920U (zh) 高分辨率视频传输系统

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22866719

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE