WO2020155037A1 - 多负载多路图传方法、控制系统和终端、无人机和服务器 - Google Patents

多负载多路图传方法、控制系统和终端、无人机和服务器 Download PDF

Info

Publication number
WO2020155037A1
WO2020155037A1 PCT/CN2019/074230 CN2019074230W WO2020155037A1 WO 2020155037 A1 WO2020155037 A1 WO 2020155037A1 CN 2019074230 W CN2019074230 W CN 2019074230W WO 2020155037 A1 WO2020155037 A1 WO 2020155037A1
Authority
WO
WIPO (PCT)
Prior art keywords
load
image data
drone
image transmission
selection instruction
Prior art date
Application number
PCT/CN2019/074230
Other languages
English (en)
French (fr)
Inventor
王景正
郑德恩
Original Assignee
深圳市大疆创新科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳市大疆创新科技有限公司 filed Critical 深圳市大疆创新科技有限公司
Priority to CN201980004941.3A priority Critical patent/CN111279691B/zh
Priority to PCT/CN2019/074230 priority patent/WO2020155037A1/zh
Publication of WO2020155037A1 publication Critical patent/WO2020155037A1/zh

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
    • H04N7/181—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast for receiving images from a plurality of remote sources
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast

Definitions

  • This application relates to the field of control technology, in particular to a multi-load multi-channel image transmission method, a control system, a control terminal, an unmanned aerial vehicle and a server.
  • the receiving channel of camera data cannot be opened adaptively, which will open all the receiving ports of possible camera types, which wastes system resources.
  • the wireless bandwidth is low.
  • the bit rate of the image transmission video stream is low, and the video definition is low.
  • bandwidth adaptive transmission the current network bandwidth situation will be sent to the camera load side through a message.
  • the camera load will perform bit rate matching and transmit the image transmission data of the matching bandwidth to the drone, and the drone will perform transparent transmission.
  • the bandwidth feedback link is too long, which will cause the actual data to be transmitted to the wireless network card node, the network has changed, resulting in image transmission data blurring.
  • Image transmission is essentially to transmit multiple load data on the aircraft to the remote control, but in fact, the app and the large screen can only display one or two of them when displaying, which causes a waste of wireless network bandwidth. It further reduces the video definition, especially at the edge of low signal, which will bring poorer image transmission quality.
  • the embodiment of the application provides a method, control system, control terminal, drone, and server for multi-load and multi-channel image transmission, so as to remove the limitation that data can only be transmitted through a single channel, and realize multi-channel real-time transmission of image transmission data.
  • the working scenarios of man-machine applications in the industry provide more possibilities.
  • the first aspect of the embodiments of the present application provides a multi-load and multi-channel image transmission method for controlling a terminal, including:
  • the selection instruction includes the selected load type, and the selection instruction is sent to the drone so that the drone opens the corresponding communication port according to the load type;
  • the technical solution of the second aspect of this application provides a multi-load and multi-channel image transmission method for UAVs.
  • the UAV includes the UAV body and multiple loads arranged on the UAV body.
  • Road map transmission methods include:
  • the selection instruction includes the selected load type
  • the image data of the corresponding load is sent to the control terminal through the image transmission channel.
  • control system including a control terminal and a drone, wherein:
  • the control terminal responds to multiple load selection instructions, the selection instruction includes the selected load type, and sends the selection instruction to the drone so that the drone opens the corresponding communication port according to the load type;
  • the UAV determines the corresponding image transmission channel according to the load type in the selection instruction
  • the UAV sends the image data of the corresponding load to the control terminal through the image transmission channel, so that the control terminal can display the image data.
  • the technical solution of the fourth aspect of the present application provides a control terminal, including: a remote control device, a display device, and a processor, wherein the processor is used for:
  • the selection instruction includes the selected load type, and the selection instruction is sent to the drone so that the drone opens the corresponding communication port according to the load type;
  • the technical solution of the fifth aspect of the present application provides an unmanned aerial vehicle, which includes an unmanned aerial vehicle body, a plurality of loads arranged on the unmanned aerial vehicle body, and a processor, wherein the processor is used for:
  • the selection instruction includes the selected load type
  • the image data of the corresponding load is sent to the control terminal through the image transmission channel.
  • the technical solution of the sixth aspect of the present application provides a server, including a processor, and the processor is configured to:
  • the selection instruction includes the selected load type, and send the selection instruction to the drone so that the drone opens the corresponding communication port according to the load type;
  • the technical solution of the seventh aspect of the present application provides a computer-readable storage medium on which a computer program is stored.
  • the computer program is executed by a processor, the multi-load multiple image transmission as provided in the first aspect of the embodiments of the present application is realized.
  • the steps of the method implement the steps of the multi-load multi-channel image transmission method provided in the second aspect of the embodiment of the present application.
  • the multi-load and multi-channel image transmission method, control system, control terminal, drone, and server provided by the embodiments of the present application can realize the following by simplifying the link process:
  • Multi-channel data (at least two channels) for real-time transmission.
  • the main road image transmission can use the on-demand-i (on-demand I frame) encoding method to achieve a higher bit rate compression ratio and bring higher image transmission clarity.
  • Two channels of data are automatically selected for real-time transmission from multiple (at least 3) load equipment.
  • it is video data, it will be re-encoded and transmitted, and the re-encoded video bit rate matches the current bandwidth; the best bandwidth adaptation mechanism can be realized to avoid blurring in the main road image transmission
  • the bandwidth feedback mechanism can be used to transparently transmit the data to reduce excessive occupation of system resources.
  • FIG. 1 shows a schematic structural block diagram of an unmanned aerial vehicle system according to an embodiment of the present application
  • FIG. 2 shows a schematic flowchart of a multi-load multi-channel image transmission method running on a control terminal according to an embodiment of the present application
  • Fig. 3 shows a schematic diagram of a two-channel image transmission process of a specific embodiment of the present application
  • FIG. 4 shows a schematic diagram of a flow chart of opening a video transmission channel and performing data transmission in a specific embodiment of the present application
  • FIG. 5 shows a schematic flowchart of a multi-load and multi-channel image transmission method running on a drone according to an embodiment of the present application
  • FIG. 6 shows a schematic flowchart of a load port opening and closing mechanism of a specific embodiment of the present application
  • FIG. 7 shows a structural block diagram of the control system of an embodiment of the present application.
  • Figure 8 shows a structural diagram of a control terminal according to an embodiment of the present application.
  • Fig. 9 shows a structural diagram of an unmanned aerial vehicle according to an embodiment of the present application.
  • a component when referred to as being "fixed to” another component, it can be directly on the other component or a central component may also exist.
  • a component When a component is considered to be “connected” to another component, it can be directly connected to another component or a centered component may exist at the same time.
  • FIG. 1 is a schematic architecture diagram of an unmanned aerial vehicle system 10 provided according to an embodiment of the present application.
  • the UAV system 10 may include a control terminal 110 of the UAV and the UAV 120.
  • the drone 120 may be a single-rotor or multi-rotor drone.
  • the drone 120 may include a power system 102, a control system 104, and a fuselage.
  • the fuselage may include a center frame and one or more arms connected to the center frame, and the one or more arms extend radially from the center frame.
  • the drone may also include a tripod, wherein the tripod is connected to the fuselage for supporting the drone when it is landing.
  • the power system 102 may include one or more motors 1022, and the motors 1022 are used to provide power to the drone 120, and the power enables the drone 120 to realize movement of one or more degrees of freedom.
  • the control system 104 may include a controller 1042 and a sensing system 1044.
  • the sensing system 1044 is used to measure the status information of the drone 120 and/or the information of the environment in which the drone 120 is located, where the status information may include attitude information, position information, remaining power information, and the like.
  • the information of the environment may include the depth of the environment, the atmospheric pressure of the environment, the humidity of the environment, the temperature of the environment, and so on.
  • the sensing system 1044 may include, for example, at least one of sensors such as a barometer, a gyroscope, an ultrasonic sensor, an electronic compass, an inertial measurement unit, a visual sensor, a global navigation satellite system, and a barometer.
  • the global navigation satellite system may be a global positioning system (Global Positioning System, GPS).
  • the controller 1042 is used to control various operations of the drone.
  • the controller 1042 can control the movement of the drone, and for another example, the controller 1042 can control the sensor system 1044 of the drone to collect data.
  • the drone 120 may include a photographing device 1064.
  • the photographing device 1064 may be, for example, a device for capturing images such as a camera or a video camera.
  • the photographing device 1064 may communicate with the controller 1042, and is configured in the controller 1042. Shooting under control, the controller 1042 can also control the drone 120 according to the image captured by the camera 1064.
  • the drone 120 further includes a pan/tilt 106.
  • the pan/tilt 106 may include a motor 1062.
  • the pan/tilt 106 is used to carry a camera 1064.
  • the controller 1042 may control the movement of the pan/tilt 106 through the motor. It should be understood that the pan-tilt 106 may be independent of the drone 120 or a part of the drone 120.
  • the camera 1064 may be fixedly connected to the body of the drone 120.
  • the UAV 120 also includes a transmission device 108. Under the control of the controller 1042, the transmission device 108 can send data collected by the sensing system 1044 and/or the camera 1064 to the control terminal 110.
  • the control terminal 110 may include a transmission device (not shown), the transmission device of the control terminal may establish a wireless communication connection with the transmission device 108 of the drone 120, and the transmission device of the control terminal may receive data sent by the transmission device 108.
  • control The terminal 110 may also send a control instruction to the drone 120 through a transmission device configured by itself.
  • the control terminal 110 may include a control device 1102 and a display device 1104.
  • the control device 1102 can control various operations of the control terminal.
  • the control device 1102 can control the transmission device to receive the data sent by the drone 120 through the transmission device 108; for another example, the control device 1102 can control the display device 1104 to display the sent data, where the data can include the data captured by the camera 1064. Environment image, posture information, location information, power information, etc.
  • control terminal in the foregoing part may include one or more processors, where the one or more processors may work individually or in cooperation.
  • FIG. 2 is a flowchart of a multi-load and multi-channel image transmission method provided by an embodiment of the application.
  • the multi-load multi-channel image transmission method described in this embodiment can be applied to a control terminal. As shown in Figure 2, the method in this embodiment may include:
  • the selection instruction includes the selected load type, and the selection instruction is sent to the drone, so that the drone opens a corresponding communication port according to the load type.
  • the execution subject of the foregoing multi-load and multi-channel image transmission method may be a remote control device.
  • the remote control device may be a component of the control terminal, that is, the control terminal includes a remote control device.
  • a part of the remote control device can be installed on the control terminal, and a part of the remote control device can be installed on the drone.
  • the control terminal also includes a display device, which is electrically connected with the remote control device to display a corresponding program interface, wherein the display device may be a touch display device.
  • the remote control device and the display device can be separated and communicate via wired or wireless means, or they can be integrated.
  • UAVs usually have multiple loads. In order to facilitate user manipulation and timely access to information, it is often necessary to transmit the image transmission data of multiple loads to users in real time. In this case, multiple loads on the current drone need to be selected. Select the image data of multiple loads required by the user to transmit to the control terminal. As mentioned earlier, by responding to the selection instructions of multiple loads, the drone sends a selection instruction to turn on the loads in the drone, that is, multiple loads are selected so as to transmit the data of these selected loads to the control.
  • the selection instruction includes the selected load type, so that it is convenient to select the load; the selection instruction is sent to the drone, so that the drone opens the corresponding communication port according to the load type, so that the selected multiple loads ( That is, the data transmission channel corresponding to the load) is opened, so that the selected load can transmit data to the control terminal through the corresponding communication port.
  • the selected multiple loads That is, the data transmission channel corresponding to the load
  • the selected load can transmit data to the control terminal through the corresponding communication port.
  • the load type is the port model, interface parameters, or other type judgment parameters of the load.
  • the selection instruction can be directly sent to the drone, or forwarded to the drone through a server, so as to improve the convenience of drone control.
  • S204 Receive a plurality of image data sent back by the UAV corresponding to the selection instruction through the corresponding communication port.
  • a plurality of image data obtained by the load corresponding to the selection instruction on the drone sent by the drone to the control terminal are received, and these image data are transmitted through the communication port corresponding to the corresponding load, that is, various image data Transmission through the corresponding dedicated channel is convenient to improve transmission efficiency and transmission accuracy, reduce interference, and other communication ports that are not related to the corresponding load are not opened, which can also avoid resource waste.
  • the multiple image data it is convenient for the user to understand the flight status of the drone and the surrounding environment of the drone from multiple angles in real time, thereby facilitating the user according to the flight status of the drone and Environmental conditions, more precise and reliable control of the drone, or timely processing of the multiple image data feedback information to deal with various events in the environment where the drone is located, thereby increasing the use of the drone Convenience and timeliness.
  • the multi-load multi-channel image transmission method provided by the embodiment of the present application, by opening the corresponding communication ports for the multiple loads, and transmitting the multiple image data obtained by the multiple loads through the corresponding communication ports, multiple data is realized.
  • Real-time transmission allows users to obtain real-time image data of multiple loads from the drone, which greatly improves the convenience of the drone system.
  • the communication port corresponding to the load can also be used to reduce interference , Improve the stability and reliability of data transmission.
  • the method further includes: opening the image transmission channel corresponding to the load type in the selection instruction to receive multiple image data.
  • the data provided by different loads are different, the corresponding communication ports are different, and the corresponding image transmission channels are also different. That is, multiple loads have their own corresponding communication ports and image transmission channels.
  • dedicated It is dedicated to the road, which can ensure the accuracy and reliability of data transmission, avoid sending wrong data and affect the user's use, and improve the convenience and reliability of the UAV system.
  • the method further includes: decoding and displaying multiple image data.
  • data transmission is often not transmitted in the original form, but the data is edited through encoding or other methods to improve transmission speed and transmission reliability and stability.
  • the data may be in the form of some codes or various optical signals, electrical signals, etc., which the user cannot directly read and use.
  • the received multiple image data It is decoded and displayed, which is convenient for users to read and use, and improves the convenience of using image data.
  • the video data can be encoded into H264, MPEG4 or MPEG2 video streams.
  • the number of the image data is two, and the two image data are respectively displayed on the display screen of the control terminal.
  • the APP software and the screen can only display one or two channels of data when displaying, the number of image data is set to two channels.
  • two channels of data can be transmitted at the same time.
  • the data method can also effectively improve bandwidth utilization and improve the video clarity of image transmission, especially at the edge of low signal, which can provide better image transmission quality, thereby improving the accuracy of data obtained by users.
  • the APP will send the load type expected to be dual-channel image transmission to the aircraft.
  • the aircraft receives the message and responds, it opens the image transmission channel corresponding to the camera type and performs data transmission.
  • the remote control receives the image transmission data, and transmits the image transmission data to the APP in the form of transparent transmission
  • the method further includes: determining the code rate of each image data after decoding; and passing the image data with the highest code rate among all the image data through the display screen Display the other image data in all the image data through at least one secondary display area of the display screen.
  • the higher the bit rate the smaller the compression ratio of image data, the less distortion, and the clearer the image. Therefore, the image data with the highest bit rate among all the image data is processed through the main display area of the display screen. Display, which is convenient for the user to accurately obtain the image information and improve the convenience for the user to view the image information; other image data is displayed through at least one secondary display area of the display screen, so that the user can obtain this part of the image data. Since the area of the secondary display area is generally smaller than the area of the main display area, even if the image is somewhat distorted, the clarity of the image will not be greatly affected due to the small display area, thereby enhancing the viewing comfort of the user.
  • the method further includes: in response to the control instruction for switching display, displaying the image data in the secondary display area corresponding to the control instruction in the main display area.
  • image data with a high bit rate is not necessarily the image data that the user wants to view most.
  • the image data in the secondary display area corresponding to the control command is displayed in the main display area, adding
  • the original display area of the image data in the secondary display area improves the convenience and comfort of viewing by the user, and is especially beneficial for the user to obtain some detailed information.
  • the multiple loads may be video acquisition devices (for example, cameras), light source devices (for example, lighting), or radio devices (for example, microphones).
  • video acquisition devices for example, cameras
  • light source devices for example, lighting
  • radio devices for example, microphones
  • sending the selection instruction to the drone specifically includes: receiving the selection instruction through the display device of the control terminal; and sending the selection instruction from the display device to the remote control device of the control terminal; The selection instruction is sent to the drone through the remote control device.
  • the image data of each load is different, and the selection instruction is received through the display device, that is, the user can directly select the image data displayed on the display device to form the selection instruction. This operation is straightforward.
  • the convenience of the UAV system used by the user is improved, and the selection instruction is sent from the display device to the remote control device of the control terminal through the selection instruction; the selection instruction is sent to the unmanned aerial vehicle through the remote control device In this way, the accuracy of the selection instruction can be ensured, thereby ensuring that the image data finally transmitted from the drone to the user is the image data that the user wants to view, avoiding erroneous transmission.
  • the remote control device can directly send a selection instruction to the drone, or indirectly send a selection instruction to the drone through a server relay.
  • the above process can be realized through the top flow of Figure 3, that is, the user sends a message to open the two loads selected by the user on the APP side (ie, the display device) and transparently transmits to the drone through the remote control side (ie, the remote control device) Side so that the drone will map.
  • the APP side ie, the display device
  • the remote control side ie, the remote control device
  • receiving multiple image data sent by the UAV corresponding to the selection instruction through the corresponding communication port includes: receiving multiple image data through the remote control device of the control terminal. Image data; send the image data from the remote control device to the display device of the control terminal; perform a decoding operation on the image data through the display device, and display the decoded image data through the display device.
  • multiple image data are transmitted in the form of encoding during the data transmission process, that is, multiple image data captured by the load are first stored in the video stream and stored in the video stream.
  • the video stream Before sending to the control terminal, the video stream is decoded and re-encoded according to the bandwidth, so that the re-encoded multiple image data is sent to the control terminal. Therefore, after the remote control device receives multiple image data transmitted in encoded form, the The plurality of image data is sent by the remote control device to the display device of the control terminal; the decoding operation is performed on the plurality of image data through the display device, so that these codes can be intuitively and directly recorded The image in the same form is displayed to the user through the display device, which improves the convenience of using the UAV system.
  • the remote control device and the display device can be connected in a wired or wireless manner, and the remote control device and the display device can also be integrated.
  • the image data can be transmitted through transparent transmission between the payload and the drone body. At this time, there is no need to encode first and then decode on the drone body.
  • the drone body acquires multiple image data of the payload Then, directly according to the bandwidth of the image transmission channel corresponding to the load, determine the coding rate according to the bandwidth, and then encode and transmit a plurality of the image data to the server or the control terminal.
  • FIG. 5 is a flowchart of another multi-load and multi-channel image transmission method provided in an embodiment of the application.
  • the multi-load and multi-channel image transmission method described in this embodiment can be applied to drones; as shown in FIG. 5, the multi-load and multi-channel image transmission method in this embodiment includes:
  • S302 Receive a selection instruction sent by the control terminal, where the selection instruction includes the selected load type.
  • the execution subject of the multi-load and multi-channel image transmission method in this embodiment may be a drone.
  • the drone includes the drone body and multiple loads set on the drone body.
  • the bandwidth is limited, so it is impossible to transmit all the loads simultaneously. Therefore, it is necessary to select among multiple loads, that is, by receiving the selection instruction sent by the control terminal, select a part of the multiple loads for image transmission; the selection instruction includes the selected load type, which is convenient for narrowing the selection range and improving the selection Accuracy and reliability.
  • the selection instruction can be directly issued by the control terminal, or after the control terminal is sent to the server, and then forwarded to the drone by the server, in other words, the drone can receive the instruction directly sent by the control terminal, or it can receive Through the instructions forwarded by the server, the convenience and flexibility of drone control are improved.
  • the multiple loads may be video acquisition devices (for example, cameras), light source devices (for example, lighting), or radio devices (for example, microphones).
  • video acquisition devices for example, cameras
  • light source devices for example, lighting
  • radio devices for example, microphones
  • this instruction will carry the user to select the type of camera/load that needs to be turned on, and the instruction is received by the drone.
  • the command sent by the APP is passed to the remote control, and the remote control sends the command to the server through the wireless network module; the server then transfers the command to the drone.
  • the corresponding image transmission channel is determined according to the load type in the selection instruction, that is, different image transmission channels are provided for different load types, and the image data is transmitted through the image transmission channel corresponding to the corresponding load, that is, various image data Transmission through the corresponding dedicated channel facilitates the improvement of transmission efficiency and accuracy of transmission, reduces interference, and avoids sending wrong data and affecting user use.
  • the image data collected by multiple loads can be combined first, and then the combined image data can be transmitted, or The image data collected for each load is transmitted separately.
  • the drone After the drone receives the command, it searches for the matching camera/load type from the currently mounted cameras/loads; after finding the matching camera/load type, it opens the corresponding image transmission channel, and closes and releases other loaded cameras. Some image transmission channel resources to reduce resource waste.
  • S306 Send the image data of the corresponding load to the control terminal through the image transmission channel.
  • the image data of the corresponding load is sent to the control terminal through the image transmission channel, that is, the image data of the corresponding load is transmitted through the dedicated channel corresponding to it, which facilitates the improvement of transmission efficiency and transmission accuracy. Reduce interference and avoid sending wrong data to affect user use.
  • the method further includes: opening the image transmission channel for transmitting the image according to the selection instruction; and determining the bandwidth of the preset network mode corresponding to the image transmission channel. Specifically, by determining the bandwidth of the preset network mode corresponding to the image transmission channel, it is convenient to select a matching bit rate according to the bandwidth, so as to use the bit rate to encode the image data to be transmitted, so as to facilitate the preset network mode Perform transmission, improve transmission efficiency and transmission accuracy, and realize the best bandwidth adaptation mechanism, so that there will be no blurring in image transmission; if it is non-load data, the bandwidth feedback mechanism can be implemented to transparently transmit the data.
  • the method further includes: determining the encoding bit rate according to the bandwidth of the preset network mode; encoding the video stream at the bit rate to generate image data; and sending the image data to the control terminal.
  • the bandwidth of the network transmission channel has a great impact on the stability and reliability of data transmission, and the network bandwidth may change significantly due to network abnormalities.
  • a bandwidth adaptive encoding before sending that is, first Determine the bandwidth of the transmission channel of the preset network mode of the corresponding load, thereby facilitating the determination of a suitable bit rate according to the bandwidth, and encoding the video stream obtained through the corresponding load according to the bit rate to generate image data Therefore, it is more conducive to data transmission in the preset network mode, and the reliability and stability of transmission are improved.
  • an on-demand-i (on-demand-i frame request) encoding method is used to achieve a higher bit rate compression ratio, thereby bringing higher image transmission definition.
  • the encoding bit rate before determining the encoding bit rate according to the bandwidth of the preset network mode, it also includes: encrypting the video stream data of the payload, and transmitting the encrypted video stream data from the payload to the drone body;
  • the encrypted video stream data is decoded.
  • the video stream data is encrypted, and the encrypted video stream data is transferred from the payload to the drone body, which is beneficial to protect the video stream data, avoid leakage, and improve the security of the data;
  • the body decodes the encrypted video stream data, which is beneficial for the drone to recognize the video stream data and improve work efficiency.
  • the drone body receives the video stream data of the load through a USB virtual network card. Specifically, the drone body receives video stream data through a USB virtual network card, which is convenient to install and use, and has strong confidentiality.
  • bit rate of the data stream for image transmission needs to be smaller than the actual bandwidth.
  • the drone receives the H264 video stream data loaded by the camera through the USB virtual network card
  • the drone After receiving the data, the drone decodes the H264 video stream to obtain a decoded image.
  • the UAV obtains the bandwidth of the wireless network. (The radio frequency module of the wireless network is placed in a shared memory after the calculation is completed)
  • the UAV sends the decoded image data to the encoder for video encoding.
  • an H264 video stream is formed, which is a video stream that matches the current bandwidth size.
  • the UAV sends the encoded H264 video stream to the remote control through the wireless network channel.
  • the method further includes: judging the usage of the load, and determining the image transmission channel corresponding to the load according to the usage.
  • the use of the load includes two situations: mounting and unloading.
  • determining the image transmission channel corresponding to the load according to the use situation includes: when the use situation is in the mounted state, determining and opening the port corresponding to the load through the preset database, so as to establish the image transmission channel through the port. Transmission of image data. Specifically, as mentioned earlier, in the mounted state, the load can perform image transmission. Therefore, the port corresponding to the load is searched and opened through the preset database to establish the image transmission channel through the port to realize the transmission of image data, thereby achieving The picture transmission of the load avoids the situation that the picture transmission cannot be realized due to the arbitrary opening of the port, and the work efficiency is improved.
  • the method further includes: closing the port corresponding to the load when the use condition is an unloaded state. Specifically, when the usage is in the offload state, that is, when the load cannot perform image transmission, the port corresponding to the load is closed, thereby avoiding invalid transmission and improving work efficiency.
  • the method further includes: judging whether the port corresponding to the load is in an available state, and generating a judgment result; when the judgment result is no, releasing the port corresponding to the load.
  • data transmission can only be performed when the port is available. Therefore, when the load is in the mounted state, it is necessary to determine whether the port corresponding to the load is available. If it is available, it can be used for image transmission. If it is not available, the judgment result is no. , The port needs to be released for image transmission.
  • determining whether the port corresponding to the load is in the available state includes: determining the port address where the load is mounted; determining whether there is any remaining load on the port address, and determining whether the port corresponding to the load does not belong to the remaining load on the port address Available status.
  • the corresponding port may also correspond to other loads.
  • releasing the port corresponding to the load specifically includes: releasing the matching relationship between the remaining loads and the port address, so that the port address is in an idle state. Specifically, by removing the matching relationship between the remaining loads and the port address, the remaining loads no longer occupy the port, so that the port address can be matched with the load in the mounted state, and the image transmission of the load in the mounted state is realized. It is helpful to solve the port conflict problem of two identical loads hanging on the same UAV.
  • the UAV can currently mount at most 2 payloads and 1 fixed FPV (ie, first-person view, used for real-time image acquisition) cameras at the same time.
  • FPV first-person view, used for real-time image acquisition
  • the center board module is the most direct module connected to the camera; the center board module will push the currently mounted camera type to the image transmission module at a fixed frequency.
  • the image transmission module receives the changed camera type, and judges whether the current camera is mounted or unmounted by looking up the table
  • the method further includes: obtaining a load category; when the number of opened loads is greater than the number of image transmission channels, determining the right to use the image transmission channel according to the priority of the load category.
  • the number of image transmission channels is often smaller than the number of loads. Therefore, by obtaining the load category, the priority of the load category can be effectively determined, and the right to use the image transmission channel can be determined according to the priority. Solve the problem of multiple loads sharing one image transmission channel.
  • the load category includes FPV load or application load, or in other words, is the load of taking pictures or application load.
  • the load categories are real-time image acquisition load (that is, the aforementioned FPV load) and application load.
  • the load categories are real-time image acquisition load and application-type load.
  • the priority of application load is higher than the priority of real-time image acquisition load.
  • the data provided by application loads are more objective, and the feedback information has more reference value; and the data of such loads are generally not image data, and they require less storage space and occupy less bandwidth during transmission. Faster transmission speed is conducive to users to quickly obtain data and improve the real-time nature of information transmission.
  • FIG. 7 is a structural diagram of the control system provided by the embodiment of the present application.
  • the control system described in this embodiment includes a control terminal and an unmanned aerial vehicle.
  • the control terminal responds to a plurality of load selection instructions, the selection instruction includes the selected load type, and sends the selection instruction to the unmanned aerial vehicle.
  • the drone opens the corresponding communication port according to the load type; the drone determines the corresponding image transmission channel according to the load type in the selection instruction; the drone sends the corresponding image data of the load to the control terminal through the image transmission channel to make the control terminal Display image data.
  • control system further includes: a server, which is electrically connected to the control terminal and the drone, respectively, and the server is used for receiving selection instructions and sending the selection instructions to the drone.
  • the server is also used to receive image data sent by the drone, and send the image data to the control terminal.
  • FIG. 8 is a structural diagram of the control terminal 1000 provided in an embodiment of the present application.
  • the control terminal 1000 of this embodiment includes a remote control device 1002, a display device 1004, and a processor 1006.
  • the processor 1006 is configured to respond to multiple load selection instructions, the selection instruction includes the selected load type, and the selection instruction is sent To the drone, so that the drone opens the corresponding communication port according to the load type; receives the multiple image data sent by the drone and the load corresponding to the selection instruction through the corresponding communication port; controls the display device to display multiple Image data.
  • the processor 1006 is further configured to: open an image transmission channel corresponding to the load type in the selection instruction to receive multiple image data.
  • the processor 1006 is further configured to: decode and display multiple image data.
  • the number of image data is two, and the two image data are respectively displayed on the display screen of the control terminal.
  • the processor 1006 is further configured to: determine the decoded bit rate of each image data; display the image data with the highest bit rate among all the image data through the main display area of the display; and display the other images in all the image data.
  • the data is displayed through at least one secondary display area of the display screen.
  • the processor 1006 is further configured to: in response to the control instruction for switching display, display the image data in the secondary display area corresponding to the control instruction in the main display area.
  • the processor 1006 is configured to send a selection instruction to the drone, specifically to: receive the selection instruction through the display device 1004 of the control terminal 1000; send the selection instruction from the display device 1004 to the remote control device 1002 of the control terminal 1000 ; Send the selection instruction to the drone through the remote control device 1002.
  • the processor 1006 is configured to receive multiple image data sent back by the drone through the corresponding communication port corresponding to the load corresponding to the selection instruction, and is specifically configured to: receive multiple images through the remote control device 1002 of the control terminal 1000 Data; the image data is sent from the remote control device 1002 to the display device 1004 of the control terminal 1000; the image data is decoded through the display device 1004, and the decoded image data is displayed on the display device 1004.
  • FIG. 9 is a structural diagram of the unmanned aerial vehicle 2000 provided in an embodiment of the application.
  • the drone 2000 of this embodiment includes a drone body 2002, a plurality of loads 2004 arranged on the drone body 2002, and a processor 2006, wherein the processor 2006 is used to: receive a selection instruction sent by a control terminal, The selection instruction includes the selected load type; the corresponding image transmission channel is determined according to the load type in the selection instruction; the image data of the corresponding load is sent to the control terminal through the image transmission channel.
  • the processor 2006 is further configured to: turn on the image transmission channel for transmitting the image according to the selection instruction; and determine the bandwidth of the preset network mode corresponding to the image transmission channel.
  • the processor 2006 is further configured to: determine the code rate of encoding according to the bandwidth of the preset network mode; encode the video stream at the code rate to generate image data; and send the image data to the control terminal.
  • the processor 2006 determines the encoding bit rate according to the bandwidth of the preset network mode, the processor 2006 is also used to: encrypt the video stream data of the payload 2004, and transmit the encrypted video stream data from the payload 2004 to the unmanned On the body 2002; the drone body 2002 decodes the encrypted video stream data.
  • the drone body 2002 receives the video stream data of the payload 2004 through a USB virtual network card.
  • the processor 2006 is further configured to: determine the usage of the load 2004, and determine the image transmission channel corresponding to the load 2004 according to the usage.
  • the processor 2006 determines the image transmission channel corresponding to the load 2004 according to the use situation, which is specifically used to determine and open the port corresponding to the load 2004 through the preset database when the use situation is in the mounted state, so as to pass The port establishes an image transmission channel to realize the transmission of image data.
  • processor 2006 is further configured to: close the port corresponding to the load 2004 when the usage is in the unloaded state.
  • the processor 2006 is further configured to: determine whether the port corresponding to the load 2004 is in an available state, and generate a judgment result; when the judgment result is no, release the port corresponding to the load 2004 .
  • the processor 2006 determines whether the port corresponding to the load 2004 is in an available state, and is specifically used to: determine the port address mounted by the load 2004; determine whether there are other loads on the port address, and when there are other loads on the port address, determine whether The port corresponding to the load 2004 is not in the available state.
  • the processor 2006 releases the port corresponding to the load 2004, which is specifically used to release the matching relationship between the remaining load 2004 and the port address, so that the port address is in an idle state.
  • the processor 2006 is further configured to: obtain a load category; when the number of opened loads is greater than the number of image transmission channels, determine the right to use the image transmission channel according to the priority of the load category.
  • load categories are real-time image acquisition load 2004 and application load 2004.
  • the priority of application load is higher than the priority of real-time image acquisition load.
  • the embodiment of the present application provides a server, including a processor, the processor is configured to: receive a selection instruction sent by a control terminal, the selection instruction includes the selected load type, and send the selection instruction to the drone so that the drone can follow
  • the load type opens the corresponding communication port; the corresponding image transmission channel is determined according to the load type in the selection instruction, the image data sent by the drone is received through the image transmission channel, and the image data is sent to the control terminal.
  • the embodiment of the present application provides a computer-readable storage medium on which a computer program is stored.
  • the steps and implementations of the multi-load multiple image transmission method in the above-mentioned first aspect are implemented.
  • any process or method description in the flowchart or described in other ways herein can be understood as including one or more executable instructions for implementing specific logical functions or steps of the process. Modules, fragments, or parts of the code, and the scope of the preferred embodiment of the present application includes additional implementations, which may not be in the order shown or discussed, including in a substantially simultaneous manner or in reverse according to the functions involved Order to perform functions, which should be understood by those skilled in the art to which the embodiments of the present application belong.
  • a "computer-readable medium” can be any device that can contain, store, communicate, propagate, or transmit a program for use by an instruction execution system, device, or device or in combination with these instruction execution systems, devices, or devices.
  • computer readable media include the following: electrical connections (electronic devices) with one or more wiring, portable computer disk cases (magnetic devices), random access memory (RAM), Read only memory (ROM), erasable and editable read only memory (EPROM or flash memory), fiber optic devices, and portable compact disk read only memory (CDROM).
  • the computer-readable medium may even be paper or other suitable medium on which the program can be printed, because it can be done, for example, by optically scanning the paper or other medium, and then editing, interpreting, or other suitable methods when necessary. Process to obtain the program electronically and then store it in the computer memory.
  • each part of this application can be implemented by hardware, software, firmware or a combination thereof.
  • multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system.
  • a logic gate circuit for implementing logic functions on data signals
  • PGA programmable gate array
  • FPGA field programmable gate array
  • the functional units in the various embodiments of the present application may be integrated into one processing module, or each unit may exist alone physically, or two or more units may be integrated into one module.
  • the above-mentioned integrated modules can be implemented in the form of hardware or software functional modules. If the integrated module is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer readable storage medium.
  • the storage medium mentioned above may be a read-only memory, a magnetic disk or an optical disk, etc.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Selective Calling Equipment (AREA)
  • Closed-Circuit Television Systems (AREA)

Abstract

本申请提供了一种多负载多路图传方法、控制系统和终端、无人机和服务器，其中，多负载多路图传方法包括：响应于多个负载的选择指令，选择指令包括选择的负载类型，将选择指令发送至无人机，以使无人机根据负载类型开启对应的通信端口(S202)；接收由无人机发送的与选择指令对应负载通过对应的通信端口传回的多个图像数据(S204)；显示多个图像数据(S206)。根据本申请的技术方案，通过简化链路过程，解除了数据只能单路图传的限制，实现图传数据多路实时传输，为无人机在行业应用的工作场景提供了更多的可能性。

Description

多负载多路图传方法、控制系统和终端、无人机和服务器 技术领域
本申请涉及一种控制技术领域，尤其涉及一种多负载多路图传方法、一种控制系统、一种控制终端、一种无人机和一种服务器。
背景技术
在行业无人机产品中，往往有多负载的使用需求，这些负载可以是相机类负载，也可以是数据采集类负载。在飞行过程中，这些负载往往需要把采集到的图传、数据实时回传到操控手的设备上，在当前的无线网络图传技术中，受限于无线网络的通讯带宽和通讯质量，往往都只有单路图传能力，且图传数据格式单一，无法带来多负载相机情况下多个负载同步实时的图传显示，一些基于无线网络的双路图传技术，虽然可以进行双路图传，但是存在几个问题点：
1.相机数据的接收通道不能自适应打开，会把可能的相机类型的接收端口都打开，浪费系统资源。
2.无线带宽低，双路图传过程中，图传视频流码率低，视频清晰度低。
3.带宽自适应传输过程中，当前网络带宽情况会通过消息发送到相机负载侧，相机负载进行码率匹配并把匹配带宽的图传数据传给无人机，无人机进行透传。在这个过程中，带宽反馈链路太长，会造成实际数据传输到无线网卡节点的时候，网络已经发生变化，造成图传数据花屏。
4.图传本质上是把飞机上的多个负载数据都往遥控器传，但实际上app和大屏在显示的时候只能显示其中的1路或者2路，造成无线网络带宽的浪费，进一步降低了视频清晰度，特别是在低信号边缘，会带来比较差的图传质量。
发明内容
本申请实施例提供一种多负载多路图传方法、控制系统、控制终端、无人机和服务器，以解除数据只能单路图传的限制，实现图传数据多路实 时传输，为无人机在行业应用的工作场景提供了更多的可能性。
为了实现上述目的，本申请实施例的第一方面提供了一种多负载多路图传方法，用于控制终端，包括：
响应于多个负载的选择指令，选择指令包括选择的负载类型，将选择指令发送至无人机，以使无人机根据负载类型开启对应的通信端口；
接收由无人机发送的与选择指令对应负载通过对应的通信端口传回的多个图像数据；
显示多个图像数据。
本申请第二方面的技术方案提供了一种多负载多路图传方法，用于无人机，无人机包括无人机本体以及设于无人机本体上的多个负载，多负载多路图传方法包括：
接收由控制终端发送的选择指令，选择指令包括选择的负载类型；
根据选择指令中的负载类型确定对应的图传通道；
通过图传通道将对应的负载的图像数据发送至控制终端。
本申请第三方面的技术方案提供了一种控制系统，包括控制终端和无人机，其中，
控制终端响应于多个负载的选择指令，选择指令包括选择的负载类型，将选择指令发送至无人机，以使无人机根据负载类型开启对应的通信端口；
无人机根据选择指令中的负载类型确定对应的图传通道；
无人机通过图传通道将对应的负载的图像数据发送至控制终端，以使控制终端显示图像数据。
本申请的第四方面的技术方案提供了一种控制终端，包括：遥控装置、显示装置和处理器，其中，处理器，用于：
响应于多个负载的选择指令，选择指令包括选择的负载类型，将选择指令发送至无人机，以使无人机根据负载类型开启对应的通信端口；
接收由无人机发送的与选择指令对应负载通过对应的通信端口传回的多个图像数据；
控制显示装置显示多个图像数据。
本申请的第五方面的技术方案提供了一种无人机，包括无人机本体、设于无人机本体上的多个负载以及处理器，其中，处理器用于：
接收由控制终端发送的选择指令，选择指令包括选择的负载类型；
根据选择指令中的负载类型确定对应的图传通道；
通过图传通道将对应的负载的图像数据发送至控制终端。
本申请的第六方面的技术方案提供了一种服务器，包括处理器，处理器用于：
接收由控制终端发送的选择指令，选择指令包括选择的负载类型，并将选择指令发送至无人机，以使无人机根据负载类型开启对应的通信端口；
根据选择指令中的负载类型确定对应的图传通道，通过图传通道接收无人机发送的图像数据，并将图像数据发送至控制终端。
本申请的第七方面的技术方案提供了一种计算机可读存储介质，其上存储有计算机程序，计算机程序被处理器执行时实现如本申请实施例第一方面提供的多负载多路图传方法的步骤、实现如本申请实施例第二方面提供的多负载多路图传方法的步骤。
本申请实施例提供的多负载多路图传方法、控制系统、控制终端、无人机和服务器，通过简化链路过程，可以实现：
1.多路数据(至少为两路)进行实时传输。
2.会自动识别当前无人机挂载的负载类设备，并打开对应的数据接收模块，避免浪费系统资源，还有利于增加相机负载的类型并提高对相机负载的适配能力。
3.根据网络状态的带宽，其中主路图传能够利用on-demand-i(按需请求I帧)的编码方式，实现更高码率压缩比，带来更高的图传清晰度。
4.从多个(至少3个)负载类设备中自动选择两路数据进行实时传输。在这两路的实时传输中，如果是视频类的数据，会进行重编码传输，重编码的视频码率匹配当前带宽；可实现最佳的带宽自适应机制，以避免主路图传出现花屏的情况发生，此外，若传输的是非负载类设备的数据，可通过带宽反馈机制，对数据进行透传，以减少对系统资源的过多占用。
5.从多个负载中选择2路进行图传，可以有效提高带宽利用率，提高两路图传的图传质量。
本申请的附加方面和优点将在下面的描述部分中变得明显，或通过本申请的实践了解到。
附图说明
为了更清楚地说明本申请实施例中的技术方案，下面将对实施例描述中所需要使用的附图作一简单地介绍，显而易见地，下面描述中的附图是本申请的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动性的前提下，还可以根据这些附图获得其他的附图。
图1示出了本申请实施例的无人机系统的示意架构框图；
图2示出了本申请实施例的运行在控制终端上的多负载多路图传方法的流程示意图；
图3示出了本申请一个具体实施例的两路图像传输的流程示意图；
图4示出了本申请一个具体实施例的打开图传通道并进行数据传输的流程示意图；
图5示出了本申请实施例的运行在无人机上的多负载多路图传方法的流程示意图；
图6示出了本申请一个具体实施例的负载端口启闭机制的流程示意图；
图7示出了本申请实施例的控制系统的结构框图；
图8示出了本申请实施例的控制终端的结构图；
图9示出了本申请实施例的无人机的结构图。
具体实施方式
下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚地描述，显然，所描述的实施例仅仅是本申请一部分实施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本申请保护的范围。
需要说明的是，当组件被称为“固定于”另一个组件，它可以直接在另一个组件上或者也可以存在居中的组件。当一个组件被认为是“连接”另一个组件，它可以是直接连接到另一个组件或者可能同时存在居中组件。
除非另有定义，本文所使用的所有的技术和科学术语与属于本申请的技术领域的技术人员通常理解的含义相同。本文中在本申请的说明书中所使用的术语只是为了描述具体的实施例的目的，不是旨在于限制本申请。本文所使用的术语“及/或”包括一个或多个相关的所列项目的任意的和所有的组合。
下面结合附图，对本申请的一些实施方式作详细说明。在不冲突的情况下，下述的实施例及实施例中的特征可以相互组合。
下面结合附图，对本申请的一些实施方式作详细说明。在不冲突的情况下，下述的实施例及实施例中的特征可以相互组合。
图1为根据本申请实施例提供的无人机系统10的示意性架构图。所述无人机系统10可以包括无人机的控制终端110和无人机120。其中，所述无人机120可以单旋翼或者多旋翼无人机。
无人机120可以包括动力系统102、控制系统104和机身。其中，当无人机120具体为多旋翼无人机时，机身可以包括中心架以及与中心架连接的一个或多个机臂，一个或多个机臂呈辐射状从中心架延伸出。无人机还可以包括脚架，其中，脚架与机身连接，用于在无人机着陆时起支撑作用。
动力系统102可以包括一个或多个电机1022，电机1022用于为无人机120提供动力，该动力使得无人机120能够实现一个或多个自由度的运动。
控制系统104可以包括控制器1042和传感系统1044。传感系统1044用于测量无人机120的状态信息和/或无人机120所处的环境的信息，其中，所述状态信息可以包括姿态信息、位置信息、剩余电量信息等。所述环境的信息可以包括环境的深度、环境的气压、环境的湿度、环境的温度等等。其中，传感系统1044例如可以包括气压计、陀螺仪、超声传感器、电子罗盘、惯性测量单元、视觉传感器、全球导航卫星系统和气压计等传感器中的至少一种。例如，全球导航卫星系统可以是全球定位系统(Global Positioning System，GPS)。
控制器1042用于控制无人机的各种操作。例如，控制器1042可以控制无人机的移动，再例如，控制器1042可以控制无人机的传感系统1044采集数据。
在某些实施例中，无人机120可以包括拍摄装置1064，拍摄装置1064例如可以是照相机或摄像机等用于捕获图像的设备，拍摄装置1064可以与控制器1042通信，并在控制器1042的控制下进行拍摄，控制器1042也可以根据拍摄装置1064拍摄的图像控制无人机120。
在某些实施例中，无人机120还包括云台106，云台106可以包括电 机1062，云台106用于携带拍摄装置1064，控制器1042可以通过电机控制云台106的运动。应理解，云台106可以独立于无人机120，也可以为无人机120的一部分。在某些实施例中，所述拍摄装置1064可以固定连接在无人机120的机身上。
无人机120还包括传输设备108，在控制器1042的控制下，所述传输设备108可以将传感系统1044和/或拍摄装置1064采集的数据发送到控制终端110。控制终端110可以包括传输设备(未示出)，控制终端的传输设备可以与无人机120的传输设备108建立无线通信连接，控制终端的传输设备可以接收传输设备108发送的数据，另外，控制终端110还可以通过自身配置的传输设备向无人机120发送控制指令。
控制终端110可以包括控制装置1102和显示装置1104。控制装置1102可以控制控制终端的各种操作。例如，控制装置1102可以控制传输设备接收无人机120通过传输设备108发送的数据；再例如，控制装置1102可以控制显示装置1104显示发送的数据，其中，所述数据可以包括拍摄装置1064捕捉的环境的图像、姿态信息、位置信息、电量信息等等。
可以理解的是，前述部分的控制终端可以包括一个或多个处理器，其中，所述一个或多个处理器可以单独地或者协同地工作。
应理解，上述对于无人机系统各组成部分的命名仅是出于标识的目的，并不应理解为对本申请的实施例的限制。
本申请实施例提供一种多负载多路图传方法。图2为本申请实施例提供的多负载多路图传方法的流程图。本实施例所述的多负载多路图传方法可应用于控制终端。如图2所示，本实施例中的方法，可以包括：
S202，响应于多个负载的选择指令，所述选择指令包括选择的负载类型，将所述选择指令发送至无人机，以使所述无人机根据所述负载类型开启对应的通信端口。
具体地，上述多负载多路图传方法的执行主体可以是遥控装置。其中，遥控装置可以是控制终端的部件，即控制终端包括遥控装置。在某些情况中，遥控装置的一部分部件可以设置在控制终端上，遥控装置的一部分部件可以设置在无人机上。控制终端还包括显示装置，显示装置与遥控装置电连接，以显示对应的程序界面，其中，显示装置可以是触摸显示装置。还需说明的是，遥控装置和显示装置可以是分开的并通过有线或无线方式 通信，二者还可以是集成为一体的。
无人机通常存在多负载，为了便于用户操控和及时获取信息，往往需要将多个负载的图传数据实时传输给用户，在这个情况下，需要对当前无人机上的多个负载进行选择，选择用户需要的多个负载的图像数据往控制终端传输。如前所述，通过响应于多个负载的选择指令，向无人机发送开启无人机中负载的选择指令，即对多个负载进行选择，以便将这些被选中的负载的数据传送给控制终端；选择指令包括选择的负载类型，这样便于对负载进行选择；将选择指令发送至无人机，使所述无人机根据所述负载类型开启对应的通信端口，使选中的多个负载(即对应负载)的数据传输通道开启，从而便于被选择的负载将数据通过对应的通信端口传送给控制终端，同时，由于针对性地仅开启了与所述负载类型对应的通信端口，其它通信端口没有开启，避免了资源的浪费，有利于相机类负载的扩展。
可以理解地，负载类型为该负载的端口型号、接口参数或其余类型判断参数。
可选地，所述选择指令可以直接发送至无人机，也可以通过服务器转发至无人机，以提升无人机操控的便利性。
S204，接收由所述无人机发送的与选择指令对应负载通过对应的通信端口传回的多个图像数据。
具体地，接收由无人机向控制终端发送的无人机上与所述选择指令对应负载获取的多个图像数据，这些图像数据通过与对应负载相对应的通信端口进行传送，即各种图像数据通过对应的专用通道进行传输，便于提升传输效率和传输的准确性，减少干扰，且其它与对应负载不相关的通信端口没有开启，还可以避免资源浪费。
S206，显示所述多个图像数据。
具体地，通过显示所述多个图像数据，便于用户从多个角度实时而直观地了解到无人机的飞行情况和无人机的周边环境情况，从而便于用户根据无人机的飞行情况和环境情况，更加精确可靠地操控无人机，或者及时地针对所述多个图像数据反馈的信息进行处理，以应对无人机所在环境中的发生的各种事件，从而提升了无人机使用的便利性和及时性。
本申请实施例提供的多负载多路图传方法中，通过对多个负载开启对应的通信端口，并通过对应的通信端口将多个负载获取的多个图像数据进 行传送，实现了数据的多路实时传送，使得用户可以从无人机上获得多个负载的实时图像数据，大幅提升了无人机系统使用的便利性，且传输过程中，由于采用与负载对应的通信端口，还可以减少干扰，提升数据传输的稳定性和可靠性。
可选地，在多负载多路图传方法中，还包括：打开与选择指令中的负载类型对应的图传通道，以接收多个图像数据。具体地，不同的负载所提供的数据不同，对应的通信端口不同，对应的图传通道也不同，即多个负载中，都有各自对应的通信端口和图传通道，简而言之，专路专用，这样可以确保数据传输的准确性和可靠性，避免发送错误数据而影响用户使用，提升了无人机系统使用的便利性和可靠性。
可选地，在上述多负载多路图传方法中，还包括：对多个图像数据进行解码并显示。具体地，为了保密或者传输的方便，数据传输时往往不是以原有形态进行传输，而是通过编码或其它方式对数据进行编辑，以提升传输速度和传输的可靠性、稳定性，当控制终端接收到这些编辑后的数据时，这些数据的形式可能是一些代码或者各种光信号、电信号等等，用户无法直接读取和使用，此时，通过对接收到的所述多个图像数据进行解码并显示，从而便于用户读取和使用，提升了图像数据使用的便利性。
其中，对视频数据进行编码后可以形成为H264、MPEG4或MPEG2视频流。
可选地，在所述多负载多路图传方法中，所述图像数据的数量为两个，两个所述图像数据分别显示在所述控制终端的显示屏上。具体地，由于APP软件和屏幕在显示的时候只能显示其中的1路或者2路数据，因此将图像数据的数量设置为两路，一方面可以同时传送两路数据，相对于1路数据的图传方法，用户可以实时地获取到更详细的图像数据，避免无线网络带宽的浪费；另一方面，在显示时，两路图像数据显示在控制终端的显示屏上，相对于同时显示3路数据的方法，还可以有效提高带宽利用率，提升图传的视频清晰度，特别是在低信号边缘，能够提供较好的图传质量，从而提升用户获取的到的数据的准确性。
更具体地，选择两路负载进行图像传输，无人机上的处理框架如图3所示，在图3所示的过程中，比较关键的是图传切换开关模块，在无人机 刚开机，或者负载刚接上无人机的时候，会有如下默认处理过程：
(1)当负载A和负载B都没有连上的时候，FPV图像数据进行单路图传；
(2)当负载A和负载B其中有一个连上的时候，连上的负载A或者负载B和FPV相机进行组合，实现双路图传。
(3)当负载A和负载B两个都连上的时候，把负载A和负载B进行组合实现双路图传。
在用户实际使用过程中，会在APP上从三个相机源(FPV、1号云台相机、2号云台相机中)点击期望看到的任意两路相机数据。APP会把期望要进行双路图传的负载类型发送给飞机。飞机收到消息并进行相应，打开对应相机类型的图传通道并进行数据传输。
具体流程如下：
(1)APP发送用户选择的两个负载类型到无人机
(2)无人机收到负载类型后，和当前已经挂载的负载类型进行对比。对于这两个负载类型，分别对比。对消息包中的第一个负载，会选择打开主路图传，对消息包的第二个负载，会选择打开辅路图传。
(3)无人机上软件打开负载的无线通道后，把图传数据通过无线网络发送到遥控器
(4)遥控器收到图传数据，以透传形式把图传数据传给APP
(5)APP收到数据后进行解码(如果需要的话)并进行显示。
上述流程可通过图4进行更深入的理解。
可选地，在所述多负载多路图传方法中，还包括：确定每个所述图像数据解码后的码率；将所有所述图像数据中码率最高的图像数据通过所述显示屏的主显示区域进行显示；将所有所述图像数据中其它图像数据通过所述显示屏的至少一个副显示区域进行显示。具体地，码率越高，图像数据的压缩比例越小，失真越少，从而图像越清晰，因此，将所有所述图像数据中码率最高的图像数据通过所述显示屏的主显示区域进行显示，便于用户准确地获取所述图像信息，提升用户观看该图像信息的便利性；其它图像数据通过所述显示屏的至少一个副显示区域进行显示，是用户可以获 取这部分的图像数据，同时，由于副显示区域的面积一般都小于主显示区域的面积，这样，即使图像有些失真，由于显示面积较小，其清晰度也不会受到太大的影响，从而提升用户观看的舒适性。
进一步地，在多负载多路图传方法中，还包括：响应于切换显示的控制指令，将控制指令对应的副显示区域中的图像数据在主显示区域显示。具体地，在一些情况下，码率高的图像数据不一定是用户最想要查看的图像数据，当用户所需要查看的图像数据在副显示区域显示时，由于副显示区域的面积较小，查看不方便，尤其对于一些细节需要查看时，更加不便，此时，通过对切换显示的控制指令的响应，将所述控制指令对应的副显示区域中的图像数据在主显示区域显示，增加了原来在副显示区域中的图像数据的显示面积，提升了用户查看的便利性和舒适性，尤其有利于用户对于一些细节信息的获取。
其中，可以理解地，多个负载可以为视频获取设备(例如摄像头)，还可以为光源设备(例如打光灯)，还可以为收音设备(例如话筒)。
可选地，在多负载多路图传方法中，将选择指令发送至无人机，具体包括：通过控制终端的显示装置接收选择指令；将选择指令由显示装置发送至控制终端的遥控装置；通过遥控装置将选择指令发送至无人机。具体地，由于无人机有多个负载，每个负载的图像数据不同，通过显示装置接收选择指令，即用户可以直接选择显示装置上所显示的图像数据来形成选择指令，这样的操作直接而方便，提升了用户使用的无人机系统的便利性，且通过所述选择指令由所述显示装置发送至所述控制终端的遥控装置；通过所述遥控装置将所述选择指令发送至无人机，这样可以确保选择指令的准确性，进而保证最终从无人机传回给用户的图像数据正是用户所希望查看的图像数据，避免错误传输。可以理解地，遥控装置可以直接向无人机发送选择指令，还可通过服务器中转而间接地向无人机发送选择指令。
可以理解地，上述过程可通过图3顶部流程实现，即用户在APP侧(即显示装置)发送打开用户所选择的两个负载的消息通过遥控器侧(即遥控装置)透传至无人机侧，以使无人机将图。
可选地，在多负载多路图传方法中，接收由无人机发送的与选择指令 对应负载通过对应的通信端口传回的多个图像数据，具体包括：通过控制终端的遥控装置接收多个图像数据；将图像数据由遥控装置发送至控制终端的显示装置；通过显示装置对图像数据执行解码操作，并通过显示装置显示解码后的图像数据。具体地，如前所述，为了保密或者传输的方便，多个图像数据在数据传输过程中均以编码的形式进行传输，即负载拍摄到的多个图像数据先以视频流存储，在无人机发送至控制终端之前，根据带宽对视频流进行解码再编码，从而将再编码后的多个图像数据发送至控制终端，因此在遥控装置接收到以编码形式传输的多个图像数据后，将所述多个图像数据由所述遥控装置发送至所述控制终端的显示装置；通过所述显示装置对所述多个图像数据执行解码操作，以便于这些编码能够以直观地，与原始记录下的图像相同的形式，通过显示装置展现给用户，提升无人机系统使用的便利性。
其中，需要说明的是，遥控装置和显示装置之间可以之间通过有线或无线的方式连接，还可将遥控装置和显示装置设为一体。
可以理解，所述负载和无人机本体之间可以通过透传的方式传输图像数据，此时无需先编码，再在无人机本体上解码，当无人机本体获取负载的多个图像数据后，直接根据对应负载的图传通道的带宽，根据所述带宽确定编码的码率，然后对多个所述图像数据编码并传输给服务器或控制终端即可。
此外，也可通过所述负载和服务器或控制终端直接通信，根据对应负载的图传通道的带宽，根据所述带宽确定编码的码率，将图像数据编码后传输给服务器或控制终端即可。
本申请实施例提供了另一种多负载多路图传方法，图5为本申请实施例提供的另一种多负载多路图传方法的流程图。本实施例所述的多负载多路图传方法可应用于无人机；如图5所示，本实施例中所述多负载多路图传方法包括：
S302：接收由控制终端发送的选择指令，选择指令包括选择的负载类型。
具体地，本实施例中的多负载多路图传方法的执行主体可以是无人机。 其中，无人机包括无人机本体以及设于无人机本体上的多个负载，如前所述，无人机上有多个负载，而带宽有限，不可能对所有的负载同时进行图传，因此，需要在多个负载中进行选择，即通过接收由控制终端发送的选择指令，在多个负载中选择一部分负载进行图传；选择指令包括选择的负载类型，便于缩小选择范围，提升选择的准确性和可靠性。
可选地，选择指令可以由控制终端直接发出，也可以由控制终端发给服务器后，由服务器转发给无人机，或者说，无人机可以接收由控制终端直接发送的指令，也可接收通过服务器转发的指令，从而提升无人机控制的便利性和灵活性。
其中，可以理解地，多个负载可以为视频获取设备(例如摄像头)，还可以为光源设备(例如打光灯)，还可以为收音设备(例如话筒)。
例如，在手机等控制终端的APP上发送多个负载的选择指令，这个指令会携带用户选择需要打开的相机/负载的类型，且该指令由无人机进行接收。或者，APP发送的指令传给遥控器，遥控器通过无线网络模块把命令发送到服务器；服务器再把命令转给无人机。
S304：根据选择指令中的负载类型确定对应的图传通道。
具体地，根据选择指令中的负载类型确定对应的图传通道，即不同的负载类型设有不同的图传通道，图像数据通过与对应负载相对应的图传通道进行传送，即各种图像数据通过与其对应的专用通道进行传输，便于提升传输效率和传输的准确性，减少干扰，避免发送错误数据而影响用户使用。
可以理解地，当对应负载中的负载的数量为多个时，在进行图像数据传输时，可先对多个负载采集到的图像数据进行合并，再将合并后的图像数据进行传输，也可以对每个负载采集的图像数据分别传输。
例如，无人机收到命令后，从当前已经挂载的相机/负载中寻找匹配的相机/负载类型；找到匹配的相机/负载类型后，打开对应的图传通道，同时关闭并释放其他已有的图传通道的资源，以减少资源浪费。
S306：通过图传通道将对应的负载的图像数据发送至控制终端。
具体地，如前所述，通过图传通道将对应的负载的图像数据发送至控制终端，即对应的负载的图像数据通过与其对应的专用通道进行传输，便 于提升传输效率和传输的准确性，减少干扰，避免发送错误数据而影响用户使用。
进一步地，在多负载多路图传方法中，还包括：根据选择指令开启传输图像的图传通道；确定对应于图传通道的预设网络模式的带宽。具体地，通过确定对应于图传通道的预设网络模式的带宽，便于根据带宽选择匹配的码率，以便使用该码率对将要进行传输的图像数据进行编码，以利于在预设网络模式下进行传输，提升传输效率和传输的准确性，实现最佳的带宽自适应机制，图传不会出现花屏；若是非负载类的数据，可实现带宽反馈机制，对数据进行透传。
进一步地，在多负载多路图传方法中，还包括：根据预设网络模式的带宽确定编码的码率；以码率对视频流进行编码，生成图像数据；将图像数据发送至所述控制终端。具体地，网络传输通道的带宽对数据传输的稳定性、可靠性都有较大影响，且网络带宽可能由于网络异常而发生明显的变化，所以需要在发送之前做一个带宽自适应编码，即先确定所述对应负载的预设网络模式的传输通道的带宽，从而便于根据所述带宽确定适合的码率，并根据所述码率对通过所述对应负载获取的视频流进行编码，生成图像数据，从而更有利于在所述预设网络模式下进行数据传输，提升传输的可靠性和稳定性。
优选地，利用on-demand-i(按需请求I帧)的编码方式，实现更高码率压缩比，从而带来更高的图传清晰度。
进一步地，在根据预设网络模式的带宽确定编码的码率之前，还包括：加密负载的视频流数据，并将加密的视频流数据从负载传送到无人机本体上；无人机本体对加密的视频流数据进行解码。具体地，对视频流数据进行加密，并将加密的视频流数据从负载传送到无人机本体上，这样有利于保护所述视频流数据，避免泄密，提升数据的安全性；通过无人机本体对所述加密的视频流数据进行解码，有利于无人机对视频流数据进行识别，提升工作效率。
进一步地，所述无人机本体通过USB虚拟网卡接收负载的视频流数据。具体地，无人机本体通过USB虚拟网卡接收视频流数据，安装和使用方便， 保密性强。
可以理解地，图传的数据流的码率大小需要小于实际带宽的大小。
为了提高带宽自适应的有效性，缩短带宽自适应的链路过程，在靠近无线网卡无人机侧的软件上，对图传的码率数据进行重编码；整个的链路流程为：
(1)无人机通过USB虚拟网卡接收相机负载的H264视频流数据
(2)无人机在收到数据后对H264视频流进行解码，获得到解码一帧的图像。
(3)无人机获取无线网络的带宽大小情况。(无线网络射频模块计算完成后放在一个共享内存中)
(4)根据当前带宽大小的情况，设置编码器编出的视频码率大小。
(5)无人机把解码后的图像数据送到编码器进行视频编码。
(6)编码后形成H264视频流，该视频流为匹配适应当前带宽大小的视频流。
(7)无人机把编码后的H264视频流通过无线网络通道发送到遥控器。
可选地，在多负载多路图传方法中，还包括：判断负载的使用情况，并根据使用情况确定对应于所述负载的图传通道。具体地，负载的使用情况包括挂载和卸载两种情况，通过判断负载的具体使用情况，即判断负载是处于挂载状态还是卸载状态，在挂载状态时，负载能够进行图传，可以确定与其对应的图传通道；而卸载状态时，负载是无法进行图传的，因此需要关闭其图传通道。
进一步地，根据使用情况确定对应于负载的图传通道，具体包括：在使用情况为挂载状态的情况下，通过预设数据库确定并开启与负载对应的端口，以通过端口建立图传通道实现图像数据的传输。具体地，如前所述，在挂载状态下，负载可以进行图传，因此通过预设数据库查找并开启与负载对应的端口，以通过端口建立图传通道进而实现图像数据的传输，从而实现负载的图传，避免了随意开启端口导致无法实现图传的情况，提升了工作效率。
进一步地，在多负载多路图传方法中，还包括：在使用情况为卸载状态的情况下，关闭与负载对应的端口。具体地，在使用情况为卸载状态的情况下，即负载无法进行图传的情况下，关闭与所述负载对应的端口，从而可以避免无效传输，提升了工作效率。
更具体地，在所述负载为卸载状态时，则去当前已经打开的端口中查询该负载是否已经处于接收数据状态，如果是就关闭这个接收端口，如果不是，则说明之前挂载操作的打开过程是失败的。
进一步地，在使用情况为挂载状态的情况下，还包括：判断与负载对应的端口是否属于可用状态，生成判断结果；在判断结果为否时，释放负载对应的端口。具体地，端口可用时，才能进行数据传输，因此在负载为挂载状态时，需要先判断与负载对应的端口是否可用，如果可用则可以进行图传，如果不可用，即判断结果为否时，需要释放端口，以便进行图传。
进一步地，判断与负载对应的端口是否属于可用状态，具体包括：确定负载挂载的端口地址；确定端口地址上是否存在其余负载，在端口地址存在其余负载时，确定与负载对应的端口不属于可用状态。具体地，对于多负载的无人机而言，有可能出现一个端口对应多个负载的情况，即在前述的处于挂载状态的负载，所对应的端口可能还与其它负载对应，因此，通过确定负载挂载的端口地址；确定端口地址上是否存在其余负载，便于确定是否有其它负载在使用该端口；在端口地址存在其余负载时，确定与负载对应的端口不属于可用状态，即端口无法用于处于挂载状态的负载进行图传，需要采取进一步的措施。
进一步地，释放负载对应的端口，具体包括：解除其余负载与端口地址的匹配关系，以使端口地址处于空闲状态。具体地，通过解除其余负载与端口地址的匹配关系，使其余负载不再占用端口，从而使端口地址能够与处于挂载状态的负载相匹配，进而实现处于挂载状态的负载的图传，这样有利于解决两个相同负载挂在同一个无人机上的端口冲突问题。
在一个具体实施例中，无人机当前最多可同时挂载2个负载和1个已经固连的FPV(即第一人称视角，用于实时画面获取)相机。对于每一种负载，在无人机软件上都有一个负载属性与之对应。其中，中心板模块是 与相机相连的最直接模块；中心板模块会固定频率推送当前挂载的相机类型到图传模块。
如图6所示，当相机挂上或者卸下无人机后的处理过程如下：
(1)中心板模块推送的相机类型发生变化。
(2)图传模块收到变化的相机类型，通过查表判断当前相机是挂载还是卸载
(3)如果是挂载，则在属性列表(即上述预设数据库中)里面查询当前设备对应的接收端口参数，并打开端口进行数据接收。
(4)如果是卸载，则在当前已经打开的设备端口中查询该设备是否已经处于接收数据状态，如果是就关闭这个接收端口，如果不是，则说明之前挂载操作的打开过程是失败的。
(5)在打开接收端口的过程中，如果发现这个端口已经被前一个设备所用，但是并没有关闭，那么会对冲突的端口进行关闭处理。为了应对两个相同设备挂在同一个飞机上的情况。
可选地，在多负载多路图传方法中，还包括：获取负载类别；当开启的负载数目大于图传通道的数目时，根据负载类别的优先级确定图传通路的使用权。具体地，由于带宽等因素限制，图传通道的数目往往小于负载数目，因此，通过获取负载类别，能够有效地确定负载类别的优先级，并根据优先级，确定图传通道的使用权，以解决多个负载共用一个图传通道的问题。
需要指出的是，负载类别包括FPV负载还是应用类负载，或者说，是摄取画面的负载，还是应用类负载。
进一步地，在多负载多路图传方法中，负载类别为实时画面获取负载(即上述FPV负载)和应用类负载。具体地，负载类别为实时画面获取负载和应用类负载，这些负载分别在不同的场景有其应用的优势，甚至可以通过多个不同负载的组合带来实时多方面信息的显示，进一步提升无人机系统使用的便利性；其中，实时画面获取负载包括定焦相机、变焦相机、红外相机、双光相机等相机设备；应用类负载包括气体检测器、雷达等。
进一步地，在多负载多路图传方法中，应用类负载的优先级高于实时 画面获取负载的优先级。具体地，应用类负载提供的数据都较为客观，反馈的信息更具有参考价值；且这类负载的数据一般都不是图像数据，其所需的存储空间小，在传输时占用的带宽更少，传输速度更快，有利于用户快速获取数据，提升信息传递的实时性。
本申请的另一个实施例提供一种控制系统，图7为本申请实施例提供的控制系统的结构图。本实施例所述的控制系统包括控制终端和无人机，其中，控制终端响应于多个负载的选择指令，选择指令包括选择的负载类型，将选择指令发送至无人机，以使无人机根据负载类型开启对应的通信端口；无人机根据选择指令中的负载类型确定对应的图传通道；无人机通过图传通道将对应的负载的图像数据发送至控制终端，以使控制终端显示图像数据。
进一步地，上述控制系统中，还包括：服务器，分别与控制终端和无人机电连接，服务器用于接收选择指令，并将选择指令发送至无人机。
从而以通过服务器对无人机以及控制终端之间的信号传递实现透传，以扩大传输范围。
进一步地，服务器还用于接收无人机发送的图像数据，并将图像数据发送至控制终端。
本申请另一个实施例提供一种控制终端，图8为本申请实施例提供的控制终端1000的结构图。本实施例的控制终端1000包括遥控装置1002、显示装置1004和处理器1006，其中，处理器1006，用于：响应于多个负载的选择指令，选择指令包括选择的负载类型，将选择指令发送至无人机，以使无人机根据负载类型开启对应的通信端口；接收由无人机发送的与选择指令对应负载通过对应的通信端口传回的多个图像数据；控制显示装置显示多个图像数据。
可选地，处理器1006还用于：打开与选择指令中的负载类型对应的图传通道，以接收多个图像数据。
可选地，处理器1006还用于：对多个图像数据进行解码并显示。
可选地，图像数据的数量为两个，两个图像数据分别显示在控制终端的显示屏上。
可选地，处理器1006还用于：确定每个图像数据解码后的码率；将所有图像数据中码率最高的图像数据通过显示屏的主显示区域进行显示；将所有图像数据中其它图像数据通过显示屏的至少一个副显示区域进行显示。
进一步地，处理器1006还用于：响应于切换显示的控制指令，将控制指令对应的副显示区域中的图像数据在主显示区域显示。
可选地，处理器1006用于将选择指令发送至无人机，具体用于：通过控制终端1000的显示装置1004接收选择指令；将选择指令由显示装置1004发送至控制终端1000的遥控装置1002；通过遥控装置1002将选择指令发送至无人机。
可选地，处理器1006用于接收由无人机发送的与选择指令对应负载通过对应的通信端口传回的多个图像数据，具体用于：通过控制终端1000的遥控装置1002接收多个图像数据；将图像数据由遥控装置1002发送至控制终端1000的显示装置1004；通过显示装置1004对图像数据执行解码操作，并通过显示装置1004显示解码后的图像数据。
本申请实施例提供一种无人机，图9为本申请实施例提供的无人机2000的结构图。本实施例的无人机2000包括无人机本体2002、设于无人机本体2002上的多个负载2004以及处理器2006，其中，处理器2006用于：接收由控制终端发送的选择指令，选择指令包括选择的负载类型；根据选择指令中的负载类型确定对应的图传通道；通过图传通道将对应的负载的图像数据发送至控制终端。
进一步地，处理器2006还用于：根据选择指令开启传输图像的图传通道；确定对应于图传通道的预设网络模式的带宽。
进一步地，处理器2006还用于：根据预设网络模式的带宽确定编码的码率；以码率对视频流进行编码，生成图像数据；将图像数据发送至控制终端。
进一步地，在处理器2006根据预设网络模式的带宽确定编码的码率之前，处理器2006还用于：加密负载2004的视频流数据，并将加密的视频流数据从负载2004传送到无人机本体2002上；无人机本体2002对加密的视频流数据进行解码。
进一步地，无人机本体2002通过USB虚拟网卡接收负载2004的视频流数据。
可选地，处理器2006还用于：判断负载2004的使用情况，并根据使用情况确定对应于负载2004的图传通道。
进一步地，处理器2006根据使用情况确定对应于负载2004的图传通道，具体用于：在使用情况为挂载状态的情况下，通过预设数据库确定并开启与负载2004对应的端口，以通过端口建立图传通道实现图像数据的传输。
进一步地，处理器2006还用于：在使用情况为卸载状态的情况下，关闭与负载2004对应的端口。
进一步地，在使用情况为挂载状态的情况下，处理器2006还用于：判断与负载2004对应的端口是否属于可用状态，生成判断结果；在判断结果为否时，释放负载2004对应的端口。
进一步地，处理器2006判断与负载2004对应的端口是否属于可用状态，具体用于：确定负载2004挂载的端口地址；确定端口地址上是否存在其余负载，在端口地址存在其余负载时，确定与负载2004对应的端口不属于可用状态。
进一步地，处理器2006释放负载2004对应的端口，具体用于：解除其余负载2004与端口地址的匹配关系，以使端口地址处于空闲状态。
可选地，处理器2006还用于：获取负载类别；当开启的负载数目大于图传通路的数目时，根据负载类别的优先级确定图传通路的使用权。
进一步地，负载类别为实时画面获取负载2004和应用类负载2004。
进一步地，应用类负载的优先级高于实时画面获取负载的优先级。
本申请实施例提供一种服务器，包括处理器，处理器用于：接收由控制终端发送的选择指令，选择指令包括选择的负载类型，并将选择指令发送至无人机，以使无人机根据负载类型开启对应的通信端口；根据选择指令中的负载类型确定对应的图传通道，通过图传通道接收无人机发送的图像数据，并将图像数据发送至控制终端。
本申请的实施例提供一种计算机可读存储介质，其上存储有计算机程 序，计算机程序被处理器执行时实现如上述所述第一方面实施例的多负载多路图传方法的步骤、实现上述第二方面实施例中的所述多负载多路图传方法的步骤。
进一步地，可以理解的是，流程图中或在此以其他方式描述的任何过程或方法描述可以被理解为，表示包括一个或更多个用于实现特定逻辑功能或过程的步骤的可执行指令的代码的模块、片段或部分，并且本申请的优选实施方式的范围包括另外的实现，其中可以不按所示出或讨论的顺序，包括根据所涉及的功能按基本同时的方式或按相反的顺序，来执行功能，这应被本申请的实施例所属技术领域的技术人员所理解。
在流程图中表示或在此以其他方式描述的逻辑和/或步骤，例如，可以被认为是用于实现逻辑功能的可执行指令的定序列表，可以具体实现在任何计算机可读介质中，以供指令执行系统、装置或设备(如基于计算机的系统、包括处理器的系统或其他可以从指令执行系统、装置或设备取指令并执行指令的系统)使用，或结合这些指令执行系统、装置或设备而使用。就本说明书而言，“计算机可读介质”可以是任何可以包含、存储、通信、传播或传输程序以供指令执行系统、装置或设备或结合这些指令执行系统、装置或设备而使用的装置。计算机可读介质的更具体的示例(非穷尽性列表)包括以下：具有一个或多个布线的电连接部(电子装置)，便携式计算机盘盒(磁装置)，随机存取存储器(RAM)，只读存储器(ROM)，可擦除可编辑只读存储器(EPROM或闪速存储器)，光纤装置，以及便携式光盘只读存储器(CDROM)。另外，计算机可读介质甚至可以是可在其上打印程序的纸或其他合适的介质，因为可以例如通过对纸或其他介质进行光学扫描，接着进行编辑、解译或必要时以其他合适方式进行处理来以电子方式获得程序，然后将其存储在计算机存储器中。
应当理解，本申请的各部分可以用硬件、软件、固件或它们的组合来实现。在上述实施方式中，多个步骤或方法可以用存储在存储器中且由合适的指令执行系统执行的软件或固件来实现。例如，如果用硬件来实现，和在另一实施方式中一样，可用本领域公知的下列技术中的任一项或他们的组合来实现：具有用于对数据信号实现逻辑功能的逻辑门电路的离散逻辑电路，具有合适的组合逻辑门电路的专用集成电路，可编程门阵列(PGA)，现场可编程门阵列(FPGA)等。
本技术领域的普通技术人员可以理解实现上述实施例方法携带的全部或部分步骤是可以通过程序来指令相关的硬件完成，的程序可以存储于一种计算机可读存储介质中，该程序在执行时，包括方法实施例的步骤之一或其组合。
此外，在本申请各个实施例中的各功能单元可以集成在一个处理模块中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个模块中。上述集成的模块既可以采用硬件的形式实现，也可以采用软件功能模块的形式实现。集成的模块如果以软件功能模块的形式实现并作为独立的产品销售或使用时，也可以存储在一个计算机可读取存储介质中。
上述提到的存储介质可以是只读存储器，磁盘或光盘等。
以上仅为本申请的优选实施例而已，并不用于限制本申请，对于本领域的技术人员来说，本申请可以有各种更改和变化。凡在本申请的精神和原则之内，所作的任何修改、等同替换、改进等，均应包含在本申请的保护范围之内。

Claims (49)

  1. 一种多负载多路图传方法，用于控制终端，其中，包括：
    响应于多个负载的选择指令，所述选择指令包括选择的负载类型，将所述选择指令发送至无人机，以使所述无人机根据所述负载类型开启对应的通信端口；
    接收由所述无人机发送的与所述选择指令对应负载通过对应的通信端口传回的多个图像数据；
    显示所述多个图像数据。
  2. 根据权利要求1所述的多负载多路图传方法，其中，还包括：
    打开与所述选择指令中的负载类型对应的图传通道，以接收所述多个图像数据。
  3. 根据权利要求1所述的多负载多路图传方法，其中，还包括：
    对所述多个图像数据进行解码并显示。
  4. 根据权利要求1所述的多负载多路图传方法，其中，所述图像数据的数量为两个，两个所述图像数据分别显示在所述控制终端的显示屏上。
  5. 根据权利要求1所述的多负载多路图传方法，其中，还包括：
    确定每个所述图像数据解码后的码率；
    将所有所述图像数据中码率最高的图像数据通过所述显示屏的主显示区域进行显示；
    将所有所述图像数据中其它图像数据通过所述显示屏的至少一个副显示区域进行显示。
  6. 根据权利要求5所述的多负载多路图传方法，其中，还包括：
    响应于切换显示的控制指令，将所述控制指令对应的副显示区域中的图像数据在主显示区域显示。
  7. 根据权利要求1所述的多负载多路图传方法，其中，所述将所述选择指令发送至无人机，具体包括：
    通过所述控制终端的显示装置接收所述选择指令；
    将所述选择指令由所述显示装置发送至所述控制终端的遥控装置；
    通过所述遥控装置将所述选择指令发送至无人机。
  8. 根据权利要求1所述的多负载多路图传方法，其中，所述接收由所述无人机发送的与所述选择指令对应负载通过对应的通信端口传回的多个图像数据，具体包括：
    通过所述控制终端的遥控装置接收所述多个图像数据；
    将所述图像数据由所述遥控装置发送至所述控制终端的显示装置；
    通过所述显示装置对所述图像数据执行解码操作，并通过所述显示装置显示解码后的所述图像数据。
  9. 一种多负载多路图传方法，用于无人机，其中，包括：
    接收由控制终端发送的选择指令，所述选择指令包括选择的负载类型；
    根据所述选择指令中的负载类型确定对应的图传通道；
    通过所述图传通道将对应的负载的图像数据发送至所述控制终端。
  10. 根据权利要求9所述的多负载多路图传方法，其中，还包括：
    根据所述选择指令开启传输图像的图传通道；
    确定对应于所述图传通道的预设网络模式的带宽。
  11. 根据权利要求10所述的多负载多路图传方法，其中，还包括：
    根据所述预设网络模式的带宽确定编码的码率；
    以所述码率对视频流进行编码，生成图像数据；
    将所述图像数据发送至所述控制终端。
  12. 根据权利要求11所述的多负载多路图传方法，其中，在所述根据所述预设网络模式的带宽确定编码的码率之前，还包括：
    加密所述负载的视频流数据，并将加密的视频流数据从负载传送到无人机本体上；
    所述无人机本体对所述加密的视频流数据进行解码。
  13. 根据权利要求12所述的多负载多路图传方法，其中，所述无人机本体通过USB虚拟网卡接收负载的视频流数据。
  14. 根据权利要求9所述的多负载多路图传方法，其中，还包括：
    判断所述负载的使用情况，并根据所述使用情况确定对应于所述负载的图传通道。
  15. 根据权利要求14所述的多负载多路图传方法，其中，所述根据所述使用情况确定对应于所述负载的图传通道，具体包括：
    在所述使用情况为挂载状态的情况下，通过预设数据库确定并开启与所述负载对应的端口，以通过所述端口建立图传通道实现图像数据的传输。
  16. 根据权利要求15所述的多负载多路图传方法，其中，还包括：
    在所述使用情况为卸载状态的情况下，关闭与所述负载对应的端口。
  17. 根据权利要求16所述的多负载多路图传方法，其中，在所述使用情况为挂载状态的情况下，还包括：
    判断与所述负载对应的端口是否属于可用状态，生成判断结果；
    在所述判断结果为否时，释放所述负载对应的端口。
  18. 根据权利要求17所述的多负载多路图传方法，其中，所述判断与所述负载对应的端口是否属于可用状态，具体包括：
    确定所述负载挂载的端口地址；
    确定所述端口地址上是否存在其余负载，在所述端口地址存在其余负载时，确定与所述负载对应的端口不属于可用状态。
  19. 根据权利要求18所述的多负载多路图传方法，其中，所述释放所述负载对应的端口，具体包括：
    解除所述其余负载与所述端口地址的匹配关系，以使所述端口地址处于空闲状态。
  20. 根据权利要求9所述的多负载多路图传方法，其中，还包括：
    获取负载类别；
    当开启的负载数目大于图传通道的数目时，根据负载类别的优先级确定所述图传通道的使用权。
  21. 根据权利要求20所述的多负载多路图传方法，其中，所述负载类别为实时画面获取负载和应用类负载。
  22. 根据权利要求21所述的多负载多路图传方法，其中，所述应用类负载的优先级高于所述实时画面获取负载的优先级。
  23. 一种控制系统，其中，包括控制终端和无人机，其中，
    所述控制终端响应于多个负载的选择指令，所述选择指令包括选择的 负载类型，将所述选择指令发送至无人机，以使所述无人机根据所述负载类型开启对应的通信端口；
    所述无人机根据所述选择指令中的负载类型确定对应的图传通道；
    所述无人机通过所述图传通道将对应的负载的图像数据发送至所述控制终端，以使所述控制终端显示所述图像数据。
  24. 根据权利要求23所述的控制系统，其中，还包括：
    服务器，分别于所述控制终端和所述无人机电连接，所述服务器用于接收所述选择指令，并将所述选择指令发送至所述无人机。
  25. 根据权利要求24所述的控制系统，其中，所述服务器还用于接收所述无人机发送的图像数据，并将所述图像数据发送至所述控制终端。
  26. 一种控制终端，其中，包括：遥控装置、显示装置和处理器，其中，所述处理器，用于：
    响应于多个负载的选择指令，所述选择指令包括选择的负载类型，将所述选择指令发送至无人机，以使所述无人机根据所述负载类型开启对应的通信端口；
    接收由所述无人机发送的与所述选择指令对应负载通过对应的通信端口传回的多个图像数据；
    控制所述显示装置显示所述多个图像数据。
  27. 根据权利要求26所述的控制终端，其中，所述处理器还用于：
    打开与所述选择指令中的负载类型对应的图传通道，以接收所述多个图像数据。
  28. 根据权利要求26所述的控制终端，其中，所述处理器还用于：
    对所述多个图像数据进行解码并显示。
  29. 根据权利要求26所述的控制终端，其中，所述图像数据的数量为两个，两个所述图像数据分别显示在所述控制终端的显示屏上。
  30. 根据权利要求26所述的控制终端，其中，所述处理器还用于：
    确定每个所述图像数据解码后的码率；
    将所有所述图像数据中码率最高的图像数据通过所述显示屏的主显示区域进行显示；
    将所有所述图像数据中其它图像数据通过所述显示屏的至少一个副显示区域进行显示。
  31. 根据权利要求30所述的控制终端，其中，所述处理器还用于：
    响应于切换显示的控制指令，将所述控制指令对应的副显示区域中的图像数据在主显示区域显示。
  32. 根据权利要求26所述的控制终端，其中，所述处理器用于将所述选择指令发送至无人机，具体用于：
    通过所述控制终端的显示装置接收所述选择指令；
    将所述选择指令由所述显示装置发送至所述控制终端的遥控装置；
    通过所述遥控装置将所述选择指令发送至无人机。
  33. 根据权利要求26所述的控制终端，其中，所述处理器用于接收由所述无人机发送的与所述选择指令对应负载通过对应的通信端口传回的多个图像数据，具体用于：
    通过所述控制终端的遥控装置接收所述多个图像数据；
    将所述图像数据由所述遥控装置发送至所述控制终端的显示装置；
    通过所述显示装置对所述图像数据执行解码操作，并通过所述显示装置显示解码后的所述图像数据。
  34. 一种无人机，其中，包括无人机本体、设于所述无人机本体上的多个负载以及处理器，其中，所述处理器用于：
    接收由控制终端发送的选择指令，所述选择指令包括选择的负载类型；
    根据所述选择指令中的负载类型确定对应的图传通道；
    通过所述图传通道将对应的负载的图像数据发送至所述控制终端。
  35. 根据权利要求34所述的无人机，其中，所述处理器还用于：
    根据所述选择指令开启传输图像的图传通道；
    确定对应于所述图传通道的预设网络模式的带宽。
  36. 根据权利要求35所述的无人机，其中，所述处理器还用于：
    根据所述预设网络模式的带宽确定编码的码率；
    以所述码率对视频流进行编码，生成图像数据；
    将所述图像数据发送至所述控制终端。
  37. 根据权利要求36所述的无人机，其中，在所述处理器根据所述预设网络模式的带宽确定编码的码率之前，所述处理器还用于：
    加密所述负载的视频流数据，并将加密的视频流数据从负载传送到无人机本体上；
    所述无人机本体对所述加密的视频流数据进行解码。
  38. 根据权利要求37所述的无人机，其中，所述无人机本体通过USB虚拟网卡接收负载的视频流数据。
  39. 根据权利要求34所述的无人机，其中，所述处理器还用于：
    判断所述负载的使用情况，并根据所述使用情况确定对应于所述负载的图传通道。
  40. 根据权利要求39所述的无人机，其中，所述处理器根据所述使用情况确定对应于所述负载的图传通道，具体用于：
    在所述使用情况为挂载状态的情况下，通过预设数据库确定并开启与所述负载对应的端口，以通过所述端口建立图传通道实现图像数据的传输。
  41. 根据权利要求40所述的无人机，其中，所述处理器还用于：
    在所述使用情况为卸载状态的情况下，关闭与所述负载对应的端口。
  42. 根据权利要求41所述的无人机，其中，在所述使用情况为挂载状态的情况下，所述处理器还用于：
    判断与所述负载对应的端口是否属于可用状态，生成判断结果；
    在所述判断结果为否时，释放所述负载对应的端口。
  43. 根据权利要求42所述的无人机，其中，所述处理器判断与所述负载对应的端口是否属于可用状态，具体用于：
    确定所述负载挂载的端口地址；
    确定所述端口地址上是否存在其余负载，在所述端口地址存在其余负载时，确定与所述负载对应的端口不属于可用状态。
  44. 根据权利要求43所述的无人机，其中，所述处理器释放所述负载对应的端口，具体用于：
    解除所述其余负载与所述端口地址的匹配关系，以使所述端口地址处于空闲状态。
  45. 根据权利要求34所述的无人机，其中，所述处理器还用于：
    获取负载类别；
    当开启的负载数目大于图传通路的数目时，根据负载类别的优先级确定所述图传通路的使用权。
  46. 根据权利要求45所述的无人机，其中，所述负载类别为实时画面获取负载和应用类负载。
  47. 根据权利要求46所述的无人机，其中，所述应用类负载的优先级高于所述实时画面获取负载的优先级。
  48. 一种服务器，其中，包括处理器，所述处理器用于：
    接收由控制终端发送的选择指令，所述选择指令包括选择的负载类型，并将所述选择指令发送至无人机，以使所述无人机根据所述负载类型开启对应的通信端口；
    根据所述选择指令中的负载类型确定对应的图传通道，通过所述图传通道接收所述无人机发送的图像数据，并将所述图像数据发送至所述控制终端。
  49. 一种计算机可读存储介质，其上存储有计算机程序，其中，所述计算机程序被处理器执行时实现如权利要求1至8中任一项所述多负载多路图传方法的步骤、实现如权利要求9至22中任一项所述多负载多路图传方法的步骤。
PCT/CN2019/074230 2019-01-31 2019-01-31 多负载多路图传方法、控制系统和终端、无人机和服务器 WO2020155037A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN201980004941.3A CN111279691B (zh) 2019-01-31 2019-01-31 多负载多路图传方法、控制系统和终端、无人机和服务器
PCT/CN2019/074230 WO2020155037A1 (zh) 2019-01-31 2019-01-31 多负载多路图传方法、控制系统和终端、无人机和服务器

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2019/074230 WO2020155037A1 (zh) 2019-01-31 2019-01-31 多负载多路图传方法、控制系统和终端、无人机和服务器

Publications (1)

Publication Number Publication Date
WO2020155037A1 true WO2020155037A1 (zh) 2020-08-06

Family

ID=71001161

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/074230 WO2020155037A1 (zh) 2019-01-31 2019-01-31 多负载多路图传方法、控制系统和终端、无人机和服务器

Country Status (2)

Country Link
CN (1) CN111279691B (zh)
WO (1) WO2020155037A1 (zh)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115022685A (zh) * 2022-06-02 2022-09-06 上海普适导航科技股份有限公司 一种适用于无人机的视频传输方法及装置
CN115988315A (zh) * 2022-12-15 2023-04-18 应急管理部大数据中心 同时读取多镜头数据的倾斜摄影相机及其数据传输方法
WO2023240508A1 (zh) * 2022-06-15 2023-12-21 深圳市大疆创新科技有限公司 控制方法、装置、图传设备及系统
CN117724532A (zh) * 2023-12-18 2024-03-19 青岛九天国际飞行学院股份有限公司 一种基于LTE Cat1技术的飞行数据无线传输系统
CN118200492A (zh) * 2024-05-17 2024-06-14 广州易而达科技股份有限公司 基于Wi-Fi的视频传输方法、装置、自动飞行器及存储介质

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2022000350A1 (zh) * 2020-06-30 2022-01-06 深圳市大疆创新科技有限公司 视频传输方法、可移动平台、终端设备、系统及存储介质
CN116547925A (zh) * 2021-03-02 2023-08-04 深圳市大疆创新科技有限公司 一种用户终端、信息处理方法及信息处理系统
CN115942068B (zh) * 2023-02-27 2023-11-07 蔚来汽车科技(安徽)有限公司 用于生成虚拟现实素材的方法和装置

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20050008155A1 (en) * 2003-07-08 2005-01-13 Pacific Microwave Research, Inc. Secure digital transmitter and method of operation
US20050122914A1 (en) * 2003-07-08 2005-06-09 Pacific Microwave Research, Inc. Secure Digital Communication System for High Multi-Path Environments
CN105116907A (zh) * 2015-07-20 2015-12-02 清华大学 微型无人机数据传输及控制系统设计方法
CN108200394A (zh) * 2018-01-04 2018-06-22 广东容祺智能科技有限公司 一种支持多路图像传输的无人机系统
CN108702487A (zh) * 2017-11-20 2018-10-23 深圳市大疆创新科技有限公司 无人机的图像传输方法和装置

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20050008155A1 (en) * 2003-07-08 2005-01-13 Pacific Microwave Research, Inc. Secure digital transmitter and method of operation
US20050122914A1 (en) * 2003-07-08 2005-06-09 Pacific Microwave Research, Inc. Secure Digital Communication System for High Multi-Path Environments
CN105116907A (zh) * 2015-07-20 2015-12-02 清华大学 微型无人机数据传输及控制系统设计方法
CN108702487A (zh) * 2017-11-20 2018-10-23 深圳市大疆创新科技有限公司 无人机的图像传输方法和装置
CN108200394A (zh) * 2018-01-04 2018-06-22 广东容祺智能科技有限公司 一种支持多路图像传输的无人机系统

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115022685A (zh) * 2022-06-02 2022-09-06 上海普适导航科技股份有限公司 一种适用于无人机的视频传输方法及装置
WO2023240508A1 (zh) * 2022-06-15 2023-12-21 深圳市大疆创新科技有限公司 控制方法、装置、图传设备及系统
CN115988315A (zh) * 2022-12-15 2023-04-18 应急管理部大数据中心 同时读取多镜头数据的倾斜摄影相机及其数据传输方法
CN117724532A (zh) * 2023-12-18 2024-03-19 青岛九天国际飞行学院股份有限公司 一种基于LTE Cat1技术的飞行数据无线传输系统
CN118200492A (zh) * 2024-05-17 2024-06-14 广州易而达科技股份有限公司 基于Wi-Fi的视频传输方法、装置、自动飞行器及存储介质

Also Published As

Publication number Publication date
CN111279691A (zh) 2020-06-12
CN111279691B (zh) 2021-12-07

Similar Documents

Publication Publication Date Title
WO2020155037A1 (zh) 多负载多路图传方法、控制系统和终端、无人机和服务器
WO2020154959A1 (zh) 多负载图传方法、控制系统、控制终端、无人机和服务器
CN203573375U (zh) 一种行车记录仪及行车记录系统
US20160191458A1 (en) Method, a device, and a data transmission system for data transmission in a network system
CN108924520B (zh) 传输控制方法、装置、控制器、拍摄设备及飞行器
CN107493311B (zh) 实现操控设备的方法、装置和系统
US20170269590A1 (en) Controlled device and communication system and method utilizing the same
WO2018209575A1 (zh) 数据传输方法、设备、机器可读存储介质以及系统
US20210136578A1 (en) Data distribution from a movable object
US20200351615A1 (en) System and method for managing movable object communications
US20190177002A1 (en) Detection and signaling of conditions of an unmanned aerial vehicle
JP6250166B2 (ja) 可動物体のアプリケーション開発を支援する方法、システム及びコンピュータ読取り可能媒体
WO2019241970A1 (zh) 无人机的扬声器控制方法和设备
US11899139B2 (en) Photonic devices with redundant components and their applications
US11134526B2 (en) Automatic update of connection to a movable object
EP4315876A1 (en) Method and apparatus for providing media service
CN108702201A (zh) 无人机系统和通信方法
US20180027265A1 (en) Methods and systems for live video broadcasting from a remote location based on an overlay of audio
WO2023128695A1 (en) Deferred rendering on extended reality (xr) devices
US11830247B2 (en) Identifying antenna communication issues using unmanned aerial vehicles
US20210294720A1 (en) System and method for movable object tracking and analysis
CN110622086A (zh) 可移动物体应用框架
CN108306836B (zh) 数据传输装置、智能交互平板及数据传输方法
EP3790298B1 (en) Methods for acquiring and sending route information of unmanned aerial vehicle
JP6644756B2 (ja) 可動物体のアプリケーション開発を支援する方法、システム及びコンピュータ読取り可能媒体

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19913176

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19913176

Country of ref document: EP

Kind code of ref document: A1