US20200351615A1 - System and method for managing movable object communications - Google Patents
System and method for managing movable object communications Download PDFInfo
- Publication number
- US20200351615A1 US20200351615A1 US16/930,090 US202016930090A US2020351615A1 US 20200351615 A1 US20200351615 A1 US 20200351615A1 US 202016930090 A US202016930090 A US 202016930090A US 2020351615 A1 US2020351615 A1 US 2020351615A1
- Authority
- US
- United States
- Prior art keywords
- movable object
- object data
- data
- cache
- listener
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
- 238000000034 method Methods 0.000 title claims abstract description 103
- 238000004891 communication Methods 0.000 title claims abstract description 63
- 230000008859 change Effects 0.000 claims description 10
- 230000008569 process Effects 0.000 description 28
- 230000033001 locomotion Effects 0.000 description 24
- 238000003384 imaging method Methods 0.000 description 22
- 230000007246 mechanism Effects 0.000 description 20
- 230000015654 memory Effects 0.000 description 15
- 238000010586 diagram Methods 0.000 description 13
- 238000012545 processing Methods 0.000 description 7
- 230000001276 controlling effect Effects 0.000 description 6
- 238000005516 engineering process Methods 0.000 description 6
- 230000006870 function Effects 0.000 description 5
- 238000001514 detection method Methods 0.000 description 4
- 230000001133 acceleration Effects 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 230000005540 biological transmission Effects 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- 238000013519 translation Methods 0.000 description 2
- RZVHIXYEVGDQDX-UHFFFAOYSA-N 9,10-anthraquinone Chemical compound C1=CC=C2C(=O)C3=CC=CC=C3C(=O)C2=C1 RZVHIXYEVGDQDX-UHFFFAOYSA-N 0.000 description 1
- WHXSMMKQMYFTQS-UHFFFAOYSA-N Lithium Chemical compound [Li] WHXSMMKQMYFTQS-UHFFFAOYSA-N 0.000 description 1
- 239000002253 acid Substances 0.000 description 1
- 230000009471 action Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- OJIJEKBXJYRIBZ-UHFFFAOYSA-N cadmium nickel Chemical compound [Ni].[Cd] OJIJEKBXJYRIBZ-UHFFFAOYSA-N 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 238000000701 chemical imaging Methods 0.000 description 1
- 238000013480 data collection Methods 0.000 description 1
- 238000013523 data management Methods 0.000 description 1
- 238000013524 data verification Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000007599 discharging Methods 0.000 description 1
- 230000009977 dual effect Effects 0.000 description 1
- 239000003792 electrolyte Substances 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 238000009432 framing Methods 0.000 description 1
- 239000000446 fuel Substances 0.000 description 1
- 229910052739 hydrogen Inorganic materials 0.000 description 1
- 239000001257 hydrogen Substances 0.000 description 1
- 238000003780 insertion Methods 0.000 description 1
- 230000037431 insertion Effects 0.000 description 1
- 229910052744 lithium Inorganic materials 0.000 description 1
- 229910001416 lithium ion Inorganic materials 0.000 description 1
- 230000007774 longterm Effects 0.000 description 1
- 230000005389 magnetism Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 229910052987 metal hydride Inorganic materials 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000000704 physical effect Effects 0.000 description 1
- 238000001556 precipitation Methods 0.000 description 1
- 230000001105 regulatory effect Effects 0.000 description 1
- 238000012163 sequencing technique Methods 0.000 description 1
- 238000000060 site-specific infrared dichroism spectroscopy Methods 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
- 230000002123 temporal effect Effects 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 238000001931 thermography Methods 0.000 description 1
- 230000001052 transient effect Effects 0.000 description 1
- 238000012384 transportation and delivery Methods 0.000 description 1
- 230000001960 triggered effect Effects 0.000 description 1
- 238000013024 troubleshooting Methods 0.000 description 1
- 238000002604 ultrasonography Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G08—SIGNALLING
- G08G—TRAFFIC CONTROL SYSTEMS
- G08G5/00—Traffic control systems for aircraft, e.g. air-traffic control [ATC]
- G08G5/0047—Navigation or guidance aids for a single aircraft
- G08G5/0069—Navigation or guidance aids for a single aircraft specially adapted for an unmanned aircraft
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W4/00—Services specially adapted for wireless communication networks; Facilities therefor
- H04W4/02—Services making use of location information
- H04W4/025—Services making use of location information using location based information parameters
- H04W4/027—Services making use of location information using location based information parameters using movement velocity, acceleration information
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B64—AIRCRAFT; AVIATION; COSMONAUTICS
- B64C—AEROPLANES; HELICOPTERS
- B64C39/00—Aircraft not otherwise provided for
- B64C39/02—Aircraft not otherwise provided for characterised by special use
- B64C39/024—Aircraft not otherwise provided for characterised by special use of the remote controlled vehicle type, i.e. RPV
-
- G—PHYSICS
- G08—SIGNALLING
- G08G—TRAFFIC CONTROL SYSTEMS
- G08G5/00—Traffic control systems for aircraft, e.g. air-traffic control [ATC]
- G08G5/0047—Navigation or guidance aids for a single aircraft
- G08G5/006—Navigation or guidance aids for a single aircraft in accordance with predefined flight zones, e.g. to avoid prohibited zones
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/12—Protocols specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
-
- H04L67/2842—
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/56—Provisioning of proxy services
- H04L67/568—Storing data temporarily at an intermediate stage, e.g. caching
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04M—TELEPHONIC COMMUNICATION
- H04M1/00—Substation equipment, e.g. for use by subscribers
- H04M1/72—Mobile telephones; Cordless telephones, i.e. devices for establishing wireless links to base stations without route selection
- H04M1/724—User interfaces specially adapted for cordless or mobile telephones
- H04M1/72403—User interfaces specially adapted for cordless or mobile telephones with means for local support of applications that increase the functionality
- H04M1/72409—User interfaces specially adapted for cordless or mobile telephones with means for local support of applications that increase the functionality by interfacing with external accessories
- H04M1/72415—User interfaces specially adapted for cordless or mobile telephones with means for local support of applications that increase the functionality by interfacing with external accessories for remote control of appliances
-
- H04M1/72533—
-
- H04W12/0027—
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W12/00—Security arrangements; Authentication; Protecting privacy or anonymity
- H04W12/30—Security of mobile devices; Security of mobile applications
- H04W12/37—Managing security policies for mobile devices or for controlling mobile applications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W4/00—Services specially adapted for wireless communication networks; Facilities therefor
- H04W4/02—Services making use of location information
- H04W4/029—Location-based management or tracking services
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W4/00—Services specially adapted for wireless communication networks; Facilities therefor
- H04W4/30—Services specially adapted for particular environments, situations or purposes
- H04W4/40—Services specially adapted for particular environments, situations or purposes for vehicles, e.g. vehicle-to-pedestrians [V2P]
- H04W4/44—Services specially adapted for particular environments, situations or purposes for vehicles, e.g. vehicle-to-pedestrians [V2P] for communication between vehicles and infrastructures, e.g. vehicle-to-cloud [V2C] or vehicle-to-home [V2H]
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B64—AIRCRAFT; AVIATION; COSMONAUTICS
- B64U—UNMANNED AERIAL VEHICLES [UAV]; EQUIPMENT THEREFOR
- B64U20/00—Constructional aspects of UAVs
- B64U20/80—Arrangement of on-board electronics, e.g. avionics systems or wiring
- B64U20/87—Mounting of imaging devices, e.g. mounting of gimbals
Definitions
- the disclosed embodiments relate generally to movable objects and more particularly, but not exclusively, to systems and methods for managing communication with a movable object.
- Movable objects such as unmanned aircraft, can be used in many different fields such as film production, sporting events, disaster relief, geological study, and more. Movable objects can be configured to transmit remote data, such as video data, back to applications that interface with the movable object. Furthermore, such applications can regularly query data, such as control or state data, from the movable object.
- communications between the movable object and the application usually take place over limited bandwidth that can result in latency, which results in asynchronous access to the movable object data by the application. This problem can be worsened by poor application design that unnecessarily increases usage of limited communication bandwidth.
- a method of managing communication with a movable object comprising: receiving, at a movable object manager, a request for movable object data from a requester in a movable object environment; determining whether movable object data corresponding to the request is available in a cache associated with the movable object manager; and providing the movable object data in the cache to the requester.
- a method of managing communication with a movable object comprising: registering, via a movable object manager, a listener for listening to movable object data in a movable object environment; storing the movable object data, which is received from the movable object, in a cache associated with the movable object manager; and providing the movable object data to an application corresponding to the listener.
- a system for managing communication with a movable object comprising: one or more processors configured for: receiving, at a movable object manager, a request for movable object data from a requester in a movable object environment; determining whether movable object data corresponding to the request is available in a cache associated with the movable object manager; and providing the movable object data in the cache to the requester.
- a system for managing communication with a movable object comprising: one or more processors configured for: registering, via a movable object manager, a listener for listening to movable object data in a movable object environment; storing the movable object data, which is received from the movable object, in a cache associated with the movable object manager; and providing the movable object data to an application corresponding to the listener.
- a non-transitory computer-readable medium with instructions stored thereon that, when executed by a processor, perform the steps comprising: receiving, at a movable object manager, a request for movable object data from a requester in a movable object environment; determining whether movable object data corresponding to the request is available in a cache associated with the movable object manager; and providing the movable object data in the cache to the requester.
- a non-transitory computer-readable medium with instructions stored thereon that, when executed by a processor, perform the steps comprising: registering, via a movable object manager, a listener for listening to movable object data in a movable object environment; storing the movable object data, which is received from the movable object, in a cache associated with the movable object manager; and providing the movable object data to an application corresponding to the listener.
- FIG. 1 is an exemplary top-level block diagram illustrating an embodiment of a movable object environment that includes a user terminal in communication with a movable object through a communication link.
- FIG. 2 is an exemplary diagram illustrating an embodiment of the movable object of FIG. 1 , wherein the movable object is a UAV.
- FIG. 3 is an exemplary block diagram illustrating an embodiment of an environment of the movable object operation system of FIG. 1 .
- FIG. 4 is an exemplary diagram illustrating an embodiment of a communication protocol for use in the movable object environment of FIG. 1 .
- FIG. 5 is an exemplary diagram illustrating an embodiment of a data packet for use in the movable object environment of FIG. 1 .
- FIG. 6 is an exemplary block diagram illustrating an embodiment of a cache for use in the movable object environment of FIG. 1 .
- FIG. 7 is an exemplary block diagram illustrating an embodiment of a hardware abstraction in the cache of FIG. 6 .
- FIG. 8 is an exemplary diagram illustrating various embodiments of a movable object data key for use in the movable object environment of FIG. 1 .
- FIG. 9 is an exemplary diagram illustrating an embodiment of a method for adding a new listener corresponding to movable object data of interest.
- FIG. 10 is an exemplary top-level flow chart illustrating an embodiment of a method for managing communication with the movable object of FIG. 1 .
- FIG. 11 is an exemplary diagram illustrating an embodiment of a method for retrieving movable object data of interest.
- FIG. 12 is another exemplary top-level flow chart illustrating an embodiment of a method for managing communication with the movable object of FIG. 1 .
- FIG. 13 is exemplary flow chart illustrating further details of the method of FIG. 12 for managing communication with the movable object of FIG. 1 .
- FIG. 14 is an exemplary diagram illustrating embodiments of a method for receiving movable object data from a movable object using “push” and “push-like” mechanisms.
- the present disclosure sets forth systems and methods for managing communication with a movable object, which overcome limitations of prior systems and methods. More particularly, the present systems and methods relate to the use of a cache for regulating access to movable object data. By using a cache, potentially redundant calls to retrieve movable object data can advantageously be reduced or eliminated. The present systems and methods can thereby regulate efficient access to remote asynchronous movable object data, thereby improving usage of limited bandwidth and improving application performance. Furthermore, the present systems and methods can utilize an application programming interface (API) that offers programmers convenient and encapsulated access to the remote asynchronous movable object data. Exemplary embodiments of the present systems and methods are described below.
- API application programming interface
- the movable object environment 10 can include a user terminal 100 , which can communicate with a movable object 300 via a communication link 200 .
- a user terminal 100 can communicate with a movable object 300 via a communication link 200 .
- One or more of the user terminal 100 , the communication link 200 , and/or the movable object 300 can comprise a system for managing movable communications, as further described below.
- the user terminal 100 can be configured to interact with a user (not shown) to operate the movable object 300 and/or present data collected by the movable object 300 to the user.
- the user terminal 100 can include, for example, remote controllers (not shown), portable computers, laptops, mobile devices, handheld devices, mobile telephones (for example, smartphones), tablet devices, tablet computers, personal digital assistants, handheld consoles, portable media players, wearable devices (for example, smartwatches and head-mounted displays), and the like.
- the user terminal 100 can include one or more processors 110 that can be used to execute software.
- the user terminal 100 can include any number of processors 110 , as desired.
- each processor 110 can include one or more general purpose microprocessors (for example, single or multi-core processors), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), application-specific instruction-set processors, digital signal processing units, coprocessors, network processing units, audio processing units, encryption processing units, and the like.
- the processors 110 can be configured to perform any of the methods described herein, including but not limited to a variety of tasks relating to mobile object control, tracking, and analysis.
- the processors 110 can include specialized software and/or hardware, for example, for processing movable object tasks using an interface.
- the processors 110 can execute an analyzer that operates to perform any of the methods described herein.
- the user terminal 100 can further include one or more memories 120 (alternatively referred to herein as non-transient computer readable media).
- Suitable memories 120 can include, for example, random access memory (RAM), static RAM, dynamic RAM, read-only memory (ROM), programmable ROM, erasable programmable ROM, electrically erasable programmable ROM, flash memory, secure digital (SD) card, and the like. Instructions for performing any of the methods described herein can be stored in the memory 120 .
- the memory 120 can be placed in operative communication with the processors 110 , as desired, and instructions and/or data can be transmitted from the memory 120 to the processors 110 for execution, as desired.
- the user terminal 100 can additionally include one or more input/output devices 130 , such as buttons, a keyboard, keypad, trackball, displays, and/or a monitor.
- input/output devices 130 such as buttons, a keyboard, keypad, trackball, displays, and/or a monitor.
- Various user interface elements for example, windows, buttons, menus, icons, pop-ups, tabs, controls, cursors, insertion points, and the like) can be used to present data to and receive data from a user (not shown).
- the user terminal 100 can be configured for communication with the movable object 300 via the communication link 200 .
- the communication link 200 can include an uplink for transmitting data (such as control data and application data) from the user terminal 100 to the movable object 300 , and a downlink for transmitting data (such as telemetry data, application data, image data, and video data) from the movable object 300 to the user terminal.
- the uplink and downlink can share a single frequency using time modulation.
- the uplink and downlink can use different frequencies.
- the communication link 200 can be configured to transmit signals according to a movable object protocol 250 (shown in FIG. 4 ).
- the communication link 200 can be a wireless communication link 200 over a wireless network. Suitable wireless communications can include, for example, radio, Wireless Fidelity (WiFi), cellular, satellite, and broadcasting. In certain embodiments, the communication link 200 can be implemented over a 3G, 4G, or 5G mobile telecommunications network.
- Suitable wireless communications can include, for example, radio, Wireless Fidelity (WiFi), cellular, satellite, and broadcasting.
- WiFi Wireless Fidelity
- the communication link 200 can be implemented over a 3G, 4G, or 5G mobile telecommunications network.
- Exemplary wireless communication technologies include, but are not limited to, Global System for Mobile Communications (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband CDMA (W-CDMA), CDMA2000, IMT Single Carrier, Enhanced Data Rates for GSM Evolution (EDGE), Long-Term Evolution (LTE), LTE Advanced, Time-Division LTE (TD-LTE), High Performance Radio Local Area Network (HiperLAN), High Performance Radio Wide Area Network (HiperWAN), High Performance Radio Metropolitan Area Network (HiperMAN), Local Multipoint Distribution Service (LMDS), Worldwide Interoperability for Microwave Access (WiMAX), ZigBee, Bluetooth, Flash Orthogonal Frequency-Division Multiplexing (Flash-OFDM), High Capacity Spatial Division Multiple Access (HC-SDMA), iBurst, Universal Mobile Telecommunications System (UMTS), UMTS Time-Division Duplexing (UMTS-TDD), Evolved High Speed Packet Access (HSPA), Flash-OFDM), High
- the communication link 200 can advantageously be encrypted to prevent third party intrusion into movable object operations.
- Suitable encryption methods include, but are not limited to, internet key exchange, Internet Protocol Security (IPsec), Kerberos, point-to-point protocol, transport layer security, SSID hiding, MAC ID filtering, Static IP addressing, 802.11 security, Wired Equivalent Privacy (WEP), Wi-Fi Protected Access (WPA), WPA2, Temporal Key Integrity Protocol (TKIP), Extensible Authentication Protocol (EAP), Lightweight Extensible Authentication Protocol (LEAP), Protected Extensible Authentication Protocol (PEAP), and the like.
- IPsec Internet Protocol Security
- Kerberos Kerberos
- point-to-point protocol transport layer security
- SSID hiding MAC ID filtering
- Static IP addressing 802.11 security
- Wired Equivalent Privacy WEP
- WPA Wi-Fi Protected Access
- WPA2 Wi-Fi Protected Access
- TKIP Temporal Key Integrity Protocol
- the movable object 300 in the present systems and methods can include any type of movable object, including bicycles, automobiles, trucks, ships, boats, trains, helicopters, aircraft, robotic devices, various hybrids thereof, and the like.
- the movable object 300 is shown in some examples herein as being a flying movable object for illustrative purposes only, such examples are not meant to be limiting.
- the embodiments described herein in the context of flying movable objects can be applied to any suitable movable object 300 .
- the movable object 300 can include or more functional modules. Each of the functional modules of the movable object 300 can include one or more movable object components 310 for carrying out, for example, movement, communication, imaging, and/or other functionality.
- movable object components 310 can include one or more movement mechanisms 312 (for example, propulsion mechanisms), a sensing system 314 , and/or a communication system 316 .
- the movable object 300 is further shown as having a carrier 318 that includes a payload 319 .
- the movable object 300 can further include hardware components such as processors, memories, etc., as desired.
- Each of the movable object components 310 can be associated with one or more attributes.
- the attributes and attribute values of the movable object components 310 constitute movable object data 311 (not shown) which can be generated, retrieved, stored, and used in a manner described herein.
- the movement mechanisms 312 can include one or more rotors, propellers, blades, engines, motors, wheels, axles, magnets, nozzles, and the like.
- the movable object 300 can have one or more propulsion mechanisms.
- the movable object 300 can have multiple movement mechanisms 312 can be of the same type or different types of movement mechanisms 312 .
- the movement mechanisms 312 can be mounted on the movable object 300 using any suitable means such as a support element (for example, a drive shaft).
- the movement mechanisms 312 can be mounted on any suitable portion of the movable object 300 , such on the top, bottom, front, back, sides, or suitable combinations thereof.
- Movable object data 311 associated with the movement mechanism 312 can include, for example, data relating to positions, on/off status, and/or wear and tear status of the movement mechanism 312 .
- the movement mechanisms 312 can enable the movable object 300 to take off vertically from a surface or land vertically on a surface without requiring horizontal movement of the movable object 300 (for example, without traveling down a runway).
- the movement mechanisms 312 can be operable to enable the movable object 112 to hover at a specified position and/or orientation.
- One or more of the movement mechanisms 312 can be controlled independently of the other movement mechanisms 312 .
- the movable object 300 can have multiple horizontally oriented rotors that can provide lift and/or thrust to the movable object 300 . The rotors can be actuated to provide vertical takeoff, vertical landing, and hovering capabilities to the movable object 300 .
- one or more of the rotors can spin in a clockwise direction, while one or more of the rotors can spin in a counterclockwise direction.
- the number of clockwise rotors can be equal to the number of counterclockwise rotors.
- the rotation rate of each of the rotors can be varied independently in order to control the lift and/or thrust produced by each rotor, and thereby adjust the spatial disposition, velocity, and/or acceleration of the movable object 300 (for example, to up to three degrees of translation and up to three degrees of rotation).
- the sensing system 314 can include one or more sensors that can sense the spatial disposition, velocity, and/or acceleration of the movable object 300 (for example, with respect to various degrees of translation and various degrees of rotation).
- the one or more sensors can include sensors such as global positioning system (GPS) sensors, motion sensors, inertial sensors (for example, inertial measurement units (IMU)), proximity sensors, image sensors, attitude and heading reference systems (AHRS), radar sensors, lidar sensors, ultrasound sensors, infrared sensors, as well as other sensors that detect motion by optics, radio, sound, vibration, magnetism, and the like.
- Movable object data 311 provided by the sensing system 314 can be used to control the spatial disposition, velocity, and/or orientation of the movable object 300 (for example, using a suitable processing unit and/or control module).
- the sensing system 314 can be used to collect movable object data 311 regarding an operational environment surrounding the movable object 300 , such as weather conditions, proximity to potential obstacles, location of geographical features, location of manmade structures, and the like.
- the sensing system 314 for sensing the environment can include electro-optical sensors, thermal/infrared sensors, color or monochrome sensors, multi-spectral imaging sensors, spectrophotometers, spectrometers, thermometers, illuminometers, microphones/sonic transducers, pressure sensors, altitude sensors, flow sensors, humidity sensors, precipitation sensors, wind speed sensors, wind direction sensors, anemometers, optical rain sensors, positioning devices, accelerometers, gyroscopes, and/or others.
- the communication system 316 can enable communication with the user terminal 100 via the communication link 200 , which can include various wired and/or wireless technologies as discussed above.
- the communication system 316 can include any number of transmitters, receivers, and/or transceivers suitable for wireless communication.
- the communication can be one-way communication whereby the movable object 300 transmits movable object data 311 to the user terminal 100 , or vice-versa.
- the communication can also be two-way communication, whereby movable object data 311 can be transmitted in both directions between the movable object 300 and the user terminal 100 .
- the user terminal 100 can provide movable object data 311 that is control data to one or more of the movable object 300 , carrier 318 , and payload 319 and receive information from one or more of the movable object 300 , carrier 318 , and payload 319 (for example, position and/or motion information of the movable object, carrier or payload; data sensed by the payload such as image data captured by a payload camera; and data generated from image data captured by the payload camera).
- control data from the user terminal 100 can include instructions for relative positions, movements, actuations, or controls of the movable object, carrier, and/or payload.
- control data can result in a modification of the location and/or orientation of the movable object (for example, via control of the movement mechanisms 312 ), or a movement of the payload with respect to the movable object (for example, via control of the carrier 318 ).
- the control data from the application can result in control of the payload.
- Any payload 319 can be used with the present systems and methods.
- communications from the movable object 300 , carrier 318 and/or payload 319 can include information from one or more sensors (for example, of the sensing system 314 or of the payload 319 ) and/or data generated based on the sensing information.
- the communications may include sensed information from one or more different types of sensors, as described above. Such information may pertain to the position (for example, location or orientation), movement, or acceleration of the movable object, carrier, and/or payload.
- Movable object data 311 from the payload 319 can include data captured by the payload 319 or a sensed state of the payload 319 .
- UAV unmanned aerial vehicle
- UAVs 320 include aircraft without a human pilot onboard the vehicle whose flight is controlled autonomously or by a remote pilot (or sometimes both).
- UAVs 320 are now finding increased usage in civilian applications involving various aerial operations, such as data-gathering or delivery.
- the present movable object operation systems and methods are suitable for use with many types of UAVs 320 including, without limitation, quadcopters (also referred to a quadrotor helicopters or quad rotors), single rotor, dual rotor, trirotor, hexarotor, and octorotor rotorcraft UAVs 320 , fixed wing UAVs 320 , and hybrid rotorcraft-fixed wing UAVs 320 .
- quadcopters also referred to a quadrotor helicopters or quad rotors
- single rotor single rotor
- dual rotor trirotor
- hexarotor hexarotor
- octorotor rotorcraft UAVs 320 fixed wing UAVs 320
- hybrid rotorcraft-fixed wing UAVs 320 hybrid rotorcraft-fixed wing UAVs 320 .
- Movable object components 310 of a UAV 320 can include, for example, a flight controller 321 for controlling flight operations of the UAV 320 .
- the flight controller 321 can include one or more sensing systems 314 (described above with reference to FIG. 1 ), such as one or more GPS 322 and/or IMU 323 .
- the flight controller 321 can be operated using a processor 324 embedded within the flight controller 321 .
- the flight controller 321 can, for example, control a movement of the UAV 320 . Exemplary control of the movement includes setting target destination, velocity, altitude, attitude (pitch, roll, and yaw), and/or the like.
- the flight controller 321 can communicate with selected data collection instruments (not shown) of the UAV 320 to obtain movable object data 311 .
- the UAV 320 can also include an imaging device 325 .
- the imaging device 325 can receive incident light from an external environment and convert the incident light into digital and/or analog signals (not shown). The signals can be processed to produce an image that can be displayed to a user.
- Exemplary imaging devices 325 suitable for use with the present systems and methods include, but are not limited to, commercially-available cameras and camcorders.
- the imaging device 325 can be adapted to use additional sensors (not shown) for various applications such as stereo vision, thermography, creation of multi-spectral images, infrared detection, gamma detection, x-ray detection, and the like.
- the imaging device 325 can be controlled in various ways, such as turn the imaging device 325 on and off, and get or set parameters of the imaging device 325 (for example, imaging angle, imaging mode, photo/video size/resolution, photo/video format, zoom settings, exposure settings, and the like).
- movable object data 311 collected can include various parameters of the imaging device 325 , still image data, and/or video image data from the imaging device.
- Exemplary still image data and video image data can respectively include still images and video images captured aerially by the imaging device 325 .
- the UAV 320 can further include a gimbal 326 for precise rotational and/or translational positioning of mounted objects, such as an imaging device.
- the gimbal 326 can include one or more pivoted supports (not shown) that are mounted with respect to one or more axes of rotation, thereby allowing a mounted object to be rotationally independent with respect to a fixture to which the gimbal 326 is mounted.
- the manner of configuring and mounting the pivoted supports can vary and can include, for example, concentric ring structures and non-ring structures.
- the gimbal 326 can include any suitable number, including one, two, or three, pivoted supports.
- the gimbal 326 can include various mechanical apparatus as needed, such as ball bearings, tracks, actuators, and the like.
- the gimbal 326 can be configured to respond to external commands (for example, from a processor) directing movement of the imaging device 325 .
- Movable object data 311 (such as angles and positions) of the gimbal 326 can be retrieved, as desired.
- the UAV 320 can further include one or more batteries 327 for providing power to the UAV 320 and/or any functional modules of the UAV 320 .
- Each battery 327 can include any number of battery cells each having two electrodes (not shown) and an electrolyte (not shown) that reacts chemically to generate an electric potential, providing electric current between the electrodes.
- the battery 327 can be chargeable and/or rechargeable.
- the battery 327 can include any type of battery cells, including, but not limited to, lead-acid cells, lithium air cells, lithium-ion cells, nickel-cadmium cells, nickel-metal hydride hydrogen cells, and the like.
- the battery 327 be a smart battery whose internal state and charging/discharging are monitored by a battery manager (not shown).
- Movable object data 311 (such as remaining charge) of the battery 327 can advantageously be retrieved and monitored, as desired, for operation of the movable object 300 .
- an exemplary movable object environment 10 is shown as including a user terminal 100 , as well as a movable object 300 .
- the movable object environment 10 can, for example, be a movable object software environment.
- the movable object environment 10 can be configured using any convenient programming language, such as Java, C, C++, Python, and the like.
- the movable object environment 10 can be operated using any appropriate hardware, as described above with reference to FIG. 1 .
- the user terminal 100 can include one or more applications 160 a , 160 b , and 160 c that are installed on the user terminal 100 .
- the application 160 can be colloquially referred to as an “app.”
- the app can be made available by a vendor and kept updated by the vendor through, for example, a mobile app store.
- the applications 160 a , 160 b , and 160 c can be individually or collectively configured to invoke an interface 170 .
- the interface 170 can be an application programming interface (API) that includes one or more defined elements such as functions, methods, procedures, parameters, constants, variables, objects, modules, datatypes, exceptions, etc.
- the API can be a movable object API that includes elements for interacting with and controlling a movable object 300 .
- a software developer can, for example, program an application 160 using the API.
- the application 160 can invoke the API and execute functionality specified by the API.
- the applications 160 a , 160 b , and 160 c can interact with elements of the interface 170 through application call processes 165 , as shown in FIG. 3 .
- an application call process 165 can be a get( ) method call that is made by an application 160 to the interface 170 to retrieve a value of particular movable object data 311 .
- the application call process 165 can be a set( ) method call to set a value for particular movable object data 311 .
- Another exemplary application call process 165 is a method call to the interface 170 that commands the movable object 300 to perform one or more tasks for the movable object 300 .
- the interface 170 can be part of a movable object software development kit (SDK).
- SDK movable object software development kit
- the movable object SDK can advantageously provide common, integrated functionality for applications 160 that interact with movable objects 300 .
- applications 160 that are designed to control a flying movable object 300 can invoke functions in the movable object SDK for controlling navigation of the flying movable object 300 .
- Applications 160 relating to imaging from a movable object 300 can invoke functions in the movable object SDK for controlling an imaging device 325 (shown in FIG. 2 ) or gimbal 326 (shown in FIG. 2 ) of the movable object 300 .
- the movable object SDK can further include tools, libraries, documentation and sample code that can help a software developer to use the SDK for developing a movable object application.
- the movable object 300 can be managed using a movable object manager 180 .
- the movable object manager 180 can be located anywhere that is convenient. Though the movable object manager 180 is depicted as being a part of the user terminal 100 in FIG. 3 for illustrative purposes only, the movable object manager 180 is not limited in locality or implementation. In some embodiments, the movable object manager 180 can be part of the movable object 300 , or be any part of a movable object environment 10 .
- the movable object manager 180 can be configured with logic for determining when communication with the movable object 300 is necessary, and when such communication is unnecessary.
- the movable object manager 180 can be configured to manage the movable object 300 by, for example, receiving data, sending commands, and/or processing data in relation to the movable object 300 .
- the movable object manager 180 can be configured to manage the movable object 300 by handling application call processes 165 received through the interface 170 .
- the movable object manager 180 can convert the received application call processes 165 into protocol call processes 255 to the movable object 300 .
- the movable object manager 180 can issue protocol call processes 255 to the movable object 300 that are not triggered through the interface 170 .
- the movable object manager 180 can perform background tasks that relate to maintaining movable object performance (such as maintaining altitude, pitch, or heading of a flying movable object 300 , checking that the movable object 300 has sufficient fuel remaining, or automatically directing the movable object to avoid restricted zones such as airports).
- the movable object manager 180 can be configured to communicate with an authentication server (not shown) for providing a secure environment for communication between the user terminal 100 and the movable object 300 .
- the movable object manager 180 can include various modules, such as a communication module or data management module (not shown), as needed. As illustrated in FIG. 3 , the movable object manager 180 includes a cache 190 for managing communications with the movable object 300 . In particular, the cache 190 can store values of certain movable object data 311 of interest. The movable object data 311 can be retrieved asynchronously by an application 160 . In some embodiments, an application that desires access to the movable object data 311 can register a listener 191 (shown in FIG. 6 ) that corresponds the movable object data. The listener 191 can facilitate updating and retrieving the movable object data 311 , as desired.
- a listener 191 shown in FIG. 6
- communication between the user terminal 100 and the movable object 300 can be conducted using protocol call processes 255 that are transmitted back and forth between the user terminal 100 and the movable object 300 .
- the protocol call processes 255 can be transmitted according to a movable object protocol 250 (shown in FIG. 4 ).
- the protocol call processes 255 can include structured data that encodes signals for the movable object 300 to take specific actions, or signals from the movable object 300 indicating present status.
- the protocol call processes 255 can advantageously be encrypted to avoid third-party intrusion and unwanted interference with movable object operations. In such cases, the protocol call processes 255 can be configured to be decrypted only by verified or trusted users. In some embodiments, the protocol call processes 255 can be formatted according to a proprietary format that is decipherable only by a designer or manufacturer of the movable object 300 . Consequently, the protocol call processes 255 can be made opaque to users, and decipherable only by the designer or manufacturer for troubleshooting purposes.
- signals received by the movable object 300 can be processed by firmware 380 within the movable object 300 .
- the firmware 380 can reside, for example in a read-only memory of the movable object 300 .
- the firmware 380 can be configured to decode protocol call processes 255 received from the user terminal 100 , and send corresponding signals to the movable object components 310 a , 310 b , and 310 c of the movable object 300 . Accordingly, the protocol call processes 255 can be translated directly into physical action by the movable object components 310 .
- the movable object protocol 250 can include an application layer 251 , a network layer 252 , and a data link layer 253 .
- the application layer 251 can be used, for example, for handling application logic, such as controlling behavior of functional modules 310 of a movable object 300 (shown in FIG. 1 ).
- the network layer 252 can be used, for example, for supporting data packet routing and relaying.
- the data link layer 253 can be used, for example, for handling data framing, data verification, and data retransmission.
- the movable object protocol 250 can support communication between various modules of a UAV 320 , such as a flight control module, gimbal module, camera module and/or other modules.
- the movable object protocol 250 can be used with different communication link technologies, such as a universal asynchronous receiver/transmitter (UART) technology, a controller area network (CAN) technology, and an inter-integrated circuit (I2C) technology.
- UART universal asynchronous receiver/transmitter
- CAN controller area network
- I2C inter-integrated circuit
- the packet 260 can include a header 261 , an application header 262 , data 263 , and a tail 264 .
- the header 261 and tail 264 can include, for example, control information that a network needs for delivering the data 263 .
- the control information can include source and destination network addresses, error detection codes, and packet sequencing information.
- the application header 262 can include, for example, various sender and receiver information.
- the sender and receiver can be among different functional modules 310 on the movable object 300 and applications 160 on the user terminal 100 (shown in FIG. 1 ).
- an exemplary cache 190 is shown in a movable object environment 10 that advantageously allows access to data, such as asynchronous remote data, from a movable object 300 .
- the cache can be a part of or otherwise associated with a movable object manager 180 , as shown above with reference to FIG. 3 .
- the exemplary cache 190 is shown as including listeners 191 , a store 195 , and a hardware abstraction 197 .
- the present multi-layered configuration of the cache 190 advantageously separates states of the movable object data 311 from logic, thereby increasing the ease of implementation and testing.
- any number of listeners 191 , stores 195 , and hardware abstractions 197 can be created, used, and/or destroyed within the cache 190 , as appropriate.
- multiple applications 160 can access the cache 190 at the same time.
- access to the cache 190 can be limited to a single application 160 .
- a single hardware abstraction 197 is shown in FIG. 6 for illustrative purposes only, any number of hardware abstractions 197 can be used, as desired.
- the cache 190 can be implemented, for example, by allocating one or more blocks of memory in the memories 130 (shown in FIG. 1 ) for storing movable object data 311 . The blocks of memory can be de-allocated when the cache 190 no longer needs the blocks.
- the one or more listeners 191 can be registered for listening to and/or providing access to particular movable object data 311 .
- a listener 191 can be registered by an application 160 that desires access to the particular movable object data 311 .
- the application 160 can invoke an addListener( ) method in a movable object API to create, or instantiate, a listener 191 in the movable object environment 10 .
- the addListener( ) method can include a movable object data key 192 that identifies the movable object data 311 of interest to the listener 191 .
- the movable object data key 192 can be represented, for example, as a parameter to the addListener( ) method—for example, as a string having a defined format, or as any other suitable type or class.
- the movable object data key 192 can include different data fields (shown in FIG. 8 ) that identify a particular movable object component 310 as well as a particular attribute of the movable object component 310 .
- the cache 190 can further include a store 195 in the movable object environment 10 for storing values of movable object data 311 .
- the store 195 can be a software representation of underlying blocks of memory for storing the values of the movable object data 311 .
- the memory can be allocated and de-allocated, as needed.
- the store 195 can be represented using any suitable data structure, such as an array, a list, a hash table, a tree, a queue, and the like.
- a separate store 195 can be created for each movable object component 310 . In such cases, each value of the movable object data 311 can be referenced according to a corresponding attribute.
- a single store 195 can include movable object data 311 for multiple movable object components 310 .
- the movable object data 311 can be referenced according to an identifier (for example, a string) corresponding to the movable object component 310 , as well as an identifier (for example, a string) corresponding to the attribute of the movable object component.
- a single store 195 can be created in the movable object environment 10 for all movable object data 311 (as shown in FIG. 6 ).
- the movable object data 311 can be represented as a constant, read-only, or immutable value in the store 195 .
- the cache 190 receives a new or updated value of the movable object data 311 , a prior read-only representation of the movable object data 311 can be deleted from the store 195 , and a new read-only representation of the movable object data 311 can be created and placed in the store 195 .
- a new movable object data key 192 can be created and associated with the new representation of the movable object data 311 .
- the representation of the movable object data 311 as a constant ensures consistency of the movable object data 311 among multiple threads, thereby improving thread safety and improving application performance.
- the use of immutable representations of the movable object data 311 to handle thread-shared data advantageously avoids the thread locks and can improve performance.
- movable object data 311 can be represented in the cache 190 as a mutable or variable value.
- mechanisms such as thread locks can be used to avoid data corruption between multiple threads.
- the cache 190 can further include one or more hardware abstractions 197 in the movable object environment 10 .
- Each of the hardware abstractions 197 can correspond to particular movable object component 310 .
- each of the hardware abstractions 197 can be a software object for interacting with corresponding movable object components 310 .
- a FlightController object can be instantiated in the movable object environment 10 that corresponds to a flight controller 321 (shown in FIG. 2 ).
- a Camera object can be instantiated in the movable object environment 10 that corresponds to an imaging device 325 (shown in FIG. 2 ).
- a Battery object can be instantiated in the movable object environment 10 that corresponds to a battery 327 of a UAV 320 (shown in FIG. 2 ), and so forth.
- the hardware abstractions 197 can be organized using inheritance.
- a base Camera class can be defined as having attributes that are common to a variety of imaging devices 325 , for example, an ISO setting.
- the Camera base class can include, for example, getISO( ) and set ISO( ) methods, as desired.
- a subclass HighPerformanceCamera can be defined as inheriting from the Camera base class, and include accessors specific to a high performance imaging device 325 , as appropriate.
- one or more of the hardware abstractions 197 can be configured to correspond to movable object components 310 having one or more subcomponents.
- a hardware abstraction can be instantiated to correspond to a flight controller 321 having a GPS 322 and/or an IMU 323 .
- Movable object data 311 of the subcomponents can be accessed using a movable object data key 192 , as described above.
- a hardware abstraction 197 can be created for communication with or access to a specific corresponding movable object component 310 .
- an application 160 can create a hardware abstraction 197 using a suitable application call process 165 .
- a hardware abstraction 197 can be removed or destroyed when communication with or access to the specific corresponding movable object component 310 is no longer desired.
- the number of hardware abstractions 197 that are instantiated at any one time can advantageously be kept low so as to reduce memory usage and improve efficiency.
- the number of hardware abstractions 197 can be kept below a predetermined threshold (for example, a maximum of 20 hardware abstractions 197 ).
- the key router 198 can include a dictionary of movable object data keys 192 , as well values of the movable object data 311 associated with each of the movable object data keys 192 .
- a dictionary for the key router 198 can be represented using any suitable data structure, such as an array, a list, a hash table, a tree, a queue, and the like.
- the key router 198 can thereby define the available movable object data keys 192 representing various attributes of the movable object component 310 .
- the key router 198 can define various characteristics of each attribute, such as which type of access (for example, read access, write access, or any combination thereof) is available for each of the attributes. Such characteristics can, for example, be associated with the movable object data key 192 corresponding to each attribute of the movable object component 310 .
- the accessors 199 of the hardware abstraction 197 can include, for example, a get( ) method to retrieve the value of a particular attribute, or a set( ) method to set a value of a particular attribute.
- the specific details of the accessors 199 can be implemented by a developer, as appropriate.
- the key router 198 can be implemented as an immutable dictionary, such that the attributes of the movable object components 310 are unchanged.
- exemplary movable object data keys 192 are shown in a string format, wherein each string takes on a predetermined key pattern. Shown on the left side of FIG. 8 are exemplary key patterns, which can be as “component/key,” “component/index/key,” “component/index/subcomponent/key,” or “component/index/subcomponent/subcomponent index/key.”
- “component” can represent a movable object component 310 (for example, “camera” to represent an imaging device 325 ) to be accessed using the movable object data key 192 .
- Index can represent a numeric value corresponding to a particular movable object component 310 of a specific type (for example, “camera/1” to represent the first of two imaging devices 325 ).
- Subcomponent can represent a subcomponent of the movable object component 310 (for example, a GPS 322 of a flight controller 321 ).
- Subcomponent index can represent a numeric value of corresponding to a particular subcomponent of the movable object component 310 (for example, “gps/1” to refer to the first of two GPS 322 devices of a flight controller 321 ).
- Key can represent a particular attribute, parameter, or other data associated with the movable object component 310 or subcomponent.
- a movable object data key 192 for a satellites number attribute for a second GPS 322 device of a first flight controller 321 can be represented as the string “flightcontroller/1/gps/2/satellitesNumber,” as shown in FIG. 8 .
- string representation of a movable object data key 192 can be converted by the cache 190 into any other data type for internal use and communication within the cache 190 .
- the cache 190 can use a KeyPath object that includes methods for parsing the string representation of the movable object data key 192 , as well as includes attributes for each portion string.
- Using a KeyPath or like object advantageously avoids re-parsing the string representation of the movable object data key 192 .
- an application 160 is interested in values of a “satellites number” attribute of a GPS 322 device of a flight controller 321 of a UAV 320 (not shown).
- the application 160 has a need to be notified of changes in value of the satellites number field.
- the application 160 can access a movable object interface 180 having an addListener( ) method for registering a listener 191 .
- the application 160 can invoke the addListener( ) method in an application call process 165 that specifies a movable object data key 192 (for example, a string key “flightcontroller/1/gps/satellitesNumber”) as a parameter.
- a movable object data key 192 for example, a string key “flightcontroller/1/gps/satellitesNumber”
- a new listener 191 is created corresponding to the movable object data key 192 “flightcontroller/1/gps/satellitesNumber.” Thereafter, the listener 191 that is created can be registered with the appropriate movable object component 310 (here, the flight controller 321 and/or the GPS 322 subcomponent). The listener 191 can be associated with a hardware abstraction 197 that corresponds to the movable object component 310 . When a value of the attribute corresponding to the listen 191 is changed, the listener 191 can be used to notify the application 160 of the change and/or the new value via the appropriate movable object data 311 .
- a listener 191 is registered, via a movable object manager 180 , for listening to movable object data 311 in a movable object environment 10 .
- the listener 191 can be registered in a movable object software environment 50 (shown in FIG. 3 ).
- the listener 191 can be registered through a movable object application programming interface (API), as described above with reference to FIG. 3 .
- the movable object API can include, for example, one or more functions, methods, procedures, parameters, constants, variables, objects, modules, datatypes, exceptions, etc.
- the listener 191 can be registered through a movable object API that is part of a movable object software development kit (SDK).
- SDK movable object software development kit
- the movable object SDK can include tools, libraries, documentation, and sample code that can help a software developer use the SDK for developing a movable object application 160 .
- the listener 191 can be registered with a movable object component 310 corresponding to the specific movable object data 311 .
- a listener 191 for a GPS 322 device of a flight controller 321 can be registered with the flight controller 321 and/or the GPS 322 .
- the listener 191 can be registered with a hardware abstraction 197 corresponding to the movable object component 310 .
- a hardware abstraction 197 for the flight controller 321 and/or the GPS 322 can be instantiated during runtime of the application 160 .
- the hardware abstraction 197 can have a key router 198 that is initialized with relevant attributes of the movable object component 310 and/or various subcomponents thereof.
- the listener 191 can be registered using an appropriate movable object data key 192 that identifies the movable object component 310 .
- the movable object data key 192 can be a string having a substring that identifies the movable object component 310 (for example, “flightcontroller” to identify the flight controller 321 ).
- the movable object data key 192 can further identify an attribute of the movable object component 310 or subcomponent thereof (for example, “satellitesNumber” to identify a satellites number attribute of the GPS 322 ).
- the listener 191 can be de-registered when movable object data 191 from the movable object component 310 is no longer desired.
- the application 160 can be configured to signal to the movable object manager 180 that the application 160 is no longer interacting with the movable object 300 .
- the application 160 can send a stop signal to the movable object manager 180 .
- the movable object manager 180 loses connection with the application 160 , whereupon the listener 191 can be automatically de-registered.
- movable object data 311 which is received from the movable object 300 can be stored in a cache 190 associated with the movable object manager 180 .
- the movable object data 311 can be received using the listener 191 via a request for the movable object data 311 that uses the movable object data key 192 .
- the listener 191 can determine whether the movable object data 311 is available in the cache 190 , and/or whether the movable object data 311 is updated in the cache 190 .
- the movable object 300 can be queried for an updated value of the movable object data 311 .
- the application 160 can query the movable object 300 for the updated value of the movable object data 311 .
- the movable object 300 can be queried for the value of the movable object data 311 using a request, such as a get( ) method in the movable object interface 170 .
- the cache 190 can be configured to automatically query the movable object 300 for the updated value of the movable object data 311 upon determining that the movable object data 311 is not available or not updated.
- the request can be performed via the hardware abstraction 197 that corresponds with the movable object component 310 having the desired movable object data 311 .
- the request can be performed using a protocol call process 255 that is transmitted to the movable object 300 using the hardware abstraction 197 .
- an updated value of the movable object data 311 can be received from the movable object (for example, using a protocol call process 255 transmitted from the movable object 300 ).
- movable object data 311 that is received at 1020 can be provided to an application 160 corresponding to the listener 191 .
- the application 160 can be an application 160 for interacting with and/or controlling the movable object 300 .
- the listener 191 can be used to detect a change in a value of the movable object data 311 . The change can be detected, for example, by storing a prior value of the movable object data 311 in the cache 190 .
- the listener 191 can periodically receive an updated value of the movable object data 311 and compare the prior value to the updated value to detect a change in the value.
- the listener can notify the application 160 of the change in the movable object data 311 .
- the changed value of the movable object data 311 can be automatically provided to the application 160 .
- the movable object data 311 is not automatically provided to the application 160 , and the application can optionally request the movable object data 311 , if desired.
- FIG. 11 an exemplary process for requesting movable object data 311 is shown in accordance with the examples provided above.
- the application 160 is to request a “satellites number” attribute of a GPS 322 device of a flight controller 321 of a UAV 320 (not shown).
- a request for the movable object data 311 can be sent to the movable object manager 180 (shown in FIG. 3 ) using an interface 170 (shown in FIG. 3 ).
- the request can be sent using an application call process 165 and an appropriate movable object data key 192 .
- a get(“flightcontroller/1/gps/satellitesNumber”) method can be invoked by the application 160 .
- the movable object data 311 can be determined. Where the movable object data 311 is available and/or updated, the movable object can advantageously be provided back to the application 160 from the cache 190 without querying the movable object 300 , thereby reducing the number of protocol call processes 255 sent over potentially limited bandwidth. Where the movable object data 311 is unavailable and/or not updated, the movable object can be retrieved from the movable object 300 using the hardware abstraction 197 corresponding to the flight controller 321 of interest.
- FIG. 12 an exemplary top-level block diagram for a method 1200 for managing communication with a movable object 300 is shown.
- a request for movable object data 311 is received at a movable object manager 180 from a requester in a movable object environment 10 .
- the requester can be, for example, an application 160 (shown in FIG. 3 ) or any other process running in the movable object environment 10 .
- the request can be made as an application call process 165 using an appropriate movable object data key 192 .
- whether movable object data 311 corresponding to the request is available in a cache 190 associated with the movable object manager 180 can be determined. For example, a store 195 of the cache 190 can be evaluated for whether movable object data 311 corresponding to the movable object data key 192 exists. In some embodiments, whether movable object data 311 is updated in the cache 190 can be determined.
- Whether the movable object data 311 is updated can be determined according to a set of predetermined rules. In some embodiments, whether the movable object data 311 is updated can be determined according to a data type of the movable object data 311 . For example, certain types of movable object data 311 are expected to remain fixed for the duration of use of the movable object 300 (for example, movable object data 311 reflecting mechanical configurations of the movable object 300 , such as the number of batteries, is fixed in the duration of a mission of the movable object). In another example, some types of movable object data 311 can be regularly transmitted (or “pushed”) from the movable object 300 to the movable object manager 180 , and stored in the cache 190 . Some types of “push” data can be assumed to be updated, without need for further determination.
- whether the movable object data 311 is updated can be determined according to a time stamp t of the movable object data 311 .
- the time stamp t can be a time at which the movable object data 311 was transmitted by the movable object 300 , and/or a time at which the movable object data 311 was received by the movable object manager 180 .
- the time stamp t of the movable object data 311 can be stored in the cache 190 , as desired.
- the time stamp t of the movable object data 311 that is stored in the cache 190 can be compared with the current time to of the request to find a lapsed time ⁇ t since the movable object data 311 was last updated.
- the lapsed time ⁇ t can be compared to a predetermined threshold T to determine whether the movable object data 311 is updated or not.
- the predetermined threshold T can be specific to the type of movable object data 311 . For example, certain types of movable object data 311 (for example, the number of satellites of a GPS 322 ) need not be regularly updated, and the predetermined threshold T can be set to a large value accordingly. In some embodiments, other types of movable object data 311 (for example, the charge status of a battery 327 ) are regularly updated, and the predetermined threshold T can be set to a smaller value accordingly.
- whether the movable object data 311 is updated can be determined according to a prior value of the movable object data 311 . For example, a position of the movable object 300 can be determined to be updated where the movable object is positioned at a home base, or along an expected trajectory. If the position of the movable object 300 is unexpected (for example, off trajectory or in a restricted area), the position can be determined to be not updated.
- the movable object data 311 in the cache 190 can be provided to the requester.
- the movable object data 311 in the cache 190 can be directly provided to the requester without querying the movable object 300 , advantageously saving time and bandwidth.
- FIG. 13 further details of various embodiments of the method 1200 are shown.
- whether the movable object data 311 is updated in the cache 190 can be determined, as described above with reference to FIG. 12 . If the movable object data 311 is not updated, then the movable object 300 is queried for an update value at 1222 . At 1223 , the updated value of the movable object data 311 is subsequently received from the movable object 300 . At 1224 , the updated value the movable object data 311 is stored in the cache 190 . Alternatively, if the movable object data 311 is updated, then the movable object data 311 is retrieved from the cache 190 at 1225 . Finally, at 1231 , the movable object data 311 in the cache 190 is provided to the requester of the movable object data 311 .
- FIG. 14 exemplary diagrams are shown illustrating various embodiments for receiving movable object data 311 from a movable object 300 .
- a “push” mechanism is shown whereby movable object data 311 is automatically transmitted by the movable object 300 to the movable object manager 180 .
- Certain types of data, as desired, can be designated “push” data and automatically transmitted to the movable object manager 180 .
- the movable object data 311 can be periodically transmitted from the movable object 300 , received by the movable object manager 180 , and stored in a cache 190 .
- the frequency of transmission of such “push” data can vary, as desired.
- a “push-like” mechanism is shown that can be used when no “push” data is available.
- the movable object manager 180 can send protocol call processes 255 , such as get( ) methods in a movable object interface 170 , to retrieve the movable object data 311 from the movable object 300 .
- the movable object data 311 can be periodically retrieved using automatic get( ) methods, and stored in the cache 190 .
- both the “push” and “push-like” mechanisms of retrieving the movable object data 311 can be used for different types of movable object data 311 for maximum flexibility and efficiency.
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Aviation & Aerospace Engineering (AREA)
- Remote Sensing (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Radar, Positioning & Navigation (AREA)
- Computer Security & Cryptography (AREA)
- Computing Systems (AREA)
- Medical Informatics (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Human Computer Interaction (AREA)
- Stored Programmes (AREA)
- Mobile Radio Communication Systems (AREA)
- Telephonic Communication Services (AREA)
Abstract
Description
- This application is a continuation of application Ser. No. 16/268,055, filed on Feb. 5, 2019, which is a continuation of International Application No. PCT/CN2016/094327, filed on Aug. 10, 2016, the entire contents of both of which are incorporated herein by reference.
- A portion of the disclosure of this patent document contains material which is subject to copyright protection. The copyright owner has no objection to the facsimile reproduction by anyone of the patent document or the patent disclosure, as it appears in the Patent and Trademark Office patent file or records, but otherwise reserves all copyright rights whatsoever.
- The disclosed embodiments relate generally to movable objects and more particularly, but not exclusively, to systems and methods for managing communication with a movable object.
- Movable objects, such as unmanned aircraft, can be used in many different fields such as film production, sporting events, disaster relief, geological study, and more. Movable objects can be configured to transmit remote data, such as video data, back to applications that interface with the movable object. Furthermore, such applications can regularly query data, such as control or state data, from the movable object. However, communications between the movable object and the application usually take place over limited bandwidth that can result in latency, which results in asynchronous access to the movable object data by the application. This problem can be worsened by poor application design that unnecessarily increases usage of limited communication bandwidth. Presently, there lacks uniform methods and systems for providing efficient movable object communication without compromising performance.
- Accordingly, there is a need for systems and methods that allow for improved communication with a movable object.
- In accordance with a first aspect disclosed herein, there is set forth a method of managing communication with a movable object, comprising: receiving, at a movable object manager, a request for movable object data from a requester in a movable object environment; determining whether movable object data corresponding to the request is available in a cache associated with the movable object manager; and providing the movable object data in the cache to the requester.
- In accordance with another aspect disclosed herein, there is set forth a method of managing communication with a movable object, comprising: registering, via a movable object manager, a listener for listening to movable object data in a movable object environment; storing the movable object data, which is received from the movable object, in a cache associated with the movable object manager; and providing the movable object data to an application corresponding to the listener.
- In accordance with another aspect disclosed herein, there is set forth a system for managing communication with a movable object, comprising: one or more processors configured for: receiving, at a movable object manager, a request for movable object data from a requester in a movable object environment; determining whether movable object data corresponding to the request is available in a cache associated with the movable object manager; and providing the movable object data in the cache to the requester.
- In accordance with another aspect disclosed herein, there is set forth a system for managing communication with a movable object, comprising: one or more processors configured for: registering, via a movable object manager, a listener for listening to movable object data in a movable object environment; storing the movable object data, which is received from the movable object, in a cache associated with the movable object manager; and providing the movable object data to an application corresponding to the listener.
- In accordance with another aspect disclosed herein, there is set forth a non-transitory computer-readable medium with instructions stored thereon that, when executed by a processor, perform the steps comprising: receiving, at a movable object manager, a request for movable object data from a requester in a movable object environment; determining whether movable object data corresponding to the request is available in a cache associated with the movable object manager; and providing the movable object data in the cache to the requester.
- In accordance with another aspect disclosed herein, there is set forth a non-transitory computer-readable medium with instructions stored thereon that, when executed by a processor, perform the steps comprising: registering, via a movable object manager, a listener for listening to movable object data in a movable object environment; storing the movable object data, which is received from the movable object, in a cache associated with the movable object manager; and providing the movable object data to an application corresponding to the listener.
-
FIG. 1 is an exemplary top-level block diagram illustrating an embodiment of a movable object environment that includes a user terminal in communication with a movable object through a communication link. -
FIG. 2 is an exemplary diagram illustrating an embodiment of the movable object ofFIG. 1 , wherein the movable object is a UAV. -
FIG. 3 is an exemplary block diagram illustrating an embodiment of an environment of the movable object operation system ofFIG. 1 . -
FIG. 4 is an exemplary diagram illustrating an embodiment of a communication protocol for use in the movable object environment ofFIG. 1 . -
FIG. 5 is an exemplary diagram illustrating an embodiment of a data packet for use in the movable object environment ofFIG. 1 . -
FIG. 6 is an exemplary block diagram illustrating an embodiment of a cache for use in the movable object environment ofFIG. 1 . -
FIG. 7 is an exemplary block diagram illustrating an embodiment of a hardware abstraction in the cache ofFIG. 6 . -
FIG. 8 is an exemplary diagram illustrating various embodiments of a movable object data key for use in the movable object environment ofFIG. 1 . -
FIG. 9 is an exemplary diagram illustrating an embodiment of a method for adding a new listener corresponding to movable object data of interest. -
FIG. 10 is an exemplary top-level flow chart illustrating an embodiment of a method for managing communication with the movable object ofFIG. 1 . -
FIG. 11 is an exemplary diagram illustrating an embodiment of a method for retrieving movable object data of interest. -
FIG. 12 is another exemplary top-level flow chart illustrating an embodiment of a method for managing communication with the movable object ofFIG. 1 . -
FIG. 13 is exemplary flow chart illustrating further details of the method ofFIG. 12 for managing communication with the movable object ofFIG. 1 . -
FIG. 14 is an exemplary diagram illustrating embodiments of a method for receiving movable object data from a movable object using “push” and “push-like” mechanisms. - It should be noted that the figures are not drawn to scale and that elements of similar structures or functions are generally represented by like reference numerals for illustrative purposes throughout the figures. It also should be noted that the figures are only intended to facilitate the description of the embodiments. The figures do not illustrate every aspect of the described embodiments and do not limit the scope of the present disclosure.
- The present disclosure sets forth systems and methods for managing communication with a movable object, which overcome limitations of prior systems and methods. More particularly, the present systems and methods relate to the use of a cache for regulating access to movable object data. By using a cache, potentially redundant calls to retrieve movable object data can advantageously be reduced or eliminated. The present systems and methods can thereby regulate efficient access to remote asynchronous movable object data, thereby improving usage of limited bandwidth and improving application performance. Furthermore, the present systems and methods can utilize an application programming interface (API) that offers programmers convenient and encapsulated access to the remote asynchronous movable object data. Exemplary embodiments of the present systems and methods are described below.
- Turning now to
FIG. 1 , an exemplarymovable object environment 10 is shown in accordance with various embodiments of the present systems and methods. Themovable object environment 10 can include auser terminal 100, which can communicate with amovable object 300 via acommunication link 200. One or more of theuser terminal 100, thecommunication link 200, and/or themovable object 300 can comprise a system for managing movable communications, as further described below. - The
user terminal 100 can be configured to interact with a user (not shown) to operate themovable object 300 and/or present data collected by themovable object 300 to the user. Theuser terminal 100 can include, for example, remote controllers (not shown), portable computers, laptops, mobile devices, handheld devices, mobile telephones (for example, smartphones), tablet devices, tablet computers, personal digital assistants, handheld consoles, portable media players, wearable devices (for example, smartwatches and head-mounted displays), and the like. - As shown in
FIG. 1 , the user terminal 100 (and/or components thereof) can include one ormore processors 110 that can be used to execute software. Theuser terminal 100 can include any number ofprocessors 110, as desired. Without limitation, eachprocessor 110 can include one or more general purpose microprocessors (for example, single or multi-core processors), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), application-specific instruction-set processors, digital signal processing units, coprocessors, network processing units, audio processing units, encryption processing units, and the like. Theprocessors 110 can be configured to perform any of the methods described herein, including but not limited to a variety of tasks relating to mobile object control, tracking, and analysis. In some embodiments, theprocessors 110 can include specialized software and/or hardware, for example, for processing movable object tasks using an interface. In some embodiments, theprocessors 110 can execute an analyzer that operates to perform any of the methods described herein. - As shown in
FIG. 1 , theuser terminal 100 can further include one or more memories 120 (alternatively referred to herein as non-transient computer readable media).Suitable memories 120 can include, for example, random access memory (RAM), static RAM, dynamic RAM, read-only memory (ROM), programmable ROM, erasable programmable ROM, electrically erasable programmable ROM, flash memory, secure digital (SD) card, and the like. Instructions for performing any of the methods described herein can be stored in thememory 120. Thememory 120 can be placed in operative communication with theprocessors 110, as desired, and instructions and/or data can be transmitted from thememory 120 to theprocessors 110 for execution, as desired. - The
user terminal 100 can additionally include one or more input/output devices 130, such as buttons, a keyboard, keypad, trackball, displays, and/or a monitor. Various user interface elements (for example, windows, buttons, menus, icons, pop-ups, tabs, controls, cursors, insertion points, and the like) can be used to present data to and receive data from a user (not shown). - The
user terminal 100 can be configured for communication with themovable object 300 via thecommunication link 200. As shown inFIG. 1 , thecommunication link 200 can include an uplink for transmitting data (such as control data and application data) from theuser terminal 100 to themovable object 300, and a downlink for transmitting data (such as telemetry data, application data, image data, and video data) from themovable object 300 to the user terminal. In some embodiments, the uplink and downlink can share a single frequency using time modulation. In other embodiments, the uplink and downlink can use different frequencies. In some embodiments, thecommunication link 200 can be configured to transmit signals according to a movable object protocol 250 (shown inFIG. 4 ). - In some embodiments, the
communication link 200 can be awireless communication link 200 over a wireless network. Suitable wireless communications can include, for example, radio, Wireless Fidelity (WiFi), cellular, satellite, and broadcasting. In certain embodiments, thecommunication link 200 can be implemented over a 3G, 4G, or 5G mobile telecommunications network. Exemplary wireless communication technologies that are suitable for the present systems and methods include, but are not limited to, Global System for Mobile Communications (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband CDMA (W-CDMA), CDMA2000, IMT Single Carrier, Enhanced Data Rates for GSM Evolution (EDGE), Long-Term Evolution (LTE), LTE Advanced, Time-Division LTE (TD-LTE), High Performance Radio Local Area Network (HiperLAN), High Performance Radio Wide Area Network (HiperWAN), High Performance Radio Metropolitan Area Network (HiperMAN), Local Multipoint Distribution Service (LMDS), Worldwide Interoperability for Microwave Access (WiMAX), ZigBee, Bluetooth, Flash Orthogonal Frequency-Division Multiplexing (Flash-OFDM), High Capacity Spatial Division Multiple Access (HC-SDMA), iBurst, Universal Mobile Telecommunications System (UMTS), UMTS Time-Division Duplexing (UMTS-TDD), Evolved High Speed Packet Access (HSPA+), Time Division Synchronous Code Division Multiple Access (TD-SCDMA), Evolution-Data Optimized (EV-DO), Digital Enhanced Cordless Telecommunications (DECT) and others. - In some embodiments, the
communication link 200 can advantageously be encrypted to prevent third party intrusion into movable object operations. Suitable encryption methods include, but are not limited to, internet key exchange, Internet Protocol Security (IPsec), Kerberos, point-to-point protocol, transport layer security, SSID hiding, MAC ID filtering, Static IP addressing, 802.11 security, Wired Equivalent Privacy (WEP), Wi-Fi Protected Access (WPA), WPA2, Temporal Key Integrity Protocol (TKIP), Extensible Authentication Protocol (EAP), Lightweight Extensible Authentication Protocol (LEAP), Protected Extensible Authentication Protocol (PEAP), and the like. - The
movable object 300 in the present systems and methods can include any type of movable object, including bicycles, automobiles, trucks, ships, boats, trains, helicopters, aircraft, robotic devices, various hybrids thereof, and the like. Although themovable object 300 is shown in some examples herein as being a flying movable object for illustrative purposes only, such examples are not meant to be limiting. The embodiments described herein in the context of flying movable objects can be applied to any suitablemovable object 300. - The
movable object 300 can include or more functional modules. Each of the functional modules of themovable object 300 can include one or moremovable object components 310 for carrying out, for example, movement, communication, imaging, and/or other functionality. For example,movable object components 310 can include one or more movement mechanisms 312 (for example, propulsion mechanisms), asensing system 314, and/or acommunication system 316. Themovable object 300 is further shown as having acarrier 318 that includes apayload 319. Themovable object 300 can further include hardware components such as processors, memories, etc., as desired. Each of themovable object components 310 can be associated with one or more attributes. The attributes and attribute values of themovable object components 310 constitute movable object data 311 (not shown) which can be generated, retrieved, stored, and used in a manner described herein. - The movement mechanisms 312 can include one or more rotors, propellers, blades, engines, motors, wheels, axles, magnets, nozzles, and the like. For example, the
movable object 300 can have one or more propulsion mechanisms. In some embodiments, themovable object 300 can have multiple movement mechanisms 312 can be of the same type or different types of movement mechanisms 312. The movement mechanisms 312 can be mounted on themovable object 300 using any suitable means such as a support element (for example, a drive shaft). The movement mechanisms 312 can be mounted on any suitable portion of themovable object 300, such on the top, bottom, front, back, sides, or suitable combinations thereof.Movable object data 311 associated with the movement mechanism 312 can include, for example, data relating to positions, on/off status, and/or wear and tear status of the movement mechanism 312. - In some embodiments, the movement mechanisms 312 can enable the
movable object 300 to take off vertically from a surface or land vertically on a surface without requiring horizontal movement of the movable object 300 (for example, without traveling down a runway). Optionally, the movement mechanisms 312 can be operable to enable the movable object 112 to hover at a specified position and/or orientation. One or more of the movement mechanisms 312 can be controlled independently of the other movement mechanisms 312. For example, themovable object 300 can have multiple horizontally oriented rotors that can provide lift and/or thrust to themovable object 300. The rotors can be actuated to provide vertical takeoff, vertical landing, and hovering capabilities to themovable object 300. In some embodiments, one or more of the rotors can spin in a clockwise direction, while one or more of the rotors can spin in a counterclockwise direction. For example, the number of clockwise rotors can be equal to the number of counterclockwise rotors. The rotation rate of each of the rotors can be varied independently in order to control the lift and/or thrust produced by each rotor, and thereby adjust the spatial disposition, velocity, and/or acceleration of the movable object 300 (for example, to up to three degrees of translation and up to three degrees of rotation). - The
sensing system 314 can include one or more sensors that can sense the spatial disposition, velocity, and/or acceleration of the movable object 300 (for example, with respect to various degrees of translation and various degrees of rotation). The one or more sensors can include sensors such as global positioning system (GPS) sensors, motion sensors, inertial sensors (for example, inertial measurement units (IMU)), proximity sensors, image sensors, attitude and heading reference systems (AHRS), radar sensors, lidar sensors, ultrasound sensors, infrared sensors, as well as other sensors that detect motion by optics, radio, sound, vibration, magnetism, and the like.Movable object data 311 provided by thesensing system 314 can be used to control the spatial disposition, velocity, and/or orientation of the movable object 300 (for example, using a suitable processing unit and/or control module). - Alternatively, and/or additionally, the
sensing system 314 can be used to collectmovable object data 311 regarding an operational environment surrounding themovable object 300, such as weather conditions, proximity to potential obstacles, location of geographical features, location of manmade structures, and the like. For example, thesensing system 314 for sensing the environment can include electro-optical sensors, thermal/infrared sensors, color or monochrome sensors, multi-spectral imaging sensors, spectrophotometers, spectrometers, thermometers, illuminometers, microphones/sonic transducers, pressure sensors, altitude sensors, flow sensors, humidity sensors, precipitation sensors, wind speed sensors, wind direction sensors, anemometers, optical rain sensors, positioning devices, accelerometers, gyroscopes, and/or others. - The
communication system 316 can enable communication with theuser terminal 100 via thecommunication link 200, which can include various wired and/or wireless technologies as discussed above. Thecommunication system 316 can include any number of transmitters, receivers, and/or transceivers suitable for wireless communication. The communication can be one-way communication whereby themovable object 300 transmitsmovable object data 311 to theuser terminal 100, or vice-versa. The communication can also be two-way communication, wherebymovable object data 311 can be transmitted in both directions between themovable object 300 and theuser terminal 100. - In some embodiments, the
user terminal 100 can providemovable object data 311 that is control data to one or more of themovable object 300,carrier 318, andpayload 319 and receive information from one or more of themovable object 300,carrier 318, and payload 319 (for example, position and/or motion information of the movable object, carrier or payload; data sensed by the payload such as image data captured by a payload camera; and data generated from image data captured by the payload camera). In some embodiments, control data from theuser terminal 100 can include instructions for relative positions, movements, actuations, or controls of the movable object, carrier, and/or payload. For example, the control data can result in a modification of the location and/or orientation of the movable object (for example, via control of the movement mechanisms 312), or a movement of the payload with respect to the movable object (for example, via control of the carrier 318). The control data from the application can result in control of the payload. Anypayload 319 can be used with the present systems and methods. - In some embodiments, communications from the
movable object 300,carrier 318 and/orpayload 319 can include information from one or more sensors (for example, of thesensing system 314 or of the payload 319) and/or data generated based on the sensing information. The communications may include sensed information from one or more different types of sensors, as described above. Such information may pertain to the position (for example, location or orientation), movement, or acceleration of the movable object, carrier, and/or payload.Movable object data 311 from thepayload 319 can include data captured by thepayload 319 or a sensed state of thepayload 319. - Turning now to
FIG. 2 , an exemplarymovable object 300 is shown in amovable object environment 10 as an unmanned aerial vehicle (UAV) 320. Colloquially referred to as “drones,”UAVs 320 include aircraft without a human pilot onboard the vehicle whose flight is controlled autonomously or by a remote pilot (or sometimes both).UAVs 320 are now finding increased usage in civilian applications involving various aerial operations, such as data-gathering or delivery. The present movable object operation systems and methods are suitable for use with many types ofUAVs 320 including, without limitation, quadcopters (also referred to a quadrotor helicopters or quad rotors), single rotor, dual rotor, trirotor, hexarotor, andoctorotor rotorcraft UAVs 320, fixedwing UAVs 320, and hybrid rotorcraft-fixedwing UAVs 320. -
Movable object components 310 of aUAV 320 can include, for example, aflight controller 321 for controlling flight operations of theUAV 320. Theflight controller 321 can include one or more sensing systems 314 (described above with reference toFIG. 1 ), such as one ormore GPS 322 and/orIMU 323. Theflight controller 321 can be operated using aprocessor 324 embedded within theflight controller 321. Theflight controller 321 can, for example, control a movement of theUAV 320. Exemplary control of the movement includes setting target destination, velocity, altitude, attitude (pitch, roll, and yaw), and/or the like. Theflight controller 321 can communicate with selected data collection instruments (not shown) of theUAV 320 to obtainmovable object data 311. - The
UAV 320 can also include animaging device 325. Theimaging device 325 can receive incident light from an external environment and convert the incident light into digital and/or analog signals (not shown). The signals can be processed to produce an image that can be displayed to a user.Exemplary imaging devices 325 suitable for use with the present systems and methods include, but are not limited to, commercially-available cameras and camcorders. In some embodiments, theimaging device 325 can be adapted to use additional sensors (not shown) for various applications such as stereo vision, thermography, creation of multi-spectral images, infrared detection, gamma detection, x-ray detection, and the like. - The
imaging device 325 can be controlled in various ways, such as turn theimaging device 325 on and off, and get or set parameters of the imaging device 325 (for example, imaging angle, imaging mode, photo/video size/resolution, photo/video format, zoom settings, exposure settings, and the like). For example,movable object data 311 collected can include various parameters of theimaging device 325, still image data, and/or video image data from the imaging device. Exemplary still image data and video image data can respectively include still images and video images captured aerially by theimaging device 325. - The
UAV 320 can further include agimbal 326 for precise rotational and/or translational positioning of mounted objects, such as an imaging device. Thegimbal 326 can include one or more pivoted supports (not shown) that are mounted with respect to one or more axes of rotation, thereby allowing a mounted object to be rotationally independent with respect to a fixture to which thegimbal 326 is mounted. The manner of configuring and mounting the pivoted supports can vary and can include, for example, concentric ring structures and non-ring structures. Depending on the number of degrees of freedom desired, thegimbal 326 can include any suitable number, including one, two, or three, pivoted supports. Thegimbal 326 can include various mechanical apparatus as needed, such as ball bearings, tracks, actuators, and the like. Thegimbal 326 can be configured to respond to external commands (for example, from a processor) directing movement of theimaging device 325. Movable object data 311 (such as angles and positions) of thegimbal 326 can be retrieved, as desired. - The
UAV 320 can further include one ormore batteries 327 for providing power to theUAV 320 and/or any functional modules of theUAV 320. Eachbattery 327 can include any number of battery cells each having two electrodes (not shown) and an electrolyte (not shown) that reacts chemically to generate an electric potential, providing electric current between the electrodes. Thebattery 327 can be chargeable and/or rechargeable. Thebattery 327 can include any type of battery cells, including, but not limited to, lead-acid cells, lithium air cells, lithium-ion cells, nickel-cadmium cells, nickel-metal hydride hydrogen cells, and the like. In some embodiments, thebattery 327 be a smart battery whose internal state and charging/discharging are monitored by a battery manager (not shown). Movable object data 311 (such as remaining charge) of thebattery 327 can advantageously be retrieved and monitored, as desired, for operation of themovable object 300. - Turning now to
FIG. 3 , an exemplarymovable object environment 10 is shown as including auser terminal 100, as well as amovable object 300. Themovable object environment 10 can, for example, be a movable object software environment. Themovable object environment 10 can be configured using any convenient programming language, such as Java, C, C++, Python, and the like. Themovable object environment 10 can be operated using any appropriate hardware, as described above with reference toFIG. 1 . - As shown in
FIG. 3 , theuser terminal 100 can include one ormore applications user terminal 100. Where theuser terminal 100 is a mobile device, theapplication 160 can be colloquially referred to as an “app.” The app can be made available by a vendor and kept updated by the vendor through, for example, a mobile app store. - In some embodiments, the
applications interface 170. For example, theinterface 170 can be an application programming interface (API) that includes one or more defined elements such as functions, methods, procedures, parameters, constants, variables, objects, modules, datatypes, exceptions, etc. The API can be a movable object API that includes elements for interacting with and controlling amovable object 300. A software developer can, for example, program anapplication 160 using the API. When being implemented on theuser terminal 100, theapplication 160 can invoke the API and execute functionality specified by the API. - The
applications interface 170 through application call processes 165, as shown inFIG. 3 . For example, anapplication call process 165 can be a get( ) method call that is made by anapplication 160 to theinterface 170 to retrieve a value of particularmovable object data 311. As another example, theapplication call process 165 can be a set( ) method call to set a value for particularmovable object data 311. Another exemplaryapplication call process 165 is a method call to theinterface 170 that commands themovable object 300 to perform one or more tasks for themovable object 300. - In some embodiments, the
interface 170 can be part of a movable object software development kit (SDK). The movable object SDK can advantageously provide common, integrated functionality forapplications 160 that interact withmovable objects 300. For example,applications 160 that are designed to control a flyingmovable object 300 can invoke functions in the movable object SDK for controlling navigation of the flyingmovable object 300.Applications 160 relating to imaging from amovable object 300 can invoke functions in the movable object SDK for controlling an imaging device 325 (shown inFIG. 2 ) or gimbal 326 (shown inFIG. 2 ) of themovable object 300. The movable object SDK can further include tools, libraries, documentation and sample code that can help a software developer to use the SDK for developing a movable object application. - In some embodiments, the
movable object 300 can be managed using amovable object manager 180. Themovable object manager 180 can be located anywhere that is convenient. Though themovable object manager 180 is depicted as being a part of theuser terminal 100 inFIG. 3 for illustrative purposes only, themovable object manager 180 is not limited in locality or implementation. In some embodiments, themovable object manager 180 can be part of themovable object 300, or be any part of amovable object environment 10. Themovable object manager 180 can be configured with logic for determining when communication with themovable object 300 is necessary, and when such communication is unnecessary. - The
movable object manager 180 can be configured to manage themovable object 300 by, for example, receiving data, sending commands, and/or processing data in relation to themovable object 300. In some embodiments, themovable object manager 180 can be configured to manage themovable object 300 by handling application call processes 165 received through theinterface 170. Themovable object manager 180 can convert the received application call processes 165 into protocol call processes 255 to themovable object 300. Additionally, themovable object manager 180 can issue protocol call processes 255 to themovable object 300 that are not triggered through theinterface 170. For example, themovable object manager 180 can perform background tasks that relate to maintaining movable object performance (such as maintaining altitude, pitch, or heading of a flyingmovable object 300, checking that themovable object 300 has sufficient fuel remaining, or automatically directing the movable object to avoid restricted zones such as airports). In some embodiments, themovable object manager 180 can be configured to communicate with an authentication server (not shown) for providing a secure environment for communication between theuser terminal 100 and themovable object 300. - The
movable object manager 180 can include various modules, such as a communication module or data management module (not shown), as needed. As illustrated inFIG. 3 , themovable object manager 180 includes acache 190 for managing communications with themovable object 300. In particular, thecache 190 can store values of certainmovable object data 311 of interest. Themovable object data 311 can be retrieved asynchronously by anapplication 160. In some embodiments, an application that desires access to themovable object data 311 can register a listener 191 (shown inFIG. 6 ) that corresponds the movable object data. Thelistener 191 can facilitate updating and retrieving themovable object data 311, as desired. - In some embodiments, communication between the
user terminal 100 and themovable object 300 can be conducted using protocol call processes 255 that are transmitted back and forth between theuser terminal 100 and themovable object 300. The protocol call processes 255 can be transmitted according to a movable object protocol 250 (shown inFIG. 4 ). The protocol call processes 255 can include structured data that encodes signals for themovable object 300 to take specific actions, or signals from themovable object 300 indicating present status. - The protocol call processes 255 can advantageously be encrypted to avoid third-party intrusion and unwanted interference with movable object operations. In such cases, the protocol call processes 255 can be configured to be decrypted only by verified or trusted users. In some embodiments, the protocol call processes 255 can be formatted according to a proprietary format that is decipherable only by a designer or manufacturer of the
movable object 300. Consequently, the protocol call processes 255 can be made opaque to users, and decipherable only by the designer or manufacturer for troubleshooting purposes. - As shown in
FIG. 3 , signals received by themovable object 300 can be processed byfirmware 380 within themovable object 300. Thefirmware 380 can reside, for example in a read-only memory of themovable object 300. Thefirmware 380 can be configured to decode protocol call processes 255 received from theuser terminal 100, and send corresponding signals to themovable object components movable object 300. Accordingly, the protocol call processes 255 can be translated directly into physical action by themovable object components 310. - Turning now to
FIG. 4 , an exemplarymovable object protocol 250 for the communication link 200 (shown inFIG. 1 ) is shown in accordance with various embodiments of the present systems and methods. Themovable object protocol 250 can include anapplication layer 251, anetwork layer 252, and adata link layer 253. Theapplication layer 251 can be used, for example, for handling application logic, such as controlling behavior offunctional modules 310 of a movable object 300 (shown inFIG. 1 ). Thenetwork layer 252 can be used, for example, for supporting data packet routing and relaying. Thedata link layer 253 can be used, for example, for handling data framing, data verification, and data retransmission. - In some embodiments, the
movable object protocol 250 can support communication between various modules of aUAV 320, such as a flight control module, gimbal module, camera module and/or other modules. Themovable object protocol 250 can be used with different communication link technologies, such as a universal asynchronous receiver/transmitter (UART) technology, a controller area network (CAN) technology, and an inter-integrated circuit (I2C) technology. - Communications can be made using the
movable object protocol 250 through transmission ofpackets 260 through thecommunication link 200. Anexemplary packet 260 is shown inFIG. 5 . Thepacket 260 can include aheader 261, anapplication header 262,data 263, and atail 264. Theheader 261 andtail 264 can include, for example, control information that a network needs for delivering thedata 263. For example, the control information can include source and destination network addresses, error detection codes, and packet sequencing information. Theapplication header 262 can include, for example, various sender and receiver information. For example, the sender and receiver can be among differentfunctional modules 310 on themovable object 300 andapplications 160 on the user terminal 100 (shown inFIG. 1 ). - Turning now to
FIG. 6 , anexemplary cache 190 is shown in amovable object environment 10 that advantageously allows access to data, such as asynchronous remote data, from amovable object 300. The cache can be a part of or otherwise associated with amovable object manager 180, as shown above with reference toFIG. 3 . Theexemplary cache 190 is shown as includinglisteners 191, astore 195, and ahardware abstraction 197. The present multi-layered configuration of thecache 190 advantageously separates states of themovable object data 311 from logic, thereby increasing the ease of implementation and testing. - Generally, any number of
listeners 191,stores 195, andhardware abstractions 197 can be created, used, and/or destroyed within thecache 190, as appropriate. In some embodiments,multiple applications 160 can access thecache 190 at the same time. In some embodiments, access to thecache 190 can be limited to asingle application 160. Although asingle hardware abstraction 197 is shown inFIG. 6 for illustrative purposes only, any number ofhardware abstractions 197 can be used, as desired. Thecache 190 can be implemented, for example, by allocating one or more blocks of memory in the memories 130 (shown inFIG. 1 ) for storingmovable object data 311. The blocks of memory can be de-allocated when thecache 190 no longer needs the blocks. - The one or
more listeners 191 can be registered for listening to and/or providing access to particularmovable object data 311. In some embodiments, alistener 191 can be registered by anapplication 160 that desires access to the particularmovable object data 311. For example, theapplication 160 can invoke an addListener( ) method in a movable object API to create, or instantiate, alistener 191 in themovable object environment 10. The addListener( ) method can include a movable object data key 192 that identifies themovable object data 311 of interest to thelistener 191. The movable object data key 192 can be represented, for example, as a parameter to the addListener( ) method—for example, as a string having a defined format, or as any other suitable type or class. The movable object data key 192 can include different data fields (shown inFIG. 8 ) that identify a particularmovable object component 310 as well as a particular attribute of themovable object component 310. - The
cache 190 can further include astore 195 in themovable object environment 10 for storing values ofmovable object data 311. In some embodiments, thestore 195 can be a software representation of underlying blocks of memory for storing the values of themovable object data 311. The memory can be allocated and de-allocated, as needed. Thestore 195 can be represented using any suitable data structure, such as an array, a list, a hash table, a tree, a queue, and the like. In some embodiments, aseparate store 195 can be created for eachmovable object component 310. In such cases, each value of themovable object data 311 can be referenced according to a corresponding attribute. In other embodiments, asingle store 195 can includemovable object data 311 for multiplemovable object components 310. In such cases, themovable object data 311 can be referenced according to an identifier (for example, a string) corresponding to themovable object component 310, as well as an identifier (for example, a string) corresponding to the attribute of the movable object component. In some embodiments, asingle store 195 can be created in themovable object environment 10 for all movable object data 311 (as shown inFIG. 6 ). - In some embodiments, the
movable object data 311 can be represented as a constant, read-only, or immutable value in thestore 195. When thecache 190 receives a new or updated value of themovable object data 311, a prior read-only representation of themovable object data 311 can be deleted from thestore 195, and a new read-only representation of themovable object data 311 can be created and placed in thestore 195. In some embodiments, a new movable object data key 192 can be created and associated with the new representation of themovable object data 311. The representation of themovable object data 311 as a constant ensures consistency of themovable object data 311 among multiple threads, thereby improving thread safety and improving application performance. The use of immutable representations of themovable object data 311 to handle thread-shared data advantageously avoids the thread locks and can improve performance. Alternatively, and/or additionally,movable object data 311 can be represented in thecache 190 as a mutable or variable value. In some embodiments, mechanisms such as thread locks can be used to avoid data corruption between multiple threads. - As shown in
FIG. 6 , thecache 190 can further include one ormore hardware abstractions 197 in themovable object environment 10. Each of thehardware abstractions 197 can correspond to particularmovable object component 310. In some embodiments, each of thehardware abstractions 197 can be a software object for interacting with correspondingmovable object components 310. For example, a FlightController object can be instantiated in themovable object environment 10 that corresponds to a flight controller 321 (shown inFIG. 2 ). Similarly, a Camera object can be instantiated in themovable object environment 10 that corresponds to an imaging device 325 (shown inFIG. 2 ). Likewise, a Battery object can be instantiated in themovable object environment 10 that corresponds to abattery 327 of a UAV 320 (shown inFIG. 2 ), and so forth. - In some embodiments, the
hardware abstractions 197 can be organized using inheritance. For example, a base Camera class can be defined as having attributes that are common to a variety ofimaging devices 325, for example, an ISO setting. The Camera base class can include, for example, getISO( ) and set ISO( ) methods, as desired. A subclass HighPerformanceCamera can be defined as inheriting from the Camera base class, and include accessors specific to a highperformance imaging device 325, as appropriate. - In some embodiments, one or more of the
hardware abstractions 197 can be configured to correspond tomovable object components 310 having one or more subcomponents. For example, a hardware abstraction can be instantiated to correspond to aflight controller 321 having aGPS 322 and/or anIMU 323.Movable object data 311 of the subcomponents can be accessed using a movable object data key 192, as described above. - In some embodiments, a
hardware abstraction 197 can be created for communication with or access to a specific correspondingmovable object component 310. For example, anapplication 160 can create ahardware abstraction 197 using a suitableapplication call process 165. In some embodiments, ahardware abstraction 197 can be removed or destroyed when communication with or access to the specific correspondingmovable object component 310 is no longer desired. During runtime, the number ofhardware abstractions 197 that are instantiated at any one time can advantageously be kept low so as to reduce memory usage and improve efficiency. In some embodiments, the number ofhardware abstractions 197 can be kept below a predetermined threshold (for example, a maximum of 20 hardware abstractions 197). - Turning now to
FIG. 7 , anexemplary hardware abstraction 197 is shown as having akey router 198 and one or more accessors 199. Thekey router 198 can include a dictionary of movableobject data keys 192, as well values of themovable object data 311 associated with each of the movableobject data keys 192. A dictionary for thekey router 198 can be represented using any suitable data structure, such as an array, a list, a hash table, a tree, a queue, and the like. Thekey router 198 can thereby define the available movableobject data keys 192 representing various attributes of themovable object component 310. Furthermore, thekey router 198 can define various characteristics of each attribute, such as which type of access (for example, read access, write access, or any combination thereof) is available for each of the attributes. Such characteristics can, for example, be associated with the movable object data key 192 corresponding to each attribute of themovable object component 310. Theaccessors 199 of thehardware abstraction 197 can include, for example, a get( ) method to retrieve the value of a particular attribute, or a set( ) method to set a value of a particular attribute. The specific details of theaccessors 199 can be implemented by a developer, as appropriate. In some embodiments, thekey router 198 can be implemented as an immutable dictionary, such that the attributes of themovable object components 310 are unchanged. - Turning now to
FIG. 8 , exemplary movableobject data keys 192 are shown in a string format, wherein each string takes on a predetermined key pattern. Shown on the left side ofFIG. 8 are exemplary key patterns, which can be as “component/key,” “component/index/key,” “component/index/subcomponent/key,” or “component/index/subcomponent/subcomponent index/key.” Here, “component” can represent a movable object component 310 (for example, “camera” to represent an imaging device 325) to be accessed using the movable object data key 192. “Index” can represent a numeric value corresponding to a particularmovable object component 310 of a specific type (for example, “camera/1” to represent the first of two imaging devices 325). “Subcomponent” can represent a subcomponent of the movable object component 310 (for example, aGPS 322 of a flight controller 321). “Subcomponent index” can represent a numeric value of corresponding to a particular subcomponent of the movable object component 310 (for example, “gps/1” to refer to the first of twoGPS 322 devices of a flight controller 321). “Key” can represent a particular attribute, parameter, or other data associated with themovable object component 310 or subcomponent. For example, a movable object data key 192 for a satellites number attribute for asecond GPS 322 device of afirst flight controller 321 can be represented as the string “flightcontroller/1/gps/2/satellitesNumber,” as shown inFIG. 8 . - In some embodiments, string representation of a movable object data key 192 can be converted by the
cache 190 into any other data type for internal use and communication within thecache 190. For example, thecache 190 can use a KeyPath object that includes methods for parsing the string representation of the movable object data key 192, as well as includes attributes for each portion string. Using a KeyPath or like object advantageously avoids re-parsing the string representation of the movable object data key 192. - Turning now to
FIG. 9 , an exemplary process for registering alistener 191 is shown in accordance with the examples provided above. Here, anapplication 160 is interested in values of a “satellites number” attribute of aGPS 322 device of aflight controller 321 of a UAV 320 (not shown). In particular, theapplication 160 has a need to be notified of changes in value of the satellites number field. To do so, theapplication 160 can access amovable object interface 180 having an addListener( ) method for registering alistener 191. In particular, theapplication 160 can invoke the addListener( ) method in anapplication call process 165 that specifies a movable object data key 192 (for example, a string key “flightcontroller/1/gps/satellitesNumber”) as a parameter. - Upon invoking the addListener( ) method, a
new listener 191 is created corresponding to the movable object data key 192 “flightcontroller/1/gps/satellitesNumber.” Thereafter, thelistener 191 that is created can be registered with the appropriate movable object component 310 (here, theflight controller 321 and/or theGPS 322 subcomponent). Thelistener 191 can be associated with ahardware abstraction 197 that corresponds to themovable object component 310. When a value of the attribute corresponding to the listen 191 is changed, thelistener 191 can be used to notify theapplication 160 of the change and/or the new value via the appropriatemovable object data 311. - Turning now to
FIG. 10 , anexemplary method 1000 for managing communication with amovable object 300 is shown. At 1010, alistener 191 is registered, via amovable object manager 180, for listening tomovable object data 311 in amovable object environment 10. In some embodiments, thelistener 191 can be registered in a movable object software environment 50 (shown inFIG. 3 ). For example, thelistener 191 can be registered through a movable object application programming interface (API), as described above with reference toFIG. 3 . The movable object API can include, for example, one or more functions, methods, procedures, parameters, constants, variables, objects, modules, datatypes, exceptions, etc. that anapplication 160 can utilize to interact with and control themovable object 300. In some embodiments, thelistener 191 can be registered through a movable object API that is part of a movable object software development kit (SDK). The movable object SDK can include tools, libraries, documentation, and sample code that can help a software developer use the SDK for developing amovable object application 160. - In some embodiments, the
listener 191 can be registered with amovable object component 310 corresponding to the specificmovable object data 311. For example, as described above with respect toFIG. 9 , alistener 191 for aGPS 322 device of aflight controller 321 can be registered with theflight controller 321 and/or theGPS 322. - In some embodiments, the
listener 191 can be registered with ahardware abstraction 197 corresponding to themovable object component 310. For example, as described above with respect toFIG. 9 , ahardware abstraction 197 for theflight controller 321 and/or theGPS 322 can be instantiated during runtime of theapplication 160. Thehardware abstraction 197 can have akey router 198 that is initialized with relevant attributes of themovable object component 310 and/or various subcomponents thereof. Thelistener 191 can be registered using an appropriate movable object data key 192 that identifies themovable object component 310. For example, the movable object data key 192 can be a string having a substring that identifies the movable object component 310 (for example, “flightcontroller” to identify the flight controller 321). The movable object data key 192 can further identify an attribute of themovable object component 310 or subcomponent thereof (for example, “satellitesNumber” to identify a satellites number attribute of the GPS 322). - In some embodiments, the
listener 191 can be de-registered whenmovable object data 191 from themovable object component 310 is no longer desired. For example, theapplication 160 can be configured to signal to themovable object manager 180 that theapplication 160 is no longer interacting with themovable object 300. In some embodiments, theapplication 160 can send a stop signal to themovable object manager 180. In some embodiments, themovable object manager 180 loses connection with theapplication 160, whereupon thelistener 191 can be automatically de-registered. - At 1020,
movable object data 311 which is received from themovable object 300 can be stored in acache 190 associated with themovable object manager 180. Themovable object data 311 can be received using thelistener 191 via a request for themovable object data 311 that uses the movable object data key 192. In some embodiments, thelistener 191 can determine whether themovable object data 311 is available in thecache 190, and/or whether themovable object data 311 is updated in thecache 190. Upon determining that themovable object data 311 is not available or not updated in thecache 190, themovable object 300 can be queried for an updated value of themovable object data 311. For example, theapplication 160 can query themovable object 300 for the updated value of themovable object data 311. Themovable object 300 can be queried for the value of themovable object data 311 using a request, such as a get( ) method in themovable object interface 170. In some embodiments, thecache 190 can be configured to automatically query themovable object 300 for the updated value of themovable object data 311 upon determining that themovable object data 311 is not available or not updated. The request can be performed via thehardware abstraction 197 that corresponds with themovable object component 310 having the desiredmovable object data 311. For example, the request can be performed using aprotocol call process 255 that is transmitted to themovable object 300 using thehardware abstraction 197. After the querying, an updated value of themovable object data 311 can be received from the movable object (for example, using aprotocol call process 255 transmitted from the movable object 300). - At 1030,
movable object data 311 that is received at 1020 can be provided to anapplication 160 corresponding to thelistener 191. In some embodiments, theapplication 160 can be anapplication 160 for interacting with and/or controlling themovable object 300. In some embodiments, thelistener 191 can be used to detect a change in a value of themovable object data 311. The change can be detected, for example, by storing a prior value of themovable object data 311 in thecache 190. Thelistener 191 can periodically receive an updated value of themovable object data 311 and compare the prior value to the updated value to detect a change in the value. Upon detecting the change in value, the listener can notify theapplication 160 of the change in themovable object data 311. Furthermore, the changed value of themovable object data 311 can be automatically provided to theapplication 160. In some embodiments, themovable object data 311 is not automatically provided to theapplication 160, and the application can optionally request themovable object data 311, if desired. - Turning now to
FIG. 11 , an exemplary process for requestingmovable object data 311 is shown in accordance with the examples provided above. Here, theapplication 160 is to request a “satellites number” attribute of aGPS 322 device of aflight controller 321 of a UAV 320 (not shown). A request for themovable object data 311 can be sent to the movable object manager 180 (shown inFIG. 3 ) using an interface 170 (shown inFIG. 3 ). The request can be sent using anapplication call process 165 and an appropriate movable object data key 192. For example, a get(“flightcontroller/1/gps/satellitesNumber”) method can be invoked by theapplication 160. - Upon invocation of the request, whether the
movable object data 311 is available and/or updated in the cache 190 (for example, within astore 195 of the cache 190) can be determined. Where themovable object data 311 is available and/or updated, the movable object can advantageously be provided back to theapplication 160 from thecache 190 without querying themovable object 300, thereby reducing the number of protocol call processes 255 sent over potentially limited bandwidth. Where themovable object data 311 is unavailable and/or not updated, the movable object can be retrieved from themovable object 300 using thehardware abstraction 197 corresponding to theflight controller 321 of interest. - Turning now to
FIG. 12 , an exemplary top-level block diagram for amethod 1200 for managing communication with amovable object 300 is shown. At 1210, a request formovable object data 311 is received at amovable object manager 180 from a requester in amovable object environment 10. The requester can be, for example, an application 160 (shown inFIG. 3 ) or any other process running in themovable object environment 10. As described above with respect toFIG. 11 , the request can be made as anapplication call process 165 using an appropriate movable object data key 192. - At 1220, whether
movable object data 311 corresponding to the request is available in acache 190 associated with themovable object manager 180 can be determined. For example, astore 195 of thecache 190 can be evaluated for whethermovable object data 311 corresponding to the movable object data key 192 exists. In some embodiments, whethermovable object data 311 is updated in thecache 190 can be determined. - Whether the
movable object data 311 is updated can be determined according to a set of predetermined rules. In some embodiments, whether themovable object data 311 is updated can be determined according to a data type of themovable object data 311. For example, certain types ofmovable object data 311 are expected to remain fixed for the duration of use of the movable object 300 (for example,movable object data 311 reflecting mechanical configurations of themovable object 300, such as the number of batteries, is fixed in the duration of a mission of the movable object). In another example, some types ofmovable object data 311 can be regularly transmitted (or “pushed”) from themovable object 300 to themovable object manager 180, and stored in thecache 190. Some types of “push” data can be assumed to be updated, without need for further determination. - Alternatively, and/or additionally, whether the
movable object data 311 is updated can be determined according to a time stamp t of themovable object data 311. The time stamp t can be a time at which themovable object data 311 was transmitted by themovable object 300, and/or a time at which themovable object data 311 was received by themovable object manager 180. The time stamp t of themovable object data 311 can be stored in thecache 190, as desired. In some embodiments, the time stamp t of themovable object data 311 that is stored in thecache 190 can be compared with the current time to of the request to find a lapsed time Δt since themovable object data 311 was last updated. The lapsed time Δt can be compared to a predetermined threshold T to determine whether themovable object data 311 is updated or not. The predetermined threshold T can be specific to the type ofmovable object data 311. For example, certain types of movable object data 311 (for example, the number of satellites of a GPS 322) need not be regularly updated, and the predetermined threshold T can be set to a large value accordingly. In some embodiments, other types of movable object data 311 (for example, the charge status of a battery 327) are regularly updated, and the predetermined threshold T can be set to a smaller value accordingly. - Alternatively, and/or additionally, whether the
movable object data 311 is updated can be determined according to a prior value of themovable object data 311. For example, a position of themovable object 300 can be determined to be updated where the movable object is positioned at a home base, or along an expected trajectory. If the position of themovable object 300 is unexpected (for example, off trajectory or in a restricted area), the position can be determined to be not updated. - At 1230, the
movable object data 311 in thecache 190 can be provided to the requester. In some embodiments, where themovable object data 311 in thecache 190 is updated, themovable object data 311 can be directly provided to the requester without querying themovable object 300, advantageously saving time and bandwidth. - Turning now to
FIG. 13 , further details of various embodiments of themethod 1200 are shown. At 1221, whether themovable object data 311 is updated in thecache 190 can be determined, as described above with reference toFIG. 12 . If themovable object data 311 is not updated, then themovable object 300 is queried for an update value at 1222. At 1223, the updated value of themovable object data 311 is subsequently received from themovable object 300. At 1224, the updated value themovable object data 311 is stored in thecache 190. Alternatively, if themovable object data 311 is updated, then themovable object data 311 is retrieved from thecache 190 at 1225. Finally, at 1231, themovable object data 311 in thecache 190 is provided to the requester of themovable object data 311. - Turning now to
FIG. 14 , exemplary diagrams are shown illustrating various embodiments for receivingmovable object data 311 from amovable object 300. On the left panel ofFIG. 14 , a “push” mechanism is shown wherebymovable object data 311 is automatically transmitted by themovable object 300 to themovable object manager 180. Certain types of data, as desired, can be designated “push” data and automatically transmitted to themovable object manager 180. In some embodiments, themovable object data 311 can be periodically transmitted from themovable object 300, received by themovable object manager 180, and stored in acache 190. The frequency of transmission of such “push” data can vary, as desired. In contrast, on the right panel ofFIG. 14 , a “push-like” mechanism is shown that can be used when no “push” data is available. In such cases, themovable object manager 180 can send protocol call processes 255, such as get( ) methods in amovable object interface 170, to retrieve themovable object data 311 from themovable object 300. In some embodiments, themovable object data 311 can be periodically retrieved using automatic get( ) methods, and stored in thecache 190. In some embodiments, both the “push” and “push-like” mechanisms of retrieving themovable object data 311 can be used for different types ofmovable object data 311 for maximum flexibility and efficiency. - The disclosed embodiments are susceptible to various modifications and alternative forms, and specific examples thereof have been shown by way of example in the drawings and are herein described in detail. It should be understood, however, that the disclosed embodiments are not to be limited to the particular forms or methods disclosed, but to the contrary, the disclosed embodiments are to cover all modifications, equivalents, and alternatives.
Claims (20)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US16/930,090 US20200351615A1 (en) | 2016-08-10 | 2020-07-15 | System and method for managing movable object communications |
Applications Claiming Priority (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
PCT/CN2016/094327 WO2018027654A1 (en) | 2016-08-10 | 2016-08-10 | System and method for managing movable object communications |
US16/268,055 US10728707B2 (en) | 2016-08-10 | 2019-02-05 | System and method for managing movable object communications |
US16/930,090 US20200351615A1 (en) | 2016-08-10 | 2020-07-15 | System and method for managing movable object communications |
Related Parent Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US16/268,055 Continuation US10728707B2 (en) | 2016-08-10 | 2019-02-05 | System and method for managing movable object communications |
Publications (1)
Publication Number | Publication Date |
---|---|
US20200351615A1 true US20200351615A1 (en) | 2020-11-05 |
Family
ID=61161189
Family Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US16/268,055 Active US10728707B2 (en) | 2016-08-10 | 2019-02-05 | System and method for managing movable object communications |
US16/930,090 Abandoned US20200351615A1 (en) | 2016-08-10 | 2020-07-15 | System and method for managing movable object communications |
Family Applications Before (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US16/268,055 Active US10728707B2 (en) | 2016-08-10 | 2019-02-05 | System and method for managing movable object communications |
Country Status (3)
Country | Link |
---|---|
US (2) | US10728707B2 (en) |
CN (1) | CN109564418A (en) |
WO (1) | WO2018027654A1 (en) |
Families Citing this family (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN112400300A (en) * | 2018-08-01 | 2021-02-23 | 西门子股份公司 | Interconnection device, communication method, and system including robot |
WO2021035612A1 (en) * | 2019-08-29 | 2021-03-04 | 深圳市大疆创新科技有限公司 | Management method, device, and system for unmanned aerial vehicle, unmanned aerial vehicle, and storage medium |
CN115167316B (en) * | 2022-08-04 | 2024-05-14 | 中国核动力研究设计院 | Cooperative processing method, system and storage medium of nuclear power plant DCS platform |
Family Cites Families (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2005123502A2 (en) * | 2003-12-12 | 2005-12-29 | Advanced Ceramics Research, Inc. | Unmanned vehicle |
US20100259614A1 (en) * | 2009-04-14 | 2010-10-14 | Honeywell International Inc. | Delay Compensated Feature Target System |
CN102520729B (en) * | 2011-12-05 | 2013-05-22 | 清华大学 | Information transmission method of unmanned aerial vehicle cluster in visual navigation |
CN102768518B (en) * | 2012-07-11 | 2014-05-21 | 清华大学 | Multiple-unmanned plane platform cooperative control system |
WO2015082596A1 (en) * | 2013-12-06 | 2015-06-11 | Bae Systems Plc | Imaging method and apparatus |
US9843647B2 (en) * | 2014-02-25 | 2017-12-12 | Here Global B.V. | Method and apparatus for providing selection and prioritization of sensor data |
KR101453372B1 (en) * | 2014-04-15 | 2014-10-22 | 주식회사 스마티랩 | SYSTEM FOR MEDIATE HETEROGENEOUS DATA EXCHANGE OF IoT DEVICES IN INTERNET OF THINGS |
CN105116907A (en) * | 2015-07-20 | 2015-12-02 | 清华大学 | Method for designing data transmission and control system of miniature unmanned aerial vehicle |
-
2016
- 2016-08-10 WO PCT/CN2016/094327 patent/WO2018027654A1/en active Application Filing
- 2016-08-10 CN CN201680088144.4A patent/CN109564418A/en active Pending
-
2019
- 2019-02-05 US US16/268,055 patent/US10728707B2/en active Active
-
2020
- 2020-07-15 US US16/930,090 patent/US20200351615A1/en not_active Abandoned
Also Published As
Publication number | Publication date |
---|---|
US20190182623A1 (en) | 2019-06-13 |
US10728707B2 (en) | 2020-07-28 |
WO2018027654A1 (en) | 2018-02-15 |
CN109564418A (en) | 2019-04-02 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US20200351615A1 (en) | System and method for managing movable object communications | |
US10332406B2 (en) | Dynamic geo-fence for drone | |
US11729261B2 (en) | Systems and methods for unmanned aerial system communication | |
US20220248296A1 (en) | Managing session continuity for edge services in multi-access environments | |
US10809711B2 (en) | Electronic device and method for controlling the same | |
WO2018095158A1 (en) | Flight control method, apparatus and system for use in aircraft | |
US10149276B2 (en) | Aerial robotic vehicle antenna switching | |
US20210294327A1 (en) | System and method for movable object control | |
US20180019802A1 (en) | Managing Network Communication of a Drone | |
JP2023057120A (en) | Method for displaying information concerning control of flying body | |
US10904755B2 (en) | Data distribution from a movable object | |
US11620913B2 (en) | Movable object application framework | |
US20210294720A1 (en) | System and method for movable object tracking and analysis | |
JP6343366B1 (en) | Information transmission method at the time of disaster | |
US11659516B2 (en) | Network-based user equipment receive/transmit capability exchange for positioning | |
US20220086791A1 (en) | User equipment receive/transmit capability exchange for positioning | |
WO2022244402A1 (en) | Information processing device and information processing method | |
US20240201694A1 (en) | Mission critical data delivery booster drones for satellite data centers | |
JP2018186507A (en) | Information transfer method at disaster | |
WO2023028762A1 (en) | User-equipment-based mobility optimization | |
KR20240095645A (en) | Apparatus and Method for real-time sharing drone mission information including video streaming based on a Remote ID standard |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: SZ DJI TECHNOLOGY CO., LTD., CHINA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:ZHANG, ZHIYUAN;ZHAN, GUOHAO;AI, CHUYUE;AND OTHERS;SIGNING DATES FROM 20190128 TO 20190130;REEL/FRAME:053220/0907 |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NON FINAL ACTION MAILED |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NOTICE OF ALLOWANCE MAILED -- APPLICATION RECEIVED IN OFFICE OF PUBLICATIONS |
|
STCB | Information on status: application discontinuation |
Free format text: ABANDONED -- FAILURE TO PAY ISSUE FEE |