US20170212529A1 - Multi-sensor fusion for robust autonomous flight in indoor and outdoor environments with a rotorcraft micro-aerial vehicle (mav) - Google Patents

Multi-sensor fusion for robust autonomous flight in indoor and outdoor environments with a rotorcraft micro-aerial vehicle (mav) Download PDF

Info

Publication number
US20170212529A1
US20170212529A1 US15/165,846 US201615165846A US2017212529A1 US 20170212529 A1 US20170212529 A1 US 20170212529A1 US 201615165846 A US201615165846 A US 201615165846A US 2017212529 A1 US2017212529 A1 US 2017212529A1
Authority
US
United States
Prior art keywords
measurements
state
sensors
vehicle
gps
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Abandoned
Application number
US15/165,846
Other languages
English (en)
Inventor
R. Vijay Kumar
Shaojie Shen
Nathan Michael
Kartik Mohta
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
University of Pennsylvania Penn
Original Assignee
University of Pennsylvania Penn
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by University of Pennsylvania Penn filed Critical University of Pennsylvania Penn
Priority to US15/165,846 priority Critical patent/US20170212529A1/en
Publication of US20170212529A1 publication Critical patent/US20170212529A1/en
Priority to US15/684,700 priority patent/US10732647B2/en
Abandoned legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/10—Simultaneous control of position or course in three dimensions
    • G05D1/101—Simultaneous control of position or course in three dimensions specially adapted for aircraft
    • G05D1/102—Simultaneous control of position or course in three dimensions specially adapted for aircraft specially adapted for vertical take-off of aircraft
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B64—AIRCRAFT; AVIATION; COSMONAUTICS
    • B64C—AEROPLANES; HELICOPTERS
    • B64C39/00—Aircraft not otherwise provided for
    • B64C39/02—Aircraft not otherwise provided for characterised by special use
    • B64C39/024—Aircraft not otherwise provided for characterised by special use of the remote controlled vehicle type, i.e. RPV
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B64—AIRCRAFT; AVIATION; COSMONAUTICS
    • B64C—AEROPLANES; HELICOPTERS
    • B64C39/00—Aircraft not otherwise provided for
    • B64C39/02—Aircraft not otherwise provided for characterised by special use
    • B64C39/028—Micro-sized aircraft
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B64—AIRCRAFT; AVIATION; COSMONAUTICS
    • B64U—UNMANNED AERIAL VEHICLES [UAV]; EQUIPMENT THEREFOR
    • B64U10/00—Type of UAV
    • B64U10/80—UAVs characterised by their small size, e.g. micro air vehicles [MAV]
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B64—AIRCRAFT; AVIATION; COSMONAUTICS
    • B64U—UNMANNED AERIAL VEHICLES [UAV]; EQUIPMENT THEREFOR
    • B64U20/00—Constructional aspects of UAVs
    • B64U20/40—Modular UAVs
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
    • G01C21/10—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration
    • G01C21/12—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning
    • G01C21/16—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning by integrating acceleration or speed, i.e. inertial navigation
    • G01C21/165—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning by integrating acceleration or speed, i.e. inertial navigation combined with non-inertial navigation instruments
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
    • G01C21/10—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration
    • G01C21/12—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning
    • G01C21/16—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning by integrating acceleration or speed, i.e. inertial navigation
    • G01C21/165—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning by integrating acceleration or speed, i.e. inertial navigation combined with non-inertial navigation instruments
    • G01C21/1654—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning by integrating acceleration or speed, i.e. inertial navigation combined with non-inertial navigation instruments with electromagnetic compass
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
    • G01C21/10—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration
    • G01C21/12—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning
    • G01C21/16—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning by integrating acceleration or speed, i.e. inertial navigation
    • G01C21/165—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning by integrating acceleration or speed, i.e. inertial navigation combined with non-inertial navigation instruments
    • G01C21/1656—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning by integrating acceleration or speed, i.e. inertial navigation combined with non-inertial navigation instruments with passive imaging devices, e.g. cameras
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
    • G01C21/20—Instruments for performing navigational calculations
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
    • G01C21/20—Instruments for performing navigational calculations
    • G01C21/206—Instruments for performing navigational calculations specially adapted for indoor navigation
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S19/00—Satellite radio beacon positioning systems; Determining position, velocity or attitude using signals transmitted by such systems
    • G01S19/38—Determining a navigation solution using signals transmitted by a satellite radio beacon positioning system
    • G01S19/39—Determining a navigation solution using signals transmitted by a satellite radio beacon positioning system the satellite radio beacon positioning system transmitting time-stamped messages, e.g. GPS [Global Positioning System], GLONASS [Global Orbiting Navigation Satellite System] or GALILEO
    • G01S19/42—Determining position
    • G01S19/48—Determining position by combining or switching between position solutions derived from the satellite radio beacon positioning system and position solutions derived from a further system
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S19/00—Satellite radio beacon positioning systems; Determining position, velocity or attitude using signals transmitted by such systems
    • G01S19/38—Determining a navigation solution using signals transmitted by a satellite radio beacon positioning system
    • G01S19/39—Determining a navigation solution using signals transmitted by a satellite radio beacon positioning system the satellite radio beacon positioning system transmitting time-stamped messages, e.g. GPS [Global Positioning System], GLONASS [Global Orbiting Navigation Satellite System] or GALILEO
    • G01S19/42—Determining position
    • G01S19/48—Determining position by combining or switching between position solutions derived from the satellite radio beacon positioning system and position solutions derived from a further system
    • G01S19/485—Determining position by combining or switching between position solutions derived from the satellite radio beacon positioning system and position solutions derived from a further system whereby the further system is an optical system or imaging system
    • B64C2201/027—
    • B64C2201/108—
    • B64C2201/141—
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B64—AIRCRAFT; AVIATION; COSMONAUTICS
    • B64U—UNMANNED AERIAL VEHICLES [UAV]; EQUIPMENT THEREFOR
    • B64U10/00—Type of UAV
    • B64U10/10—Rotorcrafts
    • B64U10/13—Flying platforms
    • B64U10/14—Flying platforms with four distinct rotor axes, e.g. quadcopters
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B64—AIRCRAFT; AVIATION; COSMONAUTICS
    • B64U—UNMANNED AERIAL VEHICLES [UAV]; EQUIPMENT THEREFOR
    • B64U2101/00—UAVs specially adapted for particular uses or applications
    • B64U2101/55—UAVs specially adapted for particular uses or applications for life-saving or rescue operations; for medical use
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B64—AIRCRAFT; AVIATION; COSMONAUTICS
    • B64U—UNMANNED AERIAL VEHICLES [UAV]; EQUIPMENT THEREFOR
    • B64U2101/00—UAVs specially adapted for particular uses or applications
    • B64U2101/70—UAVs specially adapted for particular uses or applications for use inside enclosed spaces, e.g. in buildings or in vehicles
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B64—AIRCRAFT; AVIATION; COSMONAUTICS
    • B64U—UNMANNED AERIAL VEHICLES [UAV]; EQUIPMENT THEREFOR
    • B64U2201/00—UAVs characterised by their flight controls
    • B64U2201/10—UAVs characterised by their flight controls autonomous, i.e. by navigating independently from ground or air stations, e.g. by using inertial navigation systems [INS]
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B64—AIRCRAFT; AVIATION; COSMONAUTICS
    • B64U—UNMANNED AERIAL VEHICLES [UAV]; EQUIPMENT THEREFOR
    • B64U2201/00—UAVs characterised by their flight controls
    • B64U2201/20—Remote controls
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B64—AIRCRAFT; AVIATION; COSMONAUTICS
    • B64U—UNMANNED AERIAL VEHICLES [UAV]; EQUIPMENT THEREFOR
    • B64U30/00—Means for producing lift; Empennages; Arrangements thereof
    • B64U30/20—Rotors; Rotor supports

Definitions

  • the subject matter described herein relates to controlling autonomous flight in a micro-aerial vehicle. More particularly, the subject matter described herein relates to multi-sensor fusion for robust autonomous flight in indoor and outdoor environments with a rotorcraft micro-aerial vehicle (MAV).
  • MAV micro-aerial vehicle
  • Micro-aerial vehicles such as rotorcraft micro-aerial vehicles, are capable of flying autonomously. Accurate autonomous flight can be achieved provided that there is sufficient sensor data available to provide control input for the autonomous flight. For example, in some outdoor environments where a global positioning system (GPS) is available, autonomous flight can be achieved based on GPS signals. However, in environments where GPS is not available, such as indoor environments and even outdoor urban environments, autonomous flight based on GPS alone is not possible. In some indoor environments, magnetometer output may not be available or reliable due to magnetic interference caused by structures. Thus, reliance on a single modality of sensor to control flight of a rotorcraft MAV may not be desirable.
  • GPS global positioning system
  • Another goal of controlling autonomous flight of a rotorcraft MAV is smooth transition between states when a sensor modality that was not previously available becomes available. For example, when a rotorcraft MAV is flying indoors where GPS is not available and then transitions to an outdoor environment where GPS suddenly becomes available, the rotorcraft may determine that it is far off course and may attempt to correct the error by immediately moving to be on course. It is desirable that such transitions be smooth, rather than having the rotorcraft immediately make large changes in velocity and trajectory to get back on course.
  • sensor data are available to control autonomous flight in rotorcraft micro-aerial vehicles.
  • onboard cameras, laser scanners, GPS transceivers, and accelerometers can provide multiple inputs that are suitable as control inputs for controlling flight.
  • relying on any one of these sensors fails when the assumptions associated with the sensor fails. Because each type of sensor produces a unique kind of output with a unique level of uncertainty in its measurement, there exists a need for improved methods, systems, and computer readable media for multi-sensor fusion for robust autonomous flight in indoor and outdoor environments with a rotorcraft MAV.
  • the subject matter described herein includes a modular and extensible approach to integrate noisy measurements from multiple heterogeneous sensors that yield either absolute or relative observations at different and varying time intervals, and to provide smooth and globally consistent estimates of position in real time for autonomous flight.
  • IMU inertial measurement unit
  • laser scanner laser scanner
  • stereo cameras stereo cameras
  • pressure altimeter pressure altimeter
  • magnetometer magnetometer
  • GPS receiver GPS receiver
  • the subject matter described herein may be implemented in hardware, software, firmware, or any combination thereof.
  • the terms “function”, “node” or “module” as used herein refer to hardware, which may also include software and/or firmware components, for implementing the feature being described.
  • the subject matter described herein may be implemented using a computer readable medium having stored thereon computer executable instructions that when executed by the processor of a computer control the computer to perform steps.
  • Exemplary computer readable media suitable for implementing the subject matter described herein include non-transitory computer-readable media, such as disk memory devices, chip memory devices, programmable logic devices, and application specific integrated circuits.
  • a computer readable medium that implements the subject matter described herein may be located on a single device or computing platform or may be distributed across multiple devices or computing platforms.
  • FIG. 1 depicts a 1.9 kg MAV platform equipped with an IMU, laser scanner, stereo cameras, pressure altimeter, magnetometer, and GPS receiver. All the computation is performed onboard on an Intel NUC computer with 3 rd generation i3 processor;
  • FIG. 2 depicts delayed, out-of-order measurement with a priority queue. While z 4 arrives before z 2 , z 2 is first applied to the filter. z 4 is temporary stored in the queue. z 1 is discarded since it is older than t d from the current state. The covariance is only propagated up to the time where the most recent measurement is applied to the filter. The state is propagated until the most recent IMU input;
  • Pose graph SLAM produces a globally consistent graph ( FIG. 3B );
  • FIGS. 4A and 4B illustrate the alternative GPS fusion, the discrepancy between transformed GPS measurement z 5 and the non-optimized state s 5 is minimized. Fusion of such indirect GPS measurement will lead to a smooth state estimate (dashed line between s 6 and s 5 );
  • FIGS. 5A and 5B depict that the MAV maneuvers aggressively with a maximum speed of 3.5 m/s ( FIG. 5B ).
  • the horizontal position also compares well with the ground truth with slight drift ( FIG. 5A );
  • FIGS. 6A-6H depict images from the onboard camera ( FIGS. 6A-6D ) and an external camera ( FIGS. 6E-6H ). Note the vast variety of environments, including open space, trees, complex building structures, and indoor environments. We highlight the position of the MAV with a circle. Videos of the experiments are available in the video attachment and at http://mrsl.grasp.upenn.edu/shaojie/ICRA2014.mp4;
  • FIG. 8 illustrates sensor availability over time. Note that failures occurred to all sensors. This shows that multi-sensor fusion is a must for this kind of indoor-outdoor missions;
  • FIG. 9 illustrates covariance changes as the vehicle flies through a dense building area (between 200 s-300 s, top of FIG. 7 ,).
  • the GPS comes in and out due to building shadowing.
  • the covariance of x, y, and yaw increases as GPS fails and decreases as GPS resumes. Note that the body frame velocity are observable regardless of GPS measurements, and thus its covariance remains small
  • the spike in the velocity covariance is due to the vehicle directly facing the sun.
  • the X-Y covariance is calculated from the Frobenius norm of the covariance submatrix;
  • FIG. 10 depicts vehicle trajectory overlaid on a satellite map.
  • the vehicle operates in a tree-lined campus environment, where there is high risk of GPS failure during operation;
  • FIGS. 11A and 11B depict onboard ( FIG. 11A ) and external ( FIG. 11B ) camera images as the MAV autonomously flies through a tree-lined campus environment. Note the nontrivial light condition;
  • FIG. 12 is a block diagram of a rotorcraft MAV for performing multi-sensor fusion according to an embodiment of the subject matter described herein;
  • FIG. 13 is a flow chart illustrating an exemplary process for multi-sensor fusion controlling autonomous of a rotorcraft MAV according to an embodiment of the subject matter described herein;
  • FIG. 14 illustrates an experimental platform with limited onboard computation (Intel Atom 1.6 GHz processor) and sensing (two cameras with fisheye lenses and an off-the-shelf inexpensive IMU).
  • the platform mass is 740 g;
  • FIG. 15 illustrates a system architecture with update rates and information flow between modules marked
  • FIG. 16 illustrates the performance of body frame velocity estimation during autonomous tracking of the trajectory presented in Sect. VIII-A;
  • FIGS. 17A-17D illustrate the effects on feature tracking performance due to fast translation ( FIGS. 17A-17B ) and fast rotation ( FIGS. 17C-17D ).
  • the number of tracked features significantly decrease after rotation;
  • FIGS. 18A and 18B illustrate that a simulated quadrotor tracks a smooth trajectory generated from a sequence of waypoints. Trajectory regeneration takes place after a change of waypoints at 20 s;
  • FIG. 19 illustrates a finite state machine-based approach to MAV navigation that enables the operator to interact with the vehicle during experiments
  • FIGS. 20A and 20B illustrate desired, estimated and actual trajectories when the robot is commanded to follow a smooth trajectory generated from a rectangle pattern
  • FIG. 21A is a snapshot image of the indoor environment
  • FIG. 21B is the image captured by the onboard camera. Note that the floor is featureless, which can pose a challenge to approaches that rely on downward facing cameras;
  • FIGS. 22A-22C illustrate maps and estimated positions during the indoor navigation experiment. Note the nontrivial discontinuities in the pose estimates obtained via SLAM after the loop closure ( FIG. 22C );
  • FIG. 23 illustrates a final 3D map and trajectory of the outdoor experiment after closing the loop.
  • FIGS. 24A-24D contain images of autonomous navigation in a complex outdoor environment. Images from both the external video camera and the onboard camera are shown. Videos of the experiments are available at http://mrsl.grasp.upenn.edu/shaojie/IROS2013.mov.
  • Rotorcraft micro-aerial vehicles are ideal platforms for surveillance and search and rescue in confined indoor and outdoor environments due to their small size, superior mobility, and hover capability. In such missions, it is essential that the MAV is capable of autonomous flight to minimize operator workload. Robust state estimation is critical to autonomous flight especially because of the inherently fast dynamics of MAVs. Due to cost and payload constraints, most MAVs are equipped with low cost proprioceptive sensors (e.g. MEMS IMUs) that are incapable for long term state estimation. As such, exteroceptive sensors, such as GPS, cameras, and laser scanners, are usually fused with proprioceptive sensors to improve estimation accuracy. Besides the well-developed GPS-based navigation technology [1, 2].
  • MEMS IMUs low cost proprioceptive sensors
  • the main goal of this work is to develop a modular and extensible approach to integrate noisy measurements from multiple heterogeneous sensors that yield either absolute or relative observations at different and varying time intervals, and to provide smooth and globally consistent estimates of position in real time for autonomous flight.
  • the first key contribution that is central to our work, is a principled approach, building on [11], to fusing relative measurements by augmenting the vehicle state with copies of previous states to create an augmented state vector for which consistent estimates are obtained and maintained using a filtering framework.
  • a second significant contribution is our Unscented Kalman Filter (UKF) formulation in which the propagation and update steps circumvent the difficulties that result from the semi-definiteness of the covariance matrix for the augmented state.
  • UPF Unscented Kalman Filter
  • Section III we outline the modeling framework before presenting the key contributions of UKF-based sensor fusion scheme in Section IV. We bring all the ideas together in our description of the experimental platform and the experimental results in Section VI.
  • p w [x w , y w , z w ] T is the 3D position in the world frame
  • ⁇ w [ ⁇ w , ⁇ w , ⁇ w ] T is the yaw, pitch, and roll Euler angles that represent the 3-D orientation of the body in the world frame, from which a matrix Rwb that represent the rotation of a vector from the body frame to the world frame can be obtained.
  • ⁇ dot over (p) ⁇ b is the 3D velocity in the body frame.
  • b a b and b ⁇ b are the bias of the accelerometer and gyroscope, both expressed in the body frame.
  • b z w models the bias of the laser and/or pressure altimeter in the world frame.
  • v t [v a ,v ⁇ ,v b a ,v b a ,v b z ] T
  • x t+1 ⁇ ( x t ,u t ,v t ) (1)
  • u is the measurement of the body frame linear accelerations and angular velocities from the IMU.
  • v t ⁇ N(0,D t ) ⁇ m is the process noise.
  • v a and v ⁇ represent additive noise associated with the gyroscope and the accelerometer.
  • v ba , v b ⁇ , v bz model the Gaussian random walk of the gyroscope, accelerometer and altimeter bias.
  • the function f(•) is a discretized version of the continuous time dynamical equation [6].
  • Exteroceptive sensors are usually used to correct the errors in the state propagation. Following [11], we consider measurements as either being absolute or relative, depending the nature of underlying sensor. We allow arbitrary number of either absolute or relative measurement models.
  • n t+m ⁇ N(0, Q t ) ⁇ p is the measurement noise that can be either additive or not.
  • h a (•) is in general a nonlinear function.
  • An absolute measurement connects the current state with the sensor output. Examples are shown in Sect. V-B.
  • a relative measurement connects the current and the past states with the sensor output, which can be written as:
  • ⁇ is a UKF parameter.
  • ( ⁇ square root over ((n+ ⁇ )P xx ) ⁇ ) i is the i th column of the square root covariance matrix; which is usually computed via Cholesky decomposition.
  • ⁇ are called the sigma points.
  • the mean, covariance of the random vector y, and the cross-covariance between x and y, can be approximated as:
  • a measurement may not affect all components in the state x.
  • a visual odometry only affects the 6-DOF (Degree of Freedom) pose, not the velocity or the bias terms.
  • x i is an arbitrary subset of x.
  • B i binary selection matrix
  • the linear regression matrix A in (9) serves as the linear approximation of the nonlinear function (4). It is similar to the Jacobian in the EKF formulation. As such, the propagation and update steps in UKF can be performed in a similar fashion as EKF.
  • x ⁇ t ⁇ t [ x ⁇ t ⁇ t x ⁇ I t ⁇ t ]
  • P ⁇ t ⁇ t [ P t ⁇ t xx P t ⁇ t xx i P t ⁇ t x I ⁇ x P t ⁇ t x I ⁇ x I ] .
  • t+m ⁇ hacek over (x) ⁇ t+m
  • the measurements arrive out-of-order to the filter, that is, a measurement that corresponds to an earlier state arrives after the measurement that corresponds to a later state. This violates the Markov assumption of the Kalman filter. Also, due to the sensor processing delay, measurements may run behind the state propagation.
  • the priority queue essentially serves as a measurement reordering mechanism ( FIG. 2 ) for all measurements that are not older than t d from the current state.
  • FIG. 2 we always utilize the most recent IMU measurement to propagate the state forward. We, however, only propagate the covariance on demand. As illustrated in FIG. 2 , the covariance is only propagated from the time of the last measurement to the current measurement.
  • GPS and magnetometer may be available. It is straightforward to fuse the GPS as a global pose measurement and generate the optimal state estimate. However, this may not be the best for real-world applications.
  • a vehicle that operates in a GPS-denied environment may suffer from accumulated drift.
  • the vehicle gains GPS signal as illustrated in FIG. 3A , there may be large discrepancies between the GPS measurement and the estimated state (z 5 -s 5 ). Directly applying GPS as global measurements will result in undesirable behaviors in both estimation (large linearization error) and control (sudden pose change).
  • the optimal pose graph configuration can be found with available solvers [18], as shown in FIG. 3B .
  • the pose graph is disconnected if there are no relative exteroceptive measurements between two nodes. Let two pose graphs be disconnected between k ⁇ 1 and k.
  • the pose graph SLAM provides the transformation between the non-optimized s k-1 and the SLAM-optimized s k-1 + state. This transform can be utilized to transform the global GPS measurement to be aligned with s k-1 :
  • ⁇ t ⁇ 1 s k-1 ⁇ s k-1 +
  • the experimental platform shown in FIG. 1 is based on the Pelican quadrotor from Ascending Technologies, GmbH (http://www.asctec.de/). This platform is natively equipped with an AutoPilot board consisting of an IMU and a user-programmable ARM7 microcontroller.
  • the main computation unit onboard is an Intel NUC with a 1.8 GHz Core i3 processor with 8 GB of RAM and a 120 GB SSD.
  • the sensor suite includes a ublox LEA-6T GPS module, a Hokuyo UTM-30LX LiDAR and two mvBlueFOX-1VILC200 w grayscale HDR cameras with fisheye lenses that capture 752 ⁇ 480 images at 25 Hz. We use hardware triggering for frame synchronization.
  • the onboard auto exposure controller is fine tuned to enable fast adaption during rapid light condition changes.
  • a 3-D printed laser housing redirects some of the laser beams for altitude measurement.
  • the total mass of the platform is 1.87 kg.
  • the entire algorithm is developed in C++ using robot operating system (ROS) (http://www.ros.org) as the interfacing robotics middleware.
  • ROS robot operating system
  • Some onboard sensors are capable of producing absolute measurements (Sect. 111-A), here are their details:
  • z t [ ( x t w y t w ) R b w ⁇ ( x . t b y . t b ) ⁇ t w ] + n t .
  • a position tracking controller with a nonlinear error metric [20].
  • the 100 Hz filter output (Sect. IV) is used directly as the feedback for the controller.
  • the attitude controller runs at 1 kHz on the ARM processor on the MAV's AutoPilot board, while the position tracking control operates at 100 Hz on the main computer.
  • FIGS. 6A-6H The testing site spans a variety of environments, including outdoor open space, densely filled trees, cluttered building area, and indoor environments ( FIGS. 6A-6H ).
  • the MAV is autonomously controlled using the onboard state estimates.
  • a human operator always has the option of sending high level waypoints or velocity commands to the vehicle.
  • the total flight time is approximately 8 minutes, and the vehicle travels 445 meters with an average speed of 1.5 m/s.
  • FIG. 9 shows the evolution of covariance as the vehicle flies through a GPS shadowing area.
  • the global x, y and yaw error is bounded by GPS measurement, without which the error will grow unbounded. This matches the observability analysis results. It should be noted that the error on body frame velocity does not grow, regardless of the availability of GPS. The spike in velocity covariance in FIG. 9 is due to the camera facing direct sunlight.
  • FIGS. 11A and 11B depict onboard ( FIG. 11A ) and external ( FIG. 11B ) camera images as the MAV autonomously flies through a tree-lined campus environment. Note the nontrivial light condition.
  • FIG. 12 is a block diagram illustrating an MAV for performing fusing measurements from sensors that produce both absolute and relative measurements according to an embodiment of the subject matter described herein.
  • the MAV 100 includes one or more motors 102 for controlling motion of the MAV using one or more rotors 104 .
  • the Pelican Quadro Rotor available from Ascending Technologies was used. However, other rotorcraft can be substituted without departing from the scope of the subject matter described herein.
  • It also includes a controller 106 for controlling operation of the motors 102 based on sensor input.
  • a computation unit 108 includes a sensor fusion module 110 that fuses the measurements from multiple sensors and produces an output signal to controller 106 .
  • sensor fusion module 110 receives input from IMU 112 , pressure altimeter 114 , magnetometer 116 , laser scanner 118 , GPS receiver 120 , cameras 122 , and pressure altimeter 123 .
  • Sensor fusion module 110 converts relative measurements, such as those produced by laser scanner 118 and cameras 122 to measurements that depend on augmented states as described above.
  • the transformed measurements are combined using the Unscented Kalman Filter described above and output to controller 106 .
  • the signal provided as output to controller 106 serves as feedback to controller 106 for controlling position, velocity, and acceleration of MAV 100 .
  • Controller 106 also receives inputs from a trajectory estimator 124 , which estimates the trajectory of MAV 100 needed to arrive at user-specific waypoints.
  • FIG. 13 is a flow chart illustrating an exemplary process for controlling motion of a rotorcraft MAV using multi-sensor fusion according to an embodiment of the subject matter described herein.
  • input is received from sensors of multiple different modalities.
  • computation unit 108 and sensor fusion module 110 may receive input from any one or more of the sensors illustrated in FIG. 12 from which output is available at a given time.
  • relative output measurements produced by some of the sensors that depend on previous states are converted into measurements that depend on augmented states. The process of performing such conversions is described above in Section IV(A).
  • step 204 measurements from the different sensors are combined and filtered. For example, the measurements may be combined using an Unscented Kalman Filter.
  • the combined measurements are output to a trajectory generator along with a waypoint input by a user.
  • the output of the trajectory generator is used to control motion of the rotorcraft MAV.
  • an autonomous rotorcraft MAV may include a trajectory generator or estimator 124 for generating a trajectory plan for controlling a trajectory of a rotorcraft MAV during flight based on an estimated current state of the rotorcraft MAV and a waypoint input by a user.
  • the following description illustrates trajectory planning that may be performed by trajectory generator or estimator 124 according to one embodiment of the subject matter described herein.
  • the subject matter described herein includes present a system design that enables a light-weight quadrotor equipped with only forward-facing cameras and an inexpensive IMU to autonomously navigate and efficiently map complex environments.
  • the performance of the proposed system is demonstrated via experiments in complex indoor and outdoor environments.
  • Quadrotor micro-aerial vehicles are ideal platforms for surveillance and search and rescue in confined indoor and outdoor environments due to their small size and superior mobility. In such missions, it is essential that the quadrotor be autonomous to minimize operator workload. In this work, we are interested in pursuing a light-weight, off-the-shelf quadrotor to autonomously navigate complex unknown indoor and outdoor environments using only onboard sensors with the critical control computations running in real-time onboard the robot.
  • the experimental platform ( FIG. 14 ) is based on the Hummingbird quadrotor from Ascending Technologies (see http:/www.asctec/de). This off-the-shelf platform comes with an AutoPilot board that is equipped with an inexpensive IMU and a user-programmable ARM7 microcontroller.
  • the high level computer onboard includes an Intel Atom 1.6 GHz processor and 1 GB RAM. Communication between the onboard computer and a ground station is via 802.11n wireless network.
  • the only new additions to the platform are two grayscale mvBlueFOX-MLC200 w cameras with hardware HDR. All cameras are equipped with fisheye lenses. The synchronization between cameras and IMU is ensured via hardware triggering.
  • the total weight of the platform is 740 g.
  • the software architecture is shown in FIG. 15 .
  • This architecture allows us to divide the computations between the onboard low and high level processors and the offboard ground station.
  • a vision-based estimator provides 6-DOF pose estimates at 20 Hz.
  • UDF unscented Kalman filter
  • a stereo-based visual SLAM module generates a 3D voxel grid map for the high level planner.
  • the SLAM module also provides global pose correction. However, we do not directly fuse this pose correction with the vision-based state estimate since it may cause significant pose discontinuities in the event of large scale loop closures.
  • VINS Visual-Inertial
  • u ij is the unit length feature observation vector
  • d i ⁇ r j-1 ⁇ p i ⁇ .
  • the location of the feature p i can be found by solving the following linear system:
  • the 20 Hz pose estimate from the vision system alone is not sufficient to control the robot.
  • An UKF with delayed measurement compensation is used to estimate the pose and velocity of the robot at 100 Hz [14].
  • the system state is defined as:
  • r is the 3D position of the robot
  • q is the quaternion representation the 3D orientation of the robot
  • a b is the bias of the accelerometer measurement in the body frame.
  • FIG. 16 shows the comparison of the performance of the VINS estimator against the ground truth from the Vicon motion capture system 2 during autonomous tracking of a predefined trajectory (Sect. VIII-A).
  • the lack of global bundle adjustment of the VINS estimator results in long term drift in the estimated pose due to recursive formulation. We therefore introduce an odometry frame, (r j O ,R j O ) to represent such drifting behavior.
  • Visual SLAM is a widely studied area. In small workspaces, approaches that use recursive filtering [15] or parallel tracking and mapping techniques [16] yield accurate results. Large scale mapping with monocular [17] or stereo [18] cameras are achieved using pose graph-based formulations. In our system, due to the limited onboard computation resources, limited wireless transmission bandwidth, and the accuracy of the onboard estimator, a high rate visual SLAM is both unnecessary and infeasible. Therefore, our visual SLAM module runs offboard with a maximum rate of 1 Hz.
  • the pose correction from the visual SLAM d j WO which serves as the transform between the odometry frame and the world frame, is formulated such that:
  • the robot flies through all transformed waypoints using the state estimate in the odometry frame for feedback control, it will also fly through the same sets of waypoints in the world frame. Moreover, it there are large scale loop closures (i.e. large changes in d j WO ), the set of waypoints that the robot is heading towards will change significantly. However, if we are able to regenerate smooth trajectories with initial conditions equal to the current state of the robot, the transition between trajectories will be smooth and no special handling is needed within the onboard state estimator and the controller.
  • y is a collection of desired derivative values at each waypoint, which can be either free or fixed.
  • y is a collection of desired derivative values at each waypoint, which can be either free or fixed.
  • the velocity and acceleration are set to be zero for the last waypoint. For all other waypoints, only position is fixed and the trajectory generator will provides the velocity and acceleration profile.
  • a limitation of the above trajectory generation approach is the necessity of predefining the travel time between waypoints. Due to computational constraints, we do not perform any iterative time optimization [27, 28] to find the optimal segment time, but rather use a heuristic that approximates the segment time as a linear trajectory that always accelerates from and decelerates to zero speed with a constant acceleration at the beginning and end of a segment, and maintains constant velocity in the middle of a segment. This simple heuristic can help avoid excessive accelerations during short segments, and is a reasonable time approximation for long segments.
  • FIGS. 18A and 18B show in simulation a quadrotor tracking a smooth trajectory generated from a sequence of waypoints.
  • a change of waypoints and trajectory regeneration take place at 20 s.
  • the regenerated trajectory smoothly connects to the initial trajectory and the quadrotor is able to smoothly switch waypoints.
  • a position tracking controller with a nonlinear error metric [29] due to its superior performance in highly dynamical motions that involve large angle changes and significant accelerations.
  • the 100 Hz state estimate from the VINS system (Sect. III) is used directly as the feedback for the controller.
  • the attitude controller runs at 1 kHz on the ARM processor on the robot's AutoPilot board, while the position tracking control operates at 100 Hz on the Atom processor.
  • the first experiment demonstrates the ability of the proposed system to maintain globally consistent tracking. We provide a comparison with ground truth to quantify the performance.
  • the robot navigates an indoor environment with a large loop (approximately 190 m) and completes the loop within one battery charge (less than 5 minutes of flight time).
  • we present an outdoor navigation experiment that emphasizes the robustness of the proposed system against environment changes and strong wind disturbance.
  • the robot autonomously follows a smooth trajectory generated from a rectangle pattern at approximately 1 m/s.
  • the ground truth from Vicon is used to quantify the global tracking performance.
  • FIG. 20A and FIG. 20B there is slow position drift in the VINS state estimate.
  • global corrections from the offboard visual SLAM results in a globally consistent operation.
  • the robot is controlled using the VINS state estimate, although global loop closure is clearly being merged into the system. Due to the correction from the visual SLAM, the desired smooth trajectory in the odometry frame regenerates and changes over time.
  • This experiment demonstrates the performance of the proposed system in outdoor environments.
  • the experiment is conducted in a typical winter day at Philadelphia, Pa., where the wind speed goes up to 20 km/hr.
  • the total travel distance is approximately 170 m with a total duration of 166 s ( FIG. 23 ).
  • Snapshots from the video camera and images captured by the onboard camera are shown in FIGS. 24A-24D .
  • the outdoor environment is largely unstructured, consisting of trees and vegetation, demonstrating the ability of the system to also operate in unstructured environments.
  • An integrated laser- and/or GPS-based state estimation approach may be incorporated into our current system to extend the operational environments and enhance the system robustness.

Landscapes

  • Engineering & Computer Science (AREA)
  • Remote Sensing (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Automation & Control Theory (AREA)
  • Aviation & Aerospace Engineering (AREA)
  • Mechanical Engineering (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Electromagnetism (AREA)
  • Navigation (AREA)
  • Position Fixing By Use Of Radio Waves (AREA)
  • Traffic Control Systems (AREA)
  • Control Of Position, Course, Altitude, Or Attitude Of Moving Bodies (AREA)
US15/165,846 2013-11-27 2016-05-26 Multi-sensor fusion for robust autonomous flight in indoor and outdoor environments with a rotorcraft micro-aerial vehicle (mav) Abandoned US20170212529A1 (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
US15/165,846 US20170212529A1 (en) 2013-11-27 2016-05-26 Multi-sensor fusion for robust autonomous flight in indoor and outdoor environments with a rotorcraft micro-aerial vehicle (mav)
US15/684,700 US10732647B2 (en) 2013-11-27 2017-08-23 Multi-sensor fusion for robust autonomous flight in indoor and outdoor environments with a rotorcraft micro-aerial vehicle (MAV)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
US201361910022P 2013-11-27 2013-11-27
PCT/US2014/067822 WO2015105597A2 (en) 2013-11-27 2014-11-28 Multi-sensor fusion for robust autonomous flight in indoor and outdoor environments with a rotorcraft micro-aerial vehicle (mav)
US15/165,846 US20170212529A1 (en) 2013-11-27 2016-05-26 Multi-sensor fusion for robust autonomous flight in indoor and outdoor environments with a rotorcraft micro-aerial vehicle (mav)

Related Parent Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2014/067822 Continuation WO2015105597A2 (en) 2013-11-27 2014-11-28 Multi-sensor fusion for robust autonomous flight in indoor and outdoor environments with a rotorcraft micro-aerial vehicle (mav)

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US15/684,700 Continuation US10732647B2 (en) 2013-11-27 2017-08-23 Multi-sensor fusion for robust autonomous flight in indoor and outdoor environments with a rotorcraft micro-aerial vehicle (MAV)

Publications (1)

Publication Number Publication Date
US20170212529A1 true US20170212529A1 (en) 2017-07-27

Family

ID=53524461

Family Applications (2)

Application Number Title Priority Date Filing Date
US15/165,846 Abandoned US20170212529A1 (en) 2013-11-27 2016-05-26 Multi-sensor fusion for robust autonomous flight in indoor and outdoor environments with a rotorcraft micro-aerial vehicle (mav)
US15/684,700 Active US10732647B2 (en) 2013-11-27 2017-08-23 Multi-sensor fusion for robust autonomous flight in indoor and outdoor environments with a rotorcraft micro-aerial vehicle (MAV)

Family Applications After (1)

Application Number Title Priority Date Filing Date
US15/684,700 Active US10732647B2 (en) 2013-11-27 2017-08-23 Multi-sensor fusion for robust autonomous flight in indoor and outdoor environments with a rotorcraft micro-aerial vehicle (MAV)

Country Status (6)

Country Link
US (2) US20170212529A1 (de)
EP (2) EP3074832A4 (de)
JP (2) JP2016540211A (de)
CN (2) CN109885080B (de)
CA (1) CA2931632C (de)
WO (1) WO2015105597A2 (de)

Cited By (39)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20170305546A1 (en) * 2014-04-29 2017-10-26 Baidu Online Network Technology (Beijing) Co., Ltd. Autonomous navigation method and system, and map modeling method and system
US20180188374A1 (en) * 2016-12-30 2018-07-05 Hon Hai Precision Industry Co., Ltd. Navigation systerm and method for using the same
US10037028B2 (en) 2015-07-24 2018-07-31 The Trustees Of The University Of Pennsylvania Systems, devices, and methods for on-board sensing and control of micro aerial vehicles
US20180217614A1 (en) * 2017-01-19 2018-08-02 Vtrus, Inc. Indoor mapping and modular control for uavs and other autonomous vehicles, and associated systems and methods
US20190004543A1 (en) * 2017-07-03 2019-01-03 Skydio, Inc. Detecting optical discrepancies in captured images
US10234856B2 (en) * 2016-05-12 2019-03-19 Caterpillar Inc. System and method for controlling a machine
US20190127067A1 (en) * 2016-05-03 2019-05-02 Sunshine Aerial Systems, Inc. Autonomous aerial vehicle
CN109916394A (zh) * 2019-04-04 2019-06-21 山东智翼航空科技有限公司 一种融合光流位置和速度信息的组合导航算法
US10395115B2 (en) 2015-01-27 2019-08-27 The Trustees Of The University Of Pennsylvania Systems, devices, and methods for robotic remote sensing for precision agriculture
CN110427046A (zh) * 2019-07-26 2019-11-08 沈阳航空航天大学 一种三维平滑随机游走无人机群移动模型
US20200025570A1 (en) * 2017-03-29 2020-01-23 Agency For Science, Technology And Research Real time robust localization via visual inertial odometry
US10604236B2 (en) * 2016-06-01 2020-03-31 Regents Of The University Of Minnesota Fault-tolerant aircraft flight control using a subset of aerodynamic control surfaces
CN110954101A (zh) * 2019-11-13 2020-04-03 南昌大学 一种利用Vicon的无人机激光定位的调试方法
US20200130864A1 (en) * 2018-10-29 2020-04-30 California Institute Of Technology Long-duration, fully autonomous operation of rotorcraft unmanned aerial systems including energy replenishment
USD886694S1 (en) * 2017-08-11 2020-06-09 Trifo, Inc. Autonomous vehicle sensor housing
CN111337037A (zh) * 2020-05-19 2020-06-26 北京数字绿土科技有限公司 移动激光雷达slam制图装置及数据处理方法
US10732647B2 (en) 2013-11-27 2020-08-04 The Trustees Of The University Of Pennsylvania Multi-sensor fusion for robust autonomous flight in indoor and outdoor environments with a rotorcraft micro-aerial vehicle (MAV)
US10748299B2 (en) * 2018-09-24 2020-08-18 Tata Consultancy Services Limited System and method of multirotor dynamics based online scale estimation for monocular vision
US20200277053A1 (en) * 2017-09-15 2020-09-03 Syracuse University Integrated guidance and feedback control for autonomous vehicle
WO2020181255A1 (en) * 2018-03-07 2020-09-10 Skylla Technologies, Inc. Collaborative task execution with humans and robotic vehicles
CN111896008A (zh) * 2020-08-20 2020-11-06 哈尔滨工程大学 一种改进的鲁棒无迹卡尔曼滤波组合导航方法
US10884430B2 (en) 2015-09-11 2021-01-05 The Trustees Of The University Of Pennsylvania Systems and methods for generating safe trajectories for multi-vehicle teams
CN113361194A (zh) * 2021-06-04 2021-09-07 安徽农业大学 一种基于深度学习的传感器漂移校准方法、电子设备及存储介质
US11119507B2 (en) * 2018-06-27 2021-09-14 Intel Corporation Hardware accelerator for online estimation
US20210293544A1 (en) * 2016-03-11 2021-09-23 Kaarta, Inc. Laser scanner with real-time, online ego-motion estimation
US11164149B1 (en) * 2016-08-31 2021-11-02 Corvus Robotics, Inc. Method and system for warehouse inventory management using drones
US11173921B2 (en) 2018-11-19 2021-11-16 Micron Technology, Inc. Sensor fusion to determine reliability of autonomous vehicle operation
US20210356293A1 (en) * 2019-05-03 2021-11-18 Lg Electronics Inc. Robot generating map based on multi sensors and artificial intelligence and moving based on map
TWI747718B (zh) * 2020-12-14 2021-11-21 大陸商廣州昂寶電子有限公司 位移補償方法和設備及速度補償方法和設備
US20220060628A1 (en) * 2020-08-19 2022-02-24 Honeywell International Inc. Active gimbal stabilized aerial visual-inertial navigation system
US20220092766A1 (en) * 2020-09-18 2022-03-24 Spirit Aerosystems, Inc. Feature inspection system
US11343924B2 (en) * 2017-11-24 2022-05-24 SZ DJI Technology Co., Ltd. Unmanned aerial vehicle and avionics system thereof
US20220197307A1 (en) * 2020-12-18 2022-06-23 InSitu, Inc., a subsidiary of the Boeing Company Landing a vertical landing vehicle
US11402855B2 (en) 2017-07-21 2022-08-02 Nec Corporation Processing device, drive control device, data processing method, and storage medium for attitude control of moving body based on wind conditions
CN115235475A (zh) * 2022-09-23 2022-10-25 成都凯天电子股份有限公司 一种基于mcc的ekf-slam后端导航路径优化方法
US11573325B2 (en) 2016-03-11 2023-02-07 Kaarta, Inc. Systems and methods for improvements in scanning and mapping
US11814158B1 (en) 2022-04-28 2023-11-14 Beta Air, Llc Systems and methods for determining areas of discrepancy in flight for an electric aircraft
US11815601B2 (en) 2017-11-17 2023-11-14 Carnegie Mellon University Methods and systems for geo-referencing mapping systems
US11830136B2 (en) 2018-07-05 2023-11-28 Carnegie Mellon University Methods and systems for auto-leveling of point clouds and 3D models

Families Citing this family (59)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104718508B (zh) 2012-04-30 2017-09-08 宾夕法尼亚大学理事会 四旋翼机组的三维操纵
JP6080189B2 (ja) 2014-08-15 2017-02-15 エスゼット ディージェイアイ テクノロジー カンパニー リミテッドＳｚ Ｄｊｉ Ｔｅｃｈｎｏｌｏｇｙ Ｃｏ．，Ｌｔｄ インラインセンサの較正方法及び較正装置
WO2017071143A1 (en) 2015-10-30 2017-05-04 SZ DJI Technology Co., Ltd. Systems and methods for uav path planning and control
WO2017045116A1 (en) * 2015-09-15 2017-03-23 SZ DJI Technology Co., Ltd. System and method for supporting smooth target following
DE102016211805A1 (de) 2015-10-09 2017-04-13 Volkswagen Aktiengesellschaft Fusion von Positionsdaten mittels Posen-Graph
DE102015226365A1 (de) * 2015-12-21 2017-06-22 Robert Bosch Gmbh Verfahren zur Messung der Varianz in einem Messsignal, Verfahren zur Datenfusion, Computerprogramm, Maschinenlesbares Speichermedium und Vorrichtung
US10156441B2 (en) 2016-01-05 2018-12-18 Texas Instruments Incorporated Ground plane estimation in a computer vision system
CN105425818B (zh) * 2016-01-15 2018-08-31 中国人民解放军国防科学技术大学 一种无人飞行器自主安全飞行控制方法
JP6663606B2 (ja) * 2016-03-08 2020-03-13 国立大学法人京都大学 無人航空機位置推定方法及びシステム
US11567201B2 (en) 2016-03-11 2023-01-31 Kaarta, Inc. Laser scanner with real-time, online ego-motion estimation
US10989542B2 (en) 2016-03-11 2021-04-27 Kaarta, Inc. Aligning measured signal data with slam localization data and uses thereof
CN107543540B (zh) * 2016-06-27 2020-05-15 杭州海康机器人技术有限公司 一种飞行设备的数据融合和飞行模式切换方法及装置
CN106094840B (zh) * 2016-07-20 2019-03-01 深圳洲际通航投资控股有限公司 飞行控制系统及方法
CN106125751A (zh) * 2016-07-21 2016-11-16 天津津宇凯创航空科技发展有限公司 一种巡查飞行器安全控制系统
US10474148B2 (en) * 2016-07-27 2019-11-12 General Electric Company Navigating an unmanned aerial vehicle
EP3943888A1 (de) 2016-08-04 2022-01-26 Reification Inc. Verfahren für simultane lokalisierung und mapping (slam) und zugehörige vorrichtungen und systeme
CN106526542A (zh) * 2016-10-17 2017-03-22 西南大学 一种基于确定性采样的增广卡尔曼滤波方法
WO2018086133A1 (en) * 2016-11-14 2018-05-17 SZ DJI Technology Co., Ltd. Methods and systems for selective sensor fusion
WO2018125938A1 (en) * 2016-12-30 2018-07-05 DeepMap Inc. Enrichment of point cloud data for high-definition maps for autonomous vehicles
JP6275887B2 (ja) * 2017-01-05 2018-02-07 エスゼット ディージェイアイ テクノロジー カンパニー リミテッドＳｚ Ｄｊｉ Ｔｅｃｈｎｏｌｏｇｙ Ｃｏ．，Ｌｔｄ センサ較正方法及びセンサ較正装置
JP7141403B2 (ja) 2017-01-27 2022-09-22 カールタ インコーポレイテッド 実時間オンライン自己運動推定を備えたレーザスキャナ
US10386843B2 (en) * 2017-04-03 2019-08-20 Bell Helicopter Textron Inc. System and method for determining a position of a rotorcraft
KR101982822B1 (ko) * 2017-04-06 2019-05-29 명지대학교 산학협력단 멀티센서 기반의 웨어러블 실내공간정보 구축시스템
KR101956447B1 (ko) * 2017-04-20 2019-03-12 한국과학기술원 그래프 구조 기반의 무인체 위치 추정 장치 및 그 방법
CN107478220B (zh) * 2017-07-26 2021-01-15 中国科学院深圳先进技术研究院 无人机室内导航方法、装置、无人机及存储介质
CN109387192B (zh) * 2017-08-02 2022-08-26 湖南云箭格纳微信息科技有限公司 一种室内外连续定位方法及装置
CN107577646A (zh) * 2017-08-23 2018-01-12 上海莫斐信息技术有限公司 一种高精度轨迹运算方法及系统
US10591926B2 (en) * 2017-09-18 2020-03-17 Baidu Usa Llc Smooth road reference for autonomous driving vehicles based on 2D constrained smoothing spline
US10606277B2 (en) * 2017-09-18 2020-03-31 Baidu Usa Llc Speed optimization based on constrained smoothing spline for autonomous driving vehicles
CN107544533A (zh) * 2017-10-12 2018-01-05 中国人民解放军国防科技大学 多功能便携式微型无人机系统
WO2019165194A1 (en) 2018-02-23 2019-08-29 Kaarta, Inc. Methods and systems for processing and colorizing point clouds and meshes
WO2019195270A1 (en) 2018-04-03 2019-10-10 Kaarta, Inc. Methods and systems for real or near real-time point cloud map data confidence evaluation
CN110633336B (zh) * 2018-06-05 2022-08-05 杭州海康机器人技术有限公司 激光数据搜索范围的确定方法、装置及存储介质
CN109375647A (zh) * 2018-11-20 2019-02-22 中国航空工业集团公司西安航空计算技术研究所 微型多源感知计算系统
US11312379B2 (en) * 2019-02-15 2022-04-26 Rockwell Collins, Inc. Occupancy map synchronization in multi-vehicle networks
CN109931926B (zh) * 2019-04-04 2023-04-25 山东智翼航空科技有限公司 一种基于站心坐标系的小型无人机无缝自主式导航方法
CN110081881B (zh) * 2019-04-19 2022-05-10 成都飞机工业（集团）有限责任公司 一种基于无人机多传感器信息融合技术的着舰引导方法
US11565807B1 (en) 2019-06-05 2023-01-31 Gal Zuckerman Systems and methods facilitating street-level interactions between flying drones and on-road vehicles
US11389957B2 (en) * 2019-09-30 2022-07-19 Mitsubishi Electric Research Laboratories, Inc. System and design of derivative-free model learning for robotic systems
CN112578788B (zh) * 2019-09-30 2023-05-02 北京百度网讯科技有限公司 车辆避障二次规划方法、装置、设备和可读存储介质
US10717528B1 (en) * 2019-10-03 2020-07-21 Trung Vo Tran Automatic flying delivery drone in precalculated flight routes and method for delivering merchandises
KR102258505B1 (ko) * 2019-12-09 2021-05-28 금오공과대학교 산학협력단 무인항공기의 군집 내비게이션 방법
TWI715358B (zh) * 2019-12-18 2021-01-01 財團法人工業技術研究院 移動載具及其狀態估測與感測融合切換方法
CN111006694B (zh) * 2019-12-29 2022-03-18 北京理工大学 基于航迹规划的长航时惯性导航系统轨迹发生器设计方法
CN112198887B (zh) * 2019-12-31 2022-04-01 北京理工大学 一种多旋翼无人机机载计算机性能评估系统方法
CN111141277A (zh) * 2020-01-16 2020-05-12 中国地质科学院地球物理地球化学勘查研究所 航空导航系统及航空设备
CN113156926B (zh) * 2020-01-22 2024-05-17 深圳市优必选科技股份有限公司 机器人的有限状态机的建立方法、有限状态机和机器人
FR3106571B1 (fr) 2020-01-27 2022-08-12 Airbus Helicopters Drone multirotor équipé d’une protection périphérique et procédé de commande d’un tel drone multirotor
US11662472B2 (en) 2020-04-20 2023-05-30 Honeywell International Inc. Integrity monitoring of odometry measurements within a navigation system
CN111578940B (zh) * 2020-04-24 2021-05-11 哈尔滨工业大学 一种基于跨传感器迁移学习的室内单目导航方法及系统
US12007791B2 (en) 2020-05-11 2024-06-11 Soter Technology Inc Multi-drone/sensor platform with information lateralization and federated path planning
CN111722614B (zh) * 2020-06-15 2021-07-09 南京航空航天大学 一种基于广义观测器的四旋翼无人机故障估计方法
AT523734B1 (de) 2020-11-10 2021-11-15 Alpen Adria Univ Klagenfurt Verfahren und System zur Schätzung von Zustandsgrößen eines beweglichen Objekts mit modularer Sensorfusion
EP4095746A1 (de) * 2021-05-24 2022-11-30 Zenseact AB Werbewahrnehmungsentwicklung
CN113763548B (zh) * 2021-08-17 2024-02-27 同济大学 基于视觉-激光雷达耦合的贫纹理隧洞建模方法及系统
CN114489124B (zh) * 2022-01-05 2023-07-28 南京邮电大学 一种保证无人机编队一致性动作的方法
CN114967751B (zh) * 2022-06-21 2022-12-06 深圳华创电科技术有限公司 飞行器航迹追踪方法、装置、设备及存储介质
CN117250855B (zh) * 2023-11-14 2024-02-13 安徽大学 一种基于多目标优化的飞行机器人轨迹规划方法
CN117310773B (zh) * 2023-11-30 2024-02-02 山东省科学院海洋仪器仪表研究所 基于双目立体视觉的水下机器人自主定位方法及系统

Family Cites Families (85)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
FR2697796B1 (fr) 1992-11-10 1994-12-09 Sextant Avionique Dispositif d'évitement de collisions pour aéronef notamment avec le sol.
US6278945B1 (en) 1997-11-24 2001-08-21 American Gnc Corporation Fully-coupled positioning process and system thereof
US6308911B1 (en) 1998-10-30 2001-10-30 Lockheed Martin Corp. Method and apparatus for rapidly turning a vehicle in a fluid medium
US6422508B1 (en) 2000-04-05 2002-07-23 Galileo Group, Inc. System for robotic control of imaging data having a steerable gimbal mounted spectral sensor and methods
TW539866B (en) * 2001-07-20 2003-07-01 American Gnc Corp Integrated GPS/IMU method and microsystem thereof
US6876945B2 (en) 2002-03-25 2005-04-05 Nicholas Jon Emord Seamless sensory system
US8712144B2 (en) 2003-04-30 2014-04-29 Deere & Company System and method for detecting crop rows in an agricultural field
US7343232B2 (en) 2003-06-20 2008-03-11 Geneva Aerospace Vehicle control system including related methods and components
US7289906B2 (en) 2004-04-05 2007-10-30 Oregon Health & Science University Navigation system applications of sigma-point Kalman filters for nonlinear estimation and sensor fusion
US7818127B1 (en) 2004-06-18 2010-10-19 Geneva Aerospace, Inc. Collision avoidance for vehicle control systems
US7228227B2 (en) 2004-07-07 2007-06-05 The Boeing Company Bezier curve flightpath guidance using moving waypoints
US7249730B1 (en) 2004-09-23 2007-07-31 United States Of America As Represented By The Secretary Of The Army System and method for in-flight trajectory path synthesis using the time sampled output of onboard sensors
US8019544B2 (en) 2005-01-03 2011-09-13 The Boeing Company Real-time refinement method of spacecraft star tracker alignment estimates
CA2605177C (en) * 2005-04-19 2011-06-21 Jaymart Sensors, Llc Miniaturized inertial measurement unit and associated methods
US20070235592A1 (en) 2005-12-05 2007-10-11 Horn Phillippe L Minimum time or thrust separation trajectory for spacecraft emergency separation
US8050863B2 (en) 2006-03-16 2011-11-01 Gray & Company, Inc. Navigation and control system for autonomous vehicles
CN101109640A (zh) * 2006-07-19 2008-01-23 北京航空航天大学 基于视觉的无人驾驶飞机自主着陆导航系统
US7643893B2 (en) 2006-07-24 2010-01-05 The Boeing Company Closed-loop feedback control using motion capture systems
US7925049B2 (en) 2006-08-15 2011-04-12 Sri International Stereo-based visual odometry method and system
EP2089677B1 (de) * 2006-12-06 2016-06-08 Honeywell International Inc. Verfahren, vorrichtungen und systeme für verbesserte synthetische visionen und multisensor-datenfusionen zur verbesserung der betrieblichen funktionen unbemannter flugkörper
US20080195316A1 (en) 2007-02-12 2008-08-14 Honeywell International Inc. System and method for motion estimation using vision sensors
CN101676744B (zh) * 2007-10-31 2012-07-11 北京航空航天大学 一种复杂背景低信噪比下弱小目标高精度跟踪方法
WO2011126605A2 (en) 2010-02-14 2011-10-13 Trimble Navigation Limited Gnss signal processing with regional augmentation network
US9766074B2 (en) * 2008-03-28 2017-09-19 Regents Of The University Of Minnesota Vision-aided inertial navigation
US8675068B2 (en) 2008-04-11 2014-03-18 Nearmap Australia Pty Ltd Systems and methods of capturing large area images in detail including cascaded cameras and/or calibration features
US8442355B2 (en) 2008-05-23 2013-05-14 Samsung Electronics Co., Ltd. System and method for generating a multi-dimensional image
US8040981B2 (en) * 2008-07-10 2011-10-18 Xilinx, Inc. Symbol detection in a MIMO communication system
US20110082566A1 (en) 2008-09-04 2011-04-07 Herr Hugh M Implementing a stand-up sequence using a lower-extremity prosthesis or orthosis
US8521339B2 (en) * 2008-09-09 2013-08-27 Aeryon Labs Inc. Method and system for directing unmanned vehicles
JP5151833B2 (ja) * 2008-09-09 2013-02-27 日本電気株式会社 移動体位置推定システム、移動体位置推定方法、及び移動体位置推定プログラム
US20100114408A1 (en) * 2008-10-31 2010-05-06 Honeywell International Inc. Micro aerial vehicle quality of service manager
US8380362B2 (en) 2009-07-10 2013-02-19 The Boeing Company Systems and methods for remotely collaborative vehicles
CN101598556B (zh) * 2009-07-15 2011-05-04 北京航空航天大学 一种未知环境下无人机视觉/惯性组合导航方法
US8398920B2 (en) 2009-07-28 2013-03-19 The Invention Science Fund I, Llc Drinking vessels and related systems and methods
EP2280241A3 (de) 2009-07-30 2017-08-23 QinetiQ Limited Fahrzeugsteuerung
CN101655561A (zh) * 2009-09-14 2010-02-24 南京莱斯信息技术股份有限公司 基于联合卡尔曼滤波的多点定位数据与雷达数据融合方法
US8577539B1 (en) * 2010-01-27 2013-11-05 The United States Of America As Represented By The Secretary Of The Air Force Coded aperture aided navigation and geolocation systems
US9568321B2 (en) 2010-04-19 2017-02-14 Honeywell International Inc. Systems and methods for determining inertial navigation system faults
FR2959812B1 (fr) 2010-05-05 2012-11-16 Thales Sa Procede d'elaboration d'une phase de navigation dans un systeme de navigation impliquant une correlation de terrain.
US9031809B1 (en) 2010-07-14 2015-05-12 Sri International Method and apparatus for generating three-dimensional pose using multi-modal sensor fusion
US8676498B2 (en) * 2010-09-24 2014-03-18 Honeywell International Inc. Camera and inertial measurement unit integration with navigation data feedback for feature tracking
US9058633B2 (en) 2010-10-25 2015-06-16 Trimble Navigation Limited Wide-area agricultural monitoring and prediction
US8756001B2 (en) 2011-02-28 2014-06-17 Trusted Positioning Inc. Method and apparatus for improved navigation of a moving platform
US8868323B2 (en) * 2011-03-22 2014-10-21 Honeywell International Inc. Collaborative navigation using conditional updates
WO2013105926A1 (en) 2011-03-22 2013-07-18 Aerovironment Inc. Invertible aircraft
US20140032167A1 (en) 2011-04-01 2014-01-30 Physical Sciences, Inc. Multisensor Management and Data Fusion via Parallelized Multivariate Filters
US9035774B2 (en) 2011-04-11 2015-05-19 Lone Star Ip Holdings, Lp Interrogator and system employing the same
US10027952B2 (en) 2011-08-04 2018-07-17 Trx Systems, Inc. Mapping and tracking system with features in three-dimensional space
CA2848217C (en) 2011-09-14 2018-09-18 Trusted Positioning Inc. Method and apparatus for navigation with nonlinear models
CN102411371A (zh) * 2011-11-18 2012-04-11 浙江大学 一种基于多传感器服务机器人跟随系统和方法
CN103814570A (zh) 2011-11-30 2014-05-21 三菱电机株式会社 影像监视系统
FR2985581B1 (fr) 2012-01-05 2014-11-28 Parrot Procede de pilotage d'un drone a voilure tournante pour operer une prise de vue par une camera embarquee avec minimisation des mouvements perturbateurs
US9104201B1 (en) 2012-02-13 2015-08-11 C&P Technologies, Inc. Method and apparatus for dynamic swarming of airborne drones for a reconfigurable array
US8874360B2 (en) 2012-03-09 2014-10-28 Proxy Technologies Inc. Autonomous vehicle and method for coordinating the paths of multiple autonomous vehicles
CN104718508B (zh) 2012-04-30 2017-09-08 宾夕法尼亚大学理事会 四旋翼机组的三维操纵
WO2013181558A1 (en) 2012-06-01 2013-12-05 Agerpoint, Inc. Systems and methods for monitoring agricultural products
US20140008496A1 (en) 2012-07-05 2014-01-09 Zhou Ye Using handheld device to control flying object
US9004973B2 (en) 2012-10-05 2015-04-14 Qfo Labs, Inc. Remote-control flying copter and method
JP6055274B2 (ja) 2012-10-31 2016-12-27 株式会社トプコン 航空写真測定方法及び航空写真測定システム
US9723230B2 (en) 2012-11-30 2017-08-01 University Of Utah Research Foundation Multi-spectral imaging with diffractive optics
FR3000813B1 (fr) 2013-01-04 2016-04-15 Parrot Drone a voilure tournante comprenant des moyens de determination autonome de position dans un repere absolu lie au sol.
US9243916B2 (en) * 2013-02-21 2016-01-26 Regents Of The University Of Minnesota Observability-constrained vision-aided inertial navigation
WO2014165031A1 (en) 2013-03-13 2014-10-09 Double Robotics, Inc. Accessory robot for mobile device
US9536427B2 (en) 2013-03-15 2017-01-03 Carnegie Mellon University Methods and software for managing vehicle priority in a self-organizing traffic control system
US20140312165A1 (en) 2013-03-15 2014-10-23 Armen Mkrtchyan Methods, apparatus and systems for aerial assessment of ground surfaces
US20140263822A1 (en) 2013-03-18 2014-09-18 Chester Charles Malveaux Vertical take off and landing autonomous/semiautonomous/remote controlled aerial agricultural sensor platform
US9607401B2 (en) * 2013-05-08 2017-03-28 Regents Of The University Of Minnesota Constrained key frame localization and mapping for vision-aided inertial navigation
US10063782B2 (en) 2013-06-18 2018-08-28 Motorola Solutions, Inc. Method and apparatus for displaying an image from a camera
WO2014202258A1 (en) * 2013-06-21 2014-12-24 National University Of Ireland, Maynooth A method for mapping an environment
US20150321758A1 (en) 2013-08-31 2015-11-12 II Peter Christopher Sarna UAV deployment and control system
WO2015039216A1 (en) 2013-09-17 2015-03-26 Invensense, Inc. Method and system for enhanced navigation with multiple sensors assemblies
DE102014211166A1 (de) 2013-11-20 2015-05-21 Continental Teves Ag & Co. Ohg Verfahren, Fusionsfilter und System zur Fusion von Sensorsignalen mit unterschiedlichen zeitlichen Signalausgabeverzügen zu einem Fusionsdatensatz
CA2930849C (en) 2013-11-20 2022-02-08 Rowbot Systems Llc Robotic platform and method for performing multiple functions in agricultural systems
CA2931632C (en) 2013-11-27 2020-07-14 The Trustees Of The University Of Pennsylvania Multi-sensor fusion for robust autonomous flight in indoor and outdoor environments with a rotorcraft micro-aerial vehicle (mav)
US9741140B2 (en) * 2014-05-19 2017-08-22 Microsoft Technology Licensing, Llc Fast solving for loop closure using a relative state space
CN107148633B (zh) 2014-08-22 2020-12-01 克莱米特公司 用于使用无人机系统进行农艺和农业监测的方法
US9129355B1 (en) 2014-10-09 2015-09-08 State Farm Mutual Automobile Insurance Company Method and system for assessing damage to infrastructure
FR3028186A1 (fr) 2014-11-12 2016-05-13 Parrot Equipement de telecommande de drone a longue portee
WO2016076586A1 (en) 2014-11-14 2016-05-19 Lg Electronics Inc. Mobile terminal and controlling method thereof
US20160214715A1 (en) 2014-11-21 2016-07-28 Greg Meffert Systems, Methods and Devices for Collecting Data at Remote Oil and Natural Gas Sites
US20160214713A1 (en) 2014-12-19 2016-07-28 Brandon Cragg Unmanned aerial vehicle with lights, audio and video
US9915956B2 (en) 2015-01-09 2018-03-13 Workhorse Group Inc. Package delivery by means of an automated multi-copter UAS/UAV dispatched from a conventional delivery vehicle
US10395115B2 (en) 2015-01-27 2019-08-27 The Trustees Of The University Of Pennsylvania Systems, devices, and methods for robotic remote sensing for precision agriculture
US10037028B2 (en) 2015-07-24 2018-07-31 The Trustees Of The University Of Pennsylvania Systems, devices, and methods for on-board sensing and control of micro aerial vehicles
WO2017095493A2 (en) 2015-09-11 2017-06-08 The Trustees Of The University Of Pennsylvania Systems and methods for generating safe trajectories for multi-vehicle teams

Cited By (49)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10732647B2 (en) 2013-11-27 2020-08-04 The Trustees Of The University Of Pennsylvania Multi-sensor fusion for robust autonomous flight in indoor and outdoor environments with a rotorcraft micro-aerial vehicle (MAV)
US9981742B2 (en) * 2014-04-29 2018-05-29 Baidu Online Network Technology (Beijing) Co., Ltd. Autonomous navigation method and system, and map modeling method and system
US20170305546A1 (en) * 2014-04-29 2017-10-26 Baidu Online Network Technology (Beijing) Co., Ltd. Autonomous navigation method and system, and map modeling method and system
US10395115B2 (en) 2015-01-27 2019-08-27 The Trustees Of The University Of Pennsylvania Systems, devices, and methods for robotic remote sensing for precision agriculture
US10037028B2 (en) 2015-07-24 2018-07-31 The Trustees Of The University Of Pennsylvania Systems, devices, and methods for on-board sensing and control of micro aerial vehicles
US10884430B2 (en) 2015-09-11 2021-01-05 The Trustees Of The University Of Pennsylvania Systems and methods for generating safe trajectories for multi-vehicle teams
US11573325B2 (en) 2016-03-11 2023-02-07 Kaarta, Inc. Systems and methods for improvements in scanning and mapping
US20210293544A1 (en) * 2016-03-11 2021-09-23 Kaarta, Inc. Laser scanner with real-time, online ego-motion estimation
US11585662B2 (en) * 2016-03-11 2023-02-21 Kaarta, Inc. Laser scanner with real-time, online ego-motion estimation
US20190127067A1 (en) * 2016-05-03 2019-05-02 Sunshine Aerial Systems, Inc. Autonomous aerial vehicle
US10234856B2 (en) * 2016-05-12 2019-03-19 Caterpillar Inc. System and method for controlling a machine
US10604236B2 (en) * 2016-06-01 2020-03-31 Regents Of The University Of Minnesota Fault-tolerant aircraft flight control using a subset of aerodynamic control surfaces
US11164149B1 (en) * 2016-08-31 2021-11-02 Corvus Robotics, Inc. Method and system for warehouse inventory management using drones
US20180188374A1 (en) * 2016-12-30 2018-07-05 Hon Hai Precision Industry Co., Ltd. Navigation systerm and method for using the same
US10649469B2 (en) * 2017-01-19 2020-05-12 Vtrus Inc. Indoor mapping and modular control for UAVs and other autonomous vehicles, and associated systems and methods
US20180217614A1 (en) * 2017-01-19 2018-08-02 Vtrus, Inc. Indoor mapping and modular control for uavs and other autonomous vehicles, and associated systems and methods
US20200025570A1 (en) * 2017-03-29 2020-01-23 Agency For Science, Technology And Research Real time robust localization via visual inertial odometry
US11747144B2 (en) * 2017-03-29 2023-09-05 Agency For Science, Technology And Research Real time robust localization via visual inertial odometry
US11760484B2 (en) * 2017-07-03 2023-09-19 Skydio, Inc. Detecting optical discrepancies in captured images
US11323680B2 (en) * 2017-07-03 2022-05-03 Skydio, Inc. Detecting optical discrepancies in captured images
US20190004543A1 (en) * 2017-07-03 2019-01-03 Skydio, Inc. Detecting optical discrepancies in captured images
US10379545B2 (en) * 2017-07-03 2019-08-13 Skydio, Inc. Detecting optical discrepancies in captured images
US20220337798A1 (en) * 2017-07-03 2022-10-20 Skydio, Inc. Detecting Optical Discrepancies In Captured Images
US11402855B2 (en) 2017-07-21 2022-08-02 Nec Corporation Processing device, drive control device, data processing method, and storage medium for attitude control of moving body based on wind conditions
USD886694S1 (en) * 2017-08-11 2020-06-09 Trifo, Inc. Autonomous vehicle sensor housing
US20200277053A1 (en) * 2017-09-15 2020-09-03 Syracuse University Integrated guidance and feedback control for autonomous vehicle
US11815601B2 (en) 2017-11-17 2023-11-14 Carnegie Mellon University Methods and systems for geo-referencing mapping systems
US11343924B2 (en) * 2017-11-24 2022-05-24 SZ DJI Technology Co., Ltd. Unmanned aerial vehicle and avionics system thereof
WO2020181255A1 (en) * 2018-03-07 2020-09-10 Skylla Technologies, Inc. Collaborative task execution with humans and robotic vehicles
US11119507B2 (en) * 2018-06-27 2021-09-14 Intel Corporation Hardware accelerator for online estimation
US11830136B2 (en) 2018-07-05 2023-11-28 Carnegie Mellon University Methods and systems for auto-leveling of point clouds and 3D models
US10748299B2 (en) * 2018-09-24 2020-08-18 Tata Consultancy Services Limited System and method of multirotor dynamics based online scale estimation for monocular vision
US20200130864A1 (en) * 2018-10-29 2020-04-30 California Institute Of Technology Long-duration, fully autonomous operation of rotorcraft unmanned aerial systems including energy replenishment
US11866198B2 (en) * 2018-10-29 2024-01-09 California Institute Of Technology Long-duration, fully autonomous operation of rotorcraft unmanned aerial systems including energy replenishment
US11173921B2 (en) 2018-11-19 2021-11-16 Micron Technology, Inc. Sensor fusion to determine reliability of autonomous vehicle operation
CN109916394A (zh) * 2019-04-04 2019-06-21 山东智翼航空科技有限公司 一种融合光流位置和速度信息的组合导航算法
US20210356293A1 (en) * 2019-05-03 2021-11-18 Lg Electronics Inc. Robot generating map based on multi sensors and artificial intelligence and moving based on map
US11960297B2 (en) * 2019-05-03 2024-04-16 Lg Electronics Inc. Robot generating map based on multi sensors and artificial intelligence and moving based on map
CN110427046A (zh) * 2019-07-26 2019-11-08 沈阳航空航天大学 一种三维平滑随机游走无人机群移动模型
CN110954101A (zh) * 2019-11-13 2020-04-03 南昌大学 一种利用Vicon的无人机激光定位的调试方法
CN111337037A (zh) * 2020-05-19 2020-06-26 北京数字绿土科技有限公司 移动激光雷达slam制图装置及数据处理方法
US20220060628A1 (en) * 2020-08-19 2022-02-24 Honeywell International Inc. Active gimbal stabilized aerial visual-inertial navigation system
CN111896008A (zh) * 2020-08-20 2020-11-06 哈尔滨工程大学 一种改进的鲁棒无迹卡尔曼滤波组合导航方法
US20220092766A1 (en) * 2020-09-18 2022-03-24 Spirit Aerosystems, Inc. Feature inspection system
TWI747718B (zh) * 2020-12-14 2021-11-21 大陸商廣州昂寶電子有限公司 位移補償方法和設備及速度補償方法和設備
US20220197307A1 (en) * 2020-12-18 2022-06-23 InSitu, Inc., a subsidiary of the Boeing Company Landing a vertical landing vehicle
CN113361194A (zh) * 2021-06-04 2021-09-07 安徽农业大学 一种基于深度学习的传感器漂移校准方法、电子设备及存储介质
US11814158B1 (en) 2022-04-28 2023-11-14 Beta Air, Llc Systems and methods for determining areas of discrepancy in flight for an electric aircraft
CN115235475A (zh) * 2022-09-23 2022-10-25 成都凯天电子股份有限公司 一种基于mcc的ekf-slam后端导航路径优化方法

Also Published As

Publication number Publication date
CN106030430A (zh) 2016-10-12
WO2015105597A4 (en) 2015-12-10
EP3470787A1 (de) 2019-04-17
US20180088597A1 (en) 2018-03-29
JP2016540211A (ja) 2016-12-22
EP3074832A2 (de) 2016-10-05
EP3074832A4 (de) 2017-08-30
WO2015105597A2 (en) 2015-07-16
CA2931632C (en) 2020-07-14
WO2015105597A3 (en) 2015-10-01
EP3470787B1 (de) 2020-04-15
CN109885080B (zh) 2022-09-20
CA2931632A1 (en) 2015-07-16
CN109885080A (zh) 2019-06-14
JP6739463B2 (ja) 2020-08-12
JP2018156660A (ja) 2018-10-04
US10732647B2 (en) 2020-08-04

Similar Documents

Publication Publication Date Title
US10732647B2 (en) Multi-sensor fusion for robust autonomous flight in indoor and outdoor environments with a rotorcraft micro-aerial vehicle (MAV)
Shen et al. Multi-sensor fusion for robust autonomous flight in indoor and outdoor environments with a rotorcraft MAV
US10565732B2 (en) Sensor fusion using inertial and image sensors
US20200130864A1 (en) Long-duration, fully autonomous operation of rotorcraft unmanned aerial systems including energy replenishment
EP3158293B1 (de) Sensorfusion unter verwendung von trägheits- und bildsensoren
Achtelik et al. Autonomous navigation and exploration of a quadrotor helicopter in GPS-denied indoor environments
Caballero et al. Vision-based odometry and SLAM for medium and high altitude flying UAVs
WO2016187759A1 (en) Sensor fusion using inertial and image sensors
Steiner et al. A vision-aided inertial navigation system for agile high-speed flight in unmapped environments: Distribution statement a: Approved for public release, distribution unlimited
EP3734394A1 (de) Sensorfusion unter verwendung von trägheits- und bildsensoren
Daftry et al. Robust monocular flight in cluttered outdoor environments
Shen Autonomous navigation in complex indoor and outdoor environments with micro aerial vehicles
EP3627447B1 (de) System und verfahren zur mehrrotordynamikbasierten online-massstabschätzung für monokulares sehen
Hinzmann et al. Monocular visual-inertial SLAM for fixed-wing UAVs using sliding window based nonlinear optimization
Mebarki et al. Vision-based and IMU-aided scale factor-free linear velocity estimator
Ready et al. Inertially aided visual odometry for miniature air vehicles in gps-denied environments
Wheeler et al. Relative navigation in GPS-degraded environments
Nyholm Globally consistent map generation in GPS-degraded environments
Ellingson Cooperative Navigation of Fixed-Wing Micro Air Vehicles in GPS-Denied Environments
Bassolillo et al. Enhanced Attitude and Altitude Estimation for Indoor Autonomous UAVs. Drones 2022, 6, 18
Awan Observability Properties of Relative State Estimation between Two Vehicles in a GPS-Denied Environment
Sobers Jr Efficient ranging-sensor navigation methods for indoor aircraft
McLain et al. Relative Navigation in GPS Degraded Environments
Ashraf Development of a low-cost solution for the navigation of UAVs in GPS-denied environment
McLain et al. Relative Navigation Approach for Vision-based Aerial GPS-denied Navigation

Legal Events

Date Code Title Description
STCB Information on status: application discontinuation

Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION