DE102017108733B4 - System and methods for remote monitoring of drones and aircraft for safety and health surveillance - Google Patents

Abstract

Method for estimating the mass of an aircraft (100,302,401) with a propulsion system with one or more rotating parts (101,501), the method comprising the following steps:- detecting the current position of the vehicle (100,302,401) with respect to a reference point in the three space coordinates x, y, z;- detecting the instantaneous rotational speed of one or more rotating parts (101, 501) of the drive system of the vehicle;- processing both the instantaneous position of the vehicle (100, 302, 401) and the instantaneous rotational speed of the one or more rotating parts (101,501) for estimating vehicle mass and flight characteristics of said vehicle (100,302,401), characterized in that detecting the instantaneous rotational speed of the one or more rotating parts (101,501) comprises:- a Remotely measuring the surface vibration of the vehicle (100, 302, 401) using a laser Doppler vibrome ters (LDV) (406); and- signal processing of the measured surface vibration.

Description

There is an increased demand for Unmanned Aerial Vehicles (UAVs), or drones in general, for many civilian applications, e.g. B. Transport, rescue and surveillance. At the same time, the risk of illegal use of drones related to data breaches, espionage and terrorism has greatly increased. The current state of the art includes many drone detection and identification systems based on different detection technologies and processing methods. Examples include: 1) Acoustic Sensors: Drones produce relatively loud acoustic noises from their propellers, engines, and drives. Appropriate acoustic sensors are used to capture and analyze drone noises in the time and frequency domains (e.g. frequency spectrum) to identify their unique acoustic signature compared to other noises produced by non-drone noise sources. These signatures can be evaluated directly and/or compared with previously stored databases to identify drone capabilities, manufacturers and risks. Examples of this approach: US9275645B2 , DE3929077C2 , US20090257314 A1 .

2) Image and Video Surveillance: All types of optical images such as snapshots, video/film images and thermal images are used to extract signatures for the drone's shape and all appearance aspects. These signatures can be evaluated directly and/or compared to pre-stored databases to identify drone capabilities, manufacturers and risks. Example of this approach: US8446321B2 .

3) Radio Frequency Sensors: One of the detection methods is based on detecting wireless RF signals between a drone and its remote controller. These signals can be control signals or/and video streams between a drone and a remote unit. By analyzing these signals, relevant signatures can be extracted. These signatures can be evaluated directly and/or compared to pre-stored databases to identify drone capabilities, manufacturers and risks. Example of this approach: DE102007062603A1

Currently available methods are unable to provide detailed information about a targeted aircraft, such as the gross weight (including payload) or the maintenance status of the vehicle and its rotating parts. For example, safety regulations for drones in the US and Germany are based on their weight. The drone weight is an important security feature to prevent the transport of dangerous materials such as spy equipment, bombs or explosives. The information about the maintenance status, such as wear and aging of the components, defects or component errors is also important to predict and avoid vehicle failures and resulting accidents. Acoustic sensors are sensitive to background noise, which is unavoidable in noisy sensitive areas such as airports and sporting events. In addition, certain types of drones have low acoustic noise, especially at long distances. Issues like background noise and weak drone noise emissions can lead to frequent and expensive false positives during signature extraction.

A method for determining the mass of a rotary wing aircraft is from U.S. 2017/0 010 148 A1 known.

An object of the invention is to provide methods and systems for remote aircraft monitoring which have improved capabilities to reduce the disadvantages mentioned.

This object is achieved by a method for estimating the mass of an aircraft with a propulsion system with one or more rotating parts, which has the features of independent claim 1. Preferred embodiments of the method according to the invention are defined in the dependent patent claims.

The term "aircraft" as understood by the invention includes any object, manned or unmanned, that can propel itself through the air on a controlled trajectory free of ground contact. This includes but is not limited to: drones, unmanned aerial vehicles (UAVs), fixed wing aircraft, rotary wing aircraft, helicopters and missiles.

The one or more rotating parts of the vehicle may represent rotor blades, propeller blades, turbine blades, jet/gas compressors, piston engines, or similar parts.

The processing of the current location and the current rate of rotation may further comprise the steps of: recording and processing a set of consecutive recordings of the current locations of the vehicle to calculate the corresponding current speed and acceleration of the vehicle; insertion of the current one location, velocity and acceleration of the vehicle into a first set of polynomial and/or differential equations of a first set of unknown coefficients to form parametric formulas for instantaneous expended thrust, energy and power of the vehicle; substituting a set of the instantaneous rotational speeds of the rotating parts in a second set of polynomial and/or differential equations of a second set of unknown coefficients to form parametric formulas for instantaneously produced thrust, energy and power of the vehicle; Formation of a combined set of polynomial and/or differential equations from the first set of equations and the second set of equations using the following basic conditions: that the sum of the parametrized thrust force consumed and the parametrically generated thrust force equal zero and/or the sum of the parametrically generated energy and the parametrically consumed energy is equal to zero; numerically solving the combined set of generated and consumed thrust, energy and power equations, using sufficient mappings of the locations and speeds, to estimate the unknown coefficients contained in the combined set of equations; wherein at least one of the first or second set of unknown coefficients represents vehicle mass.

The processing of the set of consecutive samples of the current locations may include, but is not limited to, the application of Kalman filters, state estimators, observers, or numerical differentiators.

In some embodiments, sensing the instantaneous rotational speed of one or more rotating parts may include: remotely sensing vehicle noise using an acoustic sensor; and signal processing the acoustic noise using the time waveform and/or the frequency spectrum to extract the rotational speed of the one or more rotating parts of the vehicle.

The processing of the acoustic noise may involve the application of known signal processing methods such as frequency spectrum comparison, cyclostationary correlation, blade sweep frequency and instantaneous energy analysis.

In the method according to the invention, detecting the instantaneous rotational speed of said one or more rotating parts comprises: a remote measurement of surface vibrations of the vehicle using a Laser Doppler Vibrometer (LDV); and signal processing of the measured surface vibration.

Signal processing of the measured surface vibration may further include: applying a time waveform and/or a frequency spectrum comparison to extract the rotational speed of the one or more rotating parts of the vehicle.

The waveform and/or frequency spectrum comparison may involve an application of known signal processing techniques such as frequency spectrum comparison, cyclostationary correlation, blade sweep frequency, and/or instantaneous energy analysis.

In some embodiments, detecting the instantaneous rotational speed of said one or more rotating parts of the vehicle may include: applying signal processing to distance signals between propellers or rotor blades and a reference point measured by a laser range finder; wherein the signal processing includes extracting from the distance signal time and/or frequency features that are synchronized with the instantaneous rotational speed of the one or more rotating parts.

When measuring the distance of the vehicle from a reference point with distance measuring devices based on it, periodic fluctuations, disturbances or errors in the measured distance signal can occur, which are caused by rotating parts passing through the laser beam used for the measurement. Such variations can in turn be processed by known signal processing techniques such as time waveform or frequency spectrum analysis. Such methods may include frequency spectrum comparison, cyclostationary correlation, blade sweep frequency, angle resampling, and/or instantaneous energy analysis.

In further embodiments, the surface vibration of the vehicle measured by the Laser Doppler Vibrometer (LDV) can be recorded using digital or analog methods, and processed to monitor wear, failures and failures of said vehicle components; this processing can be integrated into condition-based maintenance.

Surface vibration processing may further include: applying a time waveform and/or frequency spectrum comparison to extract signatures compatible with Ver degree of wear, faults and malfunctions of the vehicle are correlated.

The surface vibration of the vehicle measured by the Laser Doppler Vibrometer (LDV) can also be recorded and processed via digital or analog methods and used to classify the vehicle's signatures; these signatures are stored to compare different vehicle manufacturers and flight characteristics.

In other embodiments, the vehicle's acoustic noise may be further recorded and processed via digital or analog methods to monitor vehicle component wear, failures and failures, using time waveform and/or frequency spectrum comparison to extract signatures that are correlated with the degree of wear or type of failure of the vehicle.

Vibration and/or acoustic noise recording and processing may be used in combination with or independently of the estimation methods that estimate vehicle mass as described above.

The signatures mentioned above can include, but are not limited to: the square energy of the entire vibration signal, the mean square energy of a set of bandpass filtered segments of the entire vibration signal, the frequency spectrum, the vibration bandwidth, a cyclostationary correlation, spectral kurtosis, and an angle resampling -Technology.

This invention involves three novel parts: first, a method to remotely estimate drone weight, including payload, in flight using surface vibration and the drone's current flight position. Second, two laser-based methods for remotely measuring surface vibrations of flying drones. Third, a method to extract the unique signature of the drone's surface vibration for the purpose of identifying different drones, and to undertake the monitoring of the drone's wear status, failures and malfunctions of drones as a predictive maintenance system.

• : eine schematische Zeichnung eines beweglichen Koordinatensystems eines Fahrzeugs in einem festen Bezugssystem,
• : Zeitverlaufsdiagramm und Frequenzspektrumsdiagramm der Schallemission eines Fahrzeugs,
• : ein schematisches Flussdiagramm einer Methode zur Schätzung der Fahrzeugmasse,
• : Eine schematische Zeichnung eines Systems zur Bestimmung des Wartungszustandes eines Fahrzeugs,
• : Eine schematische Zeichnung eines Verfahrens zur Bestimmung der Rotationsgeschwindigkeit von einem Rotor.

Some possible embodiments of the invention are described below with reference to the attached figures. These show the following:

• : a schematic drawing of a moving coordinate system of a vehicle in a fixed reference frame,
• : Time history diagram and frequency spectrum diagram of the noise emission of a vehicle,
• : a schematic flow diagram of a vehicle mass estimation method,
• : A schematic drawing of a system for determining the maintenance status of a vehicle,
• : A schematic drawing of a method for determining the rotational speed of a rotor.

This method is applicable to all manned and unmanned aircraft and drones, such as fixed-wing aircraft, rotorcraft and helicopters, aircraft with electric and non-electric engines and rockets. Here it is explained as an example of a multi-rotor quadcopter drone.

shows an aircraft 100 in three different positions P1, P2, P3. In the example shown, the vehicle 100 is a multi-rotor quadcopter drone having a propulsion system 101 including rotors, a body 102 and a payload 103 . The vehicle 100 has an internal coordinate system 105 with the axes U, V, W. An external reference system 110 has the axes X, Y, Z, with the Z axis preferably being anti-parallel to the direction of gravity.

wobei x der Positionsvektor des Fahrzeugs 100 in dem Bezugssystem 110 ist, m ist das Drohnengewicht in kg, g ist die Gravitationskonstante 9,81 ms^-2, T ist der Schubvektor in N, der von dem Antriebssystem 101 erzeugt wird, R ist eine 3x3 Rotationsmatrix, welche den Drohnenbewegungsrahmen 105 in Bezug auf das Bezugssystem 110 ausrichtet. Das Bezugssystem 110 kann im Boden oder in einem beweglichen Fahrzeug (nicht gezeigt) festgelegt sein. F_r ist der gesamte Reibungs- und Widerstandskräftevektor in N. Das Ziel dieser Methode ist, die Fahrzeugmasse m zu schätzen, und zwar unter Verwendung der dynamischen Bewegungsgleichung (EOM) von Fahrzeug 100 wie in Gleichung (1). Zwei Messungen sind erforderlich: die erste ist die momentane Position des Fahrzeugs 100, Vektor P (n), wobei n die Nummer der Positionsmessungen ist und P (n) = (t, x, y, z) in Bezug auf Bezugssystem XYZ, wie in dargestellt.The equation of motion (EOM) of the vehicle 100 with respect to the frame 110 can be described as follows by the following dynamic model: $$m\ddot{x}=\left[\begin{array}{c}0\\ 0\\ -mG\end{array}\right]+TR+f_{\mathrm{right}}$$

where x is the position vector of the vehicle 100 in the frame of reference 110, m is the drone weight in kg, g is the gravitational constant 9.81 ms ^-2 , T is the thrust vector in N produced by the propulsion system 101, R is a 3x3 Rotation matrix that aligns the drone motion frame 105 with respect to the frame of reference 110. The frame of reference 110 may be fixed in the ground or in a moving vehicle (not shown). F _r is the total friction and drag force vector in N. The objective of this method is to estimate the vehicle mass m using the dynamic equation of motion (EOM) of vehicle 100 as in Equation (1). Two measurements are required: the first is the current position of the vehicle 100, vector P(n), where n is the number of position measurements and P(n)=(t,x,y,z) with respect to the XYZ frame, as in FIG shown.

wobei K eine unbekannte Konstante ist. Die Rotordrehzahl ω kann per Femermittlung aus dem akustischen Geräusch des Fahrzeugs 100 extrahiert werden.There are many positioning methods for estimating P and R such as typical RF radar, laser radar, sound localization or camera positioning systems. A set of n positioning points or shots, i.e. (P(1), P(2),...P ₍ n)) are required, with a time stamp, to estimate the instantaneous velocity and acceleration vector (see Eq. (1 )). The remaining variables in Eq. (1) are: the total generated thrust, T, from the propulsion system 101 and the total friction vector, F _r . The thrust of the propulsion system 101 may be related to the rotational speed of the rotor, ω in rad/s, assuming a polynomial formula, for example in Eq. (2): $$T=\mathrm{<figure-callout\; id="2"\; label="K\; \omega "\; filenames="DE102017108733B4\_0004.png"\; state="\{\{state\}\}">K</figure-callout>}{\omega }^{}{}^2$$

where K is an unknown constant. The rotor speed ω can be extracted from the acoustic noise of the vehicle 100 by remote detection.

A method for extracting the rotational speed ω of a rotating part from an acoustic noise is in shown.

The acoustic noise is captured by a suitable acoustic sensor (e.g. microphone) to collect time waveform data. The top graph 201 shows example time waveform data obtained by plotting the output voltage of a microphone over time as the microphone captures the transient sound of a quadcopter during launch. The time waveform data is then transformed into a frequency spectrum to extract motor shaft rotation speed versus rotor blade sweep frequency. The frequency spectrum obtained from the time waveform data displayed in the chart 201 is displayed in the chart 202. In this graph 202, the power density at each frequency and point in time is displayed such that higher power densities are displayed as lighter shading. The chart 202 is also referred to as a "power spectral density" chart or PSD. A rotor speed (about 640 Hz) and its second harmonic (640x2 Hz) are indicated by lines 203,204. The presence of a third harmonic (640x3 Hz) is also evident from the diagram.

A range of engine speeds (ω ₁ , ω ₂ ..., ω _n ) are required for the next step.

wobei α eine unbekannte Konstante ist. Die Polynomordnungen in Gl. (2-3) sind Beispiele und hängen von der erforderlichen Genauigkeit des Gewichtsvorhersagesystems ab. Ein Satz von Messungen für die momentane Drohnenbeschleunigung, die von einem separaten Positionierungssystem extrahiert wird, und die augenblickliche Drehgeschwindigkeit des Motors, die aus dem akustischen Geräusch der Drohne extrahiert wird, werden zur Lösung der Gl. 1-3 verwendet, wie in dargestellt.The total friction vector F _r can be neglected or calculated as a polynomial containing the speed of the drone, for example in Eq. (3): $$f_{\mathrm{right}}=a{\dot{x}}^2$$

where α is an unknown constant. The polynomial orders in Eq. (2-3) are examples and depend on the required accuracy of the weight prediction system. A set of measurements for the drone's instantaneous acceleration, extracted by a separate positioning system, and the instantaneous motor rotation rate, extracted from the drone's acoustic noise, are used to solve Eq. 1-3 used as in shown.

shows a drone weight detection system as follows: an external positioning system 301 such as RF radar, laser radar or optical 3D positioning is used to estimate instantaneous position, speed and acceleration vectors of an aircraft 302 and to provide this data as a data set 303 for further processing.

A device 304 for measuring the speed of a rotating element of the vehicle 302 provides the respective data as a data set 305 for further processing.

Device 304 may be as described above with reference to FIG work described. Other possible methods for determining the rotational speed of rotating elements of a vehicle are described below.

Operation of device 304 typically requires precise pointing of device 304 to vehicle 302 . To this end, device 304 or an associated tracking or targeting system may retrieve vehicle 302 location data from device 301 .

In a subsequent processing step 306, a set of multiple data sets 303, 305 in the parametric dynamic equations of motion such as Eq. (1-3) to estimate unknown parameters including the mass of the vehicle 302 and other parameters 307 such as K and α. The minimum number of data sets 303, 305 must be at least equal to the number of unknown parameters in Eq. (1-3), e.g., m, K and α.

Drone acoustic noise provides a reliable signature to identify drone manufacturers and characteristics. The efficiency of current microphone technologies (e.g. condenser gate, piezotransducer, parabolic) is limited to short ranges below 100-300 meters and can be up to 1000 meters, e.g. in the case of a parabolic microphone. Also, the drone's noise must be stronger than the background noise, and this condition may not be met for noisy areas such as airports and noisy places with music or sports events.

• Die erste Methode verwendet ein Laser-Doppler-Vibrometer (LDV). Das LDV ist eine bekannte Vorrichtung, die Vibrationen einer entfernten Oberfläche mittels einer Doppler-Verschiebung der reflektierten Laserstrahlfrequenz aufgrund der Bewegung (z. B. Körpervibration) einer Zieloberfläche femmessen kann. Das neue Verfahren in dieser Erfindung ist, dass das LDV verwendet werden kann, um die mechanische Vibration einer Drohne oder irgendwelcher Luftfahrzeuge berührungslos zu messen, indem der LDV-Laserstrahl auf einen Punkt auf der äußeren Oberfläche der Drohne gerichtet wird. Die Schwingung des Drohnenkörpers beinhaltet ein Vibrationsgeräusch von den Drohnenmotoren, das beispielsweise durch das Spektrogramm in 2 verarbeitet werden kann, um die momentane Drehzahl der Drohnenmotoren zu extrahieren.

An alternative to acoustic noise is drone surface vibration, which can be monitored using laser technology in two distinct and independent methods:

• The first method uses a Laser Doppler Vibrometer (LDV). The LDV is a known device that can remotely measure vibrations of a remote surface using a Doppler shift in the reflected laser beam frequency due to movement (e.g., body vibration) of a target surface. The new method in this invention is that the LDV can be used to non-contact measure the mechanical vibration of a drone or any aircraft by aiming the LDV laser beam at a point on the outer surface of the drone. The vibration of the drone body includes vibration noise from the drone motors, which can be seen, for example, by the spectrogram in 2 can be processed to extract the instantaneous RPM of the drone motors.

The surface vibrations collected by the LDV have several advantages: The LDV can measure vibrations with large frequency ranges (e.g. > 20 kHz) better than normal acoustic sensors thanks to the laser beam sensitivity. Frequency and time characteristics of LDV oscillations can provide unique drone signatures. A database of vibration signatures from commercial drones and aircraft in general from different manufacturers can be collected to identify flight characteristics and drone manufacturers.

The LDV measures the vibration of the target surface (drone) with minimal interference with background noise. The drone vibrations can be measured with an LDV even in extremely noisy areas.

The vibration signature measured by the LVD can also be used to monitor the general health or maintenance status and wear of drones, and to search for faults or failure conditions. An example of the health rating characteristic of aircraft (e.g. drone) with rotating parts and structures is the radio frequency power to total power ratio of the vibration signature. Due to normal operational degradation, the contribution from high frequency levels, from surface vibration or acoustic noise and overall vibration noise increases. This can be used to monitor and predict remaining drone life and critical errors.

An example of a health surveillance system is in shown. A drone 401 is placed on a hangar 402 serving as a launch pad for missions. When a mission request is received (e.g., for transport, rescue, or video surveillance), the drone 401 flies from the hangar 402 along a mission path 403 to another location 404 that represents the goal of the mission. After the mission is completed, the drone 401 flies back through a health surveillance zone 405 which requires the fixed or mobile LDV 406 to check the vibration signature of the flying drone 401 by scanning the surface/body of the drone (or any aircraft) with laser beams 407 from the LDV 406 contains. The LDV 406 generates a vibration signature. The vibration signature is analyzed by a set of algorithms 408 to decide whether the drone 401 is in an acceptable condition to fly to a hangar location 402 or whether the drone 401 needs to fly to a maintenance location 409 .

An LDV 406 is relatively expensive. An alternative method of measuring the speed of rotating parts of a vehicle is described with reference to FIG described. In this method, inexpensive laser-based devices are used as follows. The second alternative method of measuring the drone vibration caused by the propellers 501 is to use the laser range finder 502. The laser range finder 502 is a common device for measuring distances by transmitting a short pulse of laser beam to a distant reflective surface, to then convert it through a photoconverter receive. The total time of the pulse travel depends on the known speed of light and the separation distance between the laser source and the reflective surface. This method can be further used to measure the rotational speeds of drone propellers 501 as follows: By directing a laser beam 503 of a laser range finder 502 onto a drone propeller/blades 501, the laser beam 503 is scattered into a reflected beam 504, and a non-reflected beam 505 through the rotating blades 501 of the drone's engines. The reflected beam 504 contains correct measurements of distance; while not reflected beam 505 indicates out-of-range error signals. The distance signal from the reflected beam 504 is synchronized with the rotating blades 501 of the drone motors. By measuring the time rate of the reflected beam 504 to the source beam 503 using a frequency analyzer or appropriate algorithms, an accurate estimate for the velocities of the drone blades 501 and hence the motors can be obtained.

Claims (6)

Method for estimating the mass of an aircraft (100,302,401) with a propulsion system with one or more rotating parts (101,501), the method comprising the following steps: - detecting the current position of the vehicle (100,302,401) with respect to a reference point in the three spatial coordinates x, y, z; - Detection of the instantaneous rotational speed of one or more rotating parts (101, 501) of the drive system of the vehicle; - processing both the instantaneous position of the vehicle (100,302,401) and the instantaneous rotational speed of the one or more rotating parts (101,501) to estimate the vehicle mass and the flight characteristics of said vehicle (100,302,401), characterized in that the detection of the instantaneous rotational speed of the one or more rotating parts (101, 501) comprises: - a remote measurement of surface vibration of the vehicle (100, 302, 401) using a Laser Doppler Vibrometer (LDV) (406); and - signal processing of the measured surface vibration. procedure after claim 1 wherein the processing of the instantaneous position and the instantaneous rate of rotation comprises the steps of: - recording and processing a set of consecutive recordings of the instantaneous positions of the vehicle (100, 302, 401) to obtain the corresponding instantaneous speed and acceleration of the vehicle (100, 302 , 401) to calculate; - substituting the instantaneous position, velocity and acceleration of the vehicle (100, 302, 401) in a first set of polynomial and/or differential equations of a first set of unknown coefficients to provide parametric formulas for instantaneous thrust consumed, energy and generate (100, 302, 401) performance of the vehicle; - substituting a set of the instantaneous rotational speeds of the rotating parts (101, 501) in a second set of polynomial and/or differential equations of a second set of unknown coefficients to derive parametric formulas for the instantaneous generated thrust, energy and power of the vehicle form (100, 302, 401); - forming a combined set of polynomial and/or differential equations from the first set of equations and the second set of equations using the basic conditions that: the sum of the parametrically consumed thrust and the parametrically generated thrust is equal to zero, and/or the the sum of the parametrically generated energy and the parametrically consumed energy is equal to zero; - numerically solving the combined set of equations for produced and consumed thrust, energy and power using sufficient recordings of positions and rotational speeds to estimate the unknown coefficients contained in the combined set of equations; - wherein at least one of the first or second set of unknown coefficients represents the mass of the vehicle (100, 302, 401). procedure after claim 1 or 2 wherein the signal processing of the measured surface vibration comprises: - applying time waveform and/or frequency spectrum comparison to extract said rotational speed of the one or more rotating parts (101,501) of the vehicle (100,302,401). procedure after claim 1 or 2 wherein: the surface vibration of the vehicle (401) measured by the Laser Doppler Vibrometer (LDV) (406) is further recorded and processed via digital or analog methods to monitor degradation, failures and failures of components of the vehicle (401). ; - this processing can be integrated into a condition-based maintenance. procedure after claim 4 wherein the processing comprises: - applying time waveform and/or frequency spectrum comparison to extract signal signatures correlated to degradation level, faults and failures of the vehicle (401). procedure after claim 1 or 2 wherein: - the surface vibration of the vehicle measured by the Laser Doppler Vibrometer (LDV) (406) is further recorded and processed by digital or analog methods to classify signatures of the vehicle (100, 302, 401); - classifying the signatures of the vehicle; These signatures are saved to to compare the manufacturer and flight characteristics of the vehicle (100, 302, 401).

Images