RU2767477C1 - Uav navigation method - Google Patents

Abstract

FIELD: physics.SUBSTANCE: invention relates to autonomous navigation of drones based on optical images of the earth's surface. Method for autonomous navigation of unmanned aerial vehicles consists in the fact that reference and working images are obtained using optoelectronic systems in infrared range. Reference images are prepared based on the terrain thermal model by solving the direct problem of radiative heat transfer taking into account the incident solar radiation flux density, air temperature and flight altitude for given areas of the flight path and corresponding periods of astronomical time. Working images are recorded during shooting in nadir using optoelectronic systems in infrared range. Obtained reference and working infrared images are subjected to threshold processing, as a result of which reference and working matrices of reference points (contours) of objects are formed. Method includes calculating and finding the maximum of the two-dimensional matrix of the cross-correlation function, and estimating the geographical position of the maximum of the cross-correlation function from the reference matrix of reference points (contours). This estimate of the position of the maximum of the cross-correlation function is used for autonomous navigation of unmanned aerial vehicles based on data of measured values of heading, roll, pitch and altitude of the unmanned aerial vehicle.EFFECT: high accuracy of determining navigation parameters of an unmanned aerial vehicle under conditions of incorrect operation of a global satellite navigation system receiver.1 cl, 3 dwg

Description

The invention relates to the field of autonomous navigation of unmanned aerial vehicles on optical images of the earth's surface.

A known method of navigating radar images is described in US patent No. 5430445, 12.31.1992, G01S 13/90, in which the reference radar image is formed on the basis of a previously obtained photograph of a given area of the earth's surface. The standard preparation procedure consists of four stages: digitizing a photograph using a scanner, selecting a section of the scene for forming a standard, contouring and classifying informative details, and generating a reference radar image. Then the radar image is processed in order to extract informative features. Such signs are local brightness gradients. After this procedure, the reference radar image is a set of objects that describe the boundaries of positive and negative brightness changes. The processed reference radar image is transferred to the memory of the onboard computer of the aircraft and is used for correlation comparison with the working radar image.

As a result of processing, a correlation matrix of two images is created. After searching for the maximum value of coincidences of informative objects of the reference and working radar image, the coordinates of the reference point (contour) are estimated, taking into account the errors of the inertial navigation system.

The disadvantage of this method is its low radio interference immunity, since only the radar image is used, and the lack of autonomous operation.

The closest in technical essence analogue of the proposed method is a method of navigating an aircraft on radar images of the earth's surface using digital terrain models [Patent RU2364887C2, IPC G01S 13/90, 20.08.2009, Bull. No. 23], selected for the prototype.

The implementation of this method is as follows. The reference radar image is calculated in the process of movement using digital terrain models prepared in advance on the basis of vector terrain maps, and the assessment of navigation errors of the inertial control system is obtained using the maximum parameters of the two-dimensional cross-correlation function of the reference and working radar image through the position indices of the maximum in the matrix of the cross-correlation function.

However, this method has the following disadvantages: the need for pre-prepared digital models and vector maps of the area, which do not always reflect the actual exposure of objects at the time of the flight of the aircraft, the need for active emission of a probing signal to obtain a radar image during the flight and low noise immunity of the receiving paths of the location system aircraft from directional interference, leading to distortion of the radar image and the appearance of false reference points (contours) or their omission.

The proposed method has a significant difference, which consists in the use of an optoelectronic system in the infrared wavelength range, which provides infrared images when shooting from unmanned aerial vehicles, while the optoelectronic systems are directed strictly down, perpendicular to the astronomical horizon relative to the unmanned aerial vehicle, i.e. e. to nadir, followed by thresholding these images with reference infrared images.

The technical result of the proposed method is to increase the accuracy of determining the navigation parameters of an unmanned aerial vehicle in the conditions of incorrect operation of the receiver of global satellite navigation systems, by carrying out navigation calculations on infrared images obtained by an optoelectronic system from an unmanned aerial vehicle, which are characterized by a high spatial resolution compared to a radar image , and allowing navigation of an unmanned aerial vehicle at any time of the day.

This technical result is achieved by the fact that for autonomous navigation of an unmanned aerial vehicle, reference and working images are obtained using an optoelectronic system in the infrared range. Reference images are prepared on the basis of a thermal terrain model by solving a direct problem of radiative heat transfer, taking into account the flux density of incident solar radiation, air temperature and flight altitude for given areas of the flight trajectory and corresponding periods of astronomical time. Working images are recorded in the process of shooting in nadir using an optoelectronic system in the infrared range. The obtained reference and working infrared images are subjected to threshold processing, as a result of which the reference and working matrices of reference points (contours) of objects are formed. A two-dimensional matrix of the cross-correlation function is calculated, its maximum is found, and the geographic position of the maximum of the cross-correlation function is estimated from the reference matrix of reference points (contours). This estimate of the position of the maximum of the cross-correlation function for autonomous navigation of the unmanned aerial vehicle is used, according to the measured values of the heading, roll, pitch and height of the unmanned aerial vehicle.

The essence of the invention lies in the fact that for the navigation of an unmanned aerial vehicle according to the data of infrared images obtained using an optoelectronic system, preliminary on the basis of digital terrain models, including a thermal terrain model and a matrix of heights, for given points in time, reference images of terrain are formed flight path of an unmanned aerial vehicle.

This procedure is carried out by solving the direct problem of radiative heat transfer based on a thermal terrain model, taking into account the flux density of the incident solar radiation, air temperature and flight altitude. The thermal terrain model is a spatial distribution of thermophysical parameters: thermal conductivity, heat capacity, density and emissivity of objects and backgrounds. The thermal model is obtained by remotely determining the spatial distribution of the thermophysical parameters of the earth's surface [Patent RU2707387C1, IPC G01J 5/00, 11/26/2019].

The reference infrared images calculated in this way are subjected to threshold processing to select reference points (contours) and their informative features - descriptors of reference points (contours), namely the area and shape that will be used for autonomous navigation of an unmanned aerial vehicle.

The need to calculate a set of multi-temporal reference infrared images is determined by the following. An infrared image of a site during the day with variable external meteorological conditions changes its contrast. This phenomenon is called the inversion of thermal contrasts, which occurs depending on weather conditions and is associated with sunrise and sunset. As a result, only those reference matrices of reference points (contours) that correspond to the astronomical time of working matrices of reference points (contours) are subject to comparison.

Figure 1 presents multi-temporal infrared images of the same area, obtained during the day. An analysis of the image data shows that the selected objects change their contrast relative to the background. So the brightness of object A (concrete slab) at 14:00 is less than the background, and at 20:00 of the same day, it is brighter than the background, in turn, for object B (model of automotive equipment), the change in brightness in relation to the background, on the contrary, decreases. Therefore, reference infrared images can be used to compare them with working infrared images only for specified time intervals of astronomical time, and, as a rule, one reference infrared image is used in a time interval not exceeding two hours.

Figure 2 shows a variant of the scheme of the device that implements the proposed method of navigation of aircraft, where:

1 - unmanned aerial vehicle;

1.1 - on-board computer with an accelerometer and a gyroscope;

1.2 - gyro-stabilized platform with an optoelectronic system in the infrared wavelength range;

1.3 - barometric altimeter;

1.4 - receiver of global satellite navigation systems;

1.5 - transmitting and receiving device of an unmanned aerial vehicle;

1.6 - unit for registering the amount of total solar radiation and measuring the temperature of the surface air layer;

2 - ground control point;

2.1 - transceiver device of the ground control station;

2.2 - database of reference images.

Block 1.1 is designed to solve the problem of autonomous navigation of an unmanned aerial vehicle, taking into account the velocity vector according to the current angles of deviation along the course, roll and pitch of the unmanned aerial vehicle, measured using accelerometers and a gyroscope.

Block 1.2 is designed to obtain images of the surface under study in the infrared range. Shooting in the infrared range is carried out in nadir.

Block 1.3 barometric altimeter is designed to accurately measure the flight altitude of an unmanned aerial vehicle and subsequent joint processing of infrared images received from the optoelectronic system with reference infrared images on the same scale.

Block 1.4 provides navigation of an unmanned aerial vehicle according to the signal of global satellite navigation systems. Under conditions of incorrect operation of the receiver of global satellite navigation systems or its failure, a control signal is generated in block 1.1 about the transition to the autonomous navigation mode using infrared images received by the optoelectronic system from an unmanned aerial vehicle.

Block 1.5, the transceiver of an unmanned aerial vehicle, provides the formation of a control channel with a ground control station (block 2).

Block 1.6 for registering the amount of total solar radiation and measuring the temperature of the surface air layer, provides an increase in the accuracy of calculating reference infrared images based on a thermal terrain model, taking into account the flux density of incident solar radiation and air temperature [Patent RU2707387C1, IPC G01J 5/00, 11/26/2019] .

Block 2 provides data transmission (reference infrared images, telemetry data) between the unmanned aerial vehicle (block 1) and the ground control station (block 2).

Block 2.1 is designed to receive telemetry data from an unmanned aerial vehicle and transfer reference infrared images from the database of block 2.2 to the aircraft.

Figure 3 shows the algorithm for determining reference points (contours) and their descriptors with geographic coordinates, comparing the working and reference matrix of reference points (contours) based on the calculation of the cross-correlation function, where:

3 - Threshold processing of reference infrared images, which is carried out using an algorithm consisting in the sequential execution of the following steps:

3.1 - Formation of thermal terrain model;

3.2 - Solving the direct problem of radiative heat transfer based on a thermal terrain model, taking into account the flux density of incident solar radiation and air temperature for given terrain sections of the flight trajectory and time points, obtaining reference infrared images;

3.3 - Median filtering of images;

3.4 - Adaptive image binarization;

3.5 - Filtering images with border closure and formation of a set of reference points (contours) and calculation of their descriptors;

3.6 - Determination of the center of gravity for each reference point (contour);

3.7 - Calculation of the geographical coordinates of the centers of gravity for each reference point (contour);

3.8 - Formation of a database of reference infrared images and their reference matrices of reference points (contours).

4 - threshold processing of working infrared images, which is carried out using an algorithm consisting in the sequential execution of the following steps:

4.1 - Obtaining infrared images from optoelectronic systems of the underlying surface of the terrain in nadir during the flight of an unmanned aerial vehicle;

4.2 - Median filtering of images;

4.3 - Adaptive image binarization;

4.4 - Filtering images with border closure and formation of a set of reference points (contours) and calculation of their descriptors;

4.5 - Determination of the center of gravity for each reference point (contour);

4.6 - Determination of reference points (contours) by the center of gravity of objects;

4.7 - Definition of descriptors of reference points (contours);

4.8 - Obtaining a working matrix of reference points (contours).

5 - Comparison of the working and reference matrix of reference points (contours) based on the calculation of the maximum of the cross-correlation function;

6 - Assignment to the working infrared image of known coordinates of control points (contours) of reference infrared images;

7 - Recording of the combined working matrix of reference points (contours);

8 - Determination of the geographical position of the unmanned aerial vehicle;

9 - Issuance of information for adjusting the route of the unmanned aerial vehicle.

The result of threshold processing of reference infrared images are reference matrices of control points (contours) with their descriptors and geographic coordinates of the center of gravity for each element of the set of control points (contours).

Directly, during the flight of an unmanned aerial vehicle at a measured altitude using a barometric altimeter (block 1.3, Fig. 2), the task of navigation in given areas of the terrain and flight path according to reference infrared images or reference matrices of reference points (contours) is solved by obtaining from the optical-electronic system of working images of the area taken in nadir with their subsequent threshold processing according to the algorithm (blocks 4.1-4.7, Fig 3). As a result of processing by this algorithm of working infrared images in near real time with a given frequency, working matrices of reference points (contours) are obtained (block 4.8, Fig 3). Using the reference and working matrices of reference points (contours) of the same terrain using the on-board computer unit 1.1 (figure 2) according to the algorithm of block 5 (figure 3), a two-dimensional matrix of the cross-correlation function is calculated, the maximum of the cross-correlation function is found, the the geographical position of the maximum of the cross-correlation function according to the reference matrix of reference points (contours) and use this estimate of the position of the maximum of the cross-correlation function for autonomous navigation of the unmanned aerial vehicle, according to the measured values of the heading, roll, pitch and height of the aircraft.

Thus, the trajectory can be selected or corrected directly during the movement of the unmanned aerial vehicle. In block 8 (figure 3) is the definition of the geographical position of the unmanned aerial vehicle. Block 9 (figure 3) provides information to correct the route of the unmanned aerial vehicle.

The implementation of procedures for preliminary and direct processing of infrared images can be carried out in the following ways:

First option. Threshold processing of the reference infrared images is carried out before the start of the flight of the unmanned aerial vehicle, and the already calculated reference matrices of reference points (contours) are loaded on board the unmanned aerial vehicle. Autonomous navigation is implemented in accordance with the algorithm of Fig.3.

Second option. On board the unmanned aerial vehicle, during its flight, using blocks 1.5 and 2.1 (figure 2), reference infrared images are loaded, which, by the onboard computer of block 1.1 (figure 2), are subjected to threshold processing according to the algorithm of figure 3 to obtain reference matrices of reference points (contours). Autonomous navigation is implemented in accordance with the algorithm of Fig.3.

Third option. When navigating an unmanned aerial vehicle according to the data of global satellite navigation systems (block 1.4, Fig. 2) at a given frequency, infrared images of the underlying surface are recorded in nadir using an optical-electronic system and this information is transmitted to the onboard computer (block 1.1, Fig. 2) for threshold processing of these infrared images with known geographical coordinates, then, according to the algorithm of blocks 6 and 7 (figure 3), reference matrices of reference points (contours) are calculated with their subsequent storage in the memory of block 1.1 (figure 2) If the receiver fails global satellite navigation systems (block 1.4, figure 2) or if an unmanned aerial vehicle enters a difficult radio navigation environment that does not allow to correctly determine its location according to the data of global satellite navigation systems, autonomous navigation of an unmanned aerial vehicle is carried out according to those stored in block 1.1 (figure 2 ) reference matrices of reference points (cont levels).

Before obtaining a working matrix of reference points (contours) of the area of the flight path, the procedure for estimating the own speed vector of the unmanned aerial vehicle and its flight altitude above the terrain in the vicinity of the reference points (contours) (block 1.3, figure 2) is performed. This is done in order to reduce the volume of calculations in the digital processing of the working matrix of reference points (contours). Upon completion of the evaluation of the own speed vector of the unmanned aerial vehicle and its height above the terrain in the vicinity of the reference points (contours), the working infrared image is subjected to threshold processing (block 4, figure 3) to obtain reference points (contours) and their geographical coordinates (block 5, figure 3). The coordinates of matching reference points (contours) of both matrixes of reference points (contours) in the geographic coordinate system carry information about the relative position of these infrared images in the horizontal plane. At the moment of mutual alignment of the working and reference matrices of reference points (contours) of one period of astronomical time (blocks 5 and 6, figure 3), the current working matrix of reference points (contours) is recorded in the database of reference matrices of reference points (contours) in block 1.1 (figure 2) and becomes a reference, after which a sequence of threshold filtering for working infrared images is performed with the calculation of the cross-correlation function according to the algorithm of figure 3.

The proposed method can be implemented by a device consisting of units and assemblies of serial production of the industry.

Claims (1)

A method for navigating unmanned aerial vehicles, which consists in obtaining reference and working images of the earth's surface subjected to threshold processing with obtaining a plurality of reference points - contours and estimating the data of the maximum of a two-dimensional cross-correlation function, characterized in that reference and working images are obtained using optoelectronic systems in infrared range, reference images are prepared on the basis of a thermal terrain model by solving a direct problem of radiative heat transfer, taking into account the flux density of incident solar radiation, air temperature and flight altitude for given areas of the flight path and the corresponding periods of astronomical time, working images are obtained in the process of shooting in infrared nadir range, perform threshold processing of reference and working infrared images, obtain reference and working matrices of reference points - contours of objects, calculate a two-dimensional matrix in relation to of the intercorrelation function, find the maximum of the intercorrelation function, estimate the geographic position of the maximum of the intercorrelation function from the reference matrix of reference points - contours, and use this estimate of the position of the maximum of the intercorrelation function for autonomous navigation of unmanned aerial vehicles according to the measured values of the heading, roll, pitch and height of the unmanned aerial vehicle.

Images