RU2782702C1 - Device for supporting object positioning - Google Patents

Abstract

FIELD: object positioning support devices.

SUBSTANCE: object positioning support device comprises at least one light source group block, consisting of a main light source block and two auxiliary light source blocks arranged in a certain way, the color of the radiation of the main source being different from the color of the auxiliary sources.

EFFECT: expansion of the range of environmental conditions for UAV landing is provided.

1 cl, 11 dwg

Description

The technical field to which the invention belongs

The present invention relates to the field of traffic control and, more specifically, to a device for supporting the positioning of an object.

State of the art

Unmanned aerial vehicles (UAVs, UAVs) are gradually becoming a part of everyday life - they perform cargo delivery, various studies, analysis of the surrounding space, object control, photo and video shooting, are used in rescue operations and find many other applications. As is the case with other modes of transport, developments are underway around the world aimed at fully automating the operation of UAVs.

One of the tasks of UAV automation is its autonomous landing. Many different methods are known from the prior art, which can be divided into 2 groups: landing on a prepared surface - for example, on a landing site, and on an unprepared surface - for example, on the ground.

Traditionally, navigation based on the global navigation satellite system (GNSS, GNSS) is used for UAV landing, however, if the task is high-precision landing at a given location and with a given orientation, then navigation directly and only on the basis of GNSS is not enough, since it provides accuracy only within 5-8 meters. One of the solutions to this problem can be the use of differential correction systems, however, in many scenarios of UAV operation, such systems may not be available.

In order to increase accuracy, many well-known solutions use additional navigation methods. Some of these methods use communication based navigation between the landing platform and the UAV, however, they require additional communication equipment both on the landing platform and on board the UAV, which complicates the system and its wide application possibilities.

There are also known methods of additional navigation based on the use of a camera installed on the UAV. The camera in a typical case is a regular on-board tool, therefore, such methods do not require the installation of additional equipment on the UAV. There are many ways in which a black-and-white marker of a square or other shape is placed at the landing site, the camera captures an image, marker recognition is performed, and thereby the exact landing site and / or the position of the camera relative to the marker is determined, which allows for a fairly accurate landing. However, such methods have a number of disadvantages: a high resolution of the camera is required, the marker must be completely captured by the camera, it is not visible from afar and it is not visible in difficult environmental conditions (night, fog, precipitation). To solve this problem, some solutions use light sources.

For example, CN 111506091 A discloses a landing control system in which a high-resolution LED screen (light-emitting diode screen) is placed on the surface of the landing platform, which shows one pattern over the entire area when the UAV is at a high altitude above the platform, and then shows 5 small 2D patterns as the UAV approached the platform. AprilTag are used as patterns. However, such a solution is very expensive and complex structurally and algorithmically, requires agreement between the platform and the UAV regarding which patterns to show at the moment in accordance with the current height of the UAV, and also requires a high screen resolution, which makes it impossible to use high-power light sources. , which significantly impairs the possibility of using it at a long range and in difficult environmental conditions. For correct recognition of such a pattern, it must be visible in its entirety. In order to compromise between the size and brightness of individual LEDs and the overall screen resolution, the screen can be increased, but then it will take up a lot of space and clutter the platform, which makes it difficult to implement it on a small platform with non-planar geometry and difficult to place other equipment on it . All these shortcomings significantly limit the possibility of wide application of such a solution.

Also known is the solution disclosed in document KR 10-1524936 B1, which describes an UAV storage and charging device containing a plurality of landing platforms. At the corners of each platform there are n*n matrices of LED and IR lamps. LED lamps work during the day, and IR lamps work at night. The matrices together form a unique pattern. When the UAV flies close enough to the platform, it detects and recognizes this pattern, determines its location, communicates it to the platform, obtains the optimal landing path and landing clearance from it, and lands. The need to use LED and IR lamps in each cell of the matrix increases the complexity and reduces the reliability of the design, and also reduces the maximum possible brightness of the lamp. In addition, such a solution, like the previous one, requires landing coordination between the platform and the UAV. Specific types of patterns and methods for their recognition are not disclosed in this document. For correct recognition of known patterns, they must be visible in their entirety, that is, the camera must capture all corners of the platform. If only one or two matrices fall into the camera's field of view, or if not all lamps in the matrix are recognized, then uncertainty may arise. If you try to display known patterns in each matrix, similar to the one used in the previous solution, then this will lead to the need to increase the resolution of the matrices, which either reduces the maximum possible brightness of the lamps, or makes each matrix unnecessarily cumbersome.

Thus, there is a need in the prior art to create a structurally simple solution that would allow autonomous landing of UAVs in a wide range of environmental conditions.

The essence of the invention

In order to overcome at least some of the aforementioned shortcomings of the prior art, the present invention is directed to a device for supporting the positioning of an object.

The object positioning support device comprises at least one block of a group of light sources,

each light source group block contains at least one main light source block and at least two auxiliary light source blocks,

the main light source block contains at least one main light source,

each of the blocks of auxiliary light sources contains at least one auxiliary light source,

the emission color of the main light source unit differs from the emission color of the auxiliary light source units,

the exact location of the light sources within the device to support the positioning of the object and their colors are known in advance to the positioned object.

Technical result

The present invention improves the efficiency of devices, systems and methods for autonomous landing of an unmanned aerial vehicle. This provides:

- simplification of the design of the landing platform compared to platforms containing high-resolution screens or complex matrices of multiple light sources;

- simplifying the design of the UAV due to the absence of the need to install additional equipment on it in addition to the camera and the control unit;

- reduced requirements for UAV camera characteristics, such as resolution and viewing angle, compared to solutions using landing platforms with high-resolution screens or complex arrays of multiple light sources;

- simplified integration with a landing platform having a non-planar geometry;

- improving accuracy under difficult environmental conditions;

- reduction of radio noise due to the absence of the need for communication between the UAV and the landing platform;

- the possibility of reducing the size of the landing platform;

- increased accuracy at large angles to the landing platform;

- increased accuracy when only part of the landing platform enters the camera's field of view;

- sufficient accuracy with successful recognition of not all light sources in the group (with an increase in the number of recognized light sources, the accuracy increases);

- increasing the recognition range of the landing platform.

It should be understood that not each of the embodiments can simultaneously provide all of these advantages over all known prior art solutions. Accordingly, some embodiments may have only some of these advantages or other advantages over some known solutions.

These and other advantages of the present invention will become apparent upon reading the following detailed description with reference to the accompanying drawings.

Brief description of the drawings

On FIG. 1 shows the UAV autonomous landing system according to the present invention.

Fig. 2 shows an equivalent circuit of a typical example of the arrangement of light sources on a landing platform.

Fig. 3 illustrates a flowchart of an autonomous landing method.

On FIG. 4 illustrates an example of the result of color segmentation of an image.

On FIG. 5 illustrates an example of grouping auxiliary light sources.

On FIG. 6 illustrates an example of correlating a main light source with a group of auxiliary light sources.

On FIG. Figures 7-9 show examples of landing platform detection in poor lighting conditions at different distances between the UAV and the landing platform.

On FIG. 10 shows a device for supporting object positioning.

On FIG. 11 shows an example of a practical implementation of a device for supporting object positioning.

It should be understood that the figures may be represented schematically and not to scale and are intended primarily to improve understanding of the present invention.

Detailed description

General overview of the landing system

Next, with reference to FIG. 1, an example of a UAV autonomous landing system according to the present invention will be described. It should be noted that this example is not limiting and is only intended to provide a general understanding of the proposed UAV landing and positioning principles. Other examples other than those shown in FIG. 1, but also within the scope of the invention.

The unmanned aerial vehicle (UAV) 110 lands on the landing platform 120 using the object positioning support device 130. The object positioning support device 130 includes 5 groups of light-emitting diodes (LED), each group of LEDs contains 4 LEDs arranged relative to each other in a square shape. 4 groups (131, 132, 133, 134) are located at the corners of the platform and form a square, and the 5th group (135) is located in the center of the square. Dotted lines in Fig. 1 from designation 130 to groups 131, 132, 133, 134, 135 indicate their exemplary lines of communication with a common control unit (not shown) of device 130 to support object positioning.

One of the LEDs in the group is the main one, the rest of the LEDs in the group are auxiliary. Each group of LEDs allows you to uniquely identify the direction to the center of the platform. To do this, the color of the main LED in each group is different from the color of the additional LEDs in that group. The color of the main LED in the central group 135 is different from the color of all other LEDs in all groups. For example, as shown in FIG. 1, the main LED in the central 5th group 135 is red, the main LEDs in the remaining 4 corner groups 131, 132, 133, 134 are green, and the auxiliary LEDs in all 5 groups 131, 132, 133, 134, 135 are Blue colour. The location of the LEDs relative to the landing platform 120 and their colors (or in other words, the pattern) are known in advance to the UAV 110.

UAV 110 uses navigational aids (eg, GPS) to approach the landing area at some altitude, where landing platform 120 should be approximately located. and an onboard autonomous landing control unit. When the camera is operating, all the LEDs of the device 130 to support object positioning are on. During processing of the images captured by the camera, the autonomous landing control unit of the UAV 110 detects at least a portion of at least one group of LEDs of the object positioning support device 130, and using the stored pattern information, determines the location and orientation of the landing platform 120 relative to the UAV 110. Than the more groups of LEDs detected by the UAV 110, the higher the positioning accuracy. The UAV 110 then approaches the landing platform 120 based on the received data, detects as many groups of LEDs of the device 130 as possible to support object positioning, ascertains the location and orientation of the landing platform 120 relative to the UAV 110, and eventually makes an autonomous landing.

In the example shown in FIG. 1, LEDs are used, but in general any other suitable light/optical (visible) source may be used.

Accordingly, the UAV camera 110 must be capable of detecting light/optical radiation. The UAV camera 110 may be a photo and/or video camera. For the purposes of the present invention, the use of a monocular camera is sufficient, but a stereo camera may also be used if desired.

In the example shown in FIG. 1, 4 groups of LEDs are used in the corners of a square and 1 group of LEDs in the center of the square, however other patterns can be used in general. Lights that are in the same group will be referred to as neighboring lights in this document. The group of lights located closest to the center of the landing platform and/or to the center of the pattern will be referred to hereinafter as the central group of lights, and the remaining groups will be called peripheral groups of lights (or corner if they are located at the corners of the conditional polygon) . It should be noted, however, that the presence of a central group of light sources is not necessary to implement the basic principles of the present invention. In addition, the number of light sources in a group may differ both within the same object positioning support device 130 and between different object positioning support devices.

An equivalent circuit of a typical 4x4 example (i.e., 4 groups of 4 LEDs arranged in a square) is shown in FIG. 2. Auxiliary LEDs 201, 202 and 203 (shown in blue) are located on the outer edges (corners) of the group relative to the center of the landing platform, and the main LED 204 (shown in green) is located on the inner edge (corner) of the group, that is, relative to its group it is directed towards the center of the landing platform. The group of LEDs 201, 202, 203 and 204 together form a square with side d _in . More specifically, under d _in in FIG. 2 means the distance between the centers of the LEDs that form the side of the square. The center of the landing platform is conventionally marked as a square 205 (shown in red). For simplicity, the remaining 3 groups are not shown in FIG. 2. For each group of LEDs, the relative vector R _c is known, which starts at the center of the group and ends at the center of the landing platform. Dotted lines in Fig. 2 shows the positions of another, central group of LEDs that could be used to indicate or clarify the orientation of the landing platform. In general, any configuration of lights within the groups and any configuration of the groups relative to the center of the landing platform can be used. Some specific examples with other configurations will be described later in this document.

In the example shown in FIG. 1, the device 130 for maintaining object positioning is part of the landing platform 120, however, in other embodiments, the device 130 may be a separate device that can be installed on the surface of the landing platform 120 or on any other surface on which the UAV 110 is to land. in this document, for convenience of description, it is the embodiment with landing on the landing platform that may be mentioned, but it should be borne in mind that other embodiments are possible.

UAV 110 for the purposes of the present invention can be considered, for example, a drone, a quadrocopter, a multicopter, an unmanned helicopter, a vertical takeoff and landing aircraft (VTOL), or any other aircraft on which a camera and an autonomous control unit are installed or can be installed. landing so that it is capable of autonomous landing.

A landing platform may also be referred to interchangeably as a landing pad, or simply a platform or platform.

Autonomous landing method

The processing performed in the UAV will be described next with reference to FIG. 3 which illustrates a flowchart of the autonomous landing method.

The landing area coordinates are known in advance by the UAV, and in step S310, the UAV may use conventional positioning aids (eg, global navigation satellite system (GNSS) or inertial navigation system) to advance to the landing area coordinates. The accuracy of typical positioning aids may not be sufficient to make a high-precision landing at a given point with a given orientation, so in general, the location where the UAV actually moved in step S310 is only a point above the approximate landing area. To further refine the location and orientation of the UAV uses the detection of the device proposed in the present invention to support the positioning of the object.

In step S320, the camera mounted on the UAV captures an image falling within its field of view. In order for at least one group of light sources of the object positioning support device to fall into the field of view, the camera must capture the approximate landing area where it is located. The larger the camera's field of view, the more likely it is that light sources will enter the field of view. If the UAV camera is fixed (stationary), then to increase the probability of light sources entering the field of view, the UAV can start shooting from a relatively high altitude. If the UAV camera is movable (rotary), then it can be directed to the approximate landing area.

The camera's field of view (i.e., the image) may be exposed to light or glow sources other than the light sources of the object positioning support device. In order to reduce the number of such extraneous light sources and the effect of ambient light, the light sources of the object positioning support device can be used with high intensity sources, and the camera can shoot at a fast shutter speed, such as 1/500, 1/1000. This embodiment is less dependent on time of day and weather conditions. In addition, in one of the embodiments of the invention, the shutter speed of the camera may vary depending on the time of day and weather conditions. Such a dependence can be predefined or adaptive - for example, based on the number of light sources recognized in the further stages in the frame and their quality.

In step S330, the autonomous landing control unit, having received the image from the camera, converts the image color model to HSV (Hue-Saturation-Value, Hue-Saturation-Value, or HSB (Hue-Saturation-Brightness, Hue-Saturation-Brightness)), if the original color model (color space, palette) used by the camera is different from HSV. In particular, there are many palettes in which the camera can shoot natively: RGB, BGR, YUYV, YUY, CMYK, and so on.

The HSV palette is used to perform image segmentation by color, since it is more difficult to perform such an operation in other palettes with high accuracy.

In step S340, the autonomous landing control unit performs auxiliary light source detection. To do this, knowing the color of the auxiliary light sources, the autonomous landing control unit searches for the contours of the auxiliary light sources in the HSV color space using color segmentation. In particular, a search is made for areas of the image whose color falls within a predetermined spectrum interval in the HSV palette, approximately corresponding to the previously known emission spectrum of auxiliary light sources. This approach may be sufficient for many applications.

In another embodiment, the found light sources can be further filtered by the intensity of the radiation - for example, it can be determined whether the intensity of the radiation of a given light source falls within a predetermined intensity interval. This approach can be useful when the surrounding space is relatively noisy in terms of multiple extraneous light sources entering the frame, similar in color to the desired light sources. The predetermined intensity range can only be limited to the lower intensity value - in which case only the least bright sources, which are most likely not the required light sources, will be discarded. The preset intensity interval can also be limited by the upper intensity value - in this case, too bright sources will also be rejected, which with a high degree of probability are not the required light sources.

In yet another embodiment, the desired intensity interval may not be predetermined, but may be determined by the autonomous landing control unit. For example, the intensity distribution density of all found light sources can be determined. Further, the intensity interval in which the distribution density is the highest can be chosen as the required one. In another embodiment, the expected number of light sources may be taken into account, and the intensity interval in which the distribution density is closest to the expected one may be selected as required. For example, the landing platform contains 12 auxiliary light sources, 30 contours of light sources are found in the image with a similar color, but with different intensities; the intensity range is divided into 5 equal intervals, and 12 contours fall into the 4th interval, 13 contours fall into the 1st interval, and the remaining 5 contours are distributed over the remaining three intervals. Since 12 brightest light sources are expected to be found, the 4th interval is chosen as the required one, and the rest are discarded. Thus, the accuracy can be improved.

For each found contour (or in other words, for each found auxiliary light source), the autonomous landing control unit finds the center (for example, by the method of k-means - by the nearest distance).

In step S350, the autonomous landing control unit performs main light source detection. To do this, knowing the color of the main light sources, the autonomous landing control unit searches for the contours of the main light sources in the HSV color space. In particular, a search is made for areas of the image whose color falls within a predetermined spectrum interval in the HSV palette, approximately corresponding to the predetermined emission spectrum of the main light sources.

For each found contour (or in other words, for each found main light source), the autonomous landing control unit finds the center (for example, by the k-means method - by the nearest distance). In this case, if only peripheral groups of light sources are used in the device to support the positioning of an object, then the search for contours and centers is performed only for the main light sources of peripheral groups. If both peripheral and central groups of light sources are used, then the contours and center are also searched for the main source of the central group.

If no light source is found in step S340 or in step S350, then such a frame will be skipped, not taken into account. The method proceeds to step S320 to acquire a new landing area image.

On FIG. 4 illustrates an example of visualizing the result of performing steps S340-S350. In particular, it is shown that a plurality of blue contours/dots, presumably corresponding to the auxiliary light sources (step S340), a plurality of red contours/dots, presumably corresponding to the main peripheral lights (step S350), and one green contour/dot, presumably corresponding to the main to the central light source (step S350). At the same time, translucent red dots are actually noise.

Once the auxiliary light sources and/or main light sources have been found separately, it is necessary to determine how they relate to each other.

To do this, first, at step S360, the autonomous landing control unit performs clustering of the auxiliary light sources (if any were found at step S340), that is, determines whether each auxiliary light source belongs to one or another group. In particular, the k-means method is used. In this case, as is known, for the k-means method, it is required to pre-specify the required number of clusters into which the found set of points should be partitioned. The present invention proposes to use the following approach:

where N is an integer number of groups (clusters) rounded up,

M _al is the total number of auxiliary lights found in the frame,

a is the number of auxiliary lights in each group.

группы. В другом примере, если бы в каждой группе использовалось по 5 вспомогательных источников света, то число групп при этом составило бы N=10/5=2 группы.For example, if 10 auxiliary lights are found in step S340, and 3 auxiliary lights are used in each group, the number of groups is

groups. In another example, if 5 auxiliary light sources were used in each group, then the number of groups would be N=10/5=2 groups.

It should be noted that the principle of determining the number N of groups according to formula (1) is actually a first approximation, a hypothesis that often turns out to be true due to the features of the proposed method. As a hypothesis test, the distance between adjacent lights in the group can be determined and compared with a predetermined interval from the minimum distance to the maximum distance between the lights in the group. For example, if initially according to formula (1) it is determined that the detected 4 light sources should be divided into N = 2 groups, and when checking it turned out that in the 2nd group the distance between adjacent light sources does not fall within a predetermined interval from 5 to 15 cm, then the method can proceed to the hypothesis of N=3 groups (obtained by an increment to the previously calculated value of N=2). In another embodiment, those light sources that do not fall within a predetermined interval may simply be ignored (discarded), and the initially determined number of groups may be retained.

Accordingly, the autonomous landing control unit performs k-means clustering of the auxiliary light sources into a certain number of groups.

On FIG. 5 illustrates an example of grouping auxiliary light sources. In particular, it is shown that using the k-means method, it was revealed that the auxiliary light sources found earlier (see Fig. 4) belong to 4 different groups.

Then, in step S370, the autonomous landing control unit determines the distances between the groups of auxiliary light sources and the found main lights (if any were found in step S350). In particular, the coordinates of the center of each found group are calculated and the matrix D of distances between the centers of all found groups and the centers of all found main light sources is calculated.

where C _i are the coordinates of the center of the i -th group (cluster),

P _j are the coordinates of the center of the j -th main light source,

N is the number of groups

K is the number of found main light sources.

Next, in step S380, the autonomous landing control unit determines correspondences between groups of auxiliary light sources and main light sources. Specifically, in the distance matrix D calculated in step S370, in each i th row, the minimum element min(D _i,j ) is determined, or in other words, in each i th group, the minimum distance d _min =min(D _{i ,j} ) to the main light source. The index j of this element D _i,j indicates that the j -th main light source from the K found main light sources corresponds to the i -th group of N groups (is in it, belongs to it).

On FIG. 6 illustrates an example of correlating a main light source with a group of auxiliary light sources. In particular, it is shown that by performing steps S370 to S380 according to the principle of minimum distance d _min , one of the main peripheral light sources (red) found in step S350 is compared with one of the auxiliary light source groups found in step S360. At the same time, the probability of an extraneous light source in the frame, falsely recognized as the main one, located even closer to the auxiliary light sources of one group than the true main light source of this group, is extremely small, since this can only happen if an extraneous light source with intensity luminosity and with a color (spectrum) that is similar to the desired main light source, is located either on the landing platform itself inside the perimeter of the group (which is practically excluded due to the fact that the configuration of the landing platform and the device to support the positioning of the object is controlled by the person using this invention), either in the space between the landing platform/object positioning support device (i.e. is a fixed overhead light source or is on an aircraft - both are unlikely). Thus, the approach with the search for the minimum distance d _min is relatively simple and at the same time highly accurate.

Thereafter, in step S390, the autonomous landing control unit determines the location and orientation of the landing area (eg, landing platform) relative to the UAV.

In particular, for each i -th group, the Perspective-n-Point (PnP) problem is solved, that is, the problem of determining the orientation and location of the camera in space from the image obtained with its help, by obtaining a perspective projection of points in the image onto the camera sensor plane. As is known, the condition for solving such a problem is n correspondences between the points of the image and the points of the object model, as well as the known relative position of the points of the object model. In this case, a well-known object model for the UAV is a device for supporting the positioning of the object and the arrangement of groups and light sources on it.

Namely, the location P _i of the center of the landing area and the orientation R _i of the center of the landing area are calculated (as mentioned above, R _i is a relative vector indicating the direction from the i th group to the center of the landing area). Specific methods for solving the PnP problem are known to those skilled in the art and are not disclosed here in more detail. As a non-limiting example, the algorithm disclosed in Y. Wu and Z. Hu, “Pnp problem revisited,” Journal of Mathematical Imaging and Vision, vol. 24, no. 1, pp. 131–141, 2006.

When the values of location P _i and orientation R _i for each i -th group are obtained, by calculating the median of these values, the refined location P and orientation R of the center of the landing area are determined:

where N is the number of groups.

Then, the autonomous landing control unit controls the UAV actuators by itself, or sends the location and orientation data of the landing area determined in step S390 to the UAV flight controller so that the UAV approaches the landing area based on the received data. Thus, the autonomous landing method further comprises step S395 (not shown in FIG. 3) in which the UAV performs movement towards the landing area.

As it approaches, when the UAV has made some next movement towards the landing area, the UAV may again repeat steps S320-S395, surveying the landing area, detecting as many light sources as possible, refining the location and orientation of the landing area relative to the UAV, and moving closer to the area. landing. Ultimately, after completing the required number of iterations, the UAV makes an autonomous landing with increased accuracy at a given point and with a given orientation, even in conditions of poor visibility.

The proposed method has been tested in practice for landing a UAV in autonomous mode on a landing platform. The landing platform used as a sample has a square shape 40x40 cm in size, the UAV beam span is comparable in size, the groups of LEDs are located at the corners of the platform at a distance of 30 cm from each other. WS2812B addressable diodes were used as light sources. The LEDs in the group are arranged so that they form a square, while the distance between them (the side of the square) is 5 cm. The UAV successfully lands at a given point with a given orientation. The positioning accuracy is approximately 1-7 cm depending on the distance between the UAV and the landing platform - the closer the UAV is to the landing platform and the more groups of light sources detected, the higher the accuracy.

Examples of detection of the landing platform in poor lighting conditions at different distances between the UAV and the landing platform are shown in Fig. 7-9. In particular, in FIG. 7 shows the case at a distance of 2 m, FIG. 8 - 5 m, in Fig. 9 - 10 m.

As seen in FIG. 7-9, the autonomous landing control unit successfully copes with the task of detecting at least some groups of LEDs, despite the night time. Meanwhile, in FIG. 7 shows that the landing platform is located at a sufficiently large angle relative to the UAV camera, but this did not prevent the detection of LED groups. Due to the features of the proposed method, even when not all groups could be found, the center of the landing platform is determined successfully.

A stationary typical camera was used on the test UAV, and even taking into account the relatively small dimensions of the UAV and the unremarkable characteristics of the camera itself, the proposed method made it possible to successfully detect the landing platform for autonomous landing already at a height of 20 m in poor visibility conditions. The use of a higher resolution camera would increase the range/altitude at which the proposed method could be effectively performed.

In fact, in the proposed method, in order to determine the location of the landing area, it is sufficient to detect at least one group of light sources. If more than one group of light sources is detected in the field of view of the UAV camera, the positioning accuracy increases. Also, the positioning accuracy increases with an increase in the number of groups of light sources on the landing platform and with an improvement in camera characteristics (resolution, aperture ratio, matrix type, optics quality, computing capabilities, data processing algorithms used, etc.).

In addition, taking into account the dimensions of the test sample mentioned above, even the size of a group of light sources of 5x5 cm turned out to be sufficient for autonomous landing of the UAV on the landing platform in poor visibility conditions, that is, the proposed method can be used even if there is little free space.

Moreover, it is possible to use a landing platform having a complex (non-planar) geometry, because it is easier to mount small separate independent groups of light sources on it than mounting a single relatively large flat pattern / LED array.

It was mentioned above that for the purposes of the present invention, if necessary, a camera with two lenses instead of one can be used. The use of a stereo camera will improve the accuracy of the proposed method due to the availability of additional data that can, for example, be averaged, compared, processed independently, selected the best, etc.

Configuration options

It has also been mentioned above that, in general, any configuration of light sources within groups and any configuration of groups relative to the center of the landing platform can be used.

In one embodiment, the configuration may be a 3x3 triangle pattern. The light sources in each group are located relative to each other to form a triangle, one of the vertices of which is directed towards the center of the landing platform and contains the main light source.

In another embodiment, the configuration may include one or more groups of light sources arranged relative to each other to form a circle or arc.

In other embodiments, the implementation of the configuration may include groups of light sources that form other geometric objects (shapes, polylines, curves, line segments).

The basic principle is that in each group there are auxiliary lights emitting in one color and a main light source emitting in another color. The relative positions and colors of the light sources known in advance to the UAV make it possible to determine the location and orientation of the landing area.

The above method for autonomous landing of the UAV provides for determining the orientation of the landing platform. It should be noted that in some applications it is sufficient to use configurations that allow only the direction of the angles of the landing platform to be determined without accurately determining the orientation of the landing platform as a whole. For example, this applies to the case where a multi-beam UAV is used, and it does not matter which of the beams will be in which corner of the landing platform when landing.

If it is necessary to land with a more precise orientation, then configurations of light sources can be used to accurately determine the orientation of the landing platform relative to the UAV. For this, for example, a central group of light sources may be used, or the color of the main light source or auxiliary light sources in one peripheral group may differ, respectively, from the color of the main light sources or auxiliary light sources in other peripheral groups, or the color of one or more auxiliary light sources may differ from the color of other auxiliary lights within the same group, or the relative position of the groups relative to each other may be asymmetrical about the center of the landing area, or the location or colors of the main lights or auxiliary lights may be asymmetrical about the center of the area landings, etc.

Next, the autonomous landing device according to the present invention will be described.

It should be understood that this device generally corresponds to the autonomous landing control unit described above and performs its functions in the method 200, and if any information is not disclosed in relation to the device, but is disclosed in relation to the method, and vice versa, then this does not imply that in device or method, this function or step cannot be performed, but is done only in order not to clutter up the description with repetition of details.

The autonomous landing device comprises a processor and a memory coupled to the processor and containing instructions that cause the processor to perform the following operations:

send a command to the camera to capture an image when the UAV has moved to the landing area, as described in relation to steps S310-S320;

receive an image of the landing area from the camera;

convert, as described in step S330, the color space of the image to the HSV color space;

detect in the image, as described in relation to step S340, at least one auxiliary light source located in the landing area, using color segmentation in the HSV color space;

to detect in the image, as described in relation to step S350, at least one main light source located in the landing area, using color segmentation in the HSV color space;

form on the image, as described in relation to the step steps S360-S380, groups of detected light sources;

determine, as described in relation to step S390, the location and orientation of the landing area by solving the PnP task; and

control the actuators of the UAV so that the UAV approaches the landing area based on the determined location and orientation of the landing area.

The device for autonomous landing can be located on the UAV itself - in this case, the complete autonomy of the UAV is ensured due to the absence of the need to communicate with the landing platform.

In another embodiment, the autonomous landing device may be external to the UAV. In such a case, the device for autonomous landing further comprises a wireless communication unit for communicating with the UAV to receive a message from the UAV about the movement to the landing area, send an image capture command to the camera, receive images from the camera installed on the UAV, and send to UAV commands for exerting a control effect on the actuators of the UAV. In this case, the UAV is released from the computational load associated with determining the location and orientation of the landing area and the formation of control actions on the actuators. In some cases, this can reduce the power consumption of the UAV and reduce the performance requirements for UAV computing modules. The characteristics of the UAV camera and its exact location relative to the UAV are either known in advance to the device for autonomous landing, or transmitted to it from the UAV in a corresponding message.

In yet another embodiment, the autonomous landing device may itself comprise a camera. The camera, processor and memory of the autonomous landing device can be located in a single housing. This is especially useful if the UAV does not include a camera but is required to perform an autonomous landing according to the present invention. In this embodiment, the device can be completely mounted on the UAV and connected by wire or wirelessly, for example, to its flight controller in order to receive from it a message about the movement to the landing area and send a control action to the UAV actuators.

Next, with reference to FIG. 10, a device for supporting object positioning will be described in more detail.

The object positioning support device comprises at least one light source group unit 1000.

Each light source group block 1000 includes a main light source block 1010, auxiliary light source blocks 1020, 1030, 1040, connectors 1050 and 1060, and a control block 1070.

Block 1010 of the main light source contains at least one main light source. In a particular example, this may be a single LED or several closely spaced LEDs forming a single light source (eg, an LED array or LED strip).

Each of the blocks 1020, 1030, 1040 auxiliary light sources contains at least one auxiliary light source. In a particular example, this may be a single LED or several closely spaced LEDs forming a single light source (eg, an LED array or LED strip). In general, the number of auxiliary light source blocks is two or more.

The emission color of the main light source block 1010 is different from the emission color of the auxiliary light source blocks 1020, 1030, 1040.

The emission color of each of the auxiliary light units 1020, 1030, 1040 may be different if necessary (for example, to increase the likelihood of determining the direction to the center of the landing area even if the contours of a small number of light sources are found), but is preferably the same, since this simplifies the design and can increase the speed of recognition.

The exact location of the lights in the landing area and their colors are known in advance to the autonomous landing device so that the PnP task for the camera located on the UAV can be successfully solved.

In various embodiments, the light source blocks 1010, 1020, 1030, 1040 in each light source group block 1000 are arranged relative to each other to form a triangle, square, circle, arc, other geometric objects (shapes, polylines, curves, line segments), or a combination thereof. .

Also, the device for supporting the positioning of the object as a whole may contain several blocks 1000 of a group of light sources - for example, 3 blocks 1000 arranged in a triangle, 4 blocks 1000 arranged in a square, 5 blocks 1000, of which 4 are located in a square, and 1 is in the center, etc.

The light source group unit 1000 may optionally have its own power supply in order to provide increased autonomy and mobility of the device to support object positioning. In another embodiment, the light source group unit 1000 is powered from an external power source. In this case, the power source external to the block 1000 of the group of light sources may still be part of the device for supporting object positioning, that is, it may contain one power source for all individual blocks 1000 included in the device. In another embodiment, the power may be external and in relation to the device to support the positioning of the object. This may slightly increase power consumption, but allows the use of more powerful light sources, resulting in improved recognition, positioning accuracy, and landing accuracy.

The control unit 1070 is optional. If each light source group unit 1000 has its own control unit, an external control unit may not be required, and the versatility and compactness of the apparatus for supporting object positioning is increased. In another embodiment, the object positioning support device may comprise one control unit for all separate units 1000 included in the device.

Connectors 1050 and 1060 are optional, they may be required to receive power from an external power supply, to receive external control, etc. The exact number and configuration of connectors depends on the requirements of the particular application.

The light source group block 1000 also contains various circuit components (not shown in FIG. 10) such as resistors, capacitors, etc., to provide the necessary operation of the light sources and other components of the block 1000.

All components of one light source block 1000 can be placed on one printed circuit board, as shown in the example in FIG. 11. By connecting different units 1000 of a group of light sources with each other using connectors, you can connect them in a series circuit, which reduces the number of wires and simplifies installation compared to connecting a star from a power source and a single control unit. At the same time, each individual block 1000 of the group of light sources still remains independent and compact, which allows the proposed device to be used to support object positioning on landing platforms with different surfaces, including non-flat ones.

In another embodiment, all blocks 1000 group of light sources can be placed on a single printed circuit board. This design is rigid and does not require measurements and calibration upon installation or high accuracy during installation, since all dimensions and relative positions of light sources are unchanged.

Application

Devices, systems and methods according to the present invention can be used for autonomous landing of UAVs, performing, in particular, the delivery of goods, various studies, analysis of the surrounding space, object control, photography and video filming, rescue operations, autonomous monitoring for the tasks of protecting farmland, cartography, remote chemical and physical analysis, control of germination and ripeness of the crop, chemical treatment, etc.

Claims (7)

An object positioning support device, comprising: at least one block of a group of light sources, wherein each light source group block contains at least one main light source block and at least two auxiliary light source blocks, the main light source block contains at least one main light source, each of the blocks of auxiliary light sources contains at least one auxiliary light source, the emission color of the main light source unit differs from the emission color of the auxiliary light source units, the exact location of the light sources within the device to support the positioning of the object and their colors are known in advance to the positioned object.

Images