CZ36280U1 - Space mapping system - Google Patents

Description

In the registration procedure, the Industrial Property Office does not determine whether the subject of the utility model meets the conditions of eligibility for protection according to § 1 of Act. E. 478/1992 Coll.

CZ 36280 UI

Space mapping system

Field of technology

The technical solution relates to a system for space mapping, in particular for controlling an unmanned aerial vehicle so that it can move autonomously in closed spaces without a GPS signal. Thanks to the designed sensor system and software, based on input instructions, the implemented autopilot fulfills the set missions, that is, systematic mapping of the space, flying through the space along the optimal route, target detection and moving to the specified coordinates, etc. The application of such a solution is mainly in hard-to-reach, dangerous or unexplored areas.

Current state of the art

Within the framework of autopilot systems for unmanned aerial vehicles, there are solutions that focus mostly on the speed and maneuverability of the drone, i.e. the unmanned aerial vehicle, in good lighting conditions.

The essence of the technical solution

The goal of this technical solution is to achieve the autonomous movement of the drone in closed spaces even in low light conditions using IR cameras and lidars. Emphasis is also placed on the universality of the system in terms of the possibility of connecting external sensors, which expand the functionality of the system (thermal cameras, surveillance cameras, infrared cameras, etc.) and the usability of the system depending on the type of mission.

The above objective is achieved by a space mapping system that includes a server that includes:

- processor,

- the first unit for wireless communication, a

- a memory unit that is communicatively connected to each other and an unmanned aerial vehicle that includes:

- at least one RGBD camera,

- a second unit for wireless communication intended for communication with the first unit for wireless communication,

- an autopilot unit for controlling the movement of an unmanned aerial vehicle, which includes a first system of sensors, which includes an inertial measurement unit for measuring the acceleration and orientation of an unmanned aerial vehicle in space,

- the processing unit, which is connected by communication with the first system of sensors and with the RGBD type camera, and

- an engine for driving an unmanned aerial vehicle.

Advantageously, the unmanned aerial vehicle includes an RGB/IR camera that is communicatively connected to the processing unit and/or a surveillance camera that is communicatively connected to the processing unit.

The autopilot unit preferably includes a first sensor array that includes said inertial measurement unit as well as a barometer and/or magnetometer.

- 1 CZ 36280 UI

Advantageously, the system also includes a second sensor system that is communicatively connected to the autopilot unit and that includes a lidar system and/or a GPS receiver.

It is also advantageous if the system further comprises at least one accumulator for driving the engine of the unmanned aerial vehicle and for powering the processing unit and the autopilot unit.

The processing unit preferably contains

- the first module for data acquisition, which is communicatively connected to the autopilot unit and to the first system of sensors,

- SLAM module, which is connected by communication with the first module for data acquisition,

- the mapping control module, which is connected by communication with the SLAM module to receive data from the SLAM module,

- visualization and input unit, which is connected by communication with the mapping control module and intended for user visualization of a 3D map from data from the mapping control module, and

- the autopilot module, which is connected by communication with the mapping control module and the autopilot unit.

Clarification of drawings

The technical solution is further described using an exemplary embodiment, which is shown in the drawings, where Fig. 1 shows the hardware part of an exemplary embodiment of the system according to this technical solution and Fig. 2 shows a diagram of an exemplary embodiment of the software equipment of the system from Fig. 1.

Examples of implementing a technical solution

An exemplary embodiment of the assembly according to this technical solution includes a server 1 and an unmanned aerial vehicle 2 (UAV).

Server 1 contains processor 1.3. memory unit 1.2. in which the database and related software application are stored, and the first unit 1.1 for wireless communication. Furthermore, the server 1 is equipped with an input interface not shown, which can be, for example, a keyboard or a touch screen.

The memory unit 1.2 also stores a database for providing parameters for flight configuration and the autopilot module 3,4, and the Rest API, which is a specific API (Application Programming Interface) architecture, which to the standard API function, i.e. the interface function for transferring data between two or more applications, it provides, for example, the ability to access application data from multiple environments or to provide data to other users.

The unmanned aerial vehicle 2 is provided with at least one engine 2.5. which is electrically connected to at least one accumulator 2.7 and communicationally connected to the autopilot unit 2.3. such as the PixhawkCube Autopilot type.

The autopilot unit 2.3 is equipped with a first set of sensors 2.31, which includes a barometer 2.311, a magnetometer 2.312 and an inertial measurement unit 2.313 (IMU), which, using a combination of gyroscopes and accelerators, provides information about acceleration and orientation in space.

A second sensor system 2.6 is also connected to the autopilot unit 23, which includes a lidar system 2.61 containing at least one lidar, for example a TeraBee Evo 60m type, and a GPS receiver 2.62 and which is communicationally connected to the autopilot unit 23. Lidar (light Detection and Raging) is a device that determines the distance of surrounding objects by emitting a laser beam and measuring the time it takes for the reflected light to travel back from said object.

-2CZ 36280 UI

The lidar system 2.61 is preferably arranged such that the individual lidars are arranged on a circle with equal mutual spacing facing radially outward.

In addition to distance measurement, lidar can also be used, for example, for the analysis of atmospheric phenomena or terrain mapping.

The unmanned aerial vehicle 2 is further equipped with a processing unit 2.1. which is preferably a supercomputer for artificial intelligence. Such a supercomputer includes a system of interconnected graphics processors. A good example of a 2.1 processing unit is the Jetson Xavier NX.

The processing unit 2, is connected by communication with the autopilot unit 2, 3, for example via a universal serial bus.

Furthermore, a 2.4 camera system is attached to the unmanned aerial vehicle 2, which includes at least a 2.41 camera of the RGBD type, i.e. a camera for high-resolution color and depth sensing. For example, an Intel D455 camera can be used. Optionally, a 2.42 RGB/IR camera (e.g. FlirDuo camera) and a 2.43 surveillance camera (e.g. Intel T265 camera) can also be included in the 2.4 camera system. The 2.42 RGB/IR camera is not essential to the control of the UAV 2, but can be used for special purposes during mission performance (more details below). The surveillance camera 2.43 primarily increases the robustness of the autonomous control system and makes it easier for the algorithms used to navigate in space.

In addition, the unmanned aerial vehicle 2 is equipped with a second unit 2.2 for wireless communication, which is adapted for communication with the first unit 1.1 for wireless communication and is preferably adapted for communication in the 4G LTE standard. The second unit 2.2 for wireless communication is communicatively connected to the processing unit 2.1.

The communication between the processing unit 2.1 and the autopilot unit 2.3, on the other hand, is realized by means of USB via the MAVLink communication protocol, used for communication with smaller unmanned aerial vehicles.

Lighting 2.8 can also be connected to the accumulator 2.7, in the form of IR or conventional lighting, illuminating the environment in the direction of view of at least some of the camera system 2.4.

The assembly of 3 modules shown in Fig. 2 is part of the processing unit 2, 1 and contains modules that enable the execution of software applications. Data from the first system 2.31 of sensors and the second system 2.6 of sensors are sent to the first module 3.1 for data acquisition, on which the relevant application runs or is executable, preferably in the Python programming language. This data is processed together with the data from the autopilot module 3,4. The first data acquisition module 3.1 is thus adapted to regularly receive synchronized data from the available sensors of the unmanned aerial vehicle 2, the data being taken via a specific dedicated port, not shown, which is part of the processing unit 2.1.

The output of the first module 3.1 is information data, i.e. telemetry from the ardupilot, e.g. GPS or YawPitchRoll (roll, pitch and roll), and additional data from the autopilot script such as droneState (status of the unmanned aerial vehicle 2). For droneState, the following states are considered together with the time stamp: initializing - UAV 2 performs the necessary initialization movements before flight without GPS, ready - UAV 2 is ready to fly and is not moving, moving - UAV 2 is moving, reached - unmanned aerial vehicle 2 reached the waypoint (target destination), and stopped - unmanned aerial vehicle 2 stopped based on lidar or loss of connection with the autopilot.

CZ 36280 UI

The processed data is sent via the first TCP port 3.9 to the SLAM module 3.2 and to the mapping control module 33. The TCP port 3.9 is a port adapted for communication using the TCP protocol.

The SLAM module 3.2 is connected and adapted for loading data and settings from a shared database and for receiving data from the first module 3.1 for data acquisition from the RGBD type camera 2.41 and also for continuously sending a pre-processed point cloud (point cloud) from 2.41 RGBD cameras, updated UAV 2 location salient maps and obtained relative positions of UAV 2 based on the image from the 2.42 RGBD/IR camera.

Part of the SLAM module 3.2 is a second data acquisition module 3.21, which receives and processes data from the RGBD type camera 2.41. The second module 3.21 is run to minimize data transfer latency. The data processed by the SLAM module 3.2 is sent via the second TCP port 3.6 to the mapping control module 33, which together with the SLAM module 3.2 works preferably in the C++ programming language.

It follows from the above that the mapping control module 33 receives data both from the first module 3.1 for data acquisition and also from the SLAM module 32. The mapping control module 33 is adapted for a complex solution of autonomous control of the unmanned aerial vehicle 2 based on e.g. input data from the user, defining the mission, directly obtained sensor data (e.g. maintaining distance from an obstacle, signaled by a lidar) or the calculated position of the unmanned aerial vehicle 2, obtained from the SLAM module 32. Advantageously, the mapping control module 33 works in the C++ programming language.

Part of the function of the mapping control module 33 is also the creation and updating of a 3D map, created from a cloud of points from a 2.41 RGBD camera and information obtained in the route planning process. The output from the mapping control module 33 is control data, i.e. instructions for controlling the engine 23 of the unmanned aerial vehicle 2. The control data is sent to the autopilot module 3.4 via the fourth TCP port 3.8. in which the data from the mapping control module 33 is interpreted or further processed, preferably in the Python language. The processed data is transmitted to the autopilot unit 23.

The control data from the mapping control module 33 is also sent via the third TCP port 3,7 to the visualization and input module 33, which enables the user to visualize a 3D map of the scanned environment with the position of the unmanned aerial vehicle 2 with a graphical user interface (GUI) for system settings and input mission, etc. The visualization and input module 33 preferably works in the C++ programming language, the software application can be run on a client device with access to the system's local network. From the visualization and input module 33, the configuration data is sent back to the mapping control module 33.

An example of the use of a system for controlling an autonomous unmanned aerial vehicle 2 can be the following rescue mission and instructions for an unmanned aerial vehicle 2: Fly and map the object, if you detect a person along the way, save their position within the created map and return to the coordinates of the starting position by the shortest route.

Although particularly advantageous exemplary embodiments of this technical solution have been described, it is clear that a person skilled in the art will easily find other possible alternatives to these embodiments. Therefore, the scope of protection is not limited to these exemplary embodiments, but rather is given by the definition of the appended claims for protection.

Claims (6)

1. A system for space mapping, characterized in that it includes a server (1) that includes: - processor (1.3), - the first unit (1.1) for wireless communication, a - memory unit (1.2), which are interconnected by communication; and an unmanned aerial vehicle (2) that includes: - at least one camera (2.41) RGBD type, - the second unit (2.2) for wireless communication intended for communication with the first unit (1.1) for wireless communication, - an autopilot unit (2.3) for controlling the movement of the unmanned aerial vehicle (2), which includes a first system (2.31) of sensors, which includes an inertial measurement unit (2.313) for measuring the acceleration and orientation of the unmanned aerial vehicle (2) in space, - the processing unit (2.1), which is communicatively connected to the first system (2.31) of sensors and to the camera (2.41) of the RGBD type, and - engine (2.5) for driving an unmanned aerial vehicle (2). 2. The system according to claim 1, characterized in that the unmanned aerial vehicle (2) contains a camera (2.42) of the RGB/IR type, which is communicatively connected to the processing unit (2.1), and/or a surveillance camera (2.43), which is communicationally connected to the processing unit (2.1). 3. System according to claim 1 or 2, characterized in that the autopilot unit (2.3) contains a first system (2.31) of sensors, which includes an inertial measurement unit (2.313) as well as a barometer (2.311) and/or a magnetometer (2.312). 4. The system according to any one of claims 1 to 3, characterized in that it further comprises a second system (2.6) of sensors which is communicatively connected to the autopilot unit (2.3) and which contains a system (2.61) of lidars and/or a GPS receiver (2.62) ). 5. The system according to any one of claims 1 to 4, characterized in that it further comprises at least one accumulator (2.7) for driving the engine (2.5) of the unmanned aerial vehicle (2) and for powering the processing unit (2.1) and the autopilot unit (2.3). . 6. System according to any one of claims 3 to 5, characterized in that the processing unit (2.1) contains - the first module (3.1) for data acquisition, which is communicatively connected to the autopilot unit (2.3) and to the first system (2.31) of sensors, - SLAM module (3.2), which is connected by communication with the first module (3.1) for data acquisition, -5 CZ 36280 UI - the mapping control module (3.3), which is connected by communication with the SLAM module (3.2) to receive data from the SLAM module (3.2), - the visualization and input unit (3.5), which is connected by communication with the mapping control module (3.3) and intended for user visualization of a 3D map from data from the mapping control module 5 (3.3), a - the autopilot module (3.4), which is connected by communication with the mapping control module (3.3) and with the autopilot unit (2.3).