KR20230100887A - Integrated mission flight control system for AI-based autonomous flight drones - Google Patents

Abstract

The present invention relates to an integrated mission flight control system for artificial intelligence-based autonomous flight, and in particular, functions that could not be provided by existing firmware-level system boards are configured as a system capable of loading operating software, thereby providing various functions necessary for autonomous flight and meeting a wide range of requirements. It can be used so that it can be mounted, and various sensors are linked to a small embedded board, and information is calculated and transmitted to the flight control device.
The present invention secures stable flight by implementing flight control by collecting information from various sensor modules linked to a system board equipped with an integrated FCC for autonomous drones applicable to unmanned aerial vehicles for various purposes such as filming and monitoring, and implementing flight control. It is possible to perform missions in conjunction, and drone application development companies can efficiently manage when using this technology, thereby reducing development time and cost.

Description

Integrated mission flight control system for AI-based autonomous flight drones}

The present invention relates to an artificial intelligence (AI)-based integrated mission flight control system for autonomous flight. and a wide range of requirements can be mounted and used easily, and various sensors can be linked to a small embedded board, and information can be calculated and transmitted to the flight controller.

In general, drones are in the limelight in various fields such as military and industrial fields in that they can easily collect information on the ground and in the air at high altitudes without risk to users and without risk of being exposed to others.

Recently, it is expressed as an unmanned aerial vehicle system to emphasize that it is an integrated system instead of an unmanned aerial vehicle having a platform-oriented meaning. It consists of control equipment designed to maneuver and control the aircraft by communication, mission equipment mounted on the unmanned aerial vehicle for missions such as sensors, and support equipment used for analysis and maintenance necessary for the operation of the unmanned aerial vehicle. It is a device operated in one system.

Drones are different from radio-controlled airplanes, which are directly controlled by external pilots, in that they can fly autonomously. There is a difference in what can be.

Today's drones have a very high level of freedom to measure their position, speed, and attitude, create the optimal route for a given mission, and fly along it, diagnosing and responding to failures on their own.

Recently, satellite navigation systems and sensors. Civilian drones equipped with cameras have been developed to transport goods. traffic control. The range of use is expanding to areas such as security.

Therefore, although worldwide attention is focused on drones, users are still required to have extensive knowledge about drones, and a lot of training time is required for delicate control with a controller.

Conventionally, when a user makes a decision on an emergency situation, there is a high risk of an accident due to failure to properly respond to an emergency situation.

That is, conventionally, since flight safety is unstable because it relies only on commercial flight control devices, it is necessary to break away from the existing method and present a new model for autonomous flight as safety accidents occur frequently.

Patent Application No. 10-2015-0044386 Patent Application No. 10-2016-0004953 Patent Application No. 10-2017-0025045

The present invention is to solve the above conventional drawbacks, and an object of the present invention is to configure a system capable of loading operating software with functions that could not be provided by existing firmware-level system boards, thereby providing various functionalization and breadth required for autonomous flight. It is to provide an integrated mission flight control system for artificial intelligence-based autonomous drones that can be loaded and used with a wide range of requirements.

In addition, the present invention provides an integrated mission flight control system for artificial intelligence-based autonomous drones that can link various sensors to a small embedded (control computer built to drive a machine) board, calculate information, and deliver it to a flight control device. are doing

The present invention for achieving the above object is a CPU for controlling each part, a DC-DC step-down regulator that provides 33V DC output at 15A required for specific components, a USB type photodiode, and a USB for the board. There are first and second USB-UART converters that use a virtual RS232 serial connection through the connection and provide direct access to the system, a connector for interfacing with the RPCM4 device on the computer module, and a hardware-controlled UART line. A HART-HART bridge that interfaces with the two modules, a PWM header that controls servo motors and other PWM devices, and a 15-pin ribbon connector that exposes the 2-lane MIPI camera system to an external high-resolution camera module, Raspberry Pi 1st and 2nd Vertical camera connector module, GPS with up to 6 pins that can be used at the customer's discretion with a custom 6-pin header, 3 interfaces for on-board USB customer devices, Microchip USB 20 high-speed hub controller A 3-port USB customer hub module, which is a USB device port used, a custom 4-pin header that provides up to 4 pins and a custom 5-pin header that provides 5 pins, and a small ASIC that accelerates the Tenso Flow Lite model using low power A transceiver that provides a network protocol controller and a physical two-wire network bus interface between the acceleration module and the controller area, a touch switch that provides user input for CPU signals, and a removable non-volatile memory, SD card of the CPU It is characterized by comprising a card slot module connected to, a USB 2 port switch sharing a single USB 20 port, and first and second USB bridges connecting the CPU to the USB 2 port switch as a USB device.

In the present invention as described above, when drone application development companies utilize this technology, they can independently develop and reuse the technology for efficient management, thereby reducing development time and cost.

In addition, the present invention configures a system capable of loading operating software with functions that could not be provided by existing firmware-level system boards, so that various functionalization and wide-ranging requirements necessary for autonomous flight can be loaded, and various sensors can be installed on a small embedded board. It has the effect of being able to control flight by interlocking and calculating information.

1 is a block diagram of an integrated mission flight control system for artificial intelligence-based autonomous drones of the present invention
2 is a conceptual diagram of the integrated flight control system of the present invention
Figure 3 is a block diagram showing the system input and output configuration of the present invention
4 is a conceptual configuration diagram of Coral TPU-based autonomous flight of the present invention.

Hereinafter, an embodiment of the present invention will be described in detail with reference to the accompanying drawings.

1 is a block diagram of an integrated mission flight control system for artificial intelligence-based autonomous drones of the present invention. A DC-DC step-down regulator (2) that provides 33V DC output, and a USB-type photodiode (3) that can connect the design to USB 20 and provide up to 3A, 50V if the USB type-C port is used, Using a virtual RS232 serial connection through the USB connection to the board, and a first USB to UART (Serial Communications) converter (5) providing direct access to the system, and a virtual RS232 serial connection through the USB connection to the board. A second USB-UART converter (6) that provides direct access to the system and a connector (7) that includes two connectors to interface with the RPCM4 device on the Raspberry Pi computer module, controlled by hardware Interface with two modules with UART lines, HART-HART bridge (8) connecting UART4 4W on Raspberry Pi CM4 connector (7) to UART7 4W on CPU (1), to control servo motors and other PWM devices 8 PWM headers (9) for the 2-lane MIPI camera system, a 15-pin ribbon connector that exposes the 2-lane MIPI camera system to an external high-resolution camera module, the Raspberry Pi vertical camera connector module (10), and a custom 6-pin header that provides up to 6 pins. It provides three interfaces in a single unit for GPS (12), unconnected analog power (13), and on-board USB client devices, and 3 USB device ports using the Microchip USB 2513 USB 20 high-speed hub controller. A port USB client hub module 14, a custom 4-pin header 15 providing up to 4 pins at the discretion of the customer, and a custom 5-pin header 16 providing up to 5 pins at the discretion of the customer; Edge TPU and self-powered control Edge TPU is a small ASIC that accelerates Tensor Flow Lite models using little power, Acceleration Module (17), Network (CAN) protocol controller and physical 2 between the controller area. A high-speed CAN transceiver 18 providing a wire-type CAN bus interface, a 49 square mm pull-down touch switch 19 providing user input for signals of the CPU 1, and a removable non-volatile memory, the CPU 1 The card slot module 20 connected to the SD card of the ), the USB 2 port switch 21 sharing a single USB 20 port, and the CPU 1 connected to the USB 2 port switch 21 as a USB device. It consists of 1 USB bridge 22 and a 2nd USB bridge 23 connecting the CPU 1 to a USB device.

2 is a conceptual diagram of the integrated flight control system of the present invention, which is configured to perform control and communication with mobile centered on the integrated FCC 24, detect each device operation with a sensor, and mount video equipment and a loading box. .

3 is a system input/output diagram of the present invention, showing the relationship between the flight control board 30 and the mission control board 31 operating by inputting flight data through mutual UART, at the flight control terminal of the flight control board 30 It is configured to be connected to the main device 32 through the main device control output, and the mission control board 31 receives sensor information from the video equipment 33, business equipment 34 and power diagnosis equipment 35, and also It is configured to perform equipment control output.

4 is a conceptual configuration diagram of autonomous flight based on Coral TPU of the present invention. As shown therein, the present invention is an artificial intelligence-based Coral TPU, and requires the artificial intelligence function provided by Google by applying Google's TPU to the flight control board. It is a service that can be used for training and prediction purposes.

The operation of the present invention configured as described above will be described.

Firstly, CPU(1) is 32-bit Arm Cortex-M7 core with double precision FPU and L1 cache, 16K bytes data and 16K bytes instruction cache, command frequency up to 480MHz, MPU and DSP, 2MB of flash memory and 1 MB of RAM.

Additionally, the 3.3V/1.5A regulator (2) is a DC-DC step-down regulator that provides a 3.3V DC output at 1.5A required by certain components, accepts an input voltage between 3.1 and 16V DC and controls the output. do.

And the 3-port USB customer hub (4) controls the first vertical camera connector (10), the second vertical camera connector (11), the card slot module (20), and the USB 2-port switch through the regulator (2). 3.3V is provided to the CPU(1).

In addition, the power muxer module 14 smoothly switches between the two power sources, and the switching power multiplexer has the highest voltage at the source to which the power source is connected.

The photodiode (3) uses a USB Type-C port and connects the design to USB 2.0 and provides up to 3A, 5.0V, which is connected to the USB of the 3-port USB client hub (4).

The 3-port USB customer hub 4 provides three interfaces in one for on-board USB client devices, and the USB 2-port switch 21 can share a single USB 2.0 port.

The first USB-UART converter (5) and the second USB-UART converter (6) are also called FTDI, and if these USB-UART converters (5) and (6) are used, various functions can be performed through USB connection to the board. .

On the other hand, the HART-HART bridge (8) interfaces with two modules with UART lines controlled by hardware, and connects the UART4 4W of the Raspberry Pi CM4 connector (7) to the CPU (1).

Transceiver 18 is also a high-speed network transceiver that provides a network (CAN) protocol controller and physical two-wire network bus interface between the controller domains.

This transceiver 18 enables reliable communication in the network FD fast phase with data rates of up to 5 Mbit/s.

At this time, the operating voltage of the transceiver 18 is between 4.5V and 5.5V, and the module supports an IO logic level of 3.3V or 5.0V between the controller and the transceiver based on the user's selection.

The connector 7 includes two connectors for interfacing with the RPCM4 device in the Raspberry Pi computer module, and the RPCM4 COM connector is compatible only with the RPCM4.

The PWM header 9 provides 8 PWM headers for controlling servo motors and other PWM devices.

On the other hand, the present invention can process connectors (signals), and the Raspberry Pi first vertical camera connector 10 and the Raspberry Pi second vertical camera connector 11 expose a two-lane MIPI camera system to an external high-resolution camera module. With a 15-pin ribbon connector, the CSI port is connected to CAM0 of the Raspberry Pi CM4 connector (7).

In addition, I2C communication is connected to connector (7), and the enable input is provided by GPIO4 on connector (7).

Meanwhile, the second vertical camera connector 11 is a 15-pin ribbon connector that exposes the 2-lane MIPI camera system to an external high-resolution camera module, and the CSI port is connected to CAM1 of the Raspberry Pi CM4 connector 7.

I2C communication is connected to I2C0 on connector (7), and the enable input is provided by GPIO5 on connector (7).

The first USB bridge 22 of the present invention connects the CPU 1 to the USB 2-port switch 21 as a USB device, and the second USB bridge 23 also connects the CPU 1 to the USB device.

On the other hand, the analog power 13 is not connected to this module.

The first header (15) provides up to 4 pins to be used at the customer's discretion, and the second header (16) is a custom 5-pin JST GH header, which provides up to 5 pins to be used at the customer's discretion. provide pins.

The acceleration module 17 of the present invention, edge TPU and self-powered control edge TPU is a small ASIC that accelerates the Tensor Flow Lite model using little power, and performs 4 trillion operations per second (4 TOPS). and performs mobile vision models such as MobileNet v2 at 2 TOPS per watt using 2 watts of power, eg, near 400 frames per second in a power efficient manner.

Machine learning processing within the device reduces latency, enhances data privacy and does not require a constant internet connection, and the module provides a host interface, either PCIe Gen2 x 1 or USB2.0.

Meanwhile, the touch switch 19 is a 4.9 square mm pull-down touch switch, and provides a user input for a signal of the CPU 1.

And, the micro card slot module 20 provides a removable non-volatile memory, and the SD card reader is an SD card of the CPU 1.

Figure 4 is a Coral TPU-based autonomous flight concept configuration diagram of the present invention, the integrated mission flight control board for autonomous flight is composed of a flight control board and an off-board to which an artificial intelligence chip is applied. The flight control board is an automatic flight and an artificial intelligence board. are separated by

In the automatic flight, when a mission is registered and the aircraft takes off, it moves to the destination, performs the mission through obstacle avoidance and route tracking, and then lands while returning to the take-off point.

In addition, the artificial intelligence board performs a learning model to obtain a camera image in the aircraft take-off state, identifies an obstacle, performs obstacle avoidance, and recognizes an object to track a path.

The present invention is an integrated mission flight control system for autonomous drones. The flight system, which relied only on conventional commercial flight controllers, has unstable flight safety and frequently causes safety accidents. do.

That is, a system board equipped with an integrated FCC (Flight Controller Computer) (23) for autonomous drones applicable to various purposes (photography, real-time disaster management, reconnaissance and surveillance, military operations, etc.) and forms of unmanned aerial vehicles It collects information from various sensor modules interlocked to implement flight control to ensure stable flight and to perform missions in conjunction with GCS (Ground Control Station).

Therefore, when drone application developers utilize this technology, they can develop and reuse it for efficient management, thereby reducing development time and cost.

1:CPU 2:Regulator
3: photodiode 5: first converter
6: Second converter 7: Connector
8: Bridge 9: PWM Header
10: first vertical camera connector 11: second vertical camera connector
12:GPS 13:Analog power
14: power muxer module 15: first header
16: second header 17: acceleration module
18: transceiver 19: touch switch
20: card slot module 21: USB 2 port switch
22: first USB bridge 23: second USB bridge
24: integrated FCC 30: flight control board
31: mission control board 32: main device
33: video equipment 34: business equipment
35: power diagnosis device

Claims (2)

A 32-bit core CPU(1) to control each part, a DC-DC step-down regulator(2) providing 33V DC output at 15A, and a USB Type-C port to connect the design with USB, up to 3A, A photodiode (3) providing 50V, and a first USB to UART converter (5) using a virtual RS232 serial connection through the USB connection to the board and providing direct access to the system, and a USB connection to the board a second USB-to-UART converter (6), which uses a virtual RS232 serial connection via a virtual RS232 serial connection and provides direct access to the system; , HART-HART bridge (8) which interfaces with two modules with hardware controlled UART lines and connects UART4 4W on connector (7) to CPU (1), and for controlling servo motors and other PWM devices. 8 PWM headers (9), a first vertical camera connector module (10) and a second vertical camera connector (11), which are 15-pin ribbon connectors that expose a 2-lane MIPI camera system to an external high-resolution camera module, and a custom 6-pin A GPS (12) that provides up to 6 pins as a header, a power muxer module (14) that provides three interfaces for on-board USB customer devices and is a USB device port that uses a Microchip USB high-speed hub controller, and a maximum of 4 A custom 4-pin header (15) that provides pins, a custom 5-pin header (16) that provides up to 5 pins, and an acceleration module (17), which is a small ASIC that uses low power with edge TPU and self-powered edge TPU, , a high-speed transceiver 18 that provides a network protocol controller and a physical two-wire network bus interface between controller areas, a touch switch 19 that provides user input for signals from the CPU 1, and a removable non-volatile memory. It provides a card slot module (20) connected to the SD card of the CPU (1), a USB 2 port switch (21) that can share a single USB 20 port, and a USB 2 port switch with the CPU (1) as a USB device. It is configured to include first and second USB bridges 22 and 23 connected to (21),
Performing missions in conjunction with GCS while implementing flight control by collecting information from various sensor modules linked to system boards equipped with an integrated FCC (24) for autonomous drones that can be applied to filming, real-time disaster management, reconnaissance and surveillance, and unmanned aerial vehicles An integrated mission flight control system for artificial intelligence-based autonomous drones, characterized in that configured to.
The method of claim 1, wherein the integrated FCC board (24),
It is composed of a flight control board 30 and a mission control board 31, and the flight control board 30 performs flight control through the output of the main device 32, and the mission control board 31 An integrated mission flight control system for artificial intelligence-based autonomous drones, characterized in that it is configured to receive sensor information from the imaging equipment 33, mission equipment 34, and power diagnosis equipment 35 and control the equipment.

Images