CN110132060A - A method of intercepting drones based on visual navigation - Google Patents

Abstract

The invention discloses a method for intercepting drones based on visual navigation. When the black flying drone needs to be intercepted, the ground intercepting drone is quickly lifted into the air. When the ground personnel cannot directly control the intercepting drone due to factors such as communication interruption, the method of the present invention can utilize the image sensor equipped with the intercepting drone. Collect image information and perform corresponding processing, so that intercepting drones rely on visual navigation algorithms to automatically identify and track the target drone, and then slowly approach it. When approaching the target drone, a capture net is placed to capture the target drone. . The method of the invention does not require ground personnel to control, relies on the airborne system to independently complete the corresponding interception action, and has a high degree of automation. Compared with other interception methods, the method of the present invention is simple and direct, convenient to deploy, reusable, good real-time, high interception success rate, can effectively avoid interference, and can intercept various low-speed and small UAV targets.

Description

A method of intercepting drones based on visual navigation

technical field

The invention belongs to the technical field of unmanned aerial vehicles, in particular to a method for intercepting unmanned aerial vehicles based on visual navigation.

Background technique

With the lowering of the technical threshold of drones, a large number of consumer drones have flooded into the market of amateur drone players. The abuse of drones has become a major obstacle to low-altitude flight control, which seriously threatens public safety. Black flying drones are those drones that do not obtain relevant flight permits and do not obey the flight rules. They have brought many negative effects to people. In order to deal with this phenomenon, various anti-unmanned aerial vehicles have been born machine method.

At present, the anti-UAV technologies in various countries mainly include sonic interference, signal interference, hacking technology, laser cannons, "anti-UAV" UAVs, and capture of radio control. The characteristics and effects are different, but in general they can be divided into Three major categories: First, the anti-drone system of interference blocking type, which uses electromagnetic guns to emit corresponding electromagnetic signals to interfere with the flight control of the drone, causing the drone to lose control and automatically land. Although this method is simple, However, with the development of UAV technology, most UAVs can continue to fly by means of visual navigation after being interfered, which cannot meet the interception task of black flying UAVs under special circumstances; The unmanned aerial vehicle system uses damaging weapons to directly shoot down black flying drones. This method will generate a large number of damage fragments, threatening the safety of the public, and cannot be used in public places; the third is monitoring and control anti-UAV systems, such methods Generally, computer technology is used to invade the UAV control system, hijack the UAV radio control, and then capture the UAV, which is technically difficult and not universally applicable. A more popular method that appears now is to intercept and capture black flying drones point-to-point. However, because the flying height of the drone is far from the ground, the ground personnel cannot directly estimate the appropriate time to release the capture net device. The success rate of UAV interception is not high. This method may also cause collision between intercepting UAVs and black flying UAVs, which seriously threatens the safety of ground personnel and equipment. Sometimes the communication between intercepting UAVs and the ground may even be completely interrupted. , the ground personnel directly lose the control of the intercepting UAV, so that the entire interception operation fails, so it is necessary to carry out corresponding technical improvements to meet the requirements of actual use.

SUMMARY OF THE INVENTION

The purpose of the present invention is to propose a method for intercepting drones based on visual navigation, by applying visual navigation to automatically control the intercepting drones to fly to the most favorable position to release the interception and capture net, greatly improving the interception of target drones Success rate.

Above-mentioned purpose of the present invention is achieved through the following technical solutions:

A method for intercepting drones based on visual navigation, comprising the following steps:

Step 1: Train the UAV detection and recognition model;

Step 2: Solve the position P _t of the target UAV; when the image device of the intercepting UAV collects the image of the target UAV, the intercepting UAV slowly approaches the target UAV, while controlling the rotation of the servo gimbal , keep the target drone in the center of the image, including:

The coordinate systems involved in the process of calculating the position and attitude of the target UAV from the coordinate position of the target UAV in the image and the rotation of the servo gimbal include: inertial coordinate system, body coordinate system and camera coordinate system. The relative position relationship between the drone, the intercepting drone and the camera, the position P _t of the target drone in the inertial coordinate system is expressed as:

P _t =P ₁ +P ₂ +P ₃ (1)

Among them, P ₁ represents the position of the intercepting drone in the inertial coordinate system; P ₂ represents the distance vector from the center of mass of the intercepting drone to the optical center of the camera in the inertial coordinate system, which is calculated by the following formula:

where P _γ represents the position of the camera in the body coordinate system, The matrix represents the rotation matrix from the coordinate system of the intercepting UAV body to the inertial coordinate system, which is determined by the attitude angle of the intercepting UAV. Sure:

in represents the roll angle of the intercepting drone, θ represents the pitch angle of the intercepting drone, and ψ represents the yaw angle of the intercepting drone;

P ₃ represents the distance vector from the camera to the target UAV in the inertial coordinate system, which satisfies the following relationship:

in is the position of the target drone in the camera coordinate system, and its coordinates are expressed as Represents the rotation matrix from the camera coordinate system to the body coordinate system. It is determined by the horizontal rotation angle α and the pitch angle β of the PTZ camera, which can be obtained from the code disc information on the PTZ. The specific expressions are as follows:

The position of the target drone is sorted out:

Let the imaging position of the center point of the target UAV in the image plane be (u ⁱ ,vi ⁾ , which is expressed as:

where M is the intrinsic parameter matrix of the camera, expressed as:

where [μ ₀ , v ₀ ] ^T is the center of the phase plane, dx, dy represent the physical size of each pixel in the x-axis and y-axis direction, which can be obtained by the calibration of the camera. The imaging position (u ⁱ , v ⁱ ) is the position coordinate of the target UAV in the image, which is determined by the monocular imaging The principle can be obtained:

Among them, OP1 is the distance between the center of the target UAV and the center of the image, which can be obtained by the image processing part, and f is the focal length of the camera. O ₀ P is the distance from the center of the lens to the target UAV, γ represents the angle between the connection line between the target UAV and the center of the lens and the optical axis;

According to formula (9), (10) can be obtained Then it can be obtained by formula (7) Thereby, the position P _t of the target UAV under the inertial coordinate system is solved;

Step 3: The intercepting drone uses machine vision to navigate to the best interception position;

After the intercepting UAV detects and identifies the target UAV, the intercepting UAV calculates the position coordinates of the target UAV according to the coordinate position of the target UAV in the image and the rotation of the servo gimbal to form a loop control command to adjust the interception in real time. UAV flight attitude, slowly approaching the target UAV. When the intercepting drone is close to the target drone, the laser ranging radar carried by the intercepting drone scans the target drone and measures the distance to it. When the intercepting drone is below the target drone , Under the condition of maintaining a safe distance, the intercepting drone slowly climbs to 2-3m above the target drone according to the movement of the target in the image and maintains the same movement state of the target drone;

Step 4: Release the intercepting and capturing net; after adjusting the position of the intercepting drone, throw the capturing net, and one end of the capturing net is wrapped around the target drone, prompting the target drone to stop flying. When the pulling force of the capturing net is greater than a certain value At the same time, the capturing net and the intercepting drone are separated, and the capturing net and the target drone fall together.

The beneficial effects of the invention are as follows: (1) the intercepting device is economical and cheap, easy to use, small in size and easy to carry, and can meet the task of anti-UAV in different situations; (2) it can also meet the requirements of different numbers and types of UAVs Interception work; (3) By adding a visual navigation module to the intercepting drone, even if the intercepting drone is completely out of the control of the ground, it can complete the interception of the target drone. In the special case of intercepting the target UAV, the interception method of the present invention has obvious advantages and broad commercial application prospects.

In addition to the objects, features and advantages described above, the present invention has other objects, features and advantages. The present invention will be described in further detail below with reference to the drawings.

Description of drawings

FIG. 1 is a complete flow chart of the method for intercepting drones based on visual navigation of the present invention.

FIG. 2 is a flowchart of the imaging, identification and tracking of the UAV relying on visual navigation in the present invention.

Figure 3 is a schematic diagram of the transformation of each coordinate system of the UAV system.

Figure 4 is a schematic diagram of the target UAV imaging.

Figure 5 is a schematic diagram of the intercepting drone in the optimal interception position.

Figure 6 is a schematic diagram of intercepting drones throwing nets to capture target drones.

Detailed ways

The present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

The specific requirements for the UAV image processing module to correctly identify the target UAV include the following:

The image sensor carried by the UAV must meet the conditions of day and night to complete the image acquisition. When the image collected by the image sensor is transmitted to the UAV image processing module, the image needs to be preprocessed to complete the target detection. Identification of drones. In order to meet the extreme requirements for detection and recognition speed in the method of the present invention, NVIDIA Jetson TX2 kit is used in the image processing module. Jetson TX2 is an artificial intelligence supercomputer module. Although it adopts energy-saving and compact size, it has high-speed computing performance. It is suitable for intelligent terminal devices such as robots, drones, and smart cameras. In order to avoid the interference of UFOs such as aerial birds, kites and hydrogen balloons on the image recognition module and improve the recognition accuracy of the UAV, a detection and recognition algorithm based on deep learning is used in the present invention.

The detection and recognition method based on deep learning described in the present invention adopts the YOLO algorithm framework to realize the detection and recognition function of the image processing module. The YOLO algorithm framework is to directly select the entire image to realize the training of the model. In this way, the distinction between the target object and the background is easier, and the detection speed is also greatly improved. The main steps of YOLO detection include:

1. Scale the input image to 448*448 size;

2. Run CNN;

3. The detection results are optimized by means of non-maximum suppression.

In conjunction with Figures 1-2, the steps to implement the method described in the present invention:

Step 1: Train the drone detection and recognition model.

(1) The first is to prepare the identification data set, collect the flight video sequences or pictures of the UAV in various scenarios from the Internet or from the experiment, including the photo collection of the UAV from various perspectives in complex backgrounds such as over the city;

(2) Screen the collected data according to the normal scene, remove the blurred photos in the data set, and ensure the quality of the training data;

(3) Labeling of the training data set, in the present invention, we only need to be able to identify the drone, use the labeling tool to label the drone in each picture in the data set, and then train all the labeled data. Obtain the UAV detection and recognition model;

(4) The detection and recognition effect of the test model, the detection speed and the recognition accuracy meet the real-time requirements.

Step 2: Find the position P _t of the target UAV.

(1) When the image device of the intercepting UAV collects the image of the target UAV, the intercepting UAV should slowly approach the target UAV, and at the same time control the rotation of the servo gimbal to keep the target UAV in the image as much as possible. In the middle, the intercepting UAV calculates the position and coordinates of the target UAV according to the coordinate position of the target UAV in the image and the rotation of the servo gimbal, adjusts the flight attitude of the intercepting UAV in real time, and slowly approaches the target UAV. The interceptor drone is guaranteed to catch up with the target drone and intercept it.

As shown in FIG. 3 , the coordinate systems involved in the process of calculating the position coordinates of the target UAV from the coordinate position of the target UAV in the image and the rotation of the servo gimbal include: the inertial coordinate system O ₁ , the body The coordinate system O ₂ and the camera coordinate system O ₃ , according to the relative positional relationship between the target UAV, the intercepting UAV and the camera, the position vector P _t of the target UAV in the inertial coordinate system can be expressed as:

P _t =P ₁ +P ₂ +P ₃ (1)

Among them, P ₁ represents the position vector of the intercepting drone in the inertial coordinate system, and the parameters such as the attitude angle of the drone can be measured by sensors such as accelerometers and gyroscopes; P ₂ represents the center of mass of the intercepting drone to The distance vector of the camera optical center in the inertial coordinate system, which can be calculated by the formula:

Among them, P _γ represents the position of the camera in the body coordinate system, which can be obtained by calibration. The matrix represents the rotation matrix from the body coordinate system to the inertial coordinate system, which is determined by the attitude angle of the intercepting drone. (roll angle, pitch angle, yaw angle) determine:

where P ₃ represents the distance vector from the camera to the target UAV in the inertial coordinate system, which satisfies the following relationship:

in is the position of the target UAV in the camera coordinate system, Represents the rotation matrix from the camera coordinate system to the body coordinate system. It is determined by the horizontal rotation angle α and the pitch angle β of the PTZ camera, which can be obtained from the code disc information on the PTZ. The specific expressions are as follows:

To sort out the positions of the available sports targets:

So, solve After that, the position P _t of the target UAV in the inertial coordinate system can be solved. Assuming that the imaging position of the center point of the target UAV in the image plane is (u ⁱ ,vi ⁾ , it can be expressed as :

where M is the intrinsic parameter matrix of the camera, which can be expressed as:

where [μ ₀ , v ₀ ] ^T is the center of the phase plane, dx, dy represent the physical size of each pixel in the x-axis and y-axis direction, which can be obtained by the calibration of the camera. The imaging position (u ⁱ , v ⁱ ) can be obtained in the image processing module, so to obtain P _t , it is necessary to obtain the depth information of the target in the camera coordinate system

As shown in Figure 4, where O ₀ is the center of the lens, point O is the intersection of the optical axis and the meridian plane, and P1 is the projected position of the center point of the target drone on the image plane. In the intercepting drone, the installation position of the laser rangefinder can be approximately considered to be at the center of the lens, so the distance O ₀ P between the center of the intercepting drone's lens and the target drone can be calculated by the laser rangefinder. can be obtained from the geometric relationship

Among them, OP1 is the distance between the center of the target drone and the center of the image, which can be obtained from the image processing part, f is the focal length of the camera, and γ represents the angle between the line connecting the target drone and the center of the lens and the optical axis.

According to formula (7), it can be obtained The height of the intercepting UAV can be measured by the flight altimeter. The position of the intercepting UAV in the inertial coordinate system is known, and the position of the camera in the body coordinate system is known, so as to solve the target UAV in the inertial coordinate system. At this _point , the result of the solution can be converted into a control command to intercept the UAV close to the target UAV;

Step 3: The intercepting drone uses machine vision to navigate to the best interception position.

After the intercepting UAV detects and identifies the target UAV, the intercepting UAV calculates the position coordinates of the target UAV according to the coordinate position of the target UAV in the image and the rotation of the servo gimbal to form a loop control command to adjust the interception in real time. UAV flight attitude, slowly approaching the target UAV:

(1) When the intercepting UAV approaches the target UAV by visual navigation, it is necessary to turn the gimbal to keep the center of the target UAV in the center of the image. When it is found that the intercepting UAV and the target UAV are flying towards each other, That is, the target drone has been kept in the center of the intercepting drone's image, but when the distance is getting closer and closer, the intercepting drone should stop approaching immediately, turn to other directions, first avoid the target drone and then slowly approach it.

(2) When the intercepting UAV is close to the target UAV, various relative positions will occur due to different interception conditions each time. In order to adjust to the best capture net placement position, the present invention considers Typical upper and lower relative positions, when the intercepting UAV approaches the target UAV, the laser ranging radar carried by the intercepting UAV scans the target UAV and measures the distance between them. When the intercepting UAV is below the target UAV, while maintaining a safe distance, the intercepting UAV slowly climbs to the position above the target UAV (2-3m) according to the movement of the target in the image Keep the same motion state as the target drone. As shown in Figure 5, after the intercepting UAV reaches the position above the target UAV, the conical area within a certain angle under the intercepting UAV is an effective intercepting area. Among them, H represents the safety distance (2m).

Step 4: Release the interception and capture net. As shown in Figure 6, after adjusting the position of the intercepting drone, throw out the capture net. One end of the capture net is connected to the intercepting drone, and the other end is downward. After the capture net is released, it will touch the target drone. And quickly wrapped around the rotor of the target UAV, prompting the target UAV rotor to stop rotating, after the catching net was wrapped around the target UAV, the target UAV fell quickly due to the loss of lift, and the capture net was pulled by the target UAV. When it reaches a certain value, the other end of the capture net is automatically released. With the separation of the capture net and the intercepting drone, the landing deceleration device on the other end will be released, so that the target drone and the capture net will land slowly together. There will be no security threat to personnel on the ground.

Step 5: After the intercepting drone completes the intercepting action, it will automatically return to home according to its own default mode.

It should be noted that in the method of this paper, the intercepting UAV can autonomously track and intercept the target UAV when it is out of the control of the ground personnel, but when the communication between the ground personnel and the intercepting UAV is carried out, the ground operator needs to When controlling the intercepting drone, the ground operator has the highest priority for the control command of the intercepting drone, and the intercepting drone will give priority to responding to the ground operation command. At the same time, it is a major advantage of the present invention that the intercepting drone automatically completes the corresponding interception action, which greatly improves the automation degree of the interception system while improving the interception success rate.

The above descriptions are only preferred embodiments of the present invention, and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and changes. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims (5)

1. a method for intercepting drone based on visual navigation, is characterized in that comprising the following steps: Step 1: Train the UAV detection and recognition model; Step 2: Solve the position P _t of the target UAV; when the image device of the intercepting UAV collects the image of the target UAV, the intercepting UAV slowly approaches the target UAV, while controlling the rotation of the servo gimbal , keep the target drone in the center of the image, including: The coordinate systems involved in the process of calculating the position and attitude of the target UAV from the coordinate position of the target UAV in the image and the rotation of the servo gimbal include: inertial coordinate system, body coordinate system and camera coordinate system. The relative position relationship between the drone, the intercepting drone and the camera, the position P _t of the target drone in the inertial coordinate system is expressed as: P _t =P ₁ +P ₂ +P ₃ (1) Among them, P ₁ represents the position of the intercepting drone in the inertial coordinate system; P ₂ represents the distance vector from the center of mass of the intercepting drone to the optical center of the camera in the inertial coordinate system, which is calculated by the following formula: where P _γ represents the position of the camera in the body coordinate system, The matrix represents the rotation matrix from the body coordinate system to the inertial coordinate system, which is determined by the attitude angle of the intercepting drone. Sure: in represents the roll angle of the intercepting drone, θ represents the pitch angle of the intercepting drone, and ψ represents the yaw angle of the intercepting drone; P ₃ represents the distance vector from the camera to the target UAV in the inertial coordinate system, which satisfies the following relationship: in is the position of the target drone in the camera coordinate system, and its coordinates are expressed as Represents the rotation matrix from the camera coordinate system to the body coordinate system, which is determined by the horizontal rotation angle α and the pitch angle β of the PTZ camera. The specific expression is as follows: The position of the target drone is sorted out: Let the imaging position of the center point of the target UAV in the image plane be ( ^{u i} , vi ), which is expressed by the pinhole model as: where M is the intrinsic parameter matrix of the camera, expressed as: where [μ ₀ , v ₀ ] ^T is the center of the phase plane, dx, dy represent the physical size of each pixel in the x-axis and y-axis direction, which is obtained by the calibration of the camera. The imaging position (u ⁱ , vi ⁾ is the position coordinate of the target drone in the image, based on the principle of monocular imaging Available: Among them, OP1 is the distance between the center of the target UAV and the center of the image, f is the focal length of the camera, O ₀ P is the distance from the center of the lens to the target UAV, γ represents the connection between the target UAV and the center of the lens and the optical axis the angle between, According to formulas (9) and (10), it can be obtained Then it can be obtained from formula (7) Thereby, the position P _t of the target UAV under the inertial coordinate system is solved; Step 3: The intercepting drone uses machine vision to navigate to the best interception position; After the intercepting UAV detects and identifies the target UAV, the intercepting UAV calculates the position coordinates of the target UAV according to the coordinate position of the target UAV in the image and the rotation of the servo gimbal to form a loop control command to adjust the interception in real time. The flying attitude of the UAV, slowly approaching the target UAV, when the intercepting UAV approaches the target UAV, the laser ranging radar carried by the intercepting UAV scans the target UAV and measures the distance to it. When the intercepting drone is below the target drone, while maintaining a safe distance, the intercepting drone slowly climbs to 2-3m above the target drone according to the movement of the target in the image, and then Keep the same motion state as the target drone; Step 4: Release the intercepting and capturing net; after adjusting the position of the intercepting drone, throw the capturing net, and one end of the capturing net is wrapped around the target drone, prompting the target drone to stop flying. When the pulling force of the capturing net is greater than a certain value At the same time, the capturing net and the intercepting drone are separated, and the capturing net and the target drone fall together. 2 . The method for intercepting drones based on visual navigation according to claim 1 , further comprising step 5: after the intercepting drones complete the interception action, they automatically return to home in their own default mode. 3 . 3. The method for intercepting drones based on visual navigation according to claim 1 or 2, wherein the camera is a CCD camera with an infrared imaging function. 4. The method for intercepting drones based on visual navigation according to claim 1 or 2, wherein the capture net is a nylon net. 5. The method for intercepting drones based on visual navigation according to claim 1 or 2, wherein the deceleration and landing device is a parachute.