CN104102218B - The perception of view-based access control model servo and bypassing method and system - Google Patents

The perception of view-based access control model servo and bypassing method and system Download PDF

Info

Publication number
CN104102218B
CN104102218B CN201410305109.6A CN201410305109A CN104102218B CN 104102218 B CN104102218 B CN 104102218B CN 201410305109 A CN201410305109 A CN 201410305109A CN 104102218 B CN104102218 B CN 104102218B
Authority
CN
China
Prior art keywords
target
angle
envelope
avoidance
visual
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201410305109.6A
Other languages
Chinese (zh)
Other versions
CN104102218A (en
Inventor
吕洋
潘泉
赵春晖
张夷斋
刘流
席庆彪
刘慧霞
吴薇
朱海峰
程承
康青青
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Northwestern Polytechnical University
Original Assignee
Northwestern Polytechnical University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Northwestern Polytechnical University filed Critical Northwestern Polytechnical University
Priority to CN201410305109.6A priority Critical patent/CN104102218B/en
Publication of CN104102218A publication Critical patent/CN104102218A/en
Application granted granted Critical
Publication of CN104102218B publication Critical patent/CN104102218B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Landscapes

  • Control Of Position, Course, Altitude, Or Attitude Of Moving Bodies (AREA)

Abstract

本发明提供一种基于视觉伺服的感知与规避系统,包括无人机、图像采集系统、视觉伺服控制系统和导航定位系统,所述视觉伺服系统包括视觉目标检测与跟踪模块、安全包络模块和视觉伺服控制器。本发明还提供一种基于所述感知与规避系统的规避方法,本发明实现了对周围空域环境的快速感知,以及在不加装任何测距传感器且不需地面操作人员进行干预和操作的情况下,独立自主完成对空间飞行目标的规避机动;本发明载荷要求低、控制精度高、具有较高的智能性,能够提高无人机的空域飞行安全能力。

The present invention provides a visual servo-based perception and avoidance system, including a drone, an image acquisition system, a visual servo control system, and a navigation and positioning system. The visual servo system includes a visual target detection and tracking module, a security envelope module and Visual Servo Controller. The present invention also provides an avoidance method based on the sense-and-avoid system. The present invention realizes rapid perception of the surrounding airspace environment, and does not require the intervention and operation of ground operators without installing any ranging sensors. Under the environment, the evasive maneuver of the space flight target can be independently completed; the invention has low load requirements, high control precision, high intelligence, and can improve the airspace flight safety capability of the UAV.

Description

基于视觉伺服的感知与规避方法及系统Perception and avoidance method and system based on visual servoing

技术领域technical field

本发明涉及无人机导航与控制领域,特别涉及一种基于视觉伺服的感知与规避方法及系统。The invention relates to the field of unmanned aerial vehicle navigation and control, in particular to a visual servo-based perception and avoidance method and system.

背景技术Background technique

近来,随着军事应用与民用领域对无人机的需求日益强烈,加上我国民用空域领域的进一步开放,未来空域将呈现多种类型功能的无人机、有人机进行空域共享和集成的局面,空域将日趋密集。在此情况下,无人机感知与规避(Sense and Avoid,即SAA)能力将成为进入空域飞行、保障无人机飞行安全的先决条件。无人机感知与规避是指无人机利用机载传感器或地面监视系统完成对空域飞行环境的监视和飞行目标的状态获取,对潜在碰撞威胁的目标进行规避路径规划,完成规避机动动作,从而保证无人机的航路飞行安全。Recently, with the increasingly strong demand for UAVs in military applications and civilian fields, coupled with the further opening of my country's civil airspace, the future airspace will present a situation where various types of functional UAVs and manned aircraft will share and integrate airspace , the airspace will become increasingly dense. In this case, the UAV's Sense and Avoid (SAA) capability will become a prerequisite for entering the airspace and ensuring the safety of UAV flight. UAV perception and avoidance means that UAVs use airborne sensors or ground surveillance systems to monitor the airspace flight environment and obtain the status of flying targets, plan evasive paths for targets with potential collision threats, and complete evasive maneuvers. Ensure the flight safety of drones.

SAA技术是目前无人机技术领域亟待解决的关键技术难题。2013年,在美国FAA(Federal Aviation Administration)发布的无人机系统空域集成路线图中,明确提出SAA能力是无人机进行国家空域飞行的必备能力。其主要功能分为:目标检测与跟踪、碰撞威胁估计、规避路径规划、规避机动。SAA technology is a key technical problem to be solved urgently in the field of UAV technology. In 2013, in the UAV system airspace integration roadmap released by the US FAA (Federal Aviation Administration), it was clearly stated that SAA capability is a necessary capability for UAVs to fly in national airspace. Its main functions are divided into: target detection and tracking, collision threat estimation, evasive path planning, and evasive maneuvering.

无人机机载SAA传感器包括T-CAS,ADS-B、雷达、激光雷达、光电、红外传感器等。其中,光电传感器作为对非合作目标非常重要和有效感知与规避手段,特别是在轻小型无人机系统载荷、任务、成本有限,无法搭载大型感知设备如雷达、激光雷达等情况下,基于光电传感器的视觉SAA系统在质量、功耗、成本方面的优势使其更易在轻小型系统集成和应用;视觉SAA被动式感知的特点适用于战场环境、隐身任务等特定任务场合;且与ADS-B、TCAS等相比,视觉SAA不依靠应答机制,不需要地面控制信息和卫星定位信息,即能够提供对非合作目标独立的感知与规避能力。因此,研究基于光电传感器的视觉SAA系统具有重要意义。UAV airborne SAA sensors include T-CAS, ADS-B, radar, lidar, photoelectric, infrared sensors, etc. Among them, photoelectric sensors are very important and effective means of perception and avoidance for non-cooperative targets, especially in the case of light and small UAV systems with limited load, tasks, and cost, and cannot carry large-scale sensing equipment such as radar and lidar. The advantages of the sensor's visual SAA system in terms of quality, power consumption, and cost make it easier to integrate and apply in light and small systems; the characteristics of passive perception of visual SAA are suitable for specific tasks such as battlefield environments and stealth tasks; and ADS-B, Compared with TCAS, etc., visual SAA does not rely on the response mechanism, does not need ground control information and satellite positioning information, that is, it can provide independent perception and avoidance capabilities for non-cooperative targets. Therefore, it is of great significance to study the visual SAA system based on photoelectric sensor.

发明内容Contents of the invention

本发明的目的在于提供一种基于视觉伺服的感知与规避方法及系统,以解决现有传感器体积大、功耗较大,不利于在轻小型无人机系统中的集成应用的问题,并且本发明能够实现对周围空域环境的快速感知,通过角度安全包络建立视觉反馈控制模型,控制无人机完成对空域目标的感知与规避。The purpose of the present invention is to provide a visual servo-based perception and avoidance method and system to solve the problems of existing sensors with large volume and high power consumption, which are not conducive to the integrated application in light and small unmanned aerial vehicles systems, and the present invention The invention can realize the rapid perception of the surrounding airspace environment, establish a visual feedback control model through the angle safety envelope, and control the UAV to complete the perception and avoidance of airspace targets.

本发明提供一种基于视觉伺服的感知与规避系统,包括:The present invention provides a perception and avoidance system based on visual servoing, including:

无人机,所述无人机上设置自动驾驶仪,所述自动驾驶仪接收飞行控制指令进行飞行并完成无人机对障碍物的规避动作;An unmanned aerial vehicle, an autopilot is set on the unmanned aerial vehicle, and the automatic pilot receives flight control instructions to fly and complete the avoidance action of the unmanned aerial vehicle to obstacles;

图像采集系统,搭载在所述无人机上,用于采集无人机的飞行空域图像信息,采用多相机多视角集中式阵列构型,并通过相机接口实时传输图像信息;The image acquisition system is mounted on the unmanned aerial vehicle and is used to acquire the flight airspace image information of the unmanned aerial vehicle, adopts a multi-camera multi-view centralized array configuration, and transmits image information in real time through the camera interface;

视觉伺服控制系统,设置在所述无人机上,包括视觉目标检测与跟踪模块、安全包络模块和视觉伺服控制器,视觉目标检测与跟踪模块接收图像采集系统传输的图像信息并实现对图像中飞行目标的检测、定位和跟踪,安全包络模块根据获取的目标状态信息进行图像平面角度安全包络的生成,并基于该安全包络生成图像规避角度,以此角度输入至视觉伺服控制器,视觉伺服控制器输出规避机动角度,并将该角度通过串口传输至自动驾驶仪完成规避动作;The visual servo control system is arranged on the unmanned aerial vehicle and includes a visual target detection and tracking module, a safety envelope module and a visual servo controller. The visual target detection and tracking module receives the image information transmitted by the image acquisition system and realizes the detection For the detection, positioning and tracking of flying targets, the safety envelope module generates the image plane angle safety envelope based on the acquired target state information, and generates image avoidance angles based on the safety envelope, and inputs this angle to the visual servo controller. The visual servo controller outputs the evasive maneuvering angle, and transmits the angle to the autopilot through the serial port to complete the evasive action;

导航定位系统,设置在所述无人机上,用于建立航路回归机制,导航定位系统检测到无人机不在航路且不存在威胁时,进行航路回归,使无人机返回原始航路。The navigation and positioning system is set on the UAV and is used to establish a route return mechanism. When the navigation and positioning system detects that the UAV is not on the route and there is no threat, it performs route regression to make the UAV return to the original route.

作为一种优选方式,本发明中所述视觉目标检测与跟踪模块采用多路高清图像处理平台DM8168。As a preferred mode, the visual target detection and tracking module in the present invention adopts the multi-channel high-definition image processing platform DM8168.

本发明还提供一种基于视觉伺服的感知与规避系统的规避方法,包括:The present invention also provides a method for avoiding the perception and avoidance system based on visual servoing, including:

步骤一,通过无人机机载图像采集系统实现对飞行视场角内的空域的完全覆盖,获取无人机的飞行空域图像信息;Step 1, realize the complete coverage of the airspace within the flight field of view through the UAV airborne image acquisition system, and obtain the flight airspace image information of the UAV;

步骤二,机载视觉伺服控制系统中视觉目标检测与跟踪模块接收步骤一中图像信息并进行处理,完成对空域图像中飞行目标的检测、定位和跟踪;Step 2, the visual target detection and tracking module in the airborne visual servo control system receives and processes the image information in step 1, and completes the detection, positioning and tracking of the flying target in the airspace image;

步骤三,根据步骤二中获取的飞行目标状态信息,视觉伺服控制系统中安全包络模块进行图像平面角度安全包络生成,通过安全包络确定无人机是否存在碰撞威胁,针对碰撞威胁,计算得到图像意义下的规避角度;Step 3: According to the state information of the flight target obtained in step 2, the safety envelope module in the visual servo control system generates the image plane angle safety envelope, and determines whether there is a collision threat to the UAV through the safety envelope. For the collision threat, calculate Obtain the avoidance angle in the image sense;

步骤四,将步骤三中的规避角度输入视觉伺服控制系统中的视觉伺服器,视觉伺服器输出需要的规避机动俯仰和偏航角度,并传送至无人机自动驾驶仪完成规避机动;Step 4, input the evasive angle in the step 3 into the visual servo in the visual servo control system, the visual servo outputs the required evasive maneuver pitch and yaw angle, and transmits to the UAV autopilot to complete the evasive maneuver;

步骤五,利用导航定位系统建立航路回归机制,当无人机不存在威胁时,使无人机返回原始航路。Step five, use the navigation and positioning system to establish a route return mechanism, and when the UAV is not threatened, the UAV will return to the original route.

作为一种优选方式,所述步骤二的具体实现方法如下:As a preferred method, the specific implementation method of the second step is as follows:

步骤一中图像采集系统获取的图像数据通过相机数据线传送至机载视觉伺服控制系统中视觉目标检测与跟踪模块,其中图像采集系统中共设置A、B、C三台相机,视觉目标检测与跟踪模块采用多路高清图像处理平台DM8168,利用DM8168实现对获取的多路视频进行处理,通过检测算法获取目标k时刻在图像采集系统中某一相机图像平面的位置信息pk=[pxk,pyk],获得目标相对于该相机的相对角度:In step 1, the image data acquired by the image acquisition system is transmitted to the visual target detection and tracking module in the airborne visual servo control system through the camera data cable. The image acquisition system is equipped with three cameras A, B, and C for visual target detection and tracking. The module adopts the multi-channel high-definition image processing platform DM8168, and uses the DM8168 to process the acquired multi-channel video, and obtains the position information p k of a certain camera image plane in the image acquisition system at time k of the target through the detection algorithm p k =[p xk ,p yk ] to get the relative angle of the target relative to the camera:

其中,w和h分别是以像素为单位的图像的宽度和高度,f为相机镜头焦距,μ为象元尺寸;Among them, w and h are the width and height of the image in pixels, respectively, f is the focal length of the camera lens, and μ is the pixel size;

根据相机安装位置,可获得目标k时刻相对于无人机的相对角度为Θk=[σkk],其中According to the installation position of the camera, the relative angle of the target relative to the UAV at moment k can be obtained as Θ k = [σ kk ], where

γk=γ′γ k = γ′

式中分别表示检测到的目标来自于图像采集系统中的相机A、相机B和相机C;In the formula Respectively indicate that the detected target comes from camera A, camera B and camera C in the image acquisition system;

根据前后时刻的角度变化确定目标的角速度Ωk=[ωxkyk]=Θk+1k,k>0。Determine the angular velocity of the target Ω k =[ω xkyk ]=Θ k+1k ,k>0 according to the angular changes at the previous and subsequent moments.

作为另一种优选方式,所述步骤三中安全包络的生成步骤及规避角度的计算步骤包括:As another preferred manner, the step of generating the safety envelope and the step of calculating the avoidance angle in the step 3 include:

①建立目标安全包络圆,圆心为目标点,选取一个半径初始值,并以固定速度进行增长;①Establish the target safety enveloping circle, the center of which is the target point, select an initial value of the radius, and grow at a fixed speed;

②根据目标尺寸建立安全包络椭圆,椭圆圆心为目标点,椭圆短轴的长度与包络圆的半径相等,椭圆长轴的长度为短轴的长度与目标在当前角速度下在预留时间内的外推长度之和,长轴方向平行于目标的运动方向;②Create a safe envelope ellipse according to the size of the target. The center of the ellipse is the target point. The length of the minor axis of the ellipse is equal to the radius of the envelope circle. The sum of the extrapolated lengths of , the long axis direction is parallel to the moving direction of the target;

③前述生成的包络圆和包络椭圆形成平面角度安全包络,该安全包络由以椭圆短轴为分割线的沿目标运动方向的半椭圆与沿目标运动反方向的半圆组成;③ The aforementioned generated envelope circle and envelope ellipse form a plane angle safety envelope, which is composed of a semi-ellipse along the direction of target motion and a semi-circle along the opposite direction of target motion with the short axis of the ellipse as the dividing line;

④基于前述平面角度安全包络进行目标威胁估计,建立基于安全包络的威胁标示函数,以此判断是否存在碰撞威胁;④ Estimate the target threat based on the aforementioned plane angle safety envelope, and establish a threat marking function based on the safety envelope to judge whether there is a collision threat;

⑤根据最小规避距离原则,在平面角度安全包络寻找距离初始目标最近的点。⑤ According to the principle of minimum avoidance distance, find the point closest to the initial target in the plane angle safety envelope.

特别地,所述步骤三中安全包络的具体生成步骤及规避角度的具体计算步骤如下:In particular, the specific generation steps of the safety envelope and the specific calculation steps of the avoidance angle in the step three are as follows:

①建立目标安全包络圆,圆心选为目标k时刻相对于无人机的相对角度位置Θk=[σk,γk],选取一个半径初始值r0,并以固定速度进行增长,则①Establish the target safety enveloping circle, select the center of the circle as the relative angular position Θ k =[σ k , γ k ] of the target relative to the UAV at time k, select an initial value r 0 of the radius, and increase it at a fixed speed, then

rk+1=rk+ε,k=1,2,3…r k+1 =r k +ε,k=1,2,3...

rk=r0,k=0r k = r 0 , k = 0

其中ε是固定增长速度,rk+1代表k+1时刻的安全包络圆半径;Where ε is a fixed growth rate, r k+1 represents the radius of the safe envelope circle at time k+1;

②根据目标尺寸建立安全包络椭圆,椭圆圆心为目标点Θk=[σkk],长轴方向平行于目标的运动方向,即目标角速度Ωk的方向,长轴和短轴的长度分别按下列公式选取:② Establish a safety envelope ellipse according to the size of the target, the center of the ellipse is the target point Θ k = [σ k , γ k ], the long axis direction is parallel to the target’s movement direction, that is, the direction of the target angular velocity Ω k , the long axis and the short axis The length is selected according to the following formula:

a=rk(1+α‖Ωk‖),a=r k (1+α‖Ω k ‖),

b=rkb=r k ,

式中,a和b分别代表包络椭圆的长轴和短轴,rk为k时刻包络圆的半径,α为角速度转化为轴长的比例因子,该因子代表规避预留时间,‖Ωk‖为目标运动角速度向量的模;In the formula, a and b represent the major axis and minor axis of the envelope ellipse respectively, r k is the radius of the envelope circle at time k, α is the scaling factor for converting angular velocity into axis length, which represents the reserved time for avoidance, ‖Ω k ‖ is the modulus of the target motion angular velocity vector;

③前述生成的包络圆和包络椭圆形成平面角度安全包络,该安全包络由以椭圆短轴为分割线的沿目标运动方向的半椭圆与沿目标运动反方向的半圆组成;③ The aforementioned generated envelope circle and envelope ellipse form a plane angle safety envelope, which is composed of a semi-ellipse along the direction of target motion and a semi-circle along the opposite direction of target motion with the short axis of the ellipse as the dividing line;

④基于前述平面角度安全包络进行目标威胁估计,建立基于时间k的威胁标示函数lk,当lk<0时,认为存在碰撞威胁,以此判断是否存在碰撞威胁,其中④ Estimate the target threat based on the above-mentioned plane angle safety envelope, and establish a threat marking function l k based on time k. When l k <0, it is considered that there is a collision threat, so as to judge whether there is a collision threat, where

其中表示相机B的主轴方向相对于无人机主轴方向的角度,通常可认为是符号函数;in Indicates the angle of the main axis direction of camera B relative to the main axis direction of the UAV, which can usually be regarded as is a sign function;

⑤当lk<0时,根据最小规避距离原则,在平面角度安全包络寻找距离最近的点s*=[σ**]作为最小规避机动点:⑤When l k <0, according to the principle of the minimum avoidance distance, find the distance in the plane angle safety envelope The nearest point s * = [σ ** ] as the minimum avoidance maneuver point:

其中代表k时刻垂直于目标角速度向量的单位向量。in Represents the unit vector perpendicular to the target angular velocity vector at time k.

作为又一种优选方式,所述步骤四中规避机动俯仰和偏航角度的具体计算步骤如下:As yet another preferred manner, the specific calculation steps of avoiding maneuver pitch and yaw angles in the step 4 are as follows:

①定义交互矩阵为① Define the interaction matrix as

在视觉伺服控制系统的视觉伺服控制器中建立速度控制器,反馈量为完成误差的指数衰减,输出速度控制量V,视觉伺服控制器表达为The speed controller is established in the visual servo controller of the visual servo control system, and the feedback amount is The exponential attenuation of the error is completed, and the output speed control value V is expressed as

式中,λ为误差指数衰减系数,表示Le的伪逆;In the formula, λ is the error exponential decay coefficient, Represents the pseudo-inverse of L e ;

由于在交互矩阵Le中无法获取目标与本机的距离d的信息,将交互矩阵进行分割如下:Since the information of the distance d between the target and the machine cannot be obtained in the interaction matrix L e , the interaction matrix is divided as follows:

式中,Lω表示Le中的角度控制分量,Lt表示速度运动控制分量;In the formula, L ω represents the angle control component in L e , and L t represents the speed motion control component;

得到距离无关的视觉伺服控制器:Get a distance-independent visual servo controller:

式中,表示Lω的伪逆;In the formula, represents the pseudo-inverse of L ω ;

为消除稳态误差,对视觉伺服控制器进行补偿,如下:In order to eliminate the steady-state error, the visual servo controller is compensated as follows:

式中,Ω表示目标运动的角速度,表示误差的方向向量,ωx、ωy、ωz分别表示ω的三个分量;In the formula, Ω represents the angular velocity of the target movement, Indicates the direction vector of the error, ω x , ω y , ω z represent the three components of ω respectively;

②输出的俯仰角和偏航角为②The output pitch angle and yaw angle are

式中,和φc(k)表示自动驾驶仪输出姿态的俯仰角和偏航角,和φ(k)表示无人机当前姿态的俯仰角和偏航角,Δt表示视觉伺服控制器输出时间间隔步长,ωy(k)和ωz(k)分别表示当前时刻视觉伺服控制器输出的俯仰角速度控制量和偏航角速度控制量,g表示重力加速度,V(k)表示当前时刻无人机的速度。In the formula, and φ c (k) represent the pitch and yaw angles of the autopilot output attitude, and φ(k) represent the pitch angle and yaw angle of the current attitude of the UAV, Δt represents the output time interval step of the visual servo controller, ω y (k) and ω z (k) represent the visual servo controller at the current moment The output pitch rate control amount and yaw rate control amount, g represents the acceleration of gravity, and V(k) represents the speed of the drone at the current moment.

作为再一种优选方式,所述步骤五中航路回归机制的具体建立步骤如下:As another preferred mode, the specific steps of establishing the route return mechanism in the step five are as follows:

①通过本机导航定位系统确定无人机是否在航路中,当本机不在航路时,进行航路回归;①Use the navigation and positioning system of the aircraft to determine whether the UAV is on the route, and return to the route when the aircraft is not on the route;

②确认时刻k的回归航路角度②Confirm the return route angle at time k

式中,代表了航路回归时的最小机动规避点位置;In the formula, Represents the position of the minimum maneuver avoidance point when the route returns;

其中Pd是当不存在规避机动时,本机在原本航路上的位置,Pd12和Pd13分别是本机在XY和XZ平面的二维位置;Where P d is the position of the own aircraft on the original route when there is no evasive maneuver, P d12 and P d13 are the two-dimensional positions of the own aircraft on the XY and XZ planes respectively;

Ps是通过本机导航定位系统获得的本机当前位置,Ps12和Ps13分别是本机在XY和XZ平面的二维位置;P s is the current position of the machine obtained through the local navigation and positioning system, and P s12 and P s13 are the two-dimensional positions of the machine on the XY and XZ planes respectively;

es是通过本机导航定位系统获得的本机方向向量,es12和es13分别是本机在XY和XZ平面的二维方向向量。e s is the local direction vector obtained by the local navigation and positioning system, and e s12 and e s13 are the two-dimensional direction vectors of the local plane on the XY and XZ planes, respectively.

本发明实现了对周围空域环境的快速感知,以及在不加装任何测距传感器且不需地面操作人员进行干预和操作的情况下,独立自主完成对空间飞行目标的规避机动;本发明载荷要求低、控制精度高、具有较高的智能性,能够提高无人机的空域飞行安全能力。The invention realizes the rapid perception of the surrounding airspace environment, and independently completes the evasive maneuver of the space flight target without installing any ranging sensor and without the intervention and operation of ground operators; the invention has low load requirements , high control precision, and high intelligence, which can improve the airspace flight safety capability of UAVs.

附图说明Description of drawings

图1为本发明的基于视觉伺服的感知与规避系统结构框图;Fig. 1 is a structural block diagram of the perception and avoidance system based on visual servoing of the present invention;

图2为本发明的基于视觉伺服的感知与规避系统的规避方法流程图;FIG. 2 is a flow chart of the avoidance method of the visual servo-based perception and avoidance system of the present invention;

图3a为本发明的图像采集系统相机阵列布置图;Figure 3a is a layout diagram of the camera array of the image acquisition system of the present invention;

图3b为本发明的图像采集系统相机阵列视场角度示意图;Fig. 3b is a schematic diagram of the field of view angle of the camera array of the image acquisition system of the present invention;

图4为本发明的图像平面角度安全包络示意图。FIG. 4 is a schematic diagram of an image plane angle security envelope of the present invention.

具体实施方式detailed description

下面结合附图和实施例对本发明做进一步说明。The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

如图1所示,本发明提供一种基于视觉伺服的感知与规避系统,包括:As shown in Figure 1, the present invention provides a perception and avoidance system based on visual servoing, including:

无人机,所述无人机上设置自动驾驶仪,所述自动驾驶仪接收飞行控制指令进行飞行并完成无人机对障碍物的规避动作;An unmanned aerial vehicle, an autopilot is set on the unmanned aerial vehicle, and the automatic pilot receives flight control instructions to fly and complete the avoidance action of the unmanned aerial vehicle to obstacles;

图像采集系统,搭载在所述无人机上,用于采集无人机的飞行空域图像信息,采用多相机多视角集中式阵列构型,并通过相机接口实时传输图像信息;The image acquisition system is mounted on the unmanned aerial vehicle and is used to acquire the flight airspace image information of the unmanned aerial vehicle, adopts a multi-camera multi-view centralized array configuration, and transmits image information in real time through the camera interface;

视觉伺服控制系统,设置在所述无人机上,包括视觉目标检测与跟踪模块、安全包络模块和视觉伺服控制器,视觉目标检测与跟踪模块接收图像采集系统传输的图像信息并实现对图像中飞行目标的检测、定位和跟踪,安全包络模块根据获取的目标状态信息进行图像平面角度安全包络的生成,并基于该安全包络生成图像规避角度,以此角度输入至视觉伺服控制器,视觉伺服控制器输出规避机动角度,并将该角度通过串口传输至自动驾驶仪完成规避动作;The visual servo control system is arranged on the unmanned aerial vehicle and includes a visual target detection and tracking module, a safety envelope module and a visual servo controller. The visual target detection and tracking module receives the image information transmitted by the image acquisition system and realizes the detection For the detection, positioning and tracking of flying targets, the safety envelope module generates the image plane angle safety envelope based on the acquired target state information, and generates image avoidance angles based on the safety envelope, and inputs this angle to the visual servo controller. The visual servo controller outputs the evasive maneuvering angle, and transmits the angle to the autopilot through the serial port to complete the evasive action;

导航定位系统,设置在所述无人机上,用于建立航路回归机制,导航定位系统检测到无人机不在航路且不存在威胁时,进行航路回归,使无人机返回原始航路。The navigation and positioning system is set on the UAV and is used to establish a route return mechanism. When the navigation and positioning system detects that the UAV is not on the route and there is no threat, it performs route regression to make the UAV return to the original route.

其中,本发明中视觉目标检测与跟踪模块采用多路高清图像处理平台DM8168。Among them, the visual target detection and tracking module in the present invention adopts the multi-channel high-definition image processing platform DM8168.

如图2所示,本发明还提供一种基于视觉伺服的感知与规避系统的规避方法,包括:As shown in Figure 2, the present invention also provides a method for avoiding the perception and avoidance system based on visual servoing, including:

步骤一,通过无人机机载图像采集系统实现对飞行视场角内的空域的完全覆盖,获取无人机的飞行空域图像信息。Step 1: Complete coverage of the airspace within the flight field of view is achieved through the UAV airborne image acquisition system, and image information of the UAV's flight airspace is acquired.

根据国际民航组织规定,飞行器空间的视场角必须分别满足:水平视场>±110°,垂直视场>±15°,因此必须对上述空域能够进行完全覆盖,且应该尽可能的获取清晰准确的信息。图像采集系统采用多相机多视角集中式阵列构型,从相机的角度来讲,即必须满足大市场和远距离高分辨成像,而单一的相机镜头很难在视场角和焦距方面兼顾。因此,采用如图3a所示的多相机集中式阵列构型,图中A、B、C均为设置在增稳云台上的相机,增稳云台能够保证清晰、水平视角图像的获取。利用多个长焦镜头的焦距拼接获得大的视场角度,具体视场角度如图3b所示,同时提高对远距离目标的探测能力。According to the regulations of the International Civil Aviation Organization, the field of view of the aircraft space must meet the following requirements: horizontal field of view>±110°, vertical field of view>±15°, so the above-mentioned airspace must be completely covered, and it should be as clear and accurate as possible Information. The image acquisition system adopts a multi-camera multi-view centralized array configuration. From the camera point of view, it must meet the needs of a large market and long-distance high-resolution imaging. However, it is difficult for a single camera lens to balance the field of view and focal length. Therefore, a multi-camera centralized array configuration as shown in Figure 3a is adopted. A, B, and C in the figure are all cameras set on the stabilized gimbal. The stabilized gimbal can ensure the acquisition of clear, horizontal perspective images. Using the focal length stitching of multiple telephoto lenses to obtain a large field of view angle, the specific field of view angle is shown in Figure 3b, and at the same time improve the detection ability of long-distance targets.

步骤二,机载视觉伺服控制系统中视觉目标检测与跟踪模块接收步骤一中图像信息并进行处理,完成对空域图像中飞行目标的检测、定位和跟踪。In step two, the visual target detection and tracking module in the airborne visual servo control system receives and processes the image information in step one, and completes the detection, positioning and tracking of the flying target in the airspace image.

在轻小型无人机中,由于通信带宽的限制,往往不能实现对高清图像的实时传播,多路高清视频的远距离传输在当前的轻小型无人机系统中很难实现。In light and small UAVs, due to the limitation of communication bandwidth, it is often impossible to realize real-time transmission of high-definition images, and the long-distance transmission of multiple high-definition videos is difficult to achieve in the current light and small UAV systems.

图像采集系统获取的图像数据通过相机数据线传送至机载视觉伺服控制系统中多路高清图像处理平台DM8168,利用DM8168实现对获取的多路视频进行处理,通过检测算法获取目标k时刻在图像采集系统中某一相机图像平面的位置信息pk=[pxk,pyk],获得目标相对于该相机的相对角度:The image data acquired by the image acquisition system is transmitted to the multi-channel high-definition image processing platform DM8168 in the airborne visual servo control system through the camera data line, and the DM8168 is used to process the acquired multi-channel video, and the target k time is acquired through the detection algorithm. The position information of a certain camera image plane in the system p k =[p xk ,p yk ], to obtain the relative angle of the target relative to the camera:

其中,w和h分别是以像素为单位的图像的宽度和高度,f为相机镜头焦距,μ为象元尺寸;Among them, w and h are the width and height of the image in pixels, respectively, f is the focal length of the camera lens, and μ is the pixel size;

根据相机安装位置,可获得目标k时刻相对于无人机的相对角度为Θk=[σkk],其中According to the installation position of the camera, the relative angle of the target relative to the UAV at moment k can be obtained as Θ k = [σ kk ], where

γk=γ′γ k = γ′

式中分别表示检测到的目标来自于图像采集系统中的相机A、相机B和相机C。In the formula Respectively indicate that the detected targets come from camera A, camera B and camera C in the image acquisition system.

根据前后时刻的角度变化确定目标的角速度Ωk=[ωxkyk]=Θk+1k,k>0。Determine the angular velocity of the target Ω k =[ω xkyk ]=Θ k+1k ,k>0 according to the angular changes at the previous and subsequent moments.

步骤三,根据步骤二中获取的飞行目标状态信息,视觉伺服控制系统中安全包络模块进行图像平面角度安全包络生成,通过安全包络确定无人机是否存在碰撞威胁,针对碰撞威胁,计算得到图像意义下的规避角度s*=[σ**]。Step 3: According to the state information of the flight target obtained in step 2, the safety envelope module in the visual servo control system generates the image plane angle safety envelope, and determines whether there is a collision threat to the UAV through the safety envelope. For the collision threat, calculate The avoidance angle s * = [σ ** ] in the image sense is obtained.

安全包络定义为表示飞行器与目标最小分离距离的光滑、封闭曲线或曲面,在视觉规避系统中,由于距离无法获取,因此无法根据目标的真实三维空间建立安全包络,如图4所示,本发明中建立一种基于角度的安全包络方法,包括如下步骤:The safety envelope is defined as a smooth, closed curve or surface that represents the minimum separation distance between the aircraft and the target. In the visual avoidance system, since the distance cannot be obtained, the safety envelope cannot be established according to the real three-dimensional space of the target, as shown in Figure 4. Set up a kind of security envelope method based on angle among the present invention, comprise the steps:

①建立目标安全包络圆,圆心为目标点,选取一个半径初始值,并以固定速度进行增长;①Establish the target safety enveloping circle, the center of which is the target point, select an initial value of the radius, and grow at a fixed speed;

②根据目标尺寸建立安全包络椭圆,椭圆圆心为目标点,椭圆短轴的长度与包络圆的半径相等,椭圆长轴的长度为短轴的长度与目标在当前角速度下在预留时间内的外推长度之和,长轴方向平行于目标的运动方向;②Create a safe envelope ellipse according to the size of the target. The center of the ellipse is the target point. The length of the minor axis of the ellipse is equal to the radius of the envelope circle. The sum of the extrapolated lengths of , the long axis direction is parallel to the moving direction of the target;

③前述生成的包络圆和包络椭圆形成平面角度安全包络,该安全包络由以椭圆短轴为分割线的沿目标运动方向的半椭圆与沿目标运动反方向的半圆组成;③ The aforementioned generated envelope circle and envelope ellipse form a plane angle safety envelope, which is composed of a semi-ellipse along the direction of target motion and a semi-circle along the opposite direction of target motion with the short axis of the ellipse as the dividing line;

④基于前述平面角度安全包络进行目标威胁估计,建立基于安全包络的威胁标示函数,以此判断是否存在碰撞威胁;④ Estimate the target threat based on the aforementioned plane angle safety envelope, and establish a threat marking function based on the safety envelope to judge whether there is a collision threat;

⑤根据最小规避距离原则,在平面角度安全包络寻找距离初始目标最近的点。⑤ According to the principle of minimum avoidance distance, find the point closest to the initial target in the plane angle safety envelope.

上述建立基于角度的安全包络方法可依据如下具体步骤进行:The above method for establishing an angle-based security envelope can be carried out according to the following specific steps:

①建立目标安全包络圆,圆心选为目标k时刻相对于无人机的相对角度位置Θk=[σkk],选取一个半径初始值r0,并以固定速度进行增长,则①Establish the target safety enveloping circle, select the center of the circle as the relative angular position Θ k =[σ kk ] of the target relative to the UAV at moment k, select an initial value r 0 of the radius, and increase it at a fixed speed, then

rk+1=rk+ε,k=1,2,3…r k+1 =r k +ε,k=1,2,3...

rk=r0,k=0r k = r 0 , k = 0

其中ε是固定增长速度,用来模拟三维空间物体在图像平面的扩张速度,rk+1代表k+1时刻的安全包络圆半径,图4中的r表示包络圆的半径;Where ε is a fixed growth rate, which is used to simulate the expansion speed of a three-dimensional space object on the image plane, r k+1 represents the radius of the safe envelope circle at the moment k+1, and r in Figure 4 represents the radius of the envelope circle;

②根据目标尺寸建立安全包络椭圆,椭圆圆心为目标点Θk=[σkk],长轴方向平行于目标的运动方向,即目标角速度Ωk的方向,图4中的a和b分别表示包络椭圆的长轴和短轴,长轴和短轴的长度分别按下列公式选取:② Establish a safety envelope ellipse according to the size of the target. The center of the ellipse is the target point Θ k = [σ k , γ k ], and the major axis direction is parallel to the moving direction of the target, which is the direction of the target angular velocity Ω k . b represent the major axis and the minor axis of the envelope ellipse respectively, and the lengths of the major axis and the minor axis are selected according to the following formulas:

a=rk(1+α‖Ωk‖),a=r k (1+α‖Ω k ‖),

b=rkb=r k ,

式中,a和b分别代表包络椭圆的长轴和短轴,rk为k时刻包络圆的半径,α为角速度转化为轴长的比例因子,该因子代表规避预留时间,‖Ωk‖为目标运动角速度向量的模,即长轴的长度为短轴的长度与目标在当前角速度下在预留时间内的外推长度之和;In the formula, a and b represent the major axis and minor axis of the envelope ellipse respectively, r k is the radius of the envelope circle at time k, α is the scaling factor for converting angular velocity into axis length, which represents the reserved time for avoidance, ‖Ω k ‖ is the modulus of the target motion angular velocity vector, that is, the length of the major axis is the sum of the length of the minor axis and the extrapolated length of the target within the reserved time under the current angular velocity;

③前述生成的包络圆和包络椭圆形成平面角度安全包络,该安全包络由以椭圆短轴为分割线的沿目标运动方向的半椭圆与沿目标运动反方向的半圆组成,具体如图4中实线部分所示;③ The envelope circle and envelope ellipse generated above form a plane angle safety envelope, which is composed of a semi-ellipse along the target movement direction and a semi-circle along the opposite direction of the target movement with the short axis of the ellipse as the dividing line, specifically as follows Shown in the solid line part in Fig. 4;

该安全包络既考虑了由于目标接近和联测误差存在造成的规避角度余量,同时安全包络也考虑了目标的运动状态;The safety envelope not only considers the avoidance angle margin due to the target approach and joint measurement error, but also considers the target's motion state;

④基于前述平面角度安全包络进行目标威胁估计,建立基于时间k的威胁标示函数lk,当lk<0时,认为存在碰撞威胁,以此判断是否存在碰撞威胁,其中④ Estimate the target threat based on the above-mentioned plane angle safety envelope, and establish a threat marking function l k based on time k. When l k <0, it is considered that there is a collision threat, so as to judge whether there is a collision threat, where

其中表示相机B的主轴方向相对于无人机主轴方向的角度,通常可认为是符号函数;in Indicates the angle of the main axis direction of camera B relative to the main axis direction of the UAV, which can usually be regarded as is a sign function;

⑤当lk<0时,根据最小规避距离原则,在平面角度安全包络寻找距离最近的点s*=[σ**]作为最小规避机动点:⑤When l k <0, according to the principle of the minimum avoidance distance, find the distance in the plane angle safety envelope The nearest point s * = [σ ** ] as the minimum avoidance maneuver point:

其中代表k时刻垂直于目标角速度向量的单位向量。in Represents the unit vector perpendicular to the target angular velocity vector at time k.

步骤四,将步骤三中的规避角度s*=[σ**]输入视觉伺服控制系统中的视觉伺服器,视觉伺服器输出需要的规避机动俯仰和偏航角度并通过串口传送至无人机自动驾驶仪完成规避机动。Step 4, input the avoidance angle s * = [σ * , γ * ] in step 3 into the visual servo in the visual servo control system, and the visual servo outputs the required evasive maneuver pitch and yaw angles And send it to the UAV autopilot through the serial port to complete the evasive maneuver.

①定义交互矩阵为① Define the interaction matrix as

在视觉伺服控制系统的视觉伺服控制器中建立速度控制器,反馈量为完成误差的指数衰减,输出速度控制量V,视觉伺服控制器表达为The speed controller is established in the visual servo controller of the visual servo control system, and the feedback amount is The exponential attenuation of the error is completed, and the output speed control value V is expressed as

式中,λ为误差指数衰减系数,表示Le的伪逆;In the formula, λ is the error exponential decay coefficient, Represents the pseudo-inverse of L e ;

由于在交互矩阵Le中无法获取目标与本机的距离d的信息,将交互矩阵进行分割如下:Since the information of the distance d between the target and the machine cannot be obtained in the interaction matrix L e , the interaction matrix is divided as follows:

式中,Lω表示Le中的角度控制分量,Lt表示速度运动控制分量;In the formula, L ω represents the angle control component in L e , and L t represents the speed motion control component;

得到距离无关的视觉伺服控制器:Get a distance-independent visual servo controller:

式中,表示Lω的伪逆;In the formula, represents the pseudo-inverse of L ω ;

为消除稳态误差,对视觉伺服控制器进行补偿,如下:In order to eliminate the steady-state error, the visual servo controller is compensated as follows:

式中,Ω表示目标运动的角速度,表示误差的方向向量,ωx、ωy、ωz分别表示ω的三个分量;In the formula, Ω represents the angular velocity of the target movement, Indicates the direction vector of the error, ω x , ω y , ω z represent the three components of ω respectively;

②输出的俯仰角和偏航角为②The output pitch angle and yaw angle are

式中,和φc(k)表示自动驾驶仪输出姿态的俯仰角和偏航角,和φ(k)表示无人机当前姿态的俯仰角和偏航角,Δt表示视觉伺服控制器输出时间间隔步长,ωy(k)和ωz(k)分别表示当前时刻视觉伺服控制器输出的俯仰角速度控制量和偏航角速度控制量,g表示重力加速度,V(k)表示当前时刻无人机的速度。In the formula, and φ c (k) represent the pitch and yaw angles of the autopilot output attitude, and φ(k) represent the pitch angle and yaw angle of the current attitude of the UAV, Δt represents the output time interval step of the visual servo controller, ω y (k) and ω z (k) represent the visual servo controller at the current moment The output pitch rate control amount and yaw rate control amount, g represents the acceleration of gravity, and V(k) represents the speed of the drone at the current moment.

步骤五,利用导航定位系统建立航路回归机制,当无人机不存在威胁时,使无人机返回原始航路。Step five, use the navigation and positioning system to establish a route return mechanism, and when the UAV is not threatened, the UAV will return to the original route.

①通过本机导航定位系统确定无人机是否在航路中,当本机不在航路时,进行航路回归;①Use the navigation and positioning system of the aircraft to determine whether the UAV is on the route, and return to the route when the aircraft is not on the route;

②确认时刻k的回归航路角度②Confirm the return route angle at time k

式中,代表了航路回归时的最小机动规避点位置;In the formula, Represents the position of the minimum maneuver avoidance point when the route returns;

其中Pd是当不存在规避机动时,本机在原本航路上的位置,Pd12和Pd13分别是本机在XY和XZ平面的二维位置;Where P d is the position of the own aircraft on the original route when there is no evasive maneuver, P d12 and P d13 are the two-dimensional positions of the own aircraft on the XY and XZ planes respectively;

Ps是通过本机导航定位系统获得的本机当前位置,Ps12和Ps13分别是本机在XY和XZ平面的二维位置;P s is the current position of the machine obtained through the local navigation and positioning system, and P s12 and P s13 are the two-dimensional positions of the machine on the XY and XZ planes respectively;

es是通过本机导航定位系统获得的本机方向向量,es12和es13分别是本机在XY和XZ平面的二维方向向量。e s is the local direction vector obtained by the local navigation and positioning system, and e s12 and e s13 are the two-dimensional direction vectors of the local plane on the XY and XZ planes, respectively.

本发明提供的方法及系统能够实现在不加装任何测距传感器的情况下,在没有操作人员干预的情况下完成对空间飞行目标的规避机动,载荷要求低、控制精度高、具有较高的智能性,能够提高无人机的空域飞行安全能力。The method and system provided by the present invention can realize the evasive maneuver of the space flight target without installing any ranging sensor and without the intervention of the operator, with low load requirements, high control precision, and high Intelligence can improve the airspace flight safety capability of drones.

Claims (7)

1.一种基于视觉伺服的感知与规避系统,其特征在于,包括:1. A perception and avoidance system based on visual servoing, characterized in that it comprises: 无人机,所述无人机上设置自动驾驶仪,所述自动驾驶仪接收飞行控制指令进行飞行并完成无人机对障碍物的规避动作;An unmanned aerial vehicle, an autopilot is set on the unmanned aerial vehicle, and the automatic pilot receives flight control instructions to fly and complete the avoidance action of the unmanned aerial vehicle to obstacles; 图像采集系统,搭载在所述无人机上,用于采集无人机的飞行空域图像信息,采用多相机多视角集中式阵列构型,并通过相机接口实时传输图像信息;The image acquisition system is mounted on the unmanned aerial vehicle and is used to acquire the flight airspace image information of the unmanned aerial vehicle, adopts a multi-camera multi-view centralized array configuration, and transmits image information in real time through the camera interface; 视觉伺服控制系统,设置在所述无人机上,包括视觉目标检测与跟踪模块、安全包络模块和视觉伺服控制器,视觉目标检测与跟踪模块接收图像采集系统传输的图像信息并实现对图像中飞行目标的检测、定位和跟踪,安全包络模块根据获取的目标状态信息进行图像平面角度安全包络的生成,并基于该安全包络生成图像规避角度,以此角度输入至视觉伺服控制器,视觉伺服控制器输出规避机动角度,并将该角度通过串口传输至自动驾驶仪完成规避动作;The visual servo control system is arranged on the unmanned aerial vehicle and includes a visual target detection and tracking module, a safety envelope module and a visual servo controller. The visual target detection and tracking module receives the image information transmitted by the image acquisition system and realizes the detection For the detection, positioning and tracking of flying targets, the safety envelope module generates the image plane angle safety envelope based on the acquired target state information, and generates image avoidance angles based on the safety envelope, and inputs this angle to the visual servo controller. The visual servo controller outputs the evasive maneuvering angle, and transmits the angle to the autopilot through the serial port to complete the evasive action; 导航定位系统,设置在所述无人机上,用于建立航路回归机制,导航定位系统检测到无人机不在航路且不存在威胁时,进行航路回归,使无人机返回原始航路。The navigation and positioning system is set on the UAV and is used to establish a route return mechanism. When the navigation and positioning system detects that the UAV is not on the route and there is no threat, it performs route regression to make the UAV return to the original route. 2.根据权利要求1所述的基于视觉伺服的感知与规避系统,其特征在于,所述视觉目标检测与跟踪模块采用多路高清图像处理平台DM8168。2. The perception and avoidance system based on visual servoing according to claim 1, wherein the visual target detection and tracking module adopts a multi-channel high-definition image processing platform DM8168. 3.一种基于视觉伺服的感知与规避系统的规避方法,其特征在于,包括:3. A method for avoiding the perception and avoidance system based on visual servoing, characterized in that it comprises: 步骤一,通过无人机机载图像采集系统实现对飞行视场角内的空域的完全覆盖,获取无人机的飞行空域图像信息;Step 1, realize the complete coverage of the airspace within the flight field of view through the UAV airborne image acquisition system, and obtain the flight airspace image information of the UAV; 步骤二,机载视觉伺服控制系统中视觉目标检测与跟踪模块接收所述步骤一中图像信息并进行处理,完成对空域图像中飞行目标的检测、定位和跟踪;Step 2, the visual target detection and tracking module in the airborne visual servo control system receives and processes the image information in the step 1, and completes the detection, positioning and tracking of the flying target in the airspace image; 所述步骤二的具体实现方法如下:The concrete realization method of described step 2 is as follows: 所述步骤一中图像采集系统获取的图像数据通过相机数据线传送至机载视觉伺服控制系统中视觉目标检测与跟踪模块,其中图像采集系统中共设置A、B、C三台相机,视觉目标检测与跟踪模块采用多路高清图像处理平台DM8168,利用DM8168实现对获取的多路视频进行处理,通过检测算法获取目标k时刻在图像采集系统中某一相机图像平面的位置信息pk=[pxk,pyk],获得目标相对于该相机的相对角度:The image data acquired by the image acquisition system in the step 1 is transmitted to the visual target detection and tracking module in the airborne visual servo control system through the camera data line, wherein the image acquisition system is equipped with three cameras A, B, and C, and the visual target detection The tracking and tracking module adopts the multi-channel high-definition image processing platform DM8168, utilizes the DM8168 to realize the processing of the acquired multi-channel video, and obtains the position information p k of a certain camera image plane in the image acquisition system at the moment k of the target through the detection algorithm p k =[p x k ,p yk ], get the relative angle of the target relative to the camera: &sigma;&sigma; &prime;&prime; == tanthe tan -- 11 pp xx kk -- ww 22 ff // &mu;&mu; &gamma;&gamma; &prime;&prime; == tanthe tan -- 11 pp ythe y kk -- hh 22 ff // &mu;&mu; 其中,w和h分别是以像素为单位的图像的宽度和高度,f为相机镜头焦距,μ为象元尺寸;Among them, w and h are the width and height of the image in pixels, respectively, f is the focal length of the camera lens, and μ is the pixel size; 根据相机安装位置,可获得目标k时刻相对于无人机的相对角度为Θk=[σkk],其中According to the installation position of the camera, the relative angle of the target relative to the UAV at moment k can be obtained as Θ k = [σ kk ], where &sigma;&sigma; kk == &sigma;&sigma; &prime;&prime; -- 44 99 &pi;&pi; ,, pp kk &Subset;&Subset; AA &sigma;&sigma; &prime;&prime; ,, pp kk &Subset;&Subset; BB &sigma;&sigma; &prime;&prime; ++ 44 99 &pi;&pi; ,, pp kk &Subset;&Subset; CC γk=γ′γ k = γ′ 式中分别表示检测到的目标来自于图像采集系统中的相机A、相机B和相机C;In the formula Respectively indicate that the detected target comes from camera A, camera B and camera C in the image acquisition system; 根据前后时刻的角度变化确定目标的角速度Ωk=[ωxkyk]=Θk+1k,k>0;Determine the angular velocity of the target Ω k =[ω xkyk ]=Θ k+1k ,k>0 according to the angle changes at the front and rear moments; 步骤三,根据所述步骤二中获取的飞行目标状态信息,视觉伺服控制系统中安全包络模块进行图像平面角度安全包络生成,通过安全包络确定无人机是否存在碰撞威胁,针对碰撞威胁,计算得到图像意义下的规避角度;Step 3, according to the state information of the flight target obtained in the step 2, the safety envelope module in the visual servo control system performs image plane angle safety envelope generation, and determines whether the UAV has a collision threat through the safety envelope, and aims at the collision threat , calculate the avoidance angle in the image sense; 步骤四,将所述步骤三中的规避角度输入视觉伺服控制系统中的视觉伺服器,视觉伺服器输出需要的规避机动俯仰和偏航角度,并传送至无人机自动驾驶仪完成规避机动;Step 4, input the evasive angle in the step 3 into the visual servo in the visual servo control system, the visual servo outputs the required evasive maneuver pitch and yaw angle, and transmits to the UAV autopilot to complete the evasive maneuver; 步骤五,利用导航定位系统建立航路回归机制,当无人机不存在威胁时,使无人机返回原始航路。Step five, use the navigation and positioning system to establish a route return mechanism, and when there is no threat to the drone, make the drone return to the original route. 4.根据权利要求3所述的规避方法,其特征在于,所述步骤三中安全包络的生成步骤及规避角度的计算步骤包括:4. avoidance method according to claim 3, is characterized in that, the generation step of safety envelope and the calculating step of avoidance angle in described step 3 comprise: ①建立目标安全包络圆,圆心为目标点,选取一个半径初始值,并以固定速度进行增长;①Establish the target safety enveloping circle, with the center of the circle as the target point, select an initial value of the radius, and grow at a fixed speed; ②根据目标尺寸建立安全包络椭圆,椭圆圆心为目标点,椭圆短轴的长度与包络圆的半径相等,椭圆长轴的长度为短轴的长度与目标在当前角速度下在预留时间内的外推长度之和,长轴方向平行于目标的运动方向;②Create a safe envelope ellipse according to the size of the target. The center of the ellipse is the target point. The length of the minor axis of the ellipse is equal to the radius of the envelope circle. The sum of the extrapolated lengths of , the long axis direction is parallel to the moving direction of the target; ③前述生成的包络圆和包络椭圆形成平面角度安全包络,该安全包络由以椭圆短轴为分割线的沿目标运动方向的半椭圆与沿目标运动反方向的半圆组成;③ The aforementioned generated envelope circle and envelope ellipse form a plane angle safety envelope, which is composed of a semi-ellipse along the direction of target motion and a semi-circle along the opposite direction of target motion with the short axis of the ellipse as the dividing line; ④基于前述平面角度安全包络进行目标威胁估计,建立基于安全包络的威胁标示函数,以此判断是否存在碰撞威胁;④ Estimate the target threat based on the aforementioned plane angle safety envelope, and establish a threat marking function based on the safety envelope to judge whether there is a collision threat; ⑤根据最小规避距离原则,在平面角度安全包络寻找距离初始目标最近的点。⑤ According to the principle of minimum avoidance distance, find the point closest to the initial target in the plane angle safety envelope. 5.根据权利要求4所述的规避方法,其特征在于,所述步骤三中安全包络的具体生成步骤及规避角度的具体计算步骤如下:5. avoidance method according to claim 4, is characterized in that, the concrete generation step of safety envelope and the concrete calculation procedure of avoidance angle in described step 3 are as follows: ①建立目标安全包络圆,圆心选为目标k时刻相对于无人机的相对角度位置Θk=[σkk],选取一个半径初始值r0,并以固定速度进行增长,则①Establish the target safety enveloping circle, select the center of the circle as the relative angular position Θ k =[σ kk ] of the target relative to the UAV at moment k, select an initial value r 0 of the radius, and increase it at a fixed speed, then rk+1=rk+ε,k=1,2,3…r k+1 =r k +ε,k=1,2,3... rk=r0,k=0r k = r 0 , k = 0 其中ε是安全包络圆半径的固定增长长度,rk+1代表k+1时刻的安全包络圆半径;Where ε is the fixed growth length of the radius of the safety envelope circle, and r k+1 represents the radius of the safety envelope circle at time k+1; ②根据目标尺寸建立安全包络椭圆,椭圆圆心为目标点Θk=[σkk],长轴方向平行于目标的运动方向,即目标角速度Ωk的方向,长轴和短轴的长度分别按下列公式选取:② Establish a safety envelope ellipse according to the size of the target, the center of the ellipse is the target point Θ k = [σ k , γ k ], the long axis direction is parallel to the target’s movement direction, that is, the direction of the target angular velocity Ω k , the long axis and the short axis The length is selected according to the following formula: a=rk(1+α‖Ωk‖),a=r k (1+α‖Ω k ‖), b=rkb=r k , 式中,a和b分别代表包络椭圆的长轴和短轴,rk为k时刻包络圆的半径,α为角速度转化为轴长的比例因子,该因子代表规避预留时间,‖Ωk‖为目标运动角速度向量的模;In the formula, a and b represent the major axis and minor axis of the envelope ellipse respectively, r k is the radius of the envelope circle at time k, α is the scaling factor for converting angular velocity into axis length, which represents the reserved time for avoidance, ‖Ω k ‖ is the modulus of the target motion angular velocity vector; ③前述生成的包络圆和包络椭圆形成平面角度安全包络,该安全包络由以椭圆短轴为分割线的沿目标运动方向的半椭圆与沿目标运动反方向的半圆组成;③ The aforementioned generated envelope circle and envelope ellipse form a plane angle safety envelope, which is composed of a semi-ellipse along the direction of target motion and a semi-circle along the opposite direction of target motion with the short axis of the ellipse as the dividing line; ④基于前述平面角度安全包络进行目标威胁估计,建立基于时间k的威胁标示函数lk,当lk<0时,存在碰撞威胁,以此判断是否存在碰撞威胁,其中④ Estimate the target threat based on the above-mentioned plane angle safety envelope, and establish a threat marking function l k based on time k. When l k <0, there is a collision threat, so as to judge whether there is a collision threat, where ll kk == sgnsgn (( (( &sigma;&sigma; ~~ -- &sigma;&sigma; kk )) 22 ++ (( &gamma;&gamma; ~~ -- &gamma;&gamma; kk )) 22 -- rr kk )) (( &Theta;&Theta; ~~ -- &Theta;&Theta; kk )) &CenterDot;&CenterDot; &Omega;&Omega; kk << 00 sgnsgn (( (( &sigma;&sigma; ~~ -- &sigma;&sigma; kk )) 22 aa ++ (( &gamma;&gamma; ~~ -- &gamma;&gamma; kk )) 22 bb -- 11 )) (( &Theta;&Theta; ~~ -- &Theta;&Theta; kk )) &CenterDot;&Center Dot; &Omega;&Omega; kk &GreaterEqual;&Greater Equal; 00 ,, 其中表示相机B的主轴方向相对于无人机主轴方向的角度,其中,sgn(·)是符号函数;in Indicates the angle of the main axis direction of camera B relative to the main axis direction of the drone, where, sgn( ) is a symbolic function; ⑤当lk<0时,根据最小规避距离原则,在平面角度安全包络寻找距离最近的点s*=[σ**]作为最小规避机动点:⑤When l k <0, according to the principle of the minimum avoidance distance, find the distance in the plane angle safety envelope The nearest point s * = [σ ** ] as the minimum avoidance maneuver point: sthe s ** == &Theta;&Theta; kk ++ (( &Theta;&Theta; ~~ -- &Theta;&Theta; kk )) || || &Theta;&Theta; ~~ -- &Theta;&Theta; kk || || rr kk ,, (( &Theta;&Theta; ~~ -- &Theta;&Theta; kk )) &CenterDot;&CenterDot; &Omega;&Omega; kk << 00 &Theta;&Theta; kk &PlusMinus;&PlusMinus; ee &perp;&perp; &Omega;&Omega; kk rr kk ,, (( &Theta;&Theta; ~~ -- &Theta;&Theta; kk )) &CenterDot;&CenterDot; &Omega;&Omega; kk &GreaterEqual;&Greater Equal; 00 ,, 其中代表k时刻垂直于目标角速度向量的单位向量。in Represents the unit vector perpendicular to the target angular velocity vector at time k. 6.根据权利要求3所述的规避方法,其特征在于,所述步骤四中规避机动俯仰和偏航角度的具体计算步骤如下:6. The evasion method according to claim 3, characterized in that the specific calculation steps of the evasive maneuver pitch and yaw angle in the step 4 are as follows: ①定义交互矩阵为① Define the interaction matrix as LL ee == -- cos&sigma;cos&sigma; kk cos&gamma;cos&gamma; kk dd -- cos&sigma;cos&sigma; kk sin&gamma;sin&gamma; kk dd sin&sigma;sin&sigma; kk sin&gamma;sin&gamma; kk -- cos&gamma;cos&gamma; kk 00 sin&gamma;sin&gamma; kk dd sin&sigma;sin&sigma; kk -- cos&gamma;cos&gamma; kk dd sin&sigma;sin&sigma; kk 00 cos&gamma;cos&gamma; kk cos&sigma;cos&sigma; kk sin&sigma;sin&sigma; kk sin&gamma;sin&gamma; kk cos&sigma;cos&sigma; kk sin&sigma;sin&sigma; kk -- 11 ,, 在视觉伺服控制系统的视觉伺服控制器中建立速度控制器,反馈量为完成误差的指数衰减,输出速度控制量V,视觉伺服控制器表达为The speed controller is established in the visual servo controller of the visual servo control system, and the feedback amount is The exponential attenuation of the error is completed, and the output speed control value V is expressed as VV == &lsqb;&lsqb; vv ,, ww &rsqb;&rsqb; == -- &lambda;L&lambda;L ee ++ ee ,, 式中,λ为误差指数衰减系数,表示Le的伪逆;In the formula, λ is the error exponential decay coefficient, Represents the pseudo-inverse of L e ; 由于在交互矩阵Le中无法获取目标与本机的距离d的信息,将交互矩阵进行分割如下:Since the information of the distance d between the target and the machine cannot be obtained in the interaction matrix L e , the interaction matrix is divided as follows: LL ee == &lsqb;&lsqb; 11 dd LL tt ,, LL &omega;&omega; &rsqb;&rsqb; ,, 式中,Lω表示Le中的角度控制分量,Lt表示速度运动控制分量;In the formula, L ω represents the angle control component in L e , and L t represents the speed motion control component; 得到距离无关的视觉伺服控制器:Get a distance-independent visual servo controller: &omega;&omega; == -- &lambda;L&lambda;L &omega;&omega; ++ ee ,, 式中,表示Lω的伪逆;In the formula, represents the pseudo-inverse of L ω ; 为消除稳态误差,对视觉伺服控制器进行补偿,如下:In order to eliminate the steady-state error, the visual servo controller is compensated as follows: &omega;&omega; == -- &lambda;L&lambda;L &omega;&omega; ++ ee ++ LL &omega;&omega; ++ (( &Omega;&Omega; ++ sthe s ** || || sthe s ** || || &epsiv;&epsiv; )) == &omega;&omega; xx &omega;&omega; ythe y &omega;&omega; zz ,, 式中,Ω表示目标运动的角速度,表示误差的方向向量,ωx、ωy、ωz分别表示ω的三个分量;In the formula, Ω represents the angular velocity of the target movement, Indicates the direction vector of the error, ω x , ω y , ω z represent the three components of ω respectively; ②输出的俯仰角和偏航角为②The output pitch angle and yaw angle are &phi;&phi; cc (( kk )) == &phi;&phi; (( kk )) ++ tanthe tan VV (( kk )) &omega;&omega; zz (( kk )) gg 式中,和φc(k)表示自动驾驶仪输出姿态的俯仰角和偏航角,和φ(k)表示无人机当前姿态的俯仰角和偏航角,Δt表示视觉伺服控制器输出时间间隔步长,ωy(k)和ωz(k)分别表示当前时刻视觉伺服控制器输出的俯仰角速度控制量和偏航角速度控制量,g表示重力加速度,V(k)表示当前时刻无人机的速度。In the formula, and φ c (k) represent the pitch and yaw angles of the autopilot output attitude, and φ(k) represent the pitch angle and yaw angle of the current attitude of the UAV, Δt represents the output time interval step of the visual servo controller, ω y (k) and ω z (k) represent the visual servo controller at the current moment The output pitch rate control amount and yaw rate control amount, g represents the acceleration of gravity, and V(k) represents the speed of the drone at the current moment. 7.根据权利要求3所述的规避方法,其特征在于,所述步骤五中航路回归机制的具体建立步骤如下:7. The avoidance method according to claim 3, characterized in that, the specific steps of establishing the route return mechanism in the step 5 are as follows: ①通过本机导航定位系统确定无人机是否在航路中,当本机不在航路时,进行航路回归;① Determine whether the UAV is on the route through the local navigation and positioning system, and return to the route when the aircraft is not on the route; ②确认时刻k的回归航路角度②Confirm the return route angle at time k &gamma;&gamma; dd ** == coscos -- 11 (( PP dd 1212 (( kk )) -- PP sthe s 1212 )) &CenterDot;&Center Dot; ee sthe s 1212 || || PP dd 1212 -- PP sthe s 1212 || || ++ &pi;&pi; 22 ,, &sigma;&sigma; dd ** == coscos -- 11 (( PP dd 1313 -- PP sthe s 1313 )) &CenterDot;&Center Dot; ee sthe s 1313 || || PP dd 1313 -- PP sthe s 1313 || || ++ 33 &pi;&pi; 44 ,, 式中,代表了航路回归时的最小机动规避点位置;In the formula, Represents the position of the minimum maneuver avoidance point when the route returns; 其中Pd是当不存在规避机动时,本机在原本航路上的位置,Pd12和Pd13分别是本机在XY和XZ平面的二维位置;Where P d is the position of the own aircraft on the original route when there is no evasive maneuver, P d12 and P d13 are the two-dimensional positions of the own aircraft on the XY and XZ planes respectively; Ps是通过本机导航定位系统获得的本机当前位置,Ps12和Ps13分别是本机在XY和XZ平面的二维位置;P s is the current position of the machine obtained through the local navigation and positioning system, and P s12 and P s13 are the two-dimensional positions of the machine on the XY and XZ planes respectively; es是通过本机导航定位系统获得的本机方向向量,es12和es13分别是本机在XY和XZ平面的二维方向向量。e s is the local direction vector obtained by the local navigation and positioning system, and e s12 and e s13 are the two-dimensional direction vectors of the local plane on the XY and XZ planes, respectively.
CN201410305109.6A 2014-06-30 2014-06-30 The perception of view-based access control model servo and bypassing method and system Active CN104102218B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201410305109.6A CN104102218B (en) 2014-06-30 2014-06-30 The perception of view-based access control model servo and bypassing method and system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201410305109.6A CN104102218B (en) 2014-06-30 2014-06-30 The perception of view-based access control model servo and bypassing method and system

Publications (2)

Publication Number Publication Date
CN104102218A CN104102218A (en) 2014-10-15
CN104102218B true CN104102218B (en) 2017-04-05

Family

ID=51670439

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201410305109.6A Active CN104102218B (en) 2014-06-30 2014-06-30 The perception of view-based access control model servo and bypassing method and system

Country Status (1)

Country Link
CN (1) CN104102218B (en)

Families Citing this family (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111506109B (en) * 2014-12-31 2023-08-25 深圳市大疆创新科技有限公司 Selective processing of sensor data
CN104539906B (en) * 2015-01-08 2017-06-09 西安费斯达自动化工程有限公司 Image/laser range finding/ADS B monitor integral system
CN104656663B (en) * 2015-02-15 2017-12-01 西北工业大学 A kind of unmanned plane formation of view-based access control model perceives and bypassing method
CN104914877A (en) * 2015-07-13 2015-09-16 沈阳飞羽航空科技有限公司 Method for perceiving and evading light unmanned aerial vehicle autonomously
EP4579190A2 (en) 2015-12-09 2025-07-02 SZ DJI Technology Co., Ltd. Systems and methods for auto-return
CN106056624A (en) * 2016-05-24 2016-10-26 西北工业大学 Unmanned aerial vehicle high-definition image small target detecting and tracking system and detecting and tracking method thereof
CN106371461A (en) * 2016-09-08 2017-02-01 河海大学常州校区 Visual servo based video tracking flight object control system and method
US10552691B2 (en) * 2017-04-25 2020-02-04 TuSimple System and method for vehicle position and velocity estimation based on camera and lidar data
CN110362098B (en) * 2018-03-26 2022-07-05 北京京东尚科信息技术有限公司 Unmanned aerial vehicle visual servo control method and device and unmanned aerial vehicle
JP6974247B2 (en) * 2018-04-27 2021-12-01 エスゼット ディージェイアイ テクノロジー カンパニー リミテッドSz Dji Technology Co., Ltd Information processing equipment, information presentation instruction method, program, and recording medium
CN109947123B (en) * 2019-02-27 2021-06-22 南京航空航天大学 A UAV path tracking and autonomous obstacle avoidance method based on sight guidance law
CN112766025B (en) * 2019-11-04 2022-03-01 上海喜通装饰材料有限公司 Airspace target state data identification system
CN110908395A (en) * 2019-11-26 2020-03-24 西北工业大学 Improved unmanned aerial vehicle flight path real-time planning method
CN112092805B (en) * 2020-09-23 2021-10-19 北京理工大学 An integrated control method and system for intelligent vehicle collision avoidance and collision damage reduction
CN112419417B (en) * 2021-01-25 2021-05-18 成都翼比特自动化设备有限公司 Unmanned aerial vehicle-based photographing point positioning method and related device
CN113271409B (en) * 2021-04-06 2022-07-01 南京极目机器人科技有限公司 Combined camera, image acquisition method and aircraft

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP2442133A1 (en) * 2010-10-18 2012-04-18 Raytheon Company Systems and methods for collision avoidance in unmanned aerial vehicles

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8380367B2 (en) * 2009-03-26 2013-02-19 The University Of North Dakota Adaptive surveillance and guidance system for vehicle collision avoidance and interception
CN102707724B (en) * 2012-06-05 2015-01-14 清华大学 Visual localization and obstacle avoidance method and system for unmanned plane
CN102854883B (en) * 2012-07-04 2014-10-22 北京航空航天大学 Modeling method for unmanned aerial vehicle (UAV) dynamic emergency collision avoidance area
CN103149939B (en) * 2013-02-26 2015-10-21 北京航空航天大学 A kind of unmanned plane dynamic target tracking of view-based access control model and localization method
CN103869822B (en) * 2014-04-01 2016-09-07 西北工业大学 The perception of many rotor wing unmanned aerial vehicles and avoidance system and bypassing method thereof

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP2442133A1 (en) * 2010-10-18 2012-04-18 Raytheon Company Systems and methods for collision avoidance in unmanned aerial vehicles

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
无人机复杂环境中跟踪运动目标的实时航路规划;梁宵 等;《北京航空航天大学学报》;20120930;第38卷(第9期);第1129-1133页 *
电力巡线无人直升机障碍规避系统;郑天茹 等;《山东电力技术》;20121231(第185期);第14-17页 *

Also Published As

Publication number Publication date
CN104102218A (en) 2014-10-15

Similar Documents

Publication Publication Date Title
CN104102218B (en) The perception of view-based access control model servo and bypassing method and system
Kumar et al. Recent developments on target tracking problems: A review
CN104656663B (en) A kind of unmanned plane formation of view-based access control model perceives and bypassing method
CN105739523B (en) A kind of police vehicle-mounted unmanned aerial vehicle monitoring system and control method
CN105353772B (en) A kind of Visual servoing control method in UAV Maneuver target locating
CN104062977B (en) Full-autonomous flight control method for quadrotor unmanned aerial vehicle based on vision SLAM
CN104537898B (en) A kind of unmanned plane of air-ground coordination perceives avoidance system and its bypassing method
Griffiths et al. Obstacle and terrain avoidance for miniature aerial vehicles
CN110347186B (en) An autonomous tracking system for ground moving targets based on bionic binocular linkage
CN102190081B (en) Vision-based fixed point robust control method for airship
CN103365297A (en) Optical flow-based four-rotor unmanned aerial vehicle flight control method
CN108733064A (en) A kind of the vision positioning obstacle avoidance system and its method of unmanned plane
Yu et al. Stereo vision based obstacle avoidance strategy for quadcopter UAV
RU195749U1 (en) Intelligent vision system for an unmanned aerial vehicle for solving navigation problems, building a three-dimensional map of the surrounding space and obstacles, and autonomous patrolling
Kumar et al. Real-time vision-based tracking of a moving terrain target from Light Weight Fixed Wing UAV using gimbal control
Miller et al. UAV navigation based on videosequences captured by the onboard video camera
US20230242250A1 (en) Aerial Vehicle Path Determination
Minwalla et al. Experimental evaluation of PICAS: An electro-optical array for non-cooperative collision sensing on unmanned aircraft systems
Miller et al. Optical Flow as a navigation means for UAV
Kim et al. Development of a vision-based recognition and position measurement system for cooperative missions of multiple heterogeneous unmanned vehicles
Bouzerzour et al. Robust vision-based sliding mode control for uncooperative ground target searching and tracking by quadrotor
Yang et al. Design, implementation, and verification of a low‐cost terminal guidance system for small fixed‐wing UAVs
CN114897935B (en) Method and system for tracking aerial target object by unmanned aerial vehicle based on virtual camera
Sato et al. A simple autonomous flight control method of quadrotor helicopter using only single Web camera
Wang et al. A vision-aided navigation system by ground-aerial vehicle cooperation for UAV in GNSS-denied environments

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant