KR100575108B1 - Method of Docking Multiple Spacecrafts Using Vision Sensor - Google Patents

Abstract

The present invention relates to a method of docking multiple aircraft using a vision sensor. More specifically, the relationship between a tracking object and a target object is obtained in two dimensions, and the positional information and attitude through an extended Kalman filter, an inertial navigation system, and a gyro. The present invention relates to a docking method of a plurality of vehicles that accurately estimates information and allows the tracking object to dock precisely to the target object through the estimated information.

The vision sensor includes a CCD camera and a plurality of beacons, and the CCD camera and the beacons are projected onto a plane to estimate the positional information and the attitude information using their geometric relationship and the extended Kalman filter.

Vision Sensor, Docking, CCD Camera, Biphone, Extended Kalman Filter, IMU, Gyro

Description

How to Dock Multiple Aircraft Using Vision Sensors {Method of Docking Multiple Spacecrafts Using Vision Sensor}             

1 is a conceptual diagram illustrating a three-dimensional relationship between beacons and a CCD camera in a multiple aircraft docking method according to the present invention.

2 is a conceptual diagram illustrating a relationship of converting a three-dimensional relationship into a two-dimensional plane by projecting beacons and a CCD camera onto a suitable two-dimensional plane in the multiple aircraft docking method according to the present invention.

3 is a conceptual diagram showing the overall appearance of the free flying robot in the multiple aircraft docking method according to the present invention.

4 is a conceptual diagram illustrating an embodiment of the present invention.

5 is a flowchart of a position and attitude state estimation algorithm included in the multiple aircraft docking method according to the present invention.

<Description of Major Codes in Drawings>

A, B, C: Actual location of beacons A, B, C

: 시선과 비콘면의 수직선 이루는 사이각

: The angle between the line of sight and the vertical line of the beacon

: 시선과 이미지평면의 수직선이 이루는 사이각

: The angle between the line of sight and the vertical line of the image plane

: 초점거리

: 추적물체와 목표물체 사이의 거리

Focal length

: Distance between tracking object and target object

: 이미지평면 상에서의 비콘 A, B, C의 좌표

: Coordinates of beacons A, B, and C on the image plane

: 평면에 투영된 비콘 사이의 거리

: Distance between beacons projected on the plane

: 비전 센서가 가용할 때의 측정행렬(측정민감도 행렬)

: Measurement matrix when the vision sensor is available (measurement sensitivity matrix)

: 비전 센서가 가용하지 않을 때의 측정행렬(측정민감도 행렬)

: Measurement matrix when the vision sensor is not available (measurement sensitivity matrix)

The present invention relates to a method of docking multiple aircraft using a vision sensor. More specifically, the relationship between a tracking object and a target object is obtained in two dimensions, and the positional information and attitude through an extended Kalman filter, an inertial navigation system, and a gyro. The present invention relates to a docking method of a plurality of vehicles that accurately estimates information and allows the tracking object to dock precisely to the target object through the estimated information.

Over the last few decades, automation issues in rendezvous or docking have been considered one of the most important and necessary technologies. Automated Rendezvous and Docking (ARD) can greatly reduce the cost and complexity of space vehicle systems. In addition, the ARD technology developed in this way can be used for the formation of the flight of the spacecraft or to inject fuel into the plane in flight, which is highly utilized. Since Russia first introduced the technology to dock at the Mir space station, the United States, Europe and Japan have been focusing on developing ARD individually.

A typical docking process can be thought of in five stages: launch, orbit transfer, relative approach, proximity operation, and final phase. The step now covered in the present invention is the fourth step, the proximity operation step. Proximity operation usually refers to when the distance between the tracker spacecraft and the target spacecraft is less than 100m. There are generally two methods for the exact navigation required in this process. The first is using Kalman Killer to use GPS. The technique has a margin of error of approximately 1 cm, but it is likely that there is a problem between GPS when it is close, and it is difficult to operate on a planet other than Earth. The other is to use vision inductive sensors to measure relative distances or postures. This technology has no problem inherent to the GPS-related technology, but has a problem that an additional vision-related technology is needed. The present invention uses the latter technique.

In order to develop such a technique, an algorithm for properly filtering the surrounding information during the docking process is necessary. The vision system to be used in the present invention can obtain accurate information in close operation, but has a disadvantage that the sampling frequency is only a few Hz due to hardware limitations. On the other hand, the Inertial Measurement Unit (IMU), including the gyro, is stable in the short term and has a sufficient sampling frequency, but emits in the long term. Therefore, using these two sensors in combination is essential for spacecraft docking technology.

The Kalman filter has been proven to be very useful in posture measurements. The Kalman filter minimizes the error between the response of the system model equations and the actual measurements, allowing for better state estimation.

1960 R.E. Discrete Kalman filtering, proposed by Kalman, is a filtering method that is widely used in many fields due to the rapid development of digital computers. Especially in the field of automation systems, guidance and navigation, Kalman filtering is an indispensable technique.

Discrete Kalman filters are used to estimate states in discrete control processes governed by linear probability differential equations such as The actually measured value is represented by Equation 2.

[Equation 1]

[Equation 2]

는

번째 스텝에서의 상태값을

는

번째 스텝에서의 출력값을 나타낸다.

은

번째 입력값을 나타내고 랜덤 변수

와

는 각각 모델링 잡음과 측정 잡음을 나타낸다. 이 두 변수는 서로 독립이고 평균이 0인 백색 잡음으로 간주한다. 수학식 3은 두 잡음의 특성을 나타낸다.here

Is

State value in the first step

Is

The output value in the first step is shown.

silver

Random variable representing the th input value

Wow

Denote modeling noise and measurement noise, respectively. These two variables are considered independent of each other and mean zero white noise. Equation 3 shows the characteristics of the two noises.

[Equation 3]

는 각각 랜덤 변수

와

가 발생할 확률함수를 나타낸다.here

Are random variables

Wow

Represents the probability function.

와 측정 잡음의 공분산

행렬은 각각의 시각과 측정마다 변하지만 일반적으로 상수라고 가정한다.Practically Covariance of Modeling Noise

And covariance of measured noise

The matrix changes with each time and measurement but is generally assumed to be a constant.

The stochastic characteristic of the discrete Kalman filter is sufficient to set the first and second moments of the state distribution as shown in Equation 4.

[Equation 4]

의 기대값을 나타내고

는 변수

의 평균을 의미 한다. 또한

는 변수

의 분산을 나타낸다. 이러한 특성들을 가진 이산 칼만 필터는 시간과 측정에 대하여 각각 업데이트를 연속적으로 해 나가도록 구성되어 있다. 수학식 5는 시간에 대한 업데이트 구조, 수학식 6은 측정에 대한 업데이트 구조를 나타낸다.here Is a variable

Represents the expected value of

Is a variable

Means the average. Also

Is a variable

The dispersion of Discrete Kalman filters with these characteristics are configured to continuously update each time and measurement. Equation 5 shows an update structure with respect to time, and Equation 6 shows an update structure with respect to the measurement.

[Equation 5]

[Equation 6]

는 칼만 필터 게인이고 나머지 변수들은 수학식 1 내지 4에서 사용된 변수들과 의미가 같다here

Is the Kalman filter gain and the remaining variables have the same meanings as the variables used in Equations 1 to 4

Discrete Kalman filters as described above are applicable to linear systems, and for nonlinear systems that are difficult to represent linearly, the form of extended Kalman filters should be used. The system to which the Extended Kalman Filter is applied may be represented by a nonlinear probability differential equation such as Equation (7).

[Equation 7]

The time and measurement update structure described by Equations 5 and 6 in the linear system is modified to Equations 8 and 9 in the nonlinear system.

[Equation 8]

[Equation 9]

The basic use of the Extended Kalman Filter is the same as the linear system.

In order to obtain position and angle information between two objects using vision, at least one CCD camera and several beacons that provide coordinates to recognize the object must be provided simultaneously. Beacons must be made of a material that can be easily detected or a light-emitting material such as an LED and attached to a known location on the target. A tracking object with a CCD camera recognizes the beacons of the target object and calculates the position and attitude of each other.

는 목표 물체에 상대적인 추적 물체의 위치를 나타내고

는 오일러 각을 나타낸다. 이 때 목표 물체의 좌표와 추적 물체의 좌표 사이의 관계를 기술하는 방향 코사인 행렬은 3-2-1 회전을 이용하면 수학식 10처럼 나타난다.1 shows the relationship between a beacon and a CCD camera in three dimensions.

Indicates the position of the tracking object relative to the target object.

Represents the Euler angle. At this time, the direction cosine matrix describing the relationship between the coordinates of the target object and the coordinates of the tracking object is expressed by Equation 10 using 3-2-1 rotation.

[Equation 10]

는 수학식 10에서 기술한 행렬의

요소이다From the geometry shown in FIG. 1, the i-th beacon and its measurements on the CCD are represented as in equation (11). Used here

Is the matrix of the equation

Element

[Equation 11]

An object of the present invention is to estimate the accurate state by combining the vision system information and the IMU information including the gyro using a conventional extended Kalman filter. Because the sampling frequency of the vision system and the sampling frequency of the IMU including the gyro are different, it is necessary to use the extended Kalman filter differently depending on the frequency. If vision system information is available, Kalman filtering is performed using both IMU and vision system information. If vision system information is not available, Kalman filtering is performed using only IMU information.

Another object of the present invention is to directly determine the relative position and posture between two objects using a vision system (camera, beacon).

The present invention comprises a vision sensor comprising a CCD camera on one side, and a beacon on the other side of the two or more aircraft docked, relative position information and attitude information of the two aircraft calculated by recognizing the beacon with the CCD camera Provides a docking method for a plurality of aircraft, characterized in that for controlling the docking using.

및 각(角)

, 각(角)

를 계산하는 것을 특징으로 하는 다수 비행체의 도킹방법을 제공한다.In addition, the present invention is provided with at least one CCD camera on the tracking object and at least two beacons on the target object, and expresses their relationship in two dimensions by projecting them on a suitable plane, the two-dimensional relationship expressed as described above Distance from the tracking object to the target

And angle

, Angle

It provides a docking method of a plurality of aircraft, characterized in that to calculate.

The present invention also provides a method for docking a plurality of aircraft, characterized by using an extended Kalman filter to update the information of the gyro and inertial navigation apparatus and the vision sensor.

In addition, the present invention uses the information of the vision sensor when the information of the vision sensor is available, and when the information of the vision sensor is not available, the inertial navigation system and gyro information to estimate the state of a plurality of aircraft, characterized in that Provides a docking method.

In addition, the present invention is characterized by using the measurement equation as shown in the following formula (a) when the information of the vision sensor is available, and using the measurement equation as shown in (b) when the information of the vision sensor is not available. It provides a docking method of a plurality of aircraft.

(a)

(a)

(b)

(b)

In addition, the present invention in the two or more flying vehicles flying on the CCD camera on one side, the beacon is attached to the other side, by using the relative position information and attitude information of the two aircraft calculated by recognizing the beacon with the CCD camera to perform the alignment flight It provides a method of alignment flight of a plurality of vehicles, characterized in that the control.

In addition, the present invention provides a method for aligning a plurality of aircraft, characterized in that to obtain the attitude information by the following equation (c), and to obtain the relative position information by the following equation (d).

(c)

(c)

(d)

(d)

In addition, the present invention is a collision avoidance flight by using the relative position information and attitude information of the two aircraft, which is calculated by recognizing the beacon with the CCD camera on one side, the beacon on the other side of the two or more flying aircraft flying; It provides a collision avoidance flight method of a plurality of aircraft, characterized in that for controlling.

In addition, the present invention provides a collision avoidance flight method of a plurality of aircraft, characterized in that to obtain the attitude information by the above formula (c), and to obtain the relative position information by the above formula (d).

In addition, the present invention uses the information of the vision sensor when the information of the vision sensor is available, and when the information of the vision sensor is not available, the inertial navigation system and gyro information to estimate the state of a plurality of aircraft, characterized in that Provide an alignment flight method.

In addition, the present invention uses the information of the vision sensor when the information of the vision sensor is available, and when the information of the vision sensor is not available, the inertial navigation system and gyro information to estimate the state of a plurality of aircraft, characterized in that Provides a way to avoid collisions.

Hereinafter, the present invention will be described in more detail.

Conventional vision-based systems are nonlinear, as shown in Equation 11, and states cannot be directly interpreted because they are related to each other. However, if a slight modification is made to Equation 11 in accordance with the two-dimensional plane motion to be used in the present invention, the relative position and attitude information can be obtained without any prior information. 2 shows the relationship between beacons and CCD cameras in two-dimensional plane motion.

의 부호는 회전각 계산에 영향을 미치기 때문에 미리 결정해 두는 것이 필요하다. 도 2에서의

는 모두 음의 값을 갖는다. 즉 방향각 psi는 수학식 12처럼 정의된다.Shown in Figure 2

Since the sign of affects the rotation angle calculation, it is necessary to decide in advance. In FIG. 2

Are all negative values. That is, the direction angle psi is defined as in Equation 12.

[Equation 12]

와 실제 비콘들 사이의 거리

, 렌즈의 초점 거리

는 알 수 있는 5개의 변수들인 반면,

와 추적 물체와 목표 물체 사이의 거리

은 알아내어야 할 3개의 변수들이다.

는 수학식 13에서처럼 간단하게 알아낼 수 있다.The pixel position of the beacons on the CCD camera

And distance between real beacons

Focal length of lens

Are five known variables,

And the distance between the tracking object and the target object

Are the three variables to figure out.

Can be found simply as in Equation 13.

[Equation 13]

는 수학식 14처럼 정의된다.The distance between the beacons on the CCD camera

Is defined as in equation (14).

[Equation 14]

,

는 수학식 15처럼 나타난다.From the geometry shown in Figure 2

,

Is represented by equation (15).

[Equation 15]

를 수학식 16처럼 계산할 수 있다.When we solve equation 13 and equation 15 together, we finally get the direction angle

Can be calculated as in Equation 16.

[Equation 16]

를 해석적으로 구할 수 있다. 비슷한 방식으로 가운데 비콘과 렌즈 사이의 거리를 수학식 17처럼 구할 수 있다.Since we know all the variables in the right term in Equation 16, we use only one CCD camera and beacons

Can be obtained analytically. In a similar manner, the distance between the center beacon and the lens can be obtained as shown in Equation 17.

[Equation 17]

와

의 크기 차이를 이용하여 나타내어지고, 거리

은 수학식 17에서 보듯이 주로 렌즈의 초점 거리에 의존함을 알 수 있다.The direction angle ψ is shown in equation (16)

Wow

Represented by the size difference of

As shown in Equation 17, it can be seen that mainly depends on the focal length of the lens.

The system to which the algorithm developed in the present invention is applied is a free flying robot system for estimating position and attitude. It has two translational degrees of freedom and one rotational degree of freedom and has a total of three degrees of freedom, and the state vector can be represented by Equation 18. Figure 3 shows the overall appearance of the free flying robot.

Equation 18

와

는 위치 정보를,

는 로봇의 방향각 변수로써 자세 정보를 나타낸다. 이 때 자유비행 로봇의 동력학 방정식과 위치 및 자세를 제어하기 위한 간단한 PD 제어기는 수학식 19처럼 기술된다.here

Wow

Location information,

Denotes the attitude information as the direction angle variable of the robot. At this time, the dynamic equation of the free flying robot and a simple PD controller for controlling the position and attitude are described as in Equation 19.

[Equation 19]

는 3×3 대각 관성 행렬이고

는 자유비행 로봇에 가해지는 힘과 토크이다. 또한

는 비행체가 목표로 하는 상태(위치 및 자세)를 나타내고

는 현재 비행체의 상태(위치 및 자세)를 나타낸다.

와

는 튜닝을 통해 결정되어야 하는 제어 게인이다.here

Is a 3 × 3 diagonal inertia matrix

Is the force and torque applied to the free flying robot. Also

Indicates the state (position and attitude) that the aircraft is aiming for

Indicates the state (position and attitude) of the current vehicle.

Wow

Is the control gain that must be determined through tuning.

The state variables used in (19) are only three, as defined in (18). However, in practice, the variables obtained in the sensor are not the variables defined in Equation 18, but their derivative form. Therefore, in order to use the information using the Kalman filter form, the system state vector must be extended as shown in Equation 20.

[Equation 20]

Rewriting Equation 19 into the form of a well-known optimal linear estimator gives the form of Equation 21.

[Equation 21]

는 최적 문제를 풀기 쉽게 하기 위해서 일정한 공분산을 가진 백색 잡음이라 가정된다.

는 샘플링 시간이고,

는 현재의 시스템 상태 벡터이다.here

Is assumed to be white noise with constant covariance to make it easier to solve the optimal problem.

Is the sampling time,

Is the current system state vector.

As shown above, since the vision-based system can obtain position and attitude information directly, and acceleration and yaw velocity from the IMU and gyro, the measurement equation can be expressed as It is described as 22.

[Equation 22]

는

번째의 측정 잡음이고 공분산 행렬은 센서 정확도에 따라 의미 있는 값으로 정한다.

Is

The measured noise and the covariance matrix are set to meaningful values according to the sensor accuracy.

정보를, IMU 및 자이로로부터

정보를 얻을 수 있기 때문에 수학식 22와 같은 측정 방정식을 사용한다. 하지만, 비전센서 정보를 이용할 수 없어서 샘플링 주파수가 높은 IMU 및 자이로 정보만을 얻을 때에는

만을 얻을 수 있기 때문에 수학식 23과 같은 측정 방정식을 이용하여 칼만 필터링을 한다.Since Equation 20 and Equation 22 describe the state equation and the measurement equation, respectively, the state can be estimated using the Kalman filtering described above. However, because the sampling frequency of the vision sensor and the IMU and gyro are different, it is not always possible to get both of them. So when you get two pieces of information at the same time,

Information from IMU and Gyro

Since information can be obtained, a measurement equation such as (22) is used. However, when only the IMU and gyro information with high sampling frequency is obtained because the vision sensor information is not available.

Since only 10,000 can be obtained, Kalman filtering is performed using a measurement equation such as Equation 23.

[Equation 23]

5 is a flowchart of a vision based system algorithm in accordance with the present invention.

The present invention can be applied to the same principle in the alignment flight and collision avoidance flight of a plurality of aircraft.

According to the present invention, it is possible to analytically obtain relative position and posture information between two objects using only one CCD camera and a few beacons in planar motion. Thus, instead of using the IMU information, including the gyro, which tends to diverge over time, it can be used with vision information to obtain better state values. While vision information can yield relatively accurate values, the sampling frequency is very small, and the IMU information including the gyro is relatively inaccurate while the sampling frequency is high. These two sensor information can be properly filtered using discrete Kalman filter to take advantage of each other.

Claims (11)

Relative position between the tracking object and the target object using a vision system using a CCD camera and beacons; And in the method of estimating the state information related to the posture information, Project the tracking object with at least one CCD camera and the target object with at least three beacons onto a two-dimensional plane to track using the pixel position of the beacons in the CCD camera, the actual distance between the beacons, and the focal length of the lens And obtaining relative position information and attitude information between the object and the target object, and estimating a state between the tracking object and the target object by Kalman filtering. 2. The method of claim 1, wherein Kalman filtering uses discrete Kalman filters for linear systems and extended Kalman filters for nonlinear systems. The method according to claim 1, the orientation angle ψ that is attitude information; And the distance R between the tracking object and the target object is represented by the following equation and p and q are the distances between the beacons in the CCD camera; w is the distance between the actual beacons; f is the focal length of the CCD camera lens; xc is the pixel position of the middle beacon of the three beacons on the CCD camera; And α represents an angle formed between the axis of the CCD camera lens and the pixel position of the intermediate beacon:

. The method of claim 1, wherein the state vector for estimating a state between the tracking object and the target object is represented by the following equation (I); And position and attitude control are performed based on Equation (II) below, wherein x and y are position information; ψ is posture information indicating a direction angle; M (m, I _Z ) is a 3 × 3 diagonal inertia matrix; U is the force and torque of the target object; X _des is the target position and posture; X is the current position and control; And K _D and K _P represent control gains to be determined through tuning.

(I);

(II). A method of docking multiple aircraft comprising the method of claim 1. A method for aligning and flying multiple vehicles, comprising the method of any one of claims 1 to 4. A method for avoiding collision of multiple vehicles, comprising the method of any one of claims 1 to 4. delete delete delete delete

Images