KR20050104901A - Method of docking multiple spacecrafts using vision sensor - Google Patents

Abstract

The present invention relates to a method of docking multiple aircraft using a vision sensor. More specifically, the relationship between a tracking object and a target object is obtained in two dimensions, and the positional information and attitude through an extended Kalman filter, an inertial navigation system, and a gyro. The present invention relates to a docking method of a plurality of vehicles that accurately estimates information and allows the tracking object to dock precisely to the target object through the estimated information.

The vision sensor includes a CCD camera and a plurality of beacons, and the CCD camera and the beacons are projected onto a plane to estimate the positional information and the attitude information using their geometric relationship and the extended Kalman filter.

Description

How to Dock Multiple Aircraft Using Vision Sensors {Method of Docking Multiple Spacecrafts Using Vision Sensor}

The present invention relates to a method of docking multiple aircraft using a vision sensor. More specifically, the relationship between a tracking object and a target object is obtained in two dimensions, and the positional information and attitude through an extended Kalman filter, an inertial navigation system, and a gyro. The present invention relates to a docking method of a plurality of vehicles that accurately estimates information and allows the tracking object to dock precisely to the target object through the estimated information.

Over the last few decades, automation issues in rendezvous or docking have been considered one of the most important and necessary technologies. Automated Rendezvous and Docking (ARD) can greatly reduce the cost and complexity of space vehicle systems. In addition, the ARD technology developed in this way can be used for the formation of the flight of the spacecraft or to inject fuel into the plane in flight, which is highly utilized. Since Russia first introduced the technology to dock at the Mir space station, the United States, Europe and Japan have been focusing on developing ARD individually.

A typical docking process can be thought of in five stages: launch, orbit transfer, relative approach, proximity operation, and final phase. The step now covered in the present invention is the fourth step, the proximity operation step. Proximity operation usually refers to when the distance between the tracker spacecraft and the target spacecraft is less than 100m. There are generally two methods for the exact navigation required in this process. The first is using Kalman Killer to use GPS. The technique has a margin of error of approximately 1 cm, but it is likely that there is a problem between GPS when it is close, and it is difficult to operate on a planet other than Earth. The other is to use vision inductive sensors to measure relative distances or postures. This technology has no problem inherent to the GPS-related technology, but has a problem that an additional vision-related technology is needed. The present invention uses the latter technique.

In order to develop such a technique, an algorithm for properly filtering the surrounding information during the docking process is necessary. The vision system to be used in the present invention can obtain accurate information in close operation, but has a disadvantage that the sampling frequency is only a few Hz due to hardware limitations. On the other hand, the Inertial Measurement Unit (IMU), including the gyro, is stable in the short term and has a sufficient sampling frequency, but emits in the long term. Therefore, using these two sensors in combination is essential for spacecraft docking technology.

The Kalman filter has been proven to be very useful in posture measurements. The Kalman filter minimizes the error between the response of the system model equations and the actual measurements, allowing for better state estimation.

1960 R.E. Discrete Kalman filtering, proposed by Kalman, is a filtering method that is widely used in many fields due to the rapid development of digital computers. Especially in the field of automation systems, guidance and navigation, Kalman filtering is an indispensable technique.

Discrete Kalman filters are used to estimate states in discrete control processes governed by linear probability differential equations such as The actually measured value is represented by Equation 2.

[Equation 1]

[Equation 2]

here Is State value in the first step Is The output value in the first step is shown. silver Random variable representing the th input value Wow Denote modeling noise and measurement noise, respectively. These two variables are considered independent of each other and mean zero white noise. Equation 3 shows the characteristics of the two noises.

[Equation 3]

here Are random variables Wow Represents the probability function.

Practically Covariance of Modeling Noise And covariance of measured noise The matrix changes with each time and measurement but is generally assumed to be a constant.

The stochastic characteristic of the discrete Kalman filter is sufficient to set the first and second moments of the state distribution as shown in Equation 4.

[Equation 4]

here Is a variable Represents the expected value of Is a variable Means the average. Also Is a variable The dispersion of Discrete Kalman filters with these characteristics are configured to continuously update each time and measurement. Equation 5 shows an update structure with respect to time, and Equation 6 shows an update structure with respect to the measurement.

[Equation 5]

[Equation 6]

here Is the Kalman filter gain and the remaining variables have the same meanings as the variables used in Equations 1 to 4

Discrete Kalman filters as described above are applicable to linear systems, and for nonlinear systems that are difficult to represent linearly, the form of extended Kalman filters should be used. The system to which the Extended Kalman Filter is applied may be represented by a nonlinear probability differential equation such as Equation (7).

[Equation 7]

The time and measurement update structure described by Equations 5 and 6 in the linear system is modified to Equations 8 and 9 in the nonlinear system.

[Equation 8]

[Equation 9]

The basic use of the Extended Kalman Filter is the same as the linear system.

In order to obtain position and angle information between two objects using vision, at least one CCD camera and several beacons that provide coordinates to recognize the object must be provided simultaneously. Beacons must be made of a material that can be easily detected or a light-emitting material such as an LED and attached to a known location on the target. A tracking object with a CCD camera recognizes the beacons of the target object and calculates the position and attitude of each other.

1 shows the relationship between a beacon and a CCD camera in three dimensions. Indicates the position of the tracking object relative to the target object. Represents the Euler angle. At this time, the direction cosine matrix describing the relationship between the coordinates of the target object and the coordinates of the tracking object is expressed by Equation 10 using 3-2-1 rotation.

[Equation 10]

From the geometry shown in FIG. 1, the i-th beacon and its measurements on the CCD are represented as in equation (11). Used here Is the matrix of the equation Element

[Equation 11]

An object of the present invention is to estimate the accurate state by combining the vision system information and the IMU information including the gyro using a conventional extended Kalman filter. Because the sampling frequency of the vision system and the sampling frequency of the IMU including the gyro are different, it is necessary to use the extended Kalman filter differently depending on the frequency. If vision system information is available, Kalman filtering is performed using both IMU and vision system information. If vision system information is not available, Kalman filtering is performed using only IMU information.

Another object of the present invention is to directly determine the relative position and posture between two objects using a vision system (camera, beacon).

The present invention comprises a vision sensor comprising a CCD camera on one side, and a beacon on the other side of the two or more aircraft docked, relative position information and attitude information of the two aircraft calculated by recognizing the beacon with the CCD camera Provides a docking method for a plurality of aircraft, characterized in that for controlling the docking using.

In addition, the present invention is provided with at least one CCD camera on the tracking object and at least two beacons on the target object, and expresses their relationship in two dimensions by projecting them on a suitable plane, the two-dimensional relationship expressed as described above Distance from the tracking object to the target And angle , Angle It provides a docking method of a plurality of aircraft, characterized in that to calculate.

The present invention also provides a method for docking a plurality of aircraft, characterized by using an extended Kalman filter to update the information of the gyro and inertial navigation apparatus and the vision sensor.

In addition, the present invention uses the information of the vision sensor when the information of the vision sensor is available, and when the information of the vision sensor is not available, the inertial navigation system and gyro information to estimate the state of a plurality of aircraft, characterized in that Provides a docking method.

In addition, the present invention is characterized by using the measurement equation as shown in the following formula (a) when the information of the vision sensor is available, and using the measurement equation as shown in (b) when the information of the vision sensor is not available. It provides a docking method of a plurality of aircraft.

(a)

(b)

In addition, the present invention in the two or more flying vehicles flying on the CCD camera on one side, the beacon is attached to the other side, by using the relative position information and attitude information of the two aircraft calculated by recognizing the beacon with the CCD camera to perform the alignment flight It provides a method of alignment flight of a plurality of vehicles, characterized in that the control.

In addition, the present invention provides a method for aligning a plurality of aircraft, characterized in that to obtain the attitude information by the following equation (c), and to obtain the relative position information by the following equation (d).

(c)

(d)

In addition, the present invention is a collision avoidance flight by using the relative position information and attitude information of the two aircraft, which is calculated by recognizing the beacon with the CCD camera on one side, the beacon on the other side of the two or more flying aircraft flying; It provides a collision avoidance flight method of a plurality of aircraft, characterized in that for controlling.

In addition, the present invention provides a collision avoidance flight method of a plurality of aircraft, characterized in that to obtain the attitude information by the above formula (c), and to obtain the relative position information by the above formula (d).

In addition, the present invention uses the information of the vision sensor when the information of the vision sensor is available, and when the information of the vision sensor is not available, the inertial navigation system and gyro information to estimate the state of a plurality of aircraft, characterized in that Provide an alignment flight method.

In addition, the present invention uses the information of the vision sensor when the information of the vision sensor is available, and when the information of the vision sensor is not available, the inertial navigation system and gyro information to estimate the state of a plurality of aircraft, characterized in that Provides a way to avoid collisions.

Hereinafter, the present invention will be described in more detail.

Conventional vision-based systems are nonlinear, as shown in Equation 11, and states cannot be directly interpreted because they are related to each other. However, if a slight modification is made to Equation 11 in accordance with the two-dimensional plane motion to be used in the present invention, the relative position and attitude information can be obtained without any prior information. 2 shows the relationship between beacons and CCD cameras in two-dimensional plane motion.

Shown in Figure 2 Since the sign of affects the rotation angle calculation, it is necessary to decide in advance. In FIG. 2 Are all negative values. That is, the direction angle psi is defined as in Equation 12.

[Equation 12]

The pixel position of the beacons on the CCD camera And distance between real beacons Focal length of lens Are five known variables, And the distance between the tracking object and the target object Are the three variables to figure out. Can be found simply as in Equation 13.

[Equation 13]

The distance between the beacons on the CCD camera Is defined as in equation (14).

[Equation 14]

From the geometry shown in Figure 2 , Is represented by equation (15).

[Equation 15]

When we solve equation 13 and equation 15 together, we finally get the direction angle Can be calculated as in Equation 16.

[Equation 16]

Since we know all the variables in the right term in Equation 16, we use only one CCD camera and beacons Can be obtained analytically. In a similar manner, the distance between the center beacon and the lens can be obtained as shown in Equation 17.

[Equation 17]

The direction angle ψ is shown in equation (16) Wow Represented by the size difference of As shown in Equation 17, it can be seen that mainly depends on the focal length of the lens.

The system to which the algorithm developed in the present invention is applied is a free flying robot system for estimating position and attitude. It has two translational degrees of freedom and one rotational degree of freedom and has a total of three degrees of freedom, and the state vector can be represented by Equation 18. Figure 3 shows the overall appearance of the free flying robot.

Equation 18

here Wow Location information, Denotes the attitude information as the direction angle variable of the robot. At this time, the dynamic equation of the free flying robot and a simple PD controller for controlling the position and attitude are described as in Equation 19.

[Equation 19]

here Is a 3 × 3 diagonal inertia matrix Is the force and torque applied to the free flying robot. Also Indicates the state (position and attitude) that the aircraft is aiming for Indicates the state (position and attitude) of the current vehicle. Wow Is the control gain that must be determined through tuning.

The state variables used in (19) are only three, as defined in (18). However, in practice, the variables obtained in the sensor are not the variables defined in Equation 18, but their derivative form. Therefore, in order to use the information using the Kalman filter form, the system state vector must be extended as shown in Equation 20.

[Equation 20]

Rewriting Equation 19 into the form of a well-known optimal linear estimator gives the form of Equation 21.

[Equation 21]

here Is assumed to be white noise with constant covariance to make it easier to solve the optimal problem. Is the sampling time, Is the current system state vector.

As shown above, since the vision-based system can obtain position and attitude information directly, and acceleration and yaw velocity from the IMU and gyro, the measurement equation can be expressed as It is described as 22.

[Equation 22]

Is The measured noise and the covariance matrix are set to meaningful values according to the sensor accuracy.

Since Equation 20 and Equation 22 describe the state equation and the measurement equation, respectively, the state can be estimated using the Kalman filtering described above. However, because the sampling frequency of the vision sensor and the IMU and gyro are different, it is not always possible to get both of them. So when you get two pieces of information at the same time, Information from IMU and Gyro Since information can be obtained, a measurement equation such as (22) is used. However, when only the IMU and gyro information with high sampling frequency is obtained because the vision sensor information is not available. Since only 10,000 can be obtained, Kalman filtering is performed using a measurement equation such as Equation 23.

[Equation 23]

5 is a flowchart of a vision based system algorithm in accordance with the present invention.

The present invention can be applied to the same principle in the alignment flight and collision avoidance flight of a plurality of aircraft.

According to the present invention, it is possible to analytically obtain relative position and posture information between two objects using only one CCD camera and a few beacons in planar motion. Thus, instead of using the IMU information, including the gyro, which tends to diverge over time, it can be used with vision information to obtain better state values. While vision information can yield relatively accurate values, the sampling frequency is very small, and the IMU information including the gyro is relatively inaccurate while the sampling frequency is high. These two sensor information can be properly filtered using discrete Kalman filter to take advantage of each other.

1 is a conceptual diagram illustrating a three-dimensional relationship between beacons and a CCD camera in a multiple aircraft docking method according to the present invention.

2 is a conceptual diagram illustrating a relationship of converting a three-dimensional relationship into a two-dimensional plane by projecting beacons and a CCD camera onto a suitable two-dimensional plane in the multiple aircraft docking method according to the present invention.

3 is a conceptual diagram showing the overall appearance of the free flying robot in the multiple aircraft docking method according to the present invention.

4 is a conceptual diagram illustrating an embodiment of the present invention.

5 is a flowchart of a position and attitude state estimation algorithm included in the multiple aircraft docking method according to the present invention.

<Description of Major Codes in Drawings>

A, B, C: Actual location of beacons A, B, C

: The angle between the line of sight and the vertical line of the beacon

: The angle between the line of sight and the vertical line of the image plane

Focal length : Distance between tracking object and target object

: Coordinates of beacons A, B, and C on the image plane

: Distance between beacons projected on the plane

: Measurement matrix when the vision sensor is available (measurement sensitivity matrix)

: Measurement matrix when the vision sensor is not available (measurement sensitivity matrix)

Claims (11)

In the two or more aircraft to be docked, a vision sensor including a CCD camera on one side and a beacon on the other side is configured, and by using the relative position information and attitude information of the two aircraft calculated by recognizing the beacon with the CCD camera. Docking method of a plurality of aircraft, characterized in that for controlling the docking. The method of claim 1, At least one CCD camera is installed in the tracking object, and at least two beacons are installed in the target object to express their relationship in two dimensions by projecting them on a proper plane. Distance of object And angle , Angle Docking method of a plurality of aircraft, characterized in that for calculating. The method of claim 1, A method of docking a plurality of aircraft, characterized by using an extended Kalman filter to update information of a gyro and inertial navigation system and the vision sensor. A method of docking a plurality of aircrafts, wherein the information of the vision sensor is used when the information of the vision sensor is available, and the state is estimated using the inertial navigation system and the gyro information when the information of the vision sensor is not available. The method of claim 4, wherein When the information of the vision sensor is available, a measurement equation such as the following (A) is used, and when the information of the vision sensor is not available, the measurement equation of the following (B) is used. Way. (A) (B) In one or more flying vehicles, a CCD camera is attached to one side and a beacon is attached to the other side, and the alignment flight is controlled by using relative position information and attitude information of the two vehicles calculated by recognizing the beacon with the CCD camera. How to align and fly multiple aircraft. The method of claim 6, Attitude information is obtained by the following equation (C), and relative position information is obtained by the following equation (D). (C) (D) In at least two flying vehicles, a CCD camera is attached to one side and a beacon is attached to the other side, and the collision avoidance flight is controlled using the relative position information and the attitude information of the two vehicles calculated by recognizing the beacon with the CCD camera. A collision avoidance flight method of a plurality of aircraft. The method of claim 8, The collision avoidance flight method of a plurality of aircraft, characterized in that to obtain the attitude information by the above formula (C), and to obtain the relative position information by the above formula (D). When the information of the vision sensor is available, the information of the vision sensor is used, and when the information of the vision sensor is not available, the alignment flight method of a plurality of aircraft, characterized by estimating the state using the inertial navigation system and gyro information. When the information of the vision sensor is available, the information of the vision sensor is used, and when the information of the vision sensor is not available, the collision avoidance flight method of a plurality of vehicles, characterized in that to estimate the state using the inertial navigation system and gyro information.