KR20180038761A - Method for target data acquisition - Google Patents

Abstract

The present invention relates to a target data acquisition method and, more specifically, relates to a target data acquisition method which estimates a speed and a position of a target with a distance between an aiming unit and the target and an orientation angle change value of a gunner.

Description

Method for target data acquisition [

The present invention relates to a target information acquiring method, and more particularly, to a target information acquiring method for estimating a target speed and position using a distance between a sighting device and a target and a change value of a posture angle of a shooter.

In general, a method of acquiring target information in an apparatus for acquiring target information has been studied in a method using a radar sensor. However, in the case of a radar sensor, the distance, direction, and altitude of the target are measured by measuring the time of the reflected wave using the radio wave emitted from the directional antenna. Due to the problems such as size, cost and power consumption, It is difficult to apply.

Therefore, another method is to use the distance information between the shooter and the target using the distance measuring sensor and the attitude sensor, and to use the direction angle change while the target is aiming at the target.

However, the posture sensor suitable for portable use has a problem that it is difficult to acquire precise target information due to a low precision and a cumulative error due to the characteristics of the posture sensor.

FIG. 1 is a conceptual diagram of a target information acquisition method according to distance and direction angle measurement values.

Figure 1 shows how the observable data correlates with the velocity vector of the target when the shooter's position is at the origin. In an ideal environment where there is no error of the attitude sensor, the vector sum of the measured data and the target movement just before is the same as the measurement vector of the next time.

However, in the case of an attitude sensor for measuring an angular velocity, a bias error and a drift error are inevitably generated. If the accuracy of the attitude sensor is not sufficient for the requirement, there is a problem that the estimation error of the target information becomes large .

Korean Patent Publication No. 10-2010-0029504

SUMMARY OF THE INVENTION The present invention has been made to solve the above-mentioned problems, and it is an object of the present invention to provide a method and an apparatus for calculating a posture angle change value by summing a posture angle change value obtained from an image sensor, It is aimed to eliminate the drift error.

It is also aimed to obtain accurate position and velocity information of a target that is steadily moving at constant speed even with extremely low cycle distance measurements using the Primary Components Analysis (PCA) method.

In order to achieve the above object, according to the present invention,

A posture measuring step of measuring a change in posture angle of a shooter while a shooter is aiming a target using a day / night camera including an image sensor;

A posture measuring step of measuring a change in the posture angle of the shooter while the shooter is aiming the target using the posture sensor;

And a sensor fusion step of calculating a single attitude angle change value by summing up the attitude angle change value measured using the attitude sensor and the attitude angle change value measured using the image sensor,

In the sensor fusion step, the attitude angle change value measured using the attitude sensor is accumulated at the rotation angle of the aiming device. When the attitude angle change value measured using the image sensor is inputted, And updates the measured attitude angle change value.

And the rotational angular velocity of the aiming device is calculated from the attitude angle change value summed in the sensor fusion step.

In the posture measurement step using the image sensor,

A feature point extraction step;

A minutia matching correspondence searching step;

And a representative angle estimation step,

The feature point is characterized by using only the feature points of the background, not the feature points of the target.

And in the feature point correspondence search step, the movement displacement in the unit of the feature point pixel is measured in successive frames.

In the representative angle estimating step, a false displacement is removed using a mathematical technique for eliminating noise among moving displacements of the minutiae points.

In the representative angle estimating step, representative moving displacements obtained by using a mathematical technique for removing the noise are converted into motion angles.

When the number of feature points is less than the threshold value or when the pixel shift displacement per frame is equal to or more than the threshold value, only the attitude angle change value using the attitude sensor is used without using the attitude angle change value using the image sensor.

After the sensor fusion step,

The target information calculating step of calculating the three-dimensional position coordinates and the velocity of the target using the positional angle change value added in the sensor fusion step and the distance information between the target number and the target measured in the distance measuring step is performed.

In the target information calculation step,

Setting initial target coordinates using distance information between the target and the shooter;

Calculating a polar coordinate of a target using the attitude angle change value calculated in the sensor fusion step and the distance information measured in the distance measuring step while the shooter is aiming the target;

Dimensional coordinates and velocity of the target by converting the polar coordinates into three-dimensional rectangular coordinates and differentiating the polar coordinates.

Before the feature point extracting step, the method further includes a cropping step of corping the center region assuming that the target is located in the center region of the image by the aim of the shooter.

The posture measuring step using the posture sensor and the posture measuring step using the image sensor are performed simultaneously.

The attitude measuring step and the distance measuring step are characterized in that they proceed simultaneously while aiming the target.

According to the present invention, it is possible to effectively remove a bias error and a drift error by calculating one posture angle change value by summing up the posture angle change value obtained from the image sensor and the posture angle change value obtained from the posture sensor It is effective.

Therefore, there is an effect that information such as the position and speed of the target can be estimated more accurately.

In addition, it is possible to acquire target information without degrading performance even in an extremely low laser range finder cycle, thereby reducing battery consumption and increasing operation time.

FIG. 1 is a conceptual diagram of a general target information acquisition method according to distance and direction angle measurement values. FIG.
2 is a conceptual diagram illustrating the position and velocity estimation of a moving target according to the present invention.
3 is a flow chart for estimating the position and velocity of a moving target according to the present invention.
FIG. 4 is a conceptual diagram illustrating a feature point corresponding search according to the present invention; FIG.
5 is a diagram of a false displacement removal code according to the present invention.
6 is a conceptual diagram of a pixel-direction angle conversion using a pinhole camera model and a camera viewing angle according to the present invention.
FIG. 7 is a diagram showing a resultant value summation of an image sensor and an attitude sensor according to the present invention. FIG.
8 is a graph of a three-dimensional straight line estimation result using principal component analysis according to the present invention.
FIG. 9 is a graph showing a yaw axial angle estimation result of a stationary target according to the present invention. FIG.
10 is a graph showing a yaw axis direction angle estimation result of a moving target according to the present invention.
11 is a distribution of prediction points of a target using the present invention.

For a better understanding of the present invention, a preferred embodiment of the present invention will be described with reference to the accompanying drawings. The embodiments of the present invention can be modified in various forms, and the scope of the present invention should not be construed as being limited to the embodiments described in detail below. The present embodiments are provided to enable those skilled in the art to more fully understand the present invention. Therefore, the shapes and the like of the elements in the drawings can be exaggeratedly expressed to emphasize a clearer description. It should be noted that the same components are denoted by the same reference numerals in the drawings. Detailed descriptions of well-known functions and constructions which may be unnecessarily obscured by the gist of the present invention are omitted.

The present invention relates to a target information acquiring method for estimating a target speed and position using a distance between a target and a target and a change value of a posture angle of a shooter.

2 is a conceptual diagram of the position and velocity estimation of a moving target according to the present invention.

When the shooter is aiming at a moving target through the aiming device, the target in the image will appear to move but not move. Therefore, it is possible to calculate the pixel shift amount of the background image by applying the image based technique to the background area excluding the central part where the target is located. If the pinhole camera model and the camera specification (viewing angle, size) Can be obtained.

FIG. 3 shows a position and velocity estimation flowchart of a moving target according to the present invention.

In the target information obtaining method of the present invention,

A posture measuring step of measuring a change in posture angle of a shooter while a shooter is aiming a target using a day / night camera including an image sensor;

A posture measuring step of measuring a change in the posture angle of the shooter while the shooter is aiming the target using the posture sensor;

A distance measuring step of measuring a distance between a shooter and a target using a distance measuring sensor;

A sensor fusion step of calculating a single attitude angle change value by summing the attitude angle change value measured using the attitude sensor and the attitude angle change value measured using the image sensor;

A target information calculation step of calculating a three-dimensional position coordinate and a velocity of the target using the posture angle change value summed in the sensor fusion step and the distance information between the target number and the target measured in the distance measurement step;

Lt; / RTI >

The attitude measuring step using the attitude sensor and the attitude measuring step using the image sensor are simultaneously performed, and the angular velocity of the aiming device is calculated from the attitude angle variation value.

In order to obtain the rotational angular velocity of the aiming device using the image-based technique described above, in the posture measuring step using the image sensor,

Cropping steps:

A feature point extraction step;

A minutia matching correspondence searching step;

The representative angle estimation step should proceed.

The feature points extracted in the posture measurement step using the image sensor extract and use only the feature points of the background rather than the feature points of the target.

In the clipping step, as shown in FIG. 4, the center is assumed to be located in the center region of the image by the aim of the shooter, and the center region is corppointed and used.

In the feature point extraction step, feature points are extracted using Harris corner detection. In the feature point correspondence search step, an optical flow technique or the like is used to calculate the displacement of the feature point pixel unit in successive frames .

In order to obtain an optimal solution, it is possible to accurately predict a very small movement because the operation as shown in Equation (1) is repeated. Therefore, it can be mainly used for a technique for calculating the position change of an unmanned robot or a surveillance camera.

In order to calculate the rotational angular velocity of the aiming device from the moving displacements of the minutiae obtained in the minutia matching step, the moving displacements of the minutiae should be represented as one representative component. Since there is a false outlier due to a noise component during the displacement, in the representative angle estimation step, RANSAC (Random Sample Consensus) technique is used to select a model supported from the largest number of displacement displacements of the feature points False displacement is classified. FIG. 5 shows codes for eliminating false displacements using Random Sample Consensus (RANSAC).

In order to obtain the moving angle from the obtained displacement, the field of view of the camera must be considered. As shown in FIG. 6, when the moving displacement is Δx pixel, the angular coordinate system can be converted from the pixel coordinate system to the angular coordinate system by using the viewing angle F ° and the image size V pixel. However, since an angle mismatch occurs between the image coordinate system and the world coordinate system due to the roll angle of the aiming device, it is necessary to perform the roll angle correction using the image rotation transformation formula of the pinhole camera model as shown in Equation (2) .

(X, Y, Z) is a 3D coordinate on the world coordinate system, and [R | t] is a rotation / movement matrix for transforming the world coordinate system into the camera coordinate system. A is a camera matrix, which means camera focal length (f), principal point (c), asymmetry coefficient (skew) and is obtained by performing camera calibration.

In the posture measuring step using the posture sensor, a rotational angle velocity is obtained by measuring a posture angle change value using a general posture sensor such as a gyro sensor.

Both the image sensor and the attitude sensor output the angles of the yaw axis and the pitch axis of the aiming device and their characteristics are different, so that they can perform better when summed. In the case of the image sensor, there is no bias error and the output error is small, but when large motion occurs or there is no background feature point, angle is not obtained and it causes a large error and operates asynchronously.

On the other hand, in the case of the posture sensor, the error is accumulated as the time passes. However, unlike the image sensor, the error is not caused by the background feature point, and the angular velocity is output at a constant speed.

,

)은 상기 조준장치의 회전 각도에 누적시키게 되고, 상기 영상 센서를 이용하여 측정된 자세각 변화값이 입력되면 현재 측정 각도를 상기 영상 센서를 이용하여 측정된 자세각 변화값(

,

)으로 갱신하여 자세 센서의 누적 오차를 개선하였으며, 출력 주기를 30Hz로 유지하였다.Therefore, in the sensor fusion step, the attitude angle change value measured using the attitude sensor

,

) Is accumulated at a rotation angle of the aiming device, and when the measured attitude angle change value is input using the image sensor, the current measured angle is calculated as the attitude angle change value

,

) To improve the cumulative error of the posture sensor, and the output period was maintained at 30 Hz.

7, if the number of the feature points is less than the threshold value or the pixel shift displacement per frame is equal to or greater than the threshold value, the posture angle change value using the image sensor And a large error is prevented from occurring by using only the attitude angle change value using the attitude sensor.

The feature point count threshold value and the pixel shift threshold value are predetermined values through experiments.

After the sensor fusing step, target information for calculating the three-dimensional position coordinates and velocity of the target using the posture angle change value summed in the sensor fusion step and the distance information between the target number and the target measured in the distance measuring step The calculation step proceeds.

In the target information calculation step,

Setting initial target coordinates using distance information between the target and the shooter;

Calculating a polar coordinate of the target using the attitude angle change value calculated in the sensor fusion step and the distance information measured in the distance measuring step while the shooter is aiming the target;

Dimensional coordinates and velocity of the target by converting the polar coordinates into three-dimensional rectangular coordinates and differentiating the polar coordinates.

The posture measurement step and the distance measurement step are simultaneously performed while aiming the target. That is, while the shooter is aiming the target, the attitude measurement and the distance measurement proceed simultaneously.

}의 극좌표 이므로 3차원 공간상의 표적 운동 {

}을 분석하기 위해서 하기의 수학식 3을 이용하여 직교 좌표계로 변환한다.The estimated angular velocity information using the attitude angle change value calculated in the sensor fusion step and the distance information measured in the distance measurement step is {

}, The target motion in the three-dimensional space {

} Is transformed into an orthogonal coordinate system using the following equation (3).

Estimation methods such as the existing EKF (Extended Kalman Filter) have a problem that the target position estimation performance is deteriorated when the input period of the sensor is low, because the dispersion of the data is large. Therefore, even when the acquisition period of the sensor is low, Using the estimable PCA - based linear estimation method, one straight line that best represents the distribution of data is generated and estimated.

The estimation of the straight line in the second plane can be obtained by eigenvalue analysis, and the vector corresponding to the highest value of the eigenvectors of the covariance matrix represents the direction of the straight line. In order to express the movement of the three-dimensional target, it must be extended to the third plane.

To estimate the third-order straight line, SVD (Singular Value Decomposition) of Equation (4) is used, and the matrix A of any m? N can be decomposed as follows.

를 고윳값 분해해서 얻은

직교행렬(orthogonal matrix)이고 V는

를 고윳값 분해해서 얻은

직교행렬이다. 마지막으로

는 를 고윳값 분해해서 나오는 고윳값(eigenvalue)들의 square root(제곱근)를 대각원소로 하는

직사각 대각행렬이다. U is

Obtained by high-resolution decomposition of

Is an orthogonal matrix and V is

Obtained by high-resolution decomposition of

Orthogonal matrix. Finally

The square root of the eigenvalues resulting from high-resolution decomposition is a diagonal element.

It is a rectangular diagonal matrix.

라고 놓으면

는

공분산 매트릭스가 된다. 따라서 SVD는 공분산 행렬의 고윳값 분해의 문제가 되어 3차원 점들로 구성된 공분산 행렬의 고유 벡터 중에서 가장 큰 고윳값에 해당하는 벡터가 직선의 방향을 나타내게 된다. 도 8은 30개의 측정치 샘플에 대해서 3차원 PCA 직선 추정을 수행한 결과이다. A matrix obtained by subtracting the average value of each row from the entire input matrix I as shown in Equation (5)

You release

The

It becomes a covariance matrix. Therefore, SVD is a problem of high-order decomposition of a covariance matrix, so that the vector corresponding to the highest value of the eigenvectors of the covariance matrix composed of three-dimensional points represents the direction of the straight line. 8 is a result of performing a three-dimensional PCA straight line estimation on 30 measurement samples.

FIG. 9 is a graph showing a yaw axial angle estimation result of a stationary target according to the present invention, and FIG. 10 is a graph showing a yaw axial angle estimation result of a moving target according to the present invention.

As a result of the test in the stationary target, it can be confirmed that the error is accumulated as time goes by due to the bias error when only the attitude sensor is used. However, when the method of the present invention is used, it can be confirmed that bias error does not occur. It has been confirmed that the method of the present invention is more accurate in the case of a target moving at a constant speed.

Monte Carlo simulation was performed to verify the performance of the target position and velocity estimating method, and noise was generated by adding the noise according to the performance of each sensor to the normal distribution. The set noise was set by referring to the data sheet and the aim test result.

FIG. 11 shows a distribution of predicted points of a target using the present invention. FIG. 11 shows a distribution of a predicted point of a target when a shot is fired from a launching platform for 2 seconds.

The target distribution of the EKF is distributed along the horizontal axis, whereas the target distribution of the PCA is concentrated at the center, and it is confirmed that the PCA technique has better estimation performance than the EKF when the target is constant velocity and the acquisition period is low. However, the motion characteristics (propulsion, aerodynamics, etc.) and environmental conditions (humidity, wind, altitude, etc.) were not taken into account.

In the target information acquisition method of the present invention as described above, a bias error and a drift are calculated by calculating one posture angle change value by summing the posture angle change value obtained from the image sensor and the posture angle change value obtained from the posture sensor, Therefore, it is possible to more accurately estimate the information such as the position and velocity of the target.

In addition, it is possible to acquire target information without degrading performance even in an extremely low laser range finder cycle, thereby reducing battery consumption and increasing operation time.

The embodiments of the target information acquisition method of the present invention described above are merely illustrative and those skilled in the art will appreciate that various modifications and equivalent embodiments are possible without departing from the scope of the present invention. You will know. It is therefore to be understood that the invention is not limited to the form set forth in the foregoing description. Accordingly, the true scope of the present invention should be determined by the technical idea of the appended claims. It is also to be understood that the invention includes all modifications, equivalents, and alternatives falling within the spirit and scope of the invention as defined by the appended claims.

Claims (12)

A method of acquiring target information using a sighting device including a distance measuring sensor, an attitude sensor, and an image sensor,
A posture measuring step of measuring a change in the posture angle of the shooter while the shooter is aiming the target by using a day / night camera including the image sensor;
A posture measuring step of measuring a posture angle change of a shooter while the shooter is aiming at a target using the posture sensor;
And a sensor fusion step of calculating a single attitude angle change value by summing up the attitude angle change value measured using the attitude sensor and the attitude angle change value measured using the image sensor,
Wherein, when the measured angle change value is measured using the image sensor, the current measured angle is calculated as the current measured angle using the image sensor, And updating the attitude angle change value measured using the image sensor. The method according to claim 1,
And the rotational angular velocity of the aiming device is calculated from the attitude angle change value. 3. The method of claim 2,
In the posture measurement step using the image sensor,
A feature point extraction step;
A minutia matching correspondence searching step;
And a representative angle estimating step,
The feature points are not feature points of the target but only feature points of the background
Wherein the target information acquisition method comprises the steps of: The method of claim 3,
Wherein in the searching for the feature point correspondence, the movement displacement in the unit of the feature point pixel is measured in successive frames. 5. The method of claim 4,
Wherein the representative angle estimating step removes the false displacement using a mathematical technique for removing noise among moving displacements of the minutiae points. 6. The method of claim 5,
Wherein in the representative angle estimating step, a representative moving displacement obtained by using a mathematical technique for removing the noise is converted into a moving angle. The method according to claim 6,
Wherein only the attitude angle change value using the attitude sensor is used instead of the attitude angle change value using the image sensor when the number of the minutiae is less than the threshold or the pixel movement displacement per frame is equal to or more than the threshold value Information acquisition method. The method according to claim 1,
After the sensor fusion step,
And a target information calculating step of calculating the three-dimensional position coordinates and the velocity of the target using the sum of the attitude angle change values summed in the sensor fusion step and the distance information between the target number and the target measured in the distance measuring step To obtain the target information. 9. The method of claim 8,
In the target information calculation step,
Setting initial target coordinates using distance information between the target and the shooter;
Calculating a polar coordinate of the target using the attitude angle change value calculated in the sensor fusion step and the distance information measured in the distance measuring step while the shooter is aiming the target;
Calculating three-dimensional coordinates and speed of the target by converting the polar coordinates into three-dimensional rectangular coordinates and differentiating the polar coordinates;
Wherein the target information obtaining step comprises: The method of claim 3,
Further comprising a cropping step of corpusing the center area assuming that the target is located in the center region of the image by the aim of the shooter before the extraction of the feature point. 9. The method of claim 8,
Wherein the attitude measuring step using the attitude sensor and the attitude measuring step using the image sensor proceed simultaneously. 12. The method of claim 11,
Wherein the attitude measuring step and the distance measuring step are simultaneously performed while aiming the target.

Images