KR102344264B1 - Electro-optical tracking apparatus capable of automatic viewing angle correction and method thereof - Google Patents

Abstract

Provided is an automatic viewing angle correction method in an electro-optical tracking apparatus mounted on a mobile system. The method comprises the steps of: tracking an image of an unmeasured stationary object without prior geographic information through an image sensor in an optical bench; obtaining reference coordinates for the unmeasured stationary object based on laser measurement distance information before and after turning of an aircraft; generating reference viewing angle information corresponding to each viewpoint based on the reference coordinates of the unmeasured stationary object and aircraft coordinates acquired from an aircraft inertial navigation system; acquiring viewing angle information on the unmeasured stationary object being tracked in the image based on an output value of an inertial sensor in the optical bench; estimating a compensation coefficient based on the reference viewing angle information and the viewing angle information; and compensating the viewing angle information of the inertial sensor based on the compensation coefficient. Therefore, the present invention can automatically derive the compensation coefficient and perform compensation in real time without user intervention only by performing an image tracking process for the stationary object during flight.

Description

ELECTRO-OPTICAL TRACKING APPARATUS CAPABLE OF AUTOMATIC VIEWING ANGLE CORRECTION AND METHOD THEREOF

The present invention relates to an electro-optical tracking device mounted on a mobile system capable of automatic gaze angle correction and a method therefor. In particular, an electro-optical tracking device mounted on a mobile system for an un-geometry stationary object without prior geographic information. To an apparatus and method for optical mechanical automatic alignment between an image sensor and an inertial sensor.

1 is a view for explaining an alignment method between an inertial sensor and an image sensor in the prior art.

For the line-of-sight alignment between the existing inertial sensor and the image sensor, the compensation coefficient measured with an optical test equipment such as a collimator during the laboratory manufacturing and calibration process was compensated to the inertial sensor axis.

For more precise alignment, when an image is acquired by driving to the coordinates of the object during flight (S10), the acquired image is subjected to post-processing on the ground to visually identify the object (S20), and the A method of deriving and applying a compensation coefficient by obtaining a pixel error up to the target object (S30) was used (S40, S50).

Since this prior art applies a manual post-processing gaze correction method, there is a problem that a flight sortie must be allocated separately for each equipment unit and for each reassembly. In addition, as a post-processing procedure, the user had to derive a compensation coefficient through visual identification of an object in the image, so the procedure was cumbersome, costly and time-consuming.

In addition, there is a problem in that the target object is limited to a target of known geographic coordinates geodesiced in advance. Therefore, in the prior art, a prior geodetic procedure was required, and the path of the aircraft for observing the target was also cumbersome to be planned in advance.

The problem to be solved by the present invention is to measure the viewing angle of an inertial sensor mounted on the same optical bench as an image sensor in an electro-optical tracking device mounted on a mobile system by using an unmeasured stationary object without prior geographic information as a target object. Provided are an electro-optical tracking device mounted on a mobile system capable of optical mechanical precision alignment based on an image sensor and automatic viewing angle correction, and a method therefor.

However, the technical task to be achieved by the present embodiment is not limited to the above-described technical task, and other technical tasks may exist.

As a technical means for achieving the above-described technical problem, the automatic viewing angle correction method in the electro-optical tracking device mounted on the mobile system according to the first aspect of the present invention is the tail side without prior geographic information through the image sensor in the optical bench. tracking the image of the detected still object; obtaining reference coordinates for the unrecognized stationary object based on laser measurement distance information before and after turning of the aircraft; generating reference line of sight information corresponding to each viewpoint based on the aircraft coordinates obtained from the aircraft inertial navigation system and the reference coordinates of the non-measured stationary object; acquiring line-of-sight information on an un-measured still object that is being tracked in the image based on an output value of an inertial sensor in the optical bench; estimating a compensation coefficient based on the reference viewing angle information and the viewing angle information; and compensating for the viewing angle information of the inertial sensor based on the compensation coefficient.

In some embodiments of the present invention, the step of obtaining the reference coordinates for the unreasonable stationary object includes the coordinates of the i-th (i is a natural number) aircraft, and the i-th obtained laser measurement distance and measurement error. The reference coordinates may be obtained by applying a square estimation method.

In some embodiments of the present invention, the generating of the reference line-of-sight information corresponding to each point of view comprises: setting the aircraft coordinates obtained from the aircraft inertial navigation system and the reference coordinates for the un-measured stationary object to a global coordinate system reference position. providing as a vector; converting the earth coordinate system reference position vector into a navigation coordinate system reference position vector; and generating reference gaze angle information for each viewpoint in the converted navigation coordinate system reference position vector.

In some embodiments of the present invention, the generating of the reference viewing angle information for each viewpoint in the converted navigation coordinate system reference position vector includes generating the heading Euler reference viewing angle information and the pitch Euler reference viewing angle information for each viewpoint can do.

In some embodiments of the present invention, the obtaining of the line-of-sight information for the still object being tracked may include obtaining the line-of-sight information based on a transfer alignment with the aircraft inertial navigation system.

In some embodiments of the present invention, the step of estimating the reference viewing angle information and the compensation coefficient based on the viewing angle information includes applying the reference viewing angle information and the viewing angle information to each transformation matrix to apply a compensation coefficient for each viewpoint. calculating ; and estimating the compensation coefficient based on an average value of Euler angles corresponding to the compensation coefficients for each time point.

In some embodiments of the present invention, the step of calculating the compensation coefficient for each viewpoint through each transformation matrix of the reference gaze angle information and the viewing angle information includes converting the viewpoint generated by the reference gaze angle information into a navigation coordinate system. The compensation coefficient for each viewpoint may be calculated based on a product of one transformation matrix and a second transformation matrix that transforms the navigation coordinate system generated by the gaze angle information into the inertial navigation device-based coordinate system.

Some embodiments of the present invention may further include storing the estimated compensation coefficient in a data storage unit.

In addition, the electro-optical tracking device capable of automatic viewing angle correction according to the second aspect of the present invention includes an image sensor that captures an image of an unreasonable still object without prior geographic information, and laser measurement distance information before and after turning of an aircraft. An optical bench having a gimbal structure including a laser range finder to acquire, an inertial sensor for generating an output value including an acceleration sensor value and a gyro sensor value corresponding to the unreasonable stationary object, An image tracking unit for tracking a still object, an inertial navigation device having a gimbal structure that calculates viewing angle information for an un-measured still object that is being tracked in the image based on an output value of the inertial sensor, and the laser measurement distance information Acquire reference coordinates for an unreasonable stationary object, and generate reference line-of-sight information corresponding to each viewpoint based on the aircraft coordinates obtained from the aircraft inertial navigation system and the reference coordinates of the unreasonable stationary object, and the reference and an inertial sensor automatic alignment device for estimating a compensation coefficient based on the gaze angle information and the gaze angle information, wherein the inertial navigation device compensates the gaze angle information of the inertial sensor based on the compensation coefficient.

In some embodiments of the present invention, the inertial sensor automatic alignment device applies the least-squares estimation method to the coordinates of the i-th (i is a natural number) aircraft, the i-th acquired laser measurement distance and measurement error, and the reference coordinates can be obtained.

In some embodiments of the present invention, the inertial sensor automatic alignment device provides the aircraft coordinates obtained from the aircraft inertial navigation system and the reference coordinates for the unmeasured stationary object as a global coordinate system reference position vector, and the global coordinate system and a reference gaze angle information generator that converts a reference position vector into a reference position vector of the navigation coordinate system and generates reference gaze angle information for each viewpoint in the converted navigation coordinate system reference position vector.

In some embodiments of the present disclosure, the reference viewing angle information generating unit may generate the heading Euler reference viewing angle information and the pitch Euler reference viewing angle information for each viewpoint.

In some embodiments of the present invention, the inertial sensor automatic alignment device calculates a compensation coefficient for each viewpoint by applying each transformation matrix of the reference viewing angle information and the viewing angle information, and corresponds to the compensation coefficient for each viewpoint. It may include a compensation coefficient estimator for estimating the compensation coefficient based on the average value of the Euler angle, and a data storage unit for storing the estimated compensation coefficient.

In some embodiments of the present invention, the compensation coefficient estimator may include a first transformation matrix that converts the viewpoint generated by the reference gaze angle information into a navigation coordinate system, and the inertial navigation device based on the navigation coordinate system generated by the gaze angle information A compensation coefficient for each viewpoint may be calculated based on the product of the second transformation matrix transformed into the coordinate system.

In addition to this, another method for implementing the present invention, another system, and a computer-readable recording medium for recording a computer program for executing the method may be further provided.

Other specific details of the invention are included in the detailed description and drawings.

According to any one of the above-described problem solving means of the present invention, there is an advantage that compensation coefficients can be automatically derived and real-time compensation is possible without user intervention only by performing an image tracking process for a stationary object in flight in a mobile system. This automatic gaze angle correction method does not require a post-processing manual analysis process, and has the advantage of not requiring additional flight sorting assignments because real-time correction is performed during operation. In addition, since more precise optical mechanical correction is possible compared to manufacturing and calibration process equipment errors, it is possible to provide more precise geo-location for image gaze through small residual error improvement.

In particular, the present invention has the advantage that it can always be performed on an unlocated stationary opportunity object, not a planned target with known geographic coordinates, while performing the image tracking process, which is a unique function of the electro-optical tracking device. In the case of a system that restricts target objects to objects with known geographic coordinates, prior geodetic information about specific observable stationary objects on the aircraft's movement path is required. There is a problem that precise geodetic maps are required. In addition, there is a problem in that the operator has to manually load the changed prior geodetic information into the system whenever the target object is changed.

In order to solve this problem, an embodiment of the present invention does not specify a target object, so it does not require a route change of an aircraft and a procedure for analyzing the observable range, and it is possible to simultaneously perform a gaze angle correction during image tracking operation, It has the advantage of enabling efficient equipment operation.

Effects of the present invention are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the following description.

The accompanying drawings below are provided to help understanding of the present embodiment, and provide embodiments together with detailed description. However, the technical features of the present embodiment are not limited to specific drawings, and features disclosed in each drawing may be combined with each other to constitute a new embodiment.
1 is a view for explaining an alignment method between an inertial sensor and an image sensor in the prior art.
2 is a view for explaining an electro-optical tracking device according to an embodiment of the present invention.
3 is a flowchart of an automatic gaze angle correction method according to an embodiment of the present invention.
4 is a view for explaining the content of determining the reference coordinates of the target object.
5 is a diagram for explaining a process of generating reference gaze angle information.
6A and 6B are diagrams for explaining a compensation coefficient estimation step and an automatic gaze alignment step.

Advantages and features of the present invention, and a method for achieving them will become apparent with reference to the embodiments described below in detail in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but may be implemented in various different forms, and only those of ordinary skill in the art to which the present invention pertains, to complete the disclosure of the present invention. It is provided to fully understand the scope of the present invention, and the present invention is only defined by the scope of the claims.

The terminology used herein is for the purpose of describing the embodiments and is not intended to limit the present invention. In this specification, the singular also includes the plural unless specifically stated otherwise in the phrase. As used herein, “comprises” and/or “comprising” does not exclude the presence or addition of one or more other components in addition to the stated components. Like reference numerals refer to like elements throughout, and "and/or" includes each and every combination of one or more of the recited elements. Although "first", "second", etc. are used to describe various elements, these elements are not limited by these terms, of course. These terms are only used to distinguish one component from another. Accordingly, it goes without saying that the first component mentioned below may be the second component within the spirit of the present invention.

Unless otherwise defined, all terms (including technical and scientific terms) used herein will have the meaning commonly understood by those of ordinary skill in the art to which this invention belongs. In addition, terms defined in a commonly used dictionary are not to be interpreted ideally or excessively unless specifically defined explicitly.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

2 is a view for explaining the electro-optical tracking device 100 according to an embodiment of the present invention.

The electro-optical tracking device 100 mounted on a mobile system capable of automatic gaze angle correction according to an embodiment of the present invention includes an optical bench 110 , an image tracking unit 120 , an inertial navigation device 130 , and an automatic inertial sensor. and an alignment device 140 .

The optical bench 110 is an optical bench having a gimbal structure, and includes an image sensor 111 , an inertial sensor 112 , a laser rangefinder 113 , and a driver 114 .

The image sensor 111 may be an IR sensor (Infrared Sensor), a Charge Coupled Device (CCD) camera, a laser image sensor (Lidar-Vision Sensor), or the like, and captures an image of a still object. In this case, an embodiment of the present invention is characterized in that an image of an unrecognized still object without prior geographic information is captured. The image tracking unit 120 tracks an unrecognized still object through the captured image.

The inertial sensor 112 includes a gyro and an accelerometer, and generates an output value including an acceleration sensor value and a gyro sensor value corresponding to an unmeasured stationary object.

The laser range finder 113 acquires laser measurement distance information before and after turning of the aircraft.

The inertial navigation device 130 has a gimbal structure, and based on an output value of the inertial sensor 112 , calculates gaze angle information for an un-measured still object that is being tracked in an image. The inertial navigation device 130 includes a transfer alignment Kalman filter unit 131 , an inertial navigation calculation unit 132 , and a compensation coefficient compensator 133 .

The transmission alignment Kalman filter unit 131 estimates the transmission alignment error by using an auxiliary sensor that provides position, speed, and attitude information, such as an aircraft inertial navigation system.

The inertial navigation calculation unit 132 receives the output value of the inertial sensor 112, calculates the position, speed, and attitude of the electro-optical tracking device 100 based on the input value, and compensates for the estimated transmission alignment error.

The compensation coefficient compensator 133 receives the compensation coefficient estimated by the inertial sensor automatic alignment device 140 and compensates the viewing angle information of the inertial sensor 112 in real time.

Next, the inertial sensor automatic alignment device 140 includes a reference coordinate obtaining unit 141 , a reference viewing angle information generating unit 142 , a compensation coefficient estimating unit 143 , and a data storage unit 144 .

The reference coordinate acquisition unit 141 acquires reference coordinates for an unmeasured stationary object based on the laser measurement distance information.

The reference line of sight information generating unit 142 performs an image tracking process for a still object that has not been measured during the flight of the aircraft, while the reference corresponding to each viewpoint in real time based on the moving coordinates of the aircraft and the reference coordinates of the unmeasured stationary object. Generates visual angle information.

The compensation coefficient estimator 143 estimates the compensation coefficient based on the reference gaze angle information and the gaze angle information from the inertial navigation device 130 .

The data storage unit 144 may be configured as, for example, a flash memory type, and when the image tracking process is finished, the estimated compensation coefficient is stored, and the compensation coefficient is provided to the inertial navigation device 130 in real time.

When there is a stored effective compensation coefficient, the inertial navigation device 130 receiving the compensation coefficient may receive the value and compensate for the gaze angle information of the inertial sensor 112 calculated in real time.

For reference, the components shown in FIG. 2 according to an embodiment of the present invention may be implemented in the form of software or hardware such as a field programmable gate array (FPGA) or an application specific integrated circuit (ASIC), and perform predetermined roles. can do.

However, the term 'components' is not limited to software or hardware, and each component may be configured to reside in an addressable storage medium or to reproduce one or more processors.

Thus, as an example, a component includes components such as software components, object-oriented software components, class components, and task components, and processes, functions, properties, procedures, sub It includes routines, segments of program code, drivers, firmware, microcode, circuitry, data, databases, data structures, tables, arrays, and variables.

Components and functions provided within the components may be combined into a smaller number of components or further divided into additional components.

Hereinafter, an automatic gaze angle correction method in the electro-optical tracking device 100 mounted on a mobile system according to an embodiment of the present invention will be described with reference to FIGS. 3 to 5B .

3 is a flowchart of an automatic gaze angle correction method according to an embodiment of the present invention.

First, an image of an unreasonable still object without prior geographic information is tracked through the image sensor 111 in the optical bench 110 (S110), and reference coordinates for an unreasonable still object based on laser measurement distance information to obtain (S120).

4 is a view for explaining the content of determining the reference coordinates of the target object.

The general method of acquiring target coordinates of an electro-optical tracking device is calculated using trigonometry or geo-location algorithm using aircraft coordinates, line of sight, and laser measurement distance or electronic altitude map information. However, in the case of an embodiment of the present invention, reference coordinates are acquired to generate reference viewing angle information, and coordinates obtained as gaze angle measurement data of the inertial sensor cannot be reference coordinates, so a general target coordinate acquisition method is applied. Can not. Accordingly, in the case of an embodiment of the present invention, the reference coordinates of the unmeasured stationary object may be calculated using the least squares method as a linear function of the laser measurement distance information and the aircraft coordinates.

)와 레이저 측정 거리 정보(

,

)를 획득한다면, 각각 구 표면(

,

)의 가능한 좌표를 획득하게 된다. 그리고 구 표면의 교점인 2개의 가능한 좌표(

,

)를 획득한다. 이 중 대상물체는 항공기의 고도보다 낮다는 제한을 둔다면 T가 기준좌표가 된다.4 is a spherical surface showing the possible positions of an unreasonable stationary object with respect to the laser measurement distance information obtained at the positions of each aircraft. two aircraft coordinates (

) and laser measurement distance information (

,

), each spherical surface (

,

) to obtain possible coordinates of And the two possible coordinates of the intersection of the sphere surfaces (

,

) is obtained. Among them, if there is a restriction that the target object is lower than the altitude of the aircraft, T becomes the reference coordinate.

는 하기 수학식 1에 의해 결정될 수 있으며, i번째(i는 자연수) 획득한 레이저 측정 거리, 측정오차, i번째 항공기 좌표를 대상으로 선형화에 기반한 반복적인 기법인 최소자승추정법(Linear Square Estimation)을 적용하여 획득할 수 있다.reference coordinates

can be determined by the following Equation 1, and the i-th (i is a natural number) obtained laser measurement distance, measurement error, and linearization-based iterative technique based on linearization targeting the i-th aircraft coordinates Least Squares Estimation (Linear Square Estimation) It can be obtained by applying

[Equation 1]

를 알 수 있으며, 실제 위치에서의 오차

)를 나타낼 수 있다. 개략적인 위치에 대해 테일러 급수에서 i개의 샘플로 확장한다면, 위치 오차를 산출할 수 있으며 이를 통해 기준좌표

도 획득할 수 있다.The reference coordinates of the target object can be obtained through the nonlinear equation of Equation 1 above. Schematic coordinates from the viewing angle information that is the output of the inertial navigation calculator 132 .

can be known, and the error in the actual position

) can be represented. If we extend the Taylor series to i samples for the approximate position, we can calculate the position error, and through this

can also be obtained.

Next, based on the aircraft coordinates obtained from the aircraft inertial navigation system 10 and the reference coordinates of the non-measured stationary object, reference line of sight information corresponding to each viewpoint is generated ( S130 ).

5 is a diagram for explaining a process of generating reference gaze angle information.

FIG. 5 shows how to acquire reference coordinates and generate reference viewing angle information while performing image tracking for an unrecognized still object without prior geographic information at several viewpoints.

The process of obtaining the reference viewing angle information for calculating the compensation coefficient is as follows.

First, the aircraft coordinates obtained from the aircraft inertial navigation system 10 and the reference coordinates of an unmeasured stationary object are expressed as a position vector based on the earth coordinate system {e}.

Next, the global coordinate system reference position vector is converted into a position vector based on the aircraft-centered navigation coordinate system {n}, and reference line of sight information for each viewpoint of the converted navigation coordinate system reference position vector is generated. In this case, as the reference viewing angle information, heading Euler reference angle information and pitch Euler reference viewing angle information for each viewpoint are generated.

The reference viewing angle information may be generated based on Equation 2 below.

[Equation 2]

는 지구 좌표계 {e}에서의 i 시점에서의 항공기 좌표를 나타내고,

는 지구 좌표계 {n}에서의 미측지된 정지물체의 좌표를 나타낸다.

(i)는 i시점에서의 지구 좌표계 {e}에서 항법 좌표계 {n}으로의 변환행렬을 나타내고,

는 i시점에서의 항법 좌표계 {n} 기준

시선벡터의 헤딩 오일러각, 그리고

는 i시점 항법 좌표계 {n} 기준

시선벡터의 피치 오일러각을 각각 나타낸다.In Equation 2 above

represents the coordinates of the aircraft at time i in the earth coordinate system {e},

denotes the coordinates of an unrecognized stationary object in the earth coordinate system {n}.

(i) represents the transformation matrix from the earth coordinate system {e} to the navigation coordinate system {n} at time i,

is based on the navigation coordinate system {n} at point i

The heading Euler angle of the gaze vector, and

is based on the i-point navigation coordinate system {n}

Each of the pitch Euler angles of the gaze vector is indicated.

Next, based on the output value of the inertial sensor 112 in the optical bench 110, information on the viewing angle of the un-measured still object being tracked in the image is acquired (S140).

In this process, the information on the viewing angle in the navigation coordinate system is measured through the intermediate inertial sensor 112 mounted on the same optical bench 110 as the image sensor 111 for tracking an unreasonable stationary object. Based on the transmission alignment with the device 10 , the viewing angle information of the inertial sensor 112 may be acquired with navigation-class performance.

Next, a compensation coefficient is estimated based on the reference gaze angle information and the gaze angle information obtained from the inertial navigation device 130 ( S150 ).

6A and 6B are diagrams for explaining a compensation coefficient estimation step and an automatic gaze alignment step.

Specifically, the reference gaze angle information and the gaze angle information obtained from the inertial navigation device 130 are applied to each transformation matrix to calculate a compensation coefficient for each viewpoint, and to the average value of Euler angle corresponding to the compensation coefficient for each viewpoint. Based on this, the compensation coefficient can be estimated.

는 다음 수학식 3에 기초하여 산출할 수 있다.First, the compensation coefficient for each time point

can be calculated based on Equation 3 below.

[Equation 3]

는 기준 시선각 정보로 생성한 시점 {los}을 항법 좌표계 {n}으로 변환하는 제1 변환행렬을 나타내고,

는 관성항법장치의 시선각 정보로 생성한 항법 좌표계 {n}에서 관성항법장치 기반 좌표계 {gins}으로 변환하는 제2 변환행렬을 나타낸다.in Equation 3

denotes a first transformation matrix that transforms a viewpoint {los} generated with reference viewing angle information into a navigation coordinate system {n},

denotes a second transformation matrix that transforms the navigation coordinate system {n} generated with the gaze angle information of the inertial navigation device into the inertial navigation device-based coordinate system {gins}.

는 제1 변환행렬과 제2 변환행렬의 곱에 기초하여 산출된다.Compensation coefficient for each time point

is calculated based on the product of the first transformation matrix and the second transformation matrix.

After the compensation coefficient for each time point is calculated, the compensation coefficient M is determined as the average value of the Euler angles corresponding to the compensation coefficient for each time point.

Next, the derived compensation coefficient M is stored in the data storage unit 144, and thereafter, the gaze angle information of the inertial sensor 112 is compensated in real time (S160).

는 다음 수학식 4와 같이 나타낼 수 있다.The gaze angle information of the inertial sensor 112 compensated by automatic gaze alignment (

can be expressed as in Equation 4 below.

[Equation 4]

6A before applying the compensation coefficient is a state in which the information on each viewing angle of the inertial sensor 112 and the image sensor 111 is not aligned. Since quadrature correction is possible, it is possible to provide more precise spatial information about the image line of sight through small residual error improvement.

In the above description, steps S110 to S160 may be further divided into additional steps or combined into fewer steps according to an embodiment of the present invention. In addition, some steps may be omitted if necessary, and the order between steps may be changed. In addition, the contents described in FIG. 2 may be applied to the automatic viewing angle correction method of FIGS. 3 to 6B even if other contents are omitted.

According to the electro-optical tracking device and method mounted on the mobile system capable of automatic viewing angle correction according to an embodiment of the present invention described above, if only the laser distance measurement information and the image tracking process are performed, a correction coefficient is automatically derived and precise There is an advantage that photomechanical correction can be performed in real time. In addition, the targeting performance is improved with precise gaze angle information of the electro-optical tracking device through real-time residual error improvement.

The automatic gaze angle correction method in the electro-optical tracking apparatus 100 according to an embodiment of the present invention described above may be implemented as a program (or application) to be executed in combination with a computer, which is hardware, and stored in a medium. have.

The above-mentioned program, in order for the computer to read the program and execute the methods implemented as a program, C, C++, JAVA, Ruby, which the processor (CPU) of the computer can read through the device interface of the computer; It may include code coded in a computer language such as machine language. Such code may include functional code related to a function defining functions necessary for executing the methods, etc., and includes an execution procedure related control code necessary for the processor of the computer to execute the functions according to a predetermined procedure. can do. In addition, this code may further include additional information necessary for the processor of the computer to execute the functions or code related to memory reference for which location (address address) in the internal or external memory of the computer should be referenced. have. In addition, when the processor of the computer needs to communicate with any other computer or server located remotely in order to execute the functions, the code uses the communication module of the computer to determine how to communicate with any other computer or server remotely. It may further include a communication-related code for whether to communicate and what information or media to transmit and receive during communication.

The storage medium is not a medium that stores data for a short moment, such as a register, a cache, a memory, etc., but a medium that stores data semi-permanently and can be read by a device. Specifically, examples of the storage medium include, but are not limited to, ROM, RAM, CD-ROM, magnetic tape, floppy disk, and an optical data storage device. That is, the program may be stored in various recording media on various servers accessible by the computer or in various recording media on the computer of the user. In addition, the medium may be distributed in a computer system connected to a network, and a computer-readable code may be stored in a distributed manner.

The above description of the present invention is for illustration, and those of ordinary skill in the art to which the present invention pertains can understand that it can be easily modified into other specific forms without changing the technical spirit or essential features of the present invention. will be. Therefore, it should be understood that the embodiments described above are illustrative in all respects and not restrictive. For example, each component described as a single type may be implemented in a dispersed form, and likewise components described as distributed may be implemented in a combined form.

The scope of the present invention is indicated by the following claims rather than the above detailed description, and all changes or modifications derived from the meaning and scope of the claims and their equivalent concepts should be interpreted as being included in the scope of the present invention. do.

10: aircraft inertial navigation system
100: electro-optical tracking device
110: optical bench
120: video tracking unit
130: inertial navigation device
140: inertial sensor automatic alignment device

Claims (14)

In the automatic gaze angle correction method in an electro-optical tracking device mounted on a mobile system,
tracking an image of an unrecognized still object without prior geographic information through an image sensor in the optical bench;
obtaining reference coordinates for the unrecognized stationary object based on laser measurement distance information before and after turning of the aircraft;
generating reference line-of-sight information corresponding to each viewpoint based on the aircraft coordinates obtained from the aircraft inertial navigation system and the reference coordinates of the non-measured stationary object;
obtaining information on the viewing angle of the un-measured still object that is being tracked in the image based on an output value of the inertial sensor in the optical bench;
estimating a compensation coefficient based on the reference viewing angle information and the viewing angle information; and
and compensating for the viewing angle information of the inertial sensor based on the compensation coefficient.
According to claim 1,
The step of obtaining the reference coordinates for the non-measured stationary object comprises:
Automatic gaze angle correction method in an electro-optical tracking device to obtain the reference coordinates by applying the least-squares estimation method to the coordinates of the i-th aircraft (i is a natural number), the laser measurement distance and the measurement error obtained in the i-th .
According to claim 1,
The step of generating the reference viewing angle information corresponding to each viewpoint includes:
providing the aircraft coordinates obtained from the aircraft inertial navigation system and the reference coordinates for the non-measured stationary object as a global reference position vector;
converting the earth coordinate system reference position vector into a navigation coordinate system reference position vector; and
and generating reference viewing angle information for each viewpoint in the converted navigation coordinate system reference position vector.
4. The method of claim 3,
The step of generating reference viewing angle information for each viewpoint in the converted navigation coordinate system reference position vector includes:
The automatic gaze angle correction method in the electro-optical tracking device to generate the heading Euler reference gaze angle information and the pitch Euler reference gaze angle information for each viewpoint.
According to claim 1,
The step of obtaining the gaze angle information for the still object being tracked in the image comprises:
An automatic gaze angle correction method in an electro-optical tracking device that acquires the gaze angle information based on a transfer alignment with the aircraft inertial navigation system.
According to claim 1,
The step of estimating a compensation coefficient based on the reference viewing angle information and the viewing angle information includes:
calculating a compensation coefficient for each viewpoint by applying the reference viewing angle information and the viewing angle information to each transformation matrix; and
and estimating the compensation coefficient based on the average value of Euler angles corresponding to the compensation coefficients for each viewpoint.
7. The method of claim 6,
The step of calculating the compensation coefficient for each viewpoint through each transformation matrix of the reference viewing angle information and the viewing angle information includes:
Based on the product of a first transformation matrix that transforms a viewpoint generated by the reference gaze angle information into a navigation coordinate system, and a second transformation matrix that transforms a viewpoint generated by the reference gaze angle information into a coordinate system based on the inertial navigation device in the navigation coordinate system generated with the gaze angle information An automatic gaze angle correction method in an electro-optical tracking device that calculates a compensation coefficient for each viewpoint.
According to claim 1,
The automatic gaze angle correction method in the electro-optical tracking device further comprising the step of storing the estimated compensation coefficient on a data storage unit.
An image sensor that captures an image of an unrecognized stationary object without prior geographic information, a laser rangefinder that acquires laser distance information before and after turning of an aircraft, and an acceleration sensor value and a gyro corresponding to the unreasonable stationary object An optical bench of a gimbal structure including an inertial sensor that generates an output value including a sensor value,
an image tracking unit for tracking the unrecognized still object from the captured image;
An inertial navigation device having a gimbal structure that calculates line-of-sight information for an un-measured stationary object being tracked in the image based on the output value of the inertial sensor, and
Based on the laser measurement distance information, the reference coordinates for the unreasonable stationary object are acquired, and based on the aircraft coordinates obtained from the aircraft inertial navigation system and the reference coordinates of the unreasonable stationary object, a reference corresponding to each time point An inertial sensor automatic aligning device for generating gaze angle information and estimating a compensation coefficient based on the reference gaze angle information and the viewing angle information,
The inertial navigation device is an electro-optical tracking device capable of automatic gaze angle correction that compensates the gaze angle information of the inertial sensor based on the compensation coefficient.
10. The method of claim 9,
The inertial sensor automatic alignment device,
An electro-optical tracking device capable of automatic line-of-sight correction that obtains the reference coordinates by applying the least-squares estimation method to the coordinates of the i-th (i is a natural number) aircraft and the i-th obtained laser measurement distance and measurement error.
10. The method of claim 9,
The inertial sensor automatic alignment device,
The aircraft coordinates obtained from the aircraft inertial navigation device and the reference coordinates for the unmeasured stationary object are provided as a global reference position vector, and the global coordinate system reference position vector is converted into a navigation coordinate system reference position vector, and the converted An electro-optical tracking device capable of automatic gaze angle correction including a reference gaze angle information generator for generating reference gaze angle information for each viewpoint in the navigation coordinate system reference position vector.
12. The method of claim 11,
The reference viewing angle information generating unit,
An electro-optical tracking device capable of automatic viewing angle correction that generates the heading Euler reference viewing angle information and the pitch Euler reference viewing angle information for each viewpoint.
10. The method of claim 9,
The inertial sensor automatic alignment device,
Compensation coefficient estimation for calculating a compensation coefficient for each viewpoint by applying the reference viewing angle information and each transformation matrix of the viewing angle information, and estimating the compensation coefficient based on the average value of Euler angles corresponding to the compensation coefficient for each viewpoint government and
An electro-optical tracking device capable of automatic gaze angle correction comprising a data storage unit for storing the estimated compensation coefficient.
14. The method of claim 13,
The compensation coefficient estimating unit,
Based on the product of a first transformation matrix that transforms a viewpoint generated by the reference gaze angle information into a navigation coordinate system, and a second transformation matrix that transforms a viewpoint generated by the reference gaze angle information into a coordinate system based on the inertial navigation device in the navigation coordinate system generated with the gaze angle information An electro-optical tracking device capable of automatic viewing angle correction that calculates a compensation coefficient for each viewpoint.

Images