KR101877232B1 - Image tracking system based on image similarity and tracking point variation and method for determining tracking stability thereof - Google Patents

Abstract

The present invention relates to an image tracker system and a tacking stability determining method thereof, capable of determining whether a tracking state is stable by using image similarity and a tracking point variation. The image tracker system includes an image sensor for sensing a tracking image; and an image tracking unit for buffering the intervention of an operator by determining the tracking stability for a target at which the operator aims by deriving similarity information and the tracking point variation from the tracking image sensed in the image sensor if the target aiming of the operator is attempted in an operation/induction control action situation of a platform.

Description

TECHNICAL FIELD The present invention relates to a video tracker system based on image similarity and tracking point variation, and a method of determining tracking stability of the same.

 The present invention relates to a video tracker system capable of determining whether a tracking state is stable by using a degree of similarity of images and a variation amount of a tracking point, and a tracking stability determination method thereof.

In general, the platform operating system is equipped with a video tracker, which is an electronic optical tracker (EOTS) that tracks and detects the target using image information obtained by the image sensor. The image tracker automatically tracks the moving target even in the presence of disturbance such as vibration in an aircraft such as a ship, a train, and a helicopter, and provides the observer with a stable line of sight to the target, Position, speed, angular velocity, etc. to the fire control system.

Such a video tracker may be generally constituted by an image sensor, a video tracker, and a driving unit. The driving unit includes all mechanical devices and an electronic control device for tracking the position of the target calculated by the video tracker do. Therefore, it is accompanied with the difficulty of simultaneously controlling the mechanical parts including the non-linear effect as well as the electron parts which are modeled and calculated relatively ideally, and the disturbance due to the mechanical vibration generated during the movement of the image tracker and the image processing Time delay greatly affect the performance of the entire image tracker.

In the image tracker constructed as described above, the tracking image directly recognized by the operator has a time delay in terms of operation principle, and the time delay characteristic affects the performance of the entire platform operating system. Especially, the tracking stability of the video tracker is a key factor affecting the performance of the platform operation system when controlling the operation / induction of the platform through the image tracking method using the video tracker.

Due to the above-described problems, prior to operating the operation / guidance function of the platform, it is determined whether or not the traceable reference value can be secured in a given situation based on the information modeled by the image tracker to discriminate the tracking stability of the image tracker The method of determining the tracking stability of the video tracker before controlling the operation / induction of the platform is used or not presented.

There is no way to operate the image tracking at a specific point in time when the platform operation / guidance control is being performed, or to counteract or buffer the possibility of failure in updating the target. Particularly, conventionally, when a target aiming attempt (command) of an operator occurs in the target searching step of the platform operation, the image buffering storage is performed to compensate the communication or system delay and perform the tracking operation.

However, conventionally, due to the target-aiming intervention of an unstable operator, a tracking error may be generated or a tracking result for controlling the platform operation may be defective.

An object of the present invention is to provide a video tracker system and a tracking stability determination method thereof that can determine the tracking stability of a video tracker system in an operator intervention in an operation / induction control operation state of a platform.

Another object of the present invention is to provide a video tracker system capable of stably maintaining the operation / induction control operation of the platform by discriminating the tracking stability using the image similarity degree and the tracking point variation amount and eliminating the instability command of the unstable operator, .

According to an aspect of the present invention, there is provided a video tracker system for controlling operation / induction of a platform according to an embodiment of the present invention includes an image sensor for detecting a tracked image; When the operator's target aiming is attempted in the operation / induction control operation situation of the platform, the degree of similarity information and the change point of the tracking point are derived from the tracking image detected by the interim sensor and the tracking stability is determined for the target aimed by the operator, And a video tracking unit for buffering the video signal.

According to an embodiment of the present invention, the tracking stability indicates whether the target can be tracked.

According to the embodiment of the present invention, when the delay occurs in the system, the image tracking unit determines the tracking stability using the buffering delay image, and if the delay does not occur, the image stabilization can be determined by buffering the predetermined image have.

According to an embodiment of the present invention, the image tracking unit extracts image similarity information by comparing reference information extracted from a tracking image sensed at a predetermined period with previously stored information, and derives a tracking point indicating a position of a target for each tracking image A vector is formed by connecting a tracking point derived from each of the tracking images to a tracking point derived from a previous image, and a tracking point variation is calculated by accumulating a sum of vector differences between the formed vectors, And a tracking stability of the target can be determined by associating the calculated tracking point variation with the previously learned data distribution.

According to an embodiment of the present invention, the reference information may include a color, a brightness difference, and a distribution of histograms of an image, and the image-tracking unit may calculate the cumulative calculated tracking point variation amount even if the extracted image similarity information is a predetermined value or more It is determined that the possibility of tracking failure is high and the tracking operation is not performed.

According to an aspect of the present invention, there is provided a tracking stability determination method of a video tracker system for controlling operation / induction of a platform according to an embodiment of the present invention includes: acquiring a tracking image periodically through an image sensor; Calculating similarity information from a tracking image obtained by the image sensor when an operator's target aiming is attempted in an operation / induction control operation state of the platform; Calculating a change amount of a tracking point by deriving a tracking point indicating a position of a target for each tracking image acquired by the image sensor; And determining the tracking stability using the calculated similarity information of the tracking image and the variation amount of the tracking point.

In the step of determining the tracking stability according to an embodiment of the present invention, the tracking stability is determined using a buffering delay image when a delay occurs in the system, and when the delay does not occur, Can be determined.

According to an exemplary embodiment of the present invention, the step of calculating the similarity information may include extracting image similarity information by comparing the reference information extracted from the target image acquired at a predetermined period with previously stored information, Differences, and distribution of histograms.

According to an embodiment of the present invention, the step of calculating the change amount of the tracking point may include: deriving a tracking point indicating a position of the target for each tracking image; And forming a vector by connecting a tracking point derived from each tracking image with a tracking point derived from a previous image, and then calculating a tracking point variation by accumulating a sum of vector differences between the vectors.

The step of discriminating the tracking stability according to an embodiment of the present invention may include determining the tracking stability of the target by associating the calculated image similarity degree information and the calculated tracking point variation with a previously learned data distribution, Even if the image similarity degree information is equal to or greater than a predetermined value, it can be determined that the possibility of tracking failure is high if the cumulative calculated tracking point variation amount is equal to or greater than a predetermined value.

According to the present invention, the tracking stability of the video tracker system, which is a platform operating system, is determined by using the image similarity information and the tracking point geometry analysis method even during the operator intervention in the platform operation / inductive control operation state of the platform operation system can do. Therefore, the present invention can prepare the target loss situation due to the tracking anxiety in the target aiming instruction by the operator, quantify and quantify the information of the image tracker necessary for the platform operation / guidance control, The performance of the video tracker system can be improved and the platform operation / induction control operation can be stably maintained, thereby achieving accurate platform operation / inductive control performance. There is an effect that can be increased.

1 is a conceptual diagram illustrating a method of measuring image similarity of an image target.
FIG. 2 is an exemplary view of the amount of image similarity information; FIG.
FIG. 3 is a graph illustrating vector motion between tracking points derived from each image frame according to an embodiment of the present invention. FIG.
4 is an example of a tracking point pattern using the vector derived from FIG.
5 is a flow chart for determining a tracking failure / success based on a change amount of a tracking point in an explorer tracking system according to an embodiment of the present invention.
6 is an example of an image in which the similarity degree is high but the similar pattern is repeated around.
FIG. 7 is a flow chart for determining tracking stability based on image similarity and tracking point variation in a video tracker system according to an embodiment of the present invention; FIG.
8 is a diagram illustrating an example of logic for determining whether or not a tracking operation is performed according to the calculated image similarity and movement amount of a tracking point in the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, preferred embodiments of the present invention will be described with reference to the accompanying drawings.

The suffix "module" and " part "for the components used in the following description are given or mixed in consideration of ease of specification, and do not have their own meaning or role.

Although the terms including ordinals such as first, second, etc. in this specification can be used to describe various elements, the elements are not limited by the terms. The terms are used only for the purpose of distinguishing one component from another. In addition, the singular forms "a," "an," and "the" include plural referents unless the context clearly dictates otherwise.

Whenever a component is referred to as "including" an element herein, it is to be understood that it may include other elements, unless the context otherwise requires.

In the following description of the embodiments of the present invention, a detailed description of related arts will be omitted when it is determined that the gist of the embodiments disclosed herein may be blurred. It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory and are intended to provide further explanation of the invention as claimed. , ≪ / RTI > equivalents, and alternatives.

The present invention proposes a method for determining the tracking stability of a platform operating system, and more particularly, a video tracker system, by using image similarity information and tracking point geometry analysis method during operator intervention in a platform operation / guidance control operation situation.

To this end, the present invention can utilize a method of determining the degree of similarity of images by comparing the reference model of the image with the target model. The degree of similarity of the image can be determined by comparing the reference model of the image with the object model, and various characteristics such as the color, brightness difference, histogram distribution, etc. of the image can be included as reference information.

The image tracker according to the present invention includes an image sensor for sensing (or photographing) a target image, a target image tracked by analyzing the target image detected by the image sensor, And a driving unit for driving the mechanical and electronic control devices so that the tracking is performed based on the tracking and discrimination results of the image tracking unit.

The driving unit may include a mechanism unit for supporting the image sensor and a servo control unit for moving the mechanism unit. The servo control unit may further include a motor and a servo amplifier, a gyroscope for stabilization, and hardware and software that combine them.

FIG. 1 is a conceptual diagram illustrating a method of measuring image similarity of an image target unit, and FIG. 2 is an example of an amount of image similarity information.

As shown in FIG. 1, the image tracker compares reference information (eg, color, brightness difference, histogram distribution) extracted from a reference model (input image) with an object model, And marks the center of the image tracking or the position of the target box as a result of the tracking. At this time, the operator judges that the tracking is normally performed when the tracking result is located in the predicted object, and determines that the tracking is failed if the tracking result is located at a place other than the predicted object.

In general, the performance of a video tracker can be classified according to the amount of image similarity information. For example, as shown in FIG. 2, if the degree of similarity of image information is high (clear weather) due to a large number of image similarity information, it can be classified as being well tracked. On the other hand, In low cases (cloudy weather), it can be classified as non-traceable. Therefore, if the similarity of the image information is used only for the target tracking, there is a possibility that the target tracking may fail if there is a similar clutter around or a certain continuous pattern (similar pattern) exists.

In order to overcome the above problems, the present invention proposes a method of determining the tracking stability by using the geometric analysis of the tracking points in addition to the image similarity.

In the present invention, the geometric analysis for the tracking point is based on the vector amount of the tracking point. The image is acquired with a certain period in the image sensor, and each image is called a frame. The image tracking unit of the image target machine derives the position of the target for each frame obtained from the image sensor. At this time, the derived target position is defined as a tracking point, and the vector amount of the tracking point can be calculated through a geometric analysis of the chronological tracking points obtained from images accumulated in a plurality of chapters.

FIG. 3 is a graph illustrating vector shifts between tracking points derived from each image frame according to an embodiment of the present invention, and FIG. 4 illustrates a tracking point pattern using the vector formed in FIG.

In FIG. 3, the target position (x _t , y _t ) obtained by deriving the position of the target in the t frame is obtained by connecting the tracking point derived from each frame to the tracking point derived from the previous frame, The trace point pattern using the defined vectors V1 and V2 can be calculated using the geometric formulas shown in FIG.

However, it is difficult to specify the value of the tracking point patterns as a feature of determining the tracking failure or tracking success of the target. This is because, when the inner product of the vectors (V1, V2) is small, it may be a value when the tracking point bounces or the target moves in a certain direction. Also, when considering the direction, it is difficult to clearly distinguish the tracking failure / success by the value of the tracking point patterns when the tracking point is shaken opposite to each other, that is, when there is a clutter on the target or when only the target consists of continuous lines .

Accordingly, the present invention proposes a scheme using the sum of vector difference magnitudes shown in FIG. 4 to determine tracking failure / success based on the amount of change of the tracking point.

FIG. 5 is a flowchart for determining a tracking failure / success based on a change amount of a tracking point in an explorer tracking system according to an exemplary embodiment of the present invention. FIG. 6 is an example of an image in which a similarity pattern is repeated .

As shown in FIG. 5, when a target aiming attempt (command) is made by the operator (or shooter) in the target searching step of the platform operation, the operator performs a procedure of determining whether or not the target is trackable (S100, S110). Such a determination procedure may be applied to a step of performing image buffer storage to compensate for delay when communication and system delay occur. In addition, the determination procedure may be performed in a step of buffering two or three video frames even if there is no delay due to communication and system. The target aiming attempt by the operator may include a target detection area setting, a target start button input, and other manipulation commands in the image.

For example, the image shown in Fig. 6 is an image in which a similar pattern is repeated in the vicinity although there are many similarity degree information. When an operator attempts to aim a target in such an image, the target points derived from the plurality of image frames become unstable as indicated by solid or dotted rectangles.

Therefore, in order to identify the above case and prevent a loss in tracking operation, the present invention buffers an image at a predetermined time and calculates a geometric error (tracking point movement amount) of the tracking point using the following equation (1) After the accumulation, it is compared with the reference value to discriminate whether it is traceable (S120)

[Equation 1]

Here, N is the number of image frames loaded in the delay buffer.

That is, when the tracking is tried in Equation (1), the cumulative value of the movement amount of the tracking point is compared with the reference value, and if the movement of the tracking point is small and smaller than the reference value, (S130, S140). On the other hand, if the movement of the tracking point occurs a lot and is larger than the reference value, it is recognized that the tracking failure possibility is high, and the process returns to step S110 without passing on to the tracking operation step as the next step and receives the operator's next aiming attempt .

FIG. 7 is a flow chart for determining the tracking stability through the image similarity and the tracking point geometric analysis in the image tracker system according to the embodiment of the present invention. FIG. 8 is a flowchart illustrating the tracking operation according to the image similarity calculated in the present invention, And the like.

As shown in FIG. 7, when the operator starts target aiming at the target search stage (S200, S210), the image tracking unit inputs the tracking image during the delay time or the buffering time when the communication and the delay caused by the system occur (S220 to S250), and the similarity degree of the image information during the delay time or the delay time due to the communication and the system is repeatedly performed Computes the tracking point variation (movement amount), which is the vector geometry analysis information.

When the delay time has elapsed, the image tracking unit may determine whether the user can track the target based on the calculated image information similarity and the tracking point variation amount (movement amount) (S270). At this time, the image tracking unit can determine whether the target can be tracked by using the degree of similarity of the image information and the amount of change (amount of movement) of the tracking point as an AND condition.

For example, as shown in FIG. 8, the image tracking unit can determine whether the target can be tracked by associating the image similarity degree information derived from each image and the tracking point variation amount (movement amount) with a previously learned data distribution (S300, S310). For example, it is determined (or determined) that target tracking is possible only when the image similarity value is larger than A and the tracking point variation amount is smaller than B in the data distribution, and the target tracking operation (S280).

As described above, according to the present invention, the tracking stability of the video tracker system, which is a platform operating system, is determined by using the image similarity information and the tracking point geometry analysis method even in the operator intervention in the platform operation / It is possible to prepare the target loss situation due to the tracking anxiety in the target aiming command and quantify and quantify the information of the image tracker necessary for platform operation / guidance control. Particularly, the present invention can improve the performance of the video tracker system by eliminating the unstable operator intervention command according to the quantified standard according to the above-described method, and can further improve the performance of the platform tracking / The effect of increasing the control performance can be expected.

The image tracking system and its tracking stability determination method based on the image similarity degree and tracking point geometry analysis according to the present invention described above can be applied to the configuration and method of the embodiments described above in a limited manner, It will be understood that the invention may be embodied in other specific forms without departing from the spirit or essential characteristics thereof. Therefore, it should be understood that the above-described embodiments are to be considered in all respects as illustrative and not restrictive.

Claims (10)

A video tracker system for controlling operation / induction of a platform,
An image sensor for detecting a tracking image; And
When an operator's target aiming is attempted in the operation / induction control operation state of the platform, the degree of similarity information and the change amount of the tracking point are derived from the tracking image detected by the image sensor to determine tracking stability with respect to the target aimed by the operator, And a video tracking unit for buffering the video signal,
The image-
(a) extracting image similarity information by comparing reference information extracted from a tracking image detected at a predetermined period with previously stored information;
(b) a tracking point indicating the position of the target is derived for each tracking image,
(c) forming a vector by connecting a tracking point derived from each tracking image with a tracking point derived from a previous image, accumulating a sum of vector differences between the formed vectors, calculating a tracking point variation,
(d) determining the tracking stability of the target by associating the extracted image similarity degree information and the calculated tracking point variation with a previously learned data distribution. 2. The method of claim 1,
Wherein the target tracker indicates whether the target can be tracked. The apparatus of claim 1, wherein the image tracking unit
Wherein the tracking stability is determined using a buffering delay image when a delay occurs in the system, and when the delay does not occur, the tracking stability is determined by buffering a predetermined image. delete 2. The method of claim 1,
The color of the image, the brightness difference, and the distribution of the histogram,
The image-
Wherein if the extracted image similarity degree information is equal to or greater than a predetermined value, it is determined that the possibility of tracking failure is high and the tracking operation is not performed if the cumulative calculated tracking point variation amount is equal to or greater than a predetermined value. A control method of a video tracker system for controlling operation / induction of a platform,
Acquiring a tracking image periodically through an image sensor;
Calculating similarity information from a tracking image obtained by the image sensor when an operator's target aiming is attempted in an operation / induction control operation state of the platform;
Calculating a change amount of a tracking point by deriving a tracking point indicating a position of a target for each tracking image acquired by the image sensor; And
And determining the tracking stability using the calculated similarity information of the tracking image and the variation amount of the tracking point,
The step of calculating the variation of the tracking point
Deriving a tracking point indicating the position of the target for each tracking image; And
And generating a vector by connecting a tracking point derived from each of the tracking images to a tracking point derived from a previous image, and then calculating a tracking point variation by accumulating a sum of vector differences between the vectors. A method for determining the tracking stability of a system. 7. The method of claim 6, wherein the step of determining the tracking stability comprises:
Wherein when the delay occurs in the system, the tracking stability is determined using the buffering delay image, and if the delay does not occur, the tracking stability is determined by buffering the predetermined image. . 7. The method of claim 6, wherein calculating the similarity information comprises:
The image similarity information is extracted by comparing the reference information extracted from the target image acquired at a predetermined cycle with the stored information,
Wherein the reference information includes a color, brightness difference, and histogram distribution of an image. delete 7. The method of claim 6, wherein the step of determining the tracking stability comprises:
Determining the tracking stability of the target by associating the calculated image similarity degree information and the calculated tracking point variation with a previously learned data distribution,
Wherein if the calculated image similarity degree information is equal to or greater than a predetermined value, it is determined that the possibility of tracking failure is high if the cumulative calculated tracking point variation amount is equal to or greater than a predetermined value.

Images