JPH04207361A - Image tracking device - Google Patents

Abstract

PURPOSE:To limit an image processing range by determining the ensuing image processing range in an image processing means according to an in-screen target prediction position. CONSTITUTION:The directing angle changing speed during turning of a directing device 2 is measured by a directing angle change speed measuring means 5 and the directing angle is determined by integrating 40 this directing angle change speed by a target position predicting means 6. This directing angle and a tracking error are added 42 to determine the target position. The target prediction position is subtracted 46 from the target position to determine a target prediction error. The target prediction position is determined from this target prediction error by a prediction filter 41 having a required prediction characteristic. Further, the directing angle is subtracted 47 from the target prediction position to determine the in-screen target prediction position and the image processing range at the time of the ensuing processing is determined the image processing means 3 according to this in-screen target prediction position. The image processing range in the ensuing frame is determined in this way.

Description

[Detailed description of the invention] (overview〕 Regarding image tracking devices.

The purpose is to provide an image tracking device that can limit the image processing range as much as possible by predicting the target position in the next processed image7, and is mounted on a pointing device that can be turned in any direction. image processing means for detecting a tracking error between the target on the screen from the image sensor and the visual axis of the image sensor, and drives the pointing device according to the tracking error to aim the visual axis. In a direct image tracking device that performs muko tracking, a pointing angle is determined by integrating a signal of a pointing angle change rate when the pointing device turns, and the target position is determined by adding the pointing angle and the tracking error; Subtracting the predicted target position from the target position to obtain a target f measurement error, and calculating the predicted target position from the target prediction error using a required prediction characteristic.1. The method is constructed by subtracting the orientation angle from the predicted target position to obtain the predicted target position within the screen, and determining the next image processing range in the image processing means according to the predicted target position within the screen.

(Industrial Application Field) The present invention relates to an image tracking device, and in particular, receives a signal from an image sensor placed in a pointing device, extracts a target object included in an image, and calculates its position on both surfaces. Calculate the driving designation rate for the directional flywheel IJ (in other words,
This invention relates to an image tracking device C that automatically tracks a target.

An image tracking device is a device that allows a pointing device to track a target by tracking the target object in an image from an image sensor.The image tracking device is a device that allows a pointing device to track a target by tracking the target object in the image from the image sensor. 5. Misuse of these techniques by "Toward" and Otsugo-1;5. In addition, image tracking devices with high precision and high stability are being used in control devices for each building and in the field of defense technology.

Such an image tracking device is capable of responding to advanced target motion and executing efficient image processing in the tracking control processing system that controls the entire tracking system. It has been demanded.

[Prior art] As a tracking control function in the tracking control processing of a conventional image tracking device, the tracking error calculation result by image processing is multiplied by 7 by a certain key, so that the tracking control function that is included in the tracking control processing of a conventional image tracking device is Its only function was to generate a drive rate command message for the device.

In the case of a target-back climbing scene where the target's motion is simple and the target can be extracted by mechanical image processing,
Such tracking control processing also makes it possible to achieve a tracking function of 1, -7 degrees.

[Problems to be solved by the invention] However, in recent years, high-precision tracking points and
-With Unku Takeno? ! With the use of an advanced image sensor and advanced cymbal system, it achieves the performance that is typical of LJ. As the image processing range becomes wider, it is difficult to achieve the desired extraction performance.
From then on, the 4L-U becomes a big challenge to the image visualization device.

Therefore, as a result of the increasing sophistication of image processing, a delay error often occurs between image acquisition 4W and tracking error calculation, which results in a further increase in the image processing range (C). ing.

The present invention is intended to solve the problems of the prior art as described above, and is designed to solve the problems of the prior art when there is a delay time element in the tracking loop, or when a gimbal etc. equipped with a pointing device becomes a large delay element. The object of the present invention is to provide an image tracking device that can rotate the image processing range as much as possible by predicting the target position in the next processed image, even when the image is being processed.

[Means to solve the problem]

FIG. 1 shows the basic configuration of the present invention.

The present invention includes an image processing means 3 for detecting a tracking error between a target on a screen from both image sensors 1 mounted on a pointing device 2'' which can be turned in any direction and a visual axis of this image sensor. In an image tracking device that drives the directing device 2 according to the tracking error to track the visual axis of the directing device to the target, the directing angle is determined by integrating the signal of the changing speed of the directing angle when the directing device 20 turns. The target position is obtained by adding this pointing angle and the tracking error, the target prediction error is obtained by subtracting the 1 target predicted position from the 20th tower position, and the target predicted position is calculated from this target prediction error using the required prediction characteristics. The orientation angle is subtracted from this predicted target position to obtain the predicted target position within the screen, and the image processing means 3 calculates the predicted target position within the screen.
This is to determine the next image processing range.

Further, the present invention uses the image processing means 3 to extract a target on the screen of the image sensor 1 mounted on the pointing device 2 which can be turned in any direction. Detects a tracking error with the visual axis of In the image tracking device configured as above, the pointing angle change speed measuring means 5 measures the pointing angle changing speed when the pointing device 2 turns, and the target position predicting means 6 integrates this pointing angle changing speed. to find the pointing angle, add this pointing angle and the tracking error to find the target position, subtract the predicted target position from this target position to find the target prediction error, and from this target prediction error have the required prediction characteristics. Prediction filter 41! Two-1. Find the predicted target position, subtract the pointing angle from this predicted target position, and then calculate [7] on the screen.
Find the predicted target position and enter the predicted target position within this screen.
7, a tracking case 1 is set in the image processing means 3 to determine the image processing range for the next processing.

Furthermore, the present invention has an image sensor 1 that is equipped with an image sensor g1 that images one target 7, is equipped with an image sensor g1 that can be turned in any direction, and is imaged by an image processing stage 3. The process of extracting a target on the screen of the sensor 1 and detecting the tracking error between the target and the visual axis of the image sensor is performed with a delay of two frames, and the closed-loop tracking control means 4 adjusts the image according to this tracking error. In an image tracking device that performs tracking control processing and generates a drive rate signal for the pointing device 2, the pointing angle change rate measuring means 5 measures the pointing angle change speed when the pointing device 20 turns, and determines the target position. In the prediction means 6, the integrator 40 integrates the rate of change in the pointing angle to obtain the pointing angle, and the adding means 42 calculates the pointing angle by 2.
The frame-delayed signal and the tracking error are added together to obtain the target position two frames before, and the first subtraction means 46 calculates the predicted target position two frames before from the target position two frames before. The target prediction error of two frames before is obtained by subtraction, and the prediction filter 41 calculates the target predicted position of one frame before using the desired prediction characteristic from the target prediction error of two frames before. The target predicted position 1 frame before 2 is delayed by 1 frame.
The second subtraction means 47 calculates the target predicted position before the frame.
By subtracting a signal whose directivity angle is delayed by one frame from this predicted target position one frame before, the predicted intra-screen target position one frame before is obtained.

Then, according to the predicted target position in the screen one frame before,
7, a tracking gate is set in the image processing means 3 to determine the image processing range in the next frame.

In this case, the present invention is configured such that the prediction filter 41 obtains the target predicted position from the target prediction error using a position-velocity model.

The present invention further provides that the prediction filter 41 performs prediction based on the prediction of the movement of the target on the screen and the prediction of the movement of the pointing device, and calculates the predicted target position from the target prediction error. .

[Operation] Figures 2 (a) and 2 (b) show the concept of the image tracking device, where (a) shows the imaging section and (b) shows the screen captured by the image sensor in the imaging section. It shows.

Image sensor 1] is a gimbal device 12 that constitutes a pointing device.
The gimbal device] 2 is mounted on the upstream vehicle, and by the operation of the gimbal device 2, it is possible to move the target 13 in the horizontal (AZ) direction and vertical (E) direction.
Directional angle, that is, L)
change the O3 angle, thereby placing target 1 within its field of view 14.
3 can be tracked.

In the screen 15 corresponding to the field of view 14, the visual axis 16 is fixed, for example, at the center thereof, and when the target 13 does not coincide with the visual axis 16, a tracking error 17 occurs between the two. A tracking gate 17 is set on the screen 15 as a certain range in which image processing for target extraction and tracking is to be performed. Note that this screen is 1. It does not necessarily have to be visually expressed.

The movement of a target within the screen of an image tracking device can be divided into two types: one based on a change in the positional relationship between the target and the image sensor, and the other based on the movement of a gimbal device equipped with an image sensor. Furthermore, factors that cause the gimbal device to move can be divided into those based on drive commands to the gimbal device and those due to spatial stabilization residual disturbance.

In a normal automatic tracking state, a deviation in the LO3 angle due to a change in the positional relationship between the target and the image sensor is corrected by a gimbal drive command in the tracking process. Generally, the positional relationship between the target and the image sensor does not change rapidly, so
It is easy to make the tracking system follow this.

On the other hand, although the deviation of the LO3 angle due to the spatial stabilization residual disturbance depends on the characteristics of the mounted moving object, it generally includes a high frequency component, and it is difficult to cover this with an image tracking system.
Technically difficult. In addition, in such cases, the higher the magnification of the optical system of the image sensor, the greater the deviation on the screen, so it is necessary to perform image processing to extract and track the target on the screen. ``We have no choice but to make the tracking gate that defines the area larger, which inevitably increases the image processing time.

In the present invention, the 1,08 angular change due to the spatial stabilization residual disturbance, which is difficult to predict, is calculated using the I=O8 angle change from the gimbal device.
Use the S rate measurements to separate them. This is possible because the tracking error signal that comes out after image processing has a delay time caused by the imaging system and processing system. , because it can be converted into an OS angle in inertial space.

FIG. 3 provides a detailed explanation of the invention.

In the figure, 21 is an image processor that acquires an image, extracts a target from the image data, and determines its position within the screen. Outputs error signal.

In the image processor 21, the processing range is generally limited to the inside of a tracking gate having a certain size in order to reduce the amount of processing. In the video acquisition function and image processing in the image processor 21t, a certain amount of time is required before outputting the tracking error signal, and this becomes a delay time element in the tracking loop.

22 performs image tracking control processing, and
A tracking control processor that drives one cymbal for two cymbals and outputs a -F signal η, which includes a closed-loop tracking control means consisting of a loop tracking filter 23, and a target position prediction processor. 241) and target position prediction means.

The closed-loop tracking filter 23 uses tracking errors to
A tracking filter with the required characteristics performs closed-loop calculation, calculates the tracking 1 = OS command rate, uses the gimbal device, and outputs a signal of -1-.
, are the main functions of conventional tracking control processing S.

The target position prediction processor 24 predicts the target position in the image frame to be image-processed next, and uses the tracking error signal from the image processor 21 and the gimbal device 1.
The signal of the gimbal LOS rate is input from the first stage of measuring the rate of change in orientation angle consisting of the LO5[,=-t measuring instrument 25 installed in A signal of a predicted target position within the screen is generated to predict the target position.

In the image processor 21, the tracking gate position is determined based on the signal of this intra-screen target predicted position.

That is, the image processor 21 acquires an image, extracts a target object on this image, and further calculates a tracking error between this target and the visual axis position. The tracking control processor 22 performs image tracking control processing and controls the gimbal device 12 on which the image sensor 11 is mounted. The LOS rate measuring device 25 built into the gimbal device 12 measures the gimbal LOS rate accompanying the movement of the gimbal device 12 and returns the measured value to the tracking control processor 22 .

At this time, in the tracking control processor 22, the closed-loop tracking filter 23 performs closed-loop control using a tracking filter having required characteristics to drive the gimbal device 12 so that the tracking error is minimized, and The position prediction processor 24 converts the gimbal ■ and OS rate signals from the I, OS rate measuring device 25 provided in the gimbal device 12 to a digital filter consisting of a delay element and a product-sum operator, and the gimbal LOS rate. The target position on the screen in the next processed image frame is predicted by performing arithmetic processing using an integrator that integrates the signal.

If the accuracy of this predicted target position within the screen is high, it is only necessary to use a small tracking gate in image processing and process only the vicinity of the target.2 Even in the case of complex images, target extraction can be performed with positive WI. , and the processing time is also shortened.

FIG. 4 shows the processing in the target position prediction processor, where 31 is a gimbal motel representing the operation of the gimbal device, 32 is a prediction filter for predicting the target position, and 32 is a prediction filter for predicting the target position;
33 is a 2-frame delay element (,Z-''), 34.35
is a one-frame delay element (2-'), 36 is an adder, 37
．． 38 is a subtracter.

In this embodiment, the delay based on the above-mentioned video acquisition and image processing in the image processor 21 is two samples (frames).

The image processor 21 uses image data from the image sensor.
The above-mentioned processing is performed in , and a tracking error signal is generated. On the other hand, in gimbal model 31,
Integrate the measured value of gimbal L-os rate (,2-1
+ MB IJ 7 1. Inertial space”-8
Generates a signal of 0 door', 0/hull [, O8 corner.

This transmission (J) is delayed by the delay element 33 to generate a signal of the gimbal visual axis two frames L.--frames earlier.

The adder 36 generates a signal of the target position two frames ago by adding the signal of the tracking error two frames ago and the signal of the gimbal visual axis two frames ago. Furthermore, in the subtracter 37, from this signal, 271,
- Generate a signal of the target prediction error two frames ago by subtracting the signal of the target predicted position two frames ago.

The prediction filter 32 calculates the target prediction error of two frames before (
Prediction is made from No. 8 according to predetermined prediction characteristics, and a signal of the previous target predicted position is generated for one frame. The delay element 35 delays this signal by one frame to generate the signal of the predicted target position two frames before.

2/Nharu Motiru 31(')・N:7・'Hull L
The OS angle signal passes through a delay element: 34 to 1 Freno signal.
Depending on the delay, the gimbal I of one frame before,
OS Generates an angle signal. The subtracter 38 generates a value at the predicted position of the eye pillow in the screen one frame before by subtracting this signal from the signal of the previous target prediction signal 1 for one frame J. Based on this signal, the tracking point that defines the image processing area is set as described above.

The predictive characteristics of the predictive filter 32 can be predicted with sufficient accuracy for normal purposes by the position-velocity model. In the example of FIG. 4, the first half of the transfer function indicates the prediction of the movement of the target on the screen7, t)1 is a coefficient for position prediction, and b2 is a coefficient for velocity prediction. The second half of the transfer function shows the Y measurement of the movement of the gimbal device. Therefore, by making a prediction based on the target prediction error using both of them, the predicted target position can be obtained. In addition, the initial l
-1OS +/-h is used to initialize the prediction filter at the start of calculation.

This method of determining the predicted position has excellent responsiveness and high accuracy, as can be seen from the simple configuration of the prediction filter (does not include delay time due to signal processing). Therefore, it is possible to reduce the size of the tracking game.

[Example] FIG. 5 shows an example of the present invention L7,
The same parts as in FIGS. 2 and 3 are indicated by the same numbers, and in the target position prediction processor 24, 40 is an integrator;
41 is a prediction filter, 42 to 45 are adders, 46.4
7 is a subtracter, 48 is a 2-frame delay element (2D), 49
52 is a one-frame delay element (D), and bl and b2 are coefficient multipliers.

In the embodiment shown in FIG. 5, input image data from the image sensor 11 is processed in the image processor 21 to generate a tracking error signal, and the signal is added to the closed-loop tracking filter 22 to generate a gimbal drive rate signal. The image sensor 1 mounted on the gimbal device 2 is generated and applied to the gimbal device 12 to drive it.
1 in the horizontal (AZ) direction and vertical (EL) direction.

Then, the movement of the gimbal device 12 at this time is 1,...
Measured using OS L-1 measuring instrument 25 and gimbal I.

Generates an OS rate signal. Gimbal LO3L.

-1・signal is usually A, Z and [悄-5 of 2
Consists of axis measurements.

In the target position prediction processor 24, the gimbal LO rate signal from the OS rail measuring device 25 is integrated by an integrator 40, and the gimbal LO signal, which is an angle signal, is obtained.
Convert to triangular signal. The integral in this case may be obtained by converting the input signal into a digital signal C1l:' and then adding the signals using an adder.

Gimbal 1 obtained in this way. The O8 angle signal is delayed by two frames by a delay element 48, and added to the tracking error signal by an adder 42 to generate a signal of the target position two frames before.The subtracter 46 generates a signal of this target position. The signal of the target predicted position two frames before is subtracted from the signal.

Then, a signal of the target prediction error of two frames before is generated.

Coefficient units bl, b2, adders 44.45, delay elements 49.
The circuit consisting of 50 realizes the prediction filter 32 shown in FIG. It is also possible to realize it using software.

By adding the signal of the target prediction error two frames before to this prediction filter, a signal of the target prediction position one frame before is generated. By adding this to the delay element 51, the signal of the target prediction error two frames before Generates a predicted position signal.

Furthermore, the signal of the aforementioned gimbal LO3 angle is transmitted to the delay element 52.
The subtracter 47 generates a signal of the predicted target position within the screen by subtracting it from the signal of the predicted target position of the previous frame. By specifying the position using this signal, the image processor 21
In , the tracking gate for the next frame is set.

In the present invention, fluctuations in the LO8 angle from the gimbal device due to spatial stabilization residual disturbance in the gimbal device equipped with the image sensor, which are included in the tracking error signal, are determined at the stage of subtracting the gimbal LO3 angle from the signal of the predicted target position. Since the signal is canceled, its influence is removed from the signal of the target predicted position within the screen.

(Effects of the Invention) As described above, according to the present invention, in an image tracking device, a target position prediction function within the screen is added to a tracking signal processing function, and the LO3 angle due to the spatial stabilization residual disturbance in the gimbal device is added. without being affected by fluctuations in
Since the target position can be predicted on the screen,
It is possible to reduce the size of the tracking gate that defines the image processing range, which makes image processing for target extraction easier and more accurate, improving processing speed, and preventing tracking from occurring because the target is not extracted. Malfunctions can be prevented.

[Brief explanation of the drawing]

Fig. 1 is a diagram showing the basic configuration of the present invention, Fig. 2(a),
(b) is a diagram showing the concept of the image tracking device, FIG. 3 is a diagram explaining the present invention in detail, FIG. 4 is a diagram showing the processing in the target position prediction processor, and FIG. 5 is an embodiment of the present invention. It is a figure which shows an example. 1 is an image sensor, 2 is a directing device, 3 is an image processing means, 4
5 is a closed loop tracking control means, 5 is a direction angle change rate measuring means, and 6 is a target position prediction means.

Claims (5)

[Claims] (1) Image processing means (3) for detecting the tracking error between the target on the screen from the image sensor (1) mounted on the pointing device (2) which can be turned in any direction and the visual axis of the image sensor. ), and the pointing device (2) according to the tracking error.
In an image tracking device that tracks the visual axis of the pointing device to a target by driving the pointing device (2), the pointing angle is determined by integrating the signal of the pointing angle change rate when the pointing device (2) turns, and the pointing angle and the tracking The target position is obtained by adding the error, the target predicted position is subtracted from the target position to obtain the target predicted error, the target predicted position is determined from the target predicted error by the required prediction characteristics, and the target position is determined from the target predicted position. An image tracking device characterized in that a predicted intra-screen target position is obtained by subtracting a corner, and an image processing range for the next processing in the image processing means (3) is determined according to the predicted intra-screen target position. (2) An image that extracts a target on the screen of an image sensor (1) mounted on a pointing device (2) that can be turned in any direction and detects the tracking error between the target and the visual axis of the image sensor. An image tracking device comprising a processing means (3) and a closed loop tracking control means (4) that performs image tracking control processing according to the tracking error and generates a drive command rate signal for the pointing device (2). , a pointing angle change rate measuring means (5) for measuring the rate of change in the pointing angle when the pointing device (2) turns; A prediction filter (41
), and a target position prediction means (6) for calculating a predicted target position in the screen by subtracting a pointing angle from the predicted target position, and a target position prediction means (6) for calculating the predicted target position in the screen according to the predicted target position in the screen. (3)
An image tracking device characterized in that a tracking gate is set for determining an image processing range in the next processing. (3) An image sensor (1) that images a target, and a pointing device (2) equipped with the image sensor (1) and capable of turning in any direction.
), an image processing means (3) for extracting a target on the screen of the image sensor (1) and detecting a tracking error between the target and the visual axis of the image sensor with a delay of two frames; An image tracking device comprising a closed-loop tracking control means (4) that performs image tracking control processing according to a tracking error and generates a drive rate signal for the pointing device (2), the turning of the pointing device (2) a pointing angle change rate measuring means (5) for measuring the changing rate of the pointing angle when the pointing angle changes;
0), an adding means (42) for calculating the target position two frames before by adding the signal obtained by delaying the pointing angle by two frames and the tracking error; a first subtraction means (46) for subtracting a target predicted position to obtain a target prediction error two frames before; and a prediction unit (46) for obtaining a target predicted position one frame before based on a required prediction characteristic from the target prediction error two frames before. Filter (41)
and a delay means (51) for obtaining the predicted target position two frames before by delaying the predicted target position one frame before by one frame, and delaying the pointing angle by one frame from the predicted target position one frame before. a second subtraction means (47
), and a tracking gate is provided to determine an image processing range for the next processing in the image processing means (3) according to the predicted target position in the screen one frame before. An image tracking device characterized by: (4) The image tracking device according to claim 2 or 3, wherein the prediction filter (41) calculates the predicted target position from the target prediction error using a position-velocity model. (5) A claim characterized in that the prediction filter (41) performs prediction based on the prediction of the movement of the target on the screen and the prediction of the movement of the pointing device, and calculates the predicted target position from the target prediction error. The image tracking device according to item 4.