JPH01184483A - Automatic tracking device for moving body - Google Patents

Abstract

PURPOSE:To improve tracking performance by detecting a state wherein automatic tracking is disabled and carrying on the automatic tracking again when it is enabled. CONSTITUTION:An azimuth arithmetic unit calculates the variation rate of the size of a moving and the shift quantity of the position on a screen from size and geometric gravity center data on the moving body which is inputted from an image processor 3 at each sampling time in parallel to normal tracking and then judges that the tracking is impossible when the variation rate and shift quantity exceed specific tracking range set values, thereby outputting an predicted value of a target azimuth based on a target azimuth and angular speed data which are one sampling time before to a driving table controller 7. At this same time, the size and position of the moving body are compared with the specific recovery range set values each time the sampling time is elapsed from said point and the normal moving body tracking is performed when said values enters the set recovery ranges.

Description

DETAILED DESCRIPTION OF THE INVENTION [Industrial Field of Application] The present invention relates to an automatic tracking device for tracking a moving object.

[Prior Art 1] Conventionally, it has been practiced to automatically track a moving object such as an aircraft that moves at high speed, detect the azimuth angle and the moving angle speed, and use the information as illumination information for various devices. It is.

[Problems to be Solved by the Invention] Under these circumstances, there is also a system that automatically tracks a moving object, which is a public object, by generating data from a video signal including the object obtained by an imaging device. It is used.

However, conventional equipment processes each frame, but uses fixed binarization as an image processing method to separate the background and moving objects, so it can adapt to changes in environmental conditions. It was difficult to do so, and it could not be used outdoors.

In addition, although there are models equipped with algorithms that adaptively respond to changes in environmental conditions and separate the background and moving objects, there are , about 60,000 pieces of data) is processed by software, which takes several seconds to several tens of seconds per screen, making it unsuitable for measuring moving objects.

Furthermore, since the imaging device in conventional devices is fixed,
If the moving object is out of the field of view of the imaging device, it cannot be measured, and it has been impossible to track the moving object that is moving at high speed.

In addition, with conventional processing equipment, if a moving object passes behind an obstacle such as a chimney, it may become impossible to identify the moving object in front or behind it, and the tracking function may stop or malfunction. equipment had to be adjusted and restarted.

Therefore, the present invention uses an algorithm that can adapt to environmental changes and reduce the number of data processed by software to several hundred per screen by converting a part of the processing into hardware. By improving the disadvantages of conventional devices such as The purpose is to

Furthermore, even if the visibility from the imaging device to the moving object is obstructed by an obstacle and tracking becomes impossible, the present invention determines the situation before and after that point, and then waits for 1 hour from that point until tracking can be restored. The purpose of the present invention is to provide a self-help tracking system for a moving object that can move an imaging device to a predicted orientation based on the target orientation and angular velocity data before the frame and perform continuous tracking operation without restart intervention by an operator. do.

[Means for Solving the Problem] An automatic tracking device for a moving object according to the present invention includes an m-image device that images a moving object, a drive stand that supports the image-capturing device freely in the imaging direction, and Input the video signal of the current frame, compare it with the model image of the previous frame, and extract only the moving object from the brightness difference.
FII The geometric center of gravity data indicating the angular difference between the direction of the geometric center of gravity of one object and the direction of the tea screen and the size of the moving object on both sides are extracted, and the image of only the background in the current frame is calculated in the next frame. Image stored as a model image −
a processing device; an azimuth calculation device that inputs geometric center of gravity data output from the image processing device and azimuth data indicating an imaging direction input separately and outputs a target azimuth calculation value of a moving object; A control signal for changing the imaging direction so that the moving object is located at the center of the screen of the imaging device is outputted to the driving stage by manually calculating the position calculation value, and the azimuth data of the imaging device is transmitted to the position calculation device. In the tracking device equipped with a drive platform control device that outputs data to Calculates the rate of change in size and amount of change in position of the moving object within the screen, and determines that tracking is impossible when the rate of change and amount of change exceed a predetermined tracking range setting value, and the drive platform control device outputs a predicted value of the target orientation based on the target orientation and angular velocity data one sampling time before, and compares the size and position of the moving object with a predetermined recovery range setting value every sampling time from that point onwards. It has a function of determining that recovery is possible only when the recovery range is within the set value, and performing normal tracking of the moving object.

[Example] Hereinafter, the present invention will be explained using the illustrated example. 1st
The figure shows a block diagram of an apparatus configuration of an embodiment of an automatic tracking device for a moving object according to the present invention.

In this embodiment, a TV camera 1 is used as an imaging device, and is supported by a camera drive stand 9 so that the image carrying direction can be adjusted.

A video signal 2 obtained from the TV camera 1 is input to an image processing device 3 provided at the next stage, and geometric center of gravity data 9 of a moving object in the TV video is calculated.

Furthermore, the TV camera 1 has a built-in zoom mechanism that automatically zooms the moving object image to the extent necessary for digital image processing within the image processing device 3, and outputs the angle of view signal θ at each sampling time. Output. Image processing 11! By using this view angle signal θ, the device 3 can calculate the geometric center of gravity data 4 by converting the geometric center of gravity position of the moving object indicated by the number of pixels into an angle.

For normal TV image signals (R8170 standard): 2:
Since it uses the 1-interlace method, 1 in 1/30 seconds
Data for a screen can be obtained, but the vertical resolution is 25
If the number of pixels is 0 or less, data can be obtained in 1/60 second.

Here, the image processing device 3 will be explained based on the detailed configuration block diagram shown in FIG. First, the video signal 2 captured by the camera 1 is the current frame (n frame).
is stored in the input image storage device 11 as input image data 21. Further, the model image storage device 17 stores in advance a model image 18 of only the background excluding the moving object one frame before (n-1 frame).

The model image data 2o and input image data 21 constituting this model image 18 are input to the image constructor 13, and are as shown in FIG. Perform the picture lI heavy Ichi.

Here, in processing the model image 18 and input image data 21 shown in FIG. Input image data 2.1 (indicating the amount of displacement in the image transport direction between frames) is used to generate and output a constituent image 19 in the current frame (n frame). This constituent image 19 will be described in detail later in the section on effects, but the rate of change in the size of the moving object between frames (1/30 sec) is IQ, 98≦≦1.02J,
In addition, the amount of displacement ΔX in position is defined as "ΔX≦0.1 pixel", so if you set the moving object to the center of the image in the initial settings, it will be 1 pixel.
It does not deviate from the center of the image as the frame progresses over time. Therefore, if the initially stored model image 18 consists only of the background, images are captured in the image & composition unit 13 every frame. A constituent image 1119 is obtained in which only the background portion of the input image data 21 adjusted to the direction increases little by little (the portion 21 is emphasized in FIG. 3).

When generating this constituent image @19, if the initial motion or the tracking motion is started in an area where no model image is stored, the brightness pixels in a predetermined range roughly corresponding to the brightness of the background at that time. can be manually input into the model image storage device 17 as dummy data of the model image data.

The image comparator 12 inputs the aforementioned constituent image 19 and the input image data 21 from the input image storage device 11, and generates a brightness difference signal 19' by comparing the brightness of these two signals for each pixel. . This brightness difference signal 19' is composed of the brightness difference for each pixel between the input image data 21 including the moving object and the constituent image of only the background in the same carrying range, so there is almost a difference in brightness between the background parts. In contrast, the brightness difference is large in areas where there is a moving object.

This luminance difference signal 19' is output to the model image constructor 14 and the binary image constructor 15. Among these, in the binary image constructor 15, based on the input luminance It difference signal 19', pixels whose absolute value of the luminance difference is greater than or equal to a certain threshold value are treated as moving objects, and those less than that are treated as a binary background. The binarized data is output to the centroid calculator 16. here,
The center of gravity calculator 16 outputs the geometric center of gravity 4 of the moving object based on the binarized image.

On the other hand, in the model image constructor 14, the binary signal 1
9', the model image data 20 and input image data 21 are also input, and a model image for the next frame (n+1 frame) is generated based on the following equation.

················two·······(
1, but Pmod (x, y, n): frame n
Luminance Pin (x, V, n) at the position (X, V) of the model image data 20 at frame n: Luminance at the position (X, l/) of the input image data 21 at frame n.

Also, the function ``uses, for example, the following impersonation function.

-(1-j)X1+X2 r(xl, x2)-□・・・・・・・・・・・・・・・
...(Threshold level used here,
and the integer m are determined experimentally.

That is, in this model image generator 14, the model image model 20 is used to cancel the moving object in the portion corresponding to the pixel of the moving object where the absolute value of the brightness difference shown in the condition (2) is the threshold level. The corresponding pixel Pled(x, y, n) is adopted.

In addition, under the condition of absolute value of brightness difference and threshold level shown in Weighting processing is performed on both data Pin (X, v, n) to create data that will become background pixels. In this way, data of only the background without moving objects is created, outputted as model image data 18 for processing in the next frame (n+1 frame), and stored in the model image storage device! ! 17.

That is, in the image processing device 3 described above, 1
Built-in model image data of only the background in front of the frame,
Compare the brightness differences based on that and model images for each frame! By updating, moving object pixels can be accurately extracted in response to changes in screen conditions (changes in brightness, etc.).

In this way, the geometric center of gravity data 4 calculated by the image processing device 3 is output to the azimuth calculation device N5 provided at the next stage.

This azimuth calculation device 5 generates a driving platform position signal 1 containing geometric center of gravity data 4 and azimuth data indicating the AM azimuth of the TV camera.
A target azimuth calculated q value, which is the azimuth of the target object, is calculated from 0 and output to the drive platform control device 7. Drive stand & lJ til
The l device 27 controls the camera drive stand 9 that supports the TV camera 1, and uses input signals from the azimuth calculation device to control the camera drive stand 9 so that the geometric center of gravity 4 is located at the center of the screen. Drive stand 1111111 outputs signal 8. As a result, the azimuth angle of the TV camera 1 is controlled so that the image of the moving object is captured at the center of the screen.

Further, the driving platform and control device 7 outputs azimuth data indicating the current Ii image position of the TV camera 1 to the azimuth calculation device 5. Therefore, the direction calculation device 5 is connected to the TV camera 1.
Measurement data 6 such as the target azimuth of the moving object and the calculated sieve value are output from the azimuth data and the geometric center of gravity data 4 on the video screen.

Here, in order to prevent the moving object from being unable to be tracked or malfunctioning if it enters behind a chimney or the like, the direction calculation device 5
I! has a function that detects when automatic tracking is not possible during the test to prevent malfunction of the device, and a function that determines the possibility of automatic tracking recovery and restores normal tracking. lI@ is provided.

Among these, detecting that automatic tracking is no longer possible is done by applying the permissible position meter change ΔX to the permissible size change rate in the screen of the moving object preset in the azimuth 2iI calculating device 5.
This is done by comparing with the ll value.

That is, when a moving object is automatically tracked normally, changes in the size and position of the moving object within the imaging screen are calculated based on its moving speed, maximum motion acceleration, and mechanical zooming speed of the imaging device 1. 1i above that value!
Since ffi is limited to &11, the upper limit value is set in advance as an allowable value, and if the actual value exceeds the allowable value, it is determined that self-help tracking is impossible.

For example, the thickness Wl of an object in the screen at a distance of 11
, the width of the object in the screen at a distance of 12 Wi, the average moving speed of the object between the two points ■, the sampling time of both sides of the dancing image as t8, the size of the object between two consecutive screens (one sampling time) 1 is the rate of change of .

If the minimum zooming time mechanically required to reach the field angle θ1 to 62 of the ms device is t7, then the rate of change in the size of the object due to the influence of zooming between two consecutive planes is 2.

From this, the rate of change in the size of the object within one sampling time within the image capture screen is KK-2.

The moving speed of the object ■1, the maximum acceleration α. , X, the position change end Δ in one sampling time within one image screen is
X becomes Δx−vt + 1/2α t2 S IaX S.

Hereinafter, numerical examples will be given for the change rate -K -2 and the change amount ΔX.

(1) The rate of change in the size of the target object between two screens due to the change in distance tltl is 1. A target object of 20m5pan is 20onL from 2000m.
Assume that the object approaches the target object in 5.3 seconds (mach1), and as a result, the angle of view of the target object changes from 0.573° to 5.72°.

1 frame in 5.3 seconds is 5.3/1/30 pieces, 1
Change in angle of view at frame gate 4! If (multiple) is set to 1, +21 The rate of change in the size of the target object between two screens due to zooming is 2.The minimum focal length is 15g+, the angle of view is 32.7°, and the maximum focal length is 1! t180#I, the minimum zooming time to reach a maximum of 11.5 times up to an angle of view of 2.8 degrees is 8 seconds, then 1.02 times. On the other hand, when the moving object moves away, the rate of change in size is is expressed as the reciprocal of K, 1/=0.98.

In this way, the rate of change in the size of the target object per frame is defined as 0.98≦ and ≦1.02.

(4) Amount of change ΔX in the position of the moving object within the screen Define the amount of change ΔX in the position as the deviation of the geometric center of gravity of the target object from the center of the screen, and treat this as approximately equal to the motion vector. If the maximum motion acceleration of a moving object is 6g, the speed change in 1/30sec is 6X9.8m/S2x 1/3G-1,96m/52
Assuming that a moving object of 0m5pan is captured by about 30 pixels on the screen, the deviation per 1/3O3eC is 1.96m/
sx 1/30sec x30 image search 7?0rrt=0.
The displacement ΔX of the position between one pixel, that is, one frame, is ΔX≦
It is defined as 0.1 pixel.

As mentioned above, the upper limit of the size change rate K and the position change rate ΔX, which are defined by the maximum predicted characteristics of the moving object to be tracked and the basic performance of the device, is 18% due to noise in the image processing system, etc.
The tolerance range obtained by adding the upper limit value of 11 errors is set in advance in the azimuth calculation device 5.

K and ΔX are calculated from the actual measured values of the size of the moving object and the geometric center of gravity data inputted from the azimuth PIJ calculation device and the image processing device 3.

In this way, if the rate of change K in the size of the object actually measured using the digital image processing method is outside the above-mentioned tolerance range, or if the amount of change ΔX in the actually measured position is outside the tolerance range. If so, it is determined that automatic tracking is no longer possible.

In a state where automatic tracking is no longer possible, the direction calculation device 5
Using the target azimuth calculation value one scan ago and the angular velocity determined by Meng's time rate of change, the target azimuth calculation value is output to the drive platform control device 7 assuming that the angular velocity is sustained, and the camera drive platform 9 is controlled by the vIII system. , this opening does not output the measurement data 6 to the outside.

Further, the azimuth calculation device 5 determines whether automatic tracking is possible to recover at every scan time of the movie signal 2 from the stage when automatic tracking is no longer possible. For this determination, a permissible range of the size and position of the object defined by the elapsed time from the size and position of the moving object one sampling time before automatic tracking becomes impossible is set for each elapsed sampling time, and the image is captured. If the actual value measured by digital image processing within the screen falls within this range, it is determined that automatic tracking has been restored.

(9 Change in size of moving object Elapsed time t1 from tracking abnormality Error of image processing system ±ε8, Current moving object size obtained from image processing i! position 3
6. If the size of the moving object just before a tracking error occurs is 18, then 1c is 30t 30t +.

1 k −ε ≦1 ≦'e kL se
When the size of the moving object falls within the range of s s c, the size of the moving object is determined to be normal.

(6) Change in position of moving object: change amount d (d=0.1 pixel) of the deviation of the geometric center of gravity from the screen center for each frame, error ±ε of the image processing system.

, Elapsed time t1 from the time of tracking abnormality determination Geometric barycenter coordinates immediately before tracking/tailing abnormality x2, y4, Current geometric barycenter coordinates Xc obtained from image processing Vl position
In the case of ``c'', lx −x1+<ci−t+ε.

When the conditions of ly −y, l<dφ and 1εp are satisfied, the change in the position of the moving object is determined to be normal.

The size and location of the moving object can be determined by determining the conditions in (5) and (6) above! ! ! When the change is normal 1, it is determined that normal tracking has returned.

When operating the device configured as described above, a tracking failure setting value and a recovery setting value are set according to the characteristics of the moving object, and an area (target area) where the moving object 16 should exist in the video screen 21 is set. An area representing the background (background area) is set in the image processing device 3. Next, while viewing the monitor screen, the camera drive base 2 is manually operated to adjust the moving object 16 to be captured within the target area.

'When the apparatus is operated at this stage, the image processing apparatus 3 obtains the geometric center of gravity data 4'' of the target object, and the drive platform control apparatus 7 outputs the current azimuth angle data of the TV camera 1.

From these data, the azimuth calculation device 5 outputs measurement data 6 such as a target azimuth calculation value of a moving object during normal tracking operation, and further performs an operation to determine whether tracking is impossible or whether the object has returned. This azimuth calculating device 5 operates according to the flowchart shown in FIG. That is, during normal tracking operation, geometric center of gravity data, object size and azimuth data are read (Sl), and from this the rate of change K in the object size and the amount of change ΔX in position are calculated (S2). ). Here, if the set value 0.98≦S1.02 and ΔX≦0.1, which are the criteria for determining whether tracking is impossible, the pixels are read out, and the actual value calculated in the previous step S2 is the set value 1 [ A comparison is made to see if it is within 111 (S3). In this step S3, if the actual measurement value is within the range of the set value, it is assumed that the tracking range is normal and the process proceeds to the next step to calculate the target azimuth calculation value and angular velocity.S4
), outputs the measurement data 6 of the moving object, and outputs the calculated target orientation value to the drive platform control device rj7 (S5), and moves on to the processing of the next frame.

Further, in the determination block of step S3, if the measured values K and ΔX exceed the set values, it is determined that there is a tracking abnormality, and the process jumps to step S9 of the parallel flow. In this step S9, a predicted value of the moving object direction is determined using the target azimuth calculation value immediately before it is determined to be abnormal and the angular velocity obtained from its time change rate, and this predicted value is used as the target azimuth calculation value to control the drive platform v. Output to device 7.

When this step S18 is completed, from the data obtained from the next frame, steps S and S in the left loop
Proceed to steps S6 and S7, which are processes with the same content as
Load the geometric center of gravity data, object size, and azimuth data and calculate the rate of change in object size and change in position by ΔX
Calculate.

Next, in step S8, the tracking recovery setting value is 30t.
30t+ε8, i k −ε S1
≦'e kL es s cl xC-x
il <d −t+εp, and ly −y, l<d−
t+ε is read out and compared with the value calculated in step S187. In this comparison in step S8,
If the measured value is within the range of the time-corrected setting Iff, tracking is recovered and the process returns to step S4, and if it is outside the range of this set value, the routine from step S9 is repeated again.

The target azimuth calculation value from the azimuth calculation device 5 is output to the drive platform control device 7, and the drive platform control device 7 receives the input
73ffi, the camera drive stand 9 is moved so that the geometric center of gravity of the moving object is located at the center of the video screen of the camera 1.
control. As a result, moving objects can always be detected by TV camera 1.
It is possible to measure the position of a moving object while capturing it within the field of view.

In this way, even under dynamic shooting conditions such as outdoors where environmental conditions etc. change, the target object and the blue mark are separated and the process of determining their geometric center of gravity is performed for each frame on the TV iiM surface, and the target object is always By controlling the camera drive base to position the object in the center of the screen, it is possible to measure the position of a moving object over a wide area that cannot be captured by a stationary camera, and even if tracking becomes impossible, the situation can be determined. and automatically return to normal tracking.

Furthermore, by using an infrared imaging device or the like instead of using a TV camera, the position of a moving object can be measured even at night.

Effects of the Invention The embodiment of the automatic tracking device for a moving object according to the present invention is as described above, and can bring about the following effects.

It is possible to generate position data by tracking a moving object that moves at high speed over a wide range of areas with changing environmental conditions, and also detects the situation when automatic tracking of a moving object is no longer possible due to an obstacle such as a chimney. At the same time, when recovery becomes possible, tracking performance can be improved by continuing automatic tracking again, and there is no need for manual reset.

This is an automatic moving object tracking device that can generate position data by tracking a moving object that moves at high speed over a wide range of areas.

In addition, moving objects are detected based on model image data of only the background from one frame before, and this model image data is updated every frame, so changes in the situation such as changes in screen brightness are detected. It is also possible to accurately track moving objects by adapting accordingly.

[Brief explanation of the drawing]

Figure 1 is a block diagram of the configuration of the embodiment device, Figure 2 is the 1v
! FIG. 3 is a conceptual diagram showing an example of the configuration of a constituent image, and FIG. 11i4 is a flowchart showing the operation within the azimuth calculation device. 1.-TV camera, 2.. Video signal, 3.. Image processing device ff, 4.. Geometric center of gravity, 5.. Orientation calculation v
t position, 6... Measurement data, 7... Drive platform control device,
8...Drive base operation signal, 9...Camera JlI! Moving base, 10... Drive base position signal, 11... Input image storage device, 12... Image comparator, 13... Image constructor,
14... Model image constructor, 15... Binary image generator, 16... Centroid operator, 17... Model image memory VtH118... Model tjA image, 19... Configuration 1
L20--model drawing data, 21--input image data.

Claims (1)

[Claims] 1. An imaging device that images a moving object and outputs a video signal, and a drive that supports this imaging device in the imaging direction and controls the imaging direction of the imaging device based on a control signal input from the outside. In an automatic tracking device for a moving object, which includes a stand and a control device that automatically tracks the moving object by processing an imaging signal from the imaging device and outputting the control signal to a drive stand, Creating a constituent image of only the background of the current frame from a model image of only the background of one frame before (n-1) generated by processing and a video signal of the current frame (n) input from the imaging device; Furthermore, the moving object is separated from the background based on the luminance difference between the video signal of the current frame and the constituent images, and geometric center of gravity data indicating the angular difference between the geometric center of gravity direction of the moving object and the imaging direction is generated. an image processing device that calculates the size of the background image in the current frame and stores the background-only image in the current frame as a model image in the next frame; and an azimuth that indicates the geometric center of gravity data and image direction output from this image processing device. In parallel with normal tracking that inputs the angle data and outputs the target azimuth calculation value of the moving object, a screen of the moving object is generated from the size and geometric center of gravity data of the moving object that are input at each sampling time from the image processing device. calculates the rate of change in size and the amount of change in position within, and when the rate of change and amount of change exceed a predetermined tracking range setting value, it is determined that tracking is impossible, and the drive platform control device is unable to track it. A predicted value of the target direction based on the constant angular velocity prediction is output using the immediately preceding target direction calculation value and angular velocity, and the size and position of the moving object are adjusted to a predetermined recovery range setting value at each sampling time from that point onwards. An azimuth calculation device that compares and performs normal tracking of the moving object by determining that recovery is possible only when the recovery range setting value is within the set value; a drive stand control device that outputs a control signal to the drive stand to control the imaging direction so that the moving object is located at the center of the screen of the image pickup device, and outputs azimuth data of the image pickup device to the azimuth calculation device. An automatic tracking device for a moving object, characterized in that the tracking device includes: