JPH05346958A - Mobile object tracking device - Google Patents

Abstract

PURPOSE:To perform mobile object tracking operation wherein a positional deviation between images in successive frames can be corrected even when the positional deviation is large. CONSTITUTION:A 1st image memory 2 is stored with an (n)th image of an image input means 1 and a 2nd image memory 3 is stored with an (n+1)th image; and its output start timing is controlled to roughly correct the positional deviation. A positioning means 4 detects positional deviation vectors of the images in the 1st and 2nd image memories 2 and 3 and the output start timing of the 2nd image memory 3 is controlled to completely correct the positional deviation between both the images. A mobile object detecting means 5 detects a mobile object from the difference between the positioned images in the 1st and 2nd image memories 2 and 3. A position calculating means 6 calculates the position of the mobile object based upon the center of a screen as an original and a rack control means 8 controls the visual field direction of the image input means 1 according to the output of the position detecting means 6.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a moving body tracking device for detecting and tracking a moving object image such as an intruding object from an image monitoring device imaged and input by an image input device such as a television camera.

[0002]

2. Description of the Related Art Conventionally, as a device for observing a predetermined place, a TV camera or the like has been installed, and an image taken by the device is output to a monitor in another room so that an observer can constantly monitor the monitor. there were.

In recent years, with the development of image processing technology, an apparatus has been developed which detects a moving body from a picked-up image and issues an alarm to detect an abnormality on behalf of a surveillance staff member. In addition, the development of a monitoring device that is equipped with a camera mount and automatically tracks the detected moving body has been attempted. As a tracking device of a moving body by such image processing, for example, iE / E / E transaction on pattern analysis and machine intelligence, PMI 4, No. 1 (198).
2), pp. 2-10 (IEEE.Transaction on Pattern An
alysis and Machine Intelligence, vol.PAMI-4, No4 (198
2), pp.2-10).

A conventional moving object tracking device using image processing is
For example, the configuration is as shown in FIG. In FIG. 5, 1 is an image input means. A first image memory 2 stores the n-th image output from the image input means 1. A second image memory 3 stores the (n + 1) th image output from the image input means 1. Reference numeral 4 is a positioning means for positioning the images of the first image memory 2 and the second image memory 3. Reference numeral 5 denotes a moving body detecting means, which detects the moving body based on the image difference between the nth image and the (n + 1) th image aligned by the alignment means 4. Reference numeral 6 denotes a position calculating means, which calculates the center of gravity of the image of the moving body obtained by the moving body detecting means 5 and calculates the target position. A mount 7 freely drives the visual field direction of the image input means 1. Numeral 8 is a gantry control means and gantry 7
To control.

The operation of the moving body tracking device configured as described above will be described below.

First, it is assumed that the n-th image input by the image input means 1 is already stored in the first image memory 2. At this time, the (n + 1) th image (image after one frame) input by the image input unit 1 is stored in the second image memory 3. As a result, consecutive frame images are stored in the first image memory 2 and the second image memory 3. Here, the alignment means 4 is used as a pre-process for detecting a moving object.
Align with. This is because the image input means 1 is driven by the gantry 6 between consecutive frames to correct the positional deviation of the images generated between the frames. The position alignment unit 4 compares the first image memory 2 and the second image memory 3 to calculate a position shift vector, and corrects the position shift of the image in the second image memory 3. As a method of calculating the position shift vector, there is basically a method of gradually shifting the images between consecutive frames and obtaining a position where the image difference is minimized. Ie n
If the image at the th is g _n (x, y) and the image at the n + 1 th is g _{n + 1} (x, y), the image g _n (x, y)
y) is shifted by ξ in the x direction and η in the y direction to obtain ξ and η that minimize Equation 1. This makes it possible to detect the position shift vector of the image between consecutive frames.

ΣΣ | g _n (x−ξ, y−η) −g _{n + 1} (x, y) | ... (1) In this way, the position shift is corrected by the n-th image and the alignment means 4. The n + 1-th image is compared by the moving body detection means 5, and if the difference between both images is obtained, only the moving body can be detected from the images. Position calculation means 6
Calculates the center of gravity of the detected moving body, calculates the deviation from the center position of the image to calculate the target position, and outputs the target position to the gantry control means 8, whereby the gantry 7 is controlled and the visual field image of the image input means 1 is displayed. A moving object can be captured in the center. By repeating the above operation, the moving body can be tracked.

[0008]

However, in the conventional apparatus as described above, when the gantry is greatly panned (or tilted), the positional deviation in the images between the stored consecutive frames becomes large, and therefore, the alignment of There is a problem that the range is exceeded and it becomes difficult to detect a moving body.

SUMMARY OF THE INVENTION The present invention solves the above-mentioned problems of the conventional apparatus, and provides a moving object tracking device capable of correcting the positional deviation without exceeding the alignment range even when the positional deviation of images between consecutive frames is large. The purpose is to provide.

[0010]

In order to achieve the above object, the present invention stores a first image memory for storing n-th image data and an n + 1-th image data, and has a variable output start timing. Two image memories, an alignment unit that aligns the images by comparing the output images of the first image memory and the output images of the second image memory, and the n-th image aligned by the alignment unit. Moving body detecting means for comparing an image with the n + 1th image to detect a moving body in the image; and position calculating means for calculating a center of gravity of the moving body detected by the moving body detecting means and calculating a target position, Means for controlling the output start timing of the second image memory by the output of the position calculation means.

The other is that, instead of the output of the position calculating means, an encoder for detecting the position of the gantry and a coordinate converting means for converting the output of the encoder into an image coordinate system are provided. Means for controlling the output start timing of the second image memory by the output of the above.

[0012]

The present invention has the following actions due to the above-mentioned configuration.

That is, the moving body is detected in the image data shown in FIG. 3A, and the positions x and y of the moving body with the center of the screen as the origin are obtained. Next, the image shown in (B) is obtained by panning (or tilting) the input image based on the obtained positions x and y. By correcting the position of this image based on the previous positions x and y, the image shown in (C) is obtained. Here, since the positional deviations Δx and Δy due to a mechanical error at the time of panning (tilting) occur between the images of (A) and (C), the alignment is performed. Thus, the moving body may be detected by comparing the images of (A) and (C).

On the other hand, instead of the above-mentioned positions x and y, the positions x'and y'obtained by converting the output of the encoder mounted on the gantry into the coordinate system of the image are used to obtain the image of (B). Even if the position is corrected, the image of (C) can be similarly obtained.

Therefore, even if the input image is greatly panned (tilted) and the images of (A) and (B) exceed the alignment range, the images of (A) and (C) may be aligned. Therefore, the positional deviation Δx, Δy due to a mechanical error or a very slight movement of the moving body between consecutive frames can be performed without exceeding the alignment range.

[0016]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS A first embodiment of the present invention will be described below with reference to FIG. In FIG. 1, 1 is an image input means. A first image memory 2 stores the n-th image data output from the image input means 1. A second image memory 3 stores the (n + 1) th image data output from the image input unit 1, and can change the output start timing. Four
Is a position alignment means for detecting a displacement vector of the image position between the first image memory 2 and the second image memory 3. Reference numeral 5 denotes a moving body detecting means, which detects the moving body based on the image difference between the first image memory 2 and the second image memory 3 which are aligned by the aligning means 4. Reference numeral 6 denotes a position calculating means for calculating the center of gravity of the image of the moving body obtained by the moving body detecting means 5 and calculating the position of the moving body with the center of the screen as the origin. A mount 7 freely drives the visual field direction of the image input means 1. Reference numeral 8 is a gantry control means for controlling the gantry 7. A register 9 stores the output of the position calculating means 6. Reference numeral 10 denotes an addition means, which adds the output of the position alignment means 4 and the output of the register 9 to obtain the second image memory 3
Controls the output start timing of. 11 is a control means,
First image memory 2, second image memory 3, alignment means 4, moving body detection means 5, position calculation means 6, register 9,
It is a means for controlling each processing procedure of the adding means 10.

The operation of the moving body tracking device configured as described above will be described below.

FIG. 4 shows the processing procedure. First,
It is assumed that the n-th processing has already been completed and the image data of the n-th frame (for example, the image of FIG. 3A) input by the image input unit 1 has been stored in the first image memory 2. Therefore, the (n + 1) th process is started.

Now, the image data in the (n + 1) th frame input by the image input means 1 in accordance with a command from the control means 11 is stored in the second image memory 3. At this time, the field-of-view image is panned (tilted) by the n-th process, and thus becomes the image of FIG. Next, the control means 11 clears the output of the alignment means (deviation amount x, y
= 0, 0), the content of the register 9 is output as it is as the output of the adding means 10. Here, the register 9 stores the contents of the position calculating means 6 at the nth position, that is, the detected positions x (n) and y (n) of the moving body in the coordinate system with the screen center as the origin at the nth position. Therefore, the addition means 10 delays (or advances) the output start timing of the second image memory 3 by x (n) in the x direction and y (n) in the y direction, so that the second image memory 3
Is an image whose position is corrected by x (n) and y (n) (see FIG. 3).
(Image of (C)) is output. At this time, the control means 11 sends the output of the first image memory 2 and the output of the second image memory 3 to the alignment means 4. Then, the alignment means 4 detects the position shift vector of the images of FIGS. 3A and 3C.

As a method of calculating the position shift vector, for example, the method quoted in the conventional example is used. That is, when the nth image is g _n (x, y) and the n + 1th image is g _{n +1} (x, y), the image g _n (x, y) is x.
Ξ and η are calculated so as to minimize Equation 1 by shifting ξ in the direction and η in the y direction.

As a result, the positional deviation vectors Δx (n + 1) and Δy (n + 1) between the images in FIGS. 3A and 3C can be detected. Then, the control means 11 outputs the position adjustment means 4, that is, the position shift vector Δx.
By outputting (n + 1), Δy (n + 1) to the adding means 10, the adding means 10 causes the content of the register 9 = x.
(N), y (n) and the displacement vector Δx (n + 1),
By adding Δy (n + 1) and delaying (advancing) the output start timing of the second image memory 3 by x (n) + Δx (n + 1) in the x direction and y (n) + Δy (n + 1) in the y direction, The second image memory 3 is x (n) + Δx
An image whose position is corrected by (n + 1), y (n) + Δy (n + 1) is output. At this time, the control means 11 sends the output of the first image memory 2 and the output of the second image memory 3 to the moving body detection means 5.

In this way, the nth image and the (n + 1) th image whose positional deviation has been corrected by the position aligning means 4 are compared by the moving body detecting means 5, and if the difference between the two images is obtained, only the moving body is selected from the images. Can be detected.

The position calculation means 6 obtains the detected center of gravity of the moving body, calculates the position x (n + 1), y (n + 1) of the moving body in the coordinate system with the image center position as the origin, and stores it in the register 9. By updating the contents and outputting the contents to the gantry control means 8, the gantry 7 is controlled and the moving body can be captured in the center of the visual field image of the image input means 1. By repeating the above operation, the moving body can be tracked.

On the other hand, instead of controlling the output start timing of the image memory from the output of the position calculating means as described above, the output obtained by exchanging the position detected by the encoder mounted on the frame with the coordinate system of the image. Thus, even if the output start timing of the image memory is controlled, the object of the present invention can be achieved similarly.

A second embodiment of the present invention will be described below with reference to FIG. In FIG. 2, 1 is an image input means and 2 is a first
An image memory, 3 is a second image memory, 4 is a positioning means, 5 is a moving body detecting means, 6 is a position calculating means,
7 is a mount, 8 is a mount control means, 9 is a register, 1
Reference numeral 0 is an adding unit, 11 is a control unit, and the above is the same as the configuration of FIG. The difference from the configuration of FIG. 1 is that there are 12 encoders and 13 coordinate exchange means.
The position of the mount is detected.

In this embodiment, the output of the encoder 12 is fed back to the gantry control means 8 for control. The coordinate exchange means 13 is means for exchanging the output of the encoder 12 into the coordinate system of the image.

The operation of the moving body tracking device configured as described above will be described below. First, n
It is assumed that the n-th image data input by the image input means 1 has been stored in the first image memory 2 after all the processing in the second processing has been completed. Therefore, the (n + 1) th process is started.

Now, the n + 1-th image data input by the image input means 1 in accordance with a command from the control means 11 is converted into the second image data.
It is stored in the image memory 3. Next, the control means 11 clears the output of the alignment means (deviation amount = 0, 0), and the content of the register 9 is output as it is as the output of the addition means 10. The register 9 stores the output of the coordinate exchange means 13 at the nth position, that is, the movement vectors x '(n) and y' (n) in the coordinate system on the screen when the gantry is driven at the nth position. Therefore, the adding means 10 delays (advances) the output start timing of the second image memory 3 by x '(n) in the x direction and y' (n) in the y direction, so that the second image memory 3 becomes x. ′ (N),
An image whose position is corrected by y '(n) is output.

At this time, the control means 11 causes the first image memory 2
And the output of the second image memory 3 are sent to the alignment means 4. Therefore, the alignment means 4 detects the displacement vectors Δx ′ (n + 1) and Δy ′ (n + 1) of both images.
Then, the control means 11 outputs the position adjustment means 4, that is, the position shift vectors Δx ′ (n + 1) and Δy ′ (n.
By outputting +1) to the adding means 10, the output start timing of the second image memory 3 is x '(n) + in the x direction.
Δx ′ (n + 1), y ′ (n) + Δy ′ (n +
By delaying (advancing) by 1), the second image memory 3 has x ′ (n) + Δx ′ (n + 1), y ′ (n) +.
An image whose position is corrected by Δy ′ (n + 1) is output. At this time, the control means 11 sends the output of the first image memory 2 and the output of the second image memory 3 to the moving body detection means 5. In this way, the n-th image and the (n + 1) -th image whose positional deviation has been corrected by the alignment means 4 are the moving body detection means 5
Then, if the difference between the two images is obtained, only the moving object can be detected from the images.

The position calculating means 6 obtains the detected center of gravity of the moving body, calculates the position of the moving body in the coordinate system with the center position of the image as the origin, and outputs the position to the gantry control means 8, whereby the gantry 7 is The moving object can be captured in the center of the visual field image of the image input means 1 under the control. At this time, the encoder 12 detects the rotation angle of the gantry when panning (tilting), and the coordinate exchange means 13 causes the movement vectors x ′ (n + 1) and y ′ (n + 1) in the coordinate system on the screen.
Is obtained, and the contents of the register 9 are updated.
By repeating the above operation, the same operation as in the first embodiment can be performed.

As described above, as in the second embodiment of the present invention, the position of the gantry is detected by the encoder, converted into the image coordinate system by the coordinate exchanging means, and the output start timing of the image memory is controlled. Even if the control position of the gantry shifts largely due to disturbance, the position shift vector is correctly detected and the position can be adjusted. Further, since the encoder is fed back to the gantry control means, there is an advantage that the gantry can be stably controlled.

In the first and second embodiments of the present invention, the alignment means may use the block matching method. Regarding the block matching method, for example, the document “Multidimensional signal processing of TV images” (written by Takahiko Fukibe,
Published by Nikkan Kogyo Shimbun, pp. 203-204). Further, it goes without saying that the control of the image output start timing in the first and second embodiments can be similarly performed even when the first image memory is used instead of the second image memory.

Further, the moving body detecting means may divide an image into small regions and perform comparison by using a statistic amount (for example, average or variance) for each small region, instead of simply extracting the difference between the images. . Further, even if the moving object is detected by extracting the edge images from the images and comparing them, the same operation can be performed.

[0034]

As described above, according to the present invention, n
Positioning the images by comparing the output image of the first image memory that stores the image data of the th frame with the output image of the second image memory that stores the image data of the (n + 1) th frame and has a variable output start timing. Then, the nth image and the (n + 1) th aligned image are compared to detect the moving body in the image, and the center of gravity of the detected moving body is calculated to start the output of the second image memory from the calculated target position. By controlling the timing, even if the field-of-view image is greatly panned (tilted) and the positional shift between the images is large, the positional shift can be corrected without exceeding the alignment range.

Further, instead of controlling the output start timing of the image memory by the position of the moving body detected as described above, it is obtained by converting the position detected by the encoder attached to the frame into the coordinate system of the image. Even if the output start timing of the image memory is controlled by the generated output, the effect of the present invention can be similarly obtained.

[Brief description of drawings]

FIG. 1 is a schematic block diagram of a moving body tracking device according to a first embodiment of the present invention.

FIG. 2 is a schematic block diagram of a moving body tracking device according to a second embodiment of the present invention.

FIG. 3 is an explanatory view of the principle of misregistration correction in the embodiment of the present invention.

FIG. 4 is an explanatory diagram of a processing procedure in the first embodiment of the present invention.

FIG. 5 is a schematic block diagram of a moving body tracking device in a conventional example.

[Explanation of symbols]

 1 Image Input Means 2 First Image Memory 3 Second Image Memory 4 Positioning Means 5 Moving Object Detecting Means 6 Position Calculating Means 7 Mounts 8 Mounts Controlling Means 9 Registers 10 Adding Means 11 Controlling Means 12 Encoders 13 Coordinate Converting Means

Claims (2)

[Claims] 1. An image input unit for sequentially inputting and outputting a field-of-view image, a first image memory for storing n-th image data output from the image input unit, and an (n + 1) -th image output unit from the image input unit. A second image memory that stores image data and has a variable output start timing; and a positioning unit that compares the output image of the first image memory and the output image of the second image memory to position the images. A moving body detecting means for comparing the nth image and the n + 1th image aligned by the aligning means to detect a moving body in the image; and a moving body detected by the moving body detecting means. Position calculating means for obtaining a center of gravity and calculating a target position, a gantry and a gantry control means for driving and controlling the visual field direction of the image inputting means by the target position output from the position calculating means. And a means for controlling the output start timing of the second image memory by the output of the position calculating means. 2. An image input means for sequentially inputting and outputting a field-of-view image, a first image memory for storing nth image data output from the image input means, and an (n + 1) th image memory output from the image input means. A second image memory that stores image data and has a variable output start timing; and a positioning unit that compares the output image of the first image memory and the output image of the second image memory to position the images. A moving body detecting means for comparing the nth image and the n + 1th image aligned by the aligning means to detect a moving body in the image; and a moving body detected by the moving body detecting means. Position calculating means for obtaining a center of gravity and calculating a target position, a gantry and a gantry control means for driving and controlling the visual field direction of the image inputting means by the target position output from the position calculating means. An encoder for detecting the position of the gantry, a coordinate conversion means for converting the output of the encoder into an image coordinate system, and a means for controlling the output start timing of the second image memory by the output of the coordinate conversion means. A moving object tracking device comprising: