JP2003006649A - Invading object tracking method and invading object monitoring device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an invading object tracking method and an invading object monitoring device with high reliability with which an invading object can be exactly tracked by setting the suitable size of a template image even when a plurality of invading objects are photographed while being overlapped or when the luminance value of pixels comprising the image of the invading object is close to the luminance value of pixels comprising the image of the background in a reference background image. SOLUTION: An operating marker to be moved corresponding to an operating means is displayed on a monitor while being superimposed on an input image picked up by an image pickup device and on the basis of the position of the operating marker, the size of the template image is set. The size of the template image set by size setting of template image is newly registered as a candidate for the size of template image, and any size candidate of the template image is selected. Then, the invading object in the field of vision of the image pickup device is detected and tracked.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a monitoring device using an image pickup device, and more particularly, to an object that has entered the image pickup field of view, which is automatically detected from a video signal input from the image pickup device and uses an operating means. Image of an object detected based on the size of the template image set as a template image, and an intruding object automatically detecting the movement of the object detected by template matching based on the template image The present invention relates to a tracking method and an intruding object monitoring device that displays the detected movement of an object on a monitoring monitor to give a predetermined notification to a monitoring person.

[0002]

2. Description of the Related Art A video surveillance device using an image pickup device such as a camera has been widely used. However, in a surveillance system using such a video surveillance device, a manned person who performs detection and tracking of an intruding object such as a person or an automobile entering the surveillance field of view while observing an image displayed on the monitor Instead of monitoring, a system that automatically detects an intruding object from an image input from an image input means such as a camera and automatically tracks the movement of the intruding object to obtain a predetermined notification or alarm action is provided. It is becoming required.

In order to realize such a monitoring system, first, an intruding object in the visual field is detected by a difference method or the like. The difference method is a comparison between an input image obtained by an imaging device such as a television camera (hereinafter referred to as a camera) and a reference background image created in advance, that is, an image in which an object to be detected is not captured, This is a processing method in which a difference in luminance value is obtained for each pixel and an area having a large difference value is detected as an object. The image of the intruding object detected in this way is registered as a template, and the position where the degree of coincidence with the template image is the maximum is detected in the images that are sequentially input. This method is called template matching, and is described, for example, in pages 149 to 153 of a book entitled "Introduction to Computer Image Processing" supervised by Hideyuki Tamura, published by Soken Shuppan in 1985. Usually, when tracking a target object using template matching, the image of the position of the target object detected by the matching process is sequentially updated as a new template in order to follow changes in the posture of the target object. These processes of the conventional technique will be described with reference to FIGS. 4 and 5.

FIG. 4 is a diagram for explaining a process of detecting an object that has entered the visual field of the image pickup apparatus by the difference method and registering the detected object as an initial template image so that it can be used for template matching. Is. In FIG. 4, 401 is the input image, 402 is the reference background image, and 40
3 is the difference image between the input image 401 and the reference background image 402, 40
4 is a binarized image of the difference image 403, 405 is a difference processing unit, 4
06 is a binarization processing unit, 407 is a human-shaped object in the input image 401, and 408 is a difference image corresponding to the human-shaped object 407.
3 human difference image, 409 is the binarized image corresponding to the human difference image 408, humanoid object in 404 (humanoid binarized image), 410 is the detected humanoid object 409 is a circumscribing rectangle, 411 is an extraction processing unit that extracts a specified region from the input image 401, 412 is an extracted image, and 413 is a template image.

In FIG. 4, first, an input image 401 of 320 × 240 pixels is input from an image pickup device such as a camera. Next, in the difference processing unit 405, the input image 401
And a difference for each pixel from the reference background image 402 created in advance is calculated, and the difference image 403 is acquired. At this time, the humanoid object 407 in the input image 401 appears in the differential image 403 as a humanoid differential image 408. Then, in the binarization processing unit 406, with respect to each pixel of the difference image 403, the value of the pixel whose difference value is less than a predetermined threshold value is “0”, and the value of the pixel which is more than the threshold value is “255” ( 1 pixel is treated as 8 bits) and the binarized image 404 is obtained. At this time, the humanoid object 40 captured in the input image 401
7 is detected as a humanoid object 409 in the binarized image 404. The above is the outline of the object detection processing process by the difference method.

Next, the circumscribed rectangle 410 of the human-shaped object 409 detected by the difference method in the binarized image 404 is detected. Next, the extraction processing unit 411 extracts the area represented by the circumscribed rectangle 410 from the input image 401. The extracted image is registered as the template image 413 in the extracted image 412. The above is the outline of the initial template image registration processing process.

FIG. 5 is a diagram for explaining a processing process for tracking an intruding object by template matching. In FIG. 5, 501, 503, 505, and 507 are times t.
Extracted images at 0-1, t0, t0 + 1, t0 + 2, 501a, 503
a, 505a, 507a are template images obtained at times t0-1, t0, t0 + 1, t0 + 2, 502, 504, 506, 508 are at times t0, t0 + 1, t0 + 2, t0 + 3. Input image, 509
a, 509b, 509c, 509d are matching processing units, 510a,
510b and 510c are template image update units.

In FIG. 5, the matching processing unit 509a,
509b, 509c, 509d are the extracted images 501, 503, 505, 5
07 Template images 501a, 503a, 50
5a, 507a and their respective input images 502, 504, 506, 50
8 is compared and the part of the input image that has the highest degree of matching with the template image is detected. The above is the template matching processing process.

Template image update units 510a, 510b, 510
c is the position of the new intruding object that has the highest degree of matching detected by the matching processing units 509a, 509b, 509c, and 509d, and the template image is updated by replacing the extracted image and template image with that image Do.
Here, template matching and template image update are performed at times t0-1, t0, t0 + 1, t0 + 2, t0 + 3 according to FIG.
Will be explained step by step.

First, a template image 501a obtained at time t0-1 and an input image 502 obtained at time t0.
Perform template matching processing using. Here, in the first processing frame of the tracking process, the template image 501a is a human-shaped object 407 detected by the difference method.
It becomes an image of. Template matching processing part 509a
Then, the template image 501a in the input image 502
The area 502c around the area corresponding to the position (area 502b) is set as the search area, the degree of coincidence with the template image 501a is obtained, and the area that most matches (the highest degree of coincidence) with the template image 501a in the search area 502c. 502a is detected (matching). Where the search area 502c is the area
Horizontal direction ± Sx pixels, vertical direction ± Sy based on 502b
It is set as a pixel. Where, for example, Sx = 30, Sy
= 10. Sx and Sy are the input images of the target to be tracked
502 Set to the apparent maximum amount of movement. In order to obtain the degree of coincidence, for example, the normalized correlation shown in Equation (1) is used.
r (x, y, u, v) can be used.

[Equation 1]

In equation (1), f () is the input image and g ()
Represents a template image, (x, y) is the coordinates in the search region 502c of the input image (called a matching target region), (u,
v) represents the upper left coordinates of the template image 501a (the coordinates when the template image was registered). However, the upper left corner of all images is the origin (0,0). W is the width of the template image and H is the height of the template image. Also, the formula
In (1), the bar above f () and g () represents the average value,
These are the average luminance value of the matching target area in the input image and the template image, respectively, and are expressed by equations (2) and (3).

[Equation 2]

[Formula 3]

The normalized correlation r (x, y, u, v) is the input image f
Width W and height H with upper left coordinates at position (x, y) of ()
Value distribution in the area of and the position of template image g ().
It represents the degree of coincidence between the luminance value distribution of the area of width W and height H with (u, v) as the upper left coordinate, and the luminance value of each pixel of the input image f () and the template image g () is If all values are the same, it becomes 1.0. Template matching is a process of detecting a portion of the input image f () that has the highest degree of matching with the template image g (). That is, with reference to the position (u, v) of the template image g (), (x, y)
Is changed to u-Sx ≤ x <u + Sx, v-Sy ≤ y <v + Sy, and equation (1)
The normalized correlation r (x, y, u, v) represented by is maximum (x, y)
Find out. In the example of FIG. 5, the intruding object is the input image 502.
Since it is located at position 502e, the region with the best match is 502a (upper left coordinates (x, y)). In addition to the normalized correlation, a region of width W and height H with the position (x, y) of the input image f () as the upper left coordinate and the template image g ()
The average value of the absolute values of the differences in the brightness values of the pixels in the area of width W and height H with the position (u, v) at the upper left coordinates may be used as the degree of coincidence. In this case, when the brightness values of the respective pixels of the input image f () and the template image g () are all the same value, it becomes 0.0. Template matching processing part 509
According to the processing result of a, the area 502b (upper left coordinates (u,
The intruding object existing in (v)) is determined to have moved to the area 502a (upper left coordinates (x, y)). Therefore, the area
An arrow 502d connecting the center of 502b and the center of region 502a is the moving direction of the intruding object. Next, in the template image updating unit 510a, the matching region 50 of the input image 502 is
2a is extracted to obtain the extracted image 503, and the template image
Update 501a as a new template image 503a.

Next, the input image 5 input at time t0 + 1
For 04, in the template matching processing unit 509b, as in the case of time t0, the area (area 5) corresponding to the position of the template image 503a in the input image 504 is displayed.
The area 504c surrounding 04b) is set as the search area, and the area 504 that most matches the template image 503a in the search area 504c.
detect a. In this example, the moving direction of the intruding object is arrow 504d. In addition, the template image update processing unit
In 510b, the matching region 504a of the input image 504
To obtain the extracted image 505 and the template image 5
Update 03a as a new template image 505a.

Next, the input image 5 input at time t0 + 2
As for 06, the template matching processing unit 509c, as in the case of time t0 + 1, determines the region (region) corresponding to the position of the template image 505a in the input image 506.
506b) around 506c as a search area, and search area 506c
Region 50 that most matches template image 504a
Detect 6a. In this example, the moving direction of the intruding object is
It becomes arrow 506d. Further, in the template image update processing unit 510c, the matching region 506 of the input image 506 is
Extract a to get the extracted image 507,
Update 505a as a new template image 507a.

Subsequently, the template matching processing unit is applied to the input image 508 input at time t0 + 3.
In 509d, as in the case of time t0 + 2, the input image 508
The area 508c around the area (area 508b) corresponding to the position of the template image 507a in FIG.
A region 508a that most matches the template image 506a in 08c is detected. In this example, the moving direction of the intruding object is arrow 508d. In this way, the input image 502
The intruding object 502e shown in the figure is arrows 502d, 504d, 506d, 50.
It can be judged that they moved in the order of 8d.

In the tracking method of an intruding object using the difference method and template matching described above, the circumscribed rectangle of the detected object detected by the difference method becomes the size of the template image (width W, height H). However, when a plurality of intruding objects are present within the field of view of the image pickup apparatus and are overlapped in the image, they are observed larger than the actual size of the intruding object. When such a phenomenon occurs, many pixels other than the intruding object are included in the template image, which makes it difficult to accurately track the intruding object.

Further, since the difference method is a processing method for detecting an intruding object based on a difference in brightness between the input image and a reference background image prepared in advance for each pixel, the brightness value of the pixels forming the image of the intruding object is detected. When the brightness value of the pixel forming the background image in the reference background image is close, the pixel is not detected. Therefore, a part of the detected object 409 is missing, or a part of the detected object 409 is detected separately, and the circumscribed rectangle of the detected object becomes smaller than the intruding object. When such a phenomenon occurs, template matching processing must be performed using a template image smaller than the intruding object, which makes it difficult to accurately track the intruding object. To solve such a problem, a method of predetermining the apparent size of the intruding object to be traced (fixing the size of the template image) is adopted. However, when this method is used, it is difficult to track the intruding object because the size of the template image cannot be set appropriately for the intruding objects having different sizes.

[0018]

DISCLOSURE OF THE INVENTION Problems to be Solved by the Invention
When a plurality of intruding objects are present in the field of view of the image pickup apparatus and are overlapped in the image, the size of the intruding object is observed to be large. Further, when the brightness value of the pixels forming the image of the intruding object and the brightness value of the pixels forming the background image in the reference background image are close to each other, the size of the intruding object is observed to be small. It was Furthermore,
Even if a method of previously determining the apparent size of an intruding object to be tracked is used, there is a drawback in that an intruding object having a different size cannot be accurately tracked. The object of the present invention is to eliminate the above-mentioned drawbacks and to form the image of the background of the reference background image and the brightness value of the pixels forming the image of the intruding object when a plurality of intruding objects are overlapped. Provided is a highly reliable intruding object tracking method and intruding object monitoring device capable of accurately tracking an intruding object by setting an appropriate template image size even when the brightness values of pixels are close to each other. To do.

[0019]

In order to achieve the above object, an intruding object tracking method according to the present invention uses an image pickup device to pick up an image of an operation marker that moves according to an operation signal input using the operation means. The size of the template image is set based on the position of the operation marker and is displayed on the monitor by superimposing the input image on the input image, the position of the operation marker, and the size of the template image. By registering a template image, an intruding object is tracked.

That is, the intruding object tracking method of the present invention detects an object in the imaging visual field from the input image obtained by imaging the imaging visual field by template matching based on the template image, and detects the detected object. In an intruding object tracking method for tracking, an input image is displayed on a display screen of a display device, and an operation marker is displayed on the display screen in a superimposed manner,
The operation marker is moved according to the instruction of the operator, and a part of the image area of the input image is set in the template image based on the position of the operation marker.

The template image set by the intruding object tracking method of the present invention cuts out a part of the image area of the input image corresponding to a polygon having the position of the operation marker as one vertex. Furthermore, at least one of the size or shape of the template image is stored as a template size candidate, and the operator can select a desired position of the input image,
By designating one of the selected template size candidates, a part of the image area of the input image is set as the template image. Further, the size or shape of the set template image is added or updated as a size candidate of the template image.

Furthermore, the intruding object monitoring apparatus of the present invention detects an object in the imaging field of view by template matching based on the template image, and when tracking the detected object, the image signals acquired by the imaging apparatus are sequentially acquired. An image input interface for converting into an input image, an operation means for inputting an instruction from a user, an image storage means for storing the input image, an image processing means for processing the input image, and a processing marker superimposed on the input image. And a display device for displaying
The image processing means moves the operation marker displayed on the image of the display device based on the input signal from the operation means, and the image processing means causes the template image to be generated based on the input signal of the operation means and the position coordinates of the operation marker. The size of the input image, the position of the operation marker, and the size of the template image based on the size of the template image. Is to track.

Further, in the method of tracking an intruding object according to the present invention, in setting the size of the template image, the size of the template image is set by using a polygon whose vertex is the position designated by the operation marker. Is. Further, in the intruding object tracking method of the present invention, the operation marker that moves according to the operation signal input using the operation unit is displayed on the monitor by superimposing the operation marker on the input image captured by the imaging device, and the operation unit is displayed. The template image size candidates are displayed according to the operation signal input using the template, and the template image size is selected from the template image size candidates according to the operation signal input using the operating means. The initial template image is registered based on the input image, the position of the operation marker, and the size of the template image, so that the intruding object is tracked. Furthermore, in the intruding object tracking method of the present invention, the size of the template image set in the size setting of the template image is newly registered as a candidate of the size of the template image, and is used in the subsequent size selection of the template image. It is something that can be done.

That is, the intruding object monitoring apparatus according to the present invention comprises an image pickup means such as a camera, an image input I / F for inputting an image picked up by the image pickup means such as a camera, and an image output from the image input I / F. A process of detecting an intruding object in the field of view of the image pickup means and storing the image memory, setting the size of the template image according to the operation signal of the operator, and tracking the intruding object based on the size of the template image. A program memory that stores the operation program of the intruder monitoring device, a CPU that performs intrusion object detection, template image size setting, and intrusion object tracking processing according to the operation program, and an image memory A work memory that temporarily stores data when performing image analysis and at least one or more actions such as the operator's voice, gesture, hand gesture, and line-of-sight direction. Image output that superimposes on the input image the operable operation means, the input I / F that inputs the operation signal of the operation means, and the predetermined operation marker that moves on the image corresponding to the operation signal of the operation means and output. I / F and
The program stored in the program memory invades by using the difference method from the input image input from the image pickup unit through the image input I / F and the reference background image previously recorded in the image memory. A means for detecting an object, a means for setting the size of the template image based on the operation signal from the operation means, and the position coordinates of the operation marker, an input image, and template matching based on the size of the template image are used. It is designed to trace intruding objects.

[0025]

FIG. 3 shows one embodiment of the intruding object monitoring apparatus of the present invention. FIG. 3 is a block diagram showing the hardware configuration of the intruding object monitoring apparatus. 30
1 is an image pickup device (hereinafter, referred to as a camera), 302 is an electric swivel base (hereinafter, referred to as a camera platform), 303 is an operation unit, and 303a
Is a first button attached to the operating means 303, 303b is a second button attached to the operating means 303, 304 is an intruding object monitoring device, 304a is an image input I / F (I / F: Interface), 30
4b is a pan head control I / F, 304c is an input I / F, 304d is an image memory, 304e is an image output I / F, 304f is an output I / F, 30
4g is CPU (CPU: Central Processing Unit), 304h
Is a program memory, 304i is a work memory, 304j is a data bus, 305 is an output monitor, and 306 is a warning light.

The camera 301 is connected to the image input I / F 304a, the camera platform 302 is connected to the platform control I / F 304b, the operation means 303 is connected to the input I / F 304c, and the output monitor 305 is connected to the image. Warning light 3 connected to output I / F 304e
06 is connected to output I / F 304f. Image input I / F 304a, pan head control I / F 304b, input I / F 304c, image memory 304d, image output I / F 304e, output I / F 304f, C
PU 304g, program memory 304h, and work memory 3
04i is connected to the data bus 304j.

In FIG. 3, a camera 301 captures an image within a target imaging field of view. The camera 301 converts the captured image into an electrical image signal (for example, NTSC image signal) and outputs the image signal to the image input I / F 304a. The image input I / F 304a is a format in which the input video signal is handled by an intruding object monitoring device (for example, width 320 pix, height 24
Convert to 0 pix, 8 bit / pix) image data and send to image memory 304d via data bus 304i. Image memory
304d stores the sent image data. The operation means 303 includes an operation direction of the operator and buttons 303a and 303b.
The state of is converted into an electrical signal (for example, contact signal), and the signal is output to the input I / F 304c. CPU 304g uses the program stored in program memory 304h
In the work memory 304i, the signal input to the input I / F 304c and the image stored in the image memory 304d are analyzed.

As a result of the above analysis, an intruding object in the visual field of the camera 301 is detected and tracked. The CPU 304g uses the work memory 3 on the input image stored in the image memory 304d.
The operation pointer is drawn (superposed) based on the operation pointer position stored in 04i, and the camera image is displayed on the output monitor 305 via the image output I / F 304e. Image output
I / F 304e outputs the signal from CPU 304g. Monitor 305
Formats that can be used by (eg NTSC video signals)
And send it to the display monitor 305. Display monitor 305
Displays a camera image, for example. In addition, CPU 304
According to the program stored in the program memory 304h, g turns on the warning light 306 via the output I / F 304f when, for example, an intruding object is detected. Output I
The / F 304f converts the signal from the CPU 304g into a format that can be used by the warning light 306 (for example, a contact signal) and sends it to the warning light 306. The warning light 306 is turned on, for example.

The embodiment described below is executed using FIG. 3, which is an example of the hardware configuration of the intruding object monitoring apparatus. Also, in each example, the image to be used has a width of 320
Described as pix, height 240 pix, and 8 bit / pix. Of course, it goes without saying that the same operation is performed even if an image having a pixel number other than this is used.

The first embodiment of the present invention will be described with reference to FIG. FIG. 2 is a flowchart showing an example of the processing operation of the first exemplary embodiment of the present invention. In the first embodiment, when an intruding object is detected by the difference method, the operator (monitor) can operate the operating means to set the size of the template image.

The processing operations from the image input step 101 to the object presence determination step 104 will be described with reference to the difference method of FIG. 4 used in the prior art. In the image input step 101, from the camera 301, for example, 320 pix width
, Input image 401 with height of 240 pix and 8 bit / pix. In the difference processing step 102, the image input step 10
The difference for each pixel between the input image 401 obtained in 1 and the reference background image 402 previously recorded in the image memory 304d is calculated, and the difference image 403 is obtained. Next, in the binarization processing step 103, threshold processing is performed on each pixel of the difference image 403 obtained in the difference processing step 103, and if the pixel value is less than the threshold value (for example, 20), the pixel value Is set to “0”, and if it is equal to or more than the threshold value, the pixel value is set to “255” to obtain the binarized image 404. Next, in the object presence determination step 104, it is determined whether or not a cluster of “255” (binarized image of a humanoid object) 409 is present in the binary image 404 obtained in the binarization processing step 103. If there is, go to operation start determination step 105,
If it does not exist, the process branches to the image input step 101.

In operation start judgment step 105, input I /
An operation signal for determining the operation signal from the operation means 303 obtained through the F 304c and starting the size setting of the template image (for example, an operation button attached to the operation means 303.
If it is an operation signal indicating that 303a is pressed), the process branches to operation marker display step 106, and if it is not an operation signal for starting the size setting of the template image, the process branches to template image registration step 111. In the operation marker display step 106, the input image (for example, the input image of FIG.
1) An operation marker (for example, an arrow mark or any other mark that can specify the movement location) that is moved by the operation of the operation means 303 is drawn (superposed on the image). Next, in template size setting step 108, the position of the operation marker at the moment when the operation button 303b is pressed is determined based on the position of the operation marker and, for example, the operation signal that the operation button 303b attached to the operation means 303 is pressed. Record as a set point in work memory 304i. In the setting completion determination step 109, for example, it is determined that the template image size setting process is completed by pressing the operation button 303a attached to the operation unit 303, and the operation button
If 303a is not pressed, it is determined that the template image size setting process has not been completed. When the setting is completed, branch to the template image registration step 111,
If not completed, the process branches to operation marker display step 106. Therefore, the operator (monitor) can set the template image by designating the image region showing the object to be tracked with the operation marker while looking at the displayed input image.

In the template image registration step 111, from the input image (for example, the input image 401 in FIG. 4),
In the template size setting step 108, a region corresponding to a polygon whose vertex is the set point recorded in the work memory 304i is cut out and used as an initial template image (for example, the initial template image 413 in FIG. 4) in the image memory. Register on 304d. Here, if it is determined in the operation start determination step 105 that the operation means 303 is not operated, the binarized image (for example, the binarized image 4 in FIG.
Object detected in 04) (eg object 409 in Figure 4)
The size of the circumscribing rectangle (eg, circumscribing rectangle 410 in FIG. 4) is used as the size of the template image, and the input image (eg, input image 401 in FIG. 4) corresponding to that position is cut out and used as the initial template. Image memory 304d as an image (eg, the initial template image 413 in FIG. 4)
Register with.

Next, from the image input step 112, an alarm /
The processing operation up to the monitor display step 116 will be described with reference to the explanatory diagram of the object tracking processing process by the template matching method of FIG. In the image input step 112, similar to the image input step 101, from the camera 301, for example, width 320 pix, height 240 pix,
Get the input image 502 of 8 bit / pix. Next, in template matching step 113, template matching processing units 509a to 509d are performed to detect (match) a region 502a having the highest degree of matching with the template image 501a in the search region 502c and detect an intruding object. Detects the moving position of.

Next, in the maximum matching degree determination step 114,
The matching degree obtained by the template matching 113 is a predetermined value (for example, when the normalized correlation is used as the matching degree).
If it is 0.5) or more, it is judged that the matching is completed normally and the process branches to the template update step 115. If it is less than the predetermined value, the matching is not completed normally (the intruding object is lost). It branches to the image input step 101. Further, in the template update step 115, the same process as the template image update unit 510a is performed, the region corresponding to the matched region 502a in the input image 502 is cut out, and the template image is updated as a new template image 503a. . In the alarm / monitor display step 116, for example, an image of the target object is displayed on the monitor 305 or a warning lamp is displayed.
For example, the 306 is turned on to notify that an intruding object is present. Then, the process returns to the image input step 112 to continue the monitoring. With the above steps,
An intruding object in the field of view of 301 is automatically detected, and if necessary, a surveillance staff operates the operation means 303 to set the size of the template image according to the apparent size of the intruding object. Can be tracked.

In the above-described first embodiment of the present invention, the polygon area having the set points set in the template size setting step 108 as vertices is used as the template image. A rectangular area having a diagonal line may be used as the template image. This example will be described with reference to FIGS. 6 and 7. FIG. 6 is an example in which there are two set points and a rectangular area having these set diagonally is a template image.
601 is the input image, 602 is the operation marker, 603 is the input image 6
A human-shaped object shown in 01, 601a and 601b are set points, and 604 is a template image area. In this example, the operation means 303 is operated to specify the set points 601a and 601b, and the template image registration step 111
Then, the rectangular area 604 is calculated based on these two set points 601a and 601b, and this is set as the area of the template image.

Next, FIG. 7 shows an example in which a set point is 12 and a polygonal area having the set point as a vertex is used as a template image. 701 is an input image, 702 is an operation marker, 703 is a humanoid object in the input image 701, 701a to 701l are set points,
Reference numeral 704 is an area used as a template image. In this example, the operation means 303 is operated to change the set point from 701a to the set point.
701l is designated, and in the template image registration step 111, a polygonal area 704 is calculated based on the set points 701a to 701l and is set as the area of the template image.

In the examples of FIGS. 6 and 7, the area of the template image is a rectangle or a dodecagon, but it may be a polygon other than this, or a circle (a center point and one point of the circumference are set points). It goes without saying that any shape such as an ellipse (two focal points and one point on the circumference are set points) may be used.

A second embodiment of the present invention will be described with reference to FIG. FIG. 1 is a flow chart showing an example of the processing operation of the second embodiment of the present invention. In the second embodiment, when an intruding object is detected by the difference method, the operator (monitor) operates the operating means and selects the template image size from the template image sizes set so far. This allows the user to select the size. FIG. 1 shows the operation marker display step 106 of the flowchart shown in FIG.
The template size candidate display step 107 is inserted between the template size setting step 108 and the template size setting step 108, and the size candidate registration step 111 is inserted between the setting end determination step 109 and the template image registration step 111.
Is inserted. Therefore, here, the size candidate display step 107 and the size candidate registration step 111 are performed.
Since the other processing operations are the same as those in FIG. 2, the description thereof will be omitted.

In the size candidate display step 107, the size candidates of the template images stored so far in the work memory 304i are displayed. As a display example, for example, as shown in FIG. 8, any one of the template image size candidates 803a, 803b, 803c, and 803d registered so far is displayed around the operation marker 802. The size candidates of the displayed template image are switched, for example, by operating the operating means while pressing the operating means 303b, and each time the switching operation is performed, the size candidate 803a to the size candidate 803b and the size candidate 803 are selected.
Switching from b to size candidate 803c, from size candidate 803c to size candidate 803d, and from size candidate 803d to size candidate 803a (in this example, the maximum number of registrations is 4). The operator (monitor) can select a template image size candidate that matches the intruder's apparent size, and use this as a template image size. When the size candidates of the template image as shown in FIG. 8 are not switched (the operation means 303b is not operated while the operation means 303b is being pressed), the rectangle, as described in the first embodiment,
It is possible to set the size of the template image such as a polygon, a circle, and an ellipse.

In the size candidate registration step 110, from the operation marker display step 106 to the setting end determination step 1
When the template image size is newly set by the process of 09 (when the template image size is not selected from the template image size candidates), the work memory is used as a new template image size candidate.
Record on 304i. At this time, when the number of template image size candidates reaches a predetermined number (for example, 10), the oldest size candidate may be deleted from the work memory 304i. Further, the size candidate of the registered template image and the size candidate of the new template image may be replaced (updated) according to the designation of the operator. Through the above processing steps, an intruding object in the field of view of the camera 301 is automatically detected, and a monitor operates the operating means 303 if necessary according to the apparent size of the intruding object, and the size of the template image is increased. It is possible to select an appropriate template image size from the candidate and track the intruding object.

[0042]

Therefore, according to the present invention, when a plurality of intruding objects are imaged in an overlapping manner, or when the brightness values of the pixels forming the image of the intruding object and the pixels forming the background image in the reference background image are reduced. By making it possible to set an appropriate template image size even when the brightness values are close, and by selecting an appropriate template image size from the template image sizes set so far. The tracking performance of an intruding object can be further improved, and the range of application of the intruding object monitoring device can be greatly expanded.

[Brief description of drawings]

FIG. 1 is a flowchart illustrating an example of a processing operation according to an embodiment of the present invention.

FIG. 2 is a flowchart for explaining an example of processing operation according to the embodiment of the present invention.

FIG. 3 is a block diagram showing an example of a configuration of an exemplary embodiment of the present invention.

FIG. 4 is a diagram for explaining an object detection processing process by a conventional difference method.

FIG. 5 is a diagram for explaining a conventional object tracking processing process.

FIG. 6 is a diagram for explaining an outline of template image size setting using a rectangle according to the present invention.

FIG. 7 is a diagram for explaining an outline of template image size setting based on a polygon of the present invention.

FIG. 8 is a diagram for explaining an outline of template image size candidate selection according to the present invention.

[Explanation of symbols]

101: image input step, 102: difference processing step,
103: Binarization processing step, 104: Object existence determination step, 105: Operation start determination step, 106: Operation marker display step, 107: Size candidate display step, 10
8: Template size setting step, 109: Setting end determination step, 110: Size candidate registration step, 11
1: Template image registration step, 112: Image input step, 113: Template matching step, 114:
Maximum matching degree determination step, 115: template updating step, 116: alarm / monitor display step, 301: camera, 302: camera platform, 303: operating means, 303a:
Operation buttons, 303b: Operation buttons, 304a: Image input I
/ F, 304b: Head control I / F, 304c: Input I / F, 304d:
Image memory, 304e: Image output I / F, 304f: Output I / F
F, 304g: CPU, 304h: Program memory, 304i:
Work memory, 304j: Data bus, 305: Output monitor, 306: Warning light, 401: Input image, 402: Reference background image, 403: Difference image, 404: Binary image, 405:
Difference processing unit, 406: Binarization processing unit, 407: Humanoid object, 408: Difference image, 409: Humanoid binary image, 41
0 :, 411: extraction processing unit, 412: extraction image, 413: template image, 509a: template matching processing unit, 509b: template matching processing unit, 509c:
Template matching processing unit, 509d: Template matching processing unit, 510a: Template image updating processing unit, 510b: Template image updating processing unit, 510c: Template image updating processing unit, 510d: Template image updating processing unit, 601: Input image, 602 : Operation marker, 60
4: Set template image size, 701: Input image, 702: Operation marker, 704: Set template image size, 801: Input image, 802: Operation marker,
803: Size of the set template image.

   ─────────────────────────────────────────────────── ─── Continued front page    F term (reference) 5C054 AA05 CA04 CC02 CD01 CE12                       CF05 DA08 EB05 FC01 FC05                       FC12 GA04 HA18                 5C084 AA02 AA07 AA14 AA18 BB06                       BB31 CC17 DD12 EE02 EE05                       FF03 GG43 GG78 HH08                 5L096 AA06 BA02 CA07 DA01 EA35                       HA05 JA03 JA09

Claims (5)

[Claims] 1. An intruding object tracking method for detecting an object in an imaging visual field from an input image obtained by imaging the imaging visual field by template matching based on a template image and tracking the detected object. , Displaying the input image on a display screen of a display device, superimposing and displaying an operation marker on the display screen, moving the operation marker according to an instruction from an operator, and based on the position of the operation marker. And a part of the image area of the input image is set in the template image. 2. The method of tracking an intruding object according to claim 1, wherein the template image is a portion of an image area of the input image corresponding to a polygon having the position of the operation marker as one vertex. A characteristic tracking method for intruding objects. 3. The intruding object tracking method according to claim 1, wherein at least one of the size and shape of the template image is stored as a template size candidate, and the operator desires the input image. A part of the image area of the input image is set to the template image by designating one of the selected template size candidates at the position of the intruding object tracking method. 4. The intruding object tracking method according to claim 3, wherein the size or shape of the set template image is added or updated as a size candidate of the template image. Method. 5. An apparatus for detecting an object in an imaging field of view imaged by an imaging device and tracking the detected object by template matching based on a template image, successively inputting video signals imaged by the imaging device. An image input interface for converting into an image, an operation means for inputting an instruction from a user, an image storage means for storing the input image, an image processing means for processing the input image, and a processing marker for the input image. A display device for displaying in a superposed manner, the image processing means, by moving the operation marker displayed on the image of the display device based on an input signal from the operation means, the image processing means, The size of the template image is set based on the input signal of the operation means and the position coordinates of the operation marker, and the input image, An initial template registration for registering the template image is performed based on the position of the operation marker and the size of the template image, and an object in the imaging visual field is tracked by template matching based on the template image. Intruder monitoring system.