JPH08205131A - Monitor system and moving body tracking method thereof - Google Patents

Abstract

PURPOSE: To provide the monitor system which can accurately track a target moving body even when there are plural moving bodies in a monitor area. CONSTITUTION: A density pattern stack part 11 is stored with density image pattern information on respective bodies, detected by a moving body detection part 10, in time series. This density pattern stack part 11 is stored with the density image pattern information corresponding to respective bodies which can not be specified as identical bodies by an identical body correspondence decision part 13 and correspond to a low matching degree and respective current bodies before the last time. Consequently, not only collation between current density image pattern information and last density image pattern information, but also collation with density image pattern information on respective moving bodies which are detected last and before are performed for the tracking process of the CPU 1.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a monitoring system for monitoring a moving object on a premises such as a building, a site, a facility, etc. by using image processing technology.

[0002]

2. Description of the Related Art Conventionally, a moving object including an intruder or an attendant is detected by using a surveillance camera, an infrared sensor or the like, and a target moving object is monitored particularly in a premises such as a building, a site or a facility. Surveillance systems have been developed.

This surveillance system detects moving objects by inputting images within a surveillance area determined according to the position of a surveillance camera at predetermined time intervals and obtaining a difference between the respective input images. Furthermore, the target moving object is detected using the determination parameters such as the size of the moving object, the moving direction and the moving speed.

In such a monitoring system, in order to detect the determination parameter and identify the target moving object,
Tracking of moving objects is required. In the tracking process, a collation process is performed to collate a moving object detected based on the image information input at the current time (T time) with a moving object detected before that (T-1 o'clock before a predetermined time). To be done. If the matching degree obtained by this matching process is greater than or equal to a predetermined value,
It is possible to track that the same object is moving within the surveillance area.

However, when there are a plurality of moving objects in the monitoring area, the objects may intersect or contact each other. In such a case, if a plurality of objects are erroneously recognized as one object and a tracking process is executed by accumulating information based on the erroneous recognition, erroneous tracking will be performed. Further, the tracking process may be interrupted at the time of erroneous recognition because the information for tracking the target moving object cannot be held.

[0006]

In the conventional monitoring system, when a plurality of moving objects exist in the monitoring area, the target moving object is erroneously recognized due to the intersection or contact of each object,
Accurate tracking may not be possible. An object of the present invention is to provide a monitoring system capable of accurately tracking a target moving object even if a plurality of moving objects exist in the monitoring area.

[0007]

In the monitoring system of the present invention, when a plurality of moving objects exist in the monitoring area, the moving object detecting means detects each moving object, and the density corresponding to each moving object is detected. It has density image pattern storage means for storing image pattern information in time series, and same object correspondence determination means for performing the same object correspondence determination process for specifying the same object in time series for each moving object. Furthermore, in the same object correspondence determination processing of the same object correspondence determination means,
It is provided with a storage means for storing the density image pattern information before that and the current density image pattern information corresponding to the degree of matching that cannot be specified as the same object, and a tracking means for tracking each moving object.

[0008]

In the present invention, the density image pattern storage means is
The density image pattern information corresponding to each moving object detected by the moving object detecting means is stored in time series. The same object correspondence determination means identifies the same object in time series for each moving object based on the degree of matching between the current density image pattern information stored in the density image pattern storage means and the previous density image pattern information. Object correspondence determination processing is performed. In the same object correspondence determination process, the storage means extracts, for each moving object, density image pattern information before that corresponding to a matching degree that cannot be identified as the same object with respect to the current density image pattern information, and The previous density image pattern information and the current density image pattern information are stored in the density image pattern storage means. As a result, in the tracking process, not only the matching process between the current density image pattern information and the previous density image pattern information, but also the matching process with the density image pattern information of each moving object detected before the previous time are executed. To be done.

[0009]

Embodiments of the present invention will be described below with reference to the drawings. 1 is a block diagram showing the main part of the monitoring system of this embodiment, FIGS. 2 and 3 are flow charts for explaining the operation of this embodiment, and FIG. 4 is a conceptual diagram for explaining the operation of this embodiment. Is. (Structure of Monitoring System) As shown in FIG. 1, the present system includes a microprocessor (CPU) 1 and a main memory 2 which constitute a central control unit for controlling the entire system.
The central processing unit is connected to the image processing system via the system bus 3. The CPU 1 controls each component of the image processing system according to a monitoring program stored in the main memory 2 in advance.

The image processing system includes a camera interface (I / F) 6, an image memory 7, a display controller 9, a moving body detecting section 10, a density pattern stack section 11, a pattern collating section 12, a same object correspondence judging section 13, The tracking result information holding unit 14 and the monitoring purpose determination unit 15 are configured by respective constituent elements, and are connected via the image bus 4.

The camera I / F 6 is connected to the camera 5 which is an image input means for inputting an image of the surveillance area. The camera I / F 6 performs A / D conversion processing on the image of the monitoring area captured by the camera 5 and outputs digital image information to the image bus 4.

The display controller 9 is connected to the CRT monitor 8 and outputs the digital image information transferred via the image bus 4 and the marking indicating the target moving object to be monitored to the CRT monitor 8 for display.

The image memory 7 is composed of a plurality of frame memories, and sequentially holds digital image information continuously input at predetermined time intervals via the camera I / F 6. As will be described later, the image memory 7 holds background image information in which no moving objects (including intruders and visitors on the premises) are present as a reference image (R).

The moving object detection unit 10 detects a plurality of moving objects in the monitoring area based on the difference image between the background image (R) held in the image memory 7 and the image (current image) input at the current time. To detect.

The density pattern stack unit 11 is a storage device for time-sequentially accumulating and saving the density image pattern information (shading pattern information) corresponding to each moving object detected by the moving body detecting unit 10. .

The pattern matching unit 12 is means for calculating the degree of matching (cross-correlation value) between the past (before the present time) density image pattern stored in the density pattern stack unit 11 and the current density image pattern. is there. The pattern matching unit 12 calculates the matching degree of each moving object by the cross correlation method, SSDA method, or composite similarity method. For example, as a specific example using the composite similarity method, the pattern matching unit 12
“Matching degree S = [(→ a) · (→ b)] / [| → a | · |
→ b |] ”is executed. Where (→ a), (→
b) is a vector symbol that means the density image pattern of the objects a and b, respectively, and | → a | and | → b | are the sum of squares of the density of each object.

The same object correspondence determination unit 13 determines the correspondence between the previous moving object and the moving object detected at the present time (determination of the same object) based on the calculated matching degree.
Specifically, it is determined that the objects with a high degree of matching are the same object, or if the degree of matching exceeds a predetermined threshold value and is within a predetermined time period, it is determined that they are the same object. .

The tracking result information holding unit 14 is a storage device for holding tracking processing result information (trajectory information, position information, size, etc.) obtained by the tracking processing described later. further,
The monitoring purpose determination unit 15 is a determination unit for specifying a target moving object according to a preset standard.

Next, the operation of this embodiment will be described. (Monitoring Process of Moving Object) First, the overall operation of the monitoring system will be described with reference to the flowchart of FIG.

First, the CPU 1 sets various reference values necessary for the monitoring process in the main memory 2 (step S1).
The various reference values include a reference value (threshold value) of the matching degree and a time set value N (T = T-1 to TN) for storing information in the density pattern stack unit 11 in time series. Be done.

The CPU 1 uses the background image (reference image R) of the monitoring area as the initial image, the camera 5 and the camera I.
It is input via / F6 and stored in the image memory 7 (steps S2 and S3). The background image is image information in a state where no moving object exists in the monitoring area.

After this, the images of the monitoring area captured by the camera 5 are continuously input at predetermined time intervals (step S4). The input image is sequentially stored in the image memory 7 via the camera I / F 6 (step S5).

The moving object detection unit 10 detects a plurality of moving objects in the monitoring area based on the difference image between the background image (R) held in the image memory 7 and the image (current image) input at the current time. It is detected (step S6). Here, CPU1
Determines whether the moving object in the monitoring area detected by the moving object detection unit 10 is a single object or a plurality of objects. In this embodiment, it is assumed that a plurality of moving objects are detected.

Next, the tracking process which is the gist of the present invention is executed (step S7). As will be described later, this tracking process is mainly executed by each component of the density pattern stack unit 11, the pattern matching unit 12, and the same object correspondence determination unit 13 in addition to the CPU 1. Tracking process result information (track information, position information, size, etc.) obtained by this tracking process
Is stored in the tracking result information holding unit 14 (step S
8).

The monitoring purpose determining section 15 uses the information held in the tracking result information holding section 14 to determine the target moving object to be monitored according to the set reference (initial position, moving speed, moving direction, etc.). It is extracted and specified (step S9).
When the extraction and identification can be performed, the CPU 1 executes a notification process of notifying the CRT monitor 8 via the display controller 9 (YES in step S10, S11). Further, the processes from step S4 are repeated until the monitoring process is completed (step S12). (Tracking Process) Next, the tracking process of this embodiment will be described with reference to the flowchart of FIG. 3 and FIG.

First, the CPU 1 cuts out density image pattern information of a moving object detected by the moving object detection unit 10 from the image memory 7 and stores it in the main memory 2 (steps S20 and S21).

Here, in the density pattern stack unit 11, the past (current time T) detected by the moving body detection unit 10 is detected.
(Before the previous) density image pattern information is held.
The CPU 1 reads past density image pattern information from the density pattern stack unit 11 (step S22).

The pattern matching unit 12 executes a matching process for calculating the matching degree between the density image pattern information of each clipped moving object and the past density image pattern information (step S23). The same object correspondence determination unit 13 executes the matching process (the same object correspondence determination process) between the moving object at the present time and the moving object before that based on the matching degree obtained by the pattern matching unit 12 (step S24). ). That is, it is determined that the objects having a high degree of matching are the same object, or if the degree of matching exceeds a predetermined threshold value and is within a predetermined time period, it is determined that the objects are the same object.

Next, in this embodiment, the density image pattern information corresponding to the moving object detected last time (before T-1) when the matching degree is equal to or less than the threshold value with respect to the moving object at the present time (T). Is extracted from the stack unit 11 (step S2
5). That is, the latest moving object in the past that does not correspond to the current moving object is extracted.

By this extraction processing, if the corresponding past density image pattern information does not exist in the stack section 11, the density image pattern information corresponding to the current (current) moving object is stored in the density pattern stack section 11. (YES in step S26, S27). When the relevant past density image pattern information is extracted, the past density image pattern information and the density image pattern information corresponding to the moving object at the present time are obtained.
1 (NO in step S26, S28).

That is, in this embodiment, when the moving object at the present time is detected, the density pattern stack section 11
Along with the density image pattern information of all the moving objects of the previous time (T-1), the density image pattern information corresponding to the latest moving object before the previous time T-1 before the matching degree is equal to or less than the threshold value (no corresponding destination). Is stored.

Therefore, the CPU 1 has the pattern matching unit 1
When the moving object tracking processing is executed by the same object correspondence determination unit 13 as 2 above, not only the density image pattern of the previous time but also the density image pattern of the latest moving object that has no corresponding destination before T-1 before Perform collation processing with information.
As a result, even if a plurality of moving objects intersect or come into contact with each other, accurate tracking processing can be executed for each moving object.

A specific example of the tracking process of this embodiment will be described with reference to FIG. First, as shown in FIG. 4A, in the density pattern stack unit 11, density image patterns P1 and P2 are time-sequentially (for the present time, for each moving object (here, object A to object D)). T from the latest T-1 time point to the T-N time point). Here, the hatched pattern P2 is the information detected at the latest point in time for the same object. Also, the dotted pattern P3
Is information indicating that detection is not possible at that time.

Next, as shown in FIG. 4B, it is assumed that, for example, at the present time T, the density image patterns corresponding to the moving objects A and B are held in the monitoring area. Through the above-described tracking processing, the density image patterns before that are read from the density pattern stack unit 11 and the matching processing is executed. At this time, as shown in FIG. 4A, the density image pattern at the latest time point T-1 is only the pattern of the objects A and D.

Now, suppose that at time T-1, the objects A,
It is assumed that B and B intersect each other, and in addition to the objects A and D, the object B actually exists in the monitoring area. The object A is detected at the time point T-1 and also detected at the time points T-2 and T-3 before that, so that the object A is accurately tracked.

On the other hand, at the present time T, the density image pattern corresponding to the object B is input, but in the matching processing of the density image pattern at time T-1, the matching degree is less than the threshold value,
It is not determined that they are the same object. That is, with the conventional method, the tracking process from before is not possible, and the object B has appeared at this point. In other words, the density pattern stack section 11 stores density image pattern information that is erroneously recognized at time T-1.

Therefore, in the present invention, in the tracking process,
The collation processing is performed not only at the time T-1 stored in the density pattern stack unit 11 but also at each latest density image pattern information that has no corresponding destination before that. That is, here, the collation processing of the density image pattern information corresponding to the object B at the time point T-2 and the density image pattern information corresponding to the object C at the time point T-3 is executed. By this matching process, as shown in FIG. 4B, at the time point T-2, the same object correspondence determination with the object B is made, so that the object B is at least in the monitoring area from the time point T-2 to the current time point (T). Can be presumed to exist within.

Then, in the density pattern stack section 11, the density image pattern information corresponding to the moving objects A and B input at the present time and the density image pattern information (which is not judged to be the same as the objects A and B) ( D) is stored. Therefore, when the next image is input, the density image pattern information held in the density pattern stack unit 11 can be used to reliably perform the tracking process for a new moving object. (Modification of this Embodiment) In this embodiment, the density pattern stack unit 11 stores all the density image pattern information input last time (time T-1) and the time T-1 of a moving object having no corresponding destination. The previous and latest density image pattern information is stored.

Here, when the moving object is a human, it is deformed with the movement, and therefore the degree of matching with the density image pattern information at the time point T-1 is not necessarily high. Therefore, in the first modified example, the density pattern stack unit 11 includes the previous pattern (T−T−) including the pattern P1 as shown in FIG.
1) Store all past (TN) density image pattern information before.

In the tracking process, from the last time (T-1) to (T-
The matching process with the density image pattern information of each moving object up to N) is performed, and the matching process with the object having the maximum matching degree is performed. In this association processing, association with the previous (T-1) object is performed (see the flowchart in FIG. 5). In FIG. 5, in the initial setting process of step S50, the number of objects at the present time (T time) m
(Here, m = 2) and the latest number of objects Pn before the last time (T-1) stored in the density pattern stack unit 11 are set. In step S57, step S5
An object corresponding to the maximum matching degree is detected from the matching degrees calculated in 4. Then, in step S58, a process of associating the detected object with the previous (T-1) object is executed.

At this time, a weighting function f with respect to time
By multiplying the matching degree by (T), the priority of the object whose association time is not long is lowered. The weighting function f (T) is "0" at (T-1-N) and (T-
1) Sometimes, a logarithmic function that gradually increases with respect to the time of "1" is more preferable than a monotonically increasing function.

Further, in the first modification, the process shown in the flowchart of FIG. 7 may be used instead of the process shown in the flowchart of FIG. Furthermore, in the second modified example, in the first modified example, from the past last time (T-1) to (T-
Although the collation processing with all the objects up to N) has been executed, the processing shown in the flowchart of FIG. 6 is executed.

That is, in the initialization process, in the N time from the process start (T = 0) to the time point (T = N),
An average value and a variance value of the matching degree of each object are calculated (step S60). Next, the matching process is executed between the density image pattern information of the target object at the present time (T time) and the latest (before T-1) density image pattern information of each object to calculate the matching degree (step S61). ).

From the calculated matching degree, the matching degree included in the range of variance calculated by the initial setting process is extracted, and the object corresponding to this matching degree is detected (step S).
62). Then, the past (T-
The matching process with each density image pattern information from 1) to (TN) is executed, and the matching process with the object having the maximum matching degree among them is executed (step S63). The process of step S63 is similar to the associating process of the first modification.

Further, by removing the oldest matching degree and adding the latest matching degree as data, the average value and the variance value of the matching degree of each object calculated by the initialization process are updated, Is used for the collation processing of (step S6)
4).

[0046]

As described in detail above, according to the present invention, in a monitoring system for tracking a target moving object from a plurality of moving objects existing in a monitoring area, image information of each object before the present time is displayed. By accumulating, the target moving object can be accurately tracked even if intersections or contacts occur between the objects. Therefore, if the monitoring system of the present invention is applied to the premises of buildings, sites, facilities, etc.,
It is possible to reliably monitor intruders and visitors to the premises.

[Brief description of drawings]

FIG. 1 is a block diagram showing a main part of a monitoring system according to an embodiment of the present invention.

FIG. 2 is a flowchart for explaining the operation of this embodiment.

FIG. 3 is a flowchart for explaining the operation of this embodiment.

FIG. 4 is a conceptual diagram for explaining the operation of this embodiment.

FIG. 5 is a flowchart for explaining a first modified example of this embodiment.

FIG. 6 is a flowchart for explaining a second modification example of the present embodiment.

FIG. 7 is a flowchart relating to a first modified example of this embodiment.

[Explanation of symbols]

1 ... CPU, 2 ... Main memory, 3 ... System bus, 4
... image bus, 5 ... camera, 6 ... camera I / F, 7 ... image memory, 8 ... CRT monitor, 9 ... display controller, 1
0 ... Moving object detection unit, 11 ... Density pattern stack unit, 1
2 ... Pattern matching unit, 13 ... Same object correspondence determination unit, 14
... tracking result information holding unit, 15 ... monitoring purpose determining unit.

─────────────────────────────────────────────────── ─── Continuation of the front page (51) Int.Cl. ⁶ Identification code Internal reference number FI technical display location G08B 25/00 510 M

Claims (4)

[Claims] 1. An image input unit for continuously inputting images in a monitoring area, a conversion unit for converting images continuously input by the image input unit into digital image information including density image pattern information, and the conversion unit. Image storage means for storing a plurality of digital image information converted by the means, and a difference image between the background image and the current image input by the image input means in the digital image information stored in the image storage means On the basis of the above, a moving object detecting means for detecting a moving object in the monitoring area, and a density image pattern storage for time-sequentially storing the density image pattern information corresponding to each moving object detected by the moving object detecting means. Means and each of the density image pattern storage means based on the degree of matching between the present time and the density image pattern information before that. The same object correspondence determination means for performing the same object correspondence determination process for specifying the same object in time series for each moving object, and before that having a matching degree that cannot identify the same object with respect to the current density image pattern information The density image pattern information of each of the moving objects is extracted, and the extracted density image pattern information before and the density image pattern information at the present time are stored in the density image pattern storage means; A monitoring system comprising: a tracking unit configured to track the moving object based on a determination result of the object correspondence determination process. 2. The same object correspondence determining means performs a matching process on current density image pattern information and all previous density image pattern information for each moving object, and detects the moving object having the maximum matching degree. The monitoring system according to claim 1, wherein the monitoring system is configured to determine that they are the same object on the series. 3. The same object correspondence determining means calculates in advance a variance value of the matching degree of the same object, and the latest density image pattern information and the current density image pattern among the density image pattern information before that are calculated. It is configured to perform collation processing with information, and to perform collation processing with the density image pattern information before that corresponding to each moving object whose degree of collation falls within the range of the dispersion value. The monitoring system according to claim 2. 4. An image input unit for continuously inputting images in the monitoring area, an image storage unit for storing a plurality of image information continuously input by the image input unit, and an image storage unit for storing the image information. A moving object detection unit that detects a moving object in the monitoring area based on a difference image between a background image and the current image input by the image input unit in the image information, and the moving object detection unit detects the moving object. A density image pattern storage means for time-sequentially storing the density image pattern information corresponding to each of the moving objects, the density image patterns of the present time and the previous density image patterns stored in the density image pattern storage means. A step of reading information, a step of executing a collation process of the current density image pattern information and the previous density image pattern information, Performing the same object correspondence determination process for specifying the same object in time series for each moving object based on the matching degree of the matching process; and specifying the same object for the current density image pattern information. A step of extracting density image pattern information before that having an unmatchable degree for each moving object, and the extracted density image pattern information before that and the density image pattern information at the present time are stored in the density image pattern storage means. A moving object tracking method comprising a step of storing and a step of tracking each moving object based on a determination result of the same object correspondence determination processing.