JPH07320063A - Image monitor device - Google Patents

Abstract

PURPOSE:To improve the performance at the time of body extraction, to reduce an erroneous report and misdetection, and to enable stable monitoring by updating the background following up rough variation due to variation in illuminance, etc. CONSTITUTION:This device is equipped with a variation direction memory (T)6 storing the direction of variation in density by pixels between time (T-DELTAt) and time T, a variation direction memory (T-DELTAT)7 storing the direction of variation in density by pixels between time (T-2DELTAt) and time (T-DELTAT), and a cursor size memory 8 wherein widths of video noises by pixels of the background are set in every partial area. And, a background update means 9 judges whether or not the background can be updated by the pixels from the variation area image from the a memory 5, the directions of density variation from the memories 7 and 8, and the video noise information from the cursor size memory 8, substitutes a monitor image value for the background image as to the judged pixels of the background which can be updated, and stores the result in a memory 10. At next time (T+Q), difference binarization is performed with this background image.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image monitoring apparatus for monitoring a dangerous area or the like by using an image picked up by a television camera or the like, detecting an object to be monitored or a specific operation, and outputting an alarm.

[0002]

2. Description of the Related Art Conventionally, in an image monitoring apparatus using an image obtained by an image pickup means such as a television camera, a difference between a background image previously created for detecting an intruding object and a monitoring image is calculated. However, a technique for detecting an intruding object or an abnormal state in a monitoring image by utilizing a changed portion of the image or a feature amount of movement is known.

Even with such a method, if there is no change in the background image, it is possible to correctly detect an intruding object or an abnormal state. However, if there is an object with a temporal change in the background (such as cloud movement or sway of trees), or if there is a change in the background itself (change in sunshine, lighting inside or outside the room, etc.), the background image and monitoring are performed. Difference between the image and the binarized image may cause fluctuations in the background other than the target to be detected, and may give a false alarm. If the threshold value is increased in order to avoid this, the monitoring target is not well extracted, and the detection error of the monitoring target object occurs. Due to these, there is a problem that the reliability of the system is lowered.

In order to avoid such problems, for example, the following methods (1) and (2) have been proposed. (1) When it is detected that the background image has changed so that the target object can be reliably extracted from the difference / binarized image, the background image is updated at an appropriate timing. As a method of detecting a change in the background image, the duration of the value is counted for each pixel, and when the duration exceeds a certain threshold value, the background of the pixel is updated (Japanese Patent Laid-Open No. 4-1110).
79). (2) Difference between background image and monitor image ・ For all pixels not extracted as change regions in the binarized image, the background image is newly updated with the monitor image (Japanese Patent Laid-Open No. 5-120432).
See the bulletin).

[0005]

However, the above methods (1) and (2) have the following problems. In the method (1), when the background image is updated, the change in the image is monitored on a pixel-by-pixel basis. Therefore, the background image is not updated until the change in each pixel from the initial background image continues for a certain period or longer. For this reason, there is a period in which it is possible to follow a gradual scene change but cannot follow a sudden background change.

Further, since the background is updated for each pixel depending on whether the change for each pixel is within a certain period of time, even if the same object in the background is subject to constant fluctuations (such as swaying of trees) and video noise, updating is performed. Pixels that are updated and pixels that are not updated may coexist, and pixels that have not been updated may be detected as a change area thereafter. In order to avoid this, it is sufficient to increase the threshold value for considering continuous change, but if this is done, conversely the extraction accuracy of the target object will decrease.

According to the method (2), all the portions that are not extracted as change areas in the monitor image are replaced with the background image. Therefore, the background image is updated quickly, but the background still contains steady fluctuations and video level fluctuations. Therefore, in order to eliminate these effects,
It is necessary to set a large threshold value for the difference / binarization. However, this also reduces the extraction accuracy of the target object. Therefore, the problem of the present invention does not reflect small amplitude fluctuations such as fluctuations in video level during fluctuations in surveillance images,
The purpose is to enable background updates that follow rough background illuminance changes.

[0008]

In order to solve such a problem, according to the present invention, an image pickup means for picking up an image of a monitoring area, a monitoring image memory for sequentially storing the imaged image data, and a background for storing a background image. An image memory, a changing area extracting means for extracting a changing area of a time-series monitoring image and a background image, a changing area image memory for storing the changing area image, and a size and shape of each pattern of the changing area image are displayed. A target object extracting means for extracting a monitoring target object by a feature amount including a motion amount extracting means for extracting a motion of the target object;
In an image monitoring device that issues an alarm stepwise depending on a target object and its position and movement, an initial background image creating means for creating an initial background image in a state where the target object is not on the screen, and the initial background is monitored as a background image. And for each pixel of the part not extracted as a change area, a background pixel updating means for setting a value obtained by smoothing (hysteresis smoothing) with a hysteresis characteristic in the time direction as a new background pixel value, Even if the background fluctuates due to changes in illuminance, the fluctuation of small amplitude due to video noise is not reflected, and the background can be updated to follow the original background fluctuation, and stable monitoring can be performed.

In contrast to the above invention, the cursor size when performing the above-mentioned hysteresis smoothing is adjusted according to the steady fluctuation depending on the background position, and the fluctuation level of the video level for each partial area according to the density level and the density gradient. A setting means for setting is provided to absorb the change in the magnitude of the change in the video level depending on the position, enable the background update to more faithfully follow the background change at each place, and perform stable monitoring.

[0010]

Function: The direction of time-dependent change in density is stored for each pixel, and the following processing is performed on the pixel that is not extracted as a change region. (A) When the changing direction of the density is continuous, the background image of the pixel is replaced with the pixel value of the monitoring image, and the background is updated. (B) If the changing direction of the density is discontinuous and the pixel value is within the cursor, the current background is held. (C) If the density change direction is discontinuous and the pixel value reaches the upper or lower edge of the cursor, replace the background image with the pixel value of the monitoring image and update the background. select.

According to the above method, the pixels which are not extracted as the change area due to the time-series density change of each pixel are used as the background with the result of smoothing with the hysteresis characteristic, while the pixels belonging to the change area are selected. On the other hand, it means holding the background pixel value. The cursor size in this case is usually set to the upper limit of the video noise, but the background may not be uniform or there may be constant fluctuations (such as swaying of trees). There are places that can be kept relatively small and places where it can be made large. Therefore, the cursor size is set for each partial area according to the degree of video noise depending on the location by examining the fluctuation of the image for a certain period, and as described above, the background update can be performed more reliably.

[0012]

1 is a block diagram showing an embodiment of the present invention. In the figure, 1 is an image pickup means such as a television camera (here, an A / D converter is also included), 2 is a monitor image memory for storing an image input at time T by the image pickup means 1, and 3 is a background image memory. Reference numeral 4 is a difference / binarization means for performing difference / binarization between the monitor image and the background image, and 5 is a change image memory for storing an output image from the difference / binarization means.

A background update block A is composed of the following parts. 6 is a change direction memory (T) which stores the change direction of the density of each pixel between time (T-Δt) and time T, and 7 is similarly a pixel between time (T-2Δt) and time (T-Δt). Change direction memory (T-Δ
T) and 8 are cursor size memories in which the width of video noise for each pixel of the background is set for each partial area.

Numeral 9 is a background updating means which is capable of updating the background for each pixel from the change area image from the memory 5, the direction of density change from the memories 7 and 8 and the video noise information from the cursor size memory 8. It is determined whether or not the background image is replaced with the monitored pixel value for the pixel determined to be background updatable and stored in the memory 10. The next time (T +
In 1), the difference / binarization is performed using this background image.

Reference numeral B denotes a determination block, which includes an object extracting means 11 for extracting a monitored object from the change area image from the memory 5, a motion extracting means 12 for detecting the motion of the extracted monitored object, and each of these means. Based on the output of the monitoring result determination means 13 for comprehensively determining the alarm level, the number of objects, the position, the type of motion, etc., and outputting a stepwise alarm.
Etc.

Here, the hysteresis smoothing will be described. FIG. 2A is an explanatory diagram showing a general example thereof. The solid line indicates the input signal, the dotted line indicates the smoothed output signal (smoothed signal), and the rectangle indicates the cursor. That is, in this example, first, the signal is updated so that the upper end of the cursor coincides with the input signal, and the signal is held when the changing direction of the input signal temporarily changes. Also, if the input signal comes to the bottom of the cursor, then
The signal is updated so that the bottom edge of the cursor matches the input signal. By this processing, it is possible to remove the small amplitude component from the input signal and obtain a signal in which only the large amplitude component remains.

FIG. 2B shows an example of smoothing according to the present invention. Each symbol is the same as in FIG. That is, since the fluctuation due to the video noise appears in both "+" and "-", the output signal is updated in such a manner that the center of the cursor coincides with the input signal. Since the changing direction of the input signal changes almost every frame due to the video noise, the changing direction of the input signal at the time separated by Δt can be checked by smoothing the input signal by means such as moving average in a short time. Is also good.

The direction of time-dependent change in density is stored for each pixel. When the change direction of density is continuous, the background image is replaced with the corresponding pixel of the input image to update the background.
When the change in density is discontinuous, the background is held until the actual pixel value reaches the upper or lower end of the cursor. When the corresponding pixel value reaches the upper or lower end of the cursor and the corresponding pixel is not extracted as the change area, it is possible to select whether or not to replace the background image with the pixel value of the monitoring image for the corresponding pixel. Here, the binarization threshold value for extracting the changed area may be the same as the cursor size or a value slightly larger than the cursor size. Further, the change direction of the input signal may be judged not only by the change of 2Δt but also by the change of N · Δt for the continuity of the change.

FIG. 3 is a flow chart showing the smoothing procedure according to the present invention. (S1) Output signal O (x, y, T-1) of the previous frame
And the difference value of the input signal I (x, y, T) exceeds the binarization level Th and is detected as a change area, the output signal O (x, y, T) is the value O of the previous frame. (X, y, T
-1) is retained (see route). (S2) The change dI (x, y, T-Δt) / dt of the input signal from T-2Δt to T-Δt, not in the change region,
Change in input signal from T-Δt to T dI (x, y,
When the sign of T) / dt is the same, the center of the cursor coincides with the input signal I (x, y, T) and the output signal O (x, y, T).
T) is updated (see route).

(S3) Not the change region, but dI (x, y,
If the signs of T-Δt) / dt and dI (x, y, T) / dt are different and the input signal does not exceed the upper and lower ends of the cursor, the output signal O (x, y, T ) Holds the value O (x, y, T-1) of the previous frame (see route). (S4) dI (x, y, T-Δt) /
If the signs of dt and dI (x, y, T) / dt are different, and if the input signal exceeds the upper and lower ends of the cursor, the output signal O (x, y, T) is input. Signal I
Update with (x, y, T), or not, is selected depending on the case (see route).

Next, a method of setting the cursor size will be described. With the above method, the cursor size is generally
Set to the upper limit of the amplitude of the small amplitude signal you want to remove. In the present invention, since the target is an image captured by a television camera or the like, when the image has a uniform background, the level of video noise (for example, about ± 5 levels) may be set to the cursor size.

However, in a general environment, there is a change in the background as well, especially where there is a steady fluctuation such as swaying of trees, or where there is a large concentration gradient (for example, a white line appears on a black background). (Part), it becomes necessary to consider noise up to a considerably large amplitude. By the way, if the maximum value and the minimum value are obtained for the background image within a fixed time in the state where the background has no change in illuminance, the magnitude of the video noise at each background location can be almost known. With this in mind, the cursor size is set for each partial area.

FIG. 4 shows a concrete example. FIG. 10A shows an example of a background image, and FIG. 9B shows the amplitude of video noise in the background image. The brighter the area, the more video noise. In this example, it can be seen that the video noise at the tree portion and the edge portion of the white line is large. In the conventional method,
A method may be considered in which a masking process is performed on a part such as a sway of a tree and the part is not monitored. However, according to the present invention, the monitoring performance is lower than that of a part without a change, but it is large. Changes in concentration level can be detected.

FIG. 5 shows a case in which a portion having a large change in video level is gradually darkened and the background is updated by the methods described as the conventional methods (1) and (2) respectively (a) ) And (b) are shown in comparison with (c) when the background is updated by the method according to the present invention. In order to simplify the explanation, in the method according to the present invention, the cursor size = binarization level,
The value levels are the same.

In FIG. 5 (a), in the part where the video level fluctuation is large, the fluctuation within a fixed time is large, so that the background is not updated easily, and this part is different from the other regions with the same binarization threshold value. -When binarized, the change area appears even if there is no target object when the background changes.

In FIG. 5B, the background is updated for all the pixels which are not detected as the change area, but when the change occurs abruptly and is once detected as the change area,
The background is not updated for this portion, and it is continuously detected as a changed area.

In FIG. 5 (c), the background is updated when the direction of change of the background changes continuously, so the change of the noise of small amplitude is not reflected, and the background changes in accordance with the rough change of the background. Has been updated. Further, in this range, the fluctuation of the background is not detected as a changed area. That is, in order to prevent the change regions from being erroneously extracted by the conventional methods (1) and (2) of FIGS. 5A and 5B, the threshold value for the difference / binarization is increased. However, it is obvious that this reduces the extraction accuracy of the target object.

[0028]

According to the present invention, a small change due to video noise is not reflected in the background change, and the background is updated by following a rough change due to a change in illuminance, etc. The advantage is that monitoring is possible. In addition, since the background image includes rough fluctuations of the background, the performance at the time of object extraction is improved, false alarms and detection errors are reduced, and stable monitoring is possible.

[Brief description of drawings]

FIG. 1 is a configuration diagram showing an embodiment of the present invention.

FIG. 2 is an explanatory diagram for explaining hysteresis smoothing.

FIG. 3 is a flowchart showing a smoothing procedure according to the present invention.

FIG. 4 is an explanatory diagram illustrating an example of the magnitude of a background image and video noise.

FIG. 5 is an explanatory diagram for explaining an example of background updating by comparing a conventional method and a method according to the present invention.

[Explanation of symbols]

1 ... Imaging means, 2 ... Monitoring image memory, 3 ... Background image memory, 4 ... Difference / binarization means, 5 ... Change image memory, 6 ...
Density change direction memory (T), 7 ... Density change direction memory (T-Δt), 8 ... Cursor size memory, 9 ... Background updating means, 10 ... New background image memory, 11 ... Object extracting means, 12 ... Motion extracting means , 13 ... Monitoring result determination means, A
... background update block, B ... decision block.

Claims (2)

[Claims] 1. An image pickup means for picking up an image of a monitoring area, a monitoring image memory for sequentially storing picked-up image data, a background image memory for storing a background image, and a change area of a time-series monitoring image and background image. Change area extracting means for extracting, change area image memory for storing the change area image, target object extracting means for extracting the monitoring target object by the feature amount including the size and shape of each pattern of the change area image, and the target In an image monitoring apparatus that includes a motion extracting unit that extracts the motion of an object, and that generates an alarm stepwise by the target object and its position and motion, an initial background that creates an initial background image in the state where the target object is not on the screen. By monitoring the image creating means and its initial background as the background image, each pixel of the part not extracted as the change area has a hysteresis characteristic in the time direction. By providing a background pixel updating unit that uses the smoothed (hysteresis smoothed) value as a new background pixel value, even if the background changes due to a change in illuminance, a small amplitude change due to video noise is not reflected, An image monitoring device that is capable of updating the background to follow the original background fluctuation and performing stable monitoring. 2. A setting for setting a cursor size when performing the hysteresis smoothing in accordance with a steady fluctuation depending on a background position, and a fluctuation level of a video level for each partial area according to a density level and a density gradient. A means is provided to absorb the change in the magnitude of the change in the video level depending on the position
The image monitoring apparatus according to claim 1, wherein the background is updated so as to more faithfully follow the background variation for each place, and the monitoring is performed stably.