JP2001325672A - Invader monitoring system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To solve a problem that the luminance of an invader and a background is not fixed all the time in extracting processing of a moving object area, the optimal moving object area can not be acquired when a threshold is fixed or suitable mapping can not be performed when the form of the moving object area includes a large quantity of ruggedness or holes in a method for mutually mapping the moving object areas within a short distance. SOLUTION: Concerning the system for monitoring an invader into a spatial range photographed by a photographing means, this system is provided with a means for calculating a frame difference, which is the difference of image information between a reference frame to become a reference and a current frame to be successively updated, a means for determining a threshold for mask extraction corresponding to the difference of the image information between the reference frame and the current frame each time the current frame is updated, a moving object area deciding means for deciding an area, where the frame difference exceeds the threshold for mask extraction, as a moving object area and an invader deciding means for deciding the invader on the basis of the information of the moving object area.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a technique for analyzing an image captured by a surveillance camera or the like and analyzing the image to obtain an intruder.
The present invention relates to a configuration of an intruder monitoring system capable of detecting an intruder.

[0002]

2. Description of the Related Art An intruder monitoring system generally detects an intruder (moving object) by acquiring an image from a monitoring camera installed in a room, a corridor, a hall, etc. of a building and performing digital analysis processing. It is assumed that.

The analysis procedure is as follows. First, a photographed image in a normal state without an intruder is used as a reference frame.
On the other hand, an image sequentially captured and acquired is set as a current frame, and difference data between the reference frame and the current frame is extracted. The extracted difference data means that a change has occurred in the image information. The extracted difference data is compared with a predetermined fixed threshold value, and the extracted difference data is compared. Acquire the moving object area from the data.

[0004] Next, mapping of a moving object area is performed between consecutive frames. Mapping is a prerequisite process for analyzing the motion of a moving object region by associating moving object regions that are close to each other as the same target object, and determines whether or not the associated moving object region appears continuously. By analyzing the information, the intruder is determined.

[0005]

However, the brightness of the intruder and the background are not always constant in the above-described moving object region extraction processing. There is a problem that an object area cannot be obtained.

Further, in the method of mapping moving object regions that are close to each other, if the shape of the moving object region includes many irregularities or holes, there is a problem that appropriate mapping cannot be performed.

[0007] In addition, if the processing is to determine only whether or not the image data (object) extracted as the moving object area appears continuously (continuous time), erroneous detection of plant or curtain shaking or the like is performed. Tend to.

In view of the above problems, an object of the present invention is to provide an intruder monitoring system that enables optimal acquisition of a moving object area and highly accurate intruder determination.

[0009]

According to a first aspect of the present invention, an intruder enters a space range imaged by the image pickup means based on an image obtained by the image pickup means on a frame-by-frame basis. A frame difference detection unit that calculates a frame difference that is a difference between image information of a reference frame serving as a reference and a current frame that is sequentially updated, among the imaging results of the imaging unit, Threshold value determining means for determining a mask extraction threshold value in accordance with a difference in image information between the reference frame and the current frame every time the current frame is updated; and Moving object determining means for determining a region exceeding the moving object region as a moving object region, and determining an intruder based on information on the moving object region Characterized in that it comprises a subscriber determination means.

According to a second aspect of the present invention, in the intruder monitoring system according to the first aspect, the frame difference detecting means detects a background area for each of the reference frame and the current frame. And, for each of the reference frame and the current frame, normalize respective local image information using an average value of the image information of the background area, thereby obtaining a normalized reference frame and a normalized current frame. A normalization unit; and a unit for obtaining the difference frame as a difference between the normalized reference frame and the normalized current frame.

According to a third aspect of the present invention, in the intruder monitoring system according to the first aspect, the intruder determining means includes a first moving object area in the reference frame and a second moving object area in the current frame. A characteristic amount calculating unit configured to calculate a predetermined type of characteristic amount with respect to the moving object region; and the first and second characteristic amounts based on a correlation between the characteristic amounts with respect to each of the first and second moving object regions. Correspondence determination means for determining the correspondence between the moving object regions, wherein the characteristic amount is at least the first and second movements in addition to the relative distance between the first and second moving object regions. It is characterized by including one of a relative relationship between the space sizes of the object regions and a relative relationship between the optical properties of the first and second moving object regions.

According to a fourth aspect of the present invention, in the intruder monitoring system according to the third aspect, the intruder determining means further specifies a time sequence of each moving object area by the correspondence determining means. And a multi-parameter for determining whether or not the temporal sequence is equivalent to an intruder by using the continuity and straightness of movement of the moving object area as parameters in the temporal sequence. And a type determining means.

According to a fifth aspect of the present invention, there is provided the intruder monitoring system according to any one of the first to fourth aspects, further comprising means for extracting a common background area in a plurality of past consecutive frames. Means for normalizing local image information other than the background area common to the plurality of continuous frames using the image information of the common background area in each frame; and means for normalizing the image other than the normalized common background area. When the variance of the information is lower than a predetermined value, the current frame is updated as a new reference frame.

[0014]

Embodiments of the present invention will be described below with reference to the drawings. The system according to the present embodiment monitors an indoor situation by a fixed surveillance camera attached indoors, and digitally processes image data obtained by the surveillance camera. It is configured as a system that issues
In the following description, the processing procedure of the image monitoring system according to the present invention is described as (1) a moving object area extraction process,
A description will be given of four processes: (2) a moving object region mapping process, (3) an intrusion determination process, and (4) a reference update process.

The overall schematic configuration of the intruder monitoring system will be described with reference to FIG. Image (digital) information captured by the surveillance camera 1 is transmitted to an interface (I
/ F) via the image processing apparatus 2 (comprising a PC or the like)
And sequentially stored in the memory 4 (disk). Among these, the image information in the state where it is determined that there is no intruder is set as a reference frame, and the latest image information acquired every moment is set as a current frame. The local image information in each part of each frame holds the detected luminance of each pixel arranged in a matrix as digital information having a predetermined data length. Further, the following description will be made on the assumption that the initial image information of the reference frame is already stored in the memory 4. And the image processing device 2
Is equipped with an MPU 3, which is configured to analyze captured image information and issue an alarm command to the alarm device 8 via the communication I / F 6.7.

As shown in the overall flowchart of FIG.
First, in step s1, an initial reference frame y0 (shown in FIG. 17A) is read from the memory 4 and preprocessing is performed. The preprocessing is an image smoothing operation performed in general image processing. The preprocessing is performed by dividing each frame into a plurality of blocks and replacing the luminance of pixels belonging to each block with the average luminance value of the pixels belonging to the block. This is a process for removing noise by performing image conversion, and a process for omitting an invalid area at a frame end. The reference frame y preprocessed in block units in this manner
0 is a reference frame y0B (FIG. 17C)
Shown).

Note that, even after such blocking, each block expresses local image information of each part of the frame, and the local area is merely changed from a pixel unit to a pixel block unit. .

Next, a loop process for each current frame that is sequentially updated is started. In the loop processing, first, in step s2, the current frame y1 (FIG. 17)
(Shown in (b)) is read from the monitoring camera 1 or a memory area for buffering the imaging output thereof, and the same preprocessing including smoothing is performed. Then, the current frame y1 preprocessed in the block unit in this manner is set as a current frame y1B (shown in FIG. 17D). At this point, the pre-processed reference frame y0B and current frame y1B are obtained. Then, step s3 is performed.

(1) Moving Object Area Extraction Processing Step s3 will be described with reference to FIGS. Step s3 is a moving object region extraction process.
As shown in FIG. 2, the process includes s31 to s34,
First, in step s31 (= processing procedure sA), a background area is obtained. The processing procedure sA is a background area extraction flow, and as shown in FIG.
It is determined whether or not the luminance is 0 (whether y0 == 0) for each pixel of 0 (step sA1). If y0 == 0, the index value Dy is set to 1.0 (step sA2). If y0 == 0, the value obtained from y1 / y0 is set to the index value Dy (step sA3). ). Therefore, the index value Dy represents the relative value of the pixel value y1 of the current frame with respect to the pixel value y0 of the reference frame for the same pixel. Therefore, if the pixel value y1 of the current frame and the pixel value y0 of the reference frame are the same, the index value Dy is the reference value “1” and the index value D
The deviation of y from the reference value “1” represents the statistical variation of the luminance of the current frame from the luminance of the reference frame for the pixel.

Next, in step sA4, the standard deviation SD of the index value Dy is obtained for each block (for example, 16 × 16 pixels). Since the standard deviation SD indicates the degree of dispersion of the index value Dy in each block, the change in the image information (change in luminance) of the current frame y1 is smaller in the block having the smaller standard deviation SD than in the reference frame y0. Means

Next, a step value (0.005 in this embodiment) is set to another index value SB (step sA5),
The histogram of the standard deviation SD is set to 0.0 every index value SB.
From 1 to 1.0 (step sA6), the cumulative frequency of the histogram is obtained (step sA7). An index exceeding 70%, for example, from the one with the smaller cumulative frequency is set as a critical index idx (step sA8),
The threshold value ST is obtained by multiplying the critical index idx by the index value SB (0.005) (step sA9).
This is based on the experience that up to about 70% of the entire area of the frame is considered to be a background area, and what percentage of the cumulative frequency exceeds the background area may be appropriately changed depending on the environment.

Then, in the loop processing for all the blocks, the standard deviation SD of the block and the threshold value ST
(Step sA10), and the standard deviation S
If D is less than the threshold value ST, the block is determined to be a background area, and the bk flag is set to 1 (step sA11). If the standard deviation SD exceeds the threshold ST, it is determined that the block is not a background area, and bk
The flag is set to 0 (step sA12). With the above processing, the background area in the block unit in the current frame y1 is determined.

In the processing procedure s3 (FIG. 2) again, the reference frame y0B and the current frame y1B
Are normalized, and the frame difference Me is extracted (step s32 (= processing procedure sB)).

The processing procedure sB is a frame difference calculation flow. As shown in FIG. 4, first, average values y0m and y1m of the background area are obtained for each of the reference frame and the current frame (step sB1). The average value y0m is the average value of the luminance of the block determined as the background area (bk flag = 1) in the above-described processing procedure sA in the reference frame y0, and is given by the following equation (1). Where N in Equation 1 is bk
[I] = 1 is the number of i that satisfies 1. The average value y1
m is obtained by replacing y0m with y1m and y0B with y1B in the following equation.

[0025]

(Equation 1)

Next, the reference frame y0B is normalized by dividing by a mean value y0m for each block,
The current frame y1B is normalized by dividing the current frame y1B by the average value y1m for each block, and a frame difference Me which is an absolute value of a difference given by the following equation (2) is obtained (step sB2). The parameter k in Equation 2 means that the frame difference Me is calculated for all the blocks of the reference frame y0B and the current frame y1B. FIG. 18A is an image represented by the frame difference Me.

[0027]

(Equation 2)

In the processing procedure s3 again, the mask extraction threshold value TH is calculated (step s33 (= processing procedure s3).
C)).

The processing procedure sC is a flow for determining a threshold value for extracting a mask. As shown in the flowchart of FIG. 5, a histogram of the frame difference Me is created (step sC).
1), calculate the cumulative frequency (step sC2). next,
Filtering is performed with a fixed threshold Tf, which is an index of what percentage of the cumulative frequency is extracted as a moving object area (step sC3), and a mask extraction threshold TH is obtained (step sC4).

That is, in the curve of the cumulative frequency distribution in which the number of blocks is counted from a small value of the frame difference Me, that is, the block whose luminance variation is small, the slope (the one obtained by applying the differential filter to the cumulative frequency distribution) is the corresponding value. The value of the frame difference Me that reaches the fixed threshold Tf is defined as the mask extraction threshold TH.

Again, in the processing procedure s3 (FIG. 2), the frame difference Me for each block obtained in the processing procedure sB described above and the mask extraction threshold T obtained in the processing procedure sC
By comparing H, a block in which the frame difference Me exceeds the mask extraction threshold TH is determined as a moving object area (hereinafter, appropriately referred to as a mask). FIG. 18B shows image information indicating the extracted mask.

As described above, the means for extracting the background area described in the processing procedure sA, the means for calculating the frame difference described in the processing procedure sB, In the system for determining a moving object area, the mask extraction threshold value is determined according to the luminance difference between the reference frame and the current frame. That is, as described in the processing procedure sC, the mask extraction threshold value is sequentially determined from the histogram of the frame difference Me. Since the mask extraction threshold value TH is determined, it is possible to extract the moving object region more reliably than before, without being affected by changes in the luminance of the intruder or the background.

Further, as described in the processing procedure sB, the normalization is a process of dividing the luminance of each block of the reference frame and the current frame by the average luminance value of the background area. In addition, since the normalization is performed using the average luminance value of only the background area, the moving object area can be stably extracted without being affected by the luminance change such as the flicker of the fluorescent lamp and the change of the sunshine condition.

FIG. 6 shows the above-described moving object region extraction processing in terms of the function of processing data.

(2) Moving Object Region Mapping Process In the overall flowchart, a moving object region mapping process (step s4) is performed. The mapping process is a process of associating the mask of the current frame y1 with the past mask in order to obtain a trajectory of the intruder. As shown in FIG. 7, the distance between masks, the area ratio, the luminance ratio Is evaluated based on

FIG. 8A shows the mask one frame before, and FIG.
(B) is an example showing the mask of the current frame,
Two masks A1 and A3 exist one frame before, and three masks B1, B2 and B3 exist in the current frame.

FIGS. 9A and 9B show the mask Aj (j = 1, 2,...) One frame before and the mask Bi (i = 1, 2,...) Of the current frame, respectively. It is a database representing attributes, and each mask Aj,
Bi has attributes of center coordinates, area, and luminance. This database is stored in the memory 4 in an updatable manner. After giving such attributes to each mask, step s4 (moving object area mapping processing) will be described with reference to the flowchart shown in FIG.

In the mapping process, a loop process is performed on each mask Aj one frame before, and a loop process is performed on each mask Aj for each mask Bi in the current frame. First, in step s41, the distance Dis, the area ratio Ara, and the luminance ratio Yra between the mask Aj one frame before and the mask Bi of the current frame are obtained. The distance Dis is a normal distance between the center (center of gravity) coordinates Aj_c and Bi_c of the masks Aj and Bi, and the area ratio Ara is the area Aj_ of the masks Aj and Bi.
a, Bi_a is a value (ratio) obtained by dividing a small one by a large one, and the luminance ratio Yra is obtained by dividing a small one of the luminances Aj_y, Bi_y of the masks Aj and Bi by a large one. Value (ratio).

Next, in step s42, the distance Dis, the area ratio Ara, and the luminance ratio Yr obtained in step s41 are obtained.
The determination processing is performed for a. In the present embodiment, Ara ≧
0.1, Dis ≦ DL, and Yra> 0.5, that is, the distance between masks is smaller than a preset value DL, the area ratio Ara is 0.1 or more, and the luminance ratio Yra Is greater than or equal to 0.5, mask Aj and mask Bi are flag f
g (i, j) is set to 1 (step s43). On the other hand, if the above condition is not satisfied, it is determined that the masks are not related to each other, and the flag fg (i, j) is set to 0 (step s44).

The area ratio Ara and the luminance ratio Yra
Are, in the present embodiment, 0.1 and 0.5, respectively.
However, the set value is not particularly limited, and is appropriately changed to a value that enables optimal mapping (association).

Next, in step s45, the mask A
The similarity F (i, j) between j and the mask Bi is obtained. The similarity F (i, j) is calculated from the distance between the masks Aj and Bi and the area ratio, and the similarity F increases as the distance decreases and the area ratio increases. The larger the similarity F is, the stronger the relevance of the masks Aj and Bi is.
In general, it is possible to use a similarity function such that the distance is a decreasing function and the area ratio is an increasing function.

After the above processing of steps s41 to s45 has been performed for all the current masks Bi, for the i for which the flag fg (i, j) is set to 1, that is, at step s42, the distance, Among the masks that have been selected as mapping candidates based on the area ratio and the luminance ratio, the similarity F
Is mapped as a mask after the mask Aj has moved (step s46).

By performing such a mapping process,
Optimum mapping processing can be performed as compared with the related art. That is,
Conventionally, simple processing of mapping masks that are close to each other was performed, so if the mask contains many irregularities and holes, the mask's center position fluctuates due to the relative movement of the mask's irregularities and holes. In addition, the reliability of the center-to-center distance of the mask before and after the movement is reduced, and there is a possibility that erroneous recognition may occur. When only the distance is used as a parameter, it is difficult to perform mapping in a state where a plurality of masks are adjacent to each other. Thus, the present invention enables quantitative and highly accurate mapping processing by using three independent parameters (distance, area ratio, and luminance ratio).

Here, the area ratio is one mode of a characteristic quantity expressing the spatial size of the mask, and the luminance ratio is one of the characteristic quantities expressing the optical properties (light reflectance, etc.) of the mask.
Since this is the mode, generally, the mapping of the mask is performed in consideration of the similarity of the space size between the masks and the similarity of the optical properties in addition to the distance between the masks.

The mask mapped in this way is assigned the same number as the mapped past mask. An object assigned a number and recognized as a moving object is called a label. FIG. 18C shows a label one frame before, and FIG. 18D shows a mask after mapping. Masks assigned the same number are displayed with the same luminance.

(3) Intrusion Determination Processing In step s4 of the overall flowchart, a label is generated by assigning a number to the moving object area (mask), and then in step s5, it is determined whether or not the label exists in the frame. . If there is at least one label, that is, if there is a mask associated (mapped) with the past mask, the center of gravity of each label is calculated and the trajectory is determined in step s6, and the time sequence corresponding to the past trajectory is calculated. Is determined on the basis of.

As shown in FIG. 11, the approach determination is made based on the straightness of the movement vector of the mask and the number of continuous movement vectors. This determination processing will be described with reference to the flowchart in FIG. First, in step s61 (= procedure sD), Dm (indicating the straightness of a moving object), Lm (similarly indicating the straightness of a moving object), len (in the past several seconds) (Indicating the number of consecutive labels).

FIG. 13 shows a parameter calculation flow (processing procedure sD). First, in step sD1, the number of past label continuations in a preset time width (for example, 3 seconds) is obtained and set to the number of continuous len. The number of consecutive labels is the number of masks assigned the same number, that is, the number of times the same label appears in past consecutive frames.

Next, in step sD2, a motion vector Vi and a difference vector Di of the motion vector are obtained.
As shown in FIG. 14, the motion vector Vi is a vector connecting the barycentric coordinates Cn (n = 1, 2,...) Of past consecutive labels (FIG. 18E shows a label locus).

Next, in step sD3, the average value VA of the absolute values of the motion vector Vi (that is, the average distance between the centers of gravity of the respective masks) is obtained. Then, a comparison process (step sD4) between the average value VA and the predetermined reference value Ep is performed. The reference value Ep is for canceling the calculation process for a mask having an average small movement amount between adjacent frames before the parameter calculation, and the average value VA is equal to the reference value Ep. If it falls below, the mask is extremely slow moving and is excluded from the determination of an intruder.

If the average value VA is larger than the reference value Ep, the following determination processing is performed in step sD5. Of the motion vectors Vi (i = 1, 2,...)
If the largest absolute value is represented by max (| vi |), a vector whose absolute value is smaller than 0.1 times the maximum absolute value is |
Vi | <max (| Vi |) × 0.1, and the number of motion vectors Vi satisfying such a condition is
For example, 50% of the number of all motion vectors (len-1)
It is determined whether the number is less.

This determination is for excluding the label from the intruder determination when the number of finely moving vectors is considerably large as compared with the maximum amount of movement between adjacent frames. This process prevents erroneous detection of objects such as plants and curtains, which have relatively fast movements but have low-speed movements frequently, and which have a pronounced bias toward the low-speed side in the velocity distribution. is there.

When this condition is satisfied, the index value D
m and Lm are calculated (step sD6). Index value Dm
Is obtained by normalizing the difference vector Di with the average value VA of the absolute values of the motion vectors obtained in step sD3.
Lm is the value obtained by dividing the number smaller than 1.0 by the number (len-2) of all the difference vectors, and Lm is the distance VL from the start point to the end point of the trajectory as the sum of the absolute values of the motion vectors Vi. It is the divided value.

The index value Dm defined as described above is based on the ratio of the difference vector Di occupied by the vector having a small absolute value.
It has a larger value as it moves in a certain direction at a constant speed. On the other hand, the index value Lm is based on the magnitude of the motion vector Vi, and has a larger value as it goes straight.

Then, in step s62 shown in FIG. 12, intrusion determination is performed. Specifically, the above step s61
Using the index values Dm, Lm and the continuous number len obtained in
Perform a comparison process between the index values Dm and Lm and the fixed threshold value,
If the condition is satisfied (for example, Dm ≧ 0.5 and Lm ≧
0.5) If it can be determined that straightness is high and if the past consecutive number len of the same label is a fixed number, it is determined that the intrusion determination condition is satisfied, and 1 is set to the ALARM flag (step s63). . If the conditions are not met, A
The LARM flag is set to 0 (step s64).

After the above-described intrusion determination processing is completed, in step s7 of the basic flowchart, it is determined whether or not to output an alarm, that is, whether or not the ALARM flag is set to 1 is determined. If so, an alarm is output (step s8).

As described above, in this embodiment, in addition to the continuity of the moving object area, the non-mixing of fibrillation (step s)
D5), uniform velocity (Dm), and straightness (Lm) are used as parameters for intrusion determination.
It is possible to prevent erroneous detection and perform highly accurate intruder determination. In particular, in the past, there was a tendency for erroneous detection of swaying of plants and curtains, etc., but according to this algorithm, a moving object area that does not move straight ahead is not judged as an intruder, so the intruder The accuracy of the judgment was improved.

If the ALARM flag is 0, it means that there is no intruder in a normal state, and the processing shifts to reference update processing.

(4) Reference Update Processing Since the frame input from the monitoring camera is affected by changes in the sunshine, the brightness is not always constant. Therefore, the above-described moving object region extraction processing can be performed more appropriately by updating the reference frame as appropriate according to a change in the environment. Further, in the method of detecting a luminance change based on a frame difference between the reference frame and the current frame, it is assumed that the reference frame includes only a background image that does not include an intruder. Therefore,
It is desirable to stably determine a frame that does not include an intruder and update the reference frame.

FIG. 15 shows a reference update processing flow. First, the past M frames including the current frame from which the moving object area (mask) has been extracted are acquired (step s91). Here, the number M is a predetermined integer of 2 or more. Next, as shown in FIG. 16A, a common background area is extracted from the acquired M frames (step s92).

Here, the common background region is a region corresponding to the logical product of the background regions of the M frames, and is a region belonging to the background region in any of the M frames.

As shown in FIG. 16B, similarly to the above-described normalization processing (step sB2 shown in FIG. 4), the M frames are respectively subjected to the luminance average value y of the background area.
Normalization processing (step s93) is performed by dividing by 1m, y2m,..., YMm.

Next, as shown in FIG. 16C, calculation processing of the luminance variance for each block is performed in the time direction. That is, in an area other than the common background normalized in step s93, between past M consecutive frames,
The luminance variance is obtained for the same set of blocks (step s94). Since the common background area is considered to have a small variation with respect to other areas and a small variance value, the calculation can be omitted.

If the luminance variance of the common block (each block belonging to the area other than the common background) does not have a variance value exceeding the fixed threshold value, it is determined that frames without fluctuation are continuous (step s95). ), The reference frame is replaced by the current frame, and the reference updating process is performed (the whole flowchart step s1).
0).

As described above, since the normalization is performed using the average luminance value of the common background region in a plurality of past frames, the reference frame is not updated because the luminance has changed even though there is no intruder. Thus, it is possible to stably determine a frame that does not include an intruder and update the reference frame. Further, since the luminance variance in the time direction is obtained only in the region other than the common background, the calculation amount can be reduced as compared with the case where the entire region is targeted.

As image information, other information such as the size of a specific color component can be used in addition to the luminance.

Further, in performing the mask mapping, for each moving object region of the reference frame and the current frame, in addition to the relative distance, one of the relative relationship of the space size and the relative relationship of the optical properties is determined. Just including it can perform more appropriate mapping than before.

In determining whether or not the moving object area is an intruder, in addition to the continuity of the movement, the degree of deviation to the lower speed side in the speed distribution of the movement, the uniform speed and the straightness Even if at least one is included, a more appropriate intruder determination than before can be performed.

[0069]

As described above, according to the first aspect of the present invention, there is provided a system for monitoring an intruder in a space range imaged by the image capturing means based on an image obtained in frame units by the image capturing means. A frame difference detection unit that calculates a frame difference that is a difference between image information between a reference frame serving as a reference and a sequentially updated current frame in the imaging result of the imaging unit; A threshold value determining unit that determines a mask extraction threshold value in accordance with a difference between image information between the reference frame and the current frame;
Since there are moving object determining means for determining an area where the frame difference exceeds the mask extraction threshold value as a moving object area, and intruder determining means for determining an intruder based on information on the moving object area, In addition, the moving object region can be reliably extracted without being affected by the change of the luminance of the intruder or the background.

According to a second aspect of the present invention, in the intruder monitoring system according to the first aspect, the frame difference detecting means detects a background area for each of the reference frame and the current frame. And, for each of the reference frame and the current frame, normalize respective local image information using an average value of the image information of the background area, thereby obtaining a normalized reference frame and a normalized current frame. Normalization means, and means for obtaining the difference frame as a difference between the normalized reference frame and the normalized current frame, so that it is not affected by luminance changes such as flicker of fluorescent lamps and changes in sunlight conditions. It is possible to extract a moving object region.

According to a third aspect of the present invention, in the intruder monitoring system according to the first aspect, the intruder determining means includes a first moving object area in the reference frame and a second moving object area in the current frame. A characteristic amount calculating unit configured to calculate a predetermined type of characteristic amount with respect to the moving object region; and the first and second characteristic amounts based on a correlation between the characteristic amounts with respect to each of the first and second moving object regions. Correspondence determination means for determining the correspondence between the moving object regions, wherein the characteristic amount is at least the first and second movements in addition to the relative distance between the first and second moving object regions. Since it includes one of the relative relationship between the space sizes of the object regions and the relative relationship between the optical properties of the first and second moving object regions, a quantitative and highly accurate mapping process is performed. It can become.

According to a fourth aspect of the present invention, in the intruder monitoring system according to the third aspect, the intruder determining means further specifies a time sequence of each moving object area by the correspondence determining means. And a multi-parameter for determining whether or not the temporal sequence is equivalent to an intruder by using the continuity and straightness of movement of the moving object area as parameters in the temporal sequence. With the provision of the type determination means, it is possible to prevent erroneous detection and perform highly accurate intruder determination. In particular, in the past, there was a tendency to erroneously detect swaying of plants and curtains, etc., but according to this algorithm, it makes a random movement and does not judge a moving object area without straightness as an intruder,
The accuracy of intruder determination can be improved.

According to a fifth aspect of the present invention, there is provided the intruder monitoring system according to any one of the first to fourth aspects, further comprising means for extracting a common background area in a plurality of past consecutive frames. Means for normalizing local image information other than the background area common to the plurality of continuous frames by using the image information of the common background area in each frame; and image information other than the normalized common background area. When the variance is smaller than a predetermined value, the current frame is updated as a new reference frame. This solves the problem that the reference frame is not updated because a luminance change has occurred even though there is no intruder. did it. Further, it is possible to stably determine a frame that does not include an intruder and update the reference frame. Furthermore, since the luminance variance in the time direction is obtained only in the common background area, the amount of calculation can be reduced and the processing efficiency can be improved as compared with the case where the entire area is targeted.

[Brief description of the drawings]

FIG. 1 is an overall flowchart of an intruder monitoring system according to the present invention.

FIG. 2 is a diagram showing a flow of a moving object area extraction process.

FIG. 3 is a diagram illustrating a flow of a background region extraction process.

FIG. 4 is a diagram showing a frame difference extraction processing flow.

FIG. 5 is a diagram showing a mask extraction threshold value determination flow.

FIG. 6 is a diagram illustrating a unit that operates until a moving object region is extracted from a reference frame and a current frame that are input data.

FIG. 7 is a diagram showing elements used for mapping a moving object area.

FIG. 8A is an embodiment diagram showing a mask of one frame before, and FIG. 8B is a diagram showing a mask of a current frame.

FIG. 9A shows attribute data of a mask one frame before,
(B) is a diagram showing the attributes of the mask of the current frame.

FIG. 10 is a diagram showing a mapping processing flow.

FIG. 11 is a diagram showing elements used for intrusion determination processing.

FIG. 12 is a diagram illustrating a flow of an intrusion determination process.

FIG. 13 is a diagram showing a parameter calculation flow.

FIG. 14 is a diagram illustrating a moving state of a mask in a past continuous frame.

FIG. 15 is a diagram showing a reference update determination processing flow.

FIG. 16 is a diagram illustrating image processing performed on a common background area in reference update determination processing.

FIG. 17 is a diagram showing an image of an input reference frame and a current frame and a state after preprocessing in the intrusion monitoring system according to the present invention.

FIG. 18 is a diagram showing a frame difference, a moving object area, and the like generated in the intrusion monitoring system according to the present invention.

FIG. 19 is an overall view of an intruder monitoring system.

[Explanation of symbols]

 Me Frame difference TH Mask extraction threshold value y0 Reference frame y0B Luminance in block units of reference frame y1 Current frame y1B Luminance in block units of current frame

Continued on front page F-term (reference) 5B057 AA19 BA30 DA08 DA15 DB02 DB09 DC03 DC06 DC08 DC14 DC19 DC33 DC36 5C054 AA01 CC03 CG06 FC12 FF02 FF06 HA18 5C084 AA02 AA07 AA14 BB05 BB32 CC17 DD11 EE01 GG41 GG6 GG43 GG41 GG43 GG43 GG43 GG41 DA03 EA12 FA32 FA33 FA35 FA52 FA59 FA60 FA62 FA66 FA67 GA08 GA10 GA19 GA34 GA51

Claims (5)

[Claims] 1. A system for monitoring an intruder in a space range imaged by an image capturing means based on an image obtained in frame units by an image capturing means, wherein: A frame difference detecting means for calculating a frame difference that is a difference between image information of a reference frame serving as a reference and a sequentially updated current frame; (b) for each update of the current frame, the reference frame and the current Threshold value determining means for determining a mask extraction threshold value according to the difference between the image information and the frame; and (c) determining a region where the frame difference exceeds the mask extraction threshold value as a moving object region. An intruder comprising: a moving object determining unit; and (d) an intruder determining unit that determines an intruder based on the information of the moving object region. Monitoring system. 2. The intruder monitoring system according to claim 1, wherein the frame difference detecting means includes: (a-1) a background detecting means for detecting a background area for each of the reference frame and the current frame; a-1) For each of the reference frame and the current frame, normalize each local image information using an average value of the image information of the background region,
Normalizing means for obtaining a normalized reference frame and a normalized current frame thereby; and (a-3) means for obtaining the difference frame as a difference between the normalized reference frame and the normalized current frame. Intruder monitoring system characterized by the following. 3. The intruder monitoring system according to claim 1, wherein the intruder determining means comprises: (d-1) a first moving object area in the reference frame and a second moving object area in the current frame. And (d-2) the first and second moving object regions based on a mutual relationship between the characteristic amounts. A correspondence determination unit that determines a correspondence between the second moving object regions, wherein the characteristic amount is at least the first and second in addition to the relative distance between the first and second moving object regions. An intruder surveillance system comprising: one of: a relative relationship between spatial sizes of two moving object regions; and a relative relationship between optical properties of the first and second moving object regions. 4. The intruder monitoring system according to claim 3, wherein said intruder determination means further comprises: (d-3) a chain specification for specifying a time chain of each moving object area by said correspondence relation determination means. Means, (d-4) in the time sequence, using the continuity and straightness of movement of the moving object area as parameters, to determine whether or not the time sequence is equivalent to an intruder. And a multi-parameter type determining means. 5. The intruder monitoring system according to claim 1, further comprising: (e) means for extracting a common background area in a plurality of past consecutive frames; Means for normalizing local image information other than the background area common to a plurality of continuous frames using the image information of the common background area in each frame, and (g) means other than the normalized common background area other than the common background area. When the variance of the image information falls below a predetermined value, the intruder monitoring system updates the current frame as a new reference frame.