JP3034101B2 - Method and apparatus for identification using motion vector - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method and an apparatus for discriminating motion vectors, and more particularly to a method and an apparatus for discriminating motion vectors suitable for monitoring unauthorized entry into a restricted area.

[0002]

2. Description of the Related Art FIG. 15 shows a conventional method for identifying an object to be monitored based on images used for monitoring unauthorized entry into a restricted area.
Shown in Images I of the monitored area are sequentially captured from an image input unit 50 such as a TV camera. In a difference image extraction unit 51, a reference image 52 in which a monitoring area when there is no monitoring target is stored in advance.
The difference between the image and the image I is extracted by subtraction. This difference image represents a change of each input image I with respect to the reference image 52 as a luminance change. The brightness change binarization unit 53 binarizes the difference image based on a predetermined threshold value in order to prepare for noise suppression and labeling in the subsequent object identification unit 54. After labeling each connected component of the binarized image, the target identifying unit 54 identifies whether or not each connected component is a monitoring target based on its shape feature. This identification result is given to the abnormality determination unit 55, and is immediately determined as abnormal.
Alternatively, it is determined that there is an abnormality after confirming that the monitored object exists in the restricted area image Iz stored in the restricted area image storage unit 56 in advance.

[0003]

However, the above-mentioned conventional apparatus has two problems: an error due to a change in illuminance and a shape error. The shape error is that when the illuminance of the monitoring area changes due to a change in solar radiation or artificial lighting, the brightness of the image slightly changes, and an error occurs in the difference image. In some cases, an erroneous determination that an object to be monitored exists even though the object does not actually exist may occur. Shape error, the shape of the object in the difference image is the illuminance change of the environment,
This is an error that does not clearly represent the actual shape of the object due to lack of brightness difference between the object and the background, lack of contrast of the object itself, and the like. It is often divided and extracted. This shape error tends to cause an erroneous determination that there is no monitoring target even though it actually exists.

[0004] Both of the above two problems occur because the luminance change itself shown in the difference image is directly binarized.

The present inventor has noted that the above problem caused by the binarization of the luminance change itself can be solved by focusing on the movement of the monitored object and using this as a reference for identification. Accordingly, it is an object of the present invention to provide a method and apparatus for identifying a monitored object by its movement.

[0006]

Means for Solving the Problems The embodiment of FIG. 1 and FIGS.
Referring to FIG. 3, the identification method using a motion vector according to the present invention divides a frame F of an observation area image continuously generated by, for example, a TV camera into blocks B having a fixed number of pixels (mxn); replacing each block in the image in the current image I _t, the motion vector V from the portion corresponding to the block in the image in the motion vector V that is, before the image I _(t-1) to the block position in the current image I _t To output a motion vector image Iv (see FIG. 6); and binarize the motion vector V to convert the motion vector image Iv to a binary image Ib.
i (see FIG. 7); removing the minute connected components in the binary image Ibi by performing contraction image processing and diffusion image processing on the binary image Ibi; each connection in the binary image Ibi after the above image processing For the component, a configuration that identifies the monitoring target in the observation area by comparing the shape characteristic of the component and the motion characteristic of the pre-transformation motion vector of the connected component with the shape characteristic and the motion characteristic of the predetermined monitoring target. Used.

[0007] An apparatus for performing the above method according to the present invention comprises:
The frame F of the observation area continuous images generated is divided into blocks B of a certain number of pixels, corresponding to the block image in each block Bt in the image in the current image I _t, the previous image I _(t-1) the motion vector image Iv by binarizing the motion vector V; motion vector image forming means 1 moves by replacing the vector V and outputs a motion vector image Iv to the block position in the current image I _t of the portion The binary image Ibi
Binarizing means 2 for converting into a binary image; and removing minute connected components by contraction image processing and diffusion image processing, and converting the shape characteristics of each connected component in the binary image Ibi and the motion characteristics of the pre-conversion motion vector of the connected component. A configuration including identification means 3 for identifying a monitoring target in the observation area by comparing the shape characteristics and the movement characteristics of the predetermined monitoring target is used.

[0008] Preferably, the binarization of the motion vector V is performed based on a predetermined threshold value. The present invention further provides a monitoring device for detecting that a monitored object identified by the above method is in a restricted area, the monitoring device for detecting that a predicted position of the monitored object is in a restricted area, and a monitored object. The above-mentioned monitoring apparatus of a system for correcting the identification of the image with respect to the image blur is provided.

[0009]

The motion vector V, which is an important component of the present invention, is obtained by a so-called block matching method. According to this method, as shown in FIG. 2, a frame F of an image is divided into blocks B each having a predetermined number of pixels, for example, m pixels in the horizontal direction and n pixels in the vertical direction, and as shown in FIG. frame F of each block Bt of frame Ft of _t the previous image _{I (t-1) (t} -1) or come from the position of the throat, for each block Bt of the current picture I _t previous image I _{(t-1 The} search is performed by pattern matching with a certain corresponding region of ( ₎₎ . By this search, the current image
Obtaining a motion vector V (see FIG. 6) for each block Bt of I _t to the previous image I _(t-1).

If the frame F has x pixels in the horizontal direction and y pixels in the vertical direction, it is assumed that the frame F is composed of (x / m) blocks in the horizontal direction and (y / n) blocks in the vertical direction. Can be represented. In the following description, the coordinates X and Y of the block coordinate system when the coordinates of the pixel coordinate system are represented by x and y are defined as follows. X = (x / m), Y = (y / m) (1)

[0011] To extract the motion vector V is in the motion vector image forming unit 1 in FIG. 1, prior to a particular block Bt of the current picture I _t to be obtained motion vector V image I
_(t-1) is sequentially moved one pixel at a time in the horizontal and vertical directions as indicated by the arrows in FIG. Every time one pixel is moved, an image difference between the specific image block Bt and a portion overlapping with the specific image block Bt in the previous image I _(t-1) is detected, and all the pixels in the specific image block Bt are detected. , The sum of the absolute values of the image differences, that is, the sum of the absolute differences E, is stored together with the movement amount (i, j) of the previous image I _(t-1) at that time. FIG. 3 (b)
Previous image I _(t-1) in horizontal direction from -μ to μ (-μ ≦ i <μ)
Move one pixel at a time, from -ν to ν in the vertical direction (-ν ≦
j <ν) indicates that the pixel has been moved one pixel at a time. The absolute difference sum E in this case can be expressed by the following equation.

[0012]

(Equation 1) (2) The 2 μx2ν absolute difference sums E calculated for each block Bt are stored for each block Bt in this manner, and the movement amount (i, i) corresponding to the minimum value in the absolute difference sums E is stored.
j) is a motion vector V. Here, the movement amount (i, j) indicates the movement of i pixels in the horizontal direction and j pixels in the vertical direction. In short, the motion vector V corresponds to the movement amount of the previous image that gives the minimum value of the absolute difference sum E.

According to the present invention, for example, the observation area image shown in FIG. 5 is captured not as the luminance of the image component but as the motion vector V of each block constituting the image. However, the size of the block is not limited to the illustrated example, and can be arbitrarily selected. In FIG. 5, it is assumed that a human being a monitoring target is moving rightward with a small animal with a shadow at the foot, and there is a grove to the left of the surveillance, which is swaying by a rightward wind.

The binarizing means 2 shown in FIG.
Iv is converted to a binary image Ibi, which is for performing image processing. The discriminating means 3 in the illustrated example first performs the contracted image processing on the binary image Ibi by the contracted diffusion means 20, converts it to, for example, the image Ibc in FIG. 8, and further performs the diffused image processing to obtain the image in FIG. Convert to Ibe. These contraction / diffusion image processing removes a small portion and a narrow portion in the binary image Ibi, as is apparent from a comparison between FIG. 6 and FIG. For example, it is possible to remove a small portion in an image due to noise, and a small portion or a narrow portion in an image for a minute object that is not monitored in the observation area. The connected components after the contraction / diffusion image processing are distinguished from each other by labeling as shown in FIG. The labeling means 22 in FIG. 1 is for this purpose.

FIG. 1 shows an example of the configuration of an identification process following image processing and labeling, and FIG. In step 101, first, the comparison of the shape feature output of the shape feature extraction means 24 is performed by the comparison means 28. In the illustrated example, the vertical width H and the horizontal width W are extracted as shape features for each connected component distinguished by labeling, and the comparison means 28 determines that both of them are within a certain range, that is, Hmin <H <Hmax and Wmin <
Objects that do not satisfy the condition of W <Wmax are identified as not monitoring objects. However, the shape characteristics used in the present invention are not limited to the vertical width and the horizontal width, and for example, the area of the connected component and the like may be included in the shape characteristics. In short, when the monitoring target is a human, the range of a representative human body shape feature is stored in advance, and the shape feature extracted from each connected component by the shape feature extraction unit 24 is within the range of the human body shape feature. By judging whether or not there is, a very large connected component or a very small connected component, for example, a label component of symbol A in FIG. 10 corresponding to a small animal or a residual noise component (not shown) is not monitored. Can be identified.

Next, in steps 102 and 103 of FIG. 13, the identification processing for the motion feature output of the motion feature extracting means 26 is compared with the comparing means.
Perform by 28. In the illustrated example, the speed V and the speed variation ΔV are extracted as motion characteristics for each connected component distinguished by labeling, and the comparison unit 28 determines that both the speed V and the speed variation ΔV are within a certain range, that is, V Those which do not satisfy the condition of <Vmax and ΔV <Vmax are determined not to be monitoring objects. In the case of FIG. 13, the absolute value | Vi of the maximum frequency of the horizontal component Vi and the vertical component Vj of the motion vector V in the connected component as the horizontal component and the vertical component of the velocity V
and | Vjc |, the absolute value of the difference between the maximum value and the minimum value | ΔVi | and | for each of the horizontal component Vi and the vertical component Vj of the motion vector V in the connected component as the speed variation ΔV
ΔVj | is the speed. That is, the horizontal component and the vertical component are identified for both the speed V and the speed variation ΔV. However, the motion feature used in the present invention is not limited to the combination of the speed and the speed variation, and for example, the acceleration may be included in the motion feature. Furthermore, the method of extracting the speed and the speed variation is not limited to the above example. For example, a median, an average,
Standard deviation, vector representation, etc. may be used. The identification by speed makes it possible, for example, to identify high-speed moving objects such as cars and birds from low-speed monitored objects, in this case humans. The identification based on the speed variation makes it possible to identify a complex moving object including a part moving in a different direction, such as a branch of a tree, from the monitoring target of the integrated movement, which is a human in this case, and corresponds to a tree in FIG. Can be excluded from the monitoring target.

As a result of the above identification, it is possible to identify only the connected component of a human with the label B as the binary detection image Ig in FIG. 11 as the monitoring target. Moreover, the identification process leading to the image Ig in FIG. 11 is based on the movement of the monitoring target, and is not based on the luminance difference. Therefore, the identification method of the present invention solves the problems of the prior art without being affected by the change in illuminance, the difference in luminance between the monitored object and the background, and the contrast of the monitored object itself. .

Thus, the object of the present invention is to provide "a method and an apparatus for identifying an object to be monitored by its movement".

[0019]

FIG. 4 shows an example of the configuration of the motion vector image forming means 1. De _Lee digital input image I to be input successively in time, the first current image by the first switching means 10a storing means 1
2a or the second current image storage means 12b is alternately stored. At the same time, the first current image storage means 12 is switched by the second switching means 10b.
b or alternately already-stored image from the second current image storage unit 12a is read in blocks as the current image I _t. At this time, the input image is written in the first current image storage unit 12a or the second current image storage unit 12b that is not currently being read, and the second current image storage unit 12b or the first image that is not currently being written is written.
It is read from the current image storage means 12a. Read current image I
_t is applied in parallel to the third switching means 10c and one input of the block matching means 14 described below.

The third switching means 10c is alternately connected to the first previous image storage means 16a or the second previous image storage means 16b, and the fourth switching means 10d is connected to the second previous image storage means 16b or the first previous image storage. It is connected alternately to the means 16a to read the previous image I _(t-1) . Also in this case, the data is written to the first previous image storage unit 16a or the second current image storage unit 16 that is not currently read, and
The data is read from the second previous image storage unit 16b or the first previous image storage unit 16a to which writing is not currently performed. The previous image I _(t-1) read out in block units is applied to the other input of the block matching means 14.

The writing time and the reading time of both the current image storage means 12a and 12b and the previous image storage means 16a and 16b are all equal. Further, the processing of the block matching means 14, that is, the above-described motion vector V calculation processing for each block is performed within the writing time or the reading time. Under such a configuration, the first through fourth switching means 10a, 10b, by operating the 10c and 10d alternately as described above, real time de _Lee digital image input successively in time Can be sequentially processed.

In this embodiment, the block matching means 14
After passing through the filter means 17, the motion vector V from the image data is used by the image forming means 18 for the motion vector image Iv. This is to prevent erroneous generation of a finite-valued motion vector V due to image signal noise or the like when there is no actual image change and no motion vector V exists. For example, the threshold value _{V t (i t, j t} ) with respect to the motion vector V is set to, Vi <Vi _t
And Vj <When Vj _t forcibly and motion vector V = 0. Here, Vi and Vi _t represents the horizontal component of the motion vector V and the motion vector threshold value Vt, respectively, Vj and Vj _t denotes a vertical component of the motion vector V and the motion vector threshold value Vt. In the embodiment of the present invention, in which the noise reduction is performed by performing the contraction image processing and the diffusion image processing after binarizing the motion vector image Iv, the threshold value is set to zero, that is, Vi _t = 0, Vj _t = 0, and the filtering means is used. 17 can be omitted.

FIG. 14 shows the configuration of a monitoring device using the above-described identification method. TV surveillance areas including off-limits areas
It is constantly monitored by image input means 5 such as a camera. The image I from the image input unit 5 is processed by the motion vector image forming unit 1, the binarizing unit 2, and the identifying unit 3 already described with reference to FIG. The value detection image Ig (FIG. 11) specifies the connected component corresponding to the monitoring target. In this embodiment, the prohibited area storage means 7 stores an image Iz including a restricted area position Z as shown in FIG. 12, for example. The abnormality determining means 8 determines whether or not the monitoring target in the binary detection image Ig (FIG. 11) overlaps the entry-prohibited area position Z in the prohibited-area image Iz. When there is an overlap, it is determined that the monitored object is in the restricted area and is abnormal. Preferably, an abnormality is determined when the number of image blocks in the overlapping portion exceeds a preset number. In the above description, the monitoring target is a person. However, the monitoring target is not limited to a person only. For example, in the case of safety management at a construction site,
Large moving objects such as automobiles, trucks, and cranes can be individually or comprehensively monitored.

In the embodiment shown in FIG. 14, detection is performed after the monitored object has entered the restricted area. For example, in a construction site or the like, detection after entry may be too late. If a large vehicle such as a truck or a crane enters an area where the ground is weak, there is a risk of causing a serious accident involving personal injury. Therefore, there is a demand for the development of a device that detects a situation where a monitoring target is predicted to enter before the monitoring target enters the restricted area.

In order to respond to this demand, an embodiment of the present invention shown in FIG. 16 first comprises an image input unit 5, a motion vector image forming unit 1, a binarizing unit 2, and a discriminating unit 3 similar to those in FIG.
For example, a diagram containing connected components corresponding to the monitoring target by
Generate a binary image Ig as shown in Fig. 11. Then, the connected component identified as the monitoring target (the shaded area in FIG. 26)
The predicted position of the connected component after a predetermined time (τ) is predicted by the first predicting means 34a (see FIG. 19) in the predicted state storing means 32 based on the motion vector V in the motion vector image corresponding to. After the lapse of the predetermined time (τ), the difference between the position of the connected component and the expected position in the binary image Ig obtained after the lapse of the predetermined time (τ) is detected by the prediction match determination unit 38 (see FIG. 19), and is equal to or less than a predetermined value. And the number N of continuous predictions among the expected positions predicted during N or more consecutive times of the predetermined time (τ) is stored in the prediction state storage means 32. To memorize. Determine the abnormality of the position of the connected component (shaded by a broken line in FIG. 26) after a time (ατ) obtained by multiplying the predetermined time (τ) by an integer (α) equal to or less than the continuous hit number N, based on the motion vector. The position of the connected component predicted by the second predicting means 34b (see FIG. 19) of the means 40 (see FIG. 19) and predicted by the second predicting means 34b (the shaded area of the broken line in FIG. 26) and the prohibition in the prohibited area storage means 7 Prohibited zone position Z in zone image Iz
The overlap with (see FIG. 26) is detected by the abnormality determination means 40.

[0026] To further explain the embodiments of the determination the predicted position, from the current image I _t of the observation area, as shown in FIG. 17, monitored by the motion vector image forming unit 1, the binarization unit 2, and identification means 3 A binary image Ig (see FIG. 11) including a connected component corresponding to the target is obtained, and the binary image Ig is converted into a detection vector image Igv in FIG. 18 by the motion vector conversion unit 30. This conversion is based on the binary image Ig and the motion vector image Iv shown in FIG.
It can be done by matching the block with.

FIG. 19 shows an example of the configuration of the predicted state storage means 32.
Show. The first prediction processing means 34a outputs the previous image I_(t-1)Is a prediction letter
When input to the state storage means 32, the detected vector image of FIG.
The motion vector V in Igv is the next current image I_tUntil
Calculate the predicted vector image Igve
This is stored in the storage means 36. FIG. 25 (a) shows the first prediction processing procedure.
The prediction by stage 34a is shown schematically. Basically, the dotted line
Previous Image I_(t-1)From solid image I_tThe movement that led to
Current image I_tNext image of the dashed line from_{(t + 1)}To the movement to
The next image I_{(t + 1)}Is the current image I_tEach block
The coordinates of the motion position are determined by the magnitude of the motion vector
Predict that it can be obtained by moving only. Note here
Note that the horizontal component Vi and the motion vector V (i, j)
The vertical component Vj is a pixel unit (a unit on the coordinate axis of the pixel coordinate system).
), But the block movement is the block of the above formula (1).
Coordinates of the coordinate system (X,Y) According to the integer multiple of the block
Performs only marker changes. Therefore, prior to the above prediction, the current image I_t
The coordinates (x, y) of the motion vector V in the pixel coordinate system
Coordinates of the check coordinate system (X,Y). This conversion is
As shown in steps 121 and 122 in the flow chart of FIG.
Current motion vector V of specific block_tHorizontal component V of
_tiAnd vertical component V _tjIs the horizontal width of the block.
Divide by half (m / 2) and half the vertical width (n / 2) to obtain the quotient Sx and
And Sy, rounding off their quotient to the nearest whole number
And the motion vector V (Sxr, Syr) in the block coordinate system
obtain. The first prediction processing means 34a calculates the coordinates (X,
YAt the next time point (t + 1)
(X ₊₁,Y ₊₁Predict to move to).

[0028] Step 123 of _{X +1 = X + Sxr ··· (} 3) Y +1 = Y + Syr ··· (4) FIG. 22 shows the operation process of the above equation. The current image I _t
The prediction vector image Igve can be calculated by performing the above-described prediction on each block of. The coordinates after the movement
(X _+1, Y ₊₁₎ motion vectors in, and before the motion vector V _t that was used when moving the image I _(t-1).
In FIG. 25 (a), the it is possible from the previous image I _(t-1) representing the motion vector V leading to the current image I _t as a line segment between the center points of the two images, in this case the motion vector V Next image I
_{(t + 1)} to the next image I _{(t + 1)} centers showing the tip from the center point of the current image I _t down to write the motion vector V in the pixel coordinates of the assumption to continue the chain line to the Absent. This is because the prediction of the position of the next image I _{(t + 1)} is not performed in the pixel coordinate system as described above, but is performed in the block coordinate system.

The prediction match determination means 38 in the prediction state storage means 32 shown in FIG.
When input to 2, the detected vector image Igv is compared with the predicted vector image Igve previously predicted by the first prediction processing means 34a. This comparison is performed by comparing motion vectors for each block. FIG. 21 shows a flow of the judgment in the prediction coincidence judging means 38, where Vit and Vjt show the horizontal component and the vertical component of the motion vector V in the block in the detection vector image Igv, and Vit _-1 and Vjt _-1. Indicates the horizontal and vertical components of the motion vector V in the block of the predicted vector image Igve at the same position as the block in the detected vector image Igv. In steps 111 and 112, the absolute value of the difference between the horizontal and vertical components of the motion vector V at an arbitrary block position in the detected vector image Igv and the motion vector V of the predicted vector image Igve at the same position Is calculated, and it is determined whether or not those absolute values are smaller than predetermined allowable values (Δi, Δj) for matching. If each of the components is within the allowable value, it is determined that the values match or predictive and the counter 3 is set as shown in step 113.
1 is added to the number N of consecutive middle times in 9. When one of the horizontal component and the vertical component is equal to or larger than the allowable value, the continuous middle number N is forcibly set to N = 1 as shown in step 114.

The first in the predicted state storage means 32 described above.
The prediction processing means 34a predicts the expected position of the monitoring object after a predetermined time (τ) from the current time t.In the embodiment of FIG. 16, a time (α) times the predetermined time (τ) is an integer (α). Monitoring is performed while predicting the position of the monitoring target after ατ). FIG. 20 shows details of the abnormality determination means 40 for this purpose. The second prediction processing unit 34b in the abnormality determination unit 40 determines that the integer (α) is equal to the continuous number of times N detected by the prediction state storage unit 32 (α =
In the case of N), the position of the monitoring target after a time (Nτ) is predicted. The principle is that the prediction was successful over the past N times, and the same goes for N times thereafter, ie, the current image
Each block Bt of I _t is the prediction calculation as being moved by N times the current motion vector V _t of the block.
FIG. 25b schematically shows the principle of the block position prediction operation after the time (Nτ). The gist of this is that the processing described for the prediction operation after the predetermined time (τ) with reference to FIG. It is only going to be about the case of doubling.
FIG. 23 is a flowchart of the processing of the second prediction processing means 34b,
The principle is that the flow of the first prediction processing means 34a in FIG. 22 is performed when the motion vector is N times larger.
It is predicted that the block at the coordinates ( X , Y ) at the original time t moves to the coordinates ( X _{+ N} , Y _{+ N} ) of the following equation shown in step 134 at the time (t + N) after the time (Nτ).

X _{+ N} = X + Sxr (5) Y _{+ N} = Y + Syr (6) where Sxr in equation (5) and Syr in (6) are Sxr and (3) in equation (3), respectively. N times Syr of (4). Detection vector image Igv
The prediction calculation of the equations (5) and (6) is performed on all the blocks to obtain a prediction vector image including the prediction position of the monitoring target after a time (Nτ), and the binarization processing is performed on the prediction vector image. This makes it possible to obtain the predicted binary image Ige after the time (Nτ). Although the case where the multiple α with respect to the time τ is N (α = N) has been described, the operation itself can be performed on an arbitrary integer α.

FIG. 24 shows the flow of processing in the second prediction processing means 34b in the abnormality determination means 40. In step 141, the area image Iz of the prohibited area storage means 7 shown in FIG.
And the current detected binary image Ig and the predicted binary image Ig after a time (Nτ) are compared, and the image Ig entering the restricted area Z is compared.
And / or determine whether the number Nb of non-zero blocks in the Ige exceeds a preset allowable value Ne. If not, terminate the calculation as there is no abnormality.
Proceed to 142. In step 142, it is determined whether or not the non-zero block that has entered the restricted area Z is only the image of the predicted binary image Ige. Judgment ends the calculation, the image Ig
If it is not only e, it is determined as "abnormal" and the calculation is terminated.

The above-described embodiments, the current image I _t, the previous image I _(t-1), the basic principle of change detection by comparison with a predetermined standard image, or prohibited area image Iz like reference picture . Therefore, an arbitrary pixel coordinates (x, y) observed zone in a position indicated by a in the reference image corresponding pixel coordinates (x, y) preconditions coincidence between the observation area in the position indicated by the current image I _t. However, when an imaging device such as a TV camera is used as an image input device, image blurring may occur due to an accidental or intentional change in the imaging direction. Causes of the accidental image blur include vibration caused by wind and the like, and a change with time of the mounting jig. When there is image blurring, the identification by the motion vector becomes inaccurate, and when this is used for the monitoring device, the monitoring target that does not actually exist cannot be detected or the monitoring target that does not actually exist is erroneously detected. May cause serious errors. Therefore,
There is a demand for the development of a device for correcting image blur.

[0034] One embodiment of the present invention shown in FIG. 27 To answer this demand, first to the current image I _t from the image input unit 5 similarly to image input unit 60 of FIG. 14, the motion vector extraction unit 67 Processing is performed to create a motion vector image Iv. The configuration of the motion vector extracting unit 67 is the same as that of the motion vector image forming means 1 in FIG. The reference area storage means 74 (see FIG. 28) stores a reference area image I _R including a reference area R which is a fixed area in the observation area. The image blur extraction unit 62 outputs the motion vector V at the position of the reference area R in the motion vector image Iv as an image blur vector D.

[0035] reference image storage unit 80 (see FIG. 29), storing a predetermined image of the observed region as a reference image I _rf, the reference image I _rf from the image blur vector D subtractive corrected reference image I by modifying means Output _rfm . The reference image I _rf, is in the current image I _t and the case be a standard image to be compared are those that do not correspond period changes, the current image I _t before image I immediately preceding the _(t-1) There are cases. Image change extraction unit 61
The difference between the corrected reference image I _rfm the current image I _t is calculated for each block, the outputs of the difference image Id. Preferably, the difference is binarized to output a binary difference image Ibi, and more preferably, a threshold value for the binarization is provided by the image change identification unit 63, but these binarization processing and threshold setting are indispensable. is not.

By comparing the difference image Id with image information of a predetermined monitoring target, the monitoring target in the observation area is identified, and a detection image Igd including the identified monitoring target is output. An example of the image information of the predetermined monitoring target when the difference image Id is the brightness difference image is a detection mask. When the binary difference image Ibi is used, after performing the contraction image processing and the diffusion image processing on the binary difference image Ibi, the shape characteristic of each connected component in the binary difference image Ibi and the connected component By comparing the motion feature of the motion vector before conversion with the shape feature and the motion feature of the predetermined monitoring target, the monitoring target in the observation area is identified, and a binary detection image Ig including the identified connected component is output.

The prohibited area storage unit 66 stores a prohibited area image Iz including the position of the prohibited area Z in the observation area, and stores a corrected prohibited area image Izm obtained by subtracting the image blur vector D from the prohibited area image Iz. Read.

The abnormality judging section 65 detects the overlap between the detected image Igd and the correction prohibited area image Izm, thereby achieving the purpose of abnormality detection. The detected image I when using the binary difference image Ibi
gd is the binary detection image Ig.

In order to improve the accuracy of the image blur vector, the test vector T is replaced with the previous image I in the motion vector image forming means 67.
_(t-1) , and an image block in which the test vector T does not match the image blur vector D at the position of the reference area R output from the image blur extraction unit 62 in accordance with this addition is extracted from the reference area R. Can be excluded. When the image input unit 60 is a TV camera, a control / drive unit that corrects the direction of the TV camera according to the image blur vector D can be provided.

FIG. 28 shows an example of the configuration of the image blur extracting unit 62, and FIG. 32 shows a flowchart of the processing steps. The reference area storage unit 74 in FIG. 28, be stored as a reference area image I _R image of the reference area R to be described below in units of blocks. In the image blur extracting unit 75 compares the motion vector image Iv outputted from the motion vector extraction unit 67 and the reference area image I _R from the reference area storage unit 74, extracts only the motion vector V in the reference area R. In step 151 of FIG. 32, a histogram of the motion vector V in the reference area R extracted as described above is created, and in the next step 152, the motion vector V giving the maximum frequency is set to the previous image I _(t-1) and extracts an image blur vector D between the current image I _t. At this stage, since the pixel coordinate system is used, in steps 153 and 154, this is converted into a shift vector H ( X , Y ) in the block coordinate system. This conversion is performed so that the image at the position of the reference area R is corrected by the shift vector H at the time of the image blur extraction for the next image I _{(t + 1), and is then given} to the comparison process of the image blur extraction means 75. , i.e. the next image I _{(t + 1)} is to the current error of the image of the reference area location to be compared with because it may have an image blur with respect to the current image I _t next image I _{(t + 1)} This is intended to be reduced by compensation for image blur. Incidentally, the conversion into the shift vector H ( X , Y ) is performed by the image blur vector D
The horizontal and vertical components of are divided by half of the horizontal size of the block (m / 2) and half of the horizontal size (n / 2), respectively, and are rounded to the nearest decimal point. . When the reference area is updated, the above-described calculation processing is performed after updating the area set value as shown in step 150.

FIG. 35 shows an example of processing for setting the reference area R by the area setting section 68 in FIG. The reference area R selected by the operation unit 69 is set while being confirmed on the display unit 70. For example, the restricted area Z is selected and set as a special reference pole for identification or a surrounding building as the reference area R, and the selection criterion is an index for recognizing the restricted area Z;
That is, vibrations due to wind and the like are small, and high image contrast is obtained. Even if an image blur occurs in the reference area R set in consideration of these standards, the image blur vectors D in the respective blocks in the reference area image have the same magnitude in the same direction as shown in FIG. . FIG. 35B shows an image blur vector D of only the reference area R.

The image blur vector D thus extracted is
It is used in the image change extraction unit 61 and the prohibited area storage unit 66 in FIG. 29 and 30 show two examples of the configuration of the image change extraction unit 61, and FIG. 31 shows the configuration of the prohibited area storage unit 66.

[0043] FIG. 29, a reference image difference image Id between the current image I _t
An embodiment of an image change extracting unit 61 of a method of extracting as a luminance change between I _ref and an I _ref will be described. The current image I _t is delayed only by the delay means 77 the time required for extraction of the image blur vector D by the motion vector extraction unit 67 and the image blur extracting unit 62, are incorporated in the current image storage unit 78. Inter-image difference unit 79 extracts by the subtraction a difference between the reference image I _ref from the reference image storage unit 80 and current image I _t from the current image storage unit 78. Before this subtraction, the reference image I _ref is subjected to a difference calculation after the coordinates are corrected based on the pixel unit correction vector M from the first image blur adding means 81. In order to avoid the influence of the illuminance variations in the observation area up to the point of the current image I _t, periodically updating the reference image I _ref.

FIG. 33 is a flowchart showing the process of calculating the correction vector M in the first image blur adding means 81. Step 16
After the updating of the reference zone R and the no-go zone Z and the reference image update in step 0 and step 161, it is determined in step 162 whether or not the update has just been performed.
After the total value of the image blur vector D is reset in step 3, the process proceeds to the next step 164. If it is not immediately after the update, the process directly proceeds to step 164 to obtain the correction vector M by adding the image blur vector D. This addition is performed separately for a horizontal component i and a vertical component j. Thus, from the time of updating of the reference image I _ref can be corrected for accumulated value of image blur caused by the time of the current image I _t. Step 165 is for resetting the correction vector M when the reference area R and the restricted area Z are updated.

[0045] Figure 30 shows an embodiment of an image change extraction section 61 of the system to be extracted as a vector V motion for each block by the block matching between the the current image I _t and the previous image I _(t-1) difference . The current image I _t is delayed by the delay means 77 by a time required for the image shake vector D extraction, incorporated alternately in the first and second current image storage unit 12a, 12b via the first switching means 10a . Second switching means 10b first and second current image storage means 12a through, the current image I _t read out alternately from 12b third switching means 10c and the block matching unit 14
And added in parallel. The image from the third switching means 10c is stored in the first and second current image storage means 16a and 16b.
Read alternately as _(t-1) . The previous image I _(t-1) of the first and second current image storage means 16a, 16b is stored in the image blur extraction means 6
Modified by the image blur vector D from 2. First and second previous image storage means 1 via the fourth switching means 10d.
Modified previous image I alternately read from 6a and 16b
_(t-1) is the motion vector V after the compared image blur corrected with the current image I _t for each block in the block matching unit 14 are extracted. The block matching process is sequentially performed between images that are temporally and continuously input by the selective switching operation of the first to fourth switching units 10a, 10b, 10c, and 10d described in detail with reference to FIG. It can be carried out. In the embodiment of FIG. 30, between does not require first image blur adding unit 81 in FIG. 29, this motion vector V after the image blur correction is the current image I _t and the previous image I _(t-1) It is because it is extracted by.

When the embodiment shown in FIG. 29 is used for the image change extraction unit 61, the difference image is binarized by the image change identification unit 63, and the changed part is the monitored object in the object identification unit 64. After identifying whether or not the detected binary image is detected, a detected binary image of only the monitoring target can be output. On the other hand, an image change extraction unit
When the embodiment of FIG. 30 is used for 61, the image change identification unit 63
After performing the identification of the presence or absence of the motion in the object identification unit 64 after identifying whether the changed portion is a monitoring object,
It is possible to output a motion presence / absence image of only the monitoring target.

FIG. 31 shows an example of the configuration of the prohibited area storage section 66, and FIG. 34 shows a flowchart of the second image blur adding means 82. Since the prohibited area image Iz sent to the determination unit 65 to compensate for image blur caused by acquired current image I _t, prohibited area image Iz
Are output after the coordinates are shifted by the shift vector H from the second image blur adding means 82. At this time, when the image change extracting unit 61 is the one shown in FIG. 29 and performs the processing in the image difference
The shift by 82 is performed in pixel units, but the image change extraction unit 61 is the one shown in FIG. 29 and the processing by the image difference means 79 is performed in block units, or the image difference extraction unit 61 shown in FIG. When the processing in the means 79 is block matching, the shift by the second image blur adding means 82 is performed in block units. The flowchart of FIG. 34 is an example of the second image blur adding means 82 used with the image change extracting unit 61 of FIG. After the reset at the time of zone update shown in steps 170 and 171, the image blur vector D is added in step 172 to obtain an integrated value. In steps 173 and 174, the integrated value is converted into a shift vector H of the coordinates ( X , Y ) in the block coordinate system. This conversion is similar to the conversion described with reference to FIG. When the second image blur adding means 82 shifts in pixel units, the integrated value of the image blur vector D in step 172 is used as it is as the correction vector M.

FIG. 36 shows a reference area image Iz
1 shows the overall configuration of a monitoring device to which the correction function of (1) is added. What is added is that the output of the area setting unit 68 is
And the output of the motion vector extraction unit 67 is connected to the area setting unit 68 only. FIG. 37 shows the configuration of the zone setting section 68 in this case. Area image generating means based on the restricted area Z and the reference area R designated by the operation unit 69
86 generates a prohibited area image Iz and reference area image I _R. Image synthesizing means 88 synthesizes the current image I _t and the both sections image Iz and I _R, and displays the synthesized result of the display unit 70 navel. In this embodiment, it is possible to set the restricted area Z and the reference area R while observing the composite image on the display unit 70. The prohibited area image Iz is set in the prohibited area storage unit 66 as it is. On the other hand, the reference area image I _R is taken into the reference area correction means 87, where it is subjected to necessary correction and then added to the image blur extraction unit 62.

FIG. 38 shows a flow chart of the processing of the reference area correcting means 87. Step 180 of the reference zone modifying means in step 181 after initialization 87 stores the horizontal component i _s and vertical components j _s test vector T (i _s, j _s) sends out to the vector extraction unit 67 moves the . In the motion vector extraction unit 67, the previous image I
_{After (t-1)} is deliberately moved by the test vector T, a motion vector V, that is, an image blur vector D is extracted for each block in the reference area R and sent to the reference area correction means 87. The horizontal component of the motion vector V is defined as a virtual blur horizontal component Di, and the vertical component of the motion vector V is defined as a virtual blur vertical component Dj. Reference area correcting means 87 at step 182 and 183, after a later motion vector V receiver, whether virtual blur horizontal component Di coincides with the horizontal component i _s of test vectors, and virtual blur vertical component Dj is the test vector Vertical component
It determines whether coincides with j _s for each block. This agreement indicates that the reference area R is a stable area without fluctuation. The reference area correcting means 87 further performs steps 184 and 1
The test vector T is transmitted by 85 and the motion vector V
The reception and the determinations in steps 182 and 183 are repeated a predetermined number of times Ns.

At steps 186 and 187, the reference area correcting means 87
Is a modification in which only the blocks in which all the results of the above determinations match in the Ns times are kept in the reference area R, and other blocks are excluded from the reference area R.
Apply to Since the reference area image I _R after the correction removes all the unstable blocks, the extraction of the image blur vector D in the image blur extraction unit 62 is highly stabilized.

FIG. 39 (a) is a schematic diagram of an image blur vector obtained from the constituent blocks of the reference area R before the correction by the reference area correction means 87, and FIG. Image blur vector D obtained from constituent blocks of area R
FIG. In FIG. 39 (b), unstable blocks have been removed, and the image blur vectors of each block all match.

FIG. 40 shows the overall configuration of a monitoring apparatus in which the function of correcting the imaging direction of a TV camera (not shown) according to the image blur vector D is added to the monitoring apparatus of FIG. What I added was
A camera control unit 90 that outputs a control signal based on the extracted image blur vector D, and a camera drive unit 91 that changes the direction of the TV camera in the image input unit 60 in response to the control signal
Only.

FIG. 41 shows the structure of the camera control section 90, and FIG. 42 shows a flow chart of the processing of the image blur adding means 92 therein.
After the reset at the time of zone update shown in steps 190 and 191 in FIG. 42, the image blur adding means 92 adds the image blur vector D in step 192 to obtain the horizontal integrated value Di and the vertical integrated value Dj. Sign reversing means of image blur adding means 92
93 generates inverted signals -Di and -Dj from the horizontal integrated value Di and the vertical integrated value Dj, and adds the inverted signals to the camera drive unit 91 as control signals. In response to this control signal, the camera drive unit 91 drives camera direction control means such as a camera platform (not shown) to change the imaging direction of the camera. By constantly performing this series of operations, the imaging direction of the camera can be constantly and automatically adjusted. That is, the image blur vector D can be negatively fed back to control the camera imaging direction so that the integrated value of the image blur vector D is always zero.

It is not always necessary to always control the camera imaging direction, and the camera may be operated only when the integrated value of the image blur vector exceeds a preset value.

[0055]

As described in detail above, the method and apparatus for identifying a motion vector according to the present invention identify an object to be monitored based on the motion of an image, and therefore have the following remarkable effects.

(A) It is possible to avoid an error in monitoring target object identification caused by a change in illuminance due to a change in sunlight or a change in illumination in the observation area.

(B) Insufficient luminance difference between the monitored object and the background
It is possible to overcome the difficulty of identifying the shape of the monitored object due to the lack of contrast of the monitored object itself.

[0058] (c) can be performed successively in time sequential processing to continuous reliable monitor de _Lee digital image received in real time basis.

(D) Since the identification is performed not only by the shape features of the image but also by the combination with the motion features, the identification accuracy can be improved.

(E) Since a TV camera or the like is used as the image input means, an inconspicuous monitoring device can be manufactured.

(F) Since the data relating to the movement of the image is used, it is possible to make a prediction, and it is possible to foresee a danger and to issue an alarm or take other measures to improve safety in equipment maintenance or the like.

(G) The TV camera or the like can be accurately corrected for the imaging direction blur caused by the wind or the like in the image input means.

(H) In comparing the images, not only an immovable reference area is set in the observation area, but also the immobility and stability are measured, and only those with high stability are selectively used as a reference to significantly improve the identification accuracy. Can be improved.

(I) Minute blurring in the imaging direction of a TV camera or the like can be negatively feedback-controlled to reduce the integrated value of image blurring to zero, thereby improving identification accuracy.

(V) While the image input device such as a TV camera is being corrected and driven by the control device for correcting the imaging direction, the blur detection and the image correction are continued, and the mechanical automatic correction driving of the image input device is performed. Accurate identification can be ensured even during.

[Brief description of the drawings]

FIG. 1 is an explanatory diagram of an embodiment of an identification device of the present invention.

FIG. 2 is an explanatory diagram of an image frame and blocks.

FIG. 3 is a diagram for explaining the principle of block matching;

FIG. 4 is an explanatory diagram of a motion vector image forming unit.

FIG. 5 is an explanatory diagram of an observation area.

FIG. 6 is an explanatory diagram of a motion vector image Iv.

FIG. 7 is an explanatory diagram of a binarization process on a motion vector image Iv.

FIG. 8 is an explanatory diagram of a result of contracted image processing on an image.

FIG. 9 is an explanatory diagram of a result of diffusion image processing on an image.

FIG. 10 is an explanatory diagram of a labeling process.

FIG. 11 is an explanatory diagram of a binary detection image.

FIG. 12 is an explanatory diagram of a prohibited area image.

FIG. 13 is a flowchart of a process performed by an object identification unit.

FIG. 14 is an explanatory diagram of one embodiment of the monitoring device of the present invention.

FIG. 15 is an explanatory diagram of a conventional technique.

FIG. 16 is an explanatory diagram of an embodiment of a monitoring device with prediction.

FIG. 17 is an explanatory diagram of an observation area in the monitoring device with prediction.

FIG. 18 is an explanatory diagram of a detection vector image.

FIG. 19 is an explanatory diagram of predicted state storage means.

FIG. 20 is an explanatory diagram of abnormality determination means.

FIG. 21 is a flowchart of a process of a prediction match determination unit.

FIG. 22 is a flowchart of the first prediction processing means.

FIG. 23 is a flowchart of the second prediction processing means.

FIG. 24 is a flowchart of the abnormality determination unit with prediction.

FIG. 25 is a schematic diagram for explaining a prediction process.

FIG. 26 is an explanatory diagram of a processed image to be subjected to abnormality determination.

FIG. 27 is an explanatory diagram of a monitoring device including image blur correction.

FIG. 28 is an explanatory diagram of an image blur detection unit.

FIG. 29 is an explanatory diagram of an embodiment of an image change extraction unit.

FIG. 30 is an explanatory diagram of another embodiment of the image change extraction unit.

FIG. 31 is an explanatory diagram of a prohibited area storage unit.

FIG. 32 is a flowchart of processing of an image blur extracting unit.

FIG. 33 is a flowchart of processing of a first image blur adding unit.

FIG. 34 is a flowchart of processing of a second image blur adding means.

FIG. 35 is an explanatory diagram of a reference area.

FIG. 36 is an explanatory diagram of an embodiment of a monitoring device with a monitoring area modification.

FIG. 37 is an explanatory diagram of an area setting unit.

FIG. 38 is a flowchart of the processing of the reference area correcting means.

FIG. 39 is an explanatory diagram of a processing effect of a reference area correcting unit.

FIG. 40 is an explanatory diagram of an embodiment of a monitoring device with camera imaging direction control.

FIG. 41 is an explanatory diagram of a camera control unit.

FIG. 42 is a flowchart of processing of a camera control unit.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 Motion vector image formation means 2 Binarization means 3 Identification means 5 Image input means 6 Motion identification means 7 Prohibited area storage means 8 Abnormality determination means 10a, 10b, 10c, 10d First, second, third, fourth cut Transformation means 12a, 12b First and second current image storage means 14 Block matching means 16a, 16b First and second previous image storage means 17 Filter means 18 Image formation means 20 Shrink diffusion means 22 Labeling means 24 Shape feature extraction means 26 Motion feature extraction means 28 Comparison means 30 Motion vector conversion means 32 Prediction state storage means 34a, 34b First and second prediction processing means 36 Storage means 38 Prediction match determination means 39 Counter 40 Abnormality determination means 42 Duplication detection means 50, 60 Image Input unit 51 Difference image extraction unit 52 Reference image 61 Image change extraction unit 62 Image blur extraction unit 63 Image change identification unit 64 Object identification unit 65 Abnormality determination unit 66 Prohibited zone storage unit 67 Motion vector extraction unit 68 Zone setting Unit 69 operation unit 70 display unit 74 reference area storage means 75 image blur extraction means 77 delay means 78 current image storage means 79 inter-image difference means 80 reference image storage means 81 first image blur addition means 82 second image blur addition means 83 Prohibited area storage means 86 Area image generation means 87 Reference area correction means 88 Image synthesis means 90 Camera control unit 91 Camera drive unit 92 Image blur addition means 93 Sign inversion means 105-193 Step

──────────────────────────────────────────────────続 き Continuation of the front page (72) Inventor Susumu Kikuchi 1-3-1 Uchisaiwaicho, Chiyoda-ku, Tokyo Inside Tokyo Electric Power Company (56) References JP-A-5-14892 (JP, A) JP-A Sho 62 -136988 (JP, A) JP-A-2-117276 (JP, A) (58) Fields investigated (Int. Cl. ⁷ , DB name) G06T 7/20 G08B 13/196 H04N 7/18 G06T 1/00

Claims (13)

(57) [Claims] 1. A continuous frames generated viewing zone image is divided into a certain number of pixels of the block; each block within an image in the current image I _t, the previous image I _(t-1) the block image in substituted on the motion vector V to the block position in the current image I _t of the portion corresponding to the outputs motion vector image Iv; the motion binary motion vector image Iv by binarizing the vector V image Ibi To convert; to remove minute connected components by contraction image processing and diffusion image processing;
A motion that identifies the monitored object in the observation area by comparing the shape characteristic of each connected component in Ibi and the motion characteristic of the motion vector before conversion of the connected component with the shape characteristic and the motion characteristic of the predetermined monitored object. Vector identification method. 2. The method according to claim 1, wherein the conversion from the motion vector image Iv to the binary image Ibi is performed by binarizing the motion vector V based on a predetermined threshold. 3. A continuous frames generated viewing zone image is divided into a certain number of pixels of the block; each block within an image in the current image I _t, the previous image I _(t-1) the block image in substituted on the motion vector V to the block position in the current image I _t of the portion corresponding to the outputs motion vector image Iv; the motion binary motion vector image Iv by binarizing the vector V image Ibi To convert; to remove minute connected components by contraction image processing and diffusion image processing;
Identifying the monitored object in the observation area by comparing the shape feature of each connected component in Ibi and the motion feature of the pre-transformation motion vector of the connected component with the shape feature and the motion feature of the predetermined monitored object; A prediction method using a motion vector obtained by predicting the position of the monitoring target using a vector V. 4. A continuous frames generated viewing zone image is divided into blocks of a fixed number of pixels, each block in the image in the current image I _t, the previous image I _(t-1) the block image in motion vector image forming means for outputting the current image I _t in the block motion vector image Iv by replacing the motion vector V to the position of a portion corresponding to, the motion vector image by binarizing the motion vector V Iv Binary Image
Binarization means for converting to Ibi; and removing minute connected components by contraction image processing and diffusion image processing to obtain a binary image Ibi
Identification means for identifying the monitored object in the observation area by comparing the shape characteristic of each connected component and the motion characteristic of the motion vector before conversion of the connected component with the shape characteristic and the motion characteristic of the predetermined monitored object. An identification device based on motion vectors. 5. The motion according to claim 4, wherein said binarizing means converts said motion vector image Iv into a binary image Ibi by binarizing said motion vector V based on a predetermined threshold value. Vector identification device. 6. The frame of the observation area continuous images generated is divided into blocks of a fixed number of pixels, the blocks in the image in each block in the image in the current image I _t, the previous image I _(t-1) motion vector image forming means for outputting the current image I _t in the block motion vector image Iv by replacing the motion vector V to the position of a portion corresponding to, the motion vector image by binarizing the motion vector V Iv Binary Image
Binarizing means for converting to Ibi; removing minute connected components by contraction image processing and diffusion image processing, and predetermining the shape characteristics of each connected component in the binary image Ibi and the motion characteristics of the pre-conversion motion vector of the connected component. Identification means for identifying the monitoring object in the observation area by comparing with the shape characteristic and the movement characteristic of the monitoring object, and outputting a binary detection image Ig containing the identified connected component; a prohibited area position in the observation area Prohibited area storage means for storing a prohibited area image Iz containing Z;
And a motion vector monitoring apparatus including an abnormality determination unit configured to detect an overlap between a monitoring target in the binary detection image Ig and a prohibited area position Z in the prohibited area image Iz. 7. A continuously the frame of the generated observation area image is divided into blocks of a fixed number of pixels, each block in the image in the current image I _t, the previous image I _(t-1) the block image in motion vector image forming means for outputting the current image I _t in the block motion vector image Iv by replacing the motion vector V to the position of a portion corresponding to, the motion vector image by binarizing the motion vector V Iv Binary Image
Binarizing means for converting to Ibi; removing minute connected components by contraction image processing and diffusion image processing, and predetermining the shape characteristics of each connected component in the binary image Ibi and the motion characteristics of the pre-conversion motion vector of the connected component. Identification means for identifying the monitoring object in the observation area by comparing with the shape characteristic and the movement characteristic of the monitoring object, and outputting a binary detection image Ig containing the identified connected component; identified as the monitoring object First predicting means for predicting an expected position of the connected component after a predetermined time (τ) based on a motion vector V in the motion vector image corresponding to the connected component; and a prohibited zone position Z in the observation zone is included. Prohibition area storage means for storing the prohibition area image Iz; and detecting the overlap between the expected position of the connected component predicted by the first prediction means and the prohibition area position Z in the prohibition area image Iz. A monitoring apparatus based on a motion vector, comprising an abnormality determining means for outputting the abnormality. 8. The apparatus according to claim 7, wherein after a lapse of the predetermined time (τ), a difference between the position of the connected component and the expected position is detected, and the difference that is equal to or smaller than a predetermined value is set to a predictive value. Prediction coincidence determination means for storing the number N of continuous predictions among the expected positions predicted during N or more consecutive times of a predetermined time (τ); A second predictor for predicting an estimated position of the connected component after a time (ατ) multiplied by an integer (α) less than or equal to the number of hits N based on the motion vector; and the second predictor predicted by the second predictor A monitoring apparatus based on a motion vector, comprising an abnormality determination unit that detects an overlap between an estimated position of a connected component and a prohibited area position Z in the prohibited area image Iz. 9. continuously the frame of the generated observation area image is divided into blocks of a fixed number of pixels, each block in the image in the current image I _t, the previous image I _(t-1) the block image in reference area image including the immobile reference area R of the observation zone; motion vector image forming means for outputting the motion vector image Iv by replacing the motion vector V in to the block position the current image I _t of the portion corresponding to the reference area storage means for storing the I _R; the reference area R of the motion in the vector image Iv
Image blur detecting means for outputting the motion vector V at the position as an image blur vector D; reference image storing means for storing a predetermined image of the observation area as a reference image I _rf ; image blur vector D from the reference image I _rf Correction means for outputting a corrected reference image _Irfm ; subtracting the current image
The difference between the corrected reference image I _t was calculated for each block, image change identifying means outputs the difference image Id; the difference image Id
Is compared with the image information of the predetermined monitoring target to identify the monitoring target in the observation area, and identification means for outputting a detection image Igd including the identified monitoring target; Prohibited area storage means for storing the included prohibited area image Iz and reading out the corrected prohibited area image Izm obtained by subtracting the image blur vector D from the prohibited area image Iz; and the detected image Igd and the corrected prohibited area image Izm. A monitoring apparatus based on a motion vector, comprising an abnormality determining means for detecting overlap with the motion vector. 10. A monitoring device according to claim 9, wherein the output difference image Id of the image change identification unit, binarization a difference between the current image I _t and the corrected reference image calculated for each block two After performing a contraction image process and a diffusion image process on the binary difference image Ibi, the binary image Ib
By comparing the shape feature of each connected component in i and the motion feature of the pre-transformation motion vector of the connected component with the shape feature and the motion feature of the predetermined monitoring target, the monitoring target in the observation area is identified and identified. A monitoring device based on a motion vector that serves as a detection image Igd output from the identification means, the binary detection image Ig including a connected component. 11. A monitoring system according to claim 9 or 10, wherein the reference image I _rf and the previous image I _(t-1), the image blur the front from the image I _(t-1) by said correction means A monitoring device based on a motion vector in which a vector D is used as a subtraction corrected reference image _Irfm . 12. The monitoring apparatus according to claim 9, 10 or 11, wherein a test vector T is added to said previous image I _(t-1) in said motion vector image forming means, and said image is generated according to said addition. A monitoring apparatus based on a motion vector, comprising a reference area correcting means for excluding, from the reference area R, an image block in which the image blur vector D output from the shake detecting means does not match the test vector T. 13. A surveillance device according to claim 9, 10 or 11, wherein said TV camera continuously outputs an image of said observation area, and control for correcting an orientation of said TV camera in accordance with said image blur vector D. A monitoring device based on a motion vector provided with a driving means.