JP3245752B2 - Image monitoring method and apparatus - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention extracts a feature amount of a moving object in an image when a visual sensor installed in a room to be monitored, a passage, a plaza, a parking lot, or the like captures the moving object. The present invention relates to an image monitoring method and apparatus for determining the presence or absence of a moving object by measuring the presence or absence of a temporal change in the feature amount.

[0002]

2. Description of the Related Art Conventionally, as a method of detecting the presence of a moving object by processing an image captured by a visual sensor, for example, a CCD camera, a moving object is detected when the number of pixels whose density has changed in an image exceeds a threshold value. A method of determining existence is known. In this method, images obtained one after another from a CCD camera are input to a system, and the number of pixels whose density has changed due to the change in brightness is searched for a long time, so that a threshold value that is not easily affected by the change in brightness is determined. It is determined, or the worker actually walks in the monitoring area, investigates the number of pixels whose density changes in the image for each input image, and determines a threshold value for detecting only the presence of a moving object (person). (In any case, the threshold value is determined empirically, the work for determining the threshold value is performed in advance, and the image monitoring is performed using the threshold value determined by the worker to be appropriate).

There has also been proposed a monitoring system in which a monitor of a monitoring center confirms the presence of a person in a monitoring area by using a monitor image by simply transmitting an image captured by a visual sensor to an image central monitoring center in a remote place. ing.

[0004]

However, in the former technique of the prior art, the presence or absence of a moving object is determined only by the total number of density change pixels, so that the sun suddenly enters from the window of the monitoring place. Or when the brightness in the monitoring area changes due to disturbance light such as the headlights of a car, or when small density change areas occur randomly and scattered in the image due to the influence of disturbance light such as in a showroom. Means that the total number of pixels whose density changes in the image may be as large as if there is a person, and is easily affected by disturbance light; a visual sensor is installed in a place where disturbance light is hard to enter. In this case, the influence of disturbance light can be reduced to some extent, but in this case, the installation location of the visual sensor is restricted, and the installation location becomes less flexible; Since the over, yet it requires skill to work, poor efficiency of the installation work of the image monitoring device;
And so on.

[0005] In the latter case, a person in an image displayed on the monitor image of the monitoring center is displayed small depending on the position of a person in the monitoring area, or when the contrast of the image is low, the person is not displayed. Because it is difficult to distinguish between the background and the background, it is required that the observer of the surveillance center pay close attention to the entire area of the image. The burden of labor is great, mental and physical fatigue is accumulated, and attention during monitoring is easily distracted; since the monitoring personnel at the monitoring center are forced to watch over the entire area of the image, one of the monitor images It takes time to check each sheet, and when urgent and prompt response is required, for example, call 110, it takes time to respond and it is too late. ; There was the problem of the like.

Accordingly, it is an object of the present invention to provide an image monitoring method and apparatus which are hardly affected by disturbance light.

Another object of the present invention is to provide an image monitoring method and apparatus which facilitates setting work by an operator.

Another object of the present invention is to provide an image monitoring method and apparatus in which an image transmitted to a remote image centralized monitoring center can be easily checked by a monitoring person at the monitoring center.

[0009]

According to a first aspect of the present invention, there is provided a visual sensor, reference image storage means for storing and updating a reference image based on an input image obtained from the visual sensor, and a reference image obtained from the visual sensor. Density change area detection means for detecting a density change area using the input image and the reference image stored in the reference image storage means; and the size, center of gravity, etc. of the density change area detected by the density change area detection means A feature value extracting means for extracting the feature value of the feature value, a feature value storing means for storing the feature value extracted by the feature value extracting means, and detecting a temporal change of the feature value extracted by the feature value extracting means. A determination means for determining the presence or absence of a moving object in a monitoring area; and a diagram defining a density change area detected by the density change area detection means when the determination means determines the presence of a moving object. A model fitting means for creating a pattern model and applying the graphic pattern model to the input image, and an image obtained as a result of applying the graphic pattern model to the input image by the model applying means to a remote monitoring center via a communication line. Image transmission means for transmitting, the image monitoring method in an image monitoring device comprising: a step of creating a difference image between an input image obtained from the visual sensor and a reference image stored in the reference image storage means; And binarizing the created difference image with a predetermined density threshold.
If there is an isolated point, the step of removing the isolated point and extracting only the set of point groups, and extracting the feature amount of the point group such as the size of the obtained point group and the barycentric coordinate position are performed. Determining whether there is a temporal change in the feature amount in the set state of the extracted point group, and determining that a moving object exists when at least one of the point group feature amounts has changed. Generating a figure pattern model to be applied to the density change area detected by the density change area detection means when it is determined that a moving object exists; and applying the figure pattern model created in the input image to the figure. An image monitoring method, comprising transmitting an image obtained as a result of applying the pattern model to a remote centralized monitoring center via a communication line.

According to a second aspect of the present invention, there is provided a visual sensor, reference image storage means for storing and updating a reference image based on an input image obtained from the visual sensor, an input image obtained from the visual sensor, and the reference image. A density change area detecting means for detecting a density change area using the reference image stored in the storage means, and a feature amount such as a size, a center of gravity, etc. of the density change area detected by the density change area detection means is extracted. A feature amount extracting unit, a feature amount storing unit that stores the feature amount extracted by the feature amount extracting unit, and a moving object in the monitoring area by detecting a temporal change of the feature amount extracted by the feature amount extracting unit. Determining means for determining the presence or absence of a moving object, and a graphic pattern model defining a density change area detected by the density change area detecting means when the determining means determines that a moving object is present. Model fitting means for fitting the graphic pattern model to the input image, and an image obtained as a result of fitting the graphic pattern model to the input image by the model fitting means is transmitted to a remote centralized monitoring center via a communication line. This is an image monitoring device provided with image transmission means.

[0011]

DESCRIPTION OF THE PREFERRED EMBODIMENTS In an image monitoring method and apparatus according to the present invention, a density change area is detected in an image taken from a visual sensor, and a point group is extracted by removing isolated points in the density change area. A feature figure such as the size of the region represented by the extracted point group, the position of the center of gravity coordinate position, etc. is obtained, and when a change over time in the feature amount is detected, a geometric figure pattern obtained by performing arithmetic processing on the point group Model (p
A pattern model is created, the figure pattern model is applied to the density change area of the captured image, and the image of the applied result is transmitted to a remote centralized monitoring center.

[0012]

The following is a description of an embodiment of the present invention. FIG. 1 is a flowchart showing the image monitoring method according to the present invention, and FIG. 2 is a block diagram showing the configuration of the image monitoring device according to the present invention. The apparatus shown in the block diagram of FIG. 2 is used for implementing the image monitoring method shown in the flowchart of FIG.

In FIG. 2, the image monitoring apparatus 1 includes a visual sensor 2, a reference image storage unit 12, a density change area detection unit 13, a feature amount extraction unit 14, and a feature amount storage unit 1.
5, determination means 16, model fitting means 17, and image transmission means 18.

Among them, the visual sensor 2 is constituted by a CCD camera installed in the monitoring area 7 and capturing an image of the monitoring area.

The reference image storage means 12 stores and updates the reference image based on the input image obtained from the visual sensor 2.

The density change area detecting means 13 detects a density change area using the input image obtained from the visual sensor 2 and the reference image stored in the reference image storage means 12.

The characteristic amount extracting means 14 extracts characteristic amounts such as the size of the density change area detected by the density change area detection means 13 and the center of gravity.

The feature quantity storage means 15 stores the feature quantity extracted by the feature quantity extraction means 14.

The judging means 16 judges the presence or absence of a moving object in the monitoring area by detecting a change over time of the feature quantity extracted by the feature quantity extracting means 14.

When the determining means 16 determines that there is a moving object, the model fitting means 17 creates a graphic pattern model for defining the density change area detected by the density change area detecting means 13, Apply the figure pattern model to the input image.

The image transmitting means 18 transmits an image obtained as a result of applying the graphic pattern model to the input image by the model applying means 17 to a remote monitoring center via a communication line.

Reference numeral 19 denotes an image receiving means provided at a remote image central monitoring center, and reference numeral 20 denotes a central monitoring means connected to the image receiving means 19.

Next, the procedure of the image monitoring method will be described with reference to FIG. First, C
An image is captured from the CD camera 2 (step 101), and a density difference image between the input image and the reference image is created by the density change area detecting means 13 (step 102). As the reference image, an image captured at a specific time or an image captured at regular time intervals is used. Here, C
The image obtained from the CD camera 2 is stored in the reference image storage unit 5, and thereafter the image obtained from the CCD camera 2 is stored and updated at regular time intervals as the reference image. The density change area detecting means 13 binarizes the difference image using a density threshold prepared in advance (step 103). This binarization is performed by setting the value to 0 when the density value is smaller than the density threshold, and setting the value to 1 when the density value is larger. Here, the point where the value at the coordinate position in the difference image is 1 will be referred to as a “point” hereinafter. And, as shown in FIG.
If an isolated point exists in the image resulting from the binarization, the isolated point is removed. Removal of this isolated point
For example, when attention is paid to a certain point in FIG. 3 and the number of points existing in eight pixels around the point is two or less, the point of interest is removed as an isolated point. Therefore, when there are three or more points having a value of 1 in the eight pixels around the target point, the point is determined to be one point in the point group, and the point is not removed as an isolated point. This removal of isolated points is performed for all points in the binarized image (step 104). As a result, if no point group is obtained, the contents of the feature amount storage unit 15 are deleted (step 10).
After step 5 and step 106), a new image is fetched from the CCD camera 2 and the above-mentioned predetermined processing is repeated. When a point group is obtained as a result of the isolated point removal, the coordinate position of the point is detected by the feature amount extracting means 14, and the barycentric coordinate position and the size of the point group region are obtained by the processing described below.

[0024] As (point group measurement of the centroid coordinate position of the region) barycentric coordinates _{G (u} g, _v g) is coordinates u, the value at v g (u, v), than the number 1 for each point From the zero-order moment m ₀₀ , the first-order moment (m ₁₀ , m
By determining the _01), the centroid coordinate values _{u g} and _{v g}
Is given by Equation 3.

(Equation 1)

(Equation 2)

(Equation 3)

(Measurement of the size of the point cloud area) Further, according to Equation 4, the second moments (m ₂₀ , m ₁₁ , m
₀₂ ) to obtain 2 around the centroid coordinate position G.
The next moment (M ₂₀ , M ₁₁ , M ₀₂ ) is given by Expression 5.

(Equation 4)

[Mathematical formula-see original document] Then, using "the maximum value M _maj and the minimum value M _{min of the} moment of inertia around a straight line passing through the center of gravity G" obtained by performing arithmetic processing on M ₂₀ , M ₁₁ and M ₀₂ , A major axis L and a minor axis S of an ellipse as a model are respectively obtained from Expression 6.

(Equation 6)

[0026] In this manner, the center of gravity coordinates G (u _{g, v g)} a characteristic amount of the point group region and the major axis L to the minor axis S of the size of the ellipse is expressed by the product (in this case represented by LS ), And the feature quantity extraction means 14 stores these in the feature quantity storage means 1
5 is additionally stored (step 107).

Whenever the density change area is detected in the input image obtained from the CCD camera 2, the feature quantity storage means 15 stores the current feature quantity, the previous feature quantity, and the two previous feature quantities. Will be stored as follows. Therefore, the current image capturing time is set to time t _n , the immediately preceding image capturing time is set to time t _n−1 , and the two previous image capturing times are set to time t _n.
When _{n-2, G tn (u} g, v g), LS tn; G
_{tn-1 (u g, v} g), LS tn-1; G
_{tn-2 (u g, v} g), LS tn-2; ··· are stored in the feature storage unit 15. Next, using the feature amount stored in the feature amount storage unit 15, the determination unit 16
, The change of the feature amount of the point cloud is detected. Determination means 16 changes the amount of time _{t n} to put the center of gravity coordinates G obtained from the number 7 a _{D tn} as _{D tn.}

Here, when the value of _Dtn is larger than a predetermined value (0 in the embodiment) (step 108), it is assumed that the barycentric coordinate position has changed. If a change in the barycentric coordinate position is detected, G _tn−1 , G _tn−2.
, The amount of change in the barycentric coordinate position at a past time is calculated, such as the amount of change one time before (D _tn-1 ), the amount of change two times before (D _tn-2 ) _,. At this time, if a change in the barycentric coordinate position is detected continuously for a predetermined number of times (three times in the embodiment), it is determined that the feature amount has changed (step 109). Further, the determination means 16
Is obtained from the number 8 the degree R _tn size variation of this ellipse by comparing the size of the previous one elliptical LS _tn-1 and the size LS _tn of this ellipse.

## _EQU8 ## If the degree _Rtn of the size change of the ellipse exceeds a predetermined value (here, _Rtn = 0.2) (step 110), _Rtn-1 and _Rtn-1 in the past data are obtained.
_tn−2 , is calculated retroactively (feature storage means 1
5 using the feature amount at the past time, which is stored in 5), and when the degree of magnitude change exceeds a predetermined value 0.2 more than three times continuously, the feature amount changes. It is determined that it has been performed (step 111).

Further, when the presence of a person is detected by the determination means 16 (that is, when an alarm is output), the model fitting means 17 processes the captured image at the time (time t _n ) to obtain the image. 2 around the specified barycentric coordinate position G
Using the next moment (M ₂₀ , M ₁₁ , M ₀₂ ),
The inclination θ of the major axis L of the ellipse is obtained (this inclination corresponds to θ in FIG. 4; see step 120).

Equation 9] The major axis at time t _n L, minor axis S, shapes that are determined by barycentric coordinates G and determined slope θ (the ellipse in the embodiment), applying to the input image at time t _n (step 121) . Here, the value 255 as the maximum density value is used (in the present system, the value 0 to the value 25).
The image is represented by a density value indicated by 5), the center of gravity coordinates G, major axis L, and the ellipse determined by the short diameter S and the inclination theta, drawn on the input image at time t _n. The image resulting from the fitting of the ellipse is transmitted from the image transmitting means 18 to the image receiving means 19 and the centralized monitoring means 20 installed at a remote image centralized monitoring center via a communication line such as a telephone line (step 122). . 5 and 6, when the visual sensor 2 captures an image of a person (moving object) in the monitoring area 7 and detects the presence of the person, the created ellipse (geometric figure pattern model) is taken in and the density of the image is taken. The result of applying the image to the change area and displaying the image of the applied result on the centralized monitoring means 20 of the remote centralized monitoring center is shown.

In the image monitoring method described in the above embodiment, the presence of a person is detected by an ellipse determined by the barycentric coordinate position and size of the density change area. Other figures may be used.
In addition, the barycentric coordinate position is used only for detecting the position in the image, and it is also possible to track the movement trajectory of the detection target (human, animal, automobile, etc.) by sequentially combining the barycentric coordinate positions. It is. Further, the feature amount of the density change area detected by the density change area detection means only needs to indicate the characteristics of the density change area, and is not limited to the barycentric coordinate position or size.

[0030]

According to the present invention as described above, the following effects can be obtained. For an image obtained from the visual sensor, for example, two-dimensional information based on the size (expansion degree) of the density change area can be used as a standard of the density change area, so that disturbance light enters the monitoring area. Even if there is light, accurate image monitoring is possible without being affected by disturbance light. In addition, the installation work of the device is completed in a short time, and skill is not required, so that the installation efficiency is good.
Further, since a graphic indicating the position and size of the density change area will be displayed in advance in the image transmitted to the centralized monitoring center, the monitor of the monitoring center can instantaneously detect the moving object on the monitor screen. The existence can be confirmed and the efficiency of image monitoring work can be improved.

[Brief description of the drawings]

FIG. 1 is a flowchart showing an image monitoring method according to the present invention.

FIG. 2 is a block diagram illustrating a configuration of an image monitoring device according to the present invention.

FIG. 3 shows an isolated point in an image when a difference image between an input image obtained from a visual sensor constituting the apparatus of the present invention and a reference image stored in a reference image storage means is binarized by a predetermined density threshold. FIG. 6 is a diagram illustrating a state where a point cloud exists.

FIG. 4 is a diagram showing a point cloud in an image obtained by binarizing a difference image between an input image obtained from a visual sensor constituting a device of the present invention and a reference image stored in a reference image storage means by a predetermined density threshold. FIG. 9 is a diagram showing that the spread of the image is defined using an ellipse as a graphic pattern model.

FIG. 5 is a diagram showing one image transmitted to a centralized monitoring center using the device shown in FIG. 2;

6 is a diagram illustrating an image transmitted to a centralized monitoring center when a short time has elapsed from the state of FIG. 5;

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 Image monitoring apparatus 2 Visual sensor 7 Monitoring area 12 Reference image storage means 13 Density change area detection means 14 Feature quantity extraction means 15 Feature quantity storage means 16 Judgment means 17 Model fitting means 18 Image transmission means 19 Image reception means 20 Centralized monitoring means

Claims (2)

(57) [Claims] 1. A visual sensor (2), reference image storage means (12) for storing and updating a reference image with an input image obtained from the visual sensor (2), and an input image obtained from the visual sensor (2) Density change area detection means (1) for detecting a density change area using the reference image stored in the reference image storage means (12).
3), a feature amount extracting unit (14) for extracting a feature amount such as a size and a center of gravity of the density changing region detected by the density changing region detecting unit (13); Determining a presence or absence of a moving object in a monitoring area by detecting a temporal change in a feature amount extracted by the feature amount extracting unit (14); Means (16), and a graphic pattern model for defining the density change area detected by the density change area detection means (13) when the determination means (16) determines that a moving object is present ,
Model fitting means (17) for fitting the graphic pattern model to the input image, and an image obtained as a result of fitting the graphic pattern model to the input image by the model fitting means (17) is transmitted to a remote monitoring center via a communication line. Image transmission means (1
8) The image monitoring method in the image monitoring device having
There are, said visual sensor (2) comprising the steps of inputting images obtained with the reference image storage means (12) to create a difference image between the reference image stored from the predetermined difference image generated density threshold If there are isolated points in the binarized result, removing the isolated points and extracting only a set of point groups; and Extracting and storing the feature values of the point group such as the size and the center of gravity coordinate position; Determining whether there is a moving object when one of them has changed; and, if determining that there is a moving object, a graphic pattern model to be applied to the density change area detected by the density change area detection means (13). Creating a pattern pattern model in the input image, and transmitting the image resulting from the fitting of the graphic pattern model to a remote centralized monitoring center via a communication line. Characteristic image monitoring method. Wherein the visual sensor (2), the reference image and the reference image storage means for storing updated (12) by the input image obtained from the visual sensor (2), the input image obtained from the visual sensor (2) Density change area detection means (1) for detecting a density change area using the reference image stored in the reference image storage means (12).
3), a feature amount extracting unit (14) for extracting a feature amount such as a size and a center of gravity of the density changing region detected by the density changing region detecting unit (13), and the feature amount extracting unit (14) extracts the feature amount. A characteristic amount storage unit (15) for storing the extracted characteristic amount, and a determination unit for determining the presence or absence of a moving object in the monitoring area by detecting a temporal change of the characteristic amount extracted by the characteristic amount extracting unit (14). (16) When the determination means (16) determines that there is a moving object, creates a graphic pattern model that defines the density change area detected by the density change area detection means (13) ;
A model fitting means (17) for fitting the graphic pattern model to the input image; and an image obtained as a result of fitting the graphic pattern model to the input image by the model fitting means (17) to a remote centralized monitoring center via a communication line. An image monitoring apparatus comprising an image transmission means (18) for transmitting.