JPH0737063A - Object monitoring device - Google Patents

Abstract

PURPOSE:To improve accuracy and to reduce false reports by eliminating the need of measuring and setting a feature amount for respective plural areas. CONSTITUTION:This device is provided with a storage part 5 for storing a scene map in which a monitoring area is divided into one or plural planes, a parameter storage part 10 for storing the parameter of the relation of a camera installation position, the monitoring area and an input picture, a storage part 11 for storing a reference value of the feature amount on the actual coordinates of an object, a transformation part 9A for transforming the feature amount of a fluctuation area into the feature amount on actual three-dimensional space coordinates by position relation and a camera parameter and an area deciding means 9B for deciding an object area front the reference value of the feature amount on the actual coordinates of the object and the coordinate- transformed feature amount.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an input image obtained through a television camera (hereinafter, also simply referred to as a camera) as an image pickup device and a background image thereof in a dangerous area in a factory or an unmanned building. The present invention relates to an object monitoring device using an image processing technique, which detects an abnormal occurrence such as intrusion or approach of a specific object, for example, a person or the like based on the change mode of (3) and outputs an alarm or records an image at the time of abnormal.

[0002]

2. Description of the Related Art Conventionally, as this type of object monitoring apparatus, for example, one as shown in FIG. 11 is known (Japanese Patent Laid-Open No. 4-3).
No. 40718). In the figure, 1 is a camera, 2 is an A / D converter for converting an analog video signal sent from the camera 1 into digital image data, 20 is a background memory for storing the image data as a background image, and 6 is A difference processing unit that calculates the difference between this background image and newly input image data, and a binarization processing unit 7 that binarizes the output of the difference processing unit 6 with a certain threshold value.

Reference numeral 8 denotes a feature amount extraction processing unit for extracting feature amounts such as area and barycentric position of the binarized data output from the binarization processing unit 7. Reference numeral 21 denotes a time difference of the feature amounts. A feature amount correspondence processing unit that compares and correlates processed images, 22 is a motion amount extraction processing unit that obtains a movement amount based on the result of association, and 15 is the presence or absence of an intruder based on the output from the motion amount extraction processing unit 22. A determination processing unit 16 that determines whether the intruder is present as a result of the determination, and an alarm output unit that issues an alarm when it is recognized that an intruder is present. Reference numeral 23 indicates when the determination processing unit 15 determines whether there is an intruder. It is an area-specific feature amount setting unit for setting the feature amount of each monitoring area to be used.

The operation will be described with reference to FIGS. 12 and 13. The image captured by the camera 1 is input to the A / D conversion unit 2 as an analog video signal, and the A / D conversion unit 2 converts this image data into digital image data. The background memory 20 stores this image data as background image data. The difference processing unit 6 performs a difference process between the background image data and the image data that is sequentially input for each pixel.

The binarization processing unit 7 outputs the output of the difference processing unit 6 as
Binarization is performed with a preset threshold value. With respect to this binarized image data, the feature amount extraction processing unit 8 causes the area S1 at t = 1 as shown in FIG.
Then, the feature amount such as the center of gravity position X1, Y1 is obtained. Area S
1 can be obtained by counting the number of pixels in the change area, and the centroid positions X1 and Y1 can be obtained by adding the addresses (X, Y) of the pixels in the change area and dividing by the area S1. .

Next, as shown in FIG. 12B, time t
The feature amounts such as the area S2 and the barycentric positions X2 and Y2 at = 2 are obtained in the same manner as above. The feature amount correspondence processing unit 21 compares the feature amounts, and associates the features that can be determined to be the same. For example, the area difference | S1-S
2 | is smaller than a certain value, and the moving distance d (d ² = (X1-
When X2) ² + (Y1−Y2) ² ) is also smaller than a certain value, it is determined that the same object has moved and association is performed.
The motion amount extraction processing unit 22 detects the moving speed (moving distance d divided by the time difference) and the direction of the associated changed area.

Here, it is assumed that the monitoring area is, for example, a nearby passage area A, a wall or fence area B, and a distant passage area C as shown in FIG. In this case, the size and amount of movement (direction and size) differ depending on the position (A, B, C) where the intruder enters. That is, the intruder in the nearby passage A is photographed larger than the intruder in the distant passage, and the size of the change area is large, and the amount of movement is also large. Also, the intruder's intrusion direction is from right to left toward the wall in the passages A and C, and from bottom to top of the wall B climbing up the wall.

Therefore, the monitoring area is divided into three areas A, B, and C, and the characteristic quantities 81, 82, and 83 are different for each area.
(Area: Sa, Sb, Sc and movement amount (size and direction);
Va, Vb, Vc) to the area-based feature amount setting unit 23 (Sa
> Sc, | Va |> | Vc |, etc.). When there is an image change or its movement, the feature amount of the image change (area or movement amount) is compared with the feature amount set in advance by the region-specific feature amount setting unit 23 for the region having the image change. ,
The determination processing unit 15 determines whether the change is due to an intruder. As a result, when it is judged that it is an intruder,
The alarm output unit 16 issues an alarm.

[0009]

However, the above-mentioned device has the following problems. (1) For example, even when the actual shape of the object to be extracted does not change, it is necessary to measure or calculate the reference feature amount such as the area / movement amount according to each region.
It takes time to determine the reference feature amount. Further, even if the monitoring area is slightly changed, the reference feature amount must be set again.

(2) When an object moves between areas, the change in the feature amount is large at the boundary surface of the areas, and the detection accuracy decreases. (3) The output of the judgment processing result is alarm on / off (ON,
Since it is only OFF), it is impossible to know how much the detected intruder's feature amount seems to be a true intruder, that is, how credible the judgment result is. Further, since the processing is not divided according to the credibility of the judgment result, if the intruder's judgment is rigorous, false alarms will increase, and if the false alarms are reduced, the intruder will be missed. Therefore, an object of the present invention is to eliminate the need to measure and calculate the characteristic amount such as the area / movement amount according to each region, and to provide a moving object monitoring device with high detection accuracy and few false alarms. .

[0011]

In order to solve such a problem, in the present invention, a reference image is created from one or a plurality of input images through a television camera, and the difference between the reference image and the input image is calculated. In an object monitoring device that monitors an object, a scene map storage unit that stores a scene map by dividing a monitoring area into one or a plurality of planes, and a camera that shows the positional relationship between the input image, the monitoring area, and the camera installation position. A parameter storage unit for storing parameters, and a coordinate conversion processing unit for converting the feature amount of the changing area into the feature amount on the actual three-dimensional space coordinates (actual coordinates) according to the positional relationship on the scene map and the camera parameter. It is characterized in that a region determining means for determining the target object region from the reference value of the feature amount in the actual coordinates of the target object and the coordinate-transformed feature amount is provided.

According to the second aspect of the present invention, 1 through the television camera is used.
A reference image is created from one or a plurality of input images, and in an object monitoring device that monitors an object based on the difference between the reference image and the input image, the feature amount of the area determined as the target object and its reference value An object certainty factor calculating means for calculating the degree of matching, an object matching certainty factor calculating means for calculating the degree of matching between the object position predicted from the past matching result and the object position actually obtained, and A target motion certainty factor calculating means for calculating a target motion likelihood for each target motion from the obtained object position trajectory and the object correspondence certainty factor is provided, and an alarm is issued based on each value of the object certainty factor and the target motion certainty factor. It is characterized by changing the pattern of.

In the first aspect of the invention, the object certainty factor calculation means for calculating the degree of coincidence between the feature amount of the area determined as the target object and its reference value, and the prediction result from the past association result. The object association certainty factor calculating means for calculating the degree of coincidence between the object position and the actually obtained object position, and the target action likelihood for each target action from the actually obtained loci of the objects and the object association certainty factor. By adding a target motion certainty factor calculating unit for calculating the, the alarm pattern can be changed based on each value of the object certainty factor and the target motion certainty factor.

[0014]

In order to determine whether the change area is the object to be monitored, the feature amount on the image of the change area is actually coordinated based on the constraint condition that the object is constrained on the surface of the stored scene map. By converting into the above feature amount and comparing this with a preset value, it is possible to eliminate the need to set a number of feature amount set values for each region and to perform detection with high accuracy. Also, an object extraction processing unit, an object association processing unit,
By obtaining the numerical value of the probability of the results together with the processing results of the target object tracking processing unit and the target motion recognition processing unit, it becomes possible to perform judgment processing and alarm output based on this certainty To reduce.

[0015]

1 is a block diagram showing an embodiment of the present invention. 2 is an explanatory diagram for specifically explaining the operation of FIG. 1, where (a) is a reference image, (b) is a scene map, and FIG.
(C) and (d) show input images at times t1 and t2, respectively. 2A and 2B are times t1 and t, respectively.
The target object in 2 is shown. Further, as shown in FIG. 3, it is assumed that the camera 1 as an imaging device is installed diagonally above the monitoring area.

First, an image is input from the camera 1 off-line, and digital image data (hereinafter, simply referred to as image data) obtained through the A / D conversion unit 2 is set on each of the ground plane, the vertical plane, and each area outside the monitoring area. The scene map as shown in FIG. 2B is created by division and stored in the scene map storage unit 5. In FIG. 2B, as an example, the values of horizon = 001H, vertical surface = 02H, and outside monitoring area = 00H are taken. Further, the camera parameters of FIG. 3 are obtained in advance and stored in the camera parameter storage unit 10.

The operation in the online state will be described. First, one or a plurality of images obtained from the camera 1 through the A / D conversion unit 2 is input to the reference image creation unit 3, and the image shown in FIG.
A reference image as shown in (a) is created. The obtained reference image is stored in the reference image storage unit 4. The camera 1 in the difference processing unit 6
And the input image obtained through the A / D conversion unit 2 and the reference image stored in the reference image storage unit 4 are input, and a difference image between them is created. At this time, the scene map storage unit 5
In the scene map stored in (1), a difference image is unconditionally set as "no change" for an area set as outside the monitoring area. The obtained difference image is binarized by the binarization processing unit 7.
, And binarize with a preset threshold value to obtain binary image data.

Next, the binary image data is input to the feature amount extraction processing section 8. The feature amount extraction processing unit 8 first labels the binary image data and calculates the feature amount (width, height, area, representative point coordinates, etc. of the circumscribed rectangle). The labeled image data has a plurality of labels L1, L2, which are originally one object, as shown in FIG.
It may be divided into L3. In this way, the extraction processing unit 9 performs a process of integrating the regions divided into a plurality into one and extracting the regions with a size and shape close to the target object.

Various processing methods can be considered in the extraction processing unit 9, but here, as reference values such as width, height, and area for integrating regions, the coordinate system of the camera 1 (hereinafter, simply referred to as camera) is used. The value in the coordinate system) is not used, but the value in the camera coordinate system converted to the actual coordinate system is used.
For example, when the representative point of the label L1 in FIG. 4A is the midpoint P1 (x1, y1) of the lower side of the circumscribed rectangle, the point P1 is on the horizon on the scene map, and therefore the actual coordinates of the point P1. Can be represented as (X1, Y1, 0). here,
It is considered that the label L1 is on the XZ plane of Y = Y1, and the feature quantities such as the width, height, and area of the label L1 in the actual coordinates in this case are calculated using the coordinate conversion processing means 9A.

First, the conversion formula from the camera coordinates (x, y) of a point existing on the ground plane to the real coordinates (X, Y, Z) is shown in FIG.
From Is represented.

An enlarged display of the label L1 of FIG. 4 (a) is shown in FIG. 4 (b). The same figure (b) shows the point P1a (x1
a, y1a), point P1b (x1b, y1b), point P1c
Let (x1c, y1c) be the lower left point, lower right point, and upper middle point of the circumscribed rectangle of the label L1, respectively. If the real coordinates (X1a, Y1a, 0) and (X1b, Y1b, 0) of P1a and P1b of the label L1 in FIG. 4 (b) are calculated using the equation (1), these coordinate values are calculated as follows: The width W1 of the label L1 in actual coordinates can be calculated as W1 = | X1b−X1a | (2).

The camera coordinates (x1c, y) of the point P1c
1c) to the real coordinates (X1c, Y1c, Z1c), the conversion formula is X1c = X1, Y1c = Y1 Therefore, the height H1 of the label L1 in the actual coordinates can be obtained as H1 = | Z1c-Z1 | = | Z1c-0 | = | Z1c | (4).

The area value of the label L1 in camera coordinates is s
1. If the area value in the real coordinates is S1, S1 is, for example, S1 = s1.W1.H1 / w1.h1 (5) (where w1 = | x1b-x1a + 1 |, h1 = | y
1c-y1 + 1 |).

Similarly, the feature amount in the real coordinate when the labels L1, L2 are combined, the feature amount in the real coordinate when the labels L1, L2, L3 are combined, and the like are calculated. The area determining means 9B compares this with a reference value of the feature quantity such as width, height, and area in the actual coordinates of the target object preset in the feature quantity storage unit 11, and determines a value closest to the target object. Decide which label to combine to take. For example, in FIG. 4A, the labels L1, L2 and L3 are combined into one area, and the label L4 is another area.

In the above, the case of Z = 0 has been described, but when Z = 0 is not satisfied, the above equation (1) may be changed to the following equation (6).

In the correspondence processing unit 12, two consecutive times t.
= 1 and t = 2, the same object is associated with the result obtained by the extraction processing unit 9. At this time, when there are a plurality of changing regions at the times t = 1 and t = 2, the changing regions having the closest positions in the real coordinates at both times are associated with each other as the same object. In the tracking processing unit 13, a plurality of consecutive 2 obtained in the correspondence processing unit 12
The position change of the real coordinates of the same object at more than two times is calculated using the time correspondence result. The target motion recognition processing unit 14 compares the change in the position data of the real coordinates obtained by the tracking processing unit 13 with the position change in the real coordinates in the preset detection target motion, and outputs the degree of coincidence. To do.

The determination processing unit 15 performs a process corresponding to the action having the highest detection priority among the actions likely obtained by the target action recognition processing unit 14. For example, if the detection priority of the intruding action is set higher in the intruding or approaching action of the two objects 18A and 18B into the intrusion prohibited area as shown in FIG. 5, the alarm output unit is based on this intruding action. Output to 16 and image recording to VTR (video tape recorder) 17 are instructed.

FIG. 6 is a block diagram showing another embodiment of the present invention. In the extraction processing unit 9, the confidence factor calculation means 9C,
The correspondence processing unit 12 is characterized in that a target object correspondence certainty degree calculation unit 12A and a target motion recognition processing unit 14 are additionally provided with a target motion certainty degree calculation unit 14A. is there. That is, the certainty factor calculation unit 9C of the extraction processing unit 9 determines whether the shape and size of the target object are the same as the target object based on the degree of coincidence between the feature amount (height, width, area, etc.) of the obtained target object in the actual coordinates and the reference value. Target object certainty factor P
Calculate o. As a method for calculating the object certainty factor Po, feature amounts such as height, width, ratio of height to width, and area of the obtained target object in actual coordinates are expressed by, for example, a fuzzy membership function, and the membership of these is expressed. There is a method of synthesizing functions to obtain Po.

On the other hand, the correlation certainty factor calculating means 12A of the correlation processing unit 12 calculates the correlation certainty factor Pt from the difference between the predicted position and the actual position of the target object. Time t = 1,
The target object predicted position at time t = 3 calculated from the target object position at t = 2 is determined from the moving speed of the target object in real coordinates and the state of the scene map on the moving route of the target object. It For example, as shown in FIGS. 7A and 7B, time t
It is assumed that the positions of the object at = 1 and t = 2 are points P1 and P2, and both are in the area of the horizon on the scene map shown in FIG. Considering the motion of an object as a linear motion at constant velocity,
From the actual coordinate values of the points P1 and P2, the actual coordinate value of t = 3 is expected to be the position of the point P3 in FIGS. 7A and 7B.

Here, when the change of the scene map on the straight line connecting the points P2 and P3 is examined, since it does not change on the horizontal plane in FIG. 7A, the position of the point P3 remains at the time t.
= 3 is the expected position of the target object. On the other hand, FIG. 7 (b)
At point P3, it changes from the horizontal plane to the vertical plane. Therefore, in this case, the predicted position of the target object at t = 3 is the intersection P3 ′ of the horizontal plane and the vertical plane. Here, the motion of the object is assumed to be a constant-velocity linear motion, and the predicted position at the next time is calculated. However, other motions may be performed as long as the movement path and the position of the object can be calculated.

The target motion certainty factor calculating means 14A of the target motion recognition processing section 14 uses the change of the object position in the real coordinates at each time and the associated certainty factor Pt obtained in the preceding stage.
The certainty factor Pm associated with the target motion is calculated. As an example thereof, a case where an approaching motion of a target object to a certain area 19 as indicated by an arrow in FIG. 8 is detected will be described. If the approaching movement is 1 and the other is 0, the distance d between the position of the real coordinate of the target object and the area 19 and the change amount Δd of the distance are obtained, and the distance is short and The larger the amount of change in the approaching distance is, the higher the possibility of approaching the region is, and the value close to 1 may be output as Pm.

Therefore, for the distance d and the distance change amount Δd, membership functions Pmd and PmΔd for outputting the values as shown in FIGS. 9A and 9B are set, respectively, and these two functions are associated with certainty. And Pt. As a synthesizing method, for example, there is a method such as Pm: max (Pmd, PmΔd) · Pt (d <d0) Pmd · PmΔd · Pt (d0 ≦ d). When it is desired to monitor the motions of a plurality of target objects at the same time, the calculation method of the target motion certainty factor Pm is set for each motion.

In the judgment processing unit 15, an alarm output level corresponding to both the object certainty factor Po and the action certainty factor Pm is set in advance for each operation, and the alarm output level is determined by this set value. The processing corresponding to the obtained alarm output level is performed. For example, as shown in Table 1, four levels of alarm output levels from 0 to 3 are set, and the operation (a) (P
o, Pm) → Correspondence of alarm output level and operation "B"
It is assumed that the correspondence of (Po, Pm) → alarm output level is set as shown in FIGS. 10 (a) and 10 (b), respectively. Further, it is assumed that the priority level of the detection operation is set as shown in Table 2.

[0034]

[Table 1]

[Table 2]

At a certain time, Po = 0.8 of the object 18A,
Pm of the motion "a" is 0.7, Pm of the motion "b" is 0.
6, Po of object 18B = 0.6, Pm of motion "a" =
In the case of 0.3 and Pm = 0.5 of the motion "b", the object 1
The alarm output level of 8A is 2.2, the alarm output level of the object 18B is 1.1, and the highest alarm output level is "2". This is a level 2 process from Table 1, that is, an alarm is output only to the observer, and the VTR
In step 17, a process of recording the image at that time with low quality is performed. Further, the operation that gives the maximum value of the alarm output level at this time is the operation "b" from the priority order of Table 2.

When it is desired to add / delete a motion to be detected, a method of calculating the motion certainty factor Pm and (Po,
Pm) → alarm output level correspondence need only be added / deleted. Further, the detection priority of a plurality of operations can be changed only by changing the priority of Table 2. Furthermore, if the values of the object certainty factor Po and the motion certainty factor Pm at this time are also recorded together, even if an erroneous report is made, it can be used as data for knowing the cause, and this data can also be obtained. In addition, by automatically improving the reference value of the feature amount and the operation definition, it is possible to make the system suitable for the installation environment and with few false alarms.

It should be noted that the extraction processing unit 9 of FIG. 6 has only the certainty factor calculation means 9C, the correspondence processing unit 12 has only the correspondence certainty factor calculation unit 12A, and the target motion recognition processing unit 14 has the target motion certainty degree calculation unit 14A. It is needless to say that each of them may be configured by only the above.

[0038]

According to the present invention, in order to determine whether or not a change area is an object to be monitored and whether its movement is a motion to be monitored, each feature quantity (width) on the image of the change area is determined. , Height, area, movement amount, etc.) are converted into feature values on the real coordinates and these are compared with preset values, so it is necessary to set several feature value reference values for each region. Since the standard value can be used, the standard value used by the observer or system engineer can be used, so the setting becomes easy. Also, for example, if not only the judgment result of whether or not an object has entered, but also the degree of certainty of the result is obtained, the subsequent processing can be changed based on this certainty. Therefore, it is possible to reduce false alarms, which is an advantage.

[Brief description of drawings]

FIG. 1 is a block diagram showing an embodiment of the present invention.

FIG. 2 is an explanatory diagram for explaining a scene map.

FIG. 3 is an explanatory diagram for explaining a coordinate conversion method.

FIG. 4 is an explanatory diagram for explaining an object extraction process according to the present invention.

FIG. 5 is an explanatory diagram for explaining a target motion recognition process according to the present invention.

FIG. 6 is a block diagram showing another embodiment of the present invention.

FIG. 7 is an explanatory diagram for explaining an expected position.

FIG. 8 is an explanatory diagram for explaining a target motion certainty factor.

FIG. 9 is a graph showing an example of a membership function.

FIG. 10 is an explanatory diagram illustrating a determination processing operation.

FIG. 11 is a block diagram showing a conventional example.

12 is an explanatory diagram for explaining the operation of FIG. 11. FIG.

FIG. 13 is an explanatory diagram for describing the feature amount for each area in FIG. 11.

[Explanation of symbols]

1 ... Camera, 2 ... A / D converter, 2A, 2B, 18A,
18B ... Target object, 3 ... Background image creation unit, 4 ... Background image storage unit, 5 ... Scene map storage unit, 6 ... Difference processing unit, 7 ...
Binarization processing unit, 8 ... Feature amount extraction processing unit, 9 ... Target object extraction processing unit, 9A ... Coordinate conversion processing unit, 9B ... Region determination unit, 9C ... Certainty factor calculation unit, 10 ... Camera parameter storage unit, 11 ... feature quantity storage unit, 12 ... association processing unit, 1
2A ... Correlation certainty factor calculation means, 13 ... Tracking processing unit, 1
4 ... Target motion confirmation processing unit, 14A ... Target motion certainty degree calculation means, 15 ... Judgment processing unit, 16 ... Warning output unit, 17 ... V
TR, 19 ... Intrusion prohibited area, 20 ... Background memory, 21 ...
Feature amount correspondence processing unit, 22 ... Motion amount extraction processing unit, 23 ...
Region-specific feature amount setting unit.

─────────────────────────────────────────────────── ─── Continuation of the front page (51) Int.Cl. ⁶ Identification code Internal reference number FI Technical indication location G08B 13/196 7323-5G H04N 7/18 D (72) Inventor Akira Shimizu Kawasaki-ku, Kawasaki-shi, Kanagawa Prefecture No. 1 Shinden Tanabe Fuji Electric Co., Ltd.

Claims (3)

[Claims] 1. An object monitoring apparatus that creates a reference image from one or a plurality of input images through a television camera and monitors an object based on the difference between the reference image and the input image. A scene map storage unit that divides into a plurality of planes and stores a scene map, a parameter storage unit that stores a camera parameter indicating a positional relationship among the input image, the monitoring area, and the camera installation position, and a feature amount of the change area in the scene. Coordinate conversion processing means for converting into a feature amount on the actual three-dimensional space coordinates (real coordinates) according to the positional relationship on the map and the camera parameters, and the coordinate conversion with the reference value of the feature amount at the actual coordinates of the target object. An object monitoring apparatus, comprising: a region determining unit that determines a target object region from the feature amount. 2. An object monitoring apparatus that creates a reference image from one or more input images through a television camera and monitors the object based on the difference between the reference image and the input image, and determines the target object. Object confidence factor calculation means for calculating the degree of coincidence between the feature amount of the area and its reference value, and object correspondence for calculating the degree of coincidence between the object position predicted from the past correspondence result and the object position actually obtained Attaching certainty factor calculating means, and a target motion certainty factor calculating means for calculating the target motion likelihood for each target motion from the trajectory of the object position actually obtained and the object association certainty factor, and the object certainty factor An object monitoring device characterized by changing an alarm pattern based on each value of a target motion certainty factor. 3. An object certainty factor calculation means for calculating a degree of coincidence between a feature amount of a region determined as the target object and its reference value, and an object position predicted from a past association result and actually obtained. Object matching certainty factor calculating means for calculating the degree of coincidence with the object position, and target motion for calculating the target motion likelihood for each target motion from the actually obtained trajectories of the objects and the object matching certainty factors. Confidence factor calculation means is added,
The object monitoring apparatus according to claim 1, wherein the alarm pattern is changed based on each value of the object certainty factor and the target motion certainty factor.