JP5334008B2 - Abnormal operation detection device - Google Patents

Description

  The present invention relates to an abnormal operation detection device that detects an abnormal operation (such as rampage) of a person from an image taken by a surveillance camera, for example, in an elevator car.

  Conventionally, as a method for detecting an abnormal movement of a person from an image taken by a surveillance camera, a specific part of the body such as a human head is detected from the photographed image, and the specific part is tracked in time and space. There is a method. However, since this method requires that a specific part of the body be photographed by the surveillance camera, it cannot be used in a narrow closed space where the installation location of the surveillance camera is limited, such as an elevator car.

Therefore, in an elevator, a method of determining a rampage operation by using two images taken by a surveillance camera and detecting the direction and size of the movement of each point on the image is used (for example, patent document). 1).
JP 2006-276969 A

  Normally, the movement of a person who is not rampant is a constant speed movement, and when it is rampant, it becomes an acceleration movement. In the method of detecting the movement of each point between two images as in Patent Document 1, it is not possible to analyze the movement of a person in detail including the acceleration motion. For this reason, if a person simply makes a large movement in the car, there is a possibility that it will be erroneously determined as a violent movement, and there is a problem in accuracy.

  The present invention has been made in view of the above points, and an object of the present invention is to provide an abnormal operation detection device that can detect an abnormal operation of a person with high accuracy using an image taken by a surveillance camera. .

The abnormal operation detection apparatus according to the present invention uses a monitoring camera that continuously captures an elevator car as a monitoring area at predetermined frame intervals, and at least three or more consecutive images captured by the monitoring camera. A motion data calculating means for calculating motion data including an acceleration element at each point above, and a statistical value representing a temporal change in the motion data at each point of each image calculated by the motion data calculating means. A first statistical value calculating means for calculating; a second statistical value calculating means for calculating a statistical value representing a spatial spread of motion data at each point of each image; and the first and second statistical values. An evaluation value calculation means for calculating an evaluation value for determining abnormal operation by multiplying the statistical value calculated by the calculation means by a predetermined weighting factor, and calculated by this evaluation value calculation means Comparing the preset threshold value, and determines the operation determination means and that person is abnormal operation in the monitoring area when the state in which the evaluation value exceeds the threshold value continues for a predetermined time, the An image of the door part of the car is extracted from the image taken by the surveillance camera, and door opening / closing detecting means for detecting the opening / closing operation of the car door from the image change, and the door opening / closing detecting means And a determination timing control means for controlling the determination timing of the abnormal operation by the operation determination means based on the opening / closing operation of the door.

  According to the present invention, the motion data of each point is calculated using at least three or more consecutive images, and the motion data is analyzed temporally and spatially, so that abnormal human movements can be accurately performed. Can be detected.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
(First embodiment)
FIG. 1 is a block diagram showing a configuration of an abnormal operation detecting apparatus according to the first embodiment of the present invention.

  As shown in FIG. 1, the abnormal motion detection apparatus according to the present embodiment includes a monitoring camera 10, a change portion extraction unit 11, a motion data calculation unit 12, a temporal statistical value calculation unit 13, a spatial statistical value calculation unit 14, and an evaluation. A value calculation unit 15 and an operation determination unit 16 are provided.

  The monitoring camera 10 is installed in a monitoring area, and continuously captures images in the monitoring area at predetermined frame intervals. In the present embodiment, as shown in FIG. 2, an elevator car 21 is assumed as the monitoring area.

  The elevator car 21 is provided with a door 22 that opens and closes when landing, and an operation panel 23 is provided beside the door 22. In addition to the destination floor button 24 for designating the destination floor, the operation panel 23 is provided with a door opening button 25 for instructing door opening, a door closing button 26 for instructing door closing, and the like. . In addition, an emergency call button 27 is provided for contacting the outside in an emergency.

  Here, the surveillance camera 10 is installed on the ceiling surface of the car 21, for example. The installation location of the monitoring camera 10 is not limited to the ceiling surface, and may be any location as long as the movement of a person in the car 21 can be monitored.

  The change portion extraction unit 11 sequentially compares each image captured by the monitoring camera 10 in units of frames, and extracts a changed portion (that is, a portion where a person has moved) in the captured image.

  The motion data calculation unit 12 uses at least three or more continuous images, uses the change portion extracted by the change portion extraction unit 11 as a processing target, and data related to the motion of each point on the image (hereinafter, “motion”). Data)). Examples of the motion data include “velocity”, “acceleration”, “angular velocity”, “correlation value”, “approximate angular velocity”, and the like.

  The temporal statistical value calculation unit 13 calculates a statistical value indicating a temporal change in the motion data of each point calculated by the motion data calculation unit 12. Specifically, a value obtained by averaging or dispersing the motion data of each point obtained from a predetermined number of images in chronological order in the time axis direction is calculated as a temporal statistical value.

  The spatial statistical value calculation unit 14 calculates a statistical value indicating the spatial spread of the motion data of each point. Specifically, a value obtained by averaging or distributing the motion data of each point of each image over the entire image (the entire frame screen) is calculated as a spatial statistical value.

  The evaluation value calculation unit 15 multiplies the statistical value calculated by the temporal statistical value calculation unit 13 and the spatial statistical value calculation unit 14 by a predetermined weighting factor to calculate an evaluation value for determining an abnormal operation.

  The operation determination unit 16 compares the evaluation value calculated by the evaluation value calculation unit 15 with a preset threshold value to determine abnormal operation (roughness). Specifically, the abnormal operation is determined when the state where the evaluation value exceeds the threshold value continues for a predetermined time. In addition to the presence or absence of abnormal operation, it is possible to output a continuous numerical value as the abnormal operation amount.

Next, the operation of this embodiment will be described.
FIG. 3 is a flowchart showing the flow of processing by the abnormal operation detection apparatus in this embodiment.

  First, each image taken continuously by the monitoring camera 10 at a constant interval (or interval T) is taken in time-sequential order (step S11). Note that t in the following “frame (t)” indicates a sampling number. The frame rate is usually 30 frames / second, but may be as low as 10 frames / second.

  Here, as shown in FIG. 4, the change portion extraction unit 11 performs a difference calculation between the frame (t−1) and the frame (t) in order to extract the movement of a person or an object from the captured image, and changes. Extract the part of the image. The portion A in the figure shows the image of the changed portion obtained by the difference calculation.

  Usually, as shown in FIG. 5, the number of points to be processed (hereinafter referred to as measurement points) in the image is fixedly determined in advance according to the size of the image, but only the changed portion is processed. By making it a target, it is possible to suppress the amount of calculation of statistical values described later.

  Next, in the motion data calculation unit 12, the movement of the measurement points narrowed down by the change portion extraction unit 11 using at least three images of the current image, the image before one frame, and the image before two frames. Data is obtained (step S13).

  As described above, the motion data includes “speed”, “acceleration”, “angular velocity”, “correlation value”, and the like. The reason why the motion data is obtained using at least three or more images is to calculate not only “speed” but also an acceleration factor. In other words, since the movement of a person who is not rampant usually moves at a constant speed, it is meaningless to examine only the speed. Therefore, it is assumed that motion data including “acceleration” is calculated using three or more images.

  Further, as a method for estimating the position of the measurement point Q on the image of the frame (t-1) moved to the image of the frame (t), the following method is used.

  That is, for a certain measurement point Q, a template in which a part of the image of the frame (t−1) is cut out is created, and the similarity is calculated while scanning the search range around the measurement point Q of the frame (t). Then, a position after the frame (t) is estimated by searching for a portion having the highest similarity.

  Next, the temporal statistical value calculation unit 13 calculates the temporal statistical value of the measurement point (step S14). This temporal statistical value is obtained by averaging or distributing the motion data of measurement points in units of n frames.

  Now, the case where the average value of acceleration and the average value of correlation values are calculated as temporal statistical values will be described in detail.

(1) Average value of acceleration The average value of acceleration tends to be small when a person moves at a constant speed, but it is effective for discriminating abnormal movement in cases where the speed changes continuously. is there.

If the coordinate position in the space of the measurement point Q in the frame (t) is q (t), and the coordinate position in the space of the measurement point Q one frame before is q (t−1), the motion vector of the measurement point Q is The magnitude Vq (t) is
Vq (t) = q (t) -q (t-1)
It becomes.

If the coordinate position of the measurement point Q two frames before is q (t−2), the magnitude Vq (t−1) of the motion vector at the measurement point Q is
Vq (t-1) = q (t-1) -q (t-2)
It becomes.

  Here, assuming that the change in the movement of the measurement point Q at time t, that is, the acceleration is αq (t), the following equation is obtained.

αq (t) = Vq (t) −Vq (t−1)
Further, when an average value of accelerations between n frames at time t (a value obtained by averaging n accelerations on the time axis) is Mαq, the following equation is obtained. Mαq represents the average value M of the acceleration α at the point q.

Mαq = Σ (αqi) / n
However, i = t to (t−n + 1), n is the number of frames.

(2) Average value of correlation values The average value of correlation values of motion vectors between frames is inversely proportional to the amount of rampage.

  As shown in FIGS. 6 and 7, the vector of the measurement point Q in the frame (t) is V1 = (x1, y1), the magnitude is dv1, and the vector in the frame (t−1) is V2 = (x2, y2). ), And the size is dv2.

Here, when the change amount (correlation value) of the direction component of the motion vector between the frame (t−1) and the frame (t) is cos θ,
cos θ = (x1x2 + y1y2) / dv1dv2
It becomes.

  Further, when Mcθq is an average value of correlation values between n frames at time t (a value obtained by averaging n correlation values on the time axis), the following equation is obtained.

Mcθq = Σ (cosθqi) / n
However, i = t to (t−n + 1), n is the number of frames.

  In addition, for example, an average value of angular velocity (rotational speed) or pseudo angular velocity (pseudo rotational speed) may be obtained.

  That is, if the angular velocity is ω and the average value of the angular velocities between n frames at time t is Mωq, it can be expressed as follows.

ω = θ / t
Mωq = | Σ (θqi) | / n
However, i = t to (t−n + 1), θ is 0 to 180 degrees, and n is the number of frames.

  Further, as shown in FIG. 8, the pseudo angular velocity is obtained by modifying the correlation value cos θ and making it proportional to the amount of rampage. Assuming that the pseudo angular velocity at time t is Cωq (t) and the average value of n pseudo angular velocities on the time axis of the measurement point Q at time t is MCωq, it can be expressed as follows.

Cωq (t) = 1−COSθ
MCωq = | Σ (Cωqi) | / n
However, i = t to (t−n + 1), θ is 0 to 180 degrees, and n is the number of frames.

  The average value of the angular velocity ω and the pseudo angular velocity Cω is proportional to the amount of rampage, and tends to increase in the case of abnormal operation that repeatedly changes direction.

  Next, the spatial statistical value calculation unit 14 calculates the spatial statistical value of the measurement point Q (step S15). This spatial statistical value represents the spatial extent of the measurement point Q, and is calculated using the statistical value calculated by the temporal statistical value calculation unit 13.

That is, when the average value of the average value Mαq on the time axis of the acceleration αq (t) at the time t at the q point is X1 (t), it can be obtained as follows.
X1 (t) = ΣMαq / m
However, m = number of all target measurement points, q = 1 to m; measurement points.

Further, when the average value of the entire image of the pseudo angular velocity Cωq (t) at time t is X2 (t), it is obtained as follows.
X2 (t) = ΣMcωq / m
However, m = number of all target measurement points, q = 1 to m; measurement points.

  Next, the evaluation value calculation unit 15 calculates an evaluation value of abnormal operation (step S16). This evaluation value is calculated by a linear expression obtained by multiplying the statistical value finally obtained by the spatial statistical value calculation unit 14 by a predetermined weight coefficient.

  That is, when the evaluation value of the abnormal operation at time t is Y (t), it is calculated by the following linear function equation.

Y (t) = a1 * X1 (t) + a2 * X2 (t) + a3 * X3 (t) +.
Here, X1 (t), X2 (t), X3 (t)... Are statistical values at each time point t obtained by the spatial statistical value calculation unit 14. Further, a1, a2, a3... Are weighting factors, and b is a constant.

  Note that the weighting coefficient is obtained in advance from an abnormal operation image (an image taken when a person is operating abnormally) and a normal operation image (an image taken when a person is performing a steady operation). The statistical value is set based on the result of multiple regression analysis or discriminant analysis. For example, if it is only divided into two categories of abnormal and normal, discriminant analysis is superior. However, when the level of abnormality is set according to the size of the abnormality, the classification is 3 or more, so regression analysis is suitable. Actually, it is classified into four categories of normal and abnormal levels 1, 2, and 3.

  The evaluation value Y calculated by the evaluation value calculation unit 15 is given to the operation determination unit 16. The operation determination unit 16 compares the evaluation value with a preset abnormality determination threshold value to determine whether or not there is an abnormal operation (step S17).

  In this case, as shown in FIG. 9 and FIG. 10, it is assumed that the movement exceeding the threshold value is not detected, but the movement exceeding the threshold is detected continuously. Note that, as indicated by the one-dot chain line in FIG. 10, even if there is a slight break, a time longer than the threshold value may be measured and detected, for example, if 3 seconds out of 5 seconds exceed the threshold value. The operation determination unit 16 determines that a person is operating abnormally in the car 21 when the evaluation value exceeds the threshold value for a predetermined time or longer (Yes in step S18) (step S19).

  When it is determined that the operation is abnormal, a notification to that effect is given to the building monitoring room, and the car 21 stops at the nearest floor and opens.

  As described above, the acceleration movement of each point is obtained from at least three time-series images, and the abnormal operation is determined based on the temporal change and spatial spread of these points. Thus, it is possible to more accurately detect a subtle movement when a person is rampaging in the car 21.

  That is, according to the embodiment of the present invention, the statistical value indicating the temporal change of the motion data of each point is calculated by the temporal statistical calculation unit 13 and thus cannot be detected only by the magnitude of the motion vector. Rapid movement of passengers can be detected. Further, in addition to the temporal statistical calculation unit 13, the spatial statistical calculation unit 14 calculates a statistical value indicating the spread of the motion data of each point in the entire image, so that one frame image can be obtained at a certain time. It is possible to detect how much the ratio is changing greatly, and it is possible to accurately detect the movement of the passenger expected to be captured at a certain ratio for each frame.

  Thus, by providing both the temporal statistical calculation unit 13 and the spatial statistical calculation unit 14, it is possible to more accurately detect a sudden movement of a passenger as a rampage operation.

  Note that the conventional method of analyzing the movement of each point (the direction and magnitude of the movement of each point) using two images is erroneously determined as a violent action even when a person moves greatly. However, if the method of the present invention is used, it is possible to eliminate such erroneous determination and detect only an abnormal operation with high accuracy.

(Second Embodiment)
Next, a second embodiment of the present invention will be described.
In the second embodiment, a weighting factor used for calculating an evaluation value or a threshold value that is a criterion for determining an abnormal operation is corrected in accordance with a shooting situation of a monitoring target.

  FIG. 11 is a block diagram showing a configuration of an abnormal operation detecting apparatus according to the second embodiment of the present invention. In addition, the same code | symbol is attached | subjected to the same part as the structure of FIG. 1 in the said 1st Embodiment, and the description shall be abbreviate | omitted.

  In the second embodiment, the abnormal operation detection device is provided with a load detection unit 31 and a correction unit 32 in addition to the configuration of FIG. The load detection unit 31 detects the loaded load of the car 21 that is a monitoring target. Specifically, a load detection sensor is installed at the bottom of the car 21, and the current load value is detected by a signal from the sensor. Based on the load value detected by the load detection unit 31, the correction unit 32 corrects the weighting coefficient used in the evaluation value calculation unit 15 or the threshold used in the operation determination unit 16 to an optimum value.

  In the image taken by the monitoring camera 10, even if a person performs the same operation, the value of the abnormal operation evaluation amount Y varies depending on the number of people who are in the car 21 at that time. For example, as shown in FIGS. 12 (a) and 12 (b), for example, when one person is moving in the car 21 and when two persons are moving, two persons are moving. However, since the amount of movement of each point on the image becomes larger, the evaluation value tends to increase.

  Therefore, when the load value detected by the load detection unit 31 is larger than a standard value (for example, a load value equivalent to one person), the correction unit 32 decreases the weighting factor stepwise according to the load value at that time. Alternatively, the threshold value is raised step by step. As a result, it is possible to more accurately detect abnormal operations while suppressing variations in evaluation values depending on the number of passengers.

  Note that the statistical value also changes depending on the distance from the monitoring camera 10 to the subject (the person who is the target of the abnormal motion detection). That is, as shown in FIGS. 13A and 13B, even if the same operation is performed in the car 21 when the distance from the monitoring camera 10 to the subject is short, the same operation is performed on the monitoring camera 10. Since the amount of movement of each point on the image increases as the distance increases, the evaluation value increases. Conversely, the evaluation value tends to decrease as the distance from the monitoring camera 10 increases.

  Therefore, in addition to the load detection unit 31, a distance estimation unit 33 that estimates the distance from the monitoring camera 10 to the subject is provided, and a weighting coefficient or a threshold value is corrected based on the distance estimated by the distance estimation unit 33. And

  As a method for estimating the distance from the monitoring camera 10 to the subject, the area of the changed portion is obtained by calculating the difference between the reference image and the captured image of the car 21 taken in advance when there is no person, and the area of the changed portion is calculated. There is a method to estimate based on. In this case, it can be estimated that the larger the area of the changed portion is, the closer the distance is, and the smaller the area is, the farther the distance is.

  If the distance estimated by the distance estimation unit 33 is larger than the standard value (for example, the center position in the car), the correction unit 32 reduces the value of the weighting factor stepwise according to the length at that time. Alternatively, the threshold value is raised step by step. Thereby, it is possible to more accurately detect an abnormal operation while suppressing variations in evaluation values due to the distance to the subject.

  When there are a plurality of people in the car 21, the positional relationship between these people and the monitoring camera 10 is substantially the same. This is based on the premise that a person's rampage is detected in the present embodiment. For example, when there are two passengers in the car 21, when one of the two passengers is rampant, This is because they are considered to be close.

(Third embodiment)
Next, a third embodiment of the present invention will be described.
In the third embodiment, when an elevator car is a monitoring target, the determination timing of the abnormal operation is controlled based on the opening / closing operation of the door of the car.

  FIG. 14 is a block diagram showing a configuration of an abnormal operation detecting apparatus according to the third embodiment of the present invention. In addition, the same code | symbol is attached | subjected to the same part as the structure of FIG. 1 in the said 1st Embodiment, and the description shall be abbreviate | omitted.

  In the third embodiment, the abnormal operation detection device is provided with a door opening / closing detection unit 41 and a determination timing control unit 42 in addition to the configuration of FIG. The door opening / closing detector 41 detects the opening / closing operation of the door 22 of the car 21 shown in FIG. Specifically, a door sensor is provided at a predetermined location of the door 22, and an opening / closing operation is detected based on a signal from the door sensor.

  As another method, an image of a portion of the door 22 may be extracted from an image taken by the monitoring camera 10, and an opening / closing operation may be detected from the image change.

  When the door 22 of the car 21 is open, there is a person going in and out, and since the car 21 is not in a closed room state, there is no point in detecting an abnormal operation. Therefore, the door opening / closing detection unit 41 detects the opening / closing operation of the door so that the abnormal operation is detected only after the door 22 is closed until the door 22 is opened on the next destination floor. The operation of the operation determination unit 16 is controlled by. Thereby, useless detection operation can be suppressed and abnormal operation can be detected efficiently only when the car 21 is in a closed room.

  In each of the above-described embodiments, the elevator car has been described as an example of the monitoring target. However, the present invention can be applied to any closed space that requires detection of abnormal operation.

  In short, the present invention is not limited to the above-described embodiments as they are, and can be embodied by modifying the components without departing from the scope of the invention in the implementation stage. In addition, various forms can be formed by appropriately combining a plurality of constituent elements disclosed in the above embodiments. For example, some components may be omitted from all the components shown in the embodiment. Furthermore, constituent elements over different embodiments may be appropriately combined.

FIG. 1 is a block diagram showing a configuration of an abnormal operation detecting apparatus according to the first embodiment of the present invention. FIG. 2 is a diagram illustrating an example of an internal configuration of an elevator car that is monitored by the abnormal operation detection device according to the embodiment. FIG. 3 is a flowchart showing the flow of processing by the abnormal operation detection apparatus in the embodiment. FIG. 4 is a diagram for explaining a method for extracting a changed portion by calculating a difference between frames in the embodiment. FIG. 5 is a diagram for explaining the narrowing down of measurement points by the difference calculation between frames in the embodiment. FIG. 6 is a diagram for explaining motion detection between frames in the embodiment. FIG. 7 is a diagram showing motion vectors between frames in the same embodiment. FIG. 8 is a diagram for explaining a method of calculating the pseudo angular velocity Cω in the same embodiment. FIG. 9 is a diagram for explaining a method for determining an abnormal operation in the embodiment, and is a diagram illustrating a change in evaluation value when there is an instantaneous movement. FIG. 10 is a diagram for explaining a method for determining an abnormal operation in the embodiment, and is a diagram illustrating a change in evaluation value when there is a continuous abnormal operation. FIG. 11 is a block diagram showing a configuration of an abnormal operation detecting apparatus according to the second embodiment of the present invention. FIG. 12 is a diagram for explaining variations in evaluation values depending on the number of people in the embodiment. FIG. 13 is a diagram for explaining variations in evaluation values depending on the distance to the subject in the embodiment. FIG. 14 is a block diagram showing a configuration of an abnormal operation detecting apparatus according to the third embodiment of the present invention.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 10 ... Surveillance camera, 11 ... Change part extraction part, 12 ... Motion data calculation part, 13 ... Temporal statistical value calculation part, 14 ... Spatial statistical value calculation part, 15 ... Evaluation value calculation part, 16 ... Motion determination part, DESCRIPTION OF SYMBOLS 21 ... Car, 22 ... Door, 23 ... Operation panel, 24 ... Destination floor button, 25 ... Door open button, 26 ... Door close button, 27 ... Emergency call button, 31 ... Load detection part, 32 ... Correction part, 33 ... distance estimation part, 41 ... door opening / closing detection part, 42 ... determination timing control part.

Claims (1)

A surveillance camera that continuously photographs the inside of the elevator car as a surveillance area at predetermined frame intervals;
Motion data calculating means for calculating motion data including an acceleration element at each point on the image using at least three or more consecutive images captured by the monitoring camera;
First statistical value calculating means for calculating a statistical value representing a temporal change in the motion data of each point of each image calculated by the motion data calculating means;
Second statistical value calculating means for calculating a statistical value representing a spatial spread of the motion data of each point of each image;
An evaluation value calculating means for calculating an evaluation value for determining an abnormal operation by multiplying the statistical value calculated by the first and second statistical value calculating means by a predetermined weight coefficient;
The evaluation value calculated by the evaluation value calculation means is compared with a preset threshold value, and a person is abnormally operating in the monitoring area when the evaluation value exceeds the threshold value for a predetermined time. An operation determination means for determining a thing,
Door opening / closing detection means for extracting an image of the door portion of the car from the image taken by the monitoring camera and detecting the door opening / closing operation of the car from the image change ;
A determination timing control means for controlling the determination timing of the abnormal operation by the operation determination means based on the opening / closing operation of the door detected by the door opening / closing detection means;
An abnormal operation detecting device comprising:

Images