JP5730099B2 - Anomaly detection device - Google Patents

Description

  The present invention relates to an anomaly detection device that detects an anomaly that has occurred in a surveillance space by analyzing a captured surveillance image. In particular, the present invention relates to an abnormality detection device that detects an abnormal situation involving a plurality of persons.

  When a burglar breaks into a store and tries to snatch money stored in a safe or cash register, the store employee or the like may threaten to report to the outside or resist, or attack the employee or give an order Sometimes, trying to control employees by forcing their actions. Such suppression actions by robbers are often taken especially when there are a plurality of persons.

  When multiple persons are controlled by a burglar, there are concerns about the delay in the detection of incidents due to delays in reporting, and delays in the relief of injured victims. For this reason, the proposal of the abnormality detection apparatus which detects automatically that a victim is in the state which cannot perform the above notification operations, and reports is desired.

  Patent Document 1 discloses a monitoring device that determines whether a victim's movement is abnormal (such as raising both hands) as a means for detecting an intrusion burglary.

Japanese Patent No. 3502853

  However, it is difficult to discriminate between an abnormal action (forced to stop) and a normal action (stop at his own will) by the method of determining the presence or absence of an abnormality from the action of a single person. That is, a situation where “a plurality of persons are forced to operate” cannot be detected, and such a situation cannot be detected as an emergency.

  Therefore, the present invention focuses on controlling employees at the timing when a burglar or the like encounters a plurality of employees, and captures a specific image change when a new person appears in the monitoring area. Therefore, an object of the present invention is to provide an anomaly detection device that automatically detects that an operation is forced by a burglar and a plurality of employees are suppressed.

One aspect of the present invention is an anomaly detection device that detects an anomaly that has occurred in the monitoring space from a monitoring image obtained by imaging the monitoring space, the image acquisition means for sequentially acquiring the monitoring image, and the monitoring image. A person area extracting means for extracting a person as a person area; a person tracking means for tracking a past person area and a current person area in association with each other in the sequentially acquired monitoring images; A person appearance determining unit that determines that a new person appears when a person region cannot be associated; an exercise change determining unit that determines whether or not there is a predetermined change in an exercise state for each associated person; and the new person from appearing within a predetermined time, the abnormality judging means for judging an abnormality when the number of people the novel person made the predetermined change in the motion state of the person existing before the occurrence is beyond, An abnormality detection device, characterized in that it comprises.

  Here, it is preferable that the abnormality determination unit sets the predetermined change that the motion state has changed from a moving state to a stationary state.

  In addition, it is preferable that the abnormality determination unit sets the predetermined change that the movement state has changed from a moving state to a stationary state within a predetermined area set in advance.

  In addition, it is preferable that the abnormality determining unit sets the predetermined change that the moving speed has changed as the exercise state.

In addition, it is preferable that the reference is a ratio of the number of persons who have a predetermined change in the exercise state in the number of persons associated with a person existing before the new person appears .

  According to the present invention, even when a plurality of persons are suppressed by a burglar or the like and a notification operation cannot be performed, such an abnormal situation can be automatically detected and a notification can be made. Furthermore, since an emergency call can be made when the operation is forced by a burglar or the like, the occurrence of a burglary case can be detected early.

It is a figure which shows the structure of the notification system in embodiment of this invention. It is a functional block diagram which shows the structure of the abnormality detection apparatus in embodiment of this invention. It is a flowchart which shows the process in the abnormality detection apparatus in embodiment of this invention.

<Basic configuration>
As shown in the schematic diagram of FIG. 1, the notification system 1 according to the embodiment of the present invention includes an abnormality detection device 2, a controller 3, and a center device 5.

  The abnormality detection device 2 is installed on the ceiling of a room such as a store as a monitoring space. The abnormality detection device 2 outputs an abnormality signal when detecting an abnormal scene occurring in the monitoring space. The abnormality detection device 2 is connected to the controller 3 via a communication line. The controller 3 is connected to a center device 5 installed in a remote place such as a security center via a communication network 4 such as a telephone line or an internet line. The abnormality signal output from the abnormality detection device 2 is transmitted to the center device 5 via the controller 3.

FIG. 2 is a functional block diagram showing the configuration of the abnormality detection device 2. As shown in FIG. 2, the abnormality detection device 2 includes an imaging unit 20, a storage unit 21, a control unit 22, and an output unit 23. The control unit 22 is connected to the imaging unit 20, the storage unit 21, and the output unit 23, and controls these components in an integrated manner. The abnormality detection device 2 can be realized by a computer including an imaging unit controlled by an abnormality detection processing program.

  The imaging unit 20 includes a monitoring camera. The imaging unit 20 is installed so as to face the monitoring space, and images the monitoring space at a predetermined time interval. The captured monitoring images of the monitoring space are sequentially output to the control unit 22. The time interval at which the monitoring image is captured is 1/5 second, for example. Hereinafter, the unit of time recorded at the time interval of imaging is referred to as time.

  In this embodiment, in order to image the whole room which is a monitoring space, the imaging unit 20 includes a fisheye lens and is installed with its optical axis directed vertically downward. However, the present invention is not limited to this, and any method may be used as long as a person region can be extracted from a monitoring image obtained by capturing a person and the exercise state can be determined.

  The storage unit 21 is a storage device for electronic information such as a ROM (Read Only Memory) and a RAM (Random Access Memory). The storage unit 21 stores various programs and various data and is accessible from the control unit 22. Various data includes a background image 210 necessary for background difference processing.

  The control unit 22 includes an arithmetic device such as a CPU (Central Processing Unit), a DSP (Digital Signal Processor), or an MCU (Micro Control Unit). The control unit 22 reads out and executes the abnormality detection processing program from the storage unit 21, thereby executing a person area extraction unit 220, a person tracking unit 221, a person appearance determination unit 222, an exercise change determination unit 223, a change number of people determination unit 224, and an abnormality. It functions as the determination means 225. Each of these means will be described later.

  The output unit 23 is a communication unit that outputs an abnormality signal to the outside of the abnormality detection device 2. When an abnormality signal is input from the abnormality determination unit 225 of the control unit 22, the output unit 23 transmits the abnormality signal to the controller 3 via the communication network 4.

<Abnormality detection processing>
Hereinafter, the abnormality detection process in the abnormality detection device 2 will be described with reference to the flowchart of FIG.

  In step S1, the abnormality detection device 2 is initialized. When the abnormality detection device 2 is powered on, initialization is performed. The initialization includes processing for storing the monitoring image immediately after startup in the storage unit 21 as the background image 210. In addition, the movement change number counter for counting the number of persons with movement changes and the new person appearance elapsed time counter for measuring the time after the new person appears are reset.

  In step S2, a monitoring image is acquired. The imaging unit 20 images the monitoring space at a predetermined cycle, and outputs the captured monitoring image data to the control unit 22. The control unit 22 acquires a monitoring image from the imaging unit 20 and stores it in the storage unit 21 in association with the imaging time.

  In step S3, a process of extracting a person area from a monitoring image obtained by capturing a person is performed. The processing here corresponds to the person region extraction means 220.

  The control unit 22 calculates a difference image obtained by calculating a difference value between feature amounts of pixels corresponding to each other in the monitoring image acquired from the imaging unit 20 and the background image 210 stored in the storage unit 21. Then, pixels whose difference value from the background image 210 is equal to or greater than the threshold are extracted as change pixels. As the feature amount, luminance, color, edge strength / direction, and the like can be used. As the threshold value, a predetermined value determined in advance may be used, or may be changed dynamically.

  As the background image 210, the image obtained at the time of initialization in step S1 may be used as it is, or may be appropriately updated by the monitoring image input in step S2. It is also preferable to perform light / shadow region removal by normalized correlation or the like after difference calculation, and removal of sesame salt-like noise by expansion / contraction processing.

  Furthermore, the control unit 22 performs a labeling process after extracting the changed pixels. The labeling process is a process of assigning the same number to a group (group) of change pixels that are spatially continuous in the difference image. By this process, an individual number is assigned to each group (group) of changed pixels. After labeling, for example, a noise removal process such as excluding a group (group) of change pixels whose area is equal to or less than a threshold is excluded from the processing target. The threshold is determined from a standard person size calculated from camera installation conditions. A group (group) of changed pixels left as a processing target is a person region.

  In step S4, a person tracking process for associating a person area estimated to be the same person in the sequentially acquired monitoring images is performed. The processing here corresponds to the person tracking means 221.

  The control unit 22 extracts the tracking feature regarding each person area extracted by the person area extracting unit 220 by the previous time and stored in the storage unit 21 and each person area extracted by the person area extracting unit 220 at the current time. Associate with. Specifically, whenever the appearance of a person area in the monitoring area is confirmed, the control unit 22 extracts the image feature of the person area as a tracking feature and stores it in the storage unit 21. Then, the tracking feature of the person area extracted at the previous time stored in the storage unit 21 is compared with the tracking feature of the person area extracted at the current time, and the similarity between the tracking features is higher than a predetermined condition. The person areas determined to be the most similar are associated with each other. By removing the associated person area and repeating the association process, all the person areas extracted at the current time are associated with the person area extracted up to the previous time.

  The tracking feature may be a color or edge histogram, the position of the center of gravity of a person region, a circumscribed rectangle (size, aspect ratio, etc.), and the like. For example, when the tracking feature is a color histogram, the color of the pixels constituting the person region extracted at the current time is examined, and a color histogram that is the number of pixels for each color bin is obtained. Then, the similarity with the color histogram of the person area extracted up to the previous time is investigated, and the person area with the highest similarity among the person areas that satisfy the predetermined similarity condition is associated with the person area at the current time. . The predetermined similarity condition can be set, for example, such that the number of pixels in each color bin is within a difference of ± 10%. The same processing can be performed for other tracking features.

  Further, if there is a person area extracted up to the previous time that is not associated with the person area extracted at the current time, the person area is that of a person who has moved out of the monitoring space. And erase from the storage unit 21. Or it is good also as what hold | maintains as a person area | region in the memory | storage part 21 for the fixed period from the time which was not linked | related, and tries said correlation process in the meantime.

  In step S5, a process of extracting a person area corresponding to a person newly appearing in the monitoring space as a new person is performed. The processing here corresponds to the person appearance determination means 222.

  In step S4, if there is a new person area that does not correspond to any existing person area, the control unit 22 determines it as a new person.

  In step S6, when it is determined that the person is a new person in step S5, the control unit 22 sets a new person appearance progress counter for the new person. If there is a person area in which the new person appearance progress counter is set, the process proceeds to step S7, and if not, the process proceeds to step S18. Note that the processing in steps S7 to S21 is executed for each of the person areas for which the new person appearance progress counter is set, but only one person area is described here for the sake of simplicity. ing.

  In steps S7 to S9, before a new person appears in the monitoring space, it is confirmed in step S4 that the person's motion state is a moving state for the person area that has existed in the monitoring space before the current time. The processing here corresponds to a part of the movement change determination means 223. The human motion state means the general state of the human movement in the monitoring space, and means, for example, the movement state, the stationary state, the movement speed of the person, and the like.

  In step S7, the person regions extracted in the monitoring image are sequentially selected. In step S8, it is determined whether or not the selected person area exists before a new person appears. In the person tracking process in step S4, the control unit 22 shifts the process to step S9 if the selected person area exists before the new person appears, otherwise shifts the process to step S16. Let

  In step S9, it is determined whether or not the selected person area was in a moving state before a new person appeared. The control unit 22 extracts a difference area between frames for the monitoring image in which the selected person area exists, and the moving state is obtained when the area of the difference area between frames in the area of the selected person area is larger than a predetermined ratio. It is determined that For example, when the inter-frame difference area of 5% or more of the area of the selected person area is included in the person area, it is determined that the selected person area is in a moving state. In addition, the control unit 22 calculates the centroid position of the person area associated with the person area selected from the monitoring image before the new person appears, and moves when the movement distance of the centroid position is larger than a predetermined reference value. It is determined that it was in a state. For example, if the moving distance of the gravity center position of the selected person area is 3 pixels or more between frames of past monitoring images, it is determined that the selected person area is in a moving state. Further, the control unit 22 obtains a circumscribed rectangle of the person area associated with the selected person area in the past monitoring image before the person area corresponding to the new person appears, and a change in the size of the circumscribed rectangle is predetermined. If it is larger than the reference value, it is determined that the moving state has occurred. For example, if the rate of change in the size of the circumscribed rectangle of the selected person area between frames of past monitoring images is 5% or more, it is determined that the selected person area is in a moving state. .

  The control unit 22 shifts the process to step S10 when any of the conditions for determining the dynamic state is satisfied, and shifts the process to step S16 if not.

  In step S10, it is investigated whether the motion state of the person who was in a moving state became a stationary state after the appearance of a new person. The processing here corresponds to a part of the movement change determination means 223.

  The control unit 22 extracts the inter-frame difference area for the monitoring image in which the selected person area exists in the same manner as in step S9, and the area of the inter-frame difference area in the area of the selected person area is based on a predetermined ratio. When it is small, it is determined that the camera is stationary. For example, when the person region includes the inter-frame difference region in less than 5% of the area of the selected person region, it is determined that the selected person region is in a stationary state. In addition, the control unit 22 calculates the gravity center position of the selected person area in the monitoring image from the time when the person area corresponding to the new person appears until the current time, and the movement distance of the gravity center position is smaller than a predetermined reference value. It is determined that the stationary state has been reached. For example, if the moving distance of the center of gravity position of the selected person area is less than 3 pixels between frames of the monitoring image, it is determined that the selected person area is in a stationary state. In addition, the control unit 22 calculates a circumscribed rectangle of the selected person area in the monitoring image from the appearance of the person area corresponding to the new person to the current time, and the change in the size of the circumscribed rectangle is a predetermined reference value. When it is smaller, it is determined that the camera is stationary. For example, when the rate of change in the size of the circumscribed rectangle of the selected person area is less than 5% between the frames of the monitoring image, it is determined that the selected person area is in a stationary state.

  In addition, when a burglar or the like enters a surveillance space as a new person, other people may move instantaneously, so do not use the surveillance image of several frames immediately after the new person appears in the surveillance space for stillness determination. It may be.

  In steps S11 to S15, processing for counting the number of still persons is performed. The processing here corresponds to a part of the change number determination means 224. In step S10, it is determined whether or not the selected person area has changed from a moving state to a stationary state. In step S11, the control unit 22 has the person area selected in step S7 in the past, If it is determined that a new person has changed to a stationary state after appearing in the monitoring space, the process proceeds to step S14. If not, the process proceeds to step S12. In step S12, it is confirmed whether or not it is already counted by the exercise change number counter as a stationary person area. The control unit 22 shifts the process to step S16 if the person area is not counted, and decrements the exercise change number counter by 1 in step S13, assuming that the person area has already been counted and is out of the stationary state. I do. On the other hand, in step S14, it is confirmed whether or not it is already counted by the exercise change number counter as a stationary person area. If the person area has already been counted, the control unit 22 shifts the process to step S16. If the person area has not been counted, the control unit 22 sets the motion change number counter to 1 in step S15. Process to increase.

  In step S <b> 16, it is determined whether or not the investigation of the transition to the stationary state has been performed for all the person areas being tracked. The control unit 22 shifts the process to step S17 if the survey has been completed for all the person areas, otherwise returns to step S7, selects a new person area, and performs the process from step S8. repeat.

  In steps S <b> 17 to S <b> 21, a process for determining whether or not a change in a person's motion state (here, a change from a moving state to a stationary state) is caused by a new person and an abnormality determination are performed. The processing here corresponds to a part of the change number determination means 224 and the abnormality determination means 225.

  In step S17, it is determined whether or not the new person appearance progress counter is less than a predetermined threshold (predetermined time). If the new person appearance progress counter is less than the predetermined threshold (predetermined time), the control unit 22 shifts the process to step S20, and if not, shifts the process to step S18. The predetermined threshold (predetermined time) is set to a value corresponding to a time during which it can be determined that the change in the exercise state is caused by a new person. The predetermined threshold (predetermined time) may be a fixed value or may be adaptively changed. The predetermined threshold value may correspond to, for example, 30 seconds.

  In step S18, the exercise change number counter is reset. That is, the control unit 22 determines the number of person areas corresponding to the person whose motion state has been changed from the moving state to the stationary state after the new person appears in the monitoring space and before the predetermined threshold (predetermined time) is counted. The exercise change number counter is reset assuming that the predetermined threshold value (reference number of persons) has not been exceeded. That is, it is determined that the number of persons who have stopped moving within a predetermined time after the appearance of a new person is less than the reference value and that the condition is not abnormal. In step S19, the new person appearance progress counter is set to zero. Thereafter, the control unit 22 returns the process to step S2.

  In step S20, it is determined whether or not the exercise change number counter is equal to or greater than a predetermined threshold (reference number of persons). If the exercise change number counter is equal to or greater than a predetermined threshold (reference number of persons), the process proceeds to step S22; otherwise, the process proceeds to step S21. Here, the predetermined threshold (reference number of persons) may be a fixed value or a value that varies adaptively. The predetermined threshold value (reference number of persons) is preferably set, for example, as a ratio to the number of persons existing before the new person appears in the monitoring space (the number of person areas being tracked). Specifically, it is preferable to set it to 80% or more of the number of people existing before the new person appears in the monitoring space (the number of person areas being tracked).

  In step S21, a new person appearance progress counter is counted up. The control unit 22 increments the new person appearance progress counter by 1, and then returns to step S2 to investigate the change in the exercise state with respect to the next time.

  In step S22, an abnormal output process is performed. When the new person appearance progress counter reaches a predetermined threshold (predetermined time) and the exercise change number counter becomes equal to or greater than the predetermined threshold (reference number), the control unit 22 It is determined that an abnormality has occurred in the monitoring space because the motion state of the vehicle has changed from the moving state to the stationary state. The control unit 22 outputs an abnormal signal to the output unit 23.

  As described above, it is determined that a plurality of persons are forced to act and cannot perform a notification operation when a plurality of persons' movement states change within a predetermined time after a new person appears in the monitoring space. Thus, it is possible to automatically detect such an abnormal situation and make an emergency call.

  Also, a plurality of conditions for abnormality determination may be set. For example, the threshold for the number of times of change of the exercise state (predetermined time) and the threshold of the number of people for the change of the exercise state (reference number of people) are set, for example, ) Have changed their exercise state, more than 80% of the number of people (number of person areas) existing before the appearance of a new person in the surveillance space within 30 seconds, or 1 minute Within 100% of the number of people (number of person areas) that existed before the new person appeared in the surveillance space within 100% or more, the exercise state was changed, and it was determined that any one of the three conditions was satisfied. You may make it do.

  In the present embodiment, an abnormality is detected by investigating whether the person area that was in the moving state in step S10 has become stationary after the appearance of the new person area. However, the present invention is not limited to this, and for example, a change in movement speed or a concentrated movement in one place can be used as a change in exercise state, and an abnormality can be detected by capturing changes in the exercise state of a plurality of persons.

  Specifically, a change in moving speed of a person (person area) that has existed in the monitoring space before the appearance of a new person is obtained. That is, the moving speed of the center of gravity of the person (person area) extracted from the monitoring image is calculated, and a person whose change in moving speed is equal to or greater than a predetermined threshold (reference speed change amount) is determined from the appearance time of the new person. If it is equal to or greater than a predetermined number of persons (reference number of persons) before the threshold value (predetermined time) is reached, it is determined that there is an abnormality. The predetermined time is set to a value corresponding to a time during which it can be determined that the speed change is caused by a new person. The predetermined threshold (predetermined time) may be fixedly set or may be adaptively set according to the size of the monitoring space. The predetermined time can be set to 1 minute, for example. Further, the predetermined number of persons (reference person number) may be set fixedly or may be set adaptively according to the number of persons existing in the monitoring space before the appearance of the new person. The reference number of persons may be set to 80% of the number of persons existing in the monitoring space before the appearance of a new person (the number of person areas being tracked). The predetermined threshold (reference speed change amount) is a value that can be used to determine whether the moving speed of the person suddenly increases or decreases rapidly, and may be set fixedly, depending on the size of the monitoring space, etc. You may set adaptively according to it.

  Further, before reaching a predetermined threshold (predetermined time) from the appearance of a new person, more than a predetermined number of persons (reference number of persons) changes from a moving state to a stationary state, and the three-dimensional position of the person (person area) is determined. If it is within a predetermined range (reference range), it may be determined that the person (person area) is concentrated and moved to one place. Here, the predetermined time is set to a value corresponding to a time during which it can be determined that the person has moved within the same range due to the appearance of the new person. The predetermined time may be fixedly set or may be adaptively set according to the size of the monitoring space. The predetermined time can be, for example, 3 minutes. In addition, the reference number of persons may be set fixedly or may be set adaptively according to the number of persons existing in the monitoring space before the appearance of the new person. The reference number of persons may be set to 80% of the number of persons (the number of tracked person areas) that existed in the monitoring space before the appearance of the new person. Further, the reference range may be set fixedly or may be set adaptively according to the size of the monitoring space. The reference range may be a range corresponding to a circle having a radius of 1 m, for example.

  As in the above embodiment, the stationary state is determined such that there is no interframe difference in the person area, the center of gravity position of the person area has not moved, the circumscribed rectangle of the person area has not changed, etc. be able to. Further, the three-dimensional position of the person can be calculated by reverse perspective transformation in consideration of the installation conditions of the monitoring camera of the imaging unit 20.

  Moreover, although the said embodiment demonstrated the aspect which implement | achieves the function of each part of the abnormality detection apparatus 1 with one computer, it is not limited to this. The functions of each part of the abnormality detection device 1 can be realized by controlling a general computer by a program, and the functions of these devices may be appropriately combined and processed by a single computer. Distributed processing may be performed by a plurality of computers connected via a network or the like.

  DESCRIPTION OF SYMBOLS 1 Report system 2 Abnormality detection apparatus 3 Controller 4 Communication network 5 Center apparatus 20 Imaging part 21 Storage part 22 Control part 23 Output part 210 Background image 220 Person area extraction means 221 Person tracking means , 222 Person appearance determination means, 223 Movement change determination means, 224 Changed number of persons determination means, 225 Abnormality determination means.

Claims (5)

An abnormality detection device that detects an abnormality occurring in the monitoring space from a monitoring image obtained by imaging the monitoring space,
Image acquisition means for sequentially acquiring the monitoring images;
Person area extracting means for extracting a person as a person area from the monitoring image;
A person tracking means for tracking a past person area and a current person area in association with each other for the same person in the sequentially acquired monitoring images;
Person appearance determination means for determining that a new person appears when the current person area cannot be associated;
Exercise change determination means for determining whether or not there is a predetermined change in exercise state for each of the associated persons;
An abnormality determining means for determining an abnormality when the number of persons having a predetermined change in the motion state of the person existing before the new person appears within a predetermined time from the appearance of the new person is greater than or equal to a reference;
An abnormality detection device comprising: The abnormality detection device according to claim 1,
The abnormality detection device according to claim 1, wherein the abnormality determination unit sets the predetermined change that the motion state has changed from a moving state to a stationary state. The abnormality detection device according to claim 2,
The abnormality detection device according to claim 1, wherein the abnormality determination unit sets the predetermined change that the motion state has changed from a moving state to a stationary state within a predetermined region set in advance. The abnormality detection device according to claim 1,
The abnormality detection device according to claim 1, wherein the abnormality determination unit sets the predetermined change as a movement speed as the exercise state. The abnormality detection device according to any one of claims 1 to 4,
The abnormality detection apparatus according to claim 1, wherein the reference is a ratio of the number of persons having a predetermined change in the exercise state in the number of persons associated with a person existing before the new person appears .

Images