JP2017111496A - Behavior monitoring prediction system and behavior monitoring prediction method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a behavior monitoring prediction system which monitors behavior of a suspicious person to predict a risk, and a behavior monitoring prediction method.SOLUTION: A behavior monitoring prediction system detects a person from an image captured by an imaging unit of a sensor device 100, transmits image data including the detected person to a computer 200 over a network N1, and determines whether a suspicious person is included in the image data. When a determination is made that a suspicious person is included in the image data, a storage unit 201 accumulates the image data. The system determines correspondence or similarity between image data and the image data accumulated in the storage unit 201. When they are determined to be correspondent or similar, a determination is made whether the accumulated image data is image data indicating a risk behavior of a suspicious person. When the accumulated image data is determined to be image data indicating a risk behavior of a suspicious person, a warning is issued to a terminal 300 over a network N2.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a behavior monitoring prediction system and a behavior monitoring prediction method.

  Conventionally, there are various techniques for monitoring the behavior of a suspicious person. For example, Patent Literature 1 discloses a technique for sensing the inside of a toilet to detect a suspicious person and reporting the suspicious person. Patent Document 2 discloses a technique for recognizing a suspicious person by collating with a preset action pattern.

JP 2007-058416 A JP 2012-133665 A

  In the above Patent Documents 1 and 2, a suspicious person can be detected and recognized. However, in these conventional techniques, in order for a person to be detected and recognized as a suspicious person, the person needs to take the action of the suspicious person. That is, when a person actually acts as a suspicious person, the person is determined to be a suspicious person and is detected and recognized.

  Such a technique cannot determine that the person is a suspicious person unless the person actually acts as a suspicious person. Therefore, it is possible to report only after the person has acted as a suspicious person, and depending on the action taken by the suspicious person, there is a possibility that the crime becomes irreversible.

  The present invention has been made in view of the above, and an object thereof is to provide a behavior monitoring prediction system and a behavior monitoring prediction method capable of predicting a danger by monitoring a suspicious person's behavior.

  In order to solve the above-mentioned problem, the behavior monitoring prediction system according to the present invention is a behavior monitoring prediction system that monitors a suspicious person's behavior and predicts a dangerous act, and the sensor device includes an imaging unit and the imaging unit. And a sensor unit that detects a person from the captured image and transmits image data including the detected person to a computer via a network. The computer includes the suspicious person in the image data. And determining that the image data includes the suspicious person, a detection processing unit that stores the image data in a storage unit, the image data, and the storage unit that stores the image data. Whether or not they match or are similar to each other, and if it is determined that they match or are similar, the stored image data is sent to the suspicious person. If it is determined whether the stored image data is image data indicating a dangerous action by the suspicious person, a notification is sent to the terminal via the network. And a prediction determination processing unit.

  Moreover, this invention is grasped | ascertained also as the behavior monitoring prediction method performed with the said behavior monitoring prediction system.

  According to the present invention, it is possible to predict the danger by monitoring the behavior of the suspicious person.

It is a figure which shows the structural example of the action monitoring prediction system in this Embodiment. It is a block diagram which shows the functional structure of a sensor apparatus. It is a figure which shows the example of a depth image. It is a figure which shows the example of detection accumulation data. It is a figure which shows the example of the report data which a display part displays. It is a figure which shows the example of the report data which a display part displays. It is a sequence diagram which shows the process sequence of an action monitoring prediction process.

  Exemplary embodiments of a behavior monitoring prediction system and a behavior monitoring prediction method according to the present invention will be described below in detail with reference to the accompanying drawings.

  FIG. 1 is a diagram illustrating a configuration example of a behavior monitoring prediction system 1000 according to the present embodiment. As illustrated in FIG. 1, the behavior monitoring prediction system 1000 determines whether an object is a suspicious person or a suspicious object based on a sensor device 100 that detects an object such as a person or an object, and information detected by the sensor device 100. And a terminal 300 that receives a report from the computer 200. An object is a person or an object monitored by this system. A suspicious person is a person who performs suspicious behavior among monitored persons, and a suspicious object is a thing handled by a suspicious person.

  The sensor device 100 and the computer 200 are connected by a network N1, and the computer 200 and the terminal 300 are connected by a network N2. Although FIG. 1 shows a case where the network N1 is configured by a general LAN (Local Area Network) or the like, and the network N2 is configured by a general wireless communication network or the like, various other networks are shown. May be used. In the following description, it is assumed that the sensor device 100 and the computer 200 are separate casings, but all or a part of them may be configured as a single casing.

  The sensor apparatus 100 is an apparatus that captures an image of an object to be measured (person O in FIG. 1) and transmits the image and information such as the width, height, and distance to the object to the computer 200. The sensor device 100 is installed in a predetermined place such as a living room or a hallway.

  FIG. 2 is a block diagram illustrating a functional configuration of the sensor device 100. As illustrated in FIG. 2, the sensor device 100 includes an imaging unit 101, a sensor unit 102, a communication unit 103, and a control unit 104. The sensor device 100 is realized by, for example, Kinect for Windows (registered trademark).

  The imaging unit 101 is an RGB camera that images an object. The sensor unit 102 is a depth sensor that detects an object and measures the distance to the object. The sensor unit 102 reads depth information including distance data obtained by the depth sensor, and outputs an image (depth image) in which an image captured by the imaging unit 101 has recognized a boundary indicating an outline or skeleton of an object. An example of the depth image will be described later with reference to FIG. The communication unit 103 transmits various information such as the RGB image captured by the imaging unit 101 and the above-described depth image and depth information to the computer 200. The control unit 104 controls the operation of each unit described above. The RGB image is an image output from the imaging unit 101 and the depth image is converted to RGB (that is, an image before being converted into a depth image).

  FIG. 3 is a diagram illustrating an example of the above-described depth image. FIG. 3 illustrates a depth image in which the person O who has the object P is captured by the sensor device 100. In the present embodiment, the case where the person is facing the front is shown. However, the sensor device 100 is provided on the wall instead of the ceiling to capture a landscape image of the person, and the video analysis unit 202 can detect the plurality of images. The size of the person O or the object P may be measured more accurately from the image.

  In FIG. 3, first, an image in a state where the person O in the hallway where the sensor device 100 is installed is carrying the object P is captured (a), and then the person O moves the object P in front of the body. An image in a state of taking a posture with both hands is taken (b), and then an image in a state in which the person O is taking a posture in which the object P faces the door R on the right side is taken. (C) shows that. Similarly, at another timing, an image in a state where the person O lifts the object P with one hand is captured (d), and then an image in which the hand of the object P lifted by the person O is captured is captured. (E). The sensor unit 102 extracts an RGB image at a predetermined timing (for example, at intervals of 1 second) from RGB images continuously captured by the imaging unit 101, and generates the above-described depth image. Next, returning to FIG. 1, the computer 200 will be described.

  The computer 200 is composed of a general PC, analyzes the RGB image and depth image received from the sensor device 100, reads depth information, and measures the width and height of the object, the distance to the object, etc. Or whether it is a suspicious object. As illustrated in FIG. 1, the computer 200 includes a storage unit 201, a detection processing unit 202, a prediction determination processing unit 203, a communication unit 204, and a control unit 205.

  The storage unit 201 includes a general storage device such as an HDD (Hard Disk Drive), and stores various types of information necessary for processing of the present system. As illustrated in FIG. 1, the storage unit 201 stores detection accumulation data 2011 and map data 2012.

  FIG. 4 is a diagram illustrating an example of the detected and accumulated data 2011. The detection accumulation data 2011 stores data determined to include an object such as a suspicious person or a suspicious object among information including an RGB image, a depth image, and depth information received from the sensor device 100.

  As illustrated in FIG. 4, the detection accumulation data 2011 indicates, for each sensor device 10, the detection date and detection time that are the date and time when the suspicious person is detected, and the type of object included in the depth image in which the suspicious person is detected. Object information, a danger flag indicating that a suspicious person included in the depth image data has taken a dangerous action, and actual depth image data, RGB image data, and depth information are stored in association with each other. In FIG. 4, for example, the sensor device 100 identified by A0001 receives the person O determined to be a suspicious person and the object P determined to be a suspicious object at 14:01:01 on October 1, 2015. An RGB image including the image is captured, and the corresponding depth image data, RGB image data, and depth information are X1. Further, the person O indicates that he / she has performed a dangerous act (at risk flag 1) from at least 14:01:01 on Oct. 1, 2015 to 14:01:30 on Oct. 1, 2015. Show. A0001 which is identification information of the sensor device 100 is associated with position information where the sensor device 100 is actually installed.

  The map data 2012 is map information of an area that is a target of the present system, for example, map information of an area where the sensor device 100 is installed. An example of the map data 2012 is not specifically shown, but is general map information as shown in the upper part of FIGS.

  The detection processing unit 202 extracts an image in which a suspicious person or a suspicious object is captured from the depth image received from the sensor device 100, and stores the extracted image in the detection accumulation data 2011. Regarding the method of identifying an image in which a suspicious person or suspicious object is captured, the depth image includes characteristics such as the shape of a predetermined person (for example, a sunglasses wearer, a mask wearer, or a balaclava wearer) The person can be determined to be a suspicious person. In addition, the depth image includes features such as the shape of a suspicious object possessed and handled by the suspicious person (for example, a gun-shaped object, a blade-shaped object, a long object of 1 m or longer). The person can be determined to be a suspicious person.

  In this example, the shape of a suspicious person (for example, the shape of a person wearing sunglasses) or the shape of a suspicious object (for example, the shape of a gun or a long object of 1 m or longer) from the depth image is previously set as a suspicious person or a suspicious person. It is stored in the storage unit 201 as an object, and when a shape that matches or similar to these shapes is included, it is determined that the depth image includes a suspicious person or a suspicious object. Of course, it may be determined whether or not a suspicious person or a suspicious object is included in the RGB image. For example, since the images shown in FIGS. 3A to 3E include long objects of 1 m or longer, the detection processing unit 202 captures suspicious objects in these depth images. It is determined that the person handling the suspicious object is a suspicious person, and the object information such as the suspicious object and the suspicious person, the detection date and the detection time thereof, the depth image data, the RGB image data The depth information is accumulated in the detection accumulation data 2011. At the timing when the detection processing unit 202 accumulates the detection accumulation data 2011, an initial value of zero is set in the danger flag.

  The prediction determination processing unit 203 compares the depth image received from the sensor device 100 with the depth image stored in the detection accumulation data 2011 at that time, and the received depth image is the accumulated depth image. Whether or not there is a dangerous act by a suspicious person included in the received depth image is determined.

  For example, when the depth image illustrated in FIG. 3A is received from the sensor device 10, the prediction determination processing unit 203 performs the depth image illustrated in FIG. 3A and the depth already accumulated in the detection accumulation data 2011. The images are compared, and a depth image that matches or is similar to each other is specified from the detected accumulated data 2011. The similarity determination is performed by, for example, determining whether the values match or are within a certain range based on the position, posture, angle, number, and the like of an object such as the person O or the object P included in the depth image. If it has the value, it can be determined that at least both are similar.

  If the prediction determination processing unit 203 determines that the two match or are similar, the danger flag of the detection accumulation data 2011 continuously accumulated from the depth image accumulated in the detection accumulation data 2011 is set. It is determined whether or not (in this example, 1 is set). When the danger flag of the detection accumulation data 2011 is set, the prediction determination processing unit 203 determines that such a suspicious person has performed a dangerous act in the past, and the suspicious person or suspicious object received from the sensor device 100. Is set to the danger flag of the detected accumulated data 2011 corresponding to the depth image including the.

  As described above, the prediction determination processing unit 203 compares the suspicious person or suspicious object included in the depth image received from the sensor device 100 with the suspicious person or suspicious object included in the past depth image. If it is at least similar, the presence / absence of a depth image when a dangerous act is subsequently performed is determined from the detected accumulated data 2011 corresponding to the depth image. And when the depth image at the time of a dangerous act being able to be specified, the same dangerous action as the said dangerous action is imaged in the depth image which is scheduled to receive following the depth image received from the sensor apparatus 100. Predict and generate report data. Thereafter, the prediction determination processing unit 203 transmits the notification data to the terminal 300 and includes a suspicious person or a suspicious object that is predicted to perform a dangerous act among the depth image received from the sensor device 100 and the subsequently received depth image. The danger flag of the detection accumulation data 2011 corresponding to the depth image being set is set.

  For example, the prediction determination processing unit 203 determines that the depth image shown in FIG. 3A received from the sensor device 100 includes a suspicious person or a suspicious object and is similar to the accumulated accumulated detection data 2011. In this case, it is determined whether or not the danger flag is set to “1” in the accumulated detected accumulated data 2011 or the detected accumulated data 2011 subsequently accumulated. Then, if the prediction determination processing unit 203 determines that “1” is set in the danger flag, the depth of FIG. 3A and the depths of FIG. 3B and FIG. The image includes suspicious persons and suspicious objects, and predicts that dangerous behavior similar to the past will occur. Then, the prediction determination processing unit 203 sets “1” to the danger flag of the detected accumulation data 2011 corresponding to these depth images, generates notification data, and transmits the notification data to the terminal 300. When the prediction determination processing unit 203 sets a danger flag, the number of depth images determined as dangerous actions increases, and it is possible to determine whether or not there is a suspicious person's dangerous action. Even when a suspicious person or a suspicious object is imaged, it is possible to accurately predict whether or not the suspicious person will perform a dangerous act.

  Examples of dangerous actions include a posture in which a person holds a gun, and a posture in which a long stick sticks a wall or door. As with the suspicious person and the suspicious object, what kind of action corresponds to the dangerous action is determined in advance and stored in the storage unit 201. Therefore, actions other than the prescribed actions are not determined as dangerous actions. For example, in the depth images shown in FIGS. 3D and 3E, since the person O lifts the object P with one hand and the person O lifts the hand of the object P, these actions are performed. Is not stored in advance in the storage unit 201 and is not determined to be the dangerous action.

  As described above, the prediction determination processing unit 203 compares the depth image received and accumulated in the past from the sensor device 100 with the depth image currently received from the sensor device 100, and then from the sensor device 100. If a depth image that is at least similar to a suspicious person or suspicious object contained in the currently received depth image is identified, and the depth image is taken when a dangerous action is taken, the same risk as in the past Predict that the action will be imaged and report immediately. Therefore, it is possible to prevent the occurrence of dangerous acts similar to the past.

  The communication unit 204 transmits various types of information such as report data after the prediction determination by the prediction determination processing unit 203 to the terminal 300. The control unit 205 controls the operation of each unit. Next, returning to FIG. 1, the terminal 300 will be described.

  The terminal 300 is composed of a general PC, and displays the notification data after the prediction determination received from the computer 200 on a display device such as a display. As illustrated in FIG. 1, the terminal 300 includes a display unit 301, a command processing unit 302, a communication unit 303, and a control unit 304.

  The display unit 201 is composed of a general display such as an LCD (Liquid Crystal Display), and displays various information related to the report, such as report data received from the computer 200.

  FIG. 5 is a diagram illustrating an example of report data displayed on the display unit 301. As shown in FIG. 5, the display unit 301 displays a map display area S that displays map data 2012 near the area where the sensor device 100 is installed, and a content display area T that displays the reported content. . In the example shown in FIG. 5, a suspicious person (person O) and a suspicious object that are predicted to perform a dangerous act on a map in the vicinity of an area in which the sensor apparatus 100 identified by A0001 is installed and can be imaged. Data on which the position of (object P) is superimposed is displayed at the top of the screen. In the lower part of the screen, a message indicating that there is a possibility that the person O may perform a dangerous act with the object P and a message requesting immediate security to the security staff are displayed.

  In the present embodiment, the case where the notification data is displayed on the terminal 300 is described, but it is considered that there are cases where the safety of people near the site should be secured before the security guard arrives. In preparation for such a case, for example, the computer 200 transmits report data to the terminal 300 and transmits the report data to the sensor device 100, and the communication unit 103 of the sensor device 100 is provided in the sensor device 100. Further, it may be notified from the speaker (not shown) that a dangerous act may be performed in the vicinity. Further, when the sensor device 100 can be connected to a mobile terminal operated by a person in the vicinity of the sensor device 100 via a public wireless LAN, notification data is transmitted to the mobile terminal by interruption. , May be displayed. For example, as shown in FIG. 6, the same map data as described above, and the same message and message indicating evacuation communication may be transmitted to the portable terminal as report data and displayed. As a result, the person in the vicinity can know that the dangerous act is taking place nearby, and can secure personal safety such as escaping immediately. Next, processing performed in this system will be described.

  FIG. 7 is a sequence diagram illustrating a processing procedure of processing (behavior monitoring prediction processing) performed in the present system. As illustrated in FIG. 7, in the behavior monitoring prediction process, the imaging unit 101 of the sensor device 100 captures an RGB image (step S701), and the sensor unit 102 detects an object such as a person or an object to generate a depth image. (Step S702), the depth image data, RGB image data, and depth information are transmitted to the computer 200 via the communication unit 703 (Step S703).

  The detection processing unit 202 of the computer 200 identifies depth image data, RGB image data, and depth information that include a suspicious person or a suspicious object from the data received from the sensor device 100, and the storage unit 201 as detection accumulation data 2011. (Step S704). The prediction determination processing unit 203 compares the depth image specified by the detection processing unit 202 with the depth image stored in the detection accumulation data 2011 (step S705).

  When the prediction determination processing unit 203 determines that the two match or are similar, the danger flag of the detection accumulation data 2011 corresponding to the accumulated depth image or the depth image accumulated thereafter is set. It is determined whether or not (step S706). If the prediction determination processing unit 203 determines that the danger flag of the detected accumulated data 2011 is set (step S706; Yes), the detected accumulated data 2011 corresponding to the depth image received in step S703 and accumulated in step S704. Is set, and notification data is generated (step S707). On the other hand, when the prediction determination processing unit 203 determines that the danger flag of the detected accumulation data 2011 is not set (step S706; No), the prediction determination processing unit 203 returns to step S704 and continues the processing.

  The prediction determination processing unit 203 transmits report data about the detected accumulation data 2011 in which the danger flag is set to the terminal 300 (step S708), and the command processing unit 302 of the terminal 300 displays the report data on the display unit 301 ( Step S709). A system administrator or a security guard can easily confirm a suspicious person who is predicted to perform a dangerous act by viewing the report data displayed on the display unit 301.

  In this way, in this system, by executing the above process, it is possible to predict the risk by monitoring the behavior of the suspicious person. Previously, the risky behavior of a suspicious person could only be determined after the suspicious person acted.However, according to this system, the behavior of a suspicious person is predicted from the behavior of a person who has been determined to be a suspicious person in the past. Immediately notify you if the action you have taken is a past dangerous act. Therefore, it is possible to notify the system administrator and the security guard before a dangerous act actually occurs.

  Each process performed by the sensor device 100, the computer 200, and the terminal 300 is actually realized by executing a program installed in the sensor device 100, the computer 200, and the terminal 300. The program may be provided by being incorporated in advance in a ROM or the like, or may be provided by being recorded on a recording medium in a file in an installable format or an executable format, or distributed.

  In the above embodiment, the case where the sensor device is installed in an open place such as a corridor has been described. However, for example, the sensor device is installed in a closed room where a camera cannot be installed, such as in a toilet. You may predict dangerous behavior. In this case, a dangerous act can be predicted even in a space where no camera is installed.

1000 Action Monitoring Prediction System 100 Sensor Device 101 Imaging Unit 102 Sensor Unit 103 Communication Unit 104 Control Unit 200 Computer 201 Storage Unit 2011 Detection Accumulated Data 2012 Map Data 202 Detection Processing Unit 203 Prediction Determination Processing Unit 204 Communication Unit 205 Control Unit 300 Terminal 301 Display unit 302 Command processing unit 303 Communication unit 304 Control unit N1, N2 Network.

Claims (4)

A behavior monitoring and prediction system that monitors the behavior of a suspicious person and predicts dangerous behavior,
The sensor device
An imaging unit;
A sensor unit that detects a person from an image captured by the imaging unit and transmits image data including the detected person to a computer via a network;
The computer
Determining whether or not the image data includes the suspicious person, and when determining that the image data includes the suspicious person, a detection processing unit that accumulates the image data in a storage unit;
Based on the image data and the image data stored in the storage unit, it is determined whether or not they match or similar, and when it is determined that they match or similar, the stored image data Is determined to be image data indicating a dangerous action by the suspicious person, and when it is determined that the stored image data is image data indicating a dangerous action by the suspicious person, the terminal is connected via the network. A prediction determination processing unit that reports to
A behavior monitoring and prediction system comprising: When it is determined that the stored image data is image data indicating a dangerous act by the suspicious person, the prediction determination processing unit receives from the sensor device and the detection processing unit accumulates in the storage unit Set the image data as image data indicating a dangerous act,
The behavior monitoring prediction system according to claim 1. When the prediction determination processing unit determines that the stored image data is image data indicating a dangerous act by the suspicious person, the map data indicating a place where the sensor device is installed and the suspicious person By generating notification data including a message indicating that there is a possibility of taking a dangerous action and a message requesting the guard to urgently go to the scene, and transmitting the generated notification data to the terminal, the notification do,
The behavior monitoring prediction system according to claim 1 or 2, wherein A behavior monitoring prediction method for monitoring a suspicious person's behavior and predicting dangerous behavior,
A person is detected from the image captured by the imaging unit of the sensor device,
Send image data including the detected person to a computer via a network,
Determining whether the suspicious person is included in the image data;
When it is determined that the suspicious person is included in the image data, the image data is stored in a storage unit,
Based on the image data and the image data stored in the storage unit, it is determined whether or not both match or are similar,
If it is determined that both match or similar, determine whether the stored image data is image data indicating a dangerous act by the suspicious person,
If it is determined that the stored image data is image data indicating a dangerous act by the suspicious person, a notification is sent to the terminal via the network.
A behavior monitoring prediction method characterized by that.

Images