JP2019053608A - Characteristic behavior detecting apparatus - Google Patents

Abstract

To provide a characteristic behavior detecting apparatus which reduces a collection period of learning data and can keep detection accuracy in detection of a characteristic behavior.SOLUTION: In a characteristic behavior detecting system 100, a characteristic behavior detecting apparatus has a data acquiring unit 31, and a newly grouping unit 342. The data acquiring unit 31 acquires sensor values of a plurality of sensors. The newly grouping unit 342 groups the plurality of sensors on the basis of behavior patterns of detection objects detected by the sensor values.SELECTED DRAWING: Figure 2

Description

  Embodiments described herein relate generally to a characteristic behavior detection apparatus.

  In recent years, for example, terminal cameras and shopping malls, where there are many unspecified people, such as surveillance cameras, have been installed to detect unusual behaviors, which are called characteristic behaviors. Has been done. The detection of the characteristic behavior is not limited to the detection of the suspicious person, and can be applied to the detection of the situation in the monitoring target area, for example, how people flow or gather differently. That is, by detecting characteristic behavior, for example, it is possible to adaptively arrange and dispatch guards.

  In order to detect a characteristic behavior, for example, setting a detection rule for each characteristic behavior is difficult and very time-consuming. For this reason, as a method for detecting characteristic behavior, a method is proposed in which normal behavior is learned by statistical processing or machine learning, and behavior different from the learned normal behavior is detected as characteristic behavior. Has reached.

Japanese Patent No. 5712868 Japanese Patent Laid-Open No. 2006-191966 JP, 2006-191975, A

  For example, in order to learn normal behavior by machine learning, several sets of data for one week including, for example, weekday data (sensor values of sensors such as surveillance cameras) and weekend data must be collected. For example, a period of learning data collection was required to some extent. Therefore, a certain period of time has been left between the introduction of the system for detecting characteristic behavior and the start of operation.

  In order to shorten the collection period of learning data, for example, data is collected from sensors such as multiple surveillance cameras installed in different environments such as different locations within a station, different stations, different facilities such as stations and stores, etc. It can be considered. However, if data obtained in different environments are mixed, the detection accuracy of characteristic behavior by machine learning (detection accuracy of behavior different from normal behavior learned by machine learning) is significantly lowered.

  The problem to be solved by the present invention is to provide a characteristic behavior detection device capable of shortening the learning data collection period and maintaining the detection accuracy of characteristic behavior.

  According to the embodiment, the characteristic behavior detection device includes an acquisition unit and a new grouping unit. The acquisition unit acquires sensor values of a plurality of sensors. The new grouping unit groups the plurality of sensors based on the behavior pattern of the target detected from the sensor values.

The figure for demonstrating the outline | summary of the characteristic action detection system of embodiment. The figure which shows the example of 1 structure of the characteristic action detection system of embodiment. The figure which shows an example of the hardware constitutions of the characteristic action detection system of embodiment. The conceptual diagram for demonstrating grouping of the sensor by the action pattern of the characteristic action detection system of embodiment. The flowchart which shows the flow of the process regarding grouping of the sensor in the characteristic action detection system of embodiment, and learning of a normal action. The flowchart which shows the flow of the process regarding the new grouping of the sensor in the characteristic action detection system of embodiment. The figure which shows an example of the group information used in the characteristic action detection system of embodiment. The figure which shows an example of the sensor information used in the characteristic action detection system of embodiment. The figure for demonstrating the outline | summary of the process with respect to the added sensor in the characteristic action detection system of embodiment. The flowchart which shows the flow of a process with respect to the added sensor in the characteristic action detection system of embodiment. The figure which shows an example of the state by which the grouping of the sensor by the action pattern of the characteristic action detection system of embodiment was performed. The flowchart which shows the flow of the process regarding collection of learning data including the normalization in the characteristic action detection system of embodiment. The flowchart which shows the flow of the process regarding the detection of the characteristic action including the normalization in the characteristic action detection system of embodiment. The figure for demonstrating an example of the effect of the grouping of the sensor by the action pattern in the characteristic action detection system of embodiment.

  Hereinafter, embodiments will be described with reference to the drawings.

  FIG. 1 is a diagram for explaining the outline of the characteristic behavior detection system 100 of the present embodiment.

  This characteristic behavior detection system 100 assumes a system that centrally executes detection of characteristic behavior in various places such as stations, stores, and multipurpose halls. More specifically, this characteristic behavior detection system 100 assumes a system that detects a characteristic behavior at various locations using a plurality of sensors 1 such as surveillance cameras installed at various locations. However, it is assumed that the sensors 1 connected to the characteristic behavior detection system 100 are unified with the same type of sensors.

  The characteristic behavior detection system 100 is a system that detects a behavior different from a normal behavior as a characteristic behavior, and uses a plurality of sensors 1 to collect learning data for learning the normal behavior. This is a system that can shorten the time and maintain the detection accuracy of characteristic behavior. Hereinafter, the mechanism provided for maintaining the detection accuracy of the characteristic behavior while using the plurality of sensors 1 having different installation locations or different installation conditions will be described in detail.

  FIG. 2 is a diagram illustrating a configuration example of the characteristic behavior detection system 100.

  As shown in FIG. 2, the characteristic behavior detection system 100 includes a plurality of sensors 1, a sensor data management device 2, and a characteristic behavior detection device 3. The characteristic behavior detection system 100 may include a reporting device 4 and an information distribution device 5. Here, although the sensor data management device 2 and the characteristic behavior detection device 3 are shown separately, they may be constructed by the same computer, for example. Further, one or both of the sensor data management device 2 and the characteristic behavior detection device 3 may be constructed by two or more computers capable of cooperating via a network, for example, for load distribution.

  Sensor values of a plurality of sensors 1 such as surveillance cameras installed in various places are transmitted to the sensor data management device 2 (data management unit 21). The method of communicating between the sensor 1 and the sensor data management device 2 is not limited to a specific method, and although it does not matter here, for example, a method of communicating via a network such as the Internet can be adopted. The sensor value is, for example, video (image) data of a monitoring camera. However, as described above, the sensor 1 is not necessarily limited to the monitoring camera, and the sensor value may be detection data such as temperature and smoke. is there.

  The sensor data management device 2 includes a data management unit 21, a sensor data DB (Data base) 22, and a sensor information DB 23.

  The data management unit 21 has a function of managing sensor values transmitted from the plurality of sensors 1 as grouping data 22A, learning data 22B, and detection data 22C using the sensor data DB 22. For example, one sensor value can be used redundantly as grouping data 22A, learning data 22B, and detection data 22C. The grouping data 22A is data used for a process (also referred to as clustering) for grouping a plurality of sensors 1 described later. The learning data 22B is data used for learning a normal action for detecting a characteristic action. The detection data 22C is data used for detecting characteristic behavior. The data management unit 21 stores these pieces of data in the sensor data DB 22 so that they can be read out in time series for each sensor 1. The data management unit 21 can also delete each data in the sensor data DB 22 for each sensor 1.

  Further, the data management unit 21 has a function of managing information (sensor information) about the sensor 1 including information that can grasp the increase / decrease in the number of sensors 1 by the sensor information DB 23. When the sensor 1 is added or removed, the sensor information DB 23 is updated by the system administrator. This update is preferably performed each time the sensor 1 is added or removed, but it is performed in batches for the sensors 1 that have been added or removed within a certain period of time. There may be. The sensor information DB 23 holds the latest (after update) sensor information and sensor information before update of at least one generation. It is preferable that the sensor information is information that can grasp each of the number of additions and removals of the sensors 1 instead of the difference of the total number of sensors 1 as the increase or decrease of the sensors 1. Furthermore, it is preferable that sensor information is information which can grasp | ascertain the number of the sensors 1 in which installation conditions were changed, for example. That is, it is preferable that the system administrator updates the sensor information DB 23 for the sensor 1 whose installation conditions are changed, for example. For example, for the sensor 1 whose installation condition has been changed, the sensor information may be updated on the assumption that the sensor 1 has been removed and newly added. Here, in this characteristic behavior detection system 100, it should be noted that the system administrator does not need to register information related to grouping of the sensors 1 in the sensor information DB 23 as sensor information or the like, for example. The reason will be described later.

  The data management unit 21 has a function of transmitting various data stored in the sensor data DB 22 and sensor information stored in the sensor information DB 23 to the characteristic behavior detection device 3 (data acquisition unit 31). The method of communicating between the sensor data management device 2 and the characteristic behavior detection device 3 is not limited to a specific method, and although it does not matter here, for example, a method of communicating via a network such as the Internet can be adopted.

  The feature behavior detection device 3 includes a data acquisition unit 31, an object detection unit 32, a feature amount calculation unit 33, a grouping unit 34, a group management unit 35, a machine learning unit 36, a feature behavior detection / identification unit 37, and a learning model DB 38. And a group information DB 39. A part or all of the data acquisition unit 31, the object detection unit 32, the feature amount calculation unit 33, the grouping unit 34, the group management unit 35, the machine learning unit 36, and the feature behavior detection / identification unit 37 may be processed by each processing procedure. May be realized by causing a processor to execute a program in which is described, or may be realized by a dedicated electronic circuit. FIG. 3 shows an example of the hardware configuration of the characteristic behavior detection device 3.

  As shown in FIG. 3, the characteristic behavior detection device 3 includes a processor 51, a main memory 52, an external storage device 53, a communication device 54, an input device 55, and a display device 56. The characteristic behavior detection device 3 includes a bus 57 for interconnecting them. As described above, the characteristic behavior detection device 3 may be constructed by two or more computers (server and file server, etc.), and FIG. 3 only schematically shows a hardware configuration example thereof. Absent.

  Here, the characteristic behavior detection device 3 is shown in FIG. 2 by the characteristic behavior detection program 3A stored in the external storage device 53 being loaded from the external storage device 53 into the main memory 52 and executed by the processor 51. The data acquisition unit 31, the object detection unit 32, the feature amount calculation unit 33, the grouping unit 34, the group management unit 35, the machine learning unit 36, and the feature behavior detection / identification unit 37 are realized. Further, the learning model DB 38, the group information DB 39, and an action pattern DB 344 described later are assumed to be built in the external storage device 53.

  The communication device 54 is a device that executes communication among the sensor data management device 2, the reporting device 4, and the information distribution device 5. The input device 55 is a device for an operator of the characteristic behavior detection device 3 to input information including a command. The display device 56 is a device for outputting information to an operator of the characteristic behavior detection device 3 or the like.

  Returning to FIG. 2, the description will be continued. The data acquisition unit 31 has a function of acquiring various data as sensor values of the sensor 1 and sensor information from the sensor data management device 2 (data management unit 21). Various data acquired by the data acquisition unit 31 is passed to the object detection unit 32 and the feature amount calculation unit 33, and the sensor information is passed to the grouping unit 34. The exchange of various data between the sensor data management device 2 (data management unit 21) and the data acquisition unit 31 can be performed as a so-called real-time process almost simultaneously with the data acquisition by the sensor 1, or sensor data After being stored in the DB 22, it may be implemented as a so-called batch process as necessary.

  The object detection unit 32 detects, for example, a person or a group of persons existing in the detection target area of the sensor 1. Hereinafter, a person or a group of persons to be detected by the object detection unit 32 may be referred to as an object or an object. Various known methods can be adopted as a method for detecting an object (here, the method is not limited). The detection result of the object by the object detection unit 32 is notified to the feature amount calculation unit 33, the grouping unit 34, the machine learning unit 36, and the feature behavior detection / identification unit 37.

  The feature amount calculation unit 33 calculates the feature amount of the object notified from the object detection unit 32. The feature amount is a value that can represent the behavior of the object, and is information including, for example, a density and a motion vector. The feature amount calculation unit 33 includes a learning / detection feature amount calculation unit 331, a motion vector calculation unit 332, and a normalization unit 333.

  The learning / detection feature quantity calculation unit 331 calculates a feature quantity of an object used for learning a normal action for detecting a characteristic action or used for detecting a characteristic action. Various known methods can be adopted as a method for calculating the feature amount of the object (here, the method is not limited).

  The motion vector calculation unit 332 calculates the feature vector of the target object notified from the target object detection unit 32 and particularly suitable for use in grouping of the sensors 1. Various known methods can be adopted as the method for calculating the motion vector (here, the method is not limited). The grouping and motion vectors of the sensors 1 will be described later.

  The learning / detection feature amount calculated by the learning / detection feature amount calculation unit 331 and the motion vector calculated by the motion vector calculation unit 332 are passed to the normalization unit 333. The normalization unit 333 performs normalization on the received learning / detection feature quantities and motion vectors to eliminate the influence of, for example, differences in installation conditions between the sensors 1. The processing of the normalization unit 333 will be described later. The learning / detection feature quantity normalized by the normalization unit 333 is passed to the machine learning unit 36 and the feature behavior detection / identification unit 37. The motion vector normalized by the normalization unit 333 is passed to the grouping unit 34 (behavior pattern determination unit 341).

  The machine learning unit 36 learns a normal action for detecting a feature action using the learning / detecting feature quantity received from the feature quantity calculation unit 33 (normalization unit 333). The feature behavior detection / identification unit 37 uses, as a feature behavior, a behavior different from the normal behavior learned by the machine learning unit 36 using the learning / detection feature amount received from the feature amount calculation unit 33 (normalization unit 333). Detect. Various known methods can be adopted as a method of learning normal behavior (here, the method is not questioned). Also, various known methods can be adopted as a method for detecting a behavior different from a normal behavior as a characteristic behavior (here, the method is not limited). When detecting the characteristic behavior, the characteristic behavior detection / identification unit 37 identifies the behavior content of the characteristic behavior. Various known methods can be adopted as the method for identifying the action content (here, the method does not matter). The result of the detection of the characteristic behavior and the identification of the behavior content by the characteristic behavior detection / identification unit 37 is notified to the reporting device 4 and the information distribution device 5, and the notification device 4 notifies the staff and uses the information distribution device 5. Information distribution to the person is executed.

  By the way, if data collected from a plurality of sensors 1 with different installation locations or different installation conditions are mixed, the learning accuracy of normal behavior by the machine learning unit 36 decreases, and accordingly, characteristic behavior detection / There is a possibility that the detection accuracy of characteristic behavior (which is behavior different from normal behavior) by the identification unit 37 is lowered. Therefore, the feature behavior detection device 3 in the feature behavior detection system 100 maintains the normal behavior learning accuracy by the machine learning unit 36, and also maintains the feature behavior detection accuracy by the feature behavior detection / identification unit 37. A grouping unit 34 and a group management unit 35 are provided. Note that the normalization unit 333 of the feature amount calculation unit 33 also maintains the normal behavior learning accuracy by the machine learning unit 36 and also contributes to maintaining the detection accuracy of the characteristic behavior by the feature behavior detection / identification unit 37. Although it is unique to the behavior detection system 100 (characteristic behavior detection device 3), for the sake of easy understanding, description will be made separately from the description of the grouping unit 34 and the group management unit 35.

  FIG. 4 is a conceptual diagram for explaining grouping of the sensors 1 by the action pattern of the grouping unit 34. With reference to FIG. 4 and FIG. 2 together, first, the behavior pattern determination unit 341 and the new grouping unit 342 of the grouping unit 34 will be described.

  As described above, the grouping unit 34 receives the notification from the object detection unit 32, and from the feature amount calculation unit 33 (calculated by the motion vector calculation unit 332 and normalized by the normalization unit 333). Receive motion vectors.

  The motion vector calculation unit 332 of the feature amount calculation unit 33 indicates, for example, the movement indicating the movement direction and the movement amount at the acquisition interval of two temporally continuous sensor values for each of the objects notified from the object detection unit 32. Calculate the vector. For example, when the sensor 1 is a surveillance camera, the motion vector calculation unit 332 calculates a motion vector indicating the movement direction and the movement amount of the object between frames. The behavior pattern determination unit 341 of the grouping unit 34 determines a behavior pattern for each object notified from the object detection unit 32 based on the motion vector received from the feature amount calculation unit 33 (STEP 1).

  For example, the behavior pattern determination unit 341 classifies the movement direction into eight directions, up, right diagonally up, right, diagonally down right, down, diagonally down left, left, diagonally up left, and the amount of movement, that is, speed. Are classified into three speeds, low speed, normal speed, and high speed. The behavior pattern determination unit 341 determines a behavior pattern of the target object from 25 ways in which 8 directions × 3 speed = 24 kinds of stops are added, and totals the results on the behavior pattern table. The direction and speed classification numbers are merely examples, and are not limited to these. The total of 25 numerical values on the behavior pattern table is the number of objects notified from the object detection unit 32. The behavior pattern determination unit 341 stores the behavior pattern table created in this way in the behavior pattern DB 344.

  The new grouping unit 342 of the grouping unit 34 groups the sensors 1 using the behavior pattern table stored in the behavior pattern DB 344. The timing at which the sensors 1 are grouped by the new grouping unit 342 includes firstly when the characteristic behavior detection system 100 is introduced. When the characteristic behavior detection system 100 is introduced, for example, when 1000 sensors 1 are installed in the entire characteristic behavior detection system 100, the process of dividing the 1000 sensors 1 into several groups is a new grouping unit as an initial process. Executed by H.342. Secondly, there is a case where the rate of increase or decrease of the sensor 1 in the characteristic behavior detection system 100 exceeds a threshold value. This increase / decrease can be acquired from the sensor information stored in the sensor information DB 23 of the sensor data management device 2. The increase / decrease of the sensor 1 is preferably not the number of differences of the total number of sensors 1 but the number of fluctuations of the sensor 1. For example, even if X units are added, Y units are removed, and the installation conditions of Z units are changed, and the ratio of the total number of | X−Y | When the ratio of the total number of X + Y + Z exceeds the threshold value, it is preferable that the process of dividing the sensor 1 into several groups is executed again. It should be noted that various methods can be employed for determining whether to perform grouping of the sensors 1, for example, comparing the number of expansions and the number of removals with threshold values.

  First, the new grouping unit 342 extracts, in a time series, the number of detected behavior patterns of each sensor 1 in a predetermined period such as one day or one week (STEP 2). For example, the new grouping unit 342 determines the number of detected 25 types of action patterns as described above, such as an action pattern that moves upward at a low speed as action 1 and an action pattern that moves at a low speed and diagonally upward to action 2. Extract by behavior pattern and time series. In addition, the new grouping unit 342 extracts the total number of detected behavior patterns in time series.

  And the new grouping part 342 calculates | requires the time change rate of the detection number for every action pattern in the predetermined period mentioned above, and transition of the total detection number of all the action patterns, and groups the sensors 1 based on these. Execute (STEP 3).

  From the time change rate of the number of detections for each behavior pattern, it is possible to capture behavioral trends and behavioral changes due to time zones. Further, the trend of the number of detections in each sensor 1 can be grasped from the transition of the total number of detections of all the behavior patterns. By using these as a reference, the new grouping unit 342 can group the sensors 1 so that the sensors 1 having a high relationship between the behavior tendency and the scale tendency belong to the same group. . Thus, the characteristic behavior detection device 3 of the present embodiment has a unique function of automatically grouping the sensors 1 based on the behavior of the object detected by the sensor 1. It should be noted that various known methods can be adopted as a grouping method according to a given standard (in this case, the method does not matter). The group management unit 35 is notified of the result of grouping the sensors 1 by the new grouping unit 342.

  When the sensor 1 is added, the additional sensor processing unit 343 of the grouping unit 34 performs processing such as which group the sensor 1 is incorporated into. The processing of the additional sensor processing unit 343 will be described later. The processing result of the additional sensor processing unit 343 is also notified to the group management unit 35.

  Upon receiving a notification from the grouping unit 34 (new grouping unit 342, additional sensor processing unit 343), the group management unit 35 creates and updates group information that is information related to the group of sensors 1. The group information is stored in the group information DB 39. Further, the group management unit 35 manages the learning model for each group indicated by the group information using the learning model DB 38.

  The group management unit 35 passes the group information stored in the group information DB 39 and the learning model stored in the learning model DB 38 to the machine learning unit 36 and the characteristic behavior detection / identification unit 37.

  The machine learning unit 36 can determine which group of sensors 1 receives the learning / detection feature amount received from the feature amount calculation unit 33 based on the group information received from the group management unit 35. The normal behavior of the object is learned using the learning model associated with the group in the learning model received from the unit 35. The machine learning unit 36 does not necessarily have a function of determining which group of sensors 1 the learning / detection feature amount received from the feature amount calculation unit 33 is based on, for example, the group information. For example, the machine learning unit 36 may perform learning for each group using one learning model given from the group management unit 35 under the control of the group management unit 35. Of course, in this case, the learning / detection feature quantity supplied to the machine learning unit 36 is controlled by the group management unit 35 only to those relating to the sensors 1 belonging to each group. The learning result of the normal action by the machine learning unit 36 is returned to the group management unit 35, and the learning model in the learning model DB 38 is updated. When the learning model in the learning model DB 38 is updated, the group management unit 35 passes the updated learning model to the feature behavior detection / identification unit 37 at a predetermined timing.

  Similarly to the machine learning unit 36, the feature behavior detection / identification unit 37 also uses the group information received from the group management unit 35 to determine which group the sensor 1 receives the learning / detection feature amount from the feature amount calculation unit 33. Since the learning model associated with the group in the learning model received from the group management unit 35 is also used, the characteristic behavior of the target object (behavior different from the normal behavior shown in the learning model) Is detected. When the characteristic behavior is detected, the characteristic behavior detection / identification unit 37 identifies the behavior content of the characteristic behavior and notifies the notification device 4 and the information distribution device 5 of the detection and recognition results. Thereby, notification to the staff by the reporting device 4 and information distribution to the user by the information distribution device 5 are executed.

  In this way, the characteristic behavior detection device 3 has a unique function of automatically grouping the sensors 1 based on the behavior of the object detected by the sensor 1, so that the installation location is different or the installation is different. Even if data collected from a plurality of sensors 1 with different conditions are mixed, the learning accuracy of normal behavior by the machine learning unit 36 is maintained, and the detection accuracy of characteristic behavior by the characteristic behavior detection / identification unit 37 is maintained. be able to. Therefore, for example, the collection period of learning data can be shortened, and the period from the introduction of the feature behavior detection system 100 to the start of operation can be shortened. On the other hand, when installing the sensor 1, the system administrator does not need to consider which group the sensor 1 is incorporated into. In other words, the system administrator does not need to register information related to the grouping of sensors 1 in the sensor information DB 23 as sensor information, for example.

  FIG. 5 is a flowchart showing a flow of processing related to grouping of sensors 1 and normal behavior learning in the characteristic behavior detection device 3.

  The characteristic behavior detection device 3 first executes a new grouping that is a grouping for all the sensors 1 in the characteristic behavior detection system 100 (step A1). This new grouping is executed by the group management unit 35, the data acquisition unit 31, the object detection unit 32, the feature amount calculation unit 33, and the grouping unit 34 under the control of the group management unit 35. Is done. FIG. 6 shows a detailed flow of processing relating to new grouping.

  The grouping unit 34 executes the processes of Step B1 to Step B16 for each of all the sensors 1. More specifically, in step B1, the sensor 1 to be processed is selected. In step B16, the presence or absence of the unprocessed sensor 1 is determined. If there is any unprocessed sensor 1, the process returns to step B1, If the sensor 1 has been processed, the process exits the loop from Step B1 to Step B16.

  When the sensor 1 to be processed is selected in step B1, the grouping unit 34 executes the processing of step B2 to step B14 for a certain period such as one day or one week. More specifically, in step B2, a time zone to be processed is selected in time series in units of, for example, a few seconds or minutes, and in step B14, it is determined whether or not the processing for a certain period has been completed. If there is a remaining time zone, the process returns to step B2, and if all the time zones for a certain period have been processed, the process exits the loop from step B2 to step B14.

  When the time zone to be processed is selected in step B2, the grouping unit 34 manages the acquisition of data acquired in the time zone selected in step B2 by the sensor 1 selected in step B1. The data acquisition unit 31 is requested via the unit 35 (step B3). For the data acquired by the data acquisition unit 31, the processes of Step B4 to Step B13 are executed. More specifically, in step B4, data to be processed is selected. In step B13, the presence or absence of unprocessed data is determined. If unprocessed data remains, the process returns to step B4, and all data is stored. If the processing has been completed, the process exits the loop from Step B4 to Step B13.

  The object detection unit 32 detects the object from the data selected as the processing object in step B4 (step B5). For the object (detected object) detected by the object detection unit 32, the processes of Step B6 to Step B12 are executed. More specifically, in step B6, a detection object to be processed is selected, and in step B12, the presence or absence of an unprocessed detection object is determined. If an unprocessed detection object remains, the process returns to step B6, and all If the detected object has been processed, the process exits the loop from Step B6 to Step B12.

  The motion vector calculation unit 332 of the feature amount calculation unit 33 uses the motion vector of the detected object selected as the processing target in step B6 and the data selected as the processing target in step B4 (and, for example, the previous data). (Step B7). The normalization unit 333 of the feature amount calculation unit 33 performs normalization on the motion vector calculated by the motion vector calculation unit 332 to eliminate the influence of, for example, a difference in installation conditions between the sensors 1 (step) B8). The processing of the normalization unit 333 will be described later.

  The behavior pattern determining unit 341 of the grouping unit 34 calculates the direction of the detected object from the motion vector calculated by the motion vector calculating unit 332 and normalized by the normalizing unit 333 (step B9). Is calculated (step B10). The behavior pattern determination unit 341 determines the behavior pattern of the detected object based on the calculated direction and speed of the detected object, and counts up the number of detected behavior patterns (step B11).

  When the sensor 1 selected as the processing target in step B1 finishes counting up the number of detected behavior patterns in step B11, the new grouping unit 342 of the grouping unit 34 performs a behavioral pattern for each behavior pattern within a certain period. The time change rate of the detected number is calculated (step B15). When the calculation of the time change rate of the number of detections for each behavior pattern in step B15 is completed for all sensors 1, the new grouping unit 342 determines the time change rate of the number of detections of the behavior pattern in each sensor 1, and Grouping is executed based on the transition of the total number of detected behavior patterns (step B17). The group management unit 35 stores the grouping result by the new grouping unit 342 and the grouping reference information as group information in the group information DB 39 (step B18).

  FIG. 7 shows an example of group information stored in the group information DB 39.

  As shown in FIG. 7, the group information is a data structure in a table format including, for example, a group ID field, a sensor ID field, a learning model ID field, and a grouping criterion information field.

  The group ID field holds an identifier of each group. The sensor ID field holds an identifier of the sensor 1 belonging to each group. In the learning model ID field, an identifier of a learning model to be used by each group (a learning model held in the learning model DB 38) is held. Although the grouping criterion information will be described later, the grouping criterion information is information indicating the territory of each group.

  Returning to FIG. When the new grouping of step A1 is completed, the characteristic behavior detection device 3 executes the processes of step A2 to step A29 for every arbitrary grouping cycle such as one month or three months. This grouping cycle is determined in order to cope with a change with time such as a seasonal change. That is, the characteristic behavior detection device 3 can adaptively respond to changes in normal behavior such as how people flow and gather in summer and winter, for example.

  Furthermore, the characteristic behavior detection device 3 executes the processing of step A3 to step A26 for every arbitrary learning cycle such as one day or one week. This learning cycle is determined to correspond to, for example, a newly installed sensor 1.

  More specifically, the group management unit 35 first checks whether the sensor information DB 23 managed by the sensor data management device 2 is updated (step A4). FIG. 8 shows an example of sensor information stored in the sensor information DB 23.

  As shown in FIG. 8, the sensor information is a data structure in a table format including, for example, an update date / time field and a sensor ID field.

  The date and time when the sensor information DB 23 (sensor information) was updated by the system administrator is held in the update date and time field. In the sensor ID field, for example, an identifier of the sensor 1 existing at the time of the update date and time is held. The sensor information is retained for at least two generations after the first update. Based on this sensor information, the added sensor 1 and the deleted sensor 1 can be recognized. Here, for example, for the sensor 1 whose installation conditions are changed, the original identifier is deleted and a new identifier is added.

  For example, the group management unit 35 can determine that the sensor information DB 23 has been updated when the sensor information DB 23 holds the update date and time after one learning cycle before that point. When only one generation of sensor information exists, the sensor information is retained as an initial setting when the characteristic behavior detection system 100 is introduced. In that case, the characteristic behavior detection device 3 determines that the sensor information DB 23 has been updated.

  If the sensor information DB 23 has been updated (step A4), then the group management unit 35 checks whether the increase / decrease in the sensor 1 between before and after the update is greater than or equal to the threshold (step A5). . As described above, the increase / decrease of the sensor 1 is preferably not the difference number of the total number of sensors 1 but the number of fluctuations of the sensor 1. When the increase / decrease in the sensor 1 is equal to or greater than the threshold (YES in step A5), the group management unit 35 includes all the existing group information and all the learning models, that is, all the group information stored in the group information DB 39. All the learning models stored in the learning model DB 38 are deleted (step A6), and the new grouping described with reference to FIG. 6 is executed again by the new grouping unit 342 or the like (step A7).

  On the other hand, when the increase / decrease of the sensor 1 is less than the threshold value (NO in step A5), the group management unit 35 checks whether the number of sensors 1 is increasing (step A8). Here, it is preferable to check whether or not the number of sensors 1 is increasing is to check whether or not the added sensor 1 exists. When the added sensor 1 exists (YES in step A8), the group management unit 35 causes the additional sensor processing unit 343 to execute a process for the added sensor 1 (step A9). Here, with reference to FIG. 9, the outline | summary of the process with respect to the added sensor 1 is demonstrated.

  The new grouping unit 342 of the grouping unit 34 calculates a sample point of each sensor 1 belonging to each group when executing the new grouping. Also, the new grouping unit 342 calculates the center point of each group from these, and calculates the distance to the sample point farthest from the center point in each group. These are held in the group information DB 39 as grouping reference information (see FIG. 7). This sample point is, for example, a position when a value obtained by quantifying a normal behavior tendency of an object detected by each sensor 1 is projected in a two-dimensional or three-dimensional space. Various known methods can be adopted as the method for calculating the sample point, the method for calculating the center point, and the method for calculating the distance from the center point to the sample point (here, the method does not matter). .

  The additional sensor processing unit 343 of the grouping unit 34 also calculates sample points for the added sensor 1. In FIG. 9, it is assumed that the symbol a1 indicates a group center point held as grouping reference information, and the symbol a2 indicates a sample point having the longest distance from the group center point. Also, it is assumed that the symbols a3-1 and a3-2 indicate sample points calculated by the additional sensor processing unit 343, respectively. The group center point a1 illustrated in FIG. 9 is the group center point closest to the sample points a3-1 and a3-2 of the added sensor 1 among the group center points of the existing group.

  The additional sensor processing unit 343 includes the distance A, which is the distance from the group center point a1 to the sample point a2 having the largest distance from the group center point, and the sample point a3- of the added sensor 1 from the group center point a1. 1 and a3-2 is compared with the distance B. If the distance B is equal to or less than the distance A (in the case of the position a3-1), the added sensor 1 is incorporated into the group. On the other hand, when the distance B exceeds the distance A (when the position is a3-2), the additional sensor processing unit 343 generates a group of only the added sensor 1. That is, a circular area having a radius from the group center point a1 to the sample point a2 indicates a group territory. In other words, the grouping criterion information indicates the territory of the group.

  Thus, when the sensor 1 is newly installed, the characteristic behavior detection device 3 can incorporate the sensor 1 into an existing group under certain conditions. Therefore, additional learning for the sensor 1 can be performed. It can be made unnecessary, and it is possible to detect the characteristic behavior by the sensor 1 using an existing learning model immediately and with high accuracy.

  FIG. 10 is a flowchart showing a detailed flow of processing (step A9 in FIG. 5) for the added sensor. This process is executed under the control of the group management unit 35 by cooperation of the group management unit 35, the data acquisition unit 31, the object detection unit 32, the feature amount calculation unit 33, and the grouping unit 34. .

  The grouping unit 34 acquires grouping reference information held in the group information DB 39 via the group management unit 35 (step C1). The grouping unit 34 performs the processing of Step C2 to Step C21 for each of the added sensors 1. Of steps C2 to C21, steps C3 to C16 are the same as steps B2 to B15 of the new grouping described with reference to FIG.

  The additional sensor processing unit 343 of the grouping unit 34 calculates which existing group center point is closest to the sample point representing the added sensor 1 (step C17). The additional sensor processing unit 343 checks whether or not the sample point representing the added sensor 1 is closer to the center point than the existing sample point having the largest distance from the center point in the group (step) C18). If close (YES in step C18), the additional sensor processing unit 343 incorporates the added sensor 1 into the group (step C19). On the other hand, if it is far away (NO in step C19), the additional sensor processing unit 343 creates a group of only the added sensor 1 (step C20).

  When the grouping unit 34 finishes incorporation of all the added sensors 1 into the existing group or creation of the group of only the added sensor 1, the group management unit 35 saves the new grouping reference information. The group information DB 39 is updated via (step C22).

  Returning to FIG. It is preferable that checking whether or not the number of sensors 1 in step A8 has increased further includes checking whether or not there is a deleted sensor 1. When the deleted sensor 1 exists (NO in step A8), the group management unit 35 deletes the information of the sensor 1 held in the group information DB 39 (step A10). Steps A9 and A10 are not necessarily executed exclusively and selectively, and both can be executed.

  Next, the group management part 35 performs the process of step A11 to step A20 about the group in which learning is not implemented. The group in which learning is not performed is, for example, the sensor 1 created by the additional sensor processing unit 343 when all the groups or sensors 1 when the grouping is performed by the new grouping unit 342 are added. Is only a group. Further, the plurality of groups in the case where one group is divided into a plurality of groups, which will be described later, also corresponds.

  For each group, the group management unit 35 checks whether there is a learning model assigned to the group (step A12). The group in which learning is not performed and the assigned learning model does not exist is, for example, an increased group when one group is divided into a plurality of groups.

  When the learning model exists (YES in step A12), the group management unit 35 reads the learning model from the learning model DB 38 (step A13). The group management unit 35 supplies the read learning model to the machine learning unit 36. On the other hand, when there is no learning model (NO in step A12), the group management unit 35 creates a new learning model (step A14) and supplies it to the machine learning unit 36.

  The object detection unit 32 detects the object from the data acquired by the sensors 1 belonging to the group to be processed (step A15), and the learning / detection feature quantity calculation unit 331 of the feature quantity calculation unit 33 is detected. The feature amount of the target object is calculated (step A16). The normalization unit 333 of the feature amount calculation unit 33 normalizes the feature amount based on the distance information (step A17). The normalization by the normalization unit 333 will be described later, including the normalization of the motion vector in step B8 in FIG. 6 (also in step C9 in FIG. 10).

  The machine learning unit 36 that receives the feature amount calculated by the feature amount calculation unit 33 performs machine learning using the learning model assigned to the group (step A18). As described above, here, a normal action for detecting a characteristic action is learned. It is assumed that various learning parameters necessary for machine learning are sufficiently adjusted. Machine learning by the machine learning unit 36 makes the learning model suitable for the group. The learning model adapted for the group is stored in the learning model DB 38 via the group management unit 35 (step A19).

  When the machine learning is finished for each of the groups for which learning is not performed, the group management unit 35 checks whether there is a group whose evaluation value at the end of learning does not satisfy the reference value (step A21). The evaluation value varies depending on the learning method. For example, in the case of supervised learning, a learning error is applied. It should be noted that various known methods can be adopted as a method for evaluating machine learning (here, the method does not matter).

  When there is a group that does not satisfy the reference value (YES in step A21), the group management unit 35 checks whether or not there is a group including two or more sensors 1 (step A22). If it exists (YES in step A22), the group management unit 35 divides the group (step A23 to step A25) in order to create a group of only the sensors 1 included in the group. And the group management part 35 repeats the process from step A11 about each divided | segmented group. By optimizing the group by dividing the group, the group management unit 35 is configured to take a countermeasure when the machine learning by the machine learning unit 36 is not successful in units of groups.

  When there is no group that does not satisfy the reference value (NO in Step A21), or when there is a group that does not satisfy the reference value but there is no group that includes two or more sensors 1 (NO in Step A22), The group management unit 35 exits the loop of step A3 to step A26. The group management unit 35 deletes all the existing group information and all the learning models for each arbitrary grouping cycle in order to cope with the change over time such as the seasonal change described above (step A27), and creates a new grouping. Is newly executed by the new grouping unit 342 or the like (step A28).

  FIG. 11 shows an example of the state of the characteristic behavior detection system 100 in which the grouping of the sensors 1 based on the behavior pattern is executed by the characteristic behavior detection device 3.

  As shown in FIG. 11, in the characteristic behavior detection system 100, a plurality of sensors 1 are grouped, and a learning model is assigned to each group. Which sensor 1 belongs to each group, which learning model is assigned to each group, and the like are managed by the group information DB 39.

  Next, the normalization of the feature amount in step A17 in FIG. 5 and the normalization of the motion vector in step B8 in FIG. 6 (the same applies to step C9 in FIG. 10) will be described.

  For example, depending on the installation conditions of the sensor 1 such as the installation height and the depression angle, the observed values such as the apparent size and speed of the object change. Therefore, the normalization unit 333 uses the distance information of the sensors 1 to eliminate the influence of differences in installation conditions between the sensors 1 and the feature amounts and movements calculated by the learning / detection feature amount calculation unit 331. The motion vector calculated by the vector calculation unit 332 is normalized. Here, the distance information is, for example, information representing the distance from the sensor 1 to the detection target position at each sample point (for example, a pixel when the sensor 1 is a monitoring camera) in the detection area of the sensor 1. The distance information is held in the sensor information DB 23 of the sensor data management device 2 and is acquired via the data acquisition unit 31. In other words, it is assumed that the sensor information DB 23 holds information regarding installation conditions of the sensor 1 including distance information as sensor information. The distance information is not necessarily included in the sensor information DB 23 and may be stored in another DB.

  FIG. 12 is a flowchart showing a flow of processing relating to collection of learning data including normalization by the normalization unit 333.

  The characteristic behavior detection device 3 executes the processes of Step D1 to Step D16 for each group that is a collection target of learning data. Moreover, the characteristic action detection apparatus 3 performs the process of step D2-step D15 for every sensor 1 in a group. Furthermore, the characteristic behavior detection device 3 executes the processes of Step D3 to Step D14 for a certain period.

  More specifically, the data acquisition unit 31 acquires, in time series, data acquired by the sensor 1 for the processing target sensor 1 in the processing target group (step D4). Moreover, the data acquisition part 31 acquires the distance information of the sensor 1 (step D5).

  The object detection unit 32 detects the object from each of the acquired data (step D7). In addition, the learning / detection feature quantity calculation unit 331 of the feature quantity calculation unit 33 calculates an arbitrary feature quantity for each detected object (step D9).

  The normalization unit 333 extracts distance information for the coordinates of the detected object from the distance information of the target sensor 1 (step D10), and normalizes the feature quantity by combining the feature quantity and the distance information (step D11). . This normalization can be executed by, for example, the following equation (1).

Here, f _i is a normalized value at coordinate i, x _i is a feature value at coordinate i, S _max is a maximum value of distance information, and S _i is a distance information value at coordinate i.

  FIG. 13 is a flowchart showing a flow of processing related to detection of characteristic behavior including normalization by the normalization unit 333.

  The characteristic behavior detection device 3 executes the processes of Step E1 to Step E9 for each arbitrary detection cycle such as 1 minute or 5 minutes. More specifically, the object detection unit 32 detects the object from data acquired by the sensors 1 belonging to the group to be processed (step E2), and the learning / detection feature quantity calculation unit of the feature quantity calculation unit 33 331 calculates the feature amount of the detected object (step E3). In addition, the normalization unit 333 of the feature amount calculation unit 33 normalizes the feature amount based on the distance information (step E4).

  The feature behavior detection / identification unit 37 that receives the feature amount calculated by the feature amount calculation unit 33 detects the feature behavior using the learning model assigned to the group (step E5). When the characteristic behavior is detected (YES in Step E6), the characteristic behavior detection / identification unit 37 identifies the behavior content of the characteristic behavior (Step E7), and the detection / identification result is notified to the reporting device 4 or the information distribution device. 5 (step E8).

  The normalization by the normalization unit 333 for the motion vector calculated by the motion vector calculation unit 332 for grouping is also normalized by the normalization unit 333 for the feature amount calculated by the learning / detection feature amount calculation unit 331. Since the process is executed based on the distance information of the sensor 1, the description thereof is omitted here.

  In this manner, the characteristic behavior detection device 3 maintains the normal behavior learning accuracy by the machine learning unit 36 by eliminating the influence of differences in installation conditions between the sensors 1, and the characteristic behavior detection / identification unit The detection accuracy of the characteristic behavior by 37 can be maintained.

  FIG. 14 is a diagram for explaining again an example of the effect on the characteristic behavior detection system 100 when the grouping of the sensors 1 based on the behavior pattern is executed by the characteristic behavior detection device 3.

  For example, as shown in FIG. 14A, a case where only one sensor 1 is installed is assumed. In this case, although it is not necessary to consider a decrease in the detection accuracy of the characteristic behavior due to a mixture of sensor values of a plurality of sensors 1, learning data (sensor value for learning a normal behavior for detecting the characteristic behavior) ) Is required for a long period of time, and there is a certain period of time from the introduction of the system for detecting characteristic behavior to the start of operation.

  Further, if data is collected from a plurality of sensors 1 as shown in FIG. 14B in order to shorten the learning data collection period, the accuracy of learning of normal actions by machine learning is reduced, and therefore, characteristics The accuracy of behavior detection decreases. This is fatal as a system for detecting characteristic behavior.

  On the other hand, as described above, in the feature behavior detection system 100 according to the present embodiment in which the feature behavior detection device 3 performs grouping of the sensors 1 based on the behavior pattern, as shown in FIG. The period is shortened and the accuracy of normal behavior learning by machine learning is maintained, and thus the detection accuracy of characteristic behavior is maintained. That is, the characteristic behavior detection apparatus 3 of the present embodiment can shorten the learning data collection period and maintain the characteristic behavior detection accuracy.

  When the background and features in the above-described embodiment are roughly classified and summarized, they can be described as follows.

  Point of interest 1: Since the normal behavior of the detection target varies depending on the installation location of the sensor, the detection accuracy decreases. For example, at a ticket gate at a station, “the direction of the flow of people is constant” and “the people continue to move”, whereas in a large plaza, “the direction of the flow of people is indefinite”, “people move and stop. This phenomenon causes a decrease in detection accuracy.

  Therefore, as a solution, each sensor is automatically grouped into a plurality of groups on the basis of “behavior” to be detected so that sensors in the same group can perform machine learning using the same learning model. did.

  As a result, sensors with the same “normal behavior” to be detected can detect characteristic behavior with the same learning model. Even if a sensor is added, the sensor can be incorporated into an existing group (an existing learning model is applied), and new learning for such addition becomes unnecessary.

  Point of interest 2: Since the observation value (such as the apparent size of the object) of the detection object changes depending on the sensor installation conditions (installation height, depression angle, etc.), the detection accuracy decreases.

  Therefore, as a solution, the learning data collected by each sensor in the group is normalized, and the learning data is handled in a unified manner. In other words, the learning data collected by each sensor is subjected to normalization processing using “distance information”, that is, information representing the distance from the sensor at each sample point (e.g., pixel) in the sensor detection area. Yes.

  As a result, data collected in different environments can be handled uniformly within the group.

  Therefore, this embodiment can maintain detection accuracy while reducing the learning data collection period.

Furthermore, the above-described embodiment includes the concept of a characteristic behavior detection method. That is, this method can be expressed as follows.
(1) A characteristic behavior detection method for a characteristic behavior detection system having an acquisition unit that acquires sensor values of a plurality of sensors, wherein the plurality of sensors are newly created based on a behavior pattern of an object detected from the sensor values. A characteristic behavior detection method comprising grouping.
(2) The new grouping is based on the time change rate of the detected number of each behavior pattern in the first period and the transition of the total value of the detected number of all behavior patterns in the first period. The characteristic behavior detection method according to (1), wherein the sensors are grouped.
(3) The characteristic behavior detection method according to (1) or (2), wherein the behavior pattern of the target object is determined based on the direction and speed of movement of the target object.
(4) When a sensor whose sensor value is acquired by the acquisition unit is added, the added sensor is incorporated into one of the existing groups based on grouping standard information indicating each territory of the existing group. Or, the characteristic behavior detection method according to any one of (1) to (3), wherein a new group including only the added sensor is generated.
(5) When the increase / decrease of the sensor whose sensor value is acquired by the acquisition unit exceeds a threshold value, the plurality of sensors are grouped by the new grouping. The characteristic action detection method as described in one.
(6) As group management, an evaluation value with respect to a result of machine learning of normal behavior of the target object for detecting characteristic behavior of the target object that is executed using sensor values of sensors belonging to each group is a reference value (5) The feature behavior detection method according to (5), wherein when there is a group that does not satisfy the above, the group is divided into groups each composed of only sensors belonging to the group.
(7) The characteristic behavior detection method according to (5) or (6), in which grouping of the plurality of sensors is performed by the new grouping for each second period as group management.
(8) Any one of (1) to (7) that normalizes a sensor value calculated for the object based on a distance from each sensor to each detection target position at a sample point in a detection area of each of the plurality of sensors. The characteristic behavior detection method according to claim 1.

  Further, the above-described embodiment is characterized by a program for performing the above (1) to (8) or a storage medium storing the program.

  Although several embodiments of the present invention have been described, these embodiments are presented by way of example and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the scope of the invention. These embodiments and modifications thereof are included in the scope and gist of the invention, and are included in the invention described in the claims and the equivalents thereof.

  DESCRIPTION OF SYMBOLS 1 ... Sensor, 2 ... Sensor data management apparatus, 3 ... Characteristic action detection apparatus, 4 ... Notification apparatus, 5 ... Information distribution apparatus, 21 ... Data management part, 22 ... Sensor data DB, 22A ... Grouping data, 22B Data for learning, 22C ... Data for detection, 23 ... Sensor information DB, 31 ... Data acquisition unit, 32 ... Object detection unit, 33 ... Feature amount calculation unit, 34 ... Grouping unit, 35 ... Group management unit, 36 ... machine learning unit, 37 ... feature behavior detection / identification unit, 38 ... learning model DB, 39 ... group information DB, 100 ... feature behavior detection system, 331 ... feature quantity calculation unit for learning / detection, 332 ... motion vector calculation unit 333 ... normalization unit, 341 ... behavior pattern determination unit, 342 ... new grouping unit, 343 ... additional sensor processing unit, 344 ... behavior pattern DB.

Claims (8)

An acquisition unit for acquiring sensor values of a plurality of sensors;
A new grouping unit that groups the plurality of sensors based on the behavior pattern of the target detected from the sensor values;
A characteristic behavior detection device comprising:   The new grouping unit determines the plurality of sensors based on a temporal change rate of the detected number of each behavior pattern in the first period and a transition of a total value of the detected number of all behavior patterns in the first period. The characteristic behavior detection apparatus according to claim 1, which is grouped.   The characteristic behavior detection device according to claim 1, further comprising a behavior pattern determination unit that determines a behavior pattern of the target based on a direction and speed of movement of the target.   When a sensor whose sensor value is acquired by the acquisition unit is added, the added sensor is incorporated into one of the existing groups based on grouping criterion information indicating each territory of the existing group, or 4. The characteristic behavior detection device according to claim 1, further comprising an additional sensor processing unit that generates a new group including only the added sensor. 5.   The group management unit according to claim 1, further comprising a group management unit that causes the new grouping unit to perform grouping of the plurality of sensors when the increase / decrease of the sensor whose sensor value is acquired by the acquisition unit exceeds a threshold value. The characteristic behavior detection device according to any one of the preceding claims.   The group management unit uses, as a reference value, an evaluation value for a result of machine learning of normal behavior of the target object for detecting a characteristic behavior of the target object that is executed using sensor values of sensors belonging to each group. The characteristic behavior detection apparatus according to claim 5, wherein when there is a less than group, the group is optimized by dividing the group into groups each composed of only sensors belonging to the group.   The characteristic behavior detection apparatus according to claim 5, wherein the group management unit causes the new grouping unit to group the plurality of sensors every second period.   The normalization part which normalizes the value calculated about the target object based on the distance from each sensor to each detection target position in the sample point of the detection area of each of the plurality of sensors. The characteristic behavior detection device according to claim 1.

Images