JP6943290B2 - Monitoring system, monitoring method and monitoring program - Google Patents

Description

The present invention relates to a monitoring system, a monitoring method, and a monitoring program for monitoring a subject who may cause abnormal behavior.

It is said that there are many inmates at risk of abnormal behavior in hundreds of correctional facilities (prisons) nationwide. Here, the abnormal behavior means all behaviors that do not follow the instructions of the prison officer or are unusual. With many abnormal behaviors, problems represented by events such as suicide and violence are discovered after the problem occurs, so if the discovery is delayed, it will lead to a serious accident such as suicide. Therefore, it has become a social issue to appropriately monitor inmates (monitored persons).

Currently, many correctional facilities monitor inmates on a regular basis (eg, every 15 minutes) by patrols by prison officers and by deploying surveillance cameras. However, even if such measures are taken, the fact is that the inmates who may cause abnormal behavior cannot be sufficiently monitored.

Since some inmates may behave abnormally between patrols, it is possible to increase the frequency of patrols. However, if the patrol frequency is increased, there is a problem that the load on the prison officer increases. It is also conceivable to increase the surveillance range by increasing the number of places where surveillance cameras are installed. However, if the number of surveillance cameras increases too much, there is also a problem that the load of visually checking the images of all places increases.

As a method for solving such a problem, Patent Document 1 describes a suicide suppression system for preventing suicide in a closed facility. The system described in Patent Document 1 detects the ecology of the monitored object by using a microwave sensor, and converts the detected signal into biometric information. Then, the above system divides the combination of biological data into various events such as heartbeat state, respiratory state, motor state, mental state, and presence / absence state that can occur in the facility, and the biological factor or life pattern is determined from the contents of each event. To guide. In addition, the system outputs appropriate status or parameters that, depending on these degrees, weight the risk of suicide or the likelihood of committing suicide.

Japanese Unexamined Patent Publication No. 2016-85577

In the method described in Patent Document 1, appropriate positions or parameters that weight the risk of suicide or the possibility of committing suicide against changes in biological factors, environmental factors and reinforcing factors, and lifestyle patterns are set, and each of them is set. By setting thresholds for the cumulative values of parameters in a situation, the risk of suicide and the performance of suicide are determined. As described above, in the method described in Patent Document 1, it is necessary to set a threshold value for each of many parameters, it is difficult to optimize for each person, and the load of the setting itself becomes very high. There is a problem that it ends up.

Further, in the method described in Patent Document 1, it is assumed that a notification is made after an event occurs, but if the discovery of a problem is delayed, a serious accident may occur. Therefore, it is desired to be able to appropriately monitor the monitored person who may behave abnormally while reducing the load on the monitor.

Therefore, an object of the present invention is to provide a monitoring system, a monitoring method, and a monitoring program capable of appropriately monitoring a monitored person who may behave abnormally while reducing the load on the monitor.

The monitoring system according to the present invention has a biometric information acquisition unit that acquires biometric information of the monitored person, and at least one of the posture and the position, which is the mode of the monitored person, based on the image or video of the monitored person. The acquired mode acquisition unit, the feature amount calculation unit that calculates the feature amount representing the characteristics of the monitored person based on the acquired biological information and the mode, and the monitored person's state as the objective variable of the monitored person. A score calculation unit that calculates a state transition score, which is a score representing the normal state of a monitored person, and a calculated state transition score by applying feature quantities to a model that uses biological information and modes as explanatory variables. Equipped with a notification unit that notifies the observer of the information of the monitored person who meets the predetermined criteria, and the notification unit records the video of the monitored person when the state transition score exceeds the first threshold value. Then, when the state transition score of the monitored person exceeds the second threshold value, the recorded video is notified to the monitor .

The monitoring method according to the present invention acquires the biological information of the monitored person, acquires at least one of the posture and the position, which is the mode of the monitored person, based on the image or the video image of the monitored person, and is acquired. For a model in which the feature amount representing the characteristics of the monitored person is calculated based on the biometric information and the mode, the state of the monitored person is used as the objective variable, and the biometric information and the mode of the monitored person are used as explanatory variables. by applying the feature amount, calculates a state transition score is the score representing the normal state of the monitored person, calculated state transition scores satisfies the predetermined reference notifies the watcher information on the person being monitored , At the time of the notification, if the state transition score exceeds the first threshold value, the video of the monitored person is recorded, and then the state transition score of the monitored person exceeds the second threshold value. In some cases, the recorded video is notified to the observer .

The monitoring program according to the present invention uses a computer for a biometric information acquisition process for acquiring the biometric information of the monitored person, and a posture and position which are the modes of the monitored person based on an image or a video image of the monitored person. A mode acquisition process for acquiring at least one, a feature amount calculation process for calculating a feature amount representing the characteristics of the monitored person based on the acquired biological information and the mode, and a monitored person with the state of the monitored person as an objective variable. A score calculation process for calculating a state transition score, which is a score representing the normal state of a monitored person, and a calculated state by applying a feature amount to a model using the biometric information and the mode as explanatory variables. A notification process is executed to notify the observer of the information of the monitored person whose transition score meets a predetermined standard, and when the state transition score exceeds the first threshold value in the notification process, the image of the monitored person is displayed. When the state transition score of the monitored person exceeds the second threshold value, the recorded video is notified to the monitor .

According to the present invention, it is possible to appropriately monitor a monitored person who may behave abnormally while reducing the load on the monitor.

It is a block diagram which shows the structural example of the 1st Embodiment of the monitoring system of this invention. It is explanatory drawing which shows the example of the biological information. It is explanatory drawing which shows the example of the feature amount generated from the biological information. It is explanatory drawing which shows the example of the feature amount generated from the biological information. It is a flowchart which shows the example of the learning process. It is explanatory drawing which shows the example of the learning data. It is a flowchart which shows the example of the process of calculating the state transition score. It is explanatory drawing which shows the example of the result of calculating the state transition score. It is explanatory drawing which shows the example of the notification timing. It is a flowchart which shows the operation example of a monitoring system. It is a block diagram which shows the structural example of the 2nd Embodiment of the monitoring system of this invention. It is a flowchart which shows the example of the process of calculating the state transition score. It is a block diagram which shows the outline of the monitoring system by this invention.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.

Embodiment 1.
FIG. 1 is a block diagram showing a configuration example of a first embodiment of the monitoring system of the present invention. The monitoring system 100 of the present embodiment includes a sensor 10, a camera 12, a storage unit 14, a biological information acquisition unit 20, a mode acquisition unit 30, a calculation unit 40, and a notification unit 50.

The sensor 10 is a sensor that detects the biological information of the monitored person. The mode of the sensor 10 is arbitrary, and is realized by, for example, a microwave sensor for detecting the ecology of the monitored person, an infrared camera, a visible light camera, a non-contact type sensor such as a microphone, or the like. If a contact-type sensor that is directly attached to the monitored person is used, there is a risk of an accident (for example, accidental ingestion) due to the use of the sensor itself. Therefore, the sensor 10 is a non-contact sensor. Is preferable.

The camera 12 is an imaging device that photographs the monitored person. The camera 12 is installed at a position where the life of the monitored person can be grasped, such as a private or shared living room of the monitored person or a corridor. When a visible light camera is used as the sensor in the sensor 10, the sensor 10 and the camera 12 may be used as a common camera.

In the present embodiment, a case where the monitoring system 100 performs monitoring based on the information acquired by the sensor 10 and the camera 12 will be described. However, the monitoring system 100 may acquire the biometric information of the monitored person and the image or video of the monitored person from another system (not shown). In this case, the surveillance system 100 may not include the sensor 10 and the camera 12.

The storage unit 14 stores various information used for processing by the calculation unit 40, which will be described later. Further, the storage unit 14 may store the information acquired by the sensor 10 and the image captured by the camera 12. The storage unit 14 is realized by, for example, a magnetic disk device or the like.

The biological information acquisition unit 20 acquires the biological information of the monitored person based on the information acquired by the sensor 10. The content of the biological information acquired by the biological information acquisition unit 20 is arbitrary. For example, when the sensor 10 is a non-contact sensor, the biological information acquisition unit 20 may acquire information such as heart rate, respiratory rate, body temperature, and vocalization that can be acquired by a microwave sensor or the like as biological information.

The mode acquisition unit 30 acquires the mode of the monitored person. Specifically, the mode acquisition unit 30 acquires the posture and position of the monitored person based on the image or video captured by the monitored person. In this case, the mode acquisition unit 30 may acquire both the posture and the position of the monitored person, or may acquire only at least one of the posture and the position. In addition, the posture may include the movement of the monitored person. It should be noted that a method of acquiring the posture and position of a subject photographed based on an image or a video is generally known, and detailed description thereof will be omitted here.

The calculation unit 40 performs various processes for determining the abnormal state of the monitored person. The abnormal state means a state when the above-mentioned abnormal behavior is occurring. The calculation unit 40 includes a feature amount calculation unit 42, a model learning unit 44, and a score calculation unit 46.

The feature amount calculation unit 42 calculates a feature amount representing the characteristics of the monitored person based on the obtained biological information and the mode (specifically, posture and position). The feature amount calculated by the feature amount calculation unit 42 is biometric information or mode, or information generated by calculation from a combination thereof, and is represented by, for example, a vector containing one or more characteristic values of the monitored person. Will be done. The information representing the feature amount is arbitrary, and examples thereof include an average value of biometric information in a specific time range, a value of a specific frequency of biometric information, a relative value of biometric information at a specific time, and a differential value of biometric information. The relative value of the biological information at a specific time is, for example, a value standardized by subtracting the average value from the biological information, and the differential value represents the slope of the value of the biological information.

Further, the method in which the feature amount calculation unit 42 performs the calculation is also arbitrary. The feature amount calculation unit 42 may calculate the feature amount by using a smoothing filter, a bandpass filter, a differential filter, a combination of a plurality of filters, or the like for the biological information and the mode in a specific time range. Further, the feature amount calculation unit 42 may calculate the feature amount by an averaging process, a Fourier transform process, a differential process, or the like without using a filter.

FIG. 2 is an explanatory diagram showing an example of biological information. FIG. 2 shows an example of the result of detecting the heartbeat interval in time series and an example of the result of detecting the skin temperature in time series. In addition, FIGS. 3 and 4 are explanatory views showing an example of a feature amount generated from biological information.

The feature amount calculation unit 42 generates a feature amount (specifically, one element of the feature amount vector) based on the biological information illustrated in FIG. For example, the time course of the value obtained by smoothing and normalizing the biometric information illustrated in FIG. 2 in a specific time range (that is, subtracting the average value and converting it to a value between 0 and 1) is illustrated in FIG. It is calculated to do. Further, the change with time of the differential value based on the biological information illustrated in FIG. 2 is calculated as illustrated in FIG. The feature amount calculation unit 42 generates a feature amount vector from the feature amount generated in this way.

The model learning unit 44 learns a model for identifying whether the monitored person is in a normal state or an abnormal state. Specifically, the model learning unit 44 learns a model in which the state of the monitored person is used as an objective variable and the biological information and mode (specifically, posture and position) of the monitored person are used as explanatory variables. The model is learned, for example, for any of the monitored persons based on the biological information and mode of the specific period in which the abnormality occurred and the biological information and mode of the specific period in which the abnormality did not occur.

In addition, the model learning unit 44 may add a feature amount calculated based on biological information and a mode to the explanatory variables. The value of the objective variable representing the state of the monitored person may be calculated by a value (0/1) that distinguishes between a normal state and an abnormal state, or may be calculated with a probability indicating the degree of the abnormal state.

FIG. 5 is a flowchart showing an example of learning processing. The feature amount calculation unit 42 calculates the feature amount Y'(t) based on the biological information X'(t) and the mode (posture / position) at a certain time t in advance (step S11). Model learning unit 44 states at time t (normal or abnormal conditions) and objective variables, using the training data to explanatory variables feature amount calculated to produce a model M _i (step S12). The model learning unit 44 may store the generated model in the storage unit 14 (step S13).

FIG. 6 is an explanatory diagram showing an example of learning data. In the example shown in FIG. 6, the learning data is a feature amount vector including N feature amounts generated from the biometric information and the mode of the monitored person acquired every minute, and the state of the monitored person at that time. (Normal: 0, Abnormal: 1) is included. The feature amount vector illustrated in FIG. 6 may include the biological information and the mode itself as the feature amount, or may include the feature amount calculated from the biological information and the mode.

As illustrated in FIG. 6, the model learning unit 44 describes the biological information and mode of a specific period in which the abnormality occurred and the biological information and mode of the specific period in which the abnormality did not occur for any of the monitored persons in the past. Based on the learning data including and, the model that identifies the state of the monitored person is learned. In the example shown in FIG. 6, the period from 22:03 to 9 on October 31, 2017 corresponds to the specific period in which the abnormality occurred. By applying the newly obtained biological information as well as the posture and position to the model learned in this way, it becomes possible to identify whether the monitored person is in a normal state or an abnormal state.

The score calculation unit 46 calculates a score (hereinafter, referred to as a state transition score) representing a normal state or an abnormal state of the monitored person by using the calculated feature amount and the above model. Specifically, the score calculation unit 46 calculates the value of the objective variable representing the state of the monitored person as the state transition score by applying the calculated feature amount to the model generated by the model learning unit 44. ..

For example, parameters representing a model (e.g., the coefficient of each explanatory variable) a vector of P, the feature quantity vectors and Y, the model M _i to calculate the state transition scores I (t) is a formula 1 illustrated below expressed. In addition, α in Equation 1 is a bias. The score calculation unit 46 may calculate the state transition score by performing a matrix operation as illustrated in Equation 1.

I (t) = P · Y ^T + α (Equation 1)

FIG. 7 is a flowchart showing an example of processing for calculating the state transition score. The feature amount calculation unit 42 calculates the feature amount Y (t) based on the biological information X (t) acquired at time t and the mode (posture / position) (step S21). The score calculation unit 46 applies the calculated feature amount Y (t) to the model to calculate the state transition score (step S22). The score calculation unit 46 outputs the calculated state transition score to the notification unit 50 (step S23).

FIG. 8 is an explanatory diagram showing an example of the result of calculating the state transition score. In the example shown in FIG. 8, the feature amount of the monitored person is calculated every minute, and the calculated feature amount is applied to the model to calculate the score (state transition score).

The notification unit 50 notifies the observer of the information of the monitored person whose calculated state transition score meets a predetermined standard. For example, when the threshold value of the state transition score to be notified is set in advance, the notification unit 50 may notify the observer of the information of the monitored person whose state transition score exceeds the threshold value. Further, for example, when it is determined that the information of the monitored person having a higher state transition score is notified, the notification unit 50 notifies the monitorer of the information of the higher monitored person having a predetermined number of state transition scores. You may.

FIG. 9 is an explanatory diagram showing an example of notification timing. When the threshold value of the state transition score to be notified is set to, for example, 0.5, the notification unit 50 may notify the observer of the information of the monitored person at the timings T1 and T2 illustrated in FIG. good. Further, the notification unit 50 may continue the notification for a certain period of time after the threshold value is exceeded in consideration of the fluctuation of the state transition score.

Further, the method in which the notification unit 50 gives a notification is also arbitrary. The notification unit 50 may notify the observer of the monitored person's information by displaying the image of the monitored person determined to perform the notification on a display device (not shown) such as a monitor. In addition, the notification unit 50 may display the information of the monitored person to be notified in a manner that can be distinguished from other monitored persons. In addition, the notification unit 50 may notify the observer of the information of the monitored person who has decided to notify the observer by voice.

Further, the notification unit 50 may provide two or more types of criteria for determining whether to notify. For example, a threshold value for determining whether or not to notify the observer of the monitored person's information may be set in two stages.

Specifically, the notification unit 50 causes the camera 12 to start recording the monitored person when the state transition score exceeds the first threshold value, and then the state transition score is the second threshold. When the value is exceeded, the recorded video may be notified to the observer. On the other hand, the notification unit 50 records when the state transition score exceeds the first threshold value and then falls below the first threshold value without exceeding the second threshold value. It may be stopped so that the recorded video is not notified. By doing so, it is possible to grasp the signs of an abnormal state.

The biological information acquisition unit 20, the mode acquisition unit 30, the calculation unit 40 (more specifically, the feature amount calculation unit 42, the model learning unit 44, and the score calculation unit 46), and the notification unit 50 are programs. It is realized by a computer processor (for example, CPU (Central Processing Unit), GPU (Graphics Processing Unit), FPGA (field-programmable gate array)) that operates according to (monitoring program).

For example, the program is stored in the storage unit 14, the processor reads the program, and according to the program, the biological information acquisition unit 20, the mode acquisition unit 30, and the calculation unit 40 (more specifically, the feature amount calculation unit 42, It may operate as a model learning unit 44, a score calculation unit 46), and a notification unit 50. Further, the function of the monitoring system may be provided in the form of Software as a Service (SaaS).

The biological information acquisition unit 20, the mode acquisition unit 30, the calculation unit 40 (more specifically, the feature amount calculation unit 42, the model learning unit 44, and the score calculation unit 46), and the notification unit 50, respectively. May be realized by dedicated hardware. Further, a part or all of each component of each device may be realized by a general-purpose or dedicated circuitry, a processor, or a combination thereof. These may be composed of a single chip or may be composed of a plurality of chips connected via a bus. A part or all of each component of each device may be realized by a combination of the above-mentioned circuit or the like and a program.

Further, when a part or all of each component of the monitoring system is realized by a plurality of information processing devices and circuits, the plurality of information processing devices and circuits may be centrally arranged or distributed. May be done. For example, the information processing device, the circuit, and the like may be realized as a form in which each of the client-server system, the cloud computing system, and the like is connected via a communication network.

Next, the operation of the monitoring system of the present embodiment will be described. FIG. 10 is a flowchart showing an operation example of the monitoring system of the present embodiment. The biological information acquisition unit 20 acquires the biological information of the monitored person from the sensor 10 (step S31). Further, the mode acquisition unit 30 acquires the posture / position of the monitored person based on the image or video captured by the camera 12 (step S32). The score calculation unit 46 calculates the state transition score based on the feature amount calculated by the feature amount calculation unit 42 and the model learned by the model learning unit 44 (step S33). The notification unit 50 determines whether or not the state transition score satisfies a predetermined criterion (step S34). When the criterion is satisfied (Yes in step S34), the notification unit 50 notifies the observer of the information of the monitored person (step S35). On the other hand, if the criteria are not satisfied (No in step S34), the process ends.

As described above, in the present embodiment, the biometric information acquisition unit 20 acquires the biometric information of the monitored person, and the mode acquisition unit 30 acquires the monitored person based on the image or video captured by the monitored person. Acquire at least one of the posture and position, which is a mode of. Further, the feature amount calculation unit 42 calculates the feature amount representing the characteristics of the monitored person based on the acquired biological information and the mode, and the score calculation unit 46 applies the feature amount to the model to be monitored. Calculate the state transition score of the person. Then, the notification unit 50 notifies the observer of the information of the monitored person whose calculated state transition score satisfies the predetermined standard. Therefore, it is possible to appropriately monitor the monitored person who may behave abnormally while reducing the load on the monitor.

For example, if you try to check all the camera images, the load on the observer will increase. Therefore, in the present embodiment, the notification unit 50 can monitor only the target person who meets the specific condition by using the state transition score, so that the load on the monitorer can be reduced. Further, if there are too many types of information (for example, evaluation parameters) to be set, the adjustment load becomes heavy. In the present embodiment, since it is sufficient to learn past cases by machine learning and set a standard for notifying the observer, it is possible to detect an abnormal state without increasing the load without requiring adjustment for each person. become.

In this embodiment, the case where the model learning unit 44 learns the model based on the past data has been described. Further, the model learning unit 44 has newly acquired learning data about the monitored person (more specifically, the biological information and mode of the monitored person during a specific period in which the abnormality has occurred, and the abnormality has occurred. The model may be updated based on biometric information and modalities for a specific period of time that has not been done. According to such a configuration, the accuracy of calculating the state transition score can be improved.

Embodiment 2.
Next, a second embodiment of the monitoring system of the present invention will be described. FIG. 11 is a block diagram showing a configuration example of a second embodiment of the monitoring system of the present invention. The monitoring system 200 of the present embodiment includes a sensor 10, a camera 12, a storage unit 14, a biological information acquisition unit 20, a mode acquisition unit 30, a calculation unit 40a, and a notification unit 50.

The configuration of the sensor 10, the camera 12, the storage unit 14, the biological information acquisition unit 20, the mode acquisition unit 30, and the notification unit 50 is the same as in the first embodiment.

The calculation unit 40a includes a feature amount calculation unit 42, a model learning unit 44a, and a score calculation unit 46a. The configuration of the feature amount calculation unit 42 is the same as that of the first embodiment.

The model learning unit 44a learns a model for identifying whether the monitored person is in a normal state or an abnormal state. In the present embodiment, the model learning unit 44a learns a model in which the state of the monitored person is used as an objective variable and the biological information and mode of the monitored person and additional information are used as explanatory variables. That is, the model learning unit 44a of the present embodiment is different from the first embodiment in that it learns a model for identifying the state of the monitored person in consideration of additional information.

The additional information includes environmental information representing the surrounding environment of the monitored person and target person information representing the attributes of the monitored person. Environmental information includes the time zone (day or night) at the time of monitoring, temperature, humidity, weather, atmospheric pressure, and the like. In addition, the target person information includes the age, gender, food and drink intake, home environment, health condition, etc. of the monitored person. Of the additional information, the environmental information may be acquired from an environmental sensor (not shown) or from another system (not shown). Of the additional information, the target person information may be acquired from another system (not shown).

The score calculation unit 46a calculates the state transition score using the calculated feature amount and additional information, and the above model.

FIG. 12 is a flowchart showing an example of processing for calculating the state transition score. The process of calculating the feature amount Y (t) is the same as in step S21 in FIG. The score calculation unit 46a applies the calculated feature amount Y (t) and additional information to the model to calculate the state transition score (step S41). Then, the score calculation unit 46a outputs the calculated state transition score to the notification unit 50 (step S23).

The biological information acquisition unit 20, the mode acquisition unit 30, the calculation unit 40a (more specifically, the feature amount calculation unit 42, the model learning unit 44a, and the score calculation unit 46a), and the notification unit 50 are programs. It is realized by the processor of the computer that operates according to (monitoring program).

As described above, in the present embodiment, the model learning unit 44a, in addition to the learning data used by the model learning unit 44 of the first embodiment, provides additional information such as environmental information representing the surrounding environment of the monitored person or the monitored person. A model for calculating a state transition score is learned using learning data including subject information representing the attributes of. Therefore, in addition to the effect of the first embodiment, it is possible to monitor the monitored person who may have abnormal behavior with higher accuracy.

In addition, the model learning unit 44a of the present embodiment also newly acquired learning data for the monitored person (more specifically, biological information and mode of the specified period in which the abnormality occurred, which was newly acquired for the monitored person, and The model may be updated based on the additional information and the biological information and mode and additional information during a specific period in which no abnormality has occurred. According to such a configuration, the accuracy of calculating the state transition score can be improved.

Next, the outline of the present invention will be described. FIG. 13 is a block diagram showing an outline of the monitoring system according to the present invention. The monitoring system 80 (for example, the monitoring system 100) according to the present invention includes a biological information acquisition unit 81 (for example, a biological information acquisition unit 20) that acquires biological information of a monitored person, and an image or video image of the monitored person. Based on the mode acquisition unit 82 (for example, the mode acquisition unit 30) that acquires at least one of the posture and the position, which is the mode of the monitored person, and the characteristics of the monitored person based on the acquired biological information and the mode. For a feature amount calculation unit 83 (for example, a feature amount calculation unit 42) that calculates a feature amount representing the above, and a model in which the state of the monitored person is used as an objective variable and the biometric information and mode of the monitored person are used as explanatory variables. The score calculation unit 84 (for example, the score calculation unit 46) that calculates the state transition score, which is a score representing the normal state of the monitored person, and the calculated state transition score are predetermined by applying the feature amount. It is provided with a notification unit 85 (for example, a notification unit 50) that notifies the observer of information on the monitored person that meets the criteria.

With such a configuration, it is possible to appropriately monitor the monitored person who may behave abnormally while reducing the load on the monitor.

Further, the monitoring system 80 is a model for calculating a state transition score used for identifying the state of the monitored person by using learning data including the biological information and mode of the monitored person and information indicating the state of the monitored person. A learning unit (for example, a model learning unit 44) for learning the above may be provided. Then, the score calculation unit 84 may calculate the state transition score by applying the feature amount to the model.

Further, the learning unit learns a model for calculating the state transition score using learning data including environmental information representing the surrounding environment of the monitored person or target person information representing the attributes of the monitored person as additional information, and scores. The calculation unit may calculate the state transition score by applying the feature amount to the model in which the state of the monitored person is used as the objective variable and the biological information and mode of the monitored person and the additional information are used as explanatory variables. ..

Specifically, the learning unit provides learning data including biological information and mode of a specific period in which an abnormality has occurred and biological information and mode of a specific period in which an abnormality has not occurred for any of the past monitored persons. The model may be trained using.

In addition, the learning unit may update the model using the newly acquired learning data for the monitored person.

Further, the biometric information acquisition unit 81 may acquire the biometric information of the monitored person from the information detected by using the non-contact sensor.

Further, the notification unit 85 may notify the observer of the information of the monitored person whose state transition score exceeds a predetermined threshold value.

In addition, the notification unit 85 may notify the observer of the information of the monitored person whose state transition score is higher than the predetermined number.

Further, the notification unit 85 records the video of the monitored person when the state transition score exceeds the first threshold value, and then the state transition score of the monitored person exceeds the second threshold value. In that case, the recorded video may be notified to the observer.

Some or all of the above embodiments may also be described, but not limited to:

(Appendix 1) A mode of acquiring at least one of the posture and the position, which are the modes of the monitored person, based on the biometric information acquisition unit that acquires the biometric information of the monitored person and the image or video of the monitored person. The acquisition unit, the feature amount calculation unit that calculates the feature amount representing the characteristics of the monitored person based on the acquired biological information and the mode, and the biological information of the monitored person and the biological information of the monitored person and the state of the monitored person as objective variables. By applying the feature amount to a model using a mode as an explanatory variable, a score calculation unit that calculates a state transition score, which is a score representing the normal state of the monitored person, and a calculated state transition score are obtained in advance. A monitoring system characterized in that it is equipped with a notification unit that notifies the monitor of the information of the monitored person who meets the set criteria.

(Appendix 2) A model for calculating the state transition score used for identifying the state of the monitored person is learned by using the learning data including the biological information and mode of the monitored person and the information indicating the state of the monitored person. The monitoring system according to Appendix 1, which includes a learning unit, and the score calculation unit calculates a state transition score by applying a feature amount to the model.

(Appendix 3) The learning unit learns a model for calculating a state transition score using learning data including environmental information representing the surrounding environment of the monitored person or target person information representing the attributes of the monitored person as additional information. Then, the score calculation unit calculates the state transition score by applying the feature amount to the model in which the state of the monitored person is used as the objective variable and the biological information and mode of the monitored person and the additional information are used as explanatory variables. The monitoring system described in Appendix 2.

(Appendix 4) The learning unit includes learning data including biological information and mode of a specific period in which an abnormality has occurred and biological information and mode of a specific period in which an abnormality has not occurred for any of the past monitored persons. The monitoring system according to any one of Appendix 2 and Appendix 3 for learning a model using the above.

(Appendix 5) The monitoring system according to any one of Appendix 2 to Appendix 4, wherein the learning unit updates a model using newly acquired learning data for the monitored person.

(Supplementary note 6) The monitoring system according to any one of Supplementary note 1 to Supplementary note 5, wherein the biometric information acquisition unit acquires biometric information of a monitored person from information detected by using a non-contact sensor.

(Appendix 7) The monitoring system according to any one of Appendix 1 to Appendix 6 in which the notification unit notifies the observer of information on a monitored person whose state transition score exceeds a predetermined threshold value. ..

(Appendix 8) The monitoring system according to any one of Appendix 1 to Appendix 6 in which the notification unit notifies the observer of information on a monitored person having a state transition score higher than a predetermined number.

(Appendix 9) The notification unit records a video of the monitored person when the state transition score exceeds the first threshold value, and then the state transition score of the monitored person reaches the second threshold value. The monitoring system according to any one of Appendix 1 to Appendix 8 for notifying the observer of the recorded video when the above value is exceeded.

(Appendix 10) The biometric information of the monitored person is acquired, and at least one of the posture and the position, which is the mode of the monitored person, is acquired based on the image or video of the monitored person, and the acquired biometric information is obtained. Based on the above and the mode, the feature amount representing the characteristics of the monitored person is calculated, the state of the monitored person is used as the objective variable, and the biological information and the mode of the monitored person are used as explanatory variables. By applying, the state transition score, which is a score representing the normal state of the monitored person, is calculated, and the monitored person is notified of the information of the monitored person whose calculated state transition score meets a predetermined standard. A characteristic monitoring method.

(Appendix 11) Using learning data including biometric information and mode of the monitored person and information indicating the state of the monitored person, a model for calculating a state transition score used for identifying the state of the monitored person is learned. , The monitoring method according to Appendix 10 for calculating a state transition score by applying a feature amount to the model.

(Appendix 12) A computer acquires at least one of the posture and position, which are the modes of the monitored person, based on the biometric information acquisition process for acquiring the biometric information of the monitored person and the image or video of the monitored person. Mode acquisition process, feature amount calculation process to calculate the feature amount representing the characteristics of the monitored person based on the acquired biological information and mode, the biometric information of the monitored person and the biometric information of the monitored person with the state of the monitored person as the objective variable. By applying the feature amount to a model using a mode as an explanatory variable, a score calculation process for calculating a state transition score, which is a score representing the normal state of the monitored person, and a calculated state transition score are obtained. A monitoring program for executing a notification process that notifies the monitor of the information of the monitored person who meets the predetermined criteria.

(Appendix 13) A model for calculating a state transition score used for identifying the state of a monitored person by using learning data including biometric information and a mode of the monitored person and information indicating the state of the monitored person on a computer. The monitoring program according to Appendix 12, wherein a learning process for learning the above is executed, and a state transition score is calculated by applying a feature amount to the model in the score calculation process.

The present invention is suitably applied to, for example, a monitoring system for monitoring a subject who may cause abnormal behavior in a prison, a detention center, a detention center, a psychiatric ward, or the like.

10 Sensor 12 Camera 14 Storage unit 20 Biological information acquisition unit 30 Mode acquisition unit 40, 40a Calculation unit 42 Feature amount calculation unit 44, 44a Model learning unit 46, 46a Score calculation unit 50 Notification unit 100, 200 Monitoring system

Claims (10)

The biometric information acquisition unit that acquires the biometric information of the monitored person,
A mode acquisition unit that acquires at least one of the posture and position, which is the mode of the monitored person, based on the image or video of the monitored person.
A feature amount calculation unit that calculates a feature amount representing the characteristics of the monitored person based on the acquired biological information and mode, and a feature amount calculation unit.
By applying the feature amount to a model whose objective variable is the state of the monitored person and the biological information and mode of the monitored person are used as explanatory variables, the state transition is a score representing the normal state of the monitored person. The score calculation unit that calculates the score and
It is equipped with a notification unit that notifies the observer of the information of the monitored person whose calculated state transition score meets a predetermined standard .
The notification unit records a video of the monitored person when the state transition score exceeds the first threshold value, and then the state transition score of the monitored person exceeds the second threshold value. A surveillance system characterized in that the recorded video is notified to the observer in the event of a failure. It is equipped with a learning unit that learns a model that calculates a state transition score used to identify the state of the monitored person using learning data that includes the biological information and mode of the monitored person and information indicating the state of the monitored person. ,
The monitoring system according to claim 1, wherein the score calculation unit calculates a state transition score by applying a feature amount to the model. The learning unit learns a model for calculating a state transition score using learning data including environmental information representing the surrounding environment of the monitored person or target person information representing the attributes of the monitored person as additional information.
The score calculation unit calculates a state transition score by applying a feature amount to a model in which the state of the monitored person is used as an objective variable and the biological information and mode of the monitored person and additional information are used as explanatory variables. Item 2. The monitoring system according to item 2. The learning unit uses learning data including biological information and mode of a specific period in which an abnormality has occurred and biological information and mode of a specific period in which an abnormality has not occurred for any of the past monitored persons. The monitoring system according to any one of claim 2 and 3. The monitoring system according to any one of claims 2 to 4, wherein the learning unit updates a model using newly acquired learning data for the monitored person. The monitoring system according to any one of claims 1 to 5, wherein the biometric information acquisition unit acquires biometric information of a monitored person from information detected by using a non-contact sensor. The monitoring system according to any one of claims 1 to 6, wherein the notification unit notifies the monitor of information on a monitored person whose state transition score exceeds a predetermined threshold value. The monitoring system according to any one of claims 1 to 6, wherein the notification unit notifies the monitor of information on a monitored person having a state transition score higher than a predetermined number. Acquires the biometric information of the person being monitored,
Based on the image or video image of the monitored person, at least one of the posture and position, which is the mode of the monitored person, is acquired.
Based on the acquired biometric information and mode, the feature amount representing the characteristics of the monitored person is calculated.
By applying the feature amount to a model whose objective variable is the state of the monitored person and the biological information and mode of the monitored person are used as explanatory variables, the state transition is a score representing the normal state of the monitored person. Calculate the score,
Notify the observer of the information of the monitored person whose calculated state transition score meets a predetermined standard, and notify the observer .
At the time of the notification, when the state transition score exceeds the first threshold value, the video of the monitored person is recorded, and then the state transition score of the monitored person exceeds the second threshold value. A monitoring method characterized in that the recorded video is notified to the observer in the event of a failure. On the computer
Biometric information acquisition processing to acquire biometric information of the monitored person,
A mode acquisition process for acquiring at least one of the posture and position of the monitored person based on the image or video captured by the monitored person.
Feature amount calculation processing that calculates the feature amount representing the feature of the monitored person based on the acquired biological information and mode,
By applying the feature amount to a model whose objective variable is the state of the monitored person and the biological information and mode of the monitored person are used as explanatory variables, the state transition is a score representing the normal state of the monitored person. Score calculation process to calculate the score, and
A notification process is executed to notify the observer of the information of the monitored person whose calculated state transition score meets a predetermined standard.
In the notification process, when the state transition score exceeds the first threshold value, the video of the monitored person is recorded, and then the state transition score of the monitored person exceeds the second threshold value. A monitoring program for notifying the observer of the recorded video in the event of a failure.

Images