JP6650305B2 - Behavior analysis system and program - Google Patents

Description

  The present invention relates to a technology for recognizing human behavior in a sensor network environment and performing behavior analysis such as detection of abnormal behavior and identification of behavior.

Conventionally, technology that recognizes human behavior by installing various and numerous sensors in homes and offices and collecting and analyzing such information via a network (hereinafter referred to as “behavior recognition technology”) has been used. .
For example, in Patent Literature 1, a large number of motion sensors for detecting the movement of the human body are installed in a solitary elderly person's house, and the behavior of the resident is recognized based on the analysis of the sensor information. Discloses a technology for notifying a remote family member or caregiver.

  There are a wide variety of sensors used in such technologies, such as technology for monitoring human behavior based on moving images, technology for estimating behavior by wearing an acceleration sensor on a human, and bonding an RFID to an object and holding the object. A behavior recognition technology based on a variety of sensors, such as a technology for recognizing the fact and a technology for estimating a flow line using a floor pressure sensor, is known.

However, it is difficult to install a camera or microphone especially when analyzing daily life behavior in an individual's house, especially considering the risk of privacy leakage of the user.
Also, wearing of the wearable sensor is troublesome for the user, and often cannot be used on a daily basis.

Further, attaching a sensor to each object or installing a floor pressure sensor afterwards increases installation cost.
Due to these restrictions, sensors that provide a lot of information on behavior are often not available, and only sensors such as pyroelectric sensors and door switch sensors that have low installation / maintenance costs and low user burden, but lack information that can be obtained, are used. Often not.
When a sensor having such a small amount of information is used, it is difficult to construct a general-purpose behavior model due to differences in living environment and behavior style that vary greatly for each individual.

Therefore, the model of the behavior specialized for the individual is learned from the data acquired for each individual, and the data section corresponding to the behavior is cut out and the label indicating the behavior corresponding to the section is included. It is often unacceptable in terms of cost to prepare personal data with a teacher signal on behavior such as assignment before introducing the system.
In addition, since the living environment and the behavior change over time, it is more unrealistic to prepare data with a teacher signal each time.

  In view of the realistic demands for reducing the burden of introducing the system and the flexibility of re-introduction, it is desirable to use an unsupervised learning technique that builds a behavior model from personal data without using a teacher signal.

  For example, in Patent Literature 2, by monitoring a moving speed, a flow line is divided into sections from a movement start to a movement stop to automatically extract action blocks, and a hidden Markov is obtained for each obtained action block. A technique of learning a generation model of observation data from data using the model is disclosed.

  Also, in the prior invention by the inventor (Patent Document 4), a technique using a character string kernel capable of efficiently performing a comprehensive analysis of a partial character string by regarding a sensor firing sequence as a character string is disclosed. I have.

In the present invention, by analyzing a subsequence in a sensor firing sequence, a section having a similar frequency pattern of the subsequence is automatically segmented as one action, and a feature amount calculated from the segmented section is used. To detect abnormalities and identify actions.
However, although the segment itself can be obtained by unsupervised learning, a teacher signal is required when performing action identification.

  In addition, as a method of modeling an action from a sensor data sequence without using a teacher signal, a technique using a document topic modeling method called Latent Dirichlet Allocation (LDA) (Non-Patent Document 2) as in Non-Patent Document 1 It has been known.

  Topic modeling assumes a probabilistic model in which words are generated depending on the topics of documents that are not observed from the outside, and estimates potential topics from only word strings included in a given document set. Also, for a new document, a topic can be estimated from the word string.

  By associating the word sequence with the sensor sequence and the topic sequence with the action sequence, the behavior can be modeled using the topic modeling method, and unsupervised learning of the action model and estimation of the action sequence can be realized without using heuristics.

  However, it is not clear whether a topic model for a document is optimal for behavioral modeling.

  In particular, in a topic model of a document such as LDA, it is desirable to assume topic multiplicity, that is, assuming that one document can have a plurality of topics. Assumptions may not be appropriate, in which case an unnecessarily complex model risks reducing learning and inference accuracy.

  In modeling behavior, it is desirable to assume Markov nature. That is, whether or not a certain sensor fires depends on the firing of the previous sensor, especially the immediately preceding sensor, and whether or not a certain action occurs depends on the previous, particularly, the immediately preceding action.

In order to introduce the Markov property of sensor firing, an extended LDA model assuming the Markov property of words as used in Non-Patent Document 3 can be used.
On the other hand, the introduction of Markov property of behavior assumes that LDA is a multi-topic model and that words in the document are generated from multiple topics, so the introduction of Markov property of document-level topics is not simple. .

In Non-Patent Document 4, a more abstract topic hierarchy corresponding to a story development in a document is provided at a higher hierarchy of a topic, and a Markov property is introduced to this hierarchy.
Although such modeling is realized using the Hierarchical Poison-Dirichlet Process (HPDP), there is a problem that the number of parameters increases and learning and inference become complicated.

JP 2003-256957 A JP 2007-249922 A JP 2011-175349 A JP 2014-087333 A

T. Huynh et al .: Discovery of activity patterns using topic models, Proc. Of Internat. Conf.on Ubiquitous Computing, pp. 10-19 (2008). D.M.Blei et al .: Latent Dirichlet Allocation, Journal of Machine Learning Research, No. 3, pp. 993-1022 (2003). H.M.Wallach: Topic Modeling: Beyond Bag-of-words, Proceedings of the internat. Conf.on Machine Learning, pp.977-984 (2006). D. Lan et al .: Sequential Latent Dirichlet Allocation, Knowledge and Information Systems, Vol. 31, No. 3, pp. 475-503 (2012). Sadamitsu Sept. et al .: Topic-based language model using mixed Dirichlet distribution, IEICE Transactions D-II, J88-D-II (9), pp.1771-1779 (2005). Etsuo Chosa: Locomotive Syndrome: A New Concept Surrounding Motor Organ Disease, Jpn.J.Rehabil.Med., Vol.50, No.1, pp.48-54 (2013). J. Verghese et al .: Abnormality of Gait as a Predictor of Non-Alzheimer's Dementia, New England Journal of Medicine, Vol.347, No.22, pp.1761-1768 (2002).

  Although the topic model of a document, LDA and its extended model, model the multitopic nature of the document, it is not realistic to assume that a sensor sequence is generated from multiple actions when modeling actions. However, there is a problem that the model and the learning / inference algorithm are unnecessarily complicated, which may degrade the accuracy of the behavior prediction.

In fact, as described later, when modeling behavior, it is better to assume that the sensor series is output from a single topic without assuming multiple topics, and the prediction accuracy of the behavior will be better. Has been confirmed.
Another challenge is to introduce the Markov property of sensors and behaviors without complicating the model as much as possible.

In the present invention, it is assumed that a word in a document is generated from a single topic by a Dirichlet distribution as a prior distribution, and a mixed Dirichlet model that models a document as a topic mixture of such a generated distribution (see Non-Patent Document 5). ) Is adopted as a behavior modeling method.
However, we extended the mixed Dirichlet model to introduce the Markov property of the sensor and the behavior, and constructed a Gibbs sampler to learn and infer the extended model efficiently and accurately. Make inferences.

According to one embodiment of the present invention, work such as assigning an activity label to sensor data is unnecessary, and the burden of introducing a system that does not require work other than installing a sensor is reduced, and low flexibility that can be re-introduced. Costly abnormality monitoring is possible.
In addition, instead of directly detecting an abnormality from a sensor series, it is possible to detect an abnormality based on whether or not an estimated behavior is frequently observed on a daily basis after abstracting to an activity level once. .

  Furthermore, it is possible to realize the abnormality monitoring that learns the daily malfunction of the sensor as one action and does not regard it as an abnormality, thereby reducing false alarms based on false sensor firing due to heat from pets and heating appliances. It also has an effect.

In another form, by presenting the estimated behavior sequence and the feature amount calculated for each behavior to the analyst in an easy-to-understand manner, it becomes possible to analyze the daily behavior pattern and problem behavior of the resident.
At this time, since the action sequence is estimated only from the firing sequence of the sensor without giving a teacher signal for segmenting the action or identifying the action, the cost required for the analysis can be significantly reduced.
Furthermore, in another form, the health of the resident is monitored on a daily basis based on the activity, the quality of sleep, and the walking speed, which are calculated based on the behavior and not from the sensor activity itself. In addition, a low-cost and reliable remote health monitoring system can be realized which automatically notifies a health report to a remote family or facility staff when a regular or specific event occurs.

1 is an overall view of an abnormality monitoring system. It is a figure explaining an action model learning part in more detail. It is a figure explaining an action model estimation part in more detail. FIG. 7 is a diagram illustrating raw sensor data (example) received by a receiving unit. FIG. 8 is a diagram illustrating a frame sequence (example) extracted from sensor raw data. It is a figure showing the structure of an action model. FIG. 4 is a diagram illustrating a counter initialization unit (InitializeCounter). FIG. 4 is a diagram for explaining a frame sequence initialization unit (Initialize). FIG. 4 is a diagram illustrating a counter updating unit (Estimate). It is a figure explaining the experimental result which compared action prediction accuracy. 1 is an overall view of a behavior analysis system. It is a figure explaining the example of everyday action visualization in an action analysis system. It is an overall view of a health management system.

  Next, an embodiment of the present invention will be described with reference to the drawings.

(Abnormality monitoring system)
(Overall configuration example)

An example of the overall configuration of the abnormality monitoring system will be described with reference to FIG.
A plurality of sensors having a low degree of privacy invasion, such as a human body sensor (a pyroelectric sensor, etc.) and a switch sensor (a magnetic proximity door opening / closing sensor, etc.), are installed in the user's home.
As these sensors, inexpensive sensors that operate on batteries for at least one year and can transmit information to the parent device by wire or wirelessly can be used.

Ideally, the pyroelectric sensor should be installed on the ceiling with a narrow detection range so that a rough flow line of the user can be obtained.
If the sensor cannot be installed on the ceiling, the sensor is installed in a grid on the wall surface, and it is possible to detect that the operation has been performed in an area where the detection ranges of X and Y intersect by firing the sensors X and Y at the same time.

  The information from these sensors is received by the receiving unit (11), and the firing sequence (sensor data sequence) of the sensors is sent to the behavior model learning unit or the behavior model estimation unit.

  In the learning mode, the sensor firing sequence is sent to the behavior model learning unit (12), the behavior model of the user is learned, and the learned model is stored in the behavior DB (14).

  In the estimation mode, the sensor firing sequence is sent to the behavior model estimation unit (13), and the sequence of the behavior is extracted with reference to the behavior model stored in the behavior DB (14).

  Subsequently, the action sequence is sent to the abnormality detection unit (15), and the probability that the action can occur from the model is calculated for each extracted action while referring to the action model.

If this probability is lower than a certain threshold value, it is determined that the behavior is abnormal, and the abnormality warning unit (16) issues a warning.
This alarm is transmitted through a network outside the home, such as the Internet, and transmitted to a monitoring center (17) or to a family member or a care worker who watches over a remote place.

The behavior model learning unit used in the learning mode will be described with reference to FIG.
First, a frame sequence is extracted from the firing sequence of the sensor used for learning (P21).

As shown in FIG. 4, the firing sequence of the sensor typically includes the sensor name (the third column) and the date and time (the first and second columns) at which the state of the sensor (the fourth column) changes intermittently. It was recorded.
Like the information that the state of the sensor has changed, the information that the sensor information has not changed is also an important information source for understanding the behavior. To information.

  Specifically, a time window (hereinafter, referred to as a frame) having a predetermined width is moved, and the IDs of the sensors that have fired in the window are arranged in one line in the order of firing time.

FIG. 5 shows a sequence of frames obtained by shifting a window having a width of 30 seconds by 10 seconds with respect to the raw data of FIG.
For example, the ID of the door sensor D001 is 63, and it can be seen in FIG. 5 that the fire occurred during the first three frames (the door was opened).

The appearance of the same ID multiple times in one frame indicates that the same sensor has fired multiple times during the frame.
Furthermore, if no sensor fires during a certain frame, a special ID, 0, is recorded as in the last frame of FIG.

At P22, P23 and P24, an action model is learned based on the extracted frame sequence.
In the present invention, the behavior is modeled using a hierarchical probabilistic model using a mixed Dirichlet distribution, but Markov property is assumed for the behavior and the sensor.

FIG. 6 shows the structure of a model when first-order Markov property is assumed.
In the figure, shaded circles represent observable random variables, and unshaded circles represent latent random variables.
Of course, the Markov property does not need to be limited to the first order, and in some cases, the same discussion can be made even if one or both of them are assumed to be the zero order.

Hereinafter, description will be made assuming first-order Markov property.
In FIG. 6, the random variable z representing the action of each frame is generated from a categorical distribution using θ ^t as a parameter, depending on the immediately preceding action t.

Further, the random variable w representing the sensor firing of each frame is generated from a categorical distribution using φ ^{t, u} as a parameter, depending on the action t assigned to the frame and the preceding sensor firing u. Further, as a prior distribution of the behavior, the sensor, and the categorical distribution of each, a Dirichlet distribution using α and β as hyperparameters is assumed as a prior distribution.

At this time, the simultaneous distribution of the action sequence Z and the firing sequence W can be written as follows.

Here, Cat represents a categorical distribution,

I can write However, T is represents the type of action, a ^{_{Σ t θ q t = 1,}} n q, t represents the frequency of action t appears immediately after the action q. On the other hand, for W

I can write However, V is represents the number of types of sensors, a ^{_{Σ w φ w t, u =}} 1, n t u, w represents the frequency of the sensor w fires immediately after the sensor u fires Behavioral t.

Dir represents the Dirichlet distribution,

However, when there is no prior information as in the first learning, it can be assumed that the hyperparameter α is uniform. That is,

, The Dirichlet distribution can be written as

In order to avoid the complexity of the description, the following description will be made on the assumption that the hyper parameters of the Dirichlet distribution are uniform.
When assuming uniformity for β, that is,

in the case of,

I can write
When such modeling is performed, from Expressions 2, 4, and 8,

Get. Here, n _{q, ·} = Σ _t n _{q, t} .
On the other hand, from Equations 3, 5, and 10,

Get. Here is a ^{_{n t u, · = Σ w}} n t u, w.
The study of behavioral model, it is necessary to estimate these frequency n _{q, t} and n ^t _{u, w.}

The procedure is described below. First, a series of counters is initialized (P22).
In FIG. 7, the counter used when assuming that both the action and the sensor have the primary Markov property, such that the action depends on the immediately preceding action and the firing of the sensor in a certain action depends on the firing of the immediately preceding sensor. The initialization method is shown.

Next, using the frame sequence extracted in P21 as input, initialization is performed to randomly assign an action to each frame (P23). FIG. 8 shows an initialization method when assuming first-order Markov property.
At this time, the frequency of the sensor that fires in each frame is counted and stored in the main memory, and this is represented as s _{u, w} (m).

Subsequently, the counter for calculating the parameters of the behavior model is updated N times for each frame (P24).
FIG. 9 shows a method of calculating a counter for calculating a parameter of an action model assuming first-order Markov property by a Markov chain Monte Carlo method using a Gibbs sampler.

If a statistic obtained by removing the contribution c (m) of the m-th frame from the statistic c is expressed as c (−m), then when the statistic of the frame other than the m-th frame is given, m The conditional probability of action in the th frame is proportional to the right side of the following equation:

Assuming that the action of the (m-1) -th frame is q and the action of the (m + 1) -th frame is r, the first term on the right side can be written as follows from Expression 11.

Also, the second term on the right side can be written as follows from Expression 12.

According to Equation 13, the behavior of the m-th frame is randomly Gibbs-sampled, and the frequency counter is updated based on the result.
Finally, the counter is stored in the action DB (P25).
Assuming first-order Markov property, the parameters of the behavior model can be calculated from the counter as follows.

Next, an action sequence estimating unit will be described with reference to FIG.
First, a frame sequence is extracted from the sensor firing sequence to be estimated (P31) by the same method as P21 of the behavior model learning unit (FIG. 2).

  Next, after reading the counter from the action DB (P32), an initial value of the action is randomly assigned to the frame sequence as in the action model learning unit (P23) (P33).

Further, similarly to the behavior model learning unit (P24), the behavior of each frame is randomly sampled and the counter is updated (P34).
Finally, the action sequence Z for the obtained frame sequence is stored in the action DB (P35).

  Therefore, the basic difference between the behavior model learning unit (FIG. 2) and the behavior model estimation unit (FIG. 3) is the difference between whether to initialize the counter to zero and whether to overwrite the counter.

If it is desired to retrain the model using additional data after learning the behavior model once, the counter may be updated without initializing it to zero, and the updated counter may be overwritten.
That is, instead of initializing the counters in FIG. 2 and P22, similarly to FIG. 3 and P32, after calling the already stored counters from the action DB, the counters are updated (P23 and P24) and the new counters are updated. The counter may be stored in the action DB (P25).

Lastly, a supplementary description of the abnormality detection unit (15) in FIG.
In order to determine whether or not a certain frame is an ordinary one under the behavior model, the likelihood of the frame may be used.
For example, given the estimated action sequence Z and the observed sensor firing sequence W, the likelihood of the m-th frame can be calculated using Expressions 18, 19 and 20.

Further, more simply, Expression 20 can be used as the likelihood.
If the likelihood is lower than a predetermined threshold or a threshold determined from learning data, for example, the minimum likelihood obtained as a result of learning, it can be determined that the behavior is extraordinary, that is, abnormal. it can.
The likelihood calculated here is stored in the action DB as one of the attributes of each frame, and is used for later analysis and the like.

FIG. 10 is a diagram comparing the result of estimating the behavior based on the behavior modeling method used in the present invention using the published data with other modeling methods.
The vertical axis of FIG. 10 represents the percentage of time when the predicted action sequence was matched with the action sequence given by the human, and the horizontal axis was the type of the behavior assumed at the time of modeling. The number T.

  In FIG. 10, the modeling method used in the present invention includes a modeling method using Latent Dirichlet Allocation described in Non-Patent Document 2 (LDA) and a modeling method using mixed Dirichlet distribution described in Non-Patent Document 5 ( DM).

  As is clear from FIG. 10, according to the present invention, it is possible to realize a great improvement in the accuracy of behavior prediction, and to improve the accuracy of abnormality detection based on whether or not the behavior is a daily activity by relying on this accuracy improvement. Becomes possible.

(Other embodiments)
(Behavior analysis system)
An example of the overall configuration of the behavior analysis system will be described with reference to FIG.
The configuration of the sensor, the reception unit (111), the behavior model learning unit (112), the behavior model estimation unit (113), and the abnormality detection unit (115) are the same as those in the abnormality monitoring system shown in FIG.

  In addition to these units, the behavior analysis system refers to the behavior model stored in the behavior DB (114) to analyze the behavior, and performs a search unit (118), a frequency measurement unit (117), and a speed measurement unit ( 116), and an action visualization unit (119) for displaying the analysis result of each unit.

Hereinafter, each unit will be described.
The speed measuring unit (116) calculates the firing speed of the sensor in each action. Now, assuming that the sensors that fired in the time section b of a certain action are s ₁ ,..., S _L and each firing time is t (s _i ) (1 ≦ i ≦ L),

In addition, if the distance d (s _i , s _j ) between the sensors is known,

Can be used.
However, [P] assumes a value of 1 when P is true and a value of 0 when P is false.

These measured values are stored in the behavior DB (114) for each behavior.
The frequency measurement unit (117) extracts a specific action under a specific condition from the action DB (114) using the search unit (118), and notifies the user of the frequency via the action visualization unit (119). Make it presentable.

For example, it is possible to measure the frequency when there is an action corresponding to sleep before and after the action corresponding to the toilet.
The designation of the condition or the action may be in accordance with an instruction from the user via the action visualization unit (119), or may be embedded with a predetermined condition.

  The search unit (118) refers to the behavior DB (114) in response to an instruction from a user via an interface provided in the behavior visualization unit or a request from the frequency measurement unit (117), and specifies a specific condition. Search for actions.

  The time / time, the ignition information of the sensor, the estimated behavior, the degree of abnormality measured by the abnormality detection unit (115), and the sensor measured by the speed measurement unit (116) can also be used as search keys. The firing speed and the action frequency measured by the frequency measuring unit (117) can be used.

  The behavior visualization unit (119) presents the information held in the behavior DB and the values measured by each unit to the user in an easy-to-understand manner, and is used for grasping the resident's behavior pattern and analyzing problem behavior.

For easy-to-understand information presentation, it may be better to perform a smoothing process on the action series output by the action model estimation unit (113).
For example, when the same action b (b _i = b, 1 ≦ i ≦ L) is continuous in the action sequence b ₁ ,..., B _L , these are merged and presented as one action b. Is one of the simplest smoothing.

Further, it is possible to consider smoothing such that the extracted actions are clustered and similar actions are merged as one action.
As the clustering method, a known method such as a k-means method can be used, but in any case, it is necessary to determine the distance or similarity between actions.

In the modeling of the behavior in the present invention, the transition probability distribution of the behavior represented by Formula 16 and the firing probability distribution of the sensor represented by Formula 17 are obtained. Therefore, two behaviors given by using these distributions are obtained. To define the distance. For example, the distance between actions a and b can be defined as follows using Jensen-Shannon Divergence.

Here, r _k = (p _k + q _k ) / 2. Of course, there can be various variations, such as not using θ for defining the distance.
As a result of performing the smoothing, when a plurality of continuous frames are merged as one action, the newly obtained time interval is input to the action estimation unit 113 again as one frame, and the action for the new frame sequence is The accuracy can be further improved by performing the estimation and the calculation of the likelihood.

FIG. 12 shows an example of visualization of data for one day, which is displayed on a display or the like connected to the behavior visualization unit (119).
At the top, a vertical line is displayed in different colors for each sensor according to the firing time of the sensor.
Below that, the action sequence estimated from the firing sequence of the sensor is displayed as a time section surrounded by a broken line.
Further, the likelihood calculated by the abnormality detection unit (115) is displayed as “normality” for each action.

  Further, in the group of actions, action labels “sleep” and “going out” given by the user using an input device such as a keyboard and a mouse connected to the action visualization unit (119) are displayed.

In the figure, the action selected by the pointer corresponds to a series of actions from getting out of the bed during sleep to going to the tray and returning to the bed.
By pointing to this action, a plurality of toilet actions performed on that day are highlighted and displayed.

By pressing the search button in this state, the selected action can be searched and displayed from the action DB using the search unit (118).
In the figure, as a result of such a search, it is displayed that the number of times that the user went to the toilet between 9:00 pm and 6:00 am in November was 20 times, and that the total time was about 100 minutes.

In addition, for the selected action, the firing rate of the sensor calculated by the speed measurement unit (116) using Expression 22 is displayed.
This value reflects the speed of travel from returning to the toilet to returning.

(Health management system)
The health management system will be described with reference to FIG.
Sensor configuration, receiving unit (131), behavior model learning unit (132), behavior model estimation unit (133), abnormality detection unit (135), speed measurement unit (136), frequency measurement unit (137), search unit ( 138) is the same as the behavior analysis system shown in FIG.

  The health report generation unit (139) automatically generates a health report using an index related to daily activity obtained as a result of monitoring of daily activity according to a template prepared in advance, at a predetermined interval or at a predetermined event. When an error occurs, the information is transmitted to a family member, facility staff, or the like via an Internet network or the like.

  As an index to be used, the degree of abnormality of the behavior calculated by the abnormality detection unit (135) can be used.

In addition, for example, the walking speed in the living room can also be an important index.
This is because a decrease in walking speed is an index for early detection of locomotive syndrome (Non-Patent Document 6) and dementia (Non-Patent Document 7).

  In order to regard the firing speed of the sensor calculated by Expression 22 as the walking speed, it is necessary to identify the movement behavior in the living room.

For this purpose, there is a method of incorporating a discriminator for identifying a walking behavior in the behavior sequence estimated in the present invention into a health management system. However, as a simpler method, a sensor firing rate calculated by Expression 22 for each behavior is described. The maximum value of the average value can be regarded as the walking speed. This is because there is a high probability that the maximum value corresponds to the indoor movement behavior. For example, a set of interval time of action t when the D _t, can be regarded as the walking speed computed by v as follows.

In addition, the quality of sleep is also a useful index. For the calculation, measured values of the time of sleep action, the frequency of sleep disturbed in a toilet or the like, and the frequency of turning over can be used. As an index, for example, sleep behavior sequence s ₁ which allows interruption of the predetermined time, ..., with respect to s _L, can be considered a sleep efficiency is calculated as follows.

Here, t _e (s) is the start time of the action s, and t _b (s) is the end time.
For this calculation, it is necessary that behaviors such as sleep and toilet have been identified, but behaviors strongly associated with these places can be identified with a simple rule-based classifier. For example, if an action that includes a sensor activity around the bed and continues for one hour or more is determined to be a sleep action, the sleep action can be identified by the same entire action that appears at other times.

In addition, the frequency of turning over can be counted by the number of sensor activities around the bed during the sleep action.
At that time, for example, the sensor around the bed activates only when approaching the periphery of the bed during the day. It can be a more reliable indicator because it can be distinguished from those that are active during the day and those that are active during the day.

Further, the daily activity can be a useful index.
According to the present invention, since a higher-order action sequence than the firing activity of the sensor can be acquired, the frequency, time, and type of the action observed in one day can be used as the activity amount. For example, if the accumulated time of T types of actions b appearing in one day is represented by t (b), the following average entropy can be used as an index of the activity amount.

However, the _{L = Σ b t (b)} .
In addition, the sum of the firing frequencies of the sensors calculated by Expression 21 can be used as an index of the activity amount. In this case, since the sum of the sensor firing frequencies in the behavior excluding going out, sleeping, and restroom can be calculated, a more reliable activity amount can be calculated.

Outing behavior identification is characterized by no sensor activity, but must be distinguished from periods of no sensor activity, such as during sleep.
According to the behavior estimation method of the present invention, since the context such as the sensor around the entrance before and after going out and the activity of the door switch sensor at the entrance is taken into consideration, the leaving behavior is distinguished from the activity still in the room. Extracted.

  The present invention is a system for identifying and analyzing the behavior of not only a human but also a moving object such as an animal in a closed space or an open space where a sensor can be installed and ignited. Or it can be widely used in similar environments.

Reference Signs List 1 action 2 sensor 3 frame 4 number of frames 5 number of sensor firing of m-th frame 6 number of types of action 7 number of types of sensor 8 times of day 9 firing rate 10 action labeling 21 search range 22 frequency / month 23 Total time / month 11,111,131 Receiving unit 12,112,132 Behavior model learning unit 13,113,133 Behavior model estimating unit 14,114,134 Behavior DB
15, 115, 135 abnormality detection unit 16 abnormality alarm unit 17 monitoring center 116, 136 speed measurement unit 117, 137 frequency measurement unit 118, 138 search unit 119 action visualization unit 139 health report generation unit

Claims (11)

An abnormality monitoring system for human behavior in a predetermined area of a closed space or an open space including at least a sensor, a reception unit, an behavior model learning unit, an behavior model estimation unit, an activity DB, and an abnormality detection unit,
Assume that both the behavior and the sensor have Markov property,
A plurality of sensors including at least a human body sensor and a switch sensor are arranged in the predetermined area,
Each sensor information transmitted from each sensor in the receiving unit is converted in time series into a sensor data string including at least the name of the sensor, the state of the sensor, and the date and time when the state changed, and the behavior model learning unit and the behavior model estimation Sent to the department,
In the behavior model learning unit, an initialization behavior model composed of a behavior sequence obtained by learning the behavior of the person based on the one sensor data sequence is obtained by initializing the sensors included in the one sensor data sequence. Both the firing frequency (hereinafter referred to as frequency counter) are stored in the action DB,
The action series obtained by estimating the action of the person based on the other sensor data string in the action model estimating unit is the frequency counter updated with the firing frequency of the sensor included in the other sensor data string. Is stored in the action DB, and the estimated action sequence is extracted and sent to the abnormality detection unit.
The probability that the action can occur is calculated based on the frequency counter with respect to the extracted action series in the abnormality detection unit,
An abnormality monitoring system, wherein an abnormality in an action related to the other sensor data sequence is determined based on the probability.   The behavior model, wherein the behavior is generated according to a categorical distribution having a Dirichlet distribution as a prior distribution, and the sensor is generated according to a categorical distribution having a Dirichlet distribution as a prior distribution depending on the behavior, a hierarchical probability The abnormality monitoring system according to claim 1, wherein the abnormality monitoring system is a model. The sensor data sequence is a sensor data sequence (called a frame sequence) in a time window of a predetermined width (called a frame) (order M),
A frame sequence (frame sequence for learning) is extracted from the one sensor data sequence for learning the behavior model,
A frame sequence (frame sequence for estimation) is extracted from the other sensor data sequence for behavior model estimation,
The random variable z representing the action of each frame is generated according to a categorical distribution depending on the action before that,
The random variable w representing the sensor firing of each frame is generated according to the categorical distribution depending on the action z assigned to the frame and the sensor firing before that,
The abnormality monitoring system according to claim 2, wherein each prior distribution of the categorical distribution is the Dirichlet distribution. The frequency counter is n _q in the action model learning unit and the behavior model _{estimator, t, n q, ·,} n t u, ·, n t u, a _v, the learning frame sequence or for the estimated The abnormality monitoring system according to claim 3, wherein the abnormality monitoring system is initialized or updated from a frame sequence by a Markov chain Monte Carlo method using Gibbs sampling.
However, n _{q, t,} the frequency of action t appears immediately after the action column q in each learning frame sequence, n ^t _{u, v,} the sensor array u in action t in each learning frame sequence has fired a frequency of the sensor w fires _{immediately, n q, · = Σ t} n q, is _t, and ^{_{n t u, · = Σ v}} n t u, v. The extraction of the behavior series in the behavior model estimating unit reads the frequency counters estimated and stored in the DB from the DB as initial values,
The frequency counter is updated by the Markov chain Monte Carlo method based on the estimation frame sequence,
The abnormality monitoring system according to claim 4, wherein an action sequence Z corresponding to each of the estimated frame sequences for estimation is stored in the action DB.   6. The abnormality monitoring system according to claim 5, wherein the abnormality detection unit determines an abnormality in the behavior of each frame of the estimation frame sequence based on the likelihood of the frame. 7. The abnormality monitoring system according to claim 6, further comprising a search unit, a frequency measurement unit, a speed measurement unit, and a behavior visualization unit that displays a search result of the search unit,
In the speed measurement unit, the firing speed of the sensor in the action related to the estimation frame is calculated and stored in the action DB for each action,
The frequency measurement unit extracts a specific behavior from the behavior DB, stores the frequency in the behavior DB,
The search of the action in the action DB using the sensor firing information in the search unit, the likelihood determined to be abnormal, the estimated action, the sensor firing rate or the specific action and its frequency as an index. And enable
When the one index is given, the search unit searches the behavior DB for the behavior for a predetermined period (day, week, month, year, etc.) and displays the behavior in the behavior visualization unit. Behavior analysis system.   8. A health management system according to claim 7, further comprising a health report generation unit in place of the behavior visualization unit, and providing a health report describing an activity amount based on the frequency, time, and type of the specific behavior. . It is a health management system according to claim 8,
Considering the calculated firing speed of the sensor as the speed of the specific action related to the movement of the area according to the firing sensor,
A health management system characterized by providing a health report in which the maximum value of the average firing speed of the action during the predetermined period is described as the walking speed of the person. It is a health management system according to claim 8,
Further comprising a discriminator capable of discriminating the behavior that lasts a predetermined time or more including the firing of the specific sensor as a sleep behavior,
A health management system for providing a health report stating sleep quality related to the sleep behavior sequence based on an interruption time of a sleep behavior sequence including the sleep behavior identified by the classifier during the predetermined period. .   A storage medium storing a program for processing the sensor data and related data of the health management system, the behavior analysis system, and the abnormality monitoring system according to any one of claims 1 to 10.

Images