EP3163545A1

EP3163545A1 - Abnormal activity detection for elderly and handicapped individuals

Info

Publication number: EP3163545A1
Application number: EP15306725.1A
Authority: EP
Inventors: Eric Gautier; Serge Defrance; Françoise Le Bolzer
Original assignee: Thomson Licensing SAS
Current assignee: Thomson Licensing SAS
Priority date: 2015-10-29
Filing date: 2015-10-29
Publication date: 2017-05-03

Abstract

A method and apparatus for creating an anomalous behavior detection log for reporting anomalous behavior of a user in an environment are describe including logging first time-stamped event data of initial behavior of the user over a period of time in the environment, estimating a location of the user in the environment using the time-stamped event data of the initial behavior of the user, creating a characterization of life routines of the user's initial behavior, logging second time-stamped event data of subsequent behavior of the user, estimating a user location sequence of the user in the environment using time-stamped event data of the user's subsequent behavior, generating user location sequence comparisons by comparing the characterization of life routines of the user's initial behavior and the estimated user location sequence of the user's subsequent behavior and creating an anomalous behavior detection log by detecting anomalous behavior using the user location sequence comparisons.

Description

FIELD

The proposed method and apparatus relates to the detection of abnormal activities of elderly and handicapped people residing in their homes or non-medical residences.

BACKGROUND

This section is intended to introduce the reader to various aspects of art, which may be related to the present embodiments that are described below. This discussion is believed to be helpful in providing the reader with background information to facilitate a better understanding of the various aspects of the present disclosure. Accordingly, it should be understood that these statements are to be read in this light.
In the domain of the care of elderly and handicapped people staying in their own homes or in non-medical residences (such as senior apartment complexes or assisted living facilities), one key area of interest is the life habits of the individual. Care workers (givers) monitoring the elderly or handicapped individuals may effectively use a report on the life patterns of the person as a tool for diagnosis of behavioral or health problems. Anomalies may be detected for further investigation and alerts may be raised for immediate action in the case of an emergency.
To monitor the functional health of a smart home resident and detect anomalous behavior, some solutions rely on automatic recognition of daily living activities like standing, sitting, sleeping, etc... This approach requires a data collection infrastructure in the home implementing generally a large number of sensors (bed and chair occupancy sensors, etc...), which can be complex and expensive to implement. In addition sensors may sometimes be considered to be intrusive by the individual (camera, wearable sensors, etc...), consequently not widely accepted by elderly or handicapped individuals. Finally these solutions are frequently based on supervised machine learning techniques and require labeling a large amount of data to learn a model for activities. This is not always feasible.
Other nonintrusive anomaly detection approaches rely on sensors located in the living environment as motion detectors or door contactors, enabling the individual not to be disturbed with the technology. These solutions, which are generally accepted by elderly and handicapped people, are frequently based on unsupervised machine learning solutions and use sensory data directly to find outliers. An outlier in the context of the present area of technology is sensor data and its corresponding sequence which, if excluded from the group or cluster, improves the accuracy and aggregation of the reminder of the group or cluster. However, such a model of a user's behavior does not have any information or knowledge as to whether the individual is sitting, standing, walking, etc.
One commonly used similarity measure is the called LCS (Longest Common Subsequence). The LCS similarity measure is commonly used to measure similarity between DNA sequences. This definition can be extended to define symbolic temporal sequences, describing the successions of sensors that are triggered. Typically, having n sensors, one symbol may be used to describe the position of the n sensors. The symbol could be a n-digit number, each digit being represented by a 0 or 1 which is the current position of one sensor.
A day (or any period of time) corresponding to a monitoring period can then be expressed (represented, described) by a symbol sequence, expressing the succession of all sensor positions. In other words, motion and door sensor activations can be processed as symbolic temporal sequences. To illustrate, Fig. 1 is an example of the "User Motion Sequence" obtained from time-stamped events representing motion sensor activations or de-activations. Each symbol is a state representing when the sensors are activated, for example: one activated sensor (i.e. motion in bedroom = mB), several activated sensor (i.e. motion in bedroom and living = mBmL) or no activated sensors (no motion is detected = nM).
However, applying similarity-based anomaly detection on symbolic temporal sequences built directly from motion and door sensor activations leads to a poor result. The reason is that temporal sequences provide sparse information which is not fully exploitable with sequence based anomaly detection, such as similarity based techniques. The time a person moves or activates a door is generally very short with respect to the sequence duration which corresponds to the monitoring period and which can last several hours. Most of the time, there is either little motion or the doors leading outside the home are closed, which leads to building sequence including essentially the state "no Motion" (as shown in Fig. 1). So, the usual similarity measures between sequences such as LCS are not discriminant enough to be efficient to measure distances between such sequences.

SUMMARY

The proposed method and apparatus concerns the implementation of an abnormal activity detection technique for elderly and handicapped people. While principally directed to detecting anomalous behavior of elderly or handicapped individuals, the proposed method and apparatus are not so limited and may be employed to monitor prisoners, children or other individuals that find themselves in restricted environments. The description of the proposed method and apparatus uses elderly or handicapped individuals as examples. In order to be accepted by elderly and handicapped individuals and not disturb them, the data collection system must be as non-intrusive as possible. Data is collected only from a minimal number of motion and doors activation sensors, which have been judiciously placed in the residence of the elderly or handicapped individual. With such limited collected data, usual anomaly detection techniques such as similarity-based techniques of temporal sequences do not perform correctly because sequences indicating sensor activations present very sparse information. The proposed method and apparatus extends the features extracted from data collected from motion and door activation sensors by producing features indicating the location of the person in her home or outside her home, in order to be applicable efficiently in the case of anomaly detection using similarity-based techniques on the temporal sequences.
Because it is preferable to rely on non-intrusive sensors, the proposed method and apparatus use the non-intrusive approach to detect outliers even if the model does not have any information or knowledge as to whether the individual is sitting, standing, walking, etc. The proposed method and apparatus categorizes a user's life routines from sensory data provided during several days by using a clustering technique and testing whether a new day of sensory measurements is an outlier or not in line with the categorized and clustered routines. The user is the elderly or handicapped individual who is the subject of the anomaly detection. Clustering is used because it is an unsupervised machine learning method enabling grouping of similar objects into respective categories according a similarity (i.e. distance) measure definition. The principle is to cluster the points of the data set and to determine the distance between a new data point and the medoids, which represent the clusters. Distances above a certain threshold are outliers.
A method and apparatus for creating an anomalous behavior detection log for reporting anomalous behavior of a user in an environment are described including logging first time-stamped event data of initial behavior of the user over a period of time in the environment, estimating a location of the user in the environment using the time-stamped event data of the initial behavior of the user, creating a characterization of life routines of the user's initial behavior, logging second time-stamped event data of subsequent behavior of the user, estimating a user location sequence using the second time-stamped event data of the user's subsequent behavior, comparing the characterization of life routines of the user's initial behavior and the estimated user location sequence of the user's subsequent behavior and creating an anomalous behavior detection log by detecting anomalous behavior using the user location sequence comparisons.

BRIEF DESCRIPTION OF THE DRAWINGS

The proposed method and apparatus is best understood from the following detailed description when read in conjunction with the accompanying drawings. The drawings include the following figures briefly described below:

Fig. 1 is an example of a user motion sequence obtained from time-stamped events representing motion sensor activations and de-activations.
Fig. 2 is a high-level flowchart of the proposed method.
Fig. 3 shows an exemplary case, where one individual is living in a residence equipped with a non-intrusive data collection infrastructure and where each main room is covered by one omni-directional motion detector installed on the ceiling.
Fig. 4 illustrates an example of user location sequence provided by time stamped events sequences of living and bedroom motion sensors.
Fig. 5 is an exemplary plot of 15 sequences (y_axis) corresponding to 15 days of sensory measures and representing the location of that person in her home between 0 to 24h (x axis).
Fig. 6 is a graph of exemplary clusters, their medoids, the cluster greatest distance and the LCS distance to an outlier sequence presented to the anomaly detection portion of the proposed method and apparatus.
Fig. 7 is a flowchart of an exemplary implementation of step 205 of Fig. 2.
Fig. 8 is a flowchart of an exemplary implementation of step 210 of Fig. 2.
Fig. 9 is a flowchart of an exemplary implementation of step 215 of Fig. 2.
Fig. 10 is a block diagram of an exemplary server that performs the proposed method.

It should be understood that the drawing(s) are for purposes of illustrating the concepts of the disclosure and is not necessarily the only possible configuration for illustrating the disclosure.

DETAILED DESCRIPTION

The present description illustrates the principles of the present disclosure. It will thus be appreciated that those skilled in the art will be able to devise various arrangements that, although not explicitly described or shown herein, embody the principles of the disclosure and are included within its scope.
All examples and conditional language recited herein are intended for educational purposes to aid the reader in understanding the principles of the disclosure and the concepts contributed by the inventor to furthering the art, and are to be construed as being without limitation to such specifically recited examples and conditions.
Moreover, all statements herein reciting principles, aspects, and embodiments of the disclosure, as well as specific examples thereof, are intended to encompass both structural and functional equivalents thereof. Additionally, it is intended that such equivalents include both currently known equivalents as well as equivalents developed in the future, i.e., any elements developed that perform the same function, regardless of structure.
Thus, for example, it will be appreciated by those skilled in the art that the block diagrams presented herein represent conceptual views of illustrative circuitry embodying the principles of the disclosure. Similarly, it will be appreciated that any flow charts, flow diagrams, state transition diagrams, pseudocode, and the like represent various processes which may be substantially represented in computer readable media and so executed by a computer or processor, whether or not such computer or processor is explicitly shown.
The functions of the various elements shown in the figures may be provided through the use of dedicated hardware as well as hardware capable of executing software in association with appropriate software. When provided by a processor, the functions may be provided by a single dedicated processor, by a single shared processor, or by a plurality of individual processors, some of which may be shared. Moreover, explicit use of the term "processor" or "controller" should not be construed to refer exclusively to hardware capable of executing software, and may implicitly include, without limitation, digital signal processor (DSP) hardware, read only memory (ROM) for storing software, random access memory (RAM), and nonvolatile storage.
Other hardware, conventional and/or custom, may also be included. Similarly, any switches shown in the figures are conceptual only. Their function may be carried out through the operation of program logic, through dedicated logic, through the interaction of program control and dedicated logic, or even manually, the particular technique being selectable by the implementer as more specifically understood from the context.
In the claims hereof, any element expressed as a means for performing a specified function is intended to encompass any way of performing that function including, for example, a) a combination of circuit elements that performs that function or b) software in any form, including, therefore, firmware, microcode or the like, combined with appropriate circuitry for executing that software to perform the function. The disclosure as defined by such claims resides in the fact that the functionalities provided by the various recited means are combined and brought together in the manner which the claims call for. It is thus regarded that any means that can provide those functionalities are equivalent to those shown herein.
A proposed method and apparatus for anomaly detection for elderly and handicapped people can be roughly broken down into three main steps as shown in Fig. 2 and explained in detail below. Prior to 205 the motion sensors and door contacts are used to collect and time stamp event data regarding the location of the elderly or handicapped individual in their residence. This collected time-stamped event data of a user's initial behavior over a period of time in the environment is logged (recorded) by the server. The environment for an elderly or handicapped user is the user's home or non-medical residence. The environment for an individual such as a prisoner or child will be different. At 205 the collected time-stamped event data of the user's initial behavior is forwarded to a server where the collected time-stamped event data is used to estimate the location of the user in their environment. At 210 the server then uses the estimation of location of the user to characterize the user's life routines. New time-stamped event data representing a user's subsequent behavior is collected and forwarded to the server where the new time-stamped event data is logged. The server uses the new time-stamped event data to estimate a user location sequence of the user in their environment. Upon receipt of newly collected time-stamped event data, which is forwarded to the server, at 215 the newly collected time-stamped event data is compared against the user's life routines to detect anomalous behavior. That is, comparisons of the sequences of the user's initial life routines against the estimated user's location sequence are generated. An anomalous behavior detection log is created based on the comparisons of the sequences of the user's initial life routines and the estimated user location sequence. The anomalous behavior detection log is used to alert a care giver of the anomalous behavior. The care giver may be a nurse, a doctor, an aide, a family member etc. In a prison or other environment, a care giver may be a person who monitors the activities of the individual being monitored. The alert may be by phone, by email, by text message or any other reasonable means.
Fig. 3 shows an exemplary case, where one individual is living in a residence equipped with a non-intrusive data collection infrastructure and where each main room is covered by one omni-directional motion detector installed on the ceiling. These main rooms are labeled the "bedroom", the "living", the "kitchen", the "bedroom2", the "office" and the corresponding motion sensors are respectively indicated as "mB", "mL", "mK", "mB2", "mO". Furthermore one external "door" is equipped with a door contact indicated as "cD".

Depending on the way data is collected and the way information is organized, in numerous cases data format can be reconstructed from data originally collected and provided by the collection infrastructure as a sequence of ordered time-stamped events. A representation of a time-stamped event (TSE) includes listing the events corresponding to the sensor activations together with the date and time at which the events occurred. The following is a short sample of such a data format for types of sensor including door activator and motion sensors:

Date	Time	Sensor	Event
2014-11-06	16:59:10.966903	cD	OPEN
2014-11-06	16:59:11.528494	mL	ON
2014-11-06	16:59:12.773843	mL	OFF
2014-11-06	16:59:13.768215	mL	ON
2014-11-06	16:59:20.492125	mL	OFF
2014-11-06	16:59:22.162185	cD	CLOSE
2014-11-06	16:59:22.934276	mK	ON
2014-11-06	16:59:24.467157	mL	ON
2014-11-06	16:59:24.536898	mK	OFF
2014-11-06	16:59:26.95258	mL	OFF
2014-11-06	16:59:28.374536	mB	ON
2014-11-06	16:59:30.897865	mB	OFF

To expand the features that could be exploited by unsupervised machine learning techniques such as similarity-based techniques applied to temporal sequences, the symbolic temporal sequences built directly from motion and door sensors (as illustrated previously) are not used. As already mentioned, the symbolic temporal sequences deliver sparse information and lead to poor results. Instead, the time-stamped event representation is used to build a one day collection of temporal sequences of symbolic states representing an estimation of the location of the individual in each room of his home or outside his home during the data collection period. The duration of these sequences can be adapted to the problem to be resolved. In case of detection of an anomalous nighttime behavior for instance one day collection sequences limited to nighttime hours (from 10pm to 9am) only can be considered.
Consider that the location of an individual in a room depends on the sensor activation and where the individual was located initially. The sensor deactivation is not taken into account. For example, suppose that the individual is located in the bedroom, so that as soon as the motion sensor located in the living room is activated, it can be estimated that the individual has relocated to the living room. The same presence estimation can be applied to the other rooms. In case the individual is located in a room and the door is open, it is assumed that the individual has gone outside. Inversely when the individual is outside and the door is activated, it is assumed that the individual has moved inside. A motion sensor should be activated in a room to locate him in the corresponding room. When several motion sensors are activated concurrently, which can happen according to the size of the room, account is taken only of the last activated sensor and then a determination is made regarding the individual's corresponding location. Fig. 4 illustrates an example of user location sequence provided by time stamped events sequences of living and bedroom motion sensors.
The proposed method and apparatus can make some location estimation errors particularly regarding the result of door activation. Indeed an individual located inside the home can open and close the door without going outside. To limit the consequence of such a condition, the motion sensors are given a priority over the door activation sensors. In that case, as soon as a motion sensor is activated in a room, the individual is deemed to be located in the corresponding room even if he was initially located outside. So this estimation error is limited to a short period of time, corresponding to the duration the individual is not detected as moving within a specific room after entering it in his house. Inversely, when an individual is located outside and activates the door without entering in the home, he will be nevertheless located inside. To limit the impact of this kind of error, when the individual is inside without activation of a motion sensor in a specific room beyond a duration threshold, the individual is deemed to be outside.
It should be noted that within localization estimation method and apparatus, motion sensors could be replaced by door activation sensors in rooms where people usually open and close the door when they use it. This is generally the case for the front door, back door or restroom door for instance.

An example of the results of the proposed location estimation method and apparatus applied to the area of elderly and handicapped care is shown in Table 1 below. The leftmost column of Table 1 indicates the initial location of the person and the first row the motion or door sensor activations. The resulting location is then given in each table cell.

Table 1

	*mB_Off*	*mB_On*	*mL_Off*	*mL_On*	*mK_Off*	*mK_On*	*mB2_Off*	*mB2_On*	*mO_Off*	*mO_On*	*cD_Close*	*cD_Open*
*Inside*	Inside	Bedroom	Inside	Living	Inside	Kitchen	Inside	Bedroom2	Inside	Office	Inside	Outside
*Bedroom*	Bedroom	Bedroom	Bedroom	Living	Bedroom	Kitchen	Bedroom	Bedroom2	Bedroom	Office	Bedroom	Outside
*Living*	Living	Bedroom	Living	Living	Living	Kitchen	Living	Bedroom2	Living	Office	Living	Outside
*Kitchen*	Kitchen	Bedroom	Kitchen	Living	Kitchen	Kitchen	Kitchen	Bedroom2	Kitchen	Office	Kitchen	Outside
*Bedroom2*	Bedroom2	Bedroom	Bedroom2	Living	Bedroom2	Kitchen	Bedroom2	Bedroom2	Bedroom2	Office	Bedroom2	Outside
*Office*	Office	Bedroom	Office	Living	Office	Kitchen	Office	Bedroom2	Office	Office	Office	Outside
*Outside*	Outside	Bedroom	Outside	Living	Outside	Kitchen	Outside	Bedroom2	Outside	Office	Outside	Inside

Such temporal symbolic sequences enable exploiting other kind of features based on the location of the individual in the home, like time spent in a specific room or outside the home, number of times the individual stays in a specific room, hour he enters the room and duration he stays in it, etc...
The method described above has been applied to a motion and door activation sensors data set placed in the home of an elderly person. Fig. 5 is an exemplary plot of 15 sequences (y_axis) corresponding to 15 days of sensory measures and representing the location of that person in her home between 0 to 24h (x axis). Each color represents the location of the individual in a room as indicated in the legend.
Once symbolic states sequences representing an estimation of the location of the individual in each room of his home or outside his home have been built, the individual's "life routines" are characterized as a learning process by applying an unsupervised clustering process on a set of learning sequences using the length of the Longest Common Subsequence (LCS) as a similarity measure.
The LCS is a popular sequence similarity measure used in biology to measure similarity between DNA sequences. It is defined as follows: given two sequences X and Z, Z is a subsequence of X if removing some symbols from X will produce Z and Z is a common subsequence of two sequences X and Y if Z is a subsequence of X and Y. As an example assume t1 = XMJYAUZ and t2 = MZJAWXU, LCS = MJAU, length of LCS = 4. The length of the LCS is a very effective measure because it measures similarity between two sequences without restricting itself to a location-based one to one match.
Once clusters have been created from the user location sequences of the learning data, clusters that are considered as significant of the individual's life routines are identified and selected. Indeed a cluster with few sequences with respect to the number of sequences of the learning set, cannot be considered as a user life routine. So a threshold is applied to retain only the clusters having at least a significant number of sequences. The threshold can be defined as a function of the number of sequences of the learning set and the number of found clusters as indicated hereafter: $SignificantSequenceThreshold = f (NbOfSequenceOfTrainingSet, NbOfClusters)$
After having identified and selected the significant clusters of the set of learning sequences, one representative sequence of each significant cluster is identified and selected. A cluster medoid (or simply medoid) is a sequence whose average dissimilarity to all the sequences in the cluster is minimal. A medoid is chosen as representative of each significant cluster.
After having determined the individual's life routines from the set of learning sequences, anomaly detection is performed on a new sequence of time stamped events corresponding to the activation/deactivation of motion or door sensors. First of all the new sequence of time stamped events to be processed is transformed into a user location sequence as described above. Then the anomaly detection process is achieved by determining whether this sequence is considered as a user's life routines or not. So for each significant cluster, the LCS distance between the current (new) processed sequence and the representative of the cluster (such as the cluster medoid) is calculated. This distance is compared with the greatest LCS distance between the medoid and the sequences of the cluster. If the former is greater than the latter the current (new) sequence is considered as anomalous (see Fig. 6). If the former is less than the latter, then the current (new) sequence is not considered as anomaly.
Fig. 7 is a flowchart of an exemplary implementation of step 205 of Fig. 2 (estimation of the user's location). At 705 the time-stamped event data of the sensors is recorded for a pre-determined period of time. The period of time may be two weeks or more. The time-stamped event data is transmitted to a server for recordation (storage). The motion sensors and door contacts are not necessarily smart devices and they do not have sufficient processing power to perform the proposed method, so the time-stamped event data is transmitted to a server for recordation (storage) and processing. The time-stamped event data collected for the pre-determined period of time is to be used as a training set (training data, learning data). At 710 the server creates user location sequences from the time-stamped event data which are used to creating a characterization of life routines of the initial behavior of the user.
Fig. 8 is a flowchart of an exemplary implementation of step 210 of Fig. 2 (characterization of user's life routines). At 801 the server creates a collection of the user location sequences. The successive individual user location sequences created at 710 across a period of time (2 weeks for example), are collected together and will be used to characterize the user's life routine at 805 (learning/training). At 805 the server applies an unsupervised clustering process to the user location sequences to identify clusters of the user location sequences. The clustering process uses a similarity measure such as LCS for example. Some other similarity measure such as the "optimal matching distance" can also be considered. At 810 the server identifies significant clusters from among the identified clusters. This is accomplished by comparing a number of user location sequences in each of the identified clusters to a threshold. At 815 the server selects a medoid to represent each significant cluster. A medoid is the user location sequence in a cluster (significant cluster) whose dissimilarity to all other user location sequences in the cluster is minimal. At 820 the server calculates a distance between the medoid representing the cluster and each user location sequence in the cluster. This is done for each significant cluster. At 825 the server determines a greatest distance from among the calculated distances between the medoid of a cluster and each user location sequence of the cluster. This is performed for each significant cluster.
Fig. 9 is a flowchart of an exemplary implementation of step 215 of Fig. 2 (detect anomalous behavior). At 915 the server calculates a distance between the medoids of each significant cluster and the new user location sequence. At 920 the server compares the distance between (the medoids of each significant cluster and the new user location sequence) and (the greatest distances between the medoids of each cluster and each user location sequence of each cluster). This is performed for each significant cluster. At 925 a test is performed to determine if the new user location sequence is within the user's (normal) life routine. If the new user location sequence is not within the user's (normal) life routine then the new user location sequence represents anomalous behavior and an anomalous behavior detection log is created. If the distance between the medoids of each significant cluster and the new user location sequence is greater than the greatest distances between the medoids of each cluster and each user location sequence of each cluster, then there is anomalous behavior (the new user location sequence is not within the user's (normal) life routine).
Fig. 10 is a block diagram of an exemplary server that performs the proposed method. There are three main modules that correspond to the functions shown in Fig. 2 - the estimate location module, the characterize user life routine module and the detect anomalies module. Each of these modules has additional functional modules within it. The motion sensors and door contacts provide input to a record data module of the estimate location module. This is time-stamped event data and is recorded for a period of time. The period of time may be two weeks or more. The time-stamped event data is transmitted to a server for recordation (storage) by the motion sensors and door contacts. The motion sensors and door contacts are not necessarily smart devices and they do not have sufficient processing power to perform the proposed method so the time-stamped event data is transmitted to a server for recordation (storage) and processing. The time-stamped event data collected for the pre-determined period of time is to be used as a training set (training data, learning data). The create user location sequence module of the estimate location module creates user location sequences from the time-stamped event data. The estimate location module logs first time-stamped event data of initial behavior of the user over a period of time (for example, two weeks) in the user's environment. The characterize user life routine module creates a characterization of life routines of the user's initial behavior. The estimate location module logs second time-stamped event data of subsequent behavior of the user. The detect anomalies module generates user location sequence comparisons by comparing the characterization of life routines of the user's initial behavior and the estimated user location sequence. The detect anomalies module then creates an anomalous behavior detection log by detecting anomalous behavior using the user location sequence comparisons.
The create collection of user location sequence module collects individual user location sequences. The collection of user location sequences is used to characterize the user's life routine in the learning/training step (805). The apply clustering process module of the characterize user life routine module receives input from the create collection of the user location sequence module, which creates a set of user location sequences for the period of time (2 weeks for example). The apply clustering process module applies an unsupervised clustering process to the collection of the user location sequences to identify clusters of the user location sequences. The clustering process uses a similarity measure such as LCS. The identify significant clusters module then identifies significant clusters from among the identified clusters. This is accomplished by comparing a number of user location sequences in each of the identified clusters to a threshold. The select medoid module then selects a medoid to represent each significant cluster. A medoid is the user location sequence whose dissimilarity to all other user location sequences in the cluster is minimal. The calculate distance module then calculates a distance between the medoid representing the cluster and each user location sequence in the cluster. This is done for each significant cluster. The calculate greatest distance module then determines a greatest distance from among the calculated distances between the medoid of a cluster and each user location sequence of the cluster. This is performed for each significant cluster.
New time-stamped event data from motion sensors and door contacts is accepted through the record data module of the estimate location module of the server. The new time-stamped event data is live data as opposed to training/learning data and is recorded. The new time-stamped event data is also forwarded to the create user location sequence module of the estimate location module of the server, which then creates a new user location sequence. The create new user location sequence module then creates a new user location sequence. The calculate distance module then calculates a distance between the medoids of each significant cluster and the new user location sequence. The compare distance module accepts the greatest distances between the medoids of each cluster and each user location sequence of each cluster and then compares the distance between (the medoids of each significant cluster and the new user location sequence) and (the greatest distances between the medoids of each cluster and each user location sequence of each cluster). This is performed for each significant cluster. The detect and report anomalies module then performs a test to determine if the new user location sequence is within the user's (normal) life routine. If the new user location sequence is not within the user's (normal) life routine then the new user location sequence represents anomalous behavior. If the distance between the medoids of each significant cluster and the new user location sequence is greater than the greatest distances between the medoids of each cluster and each user location sequence of each cluster, then there is anomalous behavior (the new user location sequence is not within the user's (normal) life routine).
Other components within the server include but are not limited to storage, one or more communications interfaces, internal communications paths which may be bus structures. Storage may include any form of RAM (DRAM, SRAM, etc.), ROM, (e.g., EPROM, EEPROM, etc.), hard disks, optical disks, flash memory, thumb drives, etc. The communications interfaces may include wired line and/or wireless interfaces, for example to receive (accept) time-stamped event data and to provide alerts or other notifications to care givers about any anomalous behavior of the individual. It should be obvious that the server described above does not only handle (process) the data for a single individual but may process the data for a plurality of elderly or handicapped individuals.
In a context as described above with a nonintrusive data collection infrastructure based only on motion and door activation sensors, detectability depends on the kind of anomalies that are to be detected. Indeed each time an anomaly to be detected is related to the fact that an individual is located in a room of his home or outside his home, it can be deemed that an estimation of the location of an individual has been carried out.
It is to be understood that the proposed method and apparatus may be implemented in various forms of hardware, software, firmware, special purpose processors (e.g., math processors, computers or processors dedicated to machine learning), or a combination thereof. Special purpose processors may include application specific integrated circuits (ASICs), reduced instruction set computers (RISCs) and/or field programmable gate arrays (FPGAs). Preferably, the proposed method and apparatus is implemented as a combination of hardware and software. Moreover, the software is preferably implemented as an application program tangibly embodied on a program storage device. The application program may be uploaded to, and executed by, a machine comprising any suitable architecture. Preferably, the machine is implemented on a computer platform having hardware such as one or more central processing units (CPU), a random access memory (RAM), and input/output (I/O) interface(s). The computer platform also includes an operating system and microinstruction code. The various processes and functions described herein may either be part of the microinstruction code or part of the application program (or a combination thereof), which is executed via the operating system. In addition, various other peripheral devices may be connected to the computer platform such as an additional data storage device and a printing device.
It should be understood that the elements shown in the figures may be implemented in various forms of hardware, software or combinations thereof. Preferably, these elements are implemented in a combination of hardware and software on one or more appropriately programmed general-purpose devices, which may include a processor, memory and input/output interfaces. Herein, the phrase "coupled" is defined to mean directly connected to or indirectly connected with through one or more intermediate components. Such intermediate components may include both hardware and software based components.
It is to be further understood that, because some of the constituent system components and method steps depicted in the accompanying figures are preferably implemented in software, the actual connections between the system components (or the process steps) may differ depending upon the manner in which the proposed method and apparatus is programmed. Given the teachings herein, one of ordinary skill in the related art will be able to contemplate these and similar implementations or configurations of the proposed method and apparatus.

Claims

A method of creating an anomalous behavior detection log for reporting anomalous behavior of a user in an environment, said method comprising:
logging (705) first time-stamped event data of initial behavior of said user over a period of time in the environment;

estimating (205) a location of the user in the environment using said time-stamped event data of said initial behavior of said user;

creating (210) a characterization of life routines of the initial behavior of said user;

logging (210) second time-stamped event data of subsequent behavior of the user;

estimating (210) a user location sequence of the user in the environment using said time-stamped event data of the subsequent behavior of the user;

generating (215, 920) user location sequence comparisons by comparing said characterization of life routines of the initial behavior of the user and said estimated user location sequence of the subsequent behavior of the user; and

creating (925) an anomalous behavior detection log by detecting anomalous behavior using said user location sequence comparisons.
The method according to claim 1, further comprising:
accepting first time-stamped event data from sensors in said environment for a period of time; and

reporting any detected anomalous behavior of said user to a care giver.
The method according to claim 1, wherein said characterization of life routines is used as training data.
The method according to claim 2, wherein said characterization of life routines further comprises:
creating a collection of user location sequences;

applying an unsupervised clustering process to said user location sequences to identify clusters of said user location sequences;

identifying significant clusters from among said identified clusters by comparing a number of user location sequences in each identified cluster to a threshold;

selecting a medoid to represent each significant cluster;

calculating a distance between said medoid of each significant cluster and each user location sequence in the medoid's cluster; and

determining a greatest distance from among said calculated distances between said medoid of each cluster and each user location sequence in the medoid's cluster.
The method according to claim 4, wherein said creating said anomalous behavior detection log by detecting anomalous behavior using said user location sequence comparisons further comprises:
accepting second time-stamped event data from sensors in said environment;

creating a second user location sequence from second time-stamped event data;

calculating a distance between said second user location sequence and said medoids of each significant cluster;

comparing said distance between (said second user location sequence and said medoids of each significant cluster) and (said greatest distance between said medoid of each significant cluster and each user location sequence in the medoid's cluster); and

determining if said second user location sequence is within said user's life routine.
The method according to claim 4, wherein a medoid is one of said user location sequences in a significant cluster whose dissimilarity to all other user location sequences in the significant cluster is minimal.
The method according to claim 4, wherein said threshold is a function of the number of sequences of the training data and a number of identified clusters.
The method according to claim 5, wherein said second user location sequence is within said user's life routine if said distance between said second user location sequence and said medoids of each significant cluster is greater than said greatest distance between said medoid of each significant cluster and each user location sequence in the medoid's cluster.
A server for creating an anomalous behavior detection log for reporting anomalous behavior of a user in an environment, comprising:
means for logging first time-stamped event data of initial behavior of said user over a period of time in the environment (11);

means for estimating a location of the user in the environment using said time-stamped event data of said initial behavior of said user (1);

means for creating a characterization of life routines of the initial behavior of the user (3);

means for logging second time-stamped event data of subsequent behavior of the user (11, 1);

means for estimating a user location sequence of the user in the environment using said time-stamped event data of the subsequent behavior of the user (1, 12);

means for generating user location sequence comparisons by comparing said characterization of life routines of the initial behavior of the user and said estimated user location sequence of the subsequent behavior of the user (2); and

means for creating an anomalous behavior detection log by detecting anomalous behavior using said user location sequence comparisons (2).
The server according to claim 9, further comprising:
means for accepting first time-stamped event data from sensors in said environment for a period of time; and

means for reporting any detected anomalous behavior of said user to a care giver.
The server according to claim 9, wherein said first characterization of life routines is used as training data.
The server according to claim 10, wherein means for first characterization of life routines further comprises:
means for creating a collection of user location sequences;

means for applying an unsupervised clustering process to said user location sequences to identify clusters of said user location sequences;

means for identifying significant clusters from among said identified clusters by comparing a number of user location sequences in each identified cluster to a threshold;

means for selecting a medoid to represent each significant cluster;

means for calculating a distance between said medoid of each significant cluster and each user location sequence in the medoid's cluster; and

means for determining a greatest distance from among said calculated distances between said medoid of each cluster and each user location sequence in the medoid's cluster.
The server according to claim 12, wherein said means for creating said anomalous behavior detection log by detecting anomalous behavior using said user location sequence comparisons further comprises:
means for accepting second time-stamped event data from sensors in said environment;

means for creating a second user location sequence from second time-stamped event data;

means for calculating a distance between said second user location sequence and said medoids of each significant cluster;

means for comparing said distance between (said second user location sequence and said medoids of each significant cluster) and (said greatest distance between said medoid of each significant cluster and each user location sequence in the medoid's cluster); and

means for determining if said second user location sequence is within said user's life routine.
The server according to claim 12, wherein a medoid is one of said user location sequences in a significant cluster whose dissimilarity to all other user location sequences in the significant cluster is minimal and wherein said threshold is a function of the number of sequences of the training data and a number of identified clusters.
The server according to claim 13, wherein said second user location sequence is within said user's life routine if said distance between said second user location sequence and said medoids of each significant cluster is greater than said greatest distance between said medoid of each significant cluster and each user location sequence in the medoid's cluster.