JP2007249922A - Nonroutine action detecting system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To automatically define routine action from the moving line of action in real environment by clustering the moving line of action in the real environment into appropriate action. <P>SOLUTION: This nonroutine action detecting system comprises a first clustering section 1201 clustering a plurality of action data generated from the time sequence moving line information of a person for every action; a first action model generating section 1202 generating an action model by modeling the plurality of action data clustered by the first clustering section 1201, for every model; a first recognition processing section 1203 recognizing the plurality of action data using the first action model to obtain a recognition result and likelihood; a first action data selecting section 1204 selecting only the action data in which the likelihood obtained by the first recognition processing section 1203 is a threshold value or more; and a second clustering section 1205 clustering the action data selected by the first action data selecting section 1204, into action shown by the recognition result of the first recognition processing section 1203. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to an extraordinary behavior detection system for detecting an extraordinary behavior of a person who is a target of behavior recognition.

  2. Description of the Related Art Conventionally, in a system for detecting an anomaly of an elderly person or a system for watching the behavior of an elderly person, a technique for detecting a person's action and recognizing the person's action (hereinafter referred to as “behavior recognition technology”) is used. ing. Also, for the general public, an attempt has been made to provide an appropriate service by inferring the person's state from living behavior. Further, as such behavior recognition technology, the following behavior recognition technology is known.

  Compare the action model generated in advance with the data that is the target of action recognition (hereinafter referred to as “recognition target data”), and recognize the action corresponding to the action model with the highest likelihood of the recognition target data. The technology to output is mentioned. Specifically, a plurality of behavior models are generated in advance from human behavior data by a statistical model generation method such as a Hidden Markov Model (HMM), and the recognition target data in the plurality of behavior models is generated. The behavior corresponding to the behavior model with the highest likelihood is output as a recognition result.

In order to detect the unusualness of behavior using behavior recognition technology, there is a method to determine the occurrence of anomalies by detecting the difference between the daily behavior template defined in advance by hand and the behavior that actually occurred It has been proposed (see, for example, Patent Document 1).
JP 2003-256957 A

  However, in the background art described above, daily actions are defined in advance by hand before detecting the occurrence of an abnormality. For this reason, it is very difficult to define all of the actions of a complex and diverse person by a combination of action characteristics, and only a limited number of person actions can be recognized.

  In addition to detecting abnormal behavior such as falls and falls, the system for monitoring elderly people's behavior detects the lifestyle patterns of elderly people while protecting their privacy so that care welfare facilities can provide better quality care. However, there is a strong demand for estimating changes in lifestyle and health conditions, and signs in the hallway.

  In order to detect the unusualness of behavior from continuous real life data, it is necessary to devise a method of dividing real life data and a method of collecting similar behaviors in divided behavior when learning the behavior.

  In view of the above problems, the present invention is a non-daily behavior detection system that can automatically define a daily behavior from a behavior flow in the real environment by clustering the behavior flow in the real environment into an appropriate behavior. I will provide a.

  A first feature of the present invention is an extraordinary behavior detection system that detects an extraordinary behavior of a person, and a plurality of behavior data generated from time-series flow line information of the person is clustered for each behavior and clustered. A first learning processing unit that generates a first behavior model obtained by modeling a plurality of behavior data for each behavior; and a second learning processing unit that generates a second behavior model by clustering the plurality of behavior data. The second learning processing unit recognizes a plurality of behavior data using the first behavior model and obtains a recognition result and a likelihood, and the likelihood obtained by the first recognition processing unit is equal to or greater than a threshold value. A first behavior data selection unit that selects only behavior data from a plurality of behavior data, and a second cluster that clusters the behavior data selected by the first behavior data selection unit into the behavior indicated by the recognition result of the first recognition processing unit A ring portion, the second clustering unit is summarized in that a non-daily activity detection system comprising a second behavior model generating unit for generating a second behavior model from the action data clustering. According to the extraordinary behavior detection system according to the first feature, the routine behavior can be automatically defined from the behavior flow line in the real environment by collecting similar behaviors and clustering them into appropriate behaviors.

  The second feature further includes third to n-th learning processing units that generate a third to n-th behavior model by clustering a plurality of behavior data in the extraordinary behavior detection system according to the first feature (n An integer of 4 or more), the third learning processing unit recognizes a plurality of behavior data using the second behavior model, and obtains a recognition result and a likelihood, and a second recognition processing unit obtains the recognition result and likelihood. The recognition result of the second recognition processing unit indicates the second behavior data selection unit that selects only the behavior data whose likelihood is equal to or greater than the threshold from the plurality of behavior data, and the behavior data selected by the second behavior data selection unit. The gist of the present invention is to include a third clustering unit that performs clustering on behavior and a third behavior model generation unit that generates a third behavior model from behavior data clustered by the third clustering unit.

  The third feature is the behavior recognition unit that recognizes behavior data to be recognized using any one of the second to n-th behavior models and detects unusual behavior in the unusual behavior detection system according to the second feature. Is further provided.

  The fourth feature is that in the extraordinary behavior detection system according to the third feature, the number of behavior data clustered into behaviors different from the previous clustering result by clustering by the second to nth learning processing units is equal to or less than a threshold value. The gist of the present invention is to further include a clustering result determination unit that uses the behavior model generated at the time of becoming the final behavior model for recognizing the behavior data to be recognized.

  The fifth feature is that in the extraordinary behavior detection system according to the third or fourth feature, the number of behavior data clustered into different behaviors is equal to or less than a threshold value by clustering by the second to nth learning processing units. The gist of the invention is to further include a behavior model selection unit that selects the behavior model generated at this time as a final behavior model for recognizing behavior data to be recognized. According to the extraordinary behavior detection system according to the fourth feature, the extraordinary behavior in the behavior can be detected by considering the internal state of the behavior.

  A sixth feature is that in the extraordinary behavior detection system according to the fifth feature, the behavior recognition unit recognizes the behavior data to be recognized using any one of the second to nth behavior models, and the recognition result and the likelihood. A third recognition processing unit that obtains a degree, and the behavior data to be recognized and the histogram corresponding to the behavior indicated by the recognition result of the third recognition processing unit are compared, and the difference value of the frequency of the movement amount per unit time is calculated. The gist is to include a difference value calculation unit and an evaluation value calculation unit that calculates an evaluation value used for detecting an extraordinary action based on the likelihood and the difference value obtained by the third recognition processing unit. According to the extraordinary behavior detection system according to the sixth feature, it is possible to obtain an evaluation value used for detecting extraordinary behavior based on the difference value and likelihood of the histogram.

  The seventh feature is that in the extraordinary behavior detection system according to the sixth feature, the behavior recognition unit accumulates the evaluation values and determines that the extraordinary behavior occurs when the cumulative evaluation value exceeds the threshold value. Is further provided. According to the extraordinary behavior detection system according to the seventh feature, the extraordinary behavior in a series of behaviors can be detected.

  The eighth feature is that in the extraordinary behavior detection system according to the first to seventh features, the stop state determination unit that determines the movement start and stop of the person from the time-series flow line information, and the time-series flow line information An area transition determination unit that determines the presence or absence of an area transition of a person, and time-series flow line information that divides time-series flow line information from movement start to movement stop, and includes time-series flow line information including area transitions in the divided time-series flow line information. A gist is to further include an action data output unit that outputs action data and action data to be recognized, and uses an area transition history indicating an area transition history as a clustering reference. According to the extraordinary behavior detection system according to the eighth feature, it is possible to perform optimal data division from continuous real life data (time-series flow line information).

  According to the present invention, it is possible to provide an extraordinary behavior detection system that can automatically define a daily behavior from a behavior flow in the real environment by clustering the behavior flow in the real environment into an appropriate behavior.

  Next, an embodiment of the present invention will be described with reference to the drawings. In the description of the drawings in the following embodiments, the same or similar parts are denoted by the same or similar reference numerals.

(Overall configuration example)
As shown in FIG. 1, the extraordinary behavior detection system according to the embodiment of the present invention is configured as a remote monitoring (remote monitoring) system for a person such as an elderly person, for example. The extraordinary behavior detection system includes a detection device 110 installed in a monitored person's residence, an extraordinary behavior detection device 1 installed in a monitoring center, and an output device 120 installed in the monitoring subject's residence. The detection device 110, the extraordinary behavior detection device 1, and the output device 120 are configured to be able to communicate with each other via a communication network 100 such as the Internet. Note that the communication network 100 is not limited to a wired network, and may be a wireless network.

  The detection device 110 is configured to detect a person's live activity line, and the sensor unit 101 that detects a person's action flow line, and the data detected by the sensor unit 101 are processed to move per unit time. A sensor data processing unit 102 that outputs quantity and coordinate data, and a communication processing unit 103 that communicates with the communication network 100 are provided. As the sensor unit 101, for example, a floor pressure sensor, a pyroelectric sensor, a wireless tag (RF-ID), an imaging device, or a combination thereof can be used. That is, as long as a person's live activity line can be detected, the sensor unit 101 can be used. In the following, an example in which a plurality of floor pressure sensors is used as the sensor unit 101 will be described.

  The extraordinary behavior detection device 1 is configured to detect a person's extraordinary behavior based on time-series flow line information received from the detection device 110 via the communication network 100, and performs various types of information processing. The apparatus 10 includes a storage device 20 that stores various types of information, and a communication processing unit 30 that communicates with the communication network 100. The processing device 10 executes a control program stored in the storage device 20 in advance, thereby generating a plurality of behavior data from the time-series flow line information, and using the plurality of behavior data, The behavior learning unit 12 that learns the behavior and the behavior recognition unit 13 that recognizes the behavior data based on the learning result of the behavior learning unit 12 and detects the extraordinary behavior are executed.

  The output device 120 includes a notification unit 121 that notifies the monitor of the occurrence of extraordinary behavior in the monitored person, a control unit 122 that controls the notification unit 121, and a communication processing unit 123 that communicates with the communication network 100. With. As the notification unit 121, a video output device, an audio output device, a home robot, or a combination thereof can be used.

  Next, with reference to the flowchart shown in FIG. 2, the operation | movement outline | summary of the extraordinary action detection apparatus 1 which concerns on embodiment of this invention is demonstrated.

  In step S <b> 101, the behavior data generation unit 11 receives time series flow line information of a person from the detection device 110 via the communication network 100, and a plurality of behavior data (behavior for learning) is received from the received time series flow line information. Data).

  In step S102, the behavior learning unit 12 learns the plurality of behavior data generated in step S101 and generates a behavior model.

  In step S <b> 103, the behavior data generation unit 11 receives time series flow line information of a person from the detection device 110 via the communication network 100, and a plurality of behavior data (recognition target behaviors) are received from the received time series flow line information. Data).

  In step S104, the behavior recognition unit 13 recognizes the behavior data obtained in step S103 using the behavior model obtained in step S102, and detects unusual behavior.

  Hereinafter, details of the behavior data generation unit 11, the behavior learning unit 12, and the behavior recognition unit 13 will be described.

(Behavior data generator)
An example of the functional configuration of the behavior data generation unit 11 is shown in FIG. The behavior data generation unit 11 includes a stop state determination unit 111 that determines the start and stop of movement of a person from time-series flow line information, and an area transition determination unit 112 that determines whether or not there is a person area transition from the time-series flow line information. And a behavior data output unit 113 that divides time-series flow line information from the start of movement to stop of movement and outputs time-series flow line information including area transitions among the divided time-series flow line information as action data. . As shown in FIG. 4, “area transition” means a room transition with a room corresponding to the layout of the monitored person's residence as an area, but a plurality of areas are defined in one room, It may be a transition between defined areas. The behavior data output unit 113 adds area transition history information indicating an area transition history to the behavior data. The area transition history information is used for clustering processing of the behavior learning unit 12 and the like.

  In addition, the points (time-series flow line information) plotted in a circle and a square in FIG. 4 indicate positions where persons exist for each unit time (for example, 5 [s]), and the amount of movement per unit time is shown respectively. Information and information on average coordinates in the y direction orthogonal to the x direction and the x direction.

  As illustrated in FIG. 4, the stop state determination unit 111 determines that a movement amount per unit time is 1 [m] or less as a stop state. Specifically, the stop state determination unit 111 determines a stop state by comparing movement amount data per unit time with a preset stop state threshold.

  The area transition determination unit 112 receives the average coordinate data as input and determines whether there is an area transition. Specifically, the area transition determination unit 112 determines the area transition by comparing the average coordinate data with a preset area threshold. In the example shown in FIG. 4, the area transition history is represented by “AB” because it moves (transitions) from room A to room B. In the following description of the embodiment, it is assumed that there are at least seven rooms “A” to “G” in the residence of the monitored person.

  Next, with reference to the flowchart shown in FIG. 5, the operation example of the action data generation part 11, ie, the detail of step S101 and S103 shown in FIG. 2, is demonstrated. However, for convenience of explanation, a case where one behavior data is generated will be described.

  In step S201, the stop state determination unit 111 compares the movement amount data per unit time with the stop state threshold value, and determines that the movement of the person is started when the stop state threshold value is greater than or equal to the stop state threshold value. Moreover, the area transition determination part 112 detects an area transition by comparing average coordinate data with an area threshold value.

  In step S202, the stop state determination unit 111 compares the movement amount data per unit time with the stop state threshold value, and determines that the movement of the person is stopped when it is less than the stop state threshold value.

  In step S203, the area transition determination unit 112 determines whether an area transition is included. If it is determined that an area transition is included, the process proceeds to step S204. If it is determined that the area transition is not included, the process ends without generating behavior data.

  In step S204, the behavior data output unit 113 generates time-series flow line information from the movement start detected in step S201 to the movement stop detected in step S202 as behavior data (data division). Further, the behavior data output unit 113 adds area transition history information to the behavior data. The generated behavior data is stored in, for example, the storage device 20 illustrated in FIG.

  Thus, according to the action data generation unit 11 according to the embodiment of the present invention, action data that is a data processing unit can be automatically generated from continuous real life data (time-series flow line information).

(Behavioral Learning Department)
An example of the functional configuration of the behavior learning unit 12 is shown in FIG. As illustrated in FIG. 6, the behavior learning unit 12 includes first to third learning processing units 1251 to 1253 and a histogram generation unit 1211. The first to third learning processing units 1251 to 1253 learn a plurality of behavior data stored in the behavior data storage unit 21 to generate behavior models (learning models), respectively. The first learning processing unit 1251 includes a first clustering unit 1201 and a first behavior model generation unit 1202. The second learning processing unit 1252 includes a first recognition processing unit 1203, a first behavior data selection unit 1204, a second clustering unit 1205, and a second behavior model generation unit 1206. The third learning processing unit 1253 includes a second recognition processing unit 1207, a second behavior data selection unit 1208, a third clustering unit 1209, and a third behavior model generation unit 1210.

  The behavior data storage unit 21, the first behavior model storage unit 22, the second behavior model storage unit 23, the third behavior model storage unit 24, and the histogram storage unit 25 illustrated in FIG. 6 are included in the storage device 20 illustrated in FIG. Is provided. The behavior data storage unit 21 stores a plurality of behavior data generated by the behavior data generation unit 11.

  The first clustering unit 1201 reads a plurality of behavior data stored in the behavior data storage unit 21, and as shown in FIG. 7A, the plurality of read behavior data “ABC-1” and “ABC-2”. , “ABC-3”, “EFC-1”, “EFC-2”, “GBAE-1”, “GBAE-2”, and “BA-1” are clustered for each action (area transition history).

  In the example shown in FIG. 7A, the action data “ABC-1”, “ABC-2”, and “ABC-3” whose action (area transition history) is “ABC” are clustered into one. Yes. Action data “EFC-1” and “EFC-2” whose actions (area transition history) are “EFC” are clustered into one. The behavior data “GBAE-1” and “GBAE-2” whose behavior (area transition history) is “GBAE” are clustered into one. The action data “BA-1” whose action (area transition history) is “BA” is clustered into one.

  As shown in FIG. 7A, the first behavior model generation unit 1202 generates a behavior model in which a plurality of behavior data clustered by the first clustering unit 1201 is modeled for each behavior (area transition history). The first behavior model is stored in the first behavior model storage unit 22.

  A specific example of the behavior model generation method will be described below. The first behavior model generation unit 1202 generates a behavior model based on behavior data including a movement amount per unit time and average coordinates in the x direction and the y direction. Specifically, the first behavior model generation unit 1202 uses a statistical model generation method such as a Hidden Markov Model (HMM) for each behavior (area transition history) such as “Behavior ABC” and “Behavior EFC”. Generate a behavior model.

  Subsequently, the first behavior model generation unit 1202 estimates the parameters of the hidden Markov model for each behavior (area transition history) such as “Behavior ABC” and “Behavior EFC” according to the Baum-Welch algorithm. The hidden Markov model parameters include the probability of transition from one state to another state (transition probability), the probability that one output symbol is output from one state, and the like. The first behavior model generation unit 1202 stores the parameters of the hidden Markov model estimated for each behavior (area transition history) such as “Behavior ABC” and “Behavior EFC” as a behavior model in the first behavior model storage unit 22. To do. In other words, the first behavior model storage unit 22 stores a plurality of first behavior models for each human behavior (area transition history).

  In addition, the first recognition processing unit 1203 reads the plurality of behavior data stored in the behavior data storage unit 21 and the first behavior model stored in the first behavior model storage unit 22, and uses the first behavior model. A plurality of behavior data are recognized, and the recognition result and likelihood are obtained as shown in FIG. Specifically, the first recognition processing unit 1203 calculates the likelihood of behavior data for a plurality of first behavior models stored in the first behavior model storage unit 22. In addition, the first recognition processing unit 1203 identifies the behavior model that maximizes the likelihood of the behavior data, and outputs the identified behavior model as a recognition result.

  In the following, a behavior model generation method will be described by taking as an example the case of using a hidden Markov model as a statistical model generation method. For example, in the hidden Markov model described above, the first recognition processing unit 1203 generates an output symbol string corresponding to the behavior data based on the behavior data including the movement amount per unit time and the average coordinates in the x direction and the y direction. get. Subsequently, the first recognition processing unit 1203 calculates a probability (likelihood) that the acquired output symbol string is output according to each hidden Markov model.

Specifically, the first recognition processing unit 1203 calculates the likelihood (P _λ ) of action data for each hidden Markov model (behavior model) according to the following equation.

_{Here, O 1, O 2, ···} O t is the output symbol of the behavior models (hidden Markov _{model), o k (1),} o k (2) ··· o k (t) is, It is an output symbol of action data, and N is the number of states. Also, π _j is the probability that the initial state is state (j), a _{i, j} is the transition probability of transition from state (i) to state (j), and b _{j, k (t)} is , The probability that the output symbol (ok _(t) ) is output from the state (j) at time t.

  Subsequently, the first behavior data sorting unit 1204 sorts only behavior data having a likelihood equal to or higher than a preset threshold (−25.0 in this case) from the plurality of behavior data. As a result, as shown in FIG. 8, the behavior data “ABC-3” and “GBAE-2” are excluded from the clustering targets by the first behavior data selection unit 1204.

  The second clustering unit 1205 clusters the behavior data selected by the first behavior data selection unit 1204 into the behavior (area transition history) indicated by the recognition result of the first recognition processing unit 1203. Therefore, as shown in FIGS. 8 and 9, the action data “ABC-1”, “ABC-2”, “EFC-2”, and “BA-1” are clustered into one. The action data “EFC-1” and “GBAE-1” are clustered into one.

  The second behavior model generation unit 1206 generates a second behavior model from the behavior data clustered by the second clustering unit 1205. Therefore, as shown in FIG. 9A, the action (area transition history) “ABC” is taken from action data “ABC-1”, “ABC-2”, “EFC-2”, and “BA-1”. A model is generated. A behavior model of behavior (area transition history) “EFC” is generated from the behavior data “EFC-1” and “GBAE-1”. The generated second behavior model is stored in the second behavior model storage unit 23.

  Further, the second recognition processing unit 1207 reads the plurality of behavior data stored in the behavior data storage unit 21 and the second behavior model stored in the second behavior model storage unit 23, and uses the second behavior model. A plurality of behavior data are recognized, and the recognition result and likelihood are obtained as shown in FIG.

  The second behavior data sorting unit 1208 sorts only behavior data whose likelihood obtained by the second recognition processing unit 1207 is equal to or greater than a preset threshold value (−25.0 in this case) from the plurality of behavior data. As a result, as shown in FIG. 10A, the behavior data “GBAE-2” is excluded from the clustering by the second behavior data selection unit 1208.

  The significance of the third clustering process is to cluster the behavior data excluded from the second clustering target, and aims to increase the robustness of the behavior model.

  Further, the third clustering unit 1209 clusters the behavior data selected by the second behavior data selection unit 1208 into the behavior (area transition history) indicated by the recognition result of the second recognition processing unit 1207. Therefore, as shown in FIG. 10A, the action data “ABC-1”, “ABC-2”, “EFC-2”, “BA-1”, and “ABC-3” are clustered into one. ing. Therefore, the action data “ABC-3” excluded in the second clustering is relieved. Also, the behavior data “EFC-1” and “GBAE-1” are clustered into one.

  The third behavior model generation unit 1210 generates a third behavior model from the behavior data clustered by the third clustering unit 1209. Therefore, as shown in FIG. 10B, the action data (ABC-1), “ABC-2”, “EFC-2”, “BA-1”, and “ABC-3” are used as actions (area transition history). ) A behavior model of “ABC” is generated. A behavior model of behavior (area transition history) “EFC” is generated from the behavior data “EFC-1” and “GBAE-1”. The generated third behavior model is stored in the third behavior model storage unit 24.

  Further, as shown in FIGS. 11A to 11C, the histogram generation unit 1211 generates a histogram indicating the frequency distribution of the movement amount per unit time for each action data clustered by the third clustering unit 1209. . Specifically, the action data “ABC-1”, “ABC-2”, “EFC-2”, “BA-1” clustered in the action (area transition history) “ABC” shown in FIG. ”And“ ABC-3 ”, a histogram indicating the frequency distribution of the amount of movement per unit time is generated. Further, the histograms of the behavior data “ABC-1”, “ABC-2”, “EFC-2”, “BA-1”, and “ABC-3” are averaged as shown in FIG. A simple histogram. In the histogram shown in FIG. 11B, since the movement amount per unit time is small, it can be determined that the action is slow as a whole.

  Further, for each of the action data “EFC-1” and “GBAE-1” clustered in the action (area transition history) “EFC” shown in FIG. Is generated. Further, the histograms of behavior data “EFC-1” and “GBAE-1” are averaged to generate a histogram as shown in FIG. In the histogram shown in FIG. 11C, since the movement amount per unit time is often large, it can be determined that the action is fast as a whole.

  Next, with reference to the flowchart shown in FIG. 12, the operation example of the action learning part 12, ie, the detail of step S102 of FIG. 2, is demonstrated.

  In step S301, the first clustering unit 1201 reads a plurality of behavior data stored in the behavior data storage unit 21, and clusters the read behavior data for each behavior (area transition history).

  In step S302, the first behavior model generation unit 1202 generates a first behavior model obtained by modeling the plurality of behavior data clustered in step S301 for each behavior (area transition history). The first behavior model is stored in the first behavior model storage unit 22.

  In step S303, the first recognition processing unit 1203 reads the plurality of behavior data stored in the behavior data storage unit 21 and the first behavior model stored in the first behavior model storage unit 22, and uses the first behavior model. To recognize a plurality of behavior data and obtain a recognition result and likelihood.

  In step S304, the first behavior data sorting unit 1204 sorts only behavior data having a likelihood equal to or greater than a threshold from a plurality of behavior data.

  In step S305, the second clustering unit 1205 clusters the selected action data into actions (area transition history) indicated by the recognition result.

  In step S306, the second behavior model generation unit 1206 generates a second behavior model from the behavior data clustered by the second clustering unit 1205. The second behavior model is stored in the second behavior model storage unit 23.

  In step S307, the second recognition processing unit 1207 reads the plurality of behavior data stored in the behavior data storage unit 21 and the second behavior model stored in the second behavior model storage unit 23, and uses the second behavior model. To recognize a plurality of behavior data and obtain a recognition result and likelihood.

  In step S308, the second behavior data sorting unit 1208 sorts only behavior data whose likelihood obtained in step S307 is equal to or greater than a threshold from a plurality of behavior data.

  In step S309, the third clustering unit 1209 clusters the behavior data selected in step S308 into the behavior (area transition history) indicated by the recognition result in step S307.

  In step S310, the third behavior model generation unit 1210 generates a third behavior model from the behavior data clustered in step S309.

  In step S311, a histogram indicating the frequency distribution of the amount of movement per unit time is generated for each action (area transition history) for the action data clustered in step S309.

  As described above, according to the behavior learning unit 12 according to the embodiment of the present invention, it is possible to automatically generate an appropriate behavior model for the behaviors of complicated and diverse persons. Therefore, it is possible to define daily life patterns while protecting privacy.

(Action Recognition Department)
An example of the functional configuration of the action recognition unit 13 is shown in FIG. The action recognition unit 13 receives the action data to be recognized, the third action model, and the histogram as inputs, and outputs an extraordinary action detection notification for notifying the monitor of the occurrence of the extraordinary action in the monitored person. The output extraordinary behavior detection notification is transmitted to the output device 120 via the communication processing unit 30 and the communication network 100 shown in FIG. As a result, the monitor can know the occurrence of the extraordinary behavior by the monitored person.

  The behavior recognition unit 13 includes a third recognition processing unit 1301, a difference value calculation unit 1302, an evaluation value calculation unit 1303, and a determination unit 1304. The evaluation value storage unit 27 is provided in the storage device 20 shown in FIG.

  The third recognition processing unit 1301 recognizes the behavior data to be recognized using the third behavior model stored in the third behavior model storage unit 24, and obtains the recognition result and likelihood. Action data 1 to 5 as shown in FIG. 14A are sequentially input to the third recognition processing unit 1301. For example, it is assumed that the action data 1 to 5 are measured from a series of actions in a life scene with a person.

  The difference value calculation unit 1302 reads a histogram corresponding to the behavior (area transition history) indicated by the recognition result of the third recognition processing unit 1301 from the histogram storage unit 25. Further, the difference value calculation unit 1302 normalizes the recognition target data by the total number of data (the number of samples), and compares it with the read histogram for each movement amount. This comparison employs a method of integrating the frequency difference of the histogram with each movement amount. As shown in FIG. 14B, the movement amount smaller than the reference value with respect to the reference value of the movement amount set in advance, that is, when the movement speed is slow, is positive when the frequency is less than the histogram data. Accumulate. On the other hand, when the movement amount larger than the reference value, that is, the movement speed is faster than the reference value of the movement amount set in advance, it is integrated as positive when the frequency is higher than the histogram data. The integrated value obtained in this way is used as a difference value. Therefore, the difference value is output as positive when there is a lot of data having a larger movement amount than the histogram data.

  As illustrated in FIG. 15A, the evaluation value calculation unit 1303 multiplies the likelihood obtained by the third recognition processing unit 1301 by the difference value calculated by the difference value calculation unit 1302 to detect the extraordinary behavior. The evaluation value used for is calculated. The evaluation value is stored in the evaluation value storage unit 27.

  As shown in FIG. 15B, the determination unit 1304 integrates the evaluation values, and determines that the extraordinary behavior occurs when the cumulative evaluation value exceeds a preset threshold value.

  Next, with reference to the flowchart shown in FIG. 16, the operation example of the action recognition part 13, ie, the detail of step S104 of FIG. 2, is demonstrated.

  In step S401, the third recognition processing unit 1301 reads the third behavior model from the third behavior model storage unit, recognizes recognition target data using the third behavior model, and obtains a recognition result and likelihood.

  In step S402, the difference value calculation unit 1302 reads a histogram corresponding to the behavior (area transition history) indicated by the recognition result obtained in step S401 from the histogram storage unit 25. Further, the difference value calculation unit 1302 normalizes the recognition target data with the total number of data, and compares the data with the histogram for each movement amount. In this way, the difference value of the frequency of the movement amount is calculated.

  In step S403, the evaluation value calculation unit 1303 obtains an evaluation value by multiplying the likelihood obtained in step S401 by the difference value calculated in step S402.

  In step S404, the determination unit 1304 integrates the evaluation values obtained in step S403.

  In step S405, the determination unit 1304 determines whether or not the cumulative evaluation value integrated in step S404 exceeds a threshold value. When it is determined that the cumulative evaluation value exceeds the threshold value, the process proceeds to step S406.

  In step S406, the determination unit 1304 determines that the action is an extraordinary action, and outputs an extraordinary action detection notification.

  Thus, according to the action recognition unit 13 according to the embodiment of the present invention, the likelihood between the action data and the action model and the similarity (histogram difference value) between the internal state of the action data are evaluated together. This makes it possible to detect unusualness in behavior. Therefore, for example, the present invention can be applied to a system that detects forgetting behavior as an extraordinary behavior. One cause of forgetfulness is that attention (consciousness) is directed elsewhere. Forgetfulness is thought to occur in a chain, such as when attention is directed elsewhere, a certain action fails, and the consciousness toward the failed action further fails another action. Therefore, it can be applied to a system for estimating whether or not forgetfulness is likely to occur thereafter by detecting the unusualness of the behavior unit.

(Experimental example)
Next, an experimental example of the extraordinary behavior detection system according to the embodiment of the present invention will be described with reference to FIG. Each experimental data shown in FIGS. 17A and 17B includes a plurality of action data each consisting of a series of actions. As shown in FIG. 17C, it can be seen that, out of 10 experiments, only 3 times of Experiments 2, 5, and 8 were erroneously recognized, and 7 times were correctly recognized.

(First modification)
In the embodiment described above, an example has been described in which the behavior learning unit 12 generates three behavior models of the first to third behavior models and uses the third behavior model as a final behavior model. However, as a first modification of the embodiment of the present invention, as shown in FIG. 18, the second behavior model may be used as the final behavior model.

(Second modification)
As illustrated in FIG. 19, the behavior learning unit 1200 according to the second modification of the embodiment of the present invention can input the behavior model stored in the second behavior model storage unit 23 to the recognition processing unit 1203 again. 6 is different from FIG. That is, the second learning processing unit 1252 is used a plurality of times to generate a plurality of behavior models.

  Further, the behavior learning unit 1200 uses the second clustering unit 1205 to perform a final behavior model generated when the number of behavior data clustered into behaviors different from the previous clustering result is equal to or less than a threshold value. An action model selection unit 1260 for selecting as an action model is further provided. Here, “behavior data clustered into behaviors different from the previous clustering result” includes behavior data whose clustering result is “none”. In the example shown in FIG. 8, the action data corresponds to action data “ABC-3”, “EFC-2”, “GBAE-1”, “GBAE-2”, and “BA-1”. The number of action data ”is five. The threshold value can be arbitrarily set.

  Next, an operation example of the behavior learning unit 1200 according to the second modified example of the embodiment of the present invention will be described with reference to the flowchart shown in FIG.

  In step S301, the first clustering unit 1201 reads a plurality of behavior data stored in the behavior data storage unit 21, and clusters the read behavior data for each behavior (area transition history).

  In step S302, the first behavior model generation unit 1202 generates a first behavior model obtained by modeling the plurality of behavior data clustered in step S301 for each behavior (area transition history). The first behavior model is stored in the first behavior model storage unit 22.

  In step S303, the first recognition processing unit 1203 reads the plurality of behavior data stored in the behavior data storage unit 21 and the first behavior model stored in the first behavior model storage unit 22, and uses the first behavior model. To recognize a plurality of behavior data and obtain a recognition result and likelihood.

  In step S304, the behavior data sorting unit 1204 sorts only behavior data having a likelihood equal to or greater than a threshold from a plurality of behavior data.

  In step S305, the second clustering unit 1205 clusters the selected action data into actions (area transition history) indicated by the recognition result.

  In step S306, the second behavior model generation unit 1206 generates a second behavior model from the behavior data clustered by the second clustering unit 1205. The second behavior model is stored in the second behavior model storage unit 23.

  In step S320, the behavior model selection unit 1260 determines whether the number of behavior data clustered in behavior (area transition history) different from the previous clustering result in step S305 is equal to or less than a threshold value. When it is determined that the number of behavior data clustered in behavior (area transition history) different from the previous clustering result is less than or equal to the threshold value, the process proceeds to step S321. If it is determined that the number of action data clustered in an action (area transition history) different from the previous clustering result exceeds the threshold, the process returns to step S303. In step S303 when the process returns, a recognition process using the second behavior model generated in step S306 is executed.

  In step S320, the behavior model selection unit 1260 selects the second behavior model generated in step S306 as a final behavior model.

  In step S311, for the behavior data clustered in step S305, a histogram indicating the frequency distribution of the amount of movement per unit time is generated for each behavior (area transition history).

  Thus, according to the behavior learning unit 1200 according to the second modification of the embodiment of the present invention, a good behavior model can be obtained by automatically repeating clustering until the end condition is satisfied.

(Other embodiments)
As mentioned above, although this invention was described by embodiment, it should not be understood that the description and drawing which form a part of this indication limit this invention. From this disclosure, various alternative embodiments, examples and operational techniques will be apparent to those skilled in the art.

  In the description of the embodiment already described, the case where the action recognition unit 13 detects unusualness in a series of actions in a life scene with a person has been described. However, you may detect unusualness in a longer period. Detection of extraordinary activity on a daily basis is effective for detecting changes in behavioral rhythms and changes in health status.

  Thus, it should be understood that the present invention includes various embodiments and the like not described herein. Therefore, the present invention is limited only by the invention specifying matters in the scope of claims reasonable from this disclosure.

It is a block diagram which shows the example of whole structure of the unusual activity detection system which concerns on embodiment of this invention. It is a flowchart which shows the operation | movement outline | summary of the extraordinary action detection apparatus which concerns on embodiment of this invention. It is a block diagram which shows the structural example of the action data generation part which concerns on embodiment of this invention. It is a figure for demonstrating the function of the action data generation part which concerns on embodiment of this invention. It is a flowchart which shows the operation example of the action data generation part which concerns on embodiment of this invention. It is a block diagram which shows the structural example of the action learning part which concerns on embodiment of this invention. FIG. 7A is a diagram for explaining the function of the first behavior model generation unit according to the embodiment of the present invention, and FIG. 7B is the function of the first recognition processing unit according to the embodiment of the present invention. It is a figure for demonstrating. It is a figure for demonstrating the function of the 1st action data selection part and 2nd clustering part which concern on embodiment of this invention. FIG. 9A is a diagram for explaining the function of the second behavior model generation unit according to the embodiment of the present invention, and FIG. 9B is the function of the second recognition processing unit according to the embodiment of the present invention. It is a figure for demonstrating. FIG. 10A is a diagram for explaining functions of the second action data selection unit and the third clustering unit according to the embodiment of the present invention, and FIG. 10B is a third diagram according to the embodiment of the present invention. It is a figure for demonstrating the function of an action model production | generation part. FIGS. 11A to 11C are diagrams for explaining the function of the histogram generation unit according to the embodiment of the present invention. It is a flowchart which shows the operation example of the action learning part which concerns on embodiment of this invention. It is a block diagram which shows the structural example of the action recognition part which concerns on embodiment of this invention. FIG. 14A is a diagram for explaining a function of the third recognition processing unit according to the embodiment of the present invention, and FIG. 14B is a diagram for explaining a function of the difference value calculating unit according to the embodiment of the present invention. It is a figure for doing. FIG. 15A is a diagram for explaining the function of the evaluation value calculation unit according to the embodiment of the present invention, and FIG. 15B is a diagram for explaining the function of the determination unit according to the embodiment of the present invention. FIG. It is a flowchart of the operation example of the action recognition part which concerns on embodiment of this invention. FIGS. 17A to 17C are diagrams illustrating experimental examples of the extraordinary behavior detection system according to the embodiment of the present invention. It is a block diagram which shows the structural example of the action learning part which concerns on the 1st modification of embodiment of this invention. It is a block diagram which shows the structural example of the action learning part which concerns on the 2nd modification of embodiment of this invention. It is a block diagram which shows the operation example of the action learning part which concerns on the 2nd modification of embodiment of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Extraordinary action detection apparatus 10 ... Processing apparatus 11 ... Action data generation part 12 ... Action learning part 13 ... Action recognition part 20 ... Memory | storage device 21 ... Action data storage part 22 ... 1st action model memory | storage part 23 ... 2nd action Model storage unit 24 ... Third behavior model storage unit 25 ... Histogram storage unit 27 ... Evaluation value storage unit 30 ... Communication processing unit 100 ... Communication network 101 ... Sensor unit 102 ... Sensor data processing unit 103 ... Communication processing unit 110 ... Detection device DESCRIPTION OF SYMBOLS 111 ... Stop state determination part 112 ... Area transition determination part 113 ... Behavior data output part 120 ... Output device 121 ... Notification part 122 ... Control part 123 ... Communication processing part 1201 ... 1st clustering part 1202 ... 1st action model production | generation part 1203 ... 1st recognition process part 1204 ... 1st action data selection part 1205 ... 2nd clustering part 1206 Second behavior model generation unit 1207 ... second recognition processing unit 1208 ... second behavior data selection unit 1209 ... third clustering unit 1210 ... third behavior model generation unit 1211 ... histogram generation unit 1251 ... first learning processing unit 1252 ... first 2 learning processing unit 1253 ... third learning processing unit 1260 ... behavior model selection unit 1301 ... third recognition processing unit 1302 ... difference value calculation unit 1303 ... evaluation value calculation unit 1304 ... determination unit

Claims (8)

An extraordinary behavior detection system for detecting an extraordinary behavior of a person,
A first learning processing unit for clustering a plurality of behavior data generated from the time-series flow line information of the person for each behavior, and generating a first behavior model in which the plurality of clustered behavior data is modeled for each behavior; ,
A second learning processing unit that clusters the plurality of behavior data to generate a second behavior model, and the second learning processing unit includes:
A first recognition processing unit that recognizes the plurality of behavior data using the first behavior model and obtains a recognition result and likelihood;
A first behavior data sorting unit that sorts only behavior data whose likelihood calculated by the first recognition processing unit is a threshold value or more from the plurality of behavior data;
A second clustering unit that clusters the behavior data selected by the first behavior data selection unit into the behavior indicated by the recognition result of the first recognition processing unit;
A non-daily behavior detection system comprising: a second behavior model generation unit that generates the second behavior model from behavior data clustered by the second clustering unit. A third to n-th learning processing unit that clusters the plurality of behavior data to generate a third to n-th behavior model (n; an integer of 4 or more);
The third learning processing unit
Recognizing the plurality of behavior data using the second behavior model, and obtaining a recognition result and likelihood;
A second behavior data sorting unit that sorts only behavior data whose likelihood obtained by the second recognition processing unit is greater than or equal to a threshold value from the plurality of behavior data;
A third clustering unit that clusters the behavior data selected by the second behavior data selection unit into the behavior indicated by the recognition result of the second recognition processing unit;
The extraordinary behavior detection system according to claim 1, further comprising: a third behavior model generation unit that generates the third behavior model from the behavior data clustered by the third clustering unit.   The non-daily behavior according to claim 2, further comprising a behavior recognition unit that recognizes behavior data to be recognized using any one of the second to n-th behavior models and detects the non-daily behavior. Detection system.   The behavior model generated when the number of behavior data clustered in a behavior different from the previous clustering result by the clustering by the second to n-th learning processing units is equal to or less than a threshold value is the behavior data to be recognized. The non-daily behavior detection system according to claim 3, further comprising a behavior model selection unit that selects as a final behavior model for recognizing a voice.   A histogram generation unit that generates, for each action, a histogram indicating a frequency distribution of the amount of movement per unit time for the action data clustered by any of the second to n-th learning processing parts. The extraordinary behavior detection system according to claim 3 or 4. The action recognition unit
A third recognition processing unit that recognizes the behavior data of the recognition target using any one of the second to nth behavior models, and obtains a recognition result and likelihood;
A difference value calculating unit that compares the action data of the recognition target and the histogram corresponding to the action indicated by the recognition result of the third recognition processing unit, and calculates a difference value of the frequency of the movement amount per unit time; ,
The evaluation value calculation unit that calculates an evaluation value used for detecting the extraordinary behavior based on the likelihood and the difference value obtained by the third recognition processing unit. Non-daily behavior detection system.   The non-daily life according to claim 6, wherein the behavior recognition unit further includes a determination unit that accumulates the evaluation values and determines that the extraordinary behavior occurs when a cumulative evaluation value exceeds a threshold value. Behavior detection system. A stop state determination unit for determining start and stop of movement of the person from the time-series flow line information;
An area transition determination unit that determines whether or not there is an area transition of the person from the time-series flow line information;
The time-series flow line information is divided from the movement start to the movement stop, and time-series flow line information including the area transition among the divided time-series flow line information is used as the action data and the action data to be recognized. The non-daily behavior detection according to any one of claims 1 to 7, further comprising: an action data output unit for outputting, wherein an area transition history indicating the history of the area transition is used as a reference for the clustering. system.

Images