JP2010146223A - Behavior extraction system, behavior extraction method, and server - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To extract content and period of fluctuating behaviors configured of a plurality of motions from sensing data obtained by measuring the movement or state of a monitoring object by a sensor. <P>SOLUTION: The behavior extraction system is provided with a sensor mounted on a monitoring object or arranged in the periphery of the monitoring object for measuring the movement or state of the monitoring object, and a computer for analyzing sensing data measured by the sensor. The behavior extraction method for extracting a behavior of the monitoring object on the basis of the sensing data includes: causing the computer to store the sensing data; creating regions in which the stored sensing data are divided by the duration of prescribed operations; creating fragments obtained by dividing each region by the changing point of the value of the sensing data; and extracting the content and period of the behavior of the monitoring object by using a rule configured of the combination of the regions and the fragments and a sequential pattern. <P>COPYRIGHT: (C)2010,JPO&amp;INPIT

Description

The present invention relates to a method and system for analyzing time-series data measured by a sensor device, the monitoring target action contents from sensor data related to biological information, operation information, and peripheral information acquired by the sensor device, and It relates to a technique for analyzing behavior patterns.

  In recent years, a sensor network system has been studied in which a small electronic circuit having a wireless communication function is added to a sensor and various information in the real world is taken into an information processing device in real time. A wide range of applications are considered for sensor network systems. For example, biological information such as acceleration and pulse is constantly monitored by a small electronic circuit that integrates a wireless circuit, processor, sensor, and battery, and the monitoring result is diagnosed by wireless communication. Medical applications have been proposed in which a health condition is determined based on a monitor result transmitted to a device or the like. In addition, applications such as monitoring the user's biological information such as acceleration and pulse, work status, user behavior such as purchase information, and determining the user status by combining these various information are also proposed. ing.

Patent Document 1 discloses a method of enumerating human action pattern candidates for each predetermined unit time from data measured by a sensor (hereinafter, data measured by a sensor is referred to as sensing data).
Patent Document 2 discloses a method of detecting a change point of acceleration from acceleration sensing data measured from a monitoring target, estimating a behavior for each change point, and recording the detected change point.

JP 2004-287539 A JP 2003-296882 A

  In the method of Patent Document 1, a waveform of sensing data serving as a reference pattern corresponding to each behavior pattern is registered in advance, an approximation degree between the sensing data and the reference pattern for each unit time is calculated, and an action pattern having a high approximation degree. Is displayed. When the same action is a very similar time lapse and is composed of extremely equal movements, the degree of approximation of the sensing data becomes high, and the above method can estimate the action.

  In the method of Patent Document 2, the period and amplitude of the numerical time-series data of each axis of acceleration are used as feature amounts, and a point of action is extracted by detecting a point where the feature amount has changed as a change point. In order to pre-register a set of period and amplitude values for motion as motion judgment conditions for motion estimation, and to estimate the content of behavior at each change point according to the motion judgment conditions, the content of the behavior consisting of repetition of a single motion And the period can be extracted.

  However, the action is not a repetition of a single action, but a plurality of actions are combined to form one action. In addition, there are fluctuations in the behavior, and the waveform of the sensing data is not necessarily equal even in the same behavior. For example, when the monitoring target is a person, even if the action is the same, it is rare to take the same movement, and the sensing data is not equal. When the monitoring target is a device, the behavior corresponds to the operating state, but there is a behavior that dynamically changes due to an external factor, and there is a problem similar to that when the monitoring target is a person.

  Here, fluctuation means that there are a plurality of patterns in the same action depending on the duration of action, the order of actions, the magnitude of actions, and the like.

  In the above-described conventional example, it is possible to automatically identify a rough operation state such as walking, exercising, and resting for the behavior of the monitoring target wearing the sensor. However, when the monitoring target is a person, commuting, desk work, meetings, business trips, housework, watching movies, dressing, shopping, eating, etc. It is not possible to determine the action content corresponding to what was being done, such as the operating state, which is the action of the device, such as operation or low operation. The action is composed of a combination of a plurality of operation states, and each operation state is only a part of the actually performed action.

  In addition, it is not possible to cope with fluctuations in behavior in the judgment of behavior by comparison with the reference pattern of sensing data in a predetermined time interval unit, and in behavior extraction that divides the change point of sensing data as the change point of behavior, When a section appears intermittently, there is a problem that the accuracy of action determination decreases due to reasons such as that one action is divided into a plurality of actions and actions are assigned to each.

  As described above, there is a problem that the behavior to be monitored cannot be extracted by repeated detection of a single operation or comparison with a predetermined reference pattern of sensing data.

  In view of the above, an object of the present invention is to extract, from sensing data, not a monitoring target motion and exercise state but a series of behaviors such as a living behavior and an operating state break and contents.

  The following is a brief description of an outline of typical inventions disclosed in the present application.

  In other words, the behavior extraction system includes a sensor node that is attached to a monitoring target and includes a sensor that acquires sensing data and a communication unit that transmits the sensing data to the server, and a server that analyzes the sensing data. The server obtains the data storage unit for storing the sensing data, the motion frequency of the monitoring target from the sensing data, extracts a predetermined operation state and its period from the motion frequency, creates a scene area with the period as a delimiter, Based on the scene segmentation unit that creates a scene fragment with the change point of the sensing data as a delimiter for the region, and the scene determination rule recorded in the data storage unit, the action content to be monitored and its period from the scene region and the scene fragment A scene listing unit for determining

  Therefore, the present invention provides a series of actions based on a plurality of types of scene areas, scene fragment extraction that is a feature amount of each scene area, a combination of scene areas and fragments, and an order pattern from sensing data obtained by measuring the movement and state of a monitoring target. It is possible to estimate the content and period of a scene with high accuracy, and to acquire data of the scene content and period together with an index of correctness of determination.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  As a first embodiment to which the present invention is applied, there is provided a life log system that supports recording of living activities by presenting scenes extracted from sensing data measured by a sensor attached to a person as candidates for human activities. Show.

  FIG. 1 is a block diagram showing an example of a system configuration of a life log system to which the present invention is applied. An example in which the bracelet type sensor node 1 provided with an acceleration sensor is used as a sensor for detecting the operation (or state) of the user of the system and the acceleration of the arm is detected as biological information will be shown. The bracelet type sensor node 1 is mounted on a user's (or participant's) arm to detect acceleration, and wireless communication is performed on acceleration detected in a predetermined cycle (hereinafter, data measured by the sensor is referred to as sensing data). When possible, the data is transmitted to the base station 102 via the antenna 101 of the base station 102, and when wired communication is possible, the data is transmitted to the client computer (PC) 103 via a USB connection or the like.

  In FIG. 1, a base station 102 or a PC 103 communicates with a plurality of bracelet type sensor nodes 1, receives sensing data corresponding to the movement of the user from each bracelet type sensor node 1, and sends it to the server 104 via the network 105. Forward. The server 104 stores the received sensing data. The server 104 analyzes the received sensing data, and creates and stores a scene representing a series of user actions as will be described later.

  The scene data created by the server 104 can be viewed or edited by a client computer (PC) 103 operated by a user of the life log system.

  FIG. 2 is a diagram showing an example of a bracelet (or wristwatch type) sensor node 1 constituting a sensor unit of a life log system to which the present invention is applied. FIG. FIG. 2B is a cross-sectional view of the bracelet type sensor node 1 viewed from the side. This bracelet type sensor node 1 mainly measures the movement of the user (wearer). The bracelet type sensor node 1 includes a case 11 for storing a sensor and a control device, and a band 12 for attaching the case 11 to a human arm.

  Inside the case 11 is stored a substrate 10 provided with a microcomputer 3, a sensor 6 and the like as shown in FIG. As an example of the sensor 6 that measures the movement of the human body (living body), an acceleration sensor that measures three-axis accelerations in the X-Y-Z directions in the figure is employed. In the present embodiment, the bracelet type sensor node 1 includes a temperature sensor (not shown), measures the user's body temperature, and outputs it as sensing data together with the acceleration.

  FIG. 3 shows a block diagram of an electronic circuit attached to the substrate 10 of the bracelet type sensor node 11. In FIG. 3, a substrate 10 includes a wireless communication unit (RF) 2 including an antenna 5 that communicates with a base station 102, a USB communication unit 39 that is wired to the PC 103, a sensor 6, and the wireless communication unit 2. The microcomputer 3 to be controlled, the real-time clock (RTC) 4 functioning as a timer for starting the microcomputer 3 intermittently, the battery 7 for supplying power to each part, and the supply of power to the sensor 6 are controlled. A switch 8 is arranged. In addition, a bypass capacitor C1 is connected between the switch 8 and the sensor 6 to prevent unnecessary power consumption by removing noise and reducing charge / discharge speed. By controlling the switch 8 so as to reduce the number of times of charging / discharging the bypass capacitor C1, wasteful power consumption can be suppressed.

  The microcomputer 3 includes a CPU 34 that executes arithmetic processing, a ROM 33 that stores programs executed by the CPU 34, a RAM 32 that stores data and the like, and an interrupt that interrupts the CPU 34 based on a signal (timer interrupt) from the RTC 4. Serial communication for transmitting and receiving signals as serial signals between the control unit 35, the A / D converter 31 that converts the analog signal output from the sensor 6 into a digital signal, and the wireless communication unit 2 and the USB communication unit 39 An interface (SCI) 36, a wireless communication unit 2, a USB communication unit 39, a parallel interface (PIO) 37 that controls the switch 8, and an oscillation unit (OSC) 30 that supplies a clock to each unit in the microcomputer 3. Consists of including. The above-described units in the microcomputer 3 are connected via a system bus 38. The RTC 4 outputs an interrupt signal (timer interrupt) at a predetermined period set in advance to the interrupt control unit 35 of the microcomputer 3 and outputs a reference clock to the SCI 36. The PIO 37 controls ON / OFF of the switch 8 in accordance with a command from the CPU 34 and controls power supply to the sensor 6.

  The bracelet type sensor node 1 starts the microcomputer 3 at a predetermined cycle (for example, 1 minute), acquires sensing data from the sensor 6, and an identifier for identifying the bracelet type sensor node 1 in the acquired sensing data. A time stamp is given and transmitted to the base station 102 or the PC 103. Note that details of the control of the bracelet type sensor node 1 can be the same as, for example, Japanese Patent Laid-Open No. 2008-59058. The bracelet type sensor node 1 may periodically transmit the sensing data acquired to the base station 102 or the PC 103 periodically.

  FIG. 4 is a block diagram showing functional elements of a life log system to which the present invention shown in FIG. 1 is applied. Sensing data transmitted from the bracelet type sensor node 1 is accumulated in the data storage unit 500 of the server 104 via the base station 102 or the PC 103.

  The server 104 includes a processor, a memory, and a storage device (not shown), and totals the sensing data measured by the acceleration of the user's arm, and collects the data totaling unit 200, the scene dividing unit 300, the scene listing unit 400, and the action acquisition The unit 500 is executed. The data totaling unit 200 calculates a total value for each unit time (for example, one minute) and stores it in the data storage unit 600. The scene division unit 300 analyzes the sensing data and the aggregated data, extracts scene regions that are individual operation states, and scene fragments that are individual motion states, and stores them in the data storage unit 600. The scene listing unit 400 extracts a scene that is a candidate for action content by applying a rule created from past data or a pre-registered scene determination rule to the extracted scene region and scene fragment, Store in the data storage unit 600. The action acquisition unit 500 presents the extracted scene to the user's client computer 103, and stores the scene selected by the user in the data storage unit 500 together with the action content input by the user.

  The identifier of the bracelet type sensor node 1 is given to the sensing data stored in the data storage unit 600, and the identifier for identifying the user (behavior content, operation state, scene, and behavior record data) For example, an identifier of the bracelet type sensor node 1) is given to identify the user.

  The scene division unit 300, the scene listing unit 400, and the action acquisition unit 500 are stored in a storage device (storage medium) as a program, for example, loaded into the memory at a predetermined timing, and executed by the processor. In the following, the server 104 executes the scene dividing unit 300 and the scene listing unit 400 every predetermined cycle (for example, 5 minutes), and the action acquisition unit 500 is executed at a timing requested by the user. An example is shown.

  FIG. 5 is a diagram showing an overall flow of data processing performed in the life log system to which the present invention is applied. First, in step S <b> 1, the sensing data transmitted from the bracelet type sensor node 1 is transferred to the server 104 by the base station 102 or the PC 103, and the sensing data is accumulated in the data storage unit 600 of the server 104. Further, when the server 104 reaches a predetermined cycle, the data totaling unit 200 is executed, and the exercise frequency per unit time is calculated from the sensing data accumulated in the data storage unit 600 and stored in the data storage unit 600. As shown in FIG. 6, the calculation result of the exercise frequency is data obtained by sorting the exercise frequency for each unit time in time series. The method of calculating the exercise frequency will be described later with reference to FIGS.

  Next, in step S2, when the server 104 reaches a predetermined cycle, the scene division unit 300 is executed, and a scene region that is a series of operation states of the user from the sensing data and the aggregated data accumulated in the data storage unit 600, And a scene fragment which is a motion constituting a scene area is created.

  This scene area creation process detects a predetermined operation state (for example, walking, sleep) that can be extracted from sensing data or aggregated data, and detects the detected operation state, and the detected operation state and the next operation. The period between the states is extracted as a scene area, and the operation contents of sleep, rest, light work, work, walking, and exercise are determined using a predetermined determination rule described later, and assigned to each scene area.

  Further, the scene fragment creation process detects, for each of the extracted scene areas, a time-series data change point from the sensing data or the total data of the sensing data in a predetermined time unit, and the change point is changed to the next change point. Is extracted as a scene fragment which is one motion unit or state unit. The change point of the sensing data or the total data is extracted as a change point when it becomes quiet from the state of intense movement. The scene fragment is assigned a fragment classification according to a method for extracting scene fragments and a method for grouping scene fragments. The scene fragment extraction method is a type of sensing data used for scene fragment extraction. The scene fragment grouping method is the content of the sensing data used to assign the segment classification value, which is the group identifier, to the scene fragment. The grouping using the momentum statistics and the similarity of the sensing data are used. The grouping includes one or more groupings such as grouping using the similarity of the aggregated data. Then, for each fragment classification, a fragment classification value as a group identifier is assigned by the fragment classification grouping method.

  Thus, the sensing data used for scene fragment extraction and the sensing data used for grouping the scene fragments are not only the same type but also different types. For example, when a person goes from indoors to outdoors, a case is assumed in which a change in temperature is significant compared to a change in acceleration. In such cases, it is more accurate to group the scene fragments extracted from the acceleration data using temperature similarity than grouping them using acceleration similarity. It becomes the index shown in. Therefore, it is possible to improve the determination accuracy by calculating data grouped using the temperature similarity and determining the action content based on a scene determination rule described later.

  Through the above processing, the scene dividing unit 300 creates a scene area and a scene fragment at predetermined intervals and stores them in the data storage unit 600.

  Next, if the scene area and scene fragment extracted by the scene dividing unit 300 in step S3 and the action contents before and after the server 104 have already been confirmed, the server 104 uses the action contents as registered scene determination rules. And a scene that is a candidate for the user's action content is created and stored in the data storage unit 600.

  The registered scene determination rule is a rule extracted from past action content, scene area, and scene fragment data for each user, or action content, scene area, scene stored in the data storage unit 600 including the user. A rule created from fragment data or a scene determination rule constructed using known knowledge. Specifically, it is a scene determination rule for determining a scene that is a candidate for action content created from a combination of action content, scene area, scene fragment, time information and an order pattern. An example of the structure of the scene determination rule will be described later with reference to FIG.

  For example, if the action content is early morning walking, the specific action content candidate can be a walk, or “walking (10-15 minutes) and resting (20-15 minutes after waking up) (20 -25 minutes) and walking (7-10 minutes), it is determined as “commuting” which is standardized in the normal life of the user. Since the action content is a combination of action contents, the action contents are often composed of a plurality of scene areas, but there are action contents determined from a single action content and time as in the above-described walk.

  Next, the action acquisition unit 500 assigns priorities to be presented from the scenes stored in the data storage unit in descending order of possibility of matching with the contents of actions performed by the user (step S4).

  Next, in step S5, the server 104 presents each scene to the client computer 103 according to the priority order. In step S6, the user who operates the client computer 103 confirms the action content corresponding to the scene extracted by the server 104, and selects the scenes presented in order of priority, so that the user can easily perform every day. Life log can be generated.

  Next, in step S7, the scene selected by the client computer 103 is confirmed as an action record.

  The life log created in this way is stored in the data storage unit 600 of the server 104 together with a user identifier and a time stamp such as the creation date and time in step S8.

  FIG. 7 is a flowchart illustrating an example of processing performed by the data totaling unit 200 of the server 104. First, the data totaling unit 200 reads sensing data corresponding to a user identifier from the data storage unit 600 (step S11). Here, the amount of sensing data read by the data totaling unit 200 may be set to a predetermined cycle (for example, 3 minutes) that is a totaling cycle of sensing data.

  Next, in step S12, the total value for every predetermined time interval (for example, 1 minute) is calculated about the acceleration data of the sensing data read by the data totaling unit 200. In the present embodiment, the number of zero crosses indicating the frequency of exercise of the wearer (user) of the bracelet type sensor node 1 within a predetermined time interval is used as the total value.

  Since the sensing data detected by the bracelet type sensor node 1 includes the X, Y, and Z axis acceleration data, the data totaling unit 200 calculates the scalar amount of the X, Y, and Z axis acceleration. , A value where the scalar amount passes 0 or a predetermined value close to 0 is calculated as the number of zero crossings, the frequency at which the number of zero crossings appears within a predetermined time interval is calculated, and this frequency of occurrence is calculated for a predetermined time interval (1 minute) Is output as the exercise frequency.

First, the scalar quantity is obtained by scalar quantity = (Xg ² + Yg ² + Zg ² ) ^1/2 where Xg, Yg, and Zg are accelerations of the respective axes.

  Next, the obtained scalar quantity is subjected to filter (bandpass filter) processing, and only a predetermined frequency band (for example, 0.1 Hz to 5 Hz) is extracted to remove a noise component. Then, as shown in FIG. 8, the number of times that the scalar amount of the acceleration subjected to the filter processing becomes a predetermined threshold (for example, 0.05 G) is calculated as the number of zero crossings. Alternatively, the number of times that the scalar amount of acceleration crosses a predetermined threshold is defined as the number of zero crossings. Then, the number of zero crossings within a predetermined time interval is obtained as the exercise frequency. Also, an integral value of the momentum within a predetermined time interval is obtained from the number of zero crossings and a scalar amount, and this integral value is used as the exercise intensity. Also, an average temperature within a predetermined time interval is obtained for the temperature included in the sensing data. The motion frequency may be summed up by counting the number of times that the acceleration value in each direction of XYZ vibrates positively and negatively (frequency) within a predetermined time in each direction. The method for calculating the number of zero crossings has been described.

  The data totaling unit 200 obtains the exercise frequency, average temperature, and exercise intensity for each predetermined time interval, generates them as total data for each predetermined time interval, as shown in FIG. accumulate. FIG. 9 is an explanatory diagram showing a total data format 700 for each predetermined time interval. The sensor data ID 702 for storing the identifier of the bracelet type sensor node 1 included in the sensing data and the wearer (life of the bracelet type sensor node 1) A user ID 701 for storing an identifier of a log system user), a measurement date 703 for storing a start time (measurement date) of a predetermined time interval, a temperature 704 for storing a temperature included in sensing data, and a data totaling unit One entry is made up of an exercise frequency 705 for storing the exercise frequency calculated by 200 and an exercise intensity 706 for storing the exercise intensity obtained by the data totaling unit 200. The user identifier may be referred to from a table (not shown) set in advance based on the identifier of the bracelet type sensor node 1.

  FIG. 10 is a flowchart showing a scene area and scene fragment creation process performed by the scene division unit 300 of the server 104. First, the scene dividing unit 300 reads sensing data and total data corresponding to a user identifier from the data storage unit 600 (step S21). Here, the amount of total data read by the scene dividing unit 300 may be set to a predetermined cycle (for example, 5 minutes) that is a cycle of scene area creation.

  Next, in step S22, a predetermined operation state is detected from the sensing data and total data read by the scene dividing unit 300, and a scene area is created. Here, the predetermined operation state is an operation state composed of a repetition of a single operation that can be detected from sensing data or aggregated data, a continuation of the same state, and a predetermined sensing data pattern, for example, walking and sleep.

  Next, in step S23, for each of the extracted scene areas, a time-series change point of the exercise frequency is detected from the total data corresponding to the start date and time and end date and time of the scene area, Are extracted as scene fragments, which are units of the same exercise, and stored in the data storage unit 600.

  Through the above processing, the scene dividing unit 300 extracts a scene area which is a unit of motion state and a scene fragment which is a unit of motion change from the sensing data and the total data.

  FIG. 11 is a flowchart showing a scene area creation process performed by the scene division unit 300.

  First, in step S31, a predetermined operation state is detected from the sensing data read from the data storage unit 600 and the total data, and set as a delimiter scene. The detected segmented scene uses the contents of the operation state as the area contents, and a set of the start date / time, end date / time, and area contents is stored in the data storage unit 600 as the scene area. Here, the method of detecting walking is that the acceleration in the vertical direction changes periodically (landing for each step), and the acceleration in the front-rear direction regularly repeats the forward and backward directions in synchronization with the acceleration in the vertical direction. Waveforms such as (velocity change at each landing), and the acceleration in the left and right direction are repeated in synchronization with the acceleration in the up and down direction (swing of the body to the left and right at each step) can be observed, and the swing of the arm overlaps Therefore, it can be determined whether or not the corresponding scene is in a walking state. It is also possible to detect the reciprocal of the zero-cross cycle as the number of steps. A known method may be used as a method for detecting the walking state from the acceleration sensor attached to the human body.

  In addition, although the frequency of exercise is very low in sleep, the human body does exercise such as turning over during sleep, so the exercise frequency does not become zero. Several methods for determining sleep are known, and a known method may be used.

  An example of a scene area based on a predetermined sensing data pattern is a rapid temperature change. A sudden temperature change is regarded as a change in the surrounding environment to be monitored, and can be a scene segmented scene. A sudden temperature change may be detected by setting a sensing data pattern when a difference between the maximum temperature and the minimum temperature in a predetermined period (for example, 10 minutes) exceeds a predetermined value.

  Next, in step 32, each period not corresponding to the above-mentioned segmented scene is set as a scene area, the table of determination values shown in FIG. 12 is referred to, and the motion frequency of each scene area is “rest”, “light work”, Compared with each determination value of “work” and “exercise”, it is determined which corresponds.

  Then, the scene division unit 300 sets the determination result as the region content that is the operation state of the scene region, and stores it in the data storage unit 600 together with the start date and time and the end date and time (step 33). Note that the table storing the determination values for determining the operation state in FIG. 12 is set in advance.

  FIG. 13 is an explanatory diagram showing a format 800 of scene area data stored in the data storage unit 600. The user ID 801 stores a user identifier, the scene area data ID 802 is an identifier of the scene area data, and the scene area. The entry date and time 803, the end date and time 804, and the area contents 805 indicating the operation state of the scene area constitute one entry.

  Through the above processing, the scene dividing unit 300 extracts a scene area.

  FIG. 14 is a flowchart showing scene fragment creation processing performed by the scene division unit 300.

  First, in step S41, a time-series change is detected from sensing data and total data corresponding to the start date / time and end date / time of each scene area, and extracted as a change point. Here, the time-series change means a case where the total data for each predetermined time interval is compared with the time intervals before and after the difference between the total data exceeds a predetermined value. Further, the average value of the total data of the scene area may be calculated, and the point where the total data value crosses the average value may be detected as the change point. In addition, the aggregated data may be tested for the difference between the average values for the values of a predetermined period before and after, and a point where a significant difference is recognized may be detected as a change point. In the present embodiment, temperature, exercise frequency, and exercise intensity may be used as total data.

  Next, in step S42, each period divided by the detected change point is set as a scene fragment, and a fragment classification as an identifier for grouping is given to each scene fragment. The scene fragments are grouped according to the similarity of the sensing data, the similarity of the aggregated data, the statistical value of the exercise frequency of the aggregated data, the statistical value of the exercise intensity, and the like, and a fragment classification as a group identifier is given. Here, the similarity between the sensing data and the aggregated data can use the correlation value with the sensing data set as a reference for each group, and the identifier of the group with the highest correlation value can be used as the fragment classification. Further, the statistical value of the exercise frequency of the total data may be grouped by range division using the average and variance of the exercise frequency corresponding to the scene fragment. The exercise intensity may be grouped similarly to the exercise frequency. A plurality of fragment classifications may be assigned to each scene fragment. Further, the aggregate data used for dividing and the aggregate data used for grouping may be different aggregate data.

  Then, the scene division unit 300 stores the set of the start date / time, the end date / time, and the fragment classification for the extracted scene fragment in the data storage unit 600 (step S43). FIG. 15 is an explanatory diagram showing a format 900 of scene fragment data stored in the data storage unit 600. A user ID 901 that is a user identifier, a scene area data ID 902 that is an identifier of corresponding scene area data, and a scene One entry is formed from the start date and time 903 of the fragment data, the end date and time 904, and the fragment classification 1 to fragment classification n (906) which are identifiers of the assigned group.

  Through the above processing, the scene dividing unit 300 extracts scene fragments.

  FIG. 16 is a flowchart showing a scene creation process performed by the scene listing unit 400. The scene processing unit searches for scene areas, scene fragments, and action recording patterns that match the scene determination rule, and extracts a scene.

  First, in step S51, a pre-registered scene determination rule corresponding to the user is read from the data storage unit 600. The scene determination rule is composed of a condition and a conclusion. When a certain condition is satisfied, a certain conclusion (scene, that is, action content) is estimated, and is recorded in advance for each user. The condition is composed of a plurality or a single scene area, a combination of action records, or an order pattern. A scene area is not only the contents of the area but also a combination of scene fragments in the period of the scene area. In addition, the scene area, scene fragment, and action record are also conditional on the elapsed time from the start date to the end date. The conclusion is composed of the scene content, start date and time, and end date and time, and is accompanied by a certainty factor with a value indicating determination accuracy as an evaluation value. The certainty factor can be calculated from past data, and may be arbitrarily determined when a scene determination rule is created based on expert knowledge.

  FIG. 17 shows an example of the structure of the scene determination rule. The scene determination rule includes a condition table 1000 and a conclusion table 1100, and is associated with a rule ID. The condition table includes a set of user ID, rule ID, order, type, item name, value, and comparison operation. The type includes a scene area, a scene fragment, an action record, and an elapsed time, a time zone, and a time as time data. The comparison calculation is an arithmetic expression for determining whether or not the scene area or action record to be determined matches. The conclusion table includes a set of user ID, rule ID, scene content, start date / time, end date / time, and certainty factor. No specific value is defined for the start date / time and end date / time, and the start date / time and end date / time are determined from the data that satisfies the conditions in the condition table by specifying the start date / time or end date / time of the data shown in the order of the condition table. Can do. Also, by specifying * (asterisk) as the user ID, it is possible to make a scene determination rule applicable not only to a specific user but also to everyone.

  For example, the scene determination rule of the rule ID 1 of the user ID 101 is “the first is“ sleep ”of the action record, and then there is a“ walking ”scene area within the morning time zone within 70 minutes, and then“ rest ”. If it is a scene area, and finally a `` walking '' scene area, the scene is `` commuting '', the start date and time is the start date and time of the first walk, the end date and time is the end date and time of the last walk, and the certainty is `` 80% "."

  Further, the scene determination rule of the rule ID 2 of the user ID 105 is “a scene area of light work, a scene fragment in which the value of fragment classification 1 is 5 and a scene fragment in which the value of fragment classification 3 is 2. If the time from the start date / time to the end date / time of the scene area is 30 minutes or more, it indicates that the scene is “meeting” and the certainty level is “60%”.

  Next, in step S52, for each of the read scene determination rules, a scene table, scene fragment, and action record input by the user stored in the data storage unit 600 are used. Those that satisfy all of the above conditions are extracted, the contents of the conclusion table are extracted as scenes, and stored as scenes in the data storage unit 600 (step S53). FIG. 18 is an explanatory diagram showing the format 1100 of the scene data. One of the user ID 1101 as the user identifier, the scene content 1102 as the action content of the scene, the start date and time 1103 of the scene data, and the end date and time 1104 is shown. Configure the entry.

  Next, in step S54, when the user inputs an action record after the time when the extracted scene is stored in the data storage unit 600, the data storage of the scene satisfying the scene determination rule is repeated by repeating step S51 and the subsequent steps. The storage in the unit 600 is repeated. When a new action record is input, the number of scene determination rules that satisfy the condition increases, a scene to be stored in the data storage unit 600 is added, and a scene with high certainty may be stored.

  Through the above processing, a scene to be presented as a candidate for action content is extracted.

  FIG. 19 shows a process in which, when the client computer 103 accepts an action history input request from a user, the action acquisition unit 500 reads the scenes stored in the data storage unit 600 and presents them to the user with priorities. It is a flowchart showing.

  First, in step S <b> 61, the client computer 103 obtains a period for presenting action content candidates input by the user via the input unit 1032.

  Next, in step S62, a scene that partially overlaps the period is read from the data storage unit 600. Here, a scene whose end date / time is after the start date / time of the period and whose start date / time is before the end date / time of the period is read. When a plurality of scenes are read out, they are sorted according to the certainty value.

  Next, in step S63, the read scene is presented as a candidate on the display unit 1031 of the client computer 103.

  Next, in step S 64, the action content, start date / time, and end date / time selected by the user using the input unit 1032 of the client computer 103 are acquired, and stored in the data storage unit 600 of the server 104 as action records.

  Through the above processing, the action content candidate created from the sensing data is presented, and the user can acquire the action record by selecting the presented candidate.

  FIG. 20 is a screen image of the action history input screen 1200 displayed on the display unit 1031 of the client computer 103. The server 104 receives a user ID or the like from the client computer 103 and displays the specified user total data 550, the scene area data 800, and the scene on the action history input screen 1200. Note that a browser can be adopted as an application running on the client computer 103.

  The action history input screen 1200 includes an exercise frequency 705 of the aggregated data 700, an exercise frequency 1201 that displays the position of the measurement date 703 as a bar graph, an operation content 1202 that displays the area content stored in the scene area data 800, and a scene area A time display 1203 for displaying the start date and time 803 and end date and time 804 of the data 800; an action content 1204 for inputting or displaying an action record; a date and time pull-down menu 1205 for setting the date and time for inputting an action record; and a plurality of scene areas A scene combination button 1206 for transmitting an instruction to manually combine the actions to the server 104, an action contents input button 1207 for inputting action contents 1203 designated by the user with a mouse cursor or the like, and an input for instructing completion of the input And an end button 1208. The illustrated action history input screen 1200 shows a state where the input of the action content 1204 has been completed.

  When the user operating the client computer 103 selects the action content input button 1207 on the action history input screen 1200 and then selects the action content 1204, the action content candidates obtained by the action acquisition unit 500 are displayed. The user selects an action content candidate by operating a mouse or the like that constitutes the input unit 1032 of the client computer 103, or manually inputs the action content if the action content candidate does not have a desired item. Can do.

  When the user selects the action content 1204, the server 104 displays the action content candidates estimated by the action content analysis unit 300 for each entry of the scene data 1100 on the action content 704. For example, when the action content 1204 corresponding to the “light work” scene from time 9:40 in FIG. 20 is selected, a candidate box 1500 including action content candidates is displayed as shown in FIG. The candidate box 1500 includes an area for displaying the action content candidate 1501 estimated by the action acquisition unit 500, and the user can input details of the action content by selecting the displayed item.

  When the user selects a candidate presented on the display unit 1031 of the client computer 103, the action history of the selected scene area data 800 is fixed, and the server 104 generates an action record and stores it in the data storage unit 600.

  Further, the action acquisition unit 500 of the server 104 updates the certainty of the scene determination rule in which the condition of the scene determination rule for extracting the candidate for the action content selected by the user matches the condition in the condition table.

  Here, the certainty factor is updated based on the condition table, the conclusion table, the scene area data and scene fragment data obtained from the sensing data, and the action content selected by the user. Specifically, among the combinations of a plurality of scene areas and scene fragments, those satisfying the condition table are extracted and the number of cases is calculated. And the scene of the conclusion table corresponding to the conditions of the said condition table is extracted, what matches the action content actually selected by the user is extracted, and the number of cases is calculated. Then, the certainty factor can be updated by calculating the ratio of the number of cases matching the action content selected by the user to the number of cases satisfying the condition table.

  In the scene determination rule table of FIG. 17, the number of cases satisfying the conditions in the condition table and the number of cases satisfying the contents of the conclusion table in addition to the conditions are stored, so that the search for the number of cases is omitted and stored. The certainty factor may be calculated by adding the number of cases.

  In this way, by updating the certainty factor, the accuracy of the certainty factor can be increased, a scene determination rule reflecting the user's preference can be created, and accurate action content candidates can be extracted. become able to.

  In the present invention, the user's exercise state is measured by the acceleration sensor of the bracelet type sensor node 1 and stored in the server 104, the measured exercise state is analyzed, and a scene showing the user's life behavior is started together with the action content. Automatically extract time and end time. Human behavior fluctuates greatly, sensing data does not match even with the same behavior content, and the behavior content cannot be estimated based on the similarity of the sensing data. The present invention creates a scene area that is a unit of a series of operation states from sensing data, a scene fragment that is a short-time motion state that constitutes the scene area, and configures action content by a combination of scene areas and an order pattern, Similarity is determined not only by the operation state of the scene area but also by a combination of scene fragments. As a result, it is possible to extract the period and content of a series of actions corresponding to the fluctuation of the action, not the state of motion or movement.

  In the above-described embodiment, an example in which the acceleration sensor of the bracelet type sensor node 1 is used to detect the movement state of the user (human body) as the life log system is shown. However, the movement state of the human body can be detected. Any biological information may be used. For example, a pulse or the number of steps can be used, or the motion state of the human body may be detected from a combination of the plurality of biological information. Further, not only the biological information but also the position of the human body may be used using position information such as GPS or a portable terminal.

  In the life log system, a monitoring target is a person and a scene corresponds to an action, but the monitoring target is not limited to a person, and may be a device. When the monitoring target is a device, it is possible to extract a scene indicating an operation state or operation content as a scene from sensing data (for example, vibration, volume) indicating the state of the device measured by a sensor attached to the device. Yes, it is possible to acquire the operating status history of the device.

  FIG. 22 shows a first modification, and shows the action contents 1203 and 1204 of the action history input screen 1200 of the above embodiment. In the action contents 1203 and 1204 in FIG. 22, a scene having a certainty degree of 80% or more is displayed during a period in which no action record is input for the day selected by the user. When a plurality of scenes are searched, the scene with the highest certainty factor is displayed. FIG. 22 (a) shows the action contents at the stage where no action record is input, and shows that the scene with a certainty degree of 80% or more is only the dress condition from 7:00 to 8:00. Next, FIG. 22B shows the action content at the stage where the candidate presented by the user is confirmed, and “dressing” is input as the action record. By storing “dress” as an action record, a scene satisfying the conditions of the scene determination rule and having a certainty degree of 80% or more is selected, and a new action from 8:00 to 9:30 is selected. This indicates that “commuting” is displayed as a content candidate. As described above, when the action record is confirmed, a scene with a high degree of certainty that satisfies the scene determination rule is detected, and the action content candidate is presented even during a period in which the action content candidate could not be presented. Is possible.

  FIG. 23 shows a second modification in which a part of the system configuration shown in FIG. 4 of the above embodiment is changed, and other configurations are the same as those of the above embodiment. Instead of the server 104 of the embodiment, the client computer 103 includes a data collection unit 200, a scene division unit 300, a scene listing unit 400, an action acquisition unit 500, and a data storage unit. , Connected to the base station 102. The configurations of the data collection unit 200, the scene division unit 300, the scene listing unit 400, the action acquisition unit 500, and the data collection unit 600 are the same as those in the above embodiment. The client computer 103 is connected to the server 104 via the network 105. The server 104 includes a data storage unit 1300 that receives and stores an action record generated by the client computer 103, and an analysis unit 1400 that performs a predetermined analysis on the action record.

  In the case of the configuration of FIG. 4, data is stored in the server. Therefore, when a network connection is possible, data can be sent regardless of the location. However, when network connection is not possible, there is a possibility that data transmission or behavior input cannot be performed. On the other hand, in the configuration of FIG. 23, a dedicated computer such as the PC 103 is required for data transmission and action input, so there are restrictions on the location, but data is sent to the PC 103 and action input is performed on the PC 103. There is an advantage that there is no need to worry about the presence or absence of network connection.

  As described above, the present invention can extract the period and content of the behavior to be monitored from the sensing data, and can be applied to a computer system that automatically creates a human behavior history and a device operating status history. It is. In particular, it can be applied to a sensor network system capable of transmitting sensing information to be monitored to a server by wireless communication.

It is a block diagram which shows an example of the system configuration | structure of the life log system to which this invention is applied. It is a figure which shows an example of a bracelet type sensor node, (a) is the schematic seen from the front of the bracelet type sensor node 1, (b) is sectional drawing which looked at the bracelet type sensor node from the side. It is an example of the block diagram of the electronic circuit attached to the base of a bracelet type sensor node. It is an example of the block diagram which shows the functional element of the life log system to which this invention is applied. It is an example of the figure which shows the whole flow of the data processing performed with the life log system to which this invention is applied. It is an example of the graph which sorted the exercise frequency for every unit time in time series. It is a flowchart which shows an example of a process of a data total part. It is a graph which shows the relationship between acceleration and time, and is an example of determination of the number of zero crossings. It is an example of the format of the total data for every predetermined time interval. It is an example of the flowchart which shows the process of a scene division part. It is an example of the flowchart which shows a scene area | region creation process. It is an example of the table of the judgment value which sets the relationship between exercise frequency and an operation state. It is an example of the format of scene area data. It is an example of the flowchart which shows a scene fragment | piece creation process. It is an example of the format of scene fragment data. It is an example of the flowchart which shows the process of a scene enumeration part. It is an example of the table of a scene determination rule. It is an example of the format of scene data. It is an example of the flowchart which shows the process of an action acquisition part. It is an example of the screen image of the action history input screen displayed on the display part of a client computer. It is an example of the screen image of the candidate box containing the candidate of action content. It is an example which shows a 1st modification and shows the update of the action content candidate presentation content of an action history input screen. It is an example of the block diagram which shows a 2nd modification and shows the functional element of a life log system.

Explanation of symbols

1 bracelet type sensor node 102 base station 103 client computer 104 server 200 data totaling unit 300 scene dividing unit 400 scene listing unit 500 action acquisition unit 600 data storage unit 1031 display unit 1032 input unit

Claims (13)

A sensor node that is mounted on a monitoring target and has a sensor that acquires sensing data and a communication unit that transmits the sensing data to a server; and a server that analyzes the sensing data,
The server
A data storage unit for accumulating the sensing data;
The motion frequency of the monitoring target is obtained from the sensing data, a predetermined motion state and its period are extracted from the motion frequency to create a scene region with the period as a break, and the change point of the sensing data for the scene region A scene division unit for creating a scene fragment with
An action extraction system comprising: a scene enumeration unit that determines an action content and a period of the monitoring target from the scene area and the scene fragment based on a scene determination rule stored in the data storage unit. The behavior extraction system according to claim 1,
The scene determination rule is an action extraction system including a scene area, a scene fragment, an order pattern thereof, and an index indicating correctness of the determination. The behavior extraction system according to claim 2,
The scene enumeration unit gives priority to the plurality of action contents according to an index indicating the correctness of the determination,
The behavior extraction system which further has the action acquisition part which displays the said action content on the display part connected according to the said priority. The behavior extraction system according to claim 3,
When the action content displayed on the display unit is selected, the action acquisition unit determines the correctness of the determination based on the selection result, the scene area, the scene fragment, and the scene determination rule. An action extraction system that updates the indicated index. The behavior extraction system according to claim 1,
The behavior extraction system, wherein the scene division unit determines the similarity of the scene fragment, classifies the scene fragment based on the similarity, and assigns an identifier for each classification. The behavior extraction system according to claim 5,
The behavior extraction system in which the scene dividing unit determines the similarity using sensing data different from the sensing data used for creating the scene area. A communication unit that receives sensing data acquired by the sensor node attached to the monitoring target; and
A data storage unit for accumulating the sensing data;
The motion frequency of the monitoring target is obtained from the sensing data, a predetermined motion state and its period are extracted from the motion frequency to create a scene region with the period as a break, and the change point of the sensing data for the scene region A scene division unit for creating a scene fragment with
A server having a scene listing unit that determines the action content and the period of the monitoring target from the scene area and the scene fragment based on the scene determination rule stored in the data storage unit; The server according to claim 7,
The scene determination rule is a server composed of a scene area, a scene fragment, an order pattern thereof, and an index indicating the correctness of the determination. The server according to claim 8, wherein
The scene enumeration unit gives priority to the plurality of action contents according to an index indicating the correctness of the determination,
The server which further has an action acquisition part which displays the above-mentioned action contents on a display part connected according to the above-mentioned priority. The server according to claim 9, wherein
When the action content displayed on the display unit is selected, the action acquisition unit determines the correctness of the determination based on the selection result, the scene area, the scene fragment, and the scene determination rule. A server that updates the indicated index. The server according to claim 7,
The server that determines the similarity of the scene fragment, classifies the scene fragment based on the similarity, and assigns an identifier for each classification. The server according to claim 11,
The scene division unit is a server that determines the similarity by using sensing data different from the sensing data used for creating the scene area. A behavior extraction method executed by a system including a sensor node that is attached to a monitoring target and acquires sensing data, and a server that analyzes the sensing data,
The sensor node transmits the sensing data to the server,
The server obtains an exercise frequency of the monitoring target from the sensing data, extracts a predetermined operation state and its period from the exercise frequency, creates a scene area with the period as a delimiter, and performs the sensing for the scene area Create a scene fragment with the data change point as a delimiter, and extract the behavior to be monitored and its duration from the scene area and the scene fragment based on the scene determination rule stored in the server Method.

Images