JP2010146221A - Behavior recording input support system, behavior input support method, and server - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To reduce the burden at the time of inputting a behavior record by presenting behavior candidates using the motions and states of a user measured by a sensor. <P>SOLUTION: The period in which the same behavior is repeated and the period in which the same state is continued are extracted from sensing data obtained by measuring the states and motions of a monitor object by a sensor node mounted on the monitor object. The scene region data divided by the periods is displayed. When the displayed scene region data is selected to decide content of the behavior of the user, thereby reducing the burden on the user who inputs behavior. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention collects a plurality of types of behavior histories by a device that measures the behavior content of the monitoring target from the sensor data related to the biological information, operation information, and peripheral information of the monitoring target, and inputs the behavior record of the user from the behavior history Related to supporting technology.

  In recent years, it has been expected to obtain new knowledge and provide services by recording and accumulating a large amount of human behavior and analyzing the large-scale data. In the Internet world, a mechanism has already been established that uses search keywords and purchase histories to send recommended advertisements for products that individuals may want.

  A similar mechanism can be considered in the real world. For example, to improve the work efficiency of the organization by recording and analyzing daily work contents, or to evaluate the regularity of meals, exercise, and life, and to prevent lifestyle-related diseases by attaching daily life records Such as providing recommended health care services, analyzing the life records and purchase histories of many people, and recommending advertisements for products that those people often buy to people living in specific lifestyle patterns Application is conceivable.

  On the other hand, a network system that takes various information in the real world into an information processing apparatus in real time has been studied. This network system is considered to have a wide range of applications, such as monitoring the user's biological information such as acceleration and pulse, work status, purchasing information, etc., and combining these various information, Applications such as determining the situation have also been proposed.

  Also, a technique is known in which sensing information at a certain time and user status are linked by user input, the user status is estimated using the linked data, and listed in descending order of confidence. (For example, refer to Patent Document 1).

  In addition, a technique for changing the display mode according to the arrival probability at the time of route search such as car navigation is known (for example, see Patent Document 2). In Patent Document 2, the arrival probability is calculated based on the past movement history, and the route is color-coded according to the arrival probability and displayed.

  Also, by predicting the destination according to the history of the situation and the current situation, and obtaining the predicted destination surrounding information in advance, the predicted destination surrounding information is displayed in order of the probability. A technique for displaying on a screen by changing the bold or color is known (for example, see Patent Document 3).

  In addition to this, a technique (for example, refer to Patent Document 4) for inferring an action between a comment input at the start of an action entered by a user and an action between comments at the end of the action or at the start of the next action (see Patent Document 4, for example) A technique for imaging an action is known (see, for example, Patent Document 5).

JP 2004-287539 A JP 2008-70377 A JP 2005-30982 A JP-A-2005-62963 JP-A-2005-122291

  It can be expected that many useful services can be provided for recording and analyzing daily behavior, but accurately recording daily behavior including time of day is a task that imposes a considerable burden on the user. Become. Therefore, for example, a method of automatically acquiring an action using a sensor attached to the body leads to a considerable labor reduction.

  However, the action is not a repetition of a single action, but a plurality of actions are combined to form one action. In addition, there are fluctuations in the behavior, and the waveform of the sensing data is not necessarily equal even in the same behavior. For example, when the monitoring target is a person, even if the action is the same, it is rare to take the same movement, and the sensing data is not equal. When the monitoring target is a device, the behavior corresponds to the operating state, but there is a behavior that dynamically changes due to an external factor, and there is a problem similar to that when the monitoring target is a person. Here, fluctuation means that there are a plurality of patterns in the same action depending on the duration of action, the order of actions, the magnitude of actions, and the like.

  In the above-described conventional example, it is possible to automatically identify rough actions such as walking, exercising, and resting for the behavior of the monitoring target wearing the sensor node. However, when the monitoring target is a person, commuting, desk work, meetings, business trips, housework, watching movies, dressing, shopping, eating, etc. It is not possible to determine the action content corresponding to what was being done, such as the operating state, which is the action of the device, such as operation or low operation. The action is composed of a combination of a plurality of operation states, and each operation state is only a part of the actually performed action.

  In addition, as in the conventional example, the behavior determination by comparing with the reference pattern of the sensing data in a predetermined time interval unit cannot cope with the behavior fluctuation, and the behavior extraction that separates the change point of the sensing data as the behavior change point Then, when the same action section appears intermittently, there is a problem that the action determination accuracy decreases due to the fact that one action is divided into a plurality of actions and actions are assigned to each. .

  As described above, as in the above-described conventional example, it is not possible to extract a behavior to be monitored by repeated detection of a single operation or comparison with a predetermined reference pattern of sensing data. Therefore, for these specific actions, the user needs to input action contents one by one, and there is a problem that a great deal of labor is required for the user to input the contents for all actions.

  Therefore, the present invention has been made in view of the above problems, and uses a combination of various information representing behavior such as a behavior pattern of a monitoring target determined from sensor measurement data and a usage status of a computer, and the like. The purpose is to estimate the behavior pattern and simplify the input of the user's behavior content.

  The following is a brief description of an outline of typical inventions disclosed in the present application.

  That is, it is an action record input support system including a sensor node, a base station, and a server. The sensor node is attached to a monitoring target and includes a sensor and a communication unit that transmits sensing data acquired by the sensor to a server. The base station is connected to the server via the network and transmits received sensing data to the server. The server operates based on a data storage unit that stores sensing data, a scene division unit that detects an operation state of a monitoring target from sensing data stored in the data storage unit, and a scene determination rule stored in the data storage unit A scene enumeration unit that calculates a plurality of behavior content candidates to be monitored from the state and a plurality of behavior content candidates displayed on a connected display unit, and a specific behavior content is selected from the displayed plurality of behavior content candidates Then, it has an action acquisition unit that determines the action content to be monitored and stores it in the data storage unit.

  The effects obtained by the representative ones of the inventions disclosed in the present application will be briefly described as follows.

  That is, therefore, the present invention simplifies the input of the action content for the action pattern of the monitoring target by estimating a plurality of action contents from the operation state of the monitoring target and presenting the action content candidates in each time zone. It is possible to reduce the labor required to create an action history.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

<First Embodiment>
As a first embodiment to which the present invention is applied, a life log system that supports recording of living behavior by presenting a scene extracted from sensing data measured by a sensor attached to a human as a candidate for human living behavior. Show.

  FIG. 1 is a block diagram showing an example of a configuration of a life log system to which the present invention is applied. The life log system of the present invention shows an example in which the bracelet type sensor node 1 having an acceleration sensor is used as a sensor for detecting the operation (or state) of the user of the system and the acceleration of the arm is detected as biological information. . The bracelet type sensor node 1 is mounted on a user's (or participant's) arm to detect acceleration, and wirelessly communicates acceleration data detected in a predetermined cycle (hereinafter, data acquired by the sensor is referred to as sensing data). Is transmitted to the base station 102 via the antenna 101 of the base station 102, and is transmitted to the client computer (PC) 103 via a USB connection or the like when wired communication is possible.

  In FIG. 1, a base station 102 and a PC 103 communicate with a plurality of bracelet type sensor nodes 1, receive sensing data corresponding to the movement of the user from each bracelet type sensor node 1, and send it to the server 104 via the network 105. Forward. The server 104 stores the received sensing data. The server 104 analyzes the received sensing data, and generates and stores a life log indicating the user's behavior history as will be described later.

  The life log generated by the server 104 can be viewed or edited by the client computer (PC) 103 operated by the user of the life log system, and the user adds detailed information of the life log generated by the server 104. can do.

  FIG. 2 is a diagram showing an example of a bracelet type (or wristwatch type) sensor node 1 constituting the sensor unit of the life log system of the present invention, and FIG. 2A is a schematic view seen from the front of the bracelet type sensor node 1. FIG. 2B is a cross-sectional view of the bracelet type sensor node 1 viewed from the side. This bracelet type sensor node 1 mainly measures the movement of the user (wearer).

  The bracelet type sensor node 1 includes a case 11 for storing a sensor and a control device, and a band 12 for attaching the case 11 to a human arm.

  Inside the case 11 is stored a substrate 10 provided with a microcomputer 3, a sensor 6 and the like as shown in FIG. As an example of the sensor 6 that measures the movement of the human body (living body), an acceleration sensor that measures three-axis accelerations in the X-Y-Z directions in the figure is employed. In the present embodiment, the bracelet type sensor node 1 includes a temperature sensor (not shown), measures the user's body temperature, and outputs it as sensing data together with the acceleration.

FIG. 3 shows a block diagram of an electronic circuit attached to the substrate 10 of the bracelet type sensor node 11. In FIG. 3, a substrate 10 includes a wireless communication unit (RF) 2 including an antenna 5 that communicates with a base station 102, a USB communication unit 39 that is wired to the PC 103, a sensor 6, and the wireless communication unit 2. The microcomputer 3 to be controlled, the real-time clock (RTC) 4 functioning as a timer for starting the microcomputer 3 intermittently, the battery 7 for supplying power to each part, and the supply of power to the sensor 6 are controlled. A switch 8 is arranged. In addition, a bypass capacitor C1 is connected between the switch 8 and the sensor 6 to prevent unnecessary power consumption by removing noise and reducing charge / discharge speed. By controlling the switch 8 so as to reduce the number of times of charging / discharging the bypass capacitor C1, wasteful power consumption can be suppressed.
The microcomputer 3 includes a CPU 34 that executes arithmetic processing, a ROM 33 that stores programs executed by the CPU 34, a RAM 32 that stores data, and an interrupt that interrupts the CPU 34 based on a signal (timer interrupt) from the RTC 4. Serial communication for transmitting and receiving signals as serial signals between the control unit 35, the A / D converter 31 that converts the analog signal output from the sensor 6 into a digital signal, and the wireless communication unit 2 and the USB communication unit 39 An interface (SCI) 36, a wireless communication unit 2, a USB communication unit 39, a parallel interface (PIO) 37 that controls the switch 8, and an oscillation unit (OSC) 30 that supplies a clock to each unit in the microcomputer 3. Consists of including. The above-described units in the microcomputer 3 are connected via a system bus 38. The RTC 4 outputs an interrupt signal (timer interrupt) at a predetermined period set in advance to the interrupt control unit 35 of the microcomputer 3 and outputs a reference clock to the SCI 36. The PIO 37 controls ON / OFF of the switch 8 in accordance with a command from the CPU 34 and controls power supply to the sensor 6.

  The bracelet type sensor node 1 starts the microcomputer 3 at a predetermined cycle (for example, 1 minute), acquires sensing data from the sensor 6, and an identifier for identifying the bracelet type sensor node 1 in the acquired sensing data. A time stamp is given and transmitted to the base station 102 or the PC 103. Note that details of the control of the bracelet type sensor node 1 can be the same as, for example, Japanese Patent Laid-Open No. 2008-59058. The bracelet type sensor node 1 may periodically transmit the sensing data acquired to the base station 102 or the PC 103 periodically.

  FIG. 4A is a block diagram showing functional elements of a life log system to which the present invention shown in FIG. 1 is applied. Sensing data transmitted from the bracelet type sensor node 1 is accumulated in the data storage unit 500 of the server 104 via the base station 102.

  The client 103 includes a display unit 1031 that displays various types of information and an input unit 1032 that allows various information to be input by user operations.

  The server 104 includes a processor, a memory, and a storage device (not shown), and totals the sensing data measured by the acceleration of the user's arm, and collects the data totaling unit 200, the scene dividing unit 300, the scene listing unit 400, and the action acquisition The unit 500 is executed. The data totaling unit 200 calculates a total value for each unit time (for example, one minute) and stores it in the data storage unit 600. The scene dividing unit 300 analyzes the sensing data and the aggregated data, extracts scene regions that are individual motion states and scene fragments that are individual motion states, and stores them in the data storage unit 600. The scene listing unit 400 extracts a scene that is a candidate for action content by applying a rule created from past data or a scene determination rule registered in advance to the extracted scene region and scene fragment, Store in the storage unit 600. The action acquisition unit 500 presents the extracted scene to the user's client computer 103 and stores the scene selected by the user in the data storage unit 600 together with the action content input by the user.

  The identifier of the bracelet type sensor node 1 is given to the sensing data stored in the data storage unit 600, and the identifier for identifying the user (behavior content, operation state, scene, and behavior record data) For example, an identifier of the bracelet type sensor node 1) is given to identify the user.

  Here, the operation state represents the state of the user's operation such as sleep, walking, rest, etc. determined based on the sensing data, and the action content represents the content of the actual action of the user, such as a walk or watching a movie.

  The scene division unit 300, the scene listing unit 400, and the action acquisition unit 500 are stored in a storage device (storage medium) as a program, for example, loaded into the memory at a predetermined timing, and executed by the processor. In the following, the server 104 executes the scene dividing unit 300 and the scene listing unit 400 every predetermined cycle (for example, 5 minutes), and the action acquisition unit 500 is executed at a timing requested by the user. An example is shown.

  FIG. 5 is a diagram showing an overall flow of data processing performed in the life log system to which the present invention is applied. First, in step S <b> 1, the sensing data transmitted from the bracelet type sensor node 1 is transferred to the server 104 by the base station 102 or the PC 103, and the sensing data is accumulated in the data storage unit 600 of the server 104. Further, when the server 104 reaches a predetermined cycle, the data totaling unit 200 is executed, and the exercise frequency per unit time is calculated from the sensing data accumulated in the data storage unit 600 and stored in the data storage unit 600. The calculation result of the exercise frequency is data obtained by sorting the exercise frequency for each unit time in time series as shown in FIG. 6 (operation frequency). The method of calculating the exercise frequency will be described later with reference to FIGS.

  Next, in step S2, when the server 104 reaches a predetermined cycle, the scene division unit 300 is executed, and a scene region that is a series of operation states of the user from the sensing data and the aggregated data accumulated in the data storage unit 600, And a scene fragment which is a motion constituting a scene area is created.

  This scene area creation process detects a predetermined operation state (for example, walking, sleep) that can be extracted from sensing data or aggregated data, and detects the detected operation state, and the detected operation state and the next operation. The period between the states is extracted as a scene area, and the operation contents of sleep, rest, light work, work, walking, and exercise are determined using a predetermined determination rule described later, and assigned to each scene area.

  Further, the scene fragment creation process detects, for each of the extracted scene areas, a time-series data change point from the sensing data or the total data of sensing data within a predetermined time, and from the change point to the next change point. Is extracted as a scene fragment, which is one motion unit, and a fragment classification is assigned for each method of scene fragment extraction. And, for each fragment classification, it is a group identifier by one or more grouping such as grouping using the statistics of momentum, grouping using the similarity of sensing data, grouping using the similarity of aggregated data, etc. Assign fragment classification values. The change point of the sensing data or the total data is extracted as a change point when it becomes quiet from the state of intense movement. Through the above processing, the scene dividing unit 300 creates a scene area and a scene fragment at predetermined intervals and stores them in the data storage unit 600. FIG. 6 illustrates an example in which operating states of sleep, walking, rest, and light work are extracted as a scene area, an example in which a scene fragment is extracted for each scene area, and an example of a relationship between the scene area and the scene fragment.

  Next, when the scene area, the scene fragment, and the preceding and following action contents extracted by the scene dividing unit 300 in step S3 are already determined, the server 104 determines the action contents according to the registered scene determination rule. A scene that is a candidate for the user's action content is created and stored in the data storage unit 600.

  The registered scene determination rule is a rule extracted from past action content, scene area, and scene fragment data for each user, or action content, scene area, scene stored in the data storage unit 600 including the user. It is a rule created from fragment data or a rule constructed using known knowledge. Specifically, it is created from a combination of action content, scene area, scene fragment, time information and an order pattern, and is used to determine a scene that is a candidate for action content. The structure of the scene determination rule and an example of scene determination will be described later with reference to FIG.

  For example, if the action content is early morning walking, the specific action content candidate can be a walk, or “walking (10-15 minutes) and resting (20-15 minutes after waking up) (20 -25 minutes) and walking (7-10 minutes), it is determined as “commuting” which is standardized in the normal life of the user. Since the action content is a combination of action contents, the action contents are often composed of a plurality of scene areas, but there are action contents determined from a single action content and time as in the above-described walk.

  Next, the action acquisition unit 500 assigns priorities to be presented from the scenes stored in the data storage unit in descending order of possibility of matching with the contents of actions performed by the user (step S4).

  Next, in step S5, the server 104 presents each scene to the client computer 103 according to the priority order. In step S6, the user who operates the client computer 103 confirms the action content corresponding to the scene extracted by the server 104, and selects the scenes presented in order of priority, so that the user can easily perform every day. Life log can be generated.

  Next, in step S7, the scene selected by the client computer 103 is confirmed as an action record.

  The life log created in this way is stored in the data storage unit 600 of the server 104 together with a user identifier and a time stamp such as the creation date and time in step S8.

  FIG. 7 is a flowchart illustrating an example of processing performed by the data totaling unit 200 of the server 104. First, the data totaling unit 200 reads sensing data corresponding to a user identifier from the data storage unit 600 (step S11). Here, the amount of sensing data read by the data totaling unit 200 may be set to a predetermined cycle (for example, 3 minutes) that is a totaling cycle of sensing data.

  Next, in step S12, the total value for every predetermined time interval (for example, 1 minute) is calculated about the acceleration data of the sensing data read by the data totaling unit 200. In the present embodiment, the number of zero crosses indicating the frequency of exercise of the wearer (user) of the bracelet type sensor node 1 within a predetermined time interval is used as the total value.

  Since the sensing data detected by the bracelet type sensor node 1 includes the X, Y, and Z axis acceleration data, the data totaling unit 200 calculates the scalar amount of the X, Y, and Z axis acceleration. , A value where the scalar amount passes 0 or a predetermined value close to 0 is calculated as the number of zero crossings, the frequency at which the number of zero crossings appears within a predetermined time interval is calculated, and this frequency of occurrence is calculated for a predetermined time interval (1 minute) Is output as the exercise frequency.

First, the scalar quantity is obtained by scalar quantity = (Xg2 + Yg2 + Zg2) ^1/2 where Xg, Yg, and Zg are accelerations of respective axes.

  Next, the obtained scalar quantity is subjected to filter (bandpass filter) processing, and only a predetermined frequency band (for example, 0.1 Hz to 5 Hz) is extracted to remove a noise component. Then, as shown in FIG. 8, the number of times that the scalar amount of the acceleration subjected to the filter processing becomes a predetermined threshold (for example, 0.05 G) is calculated as the number of zero crossings. Alternatively, the number of times that the scalar amount of acceleration crosses a predetermined threshold is defined as the number of zero crossings. Then, the number of zero crossings within a predetermined time interval is obtained as the exercise frequency. Also, an integral value of the momentum within a predetermined time interval is obtained from the number of zero crossings and a scalar amount, and this integral value is used as the exercise intensity. Also, an average temperature within a predetermined time interval is obtained for the temperature included in the sensing data. The motion frequency may be summed by counting the number of times that the acceleration value in each direction of XYZ vibrates positively and negatively (frequency) within a predetermined time in each direction. The method for calculating the number of zero crossings has been described.

  The data totaling unit 200 obtains the exercise frequency, average temperature, and exercise intensity for each predetermined time interval, generates them as total data for each predetermined time interval, as shown in FIG. accumulate. FIG. 9 is an explanatory diagram showing a total data format 700 for each predetermined time interval. The sensor data ID 702 for storing the identifier of the bracelet type sensor node 1 included in the sensing data and the wearer (life of the bracelet type sensor node 1) A user ID 701 for storing an identifier of a log system user), a measurement date 703 for storing a start time (measurement date) of a predetermined time interval, a temperature 704 for storing a temperature included in sensing data, and a data totaling unit One entry is made up of an exercise frequency 705 for storing the exercise frequency calculated by 200 and an exercise intensity 706 for storing the exercise intensity obtained by the data totaling unit 200. The user identifier may be referred to from a table (not shown) set in advance based on the identifier of the bracelet type sensor node 1.

  FIG. 10 is a flowchart showing a scene area and scene fragment creation process performed by the scene division unit 300 of the server 104. First, the scene dividing unit 300 reads sensing data and total data corresponding to a user identifier from the data storage unit 600 (step S21). Here, the amount of total data read by the scene dividing unit 300 may be set to a predetermined cycle (for example, 5 minutes) that is a cycle of scene area creation.

  Next, in step S22, a predetermined operation state is detected from the sensing data and total data read by the scene dividing unit 300, and a scene area is created. Here, the predetermined operation state is an operation state that can be detected from the sensing data or the total data, for example, walking or sleep.

  For example, in the method of detecting the walking state, the acceleration in the vertical direction changes periodically (landing for each step), and the acceleration in the front-rear direction repeats the front and rear directions regularly in synchronization with the acceleration in the vertical direction ( (Changes in speed at each landing), and the left and right accelerations repeat regularly (swing to the left and right of the body step by step) in synchronization with the vertical acceleration, and the arm swings further. Since it can be observed as a waveform, it can be determined whether or not the corresponding scene is in a walking state. It is also possible to detect the reciprocal of the zero-cross cycle as the number of steps. A known method may be used as a method for detecting the walking state from the acceleration sensor attached to the human body.

  In addition, although the exercise frequency of sleep is extremely low, since the human body exercises such as turning over during sleep, the exercise frequency does not become zero. A known method may be used as a method for determining sleep.

  Next, with reference to the determination value table shown in FIG. 27 for scene areas not corresponding to walking or sleeping, the motion frequency of each scene area is “rest”, “light work”, “work”, “exercise”. Compared with each determination value, it is determined which corresponds.

  Next, in step S23, for each of the extracted scene areas, a time-series change point of the exercise frequency is detected from the total data corresponding to the start date and time and end date and time of the scene area, Are extracted as scene fragments, which are units of the same exercise, and stored in the data storage unit 600.

  Through the above processing, the scene dividing unit 300 extracts a scene area which is a unit of motion state and a scene fragment which is a unit of motion change from the sensing data and the total data.

  FIG. 11 is an explanatory diagram showing the format of the scene area data 800 stored in the data storage unit 600. The user ID 801 stores a user identifier, the scene area data ID 802 is an identifier of the scene area data, and the scene area. The entry date and time 803, the end date and time 804, and the area contents 805 indicating the operation state of the scene area constitute one entry.

  FIG. 12 is an explanatory diagram showing a format 900 of scene fragment data stored in the data storage unit 600. The user ID 901 is a user identifier, the scene region data ID 902 is an identifier of the corresponding scene region data, and the scene. One entry is made up of the start date and time 903, the end date and time 904 of the fragment data, and the fragment classification 1 (905) to the fragment classification n (906) which are identifiers of the assigned group.

  By recording the scene area data ID, area contents, fragment classification, etc. in association with the user IDs in this way, it is possible to create a scene that is a candidate for action contents for each user.

  FIG. 13A is a flowchart showing a procedure for registering event data. Here, the event is an action performed by the user using a PC, a mobile phone or the like, and means a short-time event. For example, email transmission / reception, photography, shopping, and the like. Here, the procedure will be described using mail transmission / reception as an example.

  First, when the user transmits / receives mail by the PC 103, data indicating that the mail is transmitted / received from the PC 103, the time, and the user ID are sent to the server 104 in step S 24. Next, in step S25, the data shown in FIG. 13B is created, and in step S26, it is registered in the data storage unit 600 of the server. Here, the transmission source of the event data is not limited to the PC 103, and for example, an external device having a calculation function such as a user's mobile phone can be considered. As described above, in the life log system of the present embodiment, when a user performs an action such as mail transmission / reception by operating a PC or a mobile phone, the action is transmitted to the server as event data. Composed. And it records on the data storage part of a server with the user ID provided to event data, and the time when it acted. The timing at which the PC or mobile phone or the like transmits the event data to the server is not particularly limited, such as a predetermined time interval when the action is performed, or a timing designated by the user.

  FIG. 13B is an explanatory diagram showing a format 1100 of event data stored in the data storage unit 600. The user ID 1101 stores a user identifier, the event data ID 1102 is an event data identifier, and the event data. Event classification ID 1103 representing the type of the event, date and time 1104 of the event data, event content 1105 for storing the content of the event data such as the content indicated by the event data or a comment given by the user, and the scene area ID associated with the event One entry is formed from the related scene 1106 to be stored. The event data and the scene area data are associated with each other by the date and time of the event data and the start date and time and the end date and time of the scene area data. Further, as shown in FIG. 16C described later, the user can add a comment to the event data, and the contents are registered as event contents.

  FIG. 14 is an example of an input support screen for inputting a user's action record displayed on the display unit 1031 of the block diagram showing the functional elements of the life log system of FIG.

  14 includes a date selection unit 1040, a graph display unit 1041, an action record display unit 1042, a scene area display unit 1043, and an event display unit 1044.

  The date selection unit 1040 has a trigger function for displaying date candidates for the user to input action records and displaying data on other display units according to the user's selection.

  The graph display unit 1041 has a function of displaying data in which the exercise frequency per unit time calculated based on the acceleration collected using the bracelet type sensor node of the date selected by the date selection unit 1040 is sorted in time series. Have.

  The action record display unit 1042 has a function for displaying and editing the action record of the date selected by the date selection unit 1040.

  The scene area display unit 1043 has a function for displaying the scene area data calculated based on the acceleration collected using the bracelet type sensor node or the like of the date selected by the date selection unit 1040.

  The event display unit 1044 has a function for displaying the event of the date selected by the date selection unit 1040.

  Hereinafter, the first and second methods for displaying the action content on the action record display unit 1042 will be described. In the first method, when scene area data or event data is displayed, the user inputs a comment on the scene area data or event data and the action content is confirmed, thereby reducing the burden of action record input. It is characterized by doing.

  Further, the second method is characterized in that the burden of action record input is reduced by calculating and presenting action record candidates from scene area data or event data.

  First, the first method will be described with reference to FIGS. 15 and 16. FIG. 15 is a flowchart showing the procedure of the data display process.

  First, a date for inputting an action record is selected using the date selection unit 1040, and if there is scene area data, event data, and already registered action record data of the user's corresponding date, action record data Are extracted from the data storage unit 600 (S201, S202). The extracted scene area data, event data, and action record data are displayed in the corresponding part (S203).

FIG. 16A is a flowchart showing the procedure of the action record input support process.
First, scene area data and event data are displayed on the corresponding part of the action record input support screen (S201 to S203), and when the user selects from the scene area data and event data (S301), the selected scene area data is displayed. Event data is displayed on the action record display unit 1042 (S302). Next, perform action record editing support processing such as adding comments to the scene area data and event data displayed on the action record display section, adjusting the start / end time of the scene, adjusting the position of the event, etc. The recording input is completed (S303, S304). If all inputs have not been completed, S301 to S304 are repeated (S305).

  FIGS. 16B and 16C show an example of a screen display of the action record input support processing procedure. FIG. 16B is an example of a screen display in which scene area data and event data are displayed on corresponding portions of the action record input support screen in S201 to S203. As shown in FIG. 14, sleep, rest, light work, and walking are displayed on the scene area display unit, and an event is displayed on the event display unit. FIG. 16C is an example of the screen display of S301 to S303. By selecting scene area data or event data from the scene area display unit 1043 or the event display unit 1044, the data is displayed in the action record display unit 1042, for example, the scene area data in the lower stage and the event data in the upper stage.

  After that, the user selects the scene area data or event data displayed on the action record display unit 1042 via the input unit 1032 and inputs a comment (action content), whereby sleep, commuting, The action content such as work is displayed on the action record display section.

  As described above, according to the first method, scene area data generated by using sensor data that can be measured by the bracelet type sensor node 1 at the time of action record input and user manual input or bracelet type sensor node 1 By inputting the action record by selecting event data generated / collected using a device other than the above, it is possible to reduce the burden required for the user's action record input.

  A method of adjusting the start / end time of the scene (action content) and the event time of the action record editing support process will be described with reference to FIG.

  When the user manually adjusts the scene start / end time and event time, the scene fragment start / end time is edited as a scale, and the scene area start / end time is edited as a scale. There are cases. A scale based on the scene fragment is displayed on the scene fragment scale 1047, and a scale based on the scene area is displayed on the scene area scale 1048. These scales are displayed by the action acquisition unit by the user specifying the action record display unit 1042 via the input unit. Alternatively, it can be automatically displayed together with the scene area display. By providing such a scene fragment scale or a scene area scale, it is possible to reduce the burden of time adjustment when the user manually adjusts the start / end time and event time of action recording.

  Next, a second method for displaying the action content on the action record display unit 1042 will be described with reference to FIGS.

  FIG. 19A is a flowchart showing the procedure of the action record input support method.

  First, when a user inputs a date for inputting an action record to the date selection unit 1040, the scene listing unit extracts scene area data and event data of the corresponding date from the data storage unit 600 (S401, S402). Next, when the extracted scene area and scene fragment and the action content before and after are already determined, the action content is determined according to the registered scene determination rule, and a scene creation process is performed (S403).

  Next, the action acquisition unit automatically displays scene area data having a certainty factor larger than a preset threshold value on the action record display unit 1042 among the created scenes (S404). Furthermore, scene area data and event data are displayed on the corresponding part of the action record input support screen (S405). When the user selects a part (time) to be input in the action record display unit 1042, scene area data and event data candidates including the selected part are displayed (S406, 407). In the present embodiment, when the user selects scene area data and event data from candidates (S408), the selected scene area data and events are displayed on the action record display unit 1042 (S409).

  Next, perform action record editing work such as adding comments to the scene area data and event data displayed in the action record display section, adjusting the start and end times of the scene, and adjusting the event position. Is completed (S410, S411). If all inputs are not completed, S405 to S410 are repeated (S412). Processing such as adding a comment to scene area data and event data, adjusting the start / end time of the scene, adjusting the position of the event, and the like is the same as that described in the first method.

  FIG. 19B is an illustrative example relating to S404. In the example of FIG. 19B, the scene data “sleep” is automatically input as scene data having a certainty factor equal to or greater than a threshold value.

  FIG. 19C is an example of illustration regarding S405 to S408. When the user selects a part (time) that the user wants to input, a start / end time including the selected time and an action record candidate display unit 1060 including a plurality of different types of candidates are displayed (upper diagram in FIG. 19C). At this time, the displayed candidates are displayed in descending order of the certainty of the scene data or in descending order of frequency selected by the user in the past. When the user selects scene data from candidates, an action record is displayed on the action record display unit 1042 (the lower diagram in FIG. 19C).

  FIG. 20 is a flowchart showing a scene creation process (S403 in FIG. 19A) performed by the scene listing unit 400. The scene enumeration unit searches for scene areas, scene fragments, and action recording patterns that match the scene determination rule, and extracts scenes (action contents).

  First, in step S51, a pre-registered scene determination rule corresponding to the user is read from the data storage unit 600. The scene determination rule is composed of a condition and a conclusion, and a certain conclusion (scene) is estimated when the condition is satisfied, and is recorded in advance for each user. The condition includes a plurality of or single scene areas, a combination of action records, and an order pattern. The scene area is not only the contents of the scene area but also a combination of scene fragments in the period of the scene area. In addition, the scene area, scene fragment, and action record are also conditional on the elapsed time from the start date to the end date. On the other hand, the conclusion is composed of the scene content, the scene area or the start date and time and the end date and time of the action recording, and is accompanied by a certainty factor with a value indicating the determination accuracy as an evaluation value. The certainty factor can be calculated from past data, and may be arbitrarily determined when a scene determination rule is created based on expert knowledge.

  FIG. 21 shows an example of the structure of the scene determination rule. The scene determination rule includes a condition table 1000 and a conclusion table 1100, and is associated with a rule ID. The condition table includes a set of user ID, rule ID, order, type, item name, value, and comparison operation. The type includes a scene area, a scene fragment, an action record, and an elapsed time, a time zone, and a time as time data. The comparison calculation is an arithmetic expression for determining whether or not the scene area data to be determined matches. The conclusion table includes a set of user ID, rule ID, scene content, start date / time, end date / time, and certainty factor. No specific value is defined for the start date / time and end date / time, and the start date / time and end date / time are determined from the data that satisfies the conditions in the condition table by specifying the start date / time or end date / time of the data shown in the order of the condition table. Can do.

  For example, the scene determination rule of the rule ID 1 of the user ID 101 is “the first is an action record“ sleep ”, and then there is a“ walk ”scene area in the morning time zone within 70 minutes, and then a“ rest ”scene. In the case of the area, and finally the “walking” scene area, the scene is “commuting” and the certainty factor is “80%”. "It is shown that.

  In addition, the scene determination rule of the rule ID 2 of the user ID 105 is “a scene fragment that is a light work scene area and the value of fragment classification 1 is 5 in the scene area and a scene fragment that is 2 in fragment classification 3”. If the time from the start date / time to the end date / time of the scene area is 30 minutes or more, the scene is “meeting” and the certainty level is “60%”.

  Next, in step S52, for each of the read scene determination rules, the conditions in the condition table of the scene determination rule are determined from the scene area, scene fragment, and action record input by the user stored in the data storage unit 600. Are extracted, the contents of the conclusion table are extracted as scenes, and stored as scenes in the data storage unit 600 (step S53).

  Next, in step S54, when the user inputs an action record after the time when the extracted scene is stored in the data storage unit 600, the data storage of the scene satisfying the scene determination rule is repeated by repeating step S51 and the subsequent steps. The storage in the unit 600 is repeated. When a new action record is input, the number of scene determination rules that satisfy the condition increases, a scene to be stored in the data storage unit 600 is added, and a scene with high certainty may be stored. Through the above processing, automatically input scenes and action record candidates are created.

  As described above, according to the second method, not only rough movements such as walking, exercise, and rest are identified, but specific action content candidates are calculated and presented to the user. This makes it possible to reduce the input burden on the user when inputting action records.

  In particular, a scene area that is a unit of a series of operation states and a scene fragment that is a short-time motion state constituting the scene area are created from the sensing data. Then, action content candidates can be calculated by configuring the action content by a combination of scene areas and an order pattern, and determining similarity not only by the operation state of the scene area but also by a combination of scene fragments. As a result, it is possible to display a plurality of behavior candidates having different time intervals together with certainty levels, instead of the user's behaviors at regular time intervals. Therefore, it is possible to cope with fluctuations in the behavior of the user and to create a more accurate behavior record.

  FIG. 18 shows a format 1200 of action record data when an action is confirmed by the first and second methods described above, and is stored in the data storage unit 600. User ID 1201 for storing the identifier of the user, action record data ID 1202 that is an identifier of the action record data, start date and time 1203 of the action record data, end date and time 1204, and contents of the action record such as a comment given by the user The action record content 1205 for storing the action record and the related data 1206 for storing the ID of the scene area data or event data related to the action record constitute one entry.

  In the above-described embodiment, an example in which the acceleration sensor of the bracelet type sensor node 1 is used to detect the movement state of the user (human body) as the life log system is shown. However, the movement state of the human body can be detected. Any biological information may be used. For example, a pulse or the number of steps can be used, or the motion state of the human body may be detected from a combination of the plurality of biological information. Further, not only the biological information but also the position of the human body may be used using position information such as GPS or a portable terminal.

In the life log system, a monitoring target is a person and a scene corresponds to an action, but the monitoring target is not limited to a person, and may be a device. When the monitoring target is a device, it is possible to extract a scene indicating an operation state or operation content as a scene from sensing data (for example, vibration, volume) indicating the state of the device measured by a sensor attached to the device. Yes, it is possible to acquire the operating status history of the device.
<Second Embodiment>
FIG. 4B is a partial change of the system configuration shown in FIG. 4A of the above embodiment, and the other configurations are the same as those of the first embodiment. In place of the server 104 of the first embodiment, the client computer 103 includes a data collection unit 200, a scene division unit 300, a scene listing unit 400, an action acquisition unit 500, and a data storage unit. Are directly connected to the base station 102. The configurations of the data collection unit 200, the scene division unit 300, the scene listing unit 400, the action acquisition unit 500, and the data collection unit 600 are the same as those in the above embodiment. The client computer 103 is connected to the server 104 via the network 105. The server 104 includes a data storage unit 1300 that receives and stores an action record generated by the client computer 103, and an analysis unit 1400 that performs a predetermined analysis on the action record.

  In the case of the configuration shown in FIG. 4A, data is stored in the server. Therefore, when a network connection is possible, data can be sent regardless of the location. However, there is a problem that data cannot be sent and behavior input cannot be performed when the network connection is not possible. In the configuration shown in FIG. 4B, a dedicated computer such as the PC 103 is required for data transmission and action input, but there are restrictions on the location, but data is sent to the PC 103 and action input is performed on the PC 103. Therefore, there is an advantage that there is no need to worry about the presence or absence of network connection.

<Third Embodiment>
In the first embodiment and the second embodiment, scene area data and event data have been used as reference information for action recording input, but other data may be used. In the third embodiment, an example in which data about PC usage status and schedule data are used will be described.

  FIG. 22 is an example of an input support screen for inputting a user's action record displayed on the display unit 1031 of the block diagram showing the functional elements of the life log system of FIG.

  The input support screen example of FIG. 22 includes a date selection unit 1040, a graph display unit 1041, an action record display unit 1042, a scene area display unit 1043, an event display unit 1044, a PC usage status display unit 1045, and a schedule display unit 1046.

  The date selection unit 1040 has a trigger function for displaying date candidates for the user to input action records and displaying data on other display units according to the user's selection.

  The graph display unit 1041 has a function of displaying data in which the exercise frequency per unit time calculated based on the acceleration collected using the bracelet type sensor node of the date selected by the date selection unit 1040 is sorted in time series. Have.

  The action record display unit 1042 has a function for displaying and editing the action record of the date selected by the date selection unit 1040.

  The scene area display unit 1043 has a function for displaying a sensor scene calculated based on the acceleration collected using the bracelet type sensor node or the like of the date selected by the date selection unit 1040.

  The event display unit 1044 has a function for displaying the event of the date selected by the date selection unit 1040.

  The PC usage status display unit 1045 has a function for displaying the usage status of the computer terminal (PC) used by the user on the date selected by the date selection unit 1040.

  The schedule display unit 1046 has a function of displaying the user schedule of the date selected by the date selection unit 1040.

  The PC usage status data and schedule data are acquired by the procedure shown in the flowchart of FIG. 13A, and have the data structure shown in FIG.

  The PC usage status data is data related to the PC operation, for example, data related to activation and termination of the PC. History information such as operation of input devices such as a keyboard and a mouse can also be included. First, history information is acquired when the user operates the PC 103, and the information is sent from the PC 103 to the server 104 in step S24. The timing of sending is every hour, just before the end of the PC. Next, in step S25, the data shown in FIG. 13B is created and registered in the data storage unit 600 of the server in step S26.

  The schedule data is the user's own schedule input by the user using the schedule input tool on the PC 103. First, when the user inputs data using the schedule input tool on the PC 103, the data is sent from the PC 103 to the server 104 in step S24. Next, in step S25, the data shown in FIG. 13B is created and registered in the data storage unit 600 of the server in step S26.

  The schedule input tool may be on a user-owned mobile phone. It is also possible to use a schedule input tool on the server 104. In this case, steps S25 and S26 are directly executed from the schedule input tool on the server 104 to register data.

  22 includes a date selection unit 1040, a graph display unit 1041, an action record display unit 1042, a scene area display unit 1043, an event display unit 1044, a PC usage status display unit 1045, and a schedule display unit 1046. Although it is configured, the configuration can change depending on the number and type of data that can be acquired and the needs of the user.

  A method of adjusting the scene start / end time and event time of the action record editing support process according to the third embodiment will be described with reference to FIG.

  When the user manually adjusts the start / end time of the scene and the time of the event, the scene fragment start / end time displayed on the scale 1047 is edited as a scale, and the scale 1048 is displayed. There are a case where editing is performed using the start / end times of the scene area as a scale, and a case where editing is performed as a scale where the start times or end times of a plurality of data displayed on the scale 1049 match. The target information for displaying the scales can be switched by turning ON / OFF the changeover switches 1050 to 1053. Further, the displayed scales may be dynamically displayed as much as necessary during editing.

By adjusting in this way, it becomes possible to reduce the burden of time adjustment when the user manually adjusts the start / end time of the scene and the time of the event.
As described above, by using not only scene area data and event data, but also PC usage status data and schedule data, it is possible to record multiple data as action records, reducing the input burden on the user. can do.

  That is, after the input of one action record is completed, the action record can have a hierarchical structure by selecting the same type or other types of scene data between the start time and the end time of the action record. By inputting the action record in this way, detailed information can be easily given to the action record. An example is shown in FIG. FIG. 24 shows an example in which the selected content is displayed on the action record display unit when the user selects the content displayed on the PC usage status display unit and the schedule display unit. Specifically, for the action content “work” determined from the scene area data, “file creation (material A creation, information collection)” determined from the PC usage status, and “meeting” determined from the schedule are It is displayed.

<Fourth embodiment>
In the first embodiment to the third embodiment, it is necessary to manually input a comment in the action record editing support process, particularly when a user's feelings and emotions are input. It is also possible to input using symbols representing the user's feelings and feelings registered in advance in the data storage unit.

  FIG. 25 is a processing example for inputting the user's feelings and emotions in the action record editing support processing of the fourth embodiment.

  First, when an action record displayed on the action record display unit 1042 is selected, a symbol group representing feelings and emotions registered in advance is displayed. It becomes possible to input easily by selecting a necessary one from them.

  In addition, as shown in the display example in the lower part of FIG. 25, if an index representing the undulations of feelings / emotions is given in advance to the symbol group of feelings / emotions, a graph representing the undulations of feelings / emotions can be generated. Is possible.

  FIG. 26 shows an example of a score representing the contents of the comment candidates and the feeling / feeling undulations displayed in the comment candidate display section 1061 of FIG. The comment candidates are not all those shown in FIGS. 25 and 26, but include various things such as those showing emotions. The method of giving the score can be any method as long as it can express the undulations of feelings and emotions.

  In this way, by using symbols representing the user's feelings and feelings in combination with scene area data, event data, PC usage status data and schedule data, the feelings and feelings in actual behavior can be easily obtained. It becomes possible to record, and it is possible to reduce the input burden on the user.

  Although the embodiments of the present invention have been described above, the present invention is not limited to the above-described embodiments, and various modifications can be made, and it is possible to appropriately combine the above-described embodiments. It will be understood by the contractor.

It is a block diagram which shows an example of a structure of a life log system. It is a figure which shows an example of a bracelet type sensor node, (a) is the schematic seen from the front of the bracelet type sensor node 1, (b) is sectional drawing which looked at the bracelet type sensor node from the side. It is an example of the block diagram of the electronic circuit attached to the base of a bracelet type sensor node. It is an example of the block diagram which shows the functional element of a life log system. It is a modification of the block diagram which shows the functional element of a life log system. It is an example of the figure which shows the whole flow of the data processing performed with a life log system. It is an example of a relationship between a graph, a scene area, and a scene fragment in which exercise frequencies for each unit time are sorted in time series. It is a flowchart which shows an example of a process of a data total part. The graph which shows the relationship between acceleration and time shows an example of determination of the number of zero crossings. It is an example of the format of the total data for every predetermined time interval. It is an example of the flowchart which shows the process of a scene division part. It is an example of the format of scene area data. It is an example of the format of scene fragment data. It is an example of the flowchart which shows the procedure which registers event data. It is an example of a format of event data. It is an example of a screen structure of an action record input assistance screen. It is a flowchart which shows an example of the procedure of a data display process. It is a flowchart which shows an example of the procedure of an action record input assistance process. It is an example of the screen display which displayed scene area data and event data on the corresponding part of the action record input support screen. It is an example of the screen display at the time of action record input of an action record input assistance screen for scene area data and event data. It is an example of a screen structure of action record edit assistance processing. It is an example of a format of action record data. It is an example of the flowchart which shows the procedure of the action record input assistance method. It is an example of a screen structure of action record automatic input. It is an example of a screen structure of action record candidate display. It is an example of the flowchart showing a scene creation process. It is an example of the structure of a scene determination rule. It is an example of a screen structure of the action record input assistance screen of 3rd Embodiment. It is an example of a screen structure of the action record edit assistance process of 3rd Embodiment. It is an example of illustration when a plurality of data are included in the action record of a 3rd embodiment. It is a processing example for inputting a user's feelings, feelings, etc. in the action record editing support process of the fourth embodiment. It is an example of the score showing the content of the comment candidate of 4th Embodiment, and the ups and downs of feelings and feelings. It is an example of the table of the judgment value which sets the relationship between exercise frequency and an operation state.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Brace type sensor node 102 Base station 103 Client computer 104 Server 200 Data totaling part 300 Scene division part 400 Scene enumeration part 500 Action acquisition part 600 Data storage part 1031 Display part 1032 Input part

Claims (13)

A sensor node that includes a sensor and a communication unit that transmits sensing data acquired by the sensor to a server;
A base station connected to the server via a network and transmitting the sensing data received to the server;
Based on a data storage unit that accumulates the sensing data, a scene division unit that detects an operation state of the monitoring target from the sensing data accumulated in the data storage unit, and a scene determination rule stored in the data storage unit A scene enumeration unit that calculates a plurality of behavior content candidates to be monitored from the operation state, and the plurality of behavior content candidates displayed on a connected display unit, and specified from the displayed plurality of behavior content candidates When the action content is selected, an action acquisition unit that determines the action content to be monitored and stores it in the data storage unit,
An action record input support system comprising: The action record input support system according to claim 1,
The scene enumeration unit calculates a certainty factor for each of the plurality of candidates,
The behavior acquisition input support system, wherein the behavior acquisition unit displays the behavior content candidates having different time intervals together with the certainty factor on the display unit. The action record input support system according to claim 1,
The scene dividing unit extracts a period between the operation state and the next operation state as a scene region, detects a time-series change point of the sensing data for each scene region, and then performs a next step from the change point. An action record input support system that extracts scene fragments whose motion unit is up to the change point of the. The action record input support system according to claim 1,
The scene judgment rule consists of a condition table and a conclusion table.
The condition table and the conclusion table are related by a rule ID,
The condition table is composed of a scene area indicating the operation state of the monitoring target, a scene fragment constituting the scene area and indicating a motion unit, action recording of the monitoring target, time information, and their appearance order.
The conclusion table is an action record input support system including the order of appearance, action contents, and certainty. The action record input support system according to claim 1,
The data storage unit records data input through an external device as event data,
The action acquisition unit displays the event data on the display unit with the action content and the time axis matched. When the displayed event data is selected, the action acquisition unit is selected together with the confirmed action content. An action record input support system that displays event data. The action record input support system according to claim 1,
The behavior acquisition input support system, wherein the behavior acquisition unit displays a scale for adjusting a time width of the behavior content on the display unit. The action record input support system according to claim 1,
When the monitoring target is a person,
The data storage unit stores a symbol indicating the feeling or emotion of the person,
The action acquisition unit displays the symbol on the display unit, and when the displayed symbol is selected, the person's feelings or emotions are graphed in time series based on an index corresponding to the selected symbol. Action record input support system to display. A communication unit that receives biological information acquired by a sensor node attached to a monitoring target;
A data storage unit for storing the biological information;
A scene division unit for detecting the operation state of the monitoring target from the biological information;
A scene enumeration unit that calculates a plurality of behavior content candidates for the monitoring target from the operation state based on a scene determination rule stored in the data storage unit;
The plurality of action content candidates are displayed on the connected display unit, and when a specific action content is selected from the displayed plurality of action content candidates, the action content to be monitored is confirmed and the data An action acquisition unit stored in the storage unit;
A server comprising The server according to claim 8, wherein
The scene enumeration unit calculates a certainty factor for each of the plurality of candidates,
The said action acquisition part is a server which displays the candidate of the said action content from which a time interval differs on the said display part with the said certainty factor. The server according to claim 8, wherein
The scene dividing unit extracts a period between the operation state and the next operation state as a scene region, detects a time-series change point of the biological information for each scene region, and then performs a next step from the change point. A server that extracts scene fragments whose motion unit is up to the change point. The server according to claim 8, wherein
The scene judgment rule consists of a condition table and a conclusion table.
The condition table and the conclusion table are related by a rule ID,
The condition table is composed of a scene area indicating the operation state of the monitoring target, a scene fragment constituting the scene area and indicating a motion unit, action recording of the monitoring target, time information, and their appearance order.
The conclusion table is a server comprising the order of appearance, action content, and certainty. An action record input support method executed by a system including a sensor node that is attached to a monitoring target and acquires sensing data, and a server that analyzes the sensing data,
The sensor node transmits sensing data acquired by the sensor to the server,
The server accumulates the sensing data, detects the operation state of the monitoring target from the sensing data, and selects the action content candidate of the monitoring target from the operation state based on a scene determination rule stored in the server. Multiple calculations are performed, the plurality of action content candidates are displayed on the connected display unit, and when a specific action content is selected from the displayed plurality of action content candidates, the action content to be monitored is determined. The action record input support method stored in the server. The action record input support method according to claim 12,
The server calculates the certainty factor of each of the plurality of candidates, and displays the action content candidates having different time intervals on the display unit together with the certainty factor.

Images