JP6263135B2 - Server device operating method, server device, and computer program - Google Patents

Description

  The present invention relates to a method for operating a server device, a server device, and a computer program.

  In recent years, many techniques for estimating human behavior from sensor data have been proposed. In order to record human behavior, there are a method of recording a sensor device by attaching the sensor device to a human and a method of recording a sensor device installed in the environment. For example, there are a method for estimating a work at hand by attaching a sensor to a human arm (for example, Patent Document 1), a method for estimating a posture of a person from a camera image (for example, Patent Document 2), and the like. In these methods, learning is performed in advance and the correct label and data are associated with the mainstream.

JP 2010-271978 A JP2013-020578A

Data with large individual differences such as vital data and data with large environmental differences such as in-home data are difficult to create in advance, and in order to understand the meaning of the data, for each individual, for each environment, You need to create a correct label. Although it is conceivable to request labeling from the user regarding vital data or in-home data, the user is not always ready for labeling, and the user responds to the request immediately after the labeling request occurs. Is hard to think in real life. When requesting the user to label sensor data changes that occurred in the past, it is difficult for the user to recall the situation where the data occurred even if the sensor data is presented as it is, and the user recalls the situation correctly. The user needs to be queried with a possible expression.
An object of this invention is to provide the technique for producing | generating the question sentence which is easy to obtain the label of a correct answer.

  In order to solve the above-mentioned problem, a first aspect of the present invention is an operation method of a server device, wherein the server device has a sensor installation location around the user, a type of the sensor, and a sensor output from the sensor. A step of inputting an event expressed by a type of trigger detected by data; a step of detecting a predetermined condition as a trigger of timing when the server device sends a question to the user; and the server device The event or event pattern input to the question target section until the condition as the trigger is detected, and the same event or event pattern previously associated with the system importance is detected as a question target candidate, For each candidate question, calculate the value for selection using the corresponding system importance , Characterized in that it comprises the steps of selecting a question target candidate corresponding to the maximum of the value as a question object.

  For example, the server device generates a question sentence including a question target sentence associated with the question target in advance.

  For example, the server device detects an event or event pattern input simultaneously or continuously with the question target, calculates a value indicating the low frequency of appearance of each of the plurality of events or event patterns, and is the lowest An event or event pattern corresponding to the value indicating the appearance frequency is selected.

  For example, when the server device stores an auxiliary information sentence corresponding to the one event or event pattern for at least one event or event pattern, the event selected by the value calculated using the appearance frequency Alternatively, when the event pattern corresponds to the one event or event pattern, the question sentence is generated using the auxiliary information sentence.

  The second aspect of the present invention is a server device for inputting an event expressed by a location of a sensor around a user, a type of the sensor, and a trigger type detected by sensor data output from the sensor. Means, a means for detecting a predetermined condition as a trigger for sending a question to the user, and an event or an event pattern input to the question target section until the condition as the trigger is detected In addition, the same event or event pattern that is associated with the system importance in advance is detected as a question target candidate, and for each of the plurality of question target candidates, a value for selection is calculated using the corresponding system importance. And means for selecting a question target candidate corresponding to the maximum value as a question target.

  The third aspect of the present invention is a computer program for causing a computer to function as the server device according to the second aspect of the present invention.

  ADVANTAGE OF THE INVENTION According to this invention, the technique for producing | generating the question sentence which is easy to obtain the correct answer label can be provided.

It is a system configuration figure in an embodiment of the invention. 1 is a block diagram of Example 1. FIG. 3 is a flowchart of the first embodiment. It is an example of a trigger function. It is an example of an event log. It is an example of the correspondence table of an event and a natural sentence. It is a figure which illustrates the illumination intensity at night for 3 days. This is an example of a question target list. It is an example of an event segment. It is an example of a question object sentence table. It is an example of an auxiliary information sentence table. It is a presentation example of a question sentence. It is an example of the question object sentence table of Example 2. It is an example of the question object sentence table of Example 3. It is an example of the question object sentence table of Example 4. FIG. 10 is a block diagram of Example 4.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[Example 1]
FIG. 1 is a system configuration diagram according to an embodiment of the present invention. FIG. 2 is a block diagram of the first embodiment.

This system includes a plurality of sensors 1 arranged around a user (not shown), a server device 2 that can communicate with the sensors 1, and a user terminal 3 that can communicate with the server device 2.
Each sensor 1 includes a sensing unit 101 that acquires various sensor data, and a sensing communication unit 102 that transmits the sensor data to the server device 2.

  The server apparatus 2 includes a server communication unit 201, a sensor data log storage unit 202 that stores sensor data received from the sensor 1 by the server communication unit 201, and a specific pattern (called a trigger) from the received sensor data by a trigger function. Event detection means 203 that detects and outputs a primary analysis result called an event represented by the installation location of the target sensor 1, the type of the sensor and the type of the trigger, and the trigger (change) corresponding to the event and the event Event log storage means 204 for storing an event log including a start time and an end time, a question target detection means 205 for detecting an event as a question target candidate, and a question target candidate temporary store for temporarily storing question target candidates Means 206 and a question transmission unit for detecting timing for transmitting a question Includes a timing detection unit 207, a question generating means 208 for generating a question, a question / answer storage means 209 for storing the generated question temporarily.

  The user terminal 3 includes a user terminal communication unit 301, a presentation unit 302 that presents a question generated by the server device 2 by voice or text, and an input unit by which a user inputs an answer by voice, text, or a selection button. 303.

FIG. 3 is a flowchart of the first embodiment.
(S1)
Each sensor 1 acquires sensor data and transmits it to the server device 2.
The server device 2 receives the sensor data, and stores the time stamp and sensor data at the time of reception in the sensor data log storage unit 202.

(S3)
The event detection means 203 of the server device 2 detects the event by sequentially inputting the received sensor data into a predetermined trigger function and performing processing, and the start time and end of the event and the trigger (change) corresponding to the event The event log including the time is stored in the event log storage unit 204.

FIG. 4 is an example of a trigger function.
In the first embodiment, as a trigger function for detecting a change, an up function for detecting a monotonic increase, a down function for detecting a monotonic decrease, a stepup function in which a value rises in a step response, and a stepdown function in which a value falls in a step response It is assumed that a change function that repeats ascent and descent and a keep function that does not change the value are defined, and this function constantly monitors sensor data.

  In addition, as a trigger function, you may set the trigger function which detects a change other than the above-mentioned change. For example, an abnormal value detection algorithm may be set as a trigger function. Then, when considering the scene of monitoring the acceleration sensor installed in the cup, the event output when an abnormal value appears due to the acceleration of the cup may be cup.acc.anomaly.

  For example, when an acceleration sensor and a temperature sensor are built in a cup, if the cup is lifted or a drink is drunk with the cup, the acceleration sensor changes, and the change function detects a trigger “change”.

  In this case, an event log including an event “cup.acc.change” which means “change detected by an acceleration sensor installed in a cup” is stored in the event log storage unit 204.

  Moreover, when a cold drink or a hot drink is poured into a glass, the down function or the up function detects a trigger of down or up. When something cold is poured into the cup, the temperature of the entire cup drops, and an event called cup.temp.down is detected, meaning that a temperature sensor installed in the cup has detected a change called down. When a hot item is poured, the temperature of the entire cup rises, and an event called cup.temp.up is detected, which means that a change called up is detected by a temperature sensor installed in the cup.

  The identification information indicating the sensor installation location (cup in the above example) may be determined in advance or may be input by the user. Further, the granularity of the expression of the identification information may be a specific “cup” or the like as in the above example, or may be “device1” or the like having a high abstraction level.

FIG. 5 is an example of an event log.
When an event is detected by the trigger function, an event log containing the start time (time following start :) and end time (time following end :) of the event and the trigger (change) corresponding to the event Stored in the storage means 204.

FIG. 6 is an example of a correspondence table between events and natural sentences.
As shown in FIG. 6, the event may be stored with a semantic conversion by having a correspondence table with the meaning. For example, instead of saving it as cup.acc.change, you may save it as “Using a cup”.

  In the above description, the sensor 1 transmits sensor data to the server apparatus 2 and the server apparatus 2 drives the trigger function. However, the sensor 1 drives the trigger function and transmits a flag corresponding to the event to the server apparatus 2. May be.

  By doing so, only the flag corresponding to the event is transmitted instead of the sensor data, and the data transmission amount is reduced, so that the battery consumption of the sensor 1 is reduced.

Returning to FIG.
(S5)
The question target detection unit 205 detects the same event or event pattern (referred to as an event pattern, which will be described later in detail) registered in the question target list from the event log stored in the event log storage unit 204. If this event or event pattern is temporarily stored in the question target candidate temporary storage means 206 as a question target candidate, the process proceeds to step S7, and if the same event or event pattern registered in the question target list is not detected, Return to step S1.

  The event pattern is a pattern {e1, e2,..., EN} composed of a plurality of events e. For example, the event pattern that “the illuminance value has changed from data1 to data3 and data3 has been detected three times and then transitioned to data4” is {device1.lumi.keep@data1, device1.lumi.keep@data3, It is expressed by event expression of device1.lumi.keep@data3, device1.lumi.keep@data3, device1.lumi.keep@data4}.

  The server device 2 may register an event or event pattern in the question target list during the operation. In this case, the initial question target list may not have an event or an event pattern.

  In the first embodiment, a case where an event or an event pattern is registered in the question target list in advance will be described. Events and event patterns that do not depend on the user but depend on the environment are preferably registered in advance in the question target list in this way.

  For example, it is assumed that the user has introduced an illuminance meter (sensor 1) somewhere in the house. In addition to the illuminance meter, an acceleration sensor is installed at the entrance door, the refrigerator door, and the sofa, and the opening / closing of the entrance door and the refrigerator door and the seating on the sofa can be detected. Furthermore, it is assumed that an illuminance sensor is installed in the television and that it can detect whether the television is turned on or off. In the case of a 1LDK house, the living room, dining room, and kitchen continue as a space without being separated by doors. For example, even if an illuminance meter is installed in the living room, the illumination light from the dining room and kitchen is also To reach. When the location of the illuminometer is fixed, only the lighting in the house is the light source outside the daylight hours at night. Considering the outside hours of sunshine, there are only a limited number of combinations of lighting.When lighting is turned on in a certain combination, the illuminance is fixed unless the illuminometer is installed in a fixed location and there is an obstruction that blocks the Therefore, the illuminance measured by the illuminometer is considered to be classified into a finite cluster. If the illumination condition label is assigned to the illuminance cluster, it is possible to guess which illumination is lit by observing the illuminance. Typical actions performed in a space such as a living room, a dining room, and a kitchen are, for example, “dandan”, “meal”, and “cooking”. For this reason, it is thought that a life pattern is reflected in the event log of lighting conditions. In the first embodiment, a scene in which a user is asked for a label of an illumination condition associated with an illuminance cluster is considered.

  A method for generating a question target list will be described for an example of such a scene. This processing is given as preprocessing, and a detailed flowchart is omitted.

FIG. 7 is a diagram illustrating the illuminance at night for three days.
Assume that the user has measured the illuminance for 3 days after installing the illuminance meter in the house, as shown in FIG. After 3 days of data measurement, the illuminance outside the sunshine hours is automatically clustered. As a method of automatic clustering, a general method such as an x-means method is used.

FIG. 8 is an example of a question target list.
The illuminance of the extracted cluster is stored in the question target list as shown in FIG. Here, the name of the installed illuminometer is device1, and the illuminance obtained as a result of clustering is represented by an event expression such as device1.lumi.keep@data1. Here, the keep function is a function that detects an event called keep when constant illuminance is measured for a certain period of time, and device1.lumi.keep@data1 is `` the illuminance of device 1 is constant at the illuminance of data1 ”Is the same as the event generated when it is detected. As a result of the clustering of the illuminance, device1.lumi.keep@data4 is obtained from the four illuminance event expressions device1.lumi.keep@data1, and these and the corresponding illuminance are stored in the question target list.

Initially, the server apparatus 2 has not obtained labels such as kitchen lighting or living room lighting, and the question target list includes illuminance but does not include a label.
In the question target list, importance (referred to as system importance) for asking the label is set.

  The system importance is, for example, three levels of 0, 1, and 2. There are various methods for calculating the system importance, but here, it is as follows. For example, device1.lumi.data1 and device1.lumi.data4 have a high appearance frequency, so the system importance level is given 2. device1.lumi.data3 has a medium appearance frequency, so the system importance level Assume that 1 is given and device1.lumi.data2 has a low appearance frequency, and therefore 0 is given to the system importance.

As the simplest example of requesting a label from the user, when the illuminance of a certain cluster (referred to as cluster A) appears, the user can be inquired as to "Where is electricity on?" If the answer from the user is “living”, a label “living” is given to cluster A.
However, it is only necessary to obtain an answer before the illuminance changes, but it is not always possible for the user to answer when the illuminance condition is desired. In addition, the user unconsciously spends lighting or turning off various places. For this reason, it is not easy for the user to recall the question “Where did electricity come on?”, It is necessary to generate a question so that the user can recall the past lighting. .

Returning to FIG.
(S7)
The question transmission timing detection means 207 proceeds to step S9 if the timing for transmitting a question is detected, and returns to step S1 if not detected.
Specifically, the question transmission timing detection unit 207 proceeds to step S9 if an event or event pattern defined as the question transmission timing is detected from the event log stored in the event log storage unit 204.
In the first embodiment, it is assumed that it is defined as the question transmission timing that the user has hung on the sofa.

(S9)
First, the question generation unit 208 becomes a question target that is important and easy to recall for the user and understanding of the environment based on various importance levels from the question target candidate group stored in the question target candidate temporary storage unit 206. Select an event or event pattern. This is called question target selection.

Next, the question generation means 208 selects supplementary information used to explain the scene to be questioned most easily based on various importance levels. This is called supplemental information selection.
The question generating means 208 finally generates a question sentence by combining the event selected by the question target selection and the supplementary information selected by the supplemental information selection.

(S11)
Next, the server communication unit 201 transmits a question sentence to the user terminal 3, and the question generation process ends here.
Here, there are two types of the above-mentioned importance categories, that is, the importance from the viewpoint that the system understands the user and the importance from the viewpoint that the person can easily recall. Here, the importance of the former is the system importance, the importance of the latter is the user importance, the importance based on the segment length, the importance based on the event appearance frequency, the importance based on the elapsed time from the event occurrence, the question It defines the importance based on the complexity of the target, the importance based on the deviation from the normal value, and the importance based on the data strength. Details of various importance levels will be described later.

  In the first embodiment, one or a plurality of continuous similar events (collectively referred to as event segments) to be questioned are selected from the sum of the importance based on the system importance and the segment length, and the importance based on the event appearance frequency. Is used to select an event segment as supplementary information so as to make it easier to recall the event segment to be questioned, and generate a question sentence.

FIG. 9 is an example of an event segment.
Here, there are cases where events are detected continuously, and one event segment is from the start of detection of the same type of event to the end of detection. The case where events are continuous refers to a case where, for example, the same illuminance data1 is continuously observed and device.lumi.keep@data1 is continuously detected as shown in FIG. The smallest unit of event segment is when an event is detected only once. Segmenting event time-series data (event log) is referred to as event segmentation. In segmentation, event segments are generated.

  In the first embodiment, it is considered that the user is inquired about the label of the lighting that is lit. For example, let us consider a case where the following series of flows occur after shopping for food on the way home from work and returning home. Go home, enter the room, turn on the kitchen lights. I put my luggage in the kitchen, and I got the food I bought in the refrigerator in the kitchen. Cook in the kitchen. After cooking, turn on the dining lights, prepare meals, turn off the kitchen lights and eat. After eating, turn on the kitchen temporarily, clean the tableware, turn off the kitchen and dining lights, turn on the living room lighting, and hang it on the sofa. At this time, the question transmission timing is detected, and the process proceeds to question generation processing. The event segments generated at this time are as shown in FIG. Among the event segments, those included in the question target list are displayed with diagonal lines.

First, question target selection will be described.
In the question target selection of the first embodiment, the importance based on the system importance and the segment length is calculated for each event segment, and the event segment having the maximum sum of various importances is selected as the question target.
It is assumed that the system importance is set in advance as described above.

  The importance based on the segment length is a parameter that reflects the fact that the longer the duration of the event is, the easier it is to remember and recall. The higher the segment length, the higher the importance, and the shorter the segment length, the more important Define the degree to be low. The segment length of an event that occurs between the time when the question target candidate is detected and the time when the question transmission timing is detected is calculated, and the event segment length is set as the importance.

  For example, in the example of FIG. 9, the importance level based on the segment length of the event segment 1 is 9, the importance level based on the segment length of the event segment 5 is 5, and the importance level based on the segment length of the event segment 3 is 2.

  The sum of the importance based on the system importance and the segment length is calculated for each event segment, and the event segment having the maximum sum is selected as the question target.

  In the case of Example 1, in the event segment 1, since the system importance of device1.lumi.data1 is 2, the importance based on the segment length is 9, and the sum is 11, the event segment 1 is selected as the question target. The

Next, supplementary information selection will be described.
In the first embodiment, a question sentence that best describes the event segment 1 selected as the event segment to be questioned is generated using the importance based on the event appearance frequency. The importance based on the event appearance frequency is a value that reflects that an event pattern that occurs only once in the question target section Tq is easier to recall than an event pattern that occurs multiple times.

  For example, if the event segment selected as the question target is an event that occurs only once in the question target section Tq, the question may be made using only that event. For example, if there is only one event of lighting the lighting in the question target section Tq, it may be queried as "Where did you turn on the electricity?"

  However, in the case of Example 1, electricity is turned on and off a plurality of times, and it is not possible to ask "Where did you turn on electricity?" In such a case, it is necessary to express the event segment to be questioned in a form that is easy to recall in combination with other event segments.

  In the first embodiment, it is assumed that sensors attached to an entrance door, a refrigerator door, a sofa, and a television are used as supplementary information for expressing in an easily recallable form. Event segments used as supplementary information are candidates for event segments that occur at the same time as the event segment to be questioned or occur within the front and rear Ta. Ta is a preset threshold value. In the first embodiment, Ta is 5 minutes. Here, one cell in FIG. 9 is Ta (5 minutes). In the case of the first embodiment, the event segments 6, 7, and 8 that have the same or continuous relationship with the event segment selected as the question target are candidates for the supplemental information event segment. Here, in the event pattern composed of a plurality of events as described above, the time series relationship between the two event segments constituting the event pattern is expressed as “A → B” when event B occurs after event A, A case where event A occurs after event B is described as “B → A”, and a case where event B occurs simultaneously with event A is described as “A = B”.

  Event segment 6 using event segment 6 as supplemental information The event pattern “door.acc.change→device1.lumi.keep@data1” occurs once in the question target section, whereas event segment 7 or 8 is used as supplemental information. The device1.lumi.keep@data1=fridege.acc.change event pattern to be used occurs twice in the question target section. The closer the event pattern appears to 1, the more supplemental information is selected.

The question target selection and the supplementary information selection are expressed as functions as follows.
First, the question target is selected. When the segment number is i, the system importance is I _system (i), and the importance based on the segment length is I _length (i), i is determined to satisfy the following: Select a segment as a question target (question target selection function). Note that argmax () indicates a variable (i in this case) when the value in () is maximum.
argmax (I _system (i) + I _length (i)) (1)

For supplementary information selection, the segment number is i, the importance based on the event appearance frequency is I _frequency (i), and the negative value of the event appearance count cnt is substituted. For example, the segment is two times the number of occurrences of an event pattern I _frequency (i) is -2, I _frequency (i) of the segment is once the number of occurrences of the event pattern is -1. At this time, i satisfying the following is obtained, and the i-th segment is selected as supplementary information (supplemental information selection function).
argmax (I _frequency (i)) (2)

  At this time, the question target selection function and the supplemental information selection function are not simple linear sums, and may be summed by setting weights, for example, by increasing the influence of system importance, or as non-linear sums. Good. The importance used is not only the system importance, the importance based on the segment length, and the importance based on the event appearance frequency, but also the user importance, the importance based on the elapsed time from the event occurrence, and the complexity of the question target Any one of the importance based on the importance, the importance based on the deviation from the normal value, and the importance based on the data intensity may be used. Various importance levels may be included in both functions.

  After the question target selection and the supplementary information selection are thus performed, a question sentence is generated. The question sentence is generated by synthesizing the question target sentence and the auxiliary sentence.

In the description of FIG. 9, the event segment 1 (one kind of event pattern) consisting of two or more consecutive events of the same type is selected in the question target selection, but one event may be selected, or the event segment Other event patterns may be selected.
Further, in the supplemental information selection, event segments 6, 7, and 8 consisting of one event are selected, but an event pattern other than the event segment may be selected.

FIG. 10 is an example of the question target sentence table.
The question target sentence table is held in advance in the question generation unit 208, for example.
In the question target sentence table, events of question target candidates and event expressions of event patterns extracted in the preprocessing are registered. A question target sentence is associated with each event expression. For example, when a certain event is selected in the question target selection, the question target sentence associated with the event expression is used for synthesizing the question sentence.

FIG. 11 is an example of the auxiliary information sentence table.
For example, the auxiliary information sentence table is held in the question generation unit 208 in advance.
Events that can be detected are registered in advance in the auxiliary information sentence table, and an auxiliary information sentence that is used when this event is selected in supplementary information selection is associated with each event. In the auxiliary information sentence table, an auxiliary information sentence may be registered for the event pattern.

  In FIG. 11, A indicates an event or event pattern to be questioned, and B indicates an event that is auxiliary information. A conjunction based on the time series relationship between A and B is selected, and an auxiliary information sentence synthesized with a subject representing an event serving as auxiliary information is used for synthesizing a question sentence.

  For example, it is assumed that {device1.lumi.stepup, device1.lumi.keep @ data1} is selected as the event pattern A to be questioned. In this case, “where electricity was turned on” is selected as the question target sentence. Assume that door.acc.change is selected as event B, which is auxiliary information that supplements event pattern A easily. At this time, since the time-series relationship between A and B is B → A, “after opening the front door” is selected as the auxiliary information sentence. The auxiliary information sentence thus selected and the question target sentence are synthesized, and a question sentence “Where did you turn on the electricity after opening the entrance door?” Is generated.

  The supplemental information for expressing in a form that is easy to recall may be a log of the sensor 1, an operation log of the user terminal 3, or an operation log on the web. In addition, an event pattern constituted by a combination of event segments may be constituted by three or more event segments.

Returning to FIG.
(S21)
The user terminal communication means 301 of the user terminal 3 receives the question text, and the presentation means 302 presents the received question text to the user.

FIG. 12 is an example of presenting a question sentence.
The presenting means 302 presents the received question sentence as shown in FIG.
As a presentation method, text may be displayed, graphic expression may be used, and audio output may be used. When the user terminal 3 can also control the motor like a robot, it may be displayed together with the movement. Other presentation means may be used.
(S23)
An answer from the user is input to the input means 303 of the user terminal 3.

(S25)
The user terminal communication unit 301 transmits the input answer to the server device 2, and the answer transmission ends. The server device 2 receives the answer and stores it in the question / answer storage means 209. That is, an answer can be used as a label and can be associated with an event or event pattern selected as a question target.

  The system importance may be changed according to the answer storage status. That is, when the label from the user related to the event device1.lumi.keep@data1 is sufficiently collected and the accuracy of the label is increased, the system importance related to the event may be changed from 2 to 0.

  By generating a question sentence as described above, the system will ask a question together with a clue for the user to recall, so that it is considered that the user can easily recall an answer. In addition, since the user recognizes when the system detects a context in the real world, an effect of increasing familiarity with the system is also expected.

  In the first embodiment, a predetermined event or event pattern is detected as a condition for sending a question to the user. However, the condition is not limited to the event or event pattern. That is, it is only necessary to satisfy a predetermined condition as a timing for sending a question to the user. The same applies to the following embodiments.

[Example 2]
In the second embodiment, similarly to the first embodiment, a question target list is created in advance. In the first embodiment, the question target list is created after being introduced into the user environment. In the second embodiment, the question target list is already prepared when the question target list is introduced into the user environment.
The system configuration diagram, block diagram, and flowchart are the same as those in the first embodiment.
In the second embodiment, a situation is considered in which user preferences and personalities are collected in a dialogue.

FIG. 13 is an example of the question target sentence table of the second embodiment.
In the question target sentence table, for example, a question target sentence suitable for inquiring about a user's preference or personality label is registered.

  By collecting and accumulating user preferences and personalities through voice dialogue between the user and the system, it is considered that there is an effect that the system grasps the user in more detail. In addition, it is thought that there is an effect of increasing the familiarity with the system by not asking the taste and personality suddenly with the dialogue system but asking the event in the real world as the starting point. Specific contents for inquiring about the preference from the system include, for example, a preference for drinks and meals, a preference for how to spend breaks, and the like. It is assumed that such preference questions and sensor data are associated in advance.

  In this case, as the type of question to be generated, the sensor installation location, physical quantity, question that asks the verb object inferred from the event expressed by the trigger function, and the verb itself is unknown because the sensor is not installed Can be a question asking for a verb or a question asking for vague content. Specifically, the question to ask for the object is that if an event called cup.temp.up is detected, which means "the temperature sensor installed in the cup has detected a change of up", Can be estimated, but if there is no data other than the temperature of the cup, it is not possible to estimate what was poured. In such a case, a question may be asked about the purpose of the action of “drinking” as inferred from cup.temp.up, such as asking “What were you drinking? In addition, when an event called sofa.acc.change, which means "change detected by an accelerometer installed on the sofa," was detected, sitting on the sofa was detected, but other data was detected. If not, it is unclear what you were doing on the couch. In such a case, as a question for asking a verb, a question asking "What were you doing on the sofa?" Also, as a question that asks vague content, when an event called door.acc.change was detected, which means "change detected by an acceleration sensor installed on a door", Can be detected, but it is not possible to extract the impression of how the contents went out. In such a case, for example, as a question to ask about impressions and impressions, a question that asks, “It seems like you went out earlier, how was it?” Can be considered. By collecting answers to these questions, it is possible to accumulate user profile information.

  As a detailed flow, the parts (S1, S3) that constantly acquire sensor data and convert it into events are the same as in the first embodiment. Further, in the question target candidate detection / storage process, the part (S5) for constantly monitoring whether or not an event or event pattern to be questioned has occurred is the same as in the first embodiment.

  In the second embodiment, the question target candidate is used by using the system importance, the importance based on the segment length, the importance based on the elapsed time from the occurrence of the event, and the importance based on the complexity of the question object as parameters for question object selection. Select the question target.

As in the first embodiment, the system importance level is defined by three levels of 0, 1, and 2 in which the importance level for the system asking the label is set in correspondence with the question target list. There are various ways to set the system importance, but in the case of Example 2, the question that has not been queried yet is set to 2, the question that has been queried once is set to 1, and the answer for the relevant event is the threshold number of times. The obtained question is defined as 0. The system importance value setting method may be in other formats, and it may be designed to increase the system importance once every time a relevant question target is detected, or the question target is detected The system importance may be changed randomly each time.
The importance based on the segment length is the same as in the first embodiment.

  The importance based on the time elapsed since the event occurred is a parameter that reflects the decline of the person's memory intensity according to the time elapsed since the event occurred. Define to have a negative impact on recall. Here, an event that has passed n minutes is assumed to contribute to the question target selection function as −0.1 × n.

  The importance based on the complexity of the question target is a parameter that reflects the difference in ease of recall depending on the complexity of the object or verb that is the question target. For example, the purpose of verbs that include more intelligent tasks such as “write” and “examine”, comparing the object of verbs that represent actions that can be taken unconsciously, such as “drink” and “clean”, with questions that are questions This reflects the fact that the answer is more complicated and the answer is less likely to be recalled for a question whose word is a question target. It is assumed that the question target with a high degree of complexity and difficult to recall is defined as 0, the middle one is defined as 1, and the question target with a low degree of complexity and easy to recall is defined as 2 in advance.

The above is summarized as a question target selection function as follows.
argmax (I _system (i) + I _length (i) + I _lapse (i) + I _complexity (i)) (3)
Here, the system importance is I _system (i), the importance based on the segment length is I _length (i), the importance based on the elapsed time from the event occurrence is I _lapse (i), and based on the complexity of the question target _Let importance be I _complexity (i).

After selecting the question target from the question target candidates by the above question target selection function, the importance based on the event appearance frequency is calculated as in the first embodiment, the supplementary information is selected, the question sentence is synthesized, and the user terminal The question text is displayed in 3 and an answer is obtained.
The processing after selecting supplemental information is the same as in the first embodiment.

  Here, the parameters used in the question target selection and the supplemental information selection may be a part of the parameters used in the second embodiment, or may be interchanged. Parameters that are not used in the second embodiment may be added.

[Example 3]
Example 3 is an example relating to a label for vital data.
Vital data varies greatly from individual to individual, and it is necessary to determine the data to be labeled for each individual.
In the third embodiment, similarly to the first embodiment, a question target list is created in advance. As in the first embodiment, the question target list is created after being introduced into the user environment.

FIG. 14 is an example of the question target sentence table of the third embodiment.
In the question target sentence table, for example, a question target sentence table suitable for inquiring a label for vital data is registered.
The system configuration diagram, block diagram, and flowchart are the same as those in the first embodiment.
A period determined after the introduction of the system is set as a calibration period, and a classifier is created for data obtained in the calibration period. The vital data to be input may be electrocardiogram, pulse group, pulse, respiratory rate, myoelectricity, skin potential, ocular potential, brain wave, and the like. The classifier classifies the degree of fatigue, stress, emotion, etc. from the input vital data. The behavior of the data classified by the classifier is created as a question target list. The above is not described in the flowchart as preprocessing.

In order to give a correct answer label to the result classified by the classifier created in the preprocessing, a question is generated.
As a detailed flow, the parts (S1, S3) that constantly acquire sensor data and convert it into events are the same as in the first embodiment. Further, in the question target candidate detection / storage process, the part (S5) for constantly monitoring whether or not an event or event pattern to be questioned has occurred is the same as in the first embodiment.

  In the third embodiment, the question target selection parameters are based on the system importance, the importance based on the segment length, the importance based on the elapsed time from the event occurrence, the importance based on the deviation from the normal value, and the data intensity. A question target is selected from question target candidates using importance.

  Here, the importance based on data intensity is a parameter that sets higher importance for sensor data with higher intensity. For example, when calculating the data average of a specified section with a predetermined window width, 1 is set when the data greatly deviates from the average of the data obtained during the calibration period, and 0 is set when the average behavior is obtained. . The threshold value for classifying the two is determined from data obtained during the calibration period.

  The importance based on the degree of deviation from the normal value is a parameter that sets the importance higher for the waveform of the sensor data with a lower appearance frequency. An abnormal value detection is performed on vital data using a general abnormal value detection algorithm, and 1 is output when detected, and 0 is output when it is not detected.

  Examples of abnormal value detection algorithms include those that detect specific patterns as abnormal values, and those that detect unusual patterns as abnormal values.However, the abnormal value detection algorithm that uses data intensity differs from the normal value. When used for the importance based on, the importance based on the data intensity is equivalent.

The above is summarized as a question target selection function as follows.
argmax (I _system (i) + I _length (i) + I _lapse (i) + I _anomaly (i) + I _intensity (i)) (4)
Here, the system importance is I _system (i), the importance based on the segment length is I _length (i), the importance based on the elapsed time from the event occurrence is I _lapse (i), and the deviation from the normal value The importance based on I _anomaly (i) and the importance based on data intensity is I _intensity (i).

  After selecting the question target from the question target candidates by the above question target selection function, the importance based on the event appearance frequency is calculated as in the first embodiment, the supplementary information is selected, the question sentence is synthesized, and the user terminal Display the question text and get an answer.

  The processing after selecting supplementary information is the same as in the first embodiment. For example, when the fatigue level is detected to be very strong, cup.acc.change and cup.temp.up are once in the cup after detecting strong fatigue. Since it was detected only as a supplementary information, a question sentence “Are you tired before drinking something warm?” Is generated.

  In addition, the parameter used by question object selection and supplement information selection may use a part of parameter used in Example 3, and may be replaced. Parameters that are not used in the third embodiment may be added.

[Example 4]
Example 4 is an example relating to a label for traveling of a wheelchair.
In the fourth embodiment, similarly to the first embodiment, a question target list is created in advance. Similarly to the second embodiment, it is assumed that a question target list is already prepared when it is introduced into the user environment.

FIG. 15 is an example of the question target sentence table of the fourth embodiment.
In the question target sentence table, for example, a question target sentence suitable for inquiring a label for wheelchair travel is registered.

FIG. 16 is a block diagram of the fourth embodiment.
The server device 2 further includes an event index unit 210 that outputs a message to the user terminal 3 when the event occurs in order to impress the user on a specific event that is a question target to be particularly inquired to the user. It is assumed that the event that activates the event index unit 210 is defined in the question target list in advance.

  It is assumed that an event that outputs a message is detected by the event index unit 210, and the message is output by the user terminal 3. The form of the message may be voice output or text output, or other output form. For example, when a sudden stop event is defined in the question target list, when a sudden stop occurs, a message “Wow!” Is output from the user terminal 3 immediately after the event is detected. By outputting a message for a specific event in this way, it is considered that the event for which the message is output is impressed by the user, and the memory is strengthened so that it can be easily recalled. Here, the message to be output may be associated with an event in advance, or the message content may be dynamically changed according to the strength of a signal related to the event.

  For example, trigger functions that determine operations such as forward, backward, right turn, left turn, right turn (right U-turn), left turn (left U-turn), stop, sudden stop, detour are set in advance, and events for these Wheelchair.innertial.forward, wheelchair.innertial.backward, wheelchair.innertial.turn-right, wheelchair.innertial.turn-left, wheelchair.innertial.u-turn-right, wheelchair.innertial.u-turn-left , Wheelchair.innertial.normal-stop, wheelchair.innertial.sudden-stop, wheelchair.innertial.detour are output.

  Here, “innertial” refers to an inertial sensor, and these operations are determined by an acceleration sensor and a gyro sensor which are inertial sensors. Reasons for making U-turns, detours, and sudden stops for left-right U-turns and detours, which are considered to be related to barriers in the city, and sudden stop events that are considered to be dangerous. Think about a scene that asks. By collecting and accumulating labels on barriers and dangers through voice dialogue between the user and the system, it is considered that a barrier-free map reflecting the opinions of the wheelchair users can be created. Assume that questions and events related to barriers and dangers are associated in advance.

  In this case, as a type of question to be generated, a question asking why the event represented by the sensor installation location, physical quantity, and trigger function has occurred can be considered. Specifically, the reason for asking the reason is that if an event of wheelchair.innertial.sudden-stop was detected, which means that a sudden stop was detected by an inertial sensor installed on the wheelchair, Can be estimated, but the reason for the sudden stop cannot be estimated. In such a case, a question may be asked about the reason for the action of “sudden stop” determined from wheelchair.innertial.sudden-stop, such as asking “What did you do?” By collecting answers to these questions, it becomes possible to accumulate information on barriers and dangers of wheelchair users.

  As a detailed flow, the parts (S1, S3) that constantly acquire sensor data and convert it into events are the same as in the first embodiment. Further, in the question target candidate detection / storage process, the part (S5) for constantly monitoring whether or not an event or event pattern to be questioned has occurred is the same as in the first embodiment.

  When an event set in advance so that the event index means 210 is triggered occurs, a preset message is output.

  In the fourth embodiment, as the target selection parameters, the system importance, the user importance, the importance based on the segment length, the importance based on the elapsed time from the event occurrence, the importance based on the deviation from the normal value, data The question target is selected from the question target candidates using the importance based on the strength and the importance based on the event index.

  Here, the user importance is an importance viewed from the viewpoint of the user, and is defined in advance together with the event. For example, when not only an inertial sensor but also the system of the present embodiment has GPS, a user importance level is defined that location A is a place that is meaningful to the user and place B is a place that is not meaningful to the user. It shall be. In this case, if the user stays at the location A, 1 is returned as the user importance level, and if the user stays at the location B, 0 is returned as the user importance level. Since it is more meaningful for the user to ask about the event related to the place A than to ask about the event related to the place B, it is considered that there is an effect that the user can easily recall.

  The importance based on the event index is a parameter reflecting the effect of the message output from the system. In the case of this embodiment, 1 is assigned to an event in which a voice message is output from the system, and 0 is assigned to an event that is not. Shall be given.

The above is summarized as a question target selection function as follows.
argmax (I _system (i) + I _user (i) + I _length (i) + I _lapse (i) + I _anomaly (i) + I _intensity (i) + I _index (i)) (5)
Here, the system importance is I _system (i), the user importance is I _user (i), the importance based on the segment length is I _length (i), and the importance based on the elapsed time from the event occurrence is I _lapse ( i), the importance based on the deviation from the normal value is I _anomaly (i), the importance based on the data intensity is I _intensity (i), and the importance based on the event index is I _index (i).

After selecting the question target from the question target candidates by the above question target selection function, the importance based on the event appearance frequency is calculated as in the first embodiment, the supplementary information is selected, the question sentence is synthesized, and the user terminal Display the question text and get an answer.
The processing after selecting supplemental information is the same as in the first embodiment.

  In addition, the parameter used by question object selection and supplement information selection may use a part of parameter used in Example 4, and may be replaced. Parameters that are not used in the fourth embodiment may be added.

  In addition, the number of importance levels, the function configuration, and the like can be variously modified without departing from the spirit of the present invention. For example, the number of importance levels may not be three, or may be a continuous value rather than discrete. In each embodiment, the question target selection function and the supplementary information selection function are designed so that each parameter affects linearly, but a non-linear element may be added or a weight may be given.

  A computer program for causing a computer to function as the server apparatus 2 can be recorded on a computer-readable recording medium such as a semiconductor memory, a magnetic disk, an optical disk, a magneto-optical disk, or a magnetic tape, and a communication network such as the Internet can be used. And can be widely distributed.

DESCRIPTION OF SYMBOLS 1 Sensor 2 Server apparatus 3 User terminal 101 Sensing means 102 Sensing communication means 201 Server communication means 202 Sensor data log storage means 203 Event detection means 204 Event log storage means 205 Question target detection means 206 Question target candidate temporary storage means 207 Question transmission timing Detection means 208 Question generation means 209 Question / answer storage means 210 Event index means 301 User terminal communication means 302 Presentation means 303 Input means

Claims (6)

A method of operating a server device,
The server device inputs an event expressed by the location of the sensor around the user, the type of the sensor, and the type of trigger detected by the sensor data output from the sensor;
The server device detecting a predetermined condition as an opportunity for sending a question to the user;
The server device is an event or event pattern that is input in the question target section until the condition as the trigger is detected, and the same event or event pattern that is previously associated with the system importance is the question target candidate And calculating a value for selection using the corresponding system importance for each of the plurality of question target candidates, and selecting a question target candidate corresponding to the maximum corresponding value as a question target. A method of operating a server device, comprising: The operation method of the server device according to claim 1, wherein the server device generates a question sentence including a question target sentence associated with the question target in advance. The server device detects an event or event pattern input simultaneously or continuously with the question target, calculates a value indicating a low frequency of appearance of each of the plurality of events or event patterns, and has the lowest frequency of appearance The server device operating method according to claim 1, wherein an event or an event pattern corresponding to the value indicating is selected. When the server device stores an auxiliary information sentence corresponding to the one event or event pattern for at least one event or event pattern, the event or event selected by the value calculated using the appearance frequency The server device operating method according to claim 3, wherein, when a pattern corresponds to the one event or event pattern, the question sentence is generated using the auxiliary information sentence. A server device,
Means for inputting an event expressed by the location of the sensor around the user, the type of the sensor, and the type of trigger detected by the sensor data output from the sensor;
Means for detecting a predetermined condition as a trigger for sending a question to the user;
The event or event pattern input to the question target section until the condition as the trigger is detected, and the same event or event pattern previously associated with the system importance is detected as a question target candidate, A means for calculating a value for selection using a corresponding system importance for each of the plurality of question target candidates, and selecting a question target candidate corresponding to the maximum value as a question target. Server device. A computer program for causing a computer to function as the server device according to claim 5.

Images