JP2015152948A - Life log recording system and program thereof - Google Patents

Abstract

PROBLEM TO BE SOLVED: To easily give a label significant for a user to sensor data.SOLUTION: A life log recording system includes a sensor device 100, a server 200, a sensor data analyzer 300, and a robotic device 400. The sensor device 100 includes means for acquiring sensor data about a user, and the server 200 includes means for storing the sensor data. The sensor data analyzer 300 includes: analysis means which constructs speech for drawing out information specifying user's action if the user's action cannot be specified by referring to learning data; and means which determines whether or not the user is in a state of being capable of having a conversation with the robotic device 400. The robotic device 400 includes means for having a conversation with the user. The sensor data analyzer 300 further includes means which transmits label information giving a meaning to the sensor data, to the server on the basis of user's response.

Description

  The present invention relates to a life log recording system for labeling life logs obtained from various sensors based on a voice dialogue between a robot and a user, and a program therefor.

  In recent years, various sensors have been reduced in size and power consumption, and sensor data as life logs can be easily collected. The collected life log is used to improve the usability of services and provide various services that predict behaviors in order to understand user behavior and provide better services ( For example, see Patent Document 1 and Non-Patent Documents 1 and 2).

  On the other hand, an automatic response system using voice has been studied for a long time, and is widely used for support of a telephone support desk, car navigation system, interaction with a robot, and the like (for example, see Non-Patent Document 3). ).

  In addition, the spread of small humanoid robots that can communicate with people and can be connected to a network has begun to spread, and it is now possible to place them at home (for example, non-patented). Reference 4).

JP 2011-108055 A

Kenji Mase, “Experience Sharing Communication Based on Interaction”, IEICE Technical Report, IEICE, January 2006, PRMU 2005-143 / HIP 2005-113, p. 37-42 Haruma Noma et al., “Development of Small Wearable Devices for Measuring Individual Behavior in Sensor Networks”, Research Report of Information Processing Society of Japan, Information Processing Society of Japan, July 2007, 2007-UBI-15, p. 29-34 Komaya Kazunori et al. “Spoken dialogue system using speech act level information to interpret user utterances”, Transactions of Information Processing Society of Japan, Information Processing Society of Japan, December 2011, Vol. 52, No. 12, p. . 3374-3385 Osamu Sugiyama et al., “Extraction of Speech and Gesture Assignment Patterns for Communication Robots and Proposal of an Advanced Development Method”, IEICE Transactions, IEICE, January 2012, J95- A (1), p. 46-59

  As described above, in recent years, various sensor data can be easily collected. However, the sensor data has a low utility value if it is merely a sequence of numerical values.

  Sensor data can be used as useful data by giving a label and giving some meaning, but it has been a very laborious task to give a label to sensor data. . For example, when sensor data is very characteristic data, there was a case where automatic labeling was possible, but such cases are rare, and in many other cases, manual labeling is performed. It was necessary and very labor intensive.

  An object of the present invention made in view of such a point is to provide a life log recording system and a program thereof capable of easily giving a meaningful label to a user to sensor data.

  In order to solve the above problems, a life log recording system according to the present invention includes a sensor device, a server, a sensor data analysis device, and a robot device, and the sensor device includes means for acquiring sensor data relating to a user of the sensor device. The server has means for receiving the sensor data from the sensor device and means for storing the sensor data, and the sensor data analysis device has means for receiving the sensor data from the server; If the learning storage means stores learning user behavior as learning data and the sensor data includes data that cannot identify the user behavior even with reference to the learning data, the user behavior is identified. From the analysis means that constitutes a conversation for extracting information to be performed from the user and the sensor data, the user Means for determining whether or not it is in a state of being able to talk to a device, and means for transmitting the conversation to the robot device when the user is in a state of being able to talk to the robot device, the robot The apparatus includes means for performing the conversation with the user, and means for transmitting a user response to the conversation to the sensor data analysis apparatus. The sensor data analysis apparatus further includes: It has a means to transmit the label information which gives meaning to the sensor data to the server.

  In order to solve the above problems, a program according to the present invention is a program for operating a life log recording system including a sensor device, a server, a sensor data analysis device, and a robot device, and the sensor device includes the sensor device. Obtaining sensor data relating to the user of the user, causing the server to receive the sensor data from the sensor device, and storing the sensor data, and causing the sensor data analysis device to When the sensor data received from the server and the received sensor data includes data that cannot identify the user's behavior even if the learned user's behavior is referred to, information for identifying the user's behavior is provided to the user. Constructing a conversation to be drawn from the sensor data Performing a step of determining whether or not a user is in a state of being able to talk to the robot device, and a step of transmitting the configured conversation to the robot device when the user is in a state of being able to talk to the robot device. And causing the robot apparatus to execute the conversation with the user and transmitting the user response to the conversation to the sensor data analysis apparatus, and further causing the sensor data analysis apparatus to further execute the user response. Based on the above, a step of transmitting label information giving meaning to the sensor data to the server is executed.

  According to the present invention, a label that is meaningful to the user can be easily given to the sensor data through a natural dialogue, and a wider use of the life log can be expected because the label is attached. In addition, by having a natural conversation through the robot, the direct impression that the labeling work is performed can be eased, and continuity can be expected.

It is a block diagram which shows the structural example of the life log recording system which concerns on one Embodiment of this invention. It is a figure which shows the processing flow of the life log recording system which concerns on one Embodiment of this invention. It is an example of the data accumulate | stored in a data storage part. It is an example of the data memorize | stored in a learning data memory | storage part. It is an example of the data which a robot action transmission part transmits. It is a block diagram which shows the structural example of the life log recording system which concerns on Example 5 of this invention. It is a block diagram which shows the structural example of the life log recording system which concerns on Example 7 of this invention.

  Hereinafter, embodiments according to the present invention will be described with reference to the drawings.

  FIG. 1 is a block diagram showing a configuration example of a life log recording system according to an embodiment of the present invention. As shown in FIG. 1, the life log recording system according to an embodiment of the present invention includes a sensor device 100, a server 200, a sensor data analysis device 300, and a robot device 400.

  The sensor device 100 is a device that a user carries around on a daily basis. The sensor device 100 includes an acceleration sensor 101, a position sensor 102, a storage unit 110, and a data transmission unit 120.

  The acceleration sensor 101 detects the acceleration of the user carrying the sensor device 100 and stores the detected acceleration data in the storage unit 110 in synchronization with the time information. The acceleration data is used to obtain user movement information, step count information, and the like.

  The position sensor 102 is a sensor using, for example, GPS (Global Positioning System), and detects position data of a user carrying the sensor device 100. The position sensor 102 stores the detected position data in the storage unit 110 in synchronization with the time information.

  Hereinafter, data relating to a user acquired by various sensors included in the sensor device 100 such as acceleration data and position data will be collectively referred to as “sensor data”.

  The storage unit 110 stores sensor data detected by the acceleration sensor 101 and the position sensor 102.

  The data transmission unit 120 transmits the sensor data stored in the storage unit 110 to the server 200.

  The server 200 includes a data reception unit 210, a data storage unit 220, a data request reception unit 230, and a data transmission unit 240.

  The data receiving unit 210 receives user sensor data from the sensor device 100. Further, the data receiving unit 210 receives label information for giving meaning to the sensor data from the sensor data analyzing apparatus 300.

  The data storage unit 220 stores user sensor data received from the sensor device 100. In addition, the data storage unit 220 labels the stored sensor data based on the label information received from the sensor data analysis device 300. The label includes the type of action, the name of the place, information on the person who was together, means of transportation, and the like. Information including sensor data, label information, and time information is treated as a life log.

  The data request reception unit 230 receives a user's request for sensor data from the sensor data analysis device 300.

  The data transmission unit 240 transmits user sensor data to the sensor data analysis device 300 in response to a request from the sensor data analysis device 300.

  The sensor data analysis device 300 includes a data request unit 310, a data reception unit 320, a data analysis unit 330, a learning data storage unit 340, an utterance content configuration unit 350, a robot action transmission unit 360, and a response result reception unit. 370, a response result analysis unit 380, and a label information transmission unit 390.

  The data request unit 310 requests user sensor data from the server 200.

  The data receiving unit 320 receives user sensor data from the server 200 and transmits it to the data analysis unit 330.

  The data analysis unit 330 refers to the learning data stored in the learning data storage unit 340, analyzes the sensor data received from the server 200, and determines whether the user's action can be specified. If the data analysis unit 330 determines that the user's action can be specified, the label information transmission unit 390 causes the server 200 to transmit label information for giving meaning to the sensor data, and the learning data storage unit 340 Store the label information. If the data analysis unit 330 determines that the user's action cannot be specified, the utterance content configuration unit 350 configures a conversation or a robot action including a question for extracting label information from the user.

  The learning data storage unit 340 stores learned user behavior as learning data.

  The utterance content configuration unit 350 configures a conversation for extracting information from the user. Further, the robot action 400 is configured to be performed by the robot apparatus 400 in accordance with the conversation.

  The robot action transmission unit 360 transmits the conversation configured by the utterance content configuration unit 350 and the operation information of the robot configured based on the conversation to the robot apparatus 400.

  The response result receiving unit 370 receives the user response result from the robot apparatus 400 and transmits it to the response result analyzing unit 380.

  When the response result analysis unit 380 analyzes the user response result received from the response result reception unit 370 and determines that the user's behavior can be specified, the response result analysis unit 380 gives the label information transmission unit 390 a meaning for the sensor data. The label information is transmitted to the server 200, and the label information is stored in the learning data storage unit 340. If the response result analysis unit 380 determines that the user's behavior cannot be identified even after analyzing the user's response result, the utterance content configuration unit 350 includes a conversation or a robot including a question for extracting label information from the user. Have the action configured again.

  The label information transmission unit 390 transmits label information for giving meaning to the sensor data to the server 200.

  The robot apparatus 400 includes a robot action reception unit 410, a robot action unit 420, a response input unit 430, and a response result transmission unit 440. The robot apparatus 400 is, for example, a small humanoid robot, but may be a virtual robot such as an agent.

  The robot action receiving unit 410 receives robot action information from the sensor data analysis device 300.

  The robot action unit 420 performs utterances and operations based on the received robot action information.

  The response input unit 430 obtains an affirmative / negative response based on speech from the user or an operation such as whispering or swinging.

  The response result transmission unit 440 transmits the acquired response information to the sensor data analysis device 300.

  The block diagram of the life log recording system shown in FIG. 1 is merely an example, and a plurality of components can be combined or divided into one. Each component of each device may be included in another device. For example, the data analysis unit 330 and the response result analysis unit 380 may be integrated and configured as one analysis unit. Further, for example, part or all of the components of the sensor data analysis apparatus 300 may be included in the server 200.

  FIG. 2 is a diagram showing a processing flow of the life log recording system according to the embodiment of the present invention. Hereinafter, the processing flow of the life log recording system according to the embodiment of the present invention will be described with reference to FIG.

  In the sensor device 100, the acceleration sensor 101 always acquires user acceleration data (step S101) and temporarily stores it in the storage unit 110 (step S102). Further, the position sensor 102 always acquires user position data (step S103) and temporarily stores it in the storage unit 110 (step S104). The data transmission unit 120 transmits (uploads) the acceleration data and the position data stored in the storage unit 110 to the server 200 at a predetermined time determined every day, for example (step S105).

  In the server 200, the data receiving unit 210 receives sensor data from the sensor device 100 and receives label information from the sensor data analysis device 300 (step S201). The data receiving unit 210 stores the received sensor data and label information in the data storage unit 220 (step S202). An example of data stored in the data storage unit 220 is shown in FIG. FIG. 3A is an example of acceleration data labeled with label information, and FIG. 3B is an example of position data labeled with label information. On the other hand, when the data request receiving unit 230 receives a request from the sensor data analysis device 300 (step S203), the data transmission unit 240 reads the requested data from the data storage unit 220 (step S204), and the sensor data analysis device. It transmits to 300 (step S205).

  In the sensor data analysis apparatus 300, the data request unit 310 requests the server 200 for newly accumulated data periodically, for example, every hour (step S301). The data receiving unit 320 receives the data requested by the data requesting unit 310 from the server 200 (step S302). The data analysis unit 330 refers to the learning data stored in the learning data storage unit 340 (step S303), analyzes the data received from the server 200 (step S304), identifies the user's action, and converts the data into data. It is determined whether or not labeling is possible (step S305). An example of the learning data stored in the learning data storage unit 340 is shown in FIG.

  When analyzing the position data, the data analysis unit 330, for example, the moving speed, the required time when moving from a certain point to a certain point, the presence or absence of dwelling (the positional deviation is within a threshold value, a certain time or more, etc.) Analysis is performed based on the deviation (how far from the average position) and the trajectory itself. For example, if the data analysis unit 330 stays within a radius of 5 m from a learned place for a certain time or longer, the data analysis unit 330 determines that the place is present and performs labeling. In addition, when analyzing the acceleration data, the data analysis unit 330 performs analysis based on, for example, the differential value of the waveform or the similarity of the waveform itself. For example, the data analysis unit 330 determines that the user has taken an action such as walking / running / moving by car or train from the similarity compared to the learned waveform, and performs labeling. Do. The above description shows some examples in which the data analysis unit 330 labels sensor data, and the items to be labeled and the labeling method are not limited to these. The data analysis unit 330 can label sensor data for any item by any method.

  For example, when the user deviates from the learned steady movement route, the data analysis unit 330 cannot automatically perform the labeling because the reason is not known. Further, for example, when the user can specify that the user has eaten at a certain store, the data analysis unit 330 can specify the name of the store but cannot specify the food that was eaten. In this case, the data analysis unit 330 can automatically label up to the store name, but cannot automatically label the eaten food.

  If the data analysis unit 330 determines Yes in step S305, the data analysis unit 330 labels the data (step S306), and causes the label information transmission unit 390 to transmit the label information to the server 200 (step S307). When data analysis unit 330 determines No in step S305, that is, when some change is detected in the sensor data, but labeling cannot be performed even with reference to the learning data stored in learning data storage unit 340. Causes the utterance content configuration unit 350 to configure a conversation or a robot action including a question for identifying the user's action from the user and extracting label information for the change in the data (steps S308 and S309). When the data analysis unit 330 constructs a question for extracting label information from the user, for example, the question relating to the desired information is included in the question (for example, when it is desired to confirm whether or not the miso ramen has been eaten, “miso ramen” In the question).

  When the data analysis unit 330 determines from the sensor data acquired from the server 200 that the user is near the robot apparatus 400 and is in a state where the user can interact with the robot apparatus 400, the data analysis unit 330 sends an utterance content configuration unit to the robot action transmission unit 360. The utterance information constituted by 350 and the movement information of the robot constituted based on the utterance information are transmitted to the robot apparatus 400 (step S310). An example of data transmitted by the robot action transmission unit 360 is shown in FIG.

  The response result receiving unit 370 receives the response result of the user from the robot apparatus 400 (step S302), and this time the response result analyzing unit 380 executes processing corresponding to steps S303 to S305 executed by the data analyzing unit 330. To do. The response result analysis unit 380 analyzes the user response received by the response result reception unit 370, and determines that labeling is possible (Yes in step S305), performs labeling on the data (step S306), The information transmitting unit 390 causes the server 200 to transmit label information that gives meaning to the sensor data (step S307), and the label information is registered in the learning data storage unit 340 (step S311). The response result analysis unit 380 determines whether labeling is sufficient (step S312). If the response result analysis unit 380 determines Yes in step S312, the response result analysis unit 380 ends the process. When the response result analysis unit 380 determines No in step S312, for example, when it is determined that further conversation is necessary, for example, when it is desired to add additional label information, the process proceeds to step S308 again, and the utterance is made. Let the content composition unit 350 compose further conversations.

  In the robot apparatus 400, the robot action reception unit 410 receives robot action information from the sensor data analysis apparatus 300 (step S401). The robot action unit 420 performs utterances and actions based on the received robot action information (step S402). The response input unit 430 acquires an affirmative / negative response due to speech from the user, an operation such as whispering or swinging (step S403). The response result transmission unit 440 transmits the acquired response information to the sensor data analysis device 300 (step S404).

  In the above configuration, the data from the sensor device 100 is temporarily stored in the server 200, and then the sensor data analysis device 300 acquires the data stored in the server 200. The apparatus 300 may receive data from the sensor apparatus 100 in parallel, and the sensor data analysis apparatus 300 may analyze the data.

  Hereinafter, the processing after the sensor data analysis apparatus 300 acquires the sensor data from the server 200 will be described in detail with specific examples (Examples 1 to 9). In the following description, it is assumed that the robot apparatus 400 is a small humanoid robot at home. In addition, processing overlapping in each embodiment will be omitted as appropriate.

[Example 1]
The data analysis unit 330 determines from the user position data acquired from the server 200 that the user has stayed at a certain place around noon for a long time. However, the data analysis unit 330 has found that even when the learning data stored in the learning data storage unit 340 refers to the learning data, the user often has lunch at that time. Therefore, the location could not be specified.

In this case, the data analysis unit 330 uses the utterance content composition unit 350 to search for information necessary for labeling using the fact that the possibility of lunch is high and the location is unknown as key information.
"Did you go to eat lunch around xx?"
"What kind of shop?"
"What did you eat?"
"it was delicious?"
Make a conversation. In addition, the data analysis unit 330 causes the utterance content composition unit 350 to utter the question while the robot apparatus 400 behaves like a neck when listening to the first three questions in the above example. In addition, the last question is configured such that the robot apparatus 400 utters the question while performing an action of getting excited.

  Furthermore, the data analysis unit 330 is configured so that the user can interact with the robot apparatus 400 based on the sensor data acquired from the server 200, such as when the user returns home and eats dinner while being near the robot apparatus 400. When it is determined that there is, the robot action transmission unit 360 causes the robot apparatus 400 to transmit information in order. When the data analysis unit 330 determines from the sensor data acquired from the server 200 that, for example, in the evening time zone, the user is at the table and moves his hand with little significant movement. The user can be identified as having dinner at home.

  The robot apparatus 400 asks the user one question at a time using the robot action unit 420 based on the information received by the robot action receiving unit 410. The response result transmission unit 440 transmits the response obtained by the response input unit 430 from the user to the sensor data analysis device 300.

  The response result analyzing unit 380 analyzes the user response received from the robot apparatus 400 by the response result receiving unit 370. At this time, if the desired response cannot be obtained, the response result analyzing unit 380 configures the changed utterance content in the utterance content configuring unit 350, and the robot action transmitting unit 360 includes the changed utterance content and The robot action is transmitted to the robot apparatus 400.

In the above example, for example, when the response to the first question is affirmative, the response result analysis unit 380 causes the robot apparatus 400 to continue the utterance as originally planned. If the response to the question is negative, for example,
"So what were you doing?"
Is switched to such a question, and the robot apparatus 400 is uttered.

  In addition, for example, for the second and third questions, the response result analysis unit 380 finds nouns included in the user's response by morphological analysis, and determines whether the words are likely to be appropriate as answers. Thus, it can be determined whether or not the response is desired.

  When the response result analysis unit 380 can obtain from the user's response information that the user has eaten lunch, the name of the store where the user stayed, what the user has eaten, and the evaluation of the food These are the label information corresponding to the change of the corresponding sensor data, and the label information transmitting unit 390 transmits the label information to the server 200. The server 200 stores the label information received by the data receiving unit 210 in the data storage unit 220.

  The information (life log) stored in these data storage units 220 can be used at a later date to respond to a question such as “Where is a good place for dumplings before?” .

  The life log recording system according to the present embodiment is frequently used at the beginning of the user because learning according to the user is not sufficiently performed, so the frequency and items to ask questions increase. Such behavior can be automatically labeled without asking the user. As a result, the number of times that the robot apparatus 400 asks the user a confirmation question decreases.

  In this embodiment, a case has been described in which a change in certain sensor data is used as a trigger and the reason is heard by a question-answering conversation with a fixed task. Alternatively, a word such as “lunch” may be set as a chat keyword in a conversation system for performing chat, and a response serving as label information may be extracted from a response from the user in the chat. Alternatively, chatting and task-oriented conversation may be interlaced to start listening in a more natural conversation.

[Example 2]
The data analysis unit 330 indicates that the user has performed intense movement similar to jogging in a situation (different time zone and location) different from that when the user is jogging from the sensor data of the user acquired from the server 200. It was judged.

In this case, the data analysis unit 330 sends the utterance content configuration unit 350 to
“It seems like I was running around xx, but was there anything?”
Make a conversation. The data analysis unit 330 is in a state where it can interact with the robot apparatus 400 from the data acquired from the server 200, for example, when the user comes back from jogging that is regularly performed and is taking a breath by the robot apparatus 400. When it is determined, the robot action transmission unit 360 causes the robot apparatus 400 to transmit the utterance content.

  When the response result analysis unit 380 obtains a response from the user such as “I was about to run late for the train”, the response result analysis unit 380 indicated that it ran for the section of the sensor data, The label information transmitting unit 390 transmits label information to the server 200 so that “

  The above contents can be used as a service that uses a life log at a later date to use the labeled life log in order to give an answer such as how many minutes it takes to run when answering the time from a certain place to the station.

[Example 3]
From the sensor data of the user acquired from the server 200, the data analysis unit 330 records that the user stays somewhere at this time and eats lunch on a normal weekday. We were unable to automatically find evidence of

In this case, the data analysis unit 330 sends the utterance content configuration unit 350 to
"What did you do for lunch today?"
Make a conversation. From the data acquired from the server 200, the data analysis unit 330 is in a state where the user is at home and is near the robot apparatus 400. For example, when the user is having dinner, the data analysis unit 330 can communicate with the robot apparatus 400. When it is determined that there is, the robot action transmission unit 360 causes the robot apparatus 400 to transmit the utterance content.

  The response result analysis unit 380 does not label a specific sensor data portion when a response such as “I did not eat” is obtained from the user, but gives a label such as “without lunch” as the label of the day. As described above, the label information transmission unit 390 transmits the label information to the server 200.

In addition, the response result analysis unit 380 receives a message from the user.
"I bought a station lunch and ate it on the train going to xx"
"The eki box called xx was delicious"
If a response such as “Lunch”, “Ekiben”, or “xx Ekiben” was delicious for the data that was moving on the train in the sensor data, the label information should be added. The information transmission unit 390 transmits label information to the server 200.

[Example 4]
The data analysis unit 330 determines that the user has visited some store from the sensor data of the user acquired from the server 200 and the learning data stored in the learning data storage unit 340. However, since the data analysis unit 330 is labeled with a plurality of different store names with respect to similar changes in sensor data, the data analysis unit 330 cannot automatically narrow down.

In this case, the data analysis unit 330 sends the utterance content configuration unit 350 to, for example,
“Which store did you stop by today?”
Make a conversation. When the data analysis unit 330 determines from the data acquired from the server 200 that the user is at home, is in a state of being calm by the robot apparatus 400, and is in a state of being able to interact with the robot apparatus 400, The robot action transmission unit 360 causes the robot apparatus 400 to transmit the utterance content.

  The response result analysis unit 380 can increase the accuracy in obtaining a response from the user and labeling it.

[Example 5]
In this embodiment, the sensor device 100 includes a vital sensor 103 in addition to the acceleration sensor 101 and the position sensor 102 as shown in FIG. The vital sensor 103 can detect the heart rate, blood pressure, blood oxygen concentration, etc. of the user carrying the sensor device 100. The detected heart rate data and the like are stored in the data storage unit 220 in the same manner as the acceleration data and the position data.

  The data analysis unit 330 did not find any change in the user acceleration data and the position data acquired from the server 200, but the heart rate suddenly increased despite the fact that the data analysis unit 330 did not make any special movement. I found that there was a time.

In this case, the data analysis unit 330 sends the utterance content configuration unit 350 to, for example,
"What happened when you were in xxx?"
Make a conversation. When the data analysis unit 330 determines from the data acquired from the server 200 that the user is at home, is in a state of being calm by the robot apparatus 400, and is in a state of being able to interact with the robot apparatus 400, The robot action transmission unit 360 causes the robot apparatus 400 to transmit the utterance content.

The response result analysis unit 380 receives a response from the user.
"Meet xx teacher who was mad at junior high school"
When the response is obtained, the label information transmitting unit 390 is caused to transmit the label information to the server 200 so that the label information “Meet xx teacher” is given as the label information. However, such a unique example may not be stored in the learning data storage unit 340.

The recorded life log is "Where and when did you meet Dr. xx?"
It can be used to answer such questions.

[Example 6]
From the sensor data of the user acquired from the server 200, the data analysis unit 330 found that the user normally jogged about 10 km and only ran for 3 km today.

In this case, the data analysis unit 330 sends the utterance content configuration unit 350 to, for example,
“Why was jogging short today?”
Make a conversation. When the data analysis unit 330 determines from the data acquired from the server 200 that the user is at home, is in a state of being calm by the robot apparatus 400, and is in a state of being able to interact with the robot apparatus 400, The robot action transmission unit 360 causes the robot apparatus 400 to transmit the utterance content.

  When a response such as “it was raining” is obtained from the user, the response result analysis unit 380 determines a label that is jogging as a life log of the corresponding part and relates to a condition that the weather is raining. The label information transmitting unit 390 transmits the label information to the server 200 so that the label is also added.

  The response result analysis unit 380 further learns that weather information is also used as judgment data as learning data, and when sensor data that can be given a “jogging” action label is found, If the weather information is acquired from the Web and the weather is raining, it may be determined that the distance is a normal value even if the distance is short, and the number of questions to the user may be reduced.

[Example 7]
In this embodiment, as shown in FIG. 7, the sensor device 100 includes a camera 104 that acquires an image and a microphone 105 that acquires sound in accordance with the acceleration sensor 101 and the position sensor 102. The video that is always acquired by the camera 104 and the audio that is always acquired by the microphone 105 are stored in the data storage unit 220 in the same manner as the acceleration data and the position data. However, since the processing of video and audio is heavy, in normal times, analysis such as acceleration data and position data is not performed, and video and audio are only stored in the data storage unit 220. To do. Note that the camera 104 may acquire a still image instead of a video.

  The data analysis unit 330 has found a change that the user has stopped at a place that does not normally stop from the sensor data of the user acquired from the server 200. However, the data analysis unit 330 estimates that labeling cannot be performed or is insufficient only from data used in normal labeling.

  In this case, as an additional analysis, the data analysis unit 330 acquires and analyzes a video registered as other media information that is a candidate for an information acquisition source. The data analysis unit 330 analyzes the video in the corresponding time zone and finds, for example, that ramen has been captured in the video for a long time.

In this case, the data analysis unit 330 sends the utterance content configuration unit 350 to, for example,
"Did you go to eat ramen today?"
Make a conversation. When the data analysis unit 330 determines from the data acquired from the server 200 that the user is at home, is in a state of being calm by the robot apparatus 400, and is in a state of being able to interact with the robot apparatus 400, The robot action transmission unit 360 causes the robot apparatus 400 to transmit the utterance content.

  As described above, by asking an appropriate question that the life log recording system understands the user himself / herself, it is possible to make the user feel familiar and to create a situation where the user can easily use the system.

[Example 8]
The data analysis unit 330 indicates that, from the sensor data of the user acquired from the server 200, the user has been to the fitness club, but there is no state of exercising longer than usual for a certain time zone. I found it.

  In this case, the data analysis unit 330 analyzes the user's own mail or SNS (Social Network Service) registered as another media information that is a candidate for an information acquisition source, or a reply to those from another person.

  As a result of the above analysis, the data analysis unit 330 found that it seems that a meal was taken at a restaurant attached to fitness.

In this case, the data analysis unit 330 sends the utterance content configuration unit 350 to, for example,
“Did you eat dinner at fitness?”
Make a conversation. When the data analysis unit 330 determines from the data acquired from the server 200 that the user is at home, is in a state of being calm by the robot apparatus 400, and is in a state of being able to interact with the robot apparatus 400, The robot action transmission unit 360 causes the robot apparatus 400 to transmit the utterance content.

  As described above, by asking an appropriate question that the life log recording system understands the user himself / herself, it is possible to make the user feel familiar and to create a situation where the user can easily use the system.

[Example 9]
The data analysis unit 330 indicates that, from the sensor data of the user acquired from the server 200, the user has been to the fitness club, but there is no state of exercising longer than usual for a certain time zone. I found it.

  In this case, the data analysis unit 330 sends an e-mail or SNS of a person who has been registered as a friend at an SNS or the like who is in the same place (fitness) in the same time zone registered as another media information that is a candidate for an information acquisition source. Is analyzed.

  As a result of the above analysis, the data analysis unit 330 found out that it seemed that a meal was taken together at a restaurant attached to fitness.

In this case, the data analysis unit 330 sends the utterance content configuration unit 350 to, for example,
“Did you eat dinner with xx at fitness?”
Make a conversation. When the data analysis unit 330 determines from the data acquired from the server 200 that the user is at home, is in a state of being calm by the robot apparatus 400, and is in a state of being able to interact with the robot apparatus 400, The robot action transmission unit 360 causes the robot apparatus 400 to transmit the utterance content.

  As described above, by asking an appropriate question that the life log recording system understands the user himself / herself, it is possible to make the user feel familiar and to create a situation where the user can easily use the system.

  Thus, according to the present invention, when learning data learned in the past cannot identify a user's action and cannot automatically label it, the robot apparatus identifies the user's action by talking with the user. In order to extract information, a label meaningful to the user can be easily given through a natural dialogue. In addition, since the labeling is performed, a wider utilization of the life log can be expected.

  Further, according to the present invention, it is determined from the sensor data whether or not the user can talk to the robot apparatus, and when the user can talk to the robot apparatus, the robot apparatus is allowed to talk to the user. Thereby, the user can have a natural conversation with the robot apparatus in a calm state, and the direct impression that the labeling work is being performed can be relieved. As a result, continuous use by the user can be expected.

  Although the present invention has been described based on the drawings and examples, it should be noted that those skilled in the art can easily make various modifications and corrections based on the present disclosure. Therefore, it should be noted that these variations and modifications are included in the scope of the present invention. For example, the functions included in each component, each step, etc. can be rearranged so that there is no logical contradiction, and multiple components, steps, etc. can be combined or divided into one It is. Further, although the present invention has been described mainly with respect to the apparatus, the present invention can also be realized as a method, a program executed by a processor included in the apparatus, or a storage medium storing the program, and is within the scope of the present invention. It should be understood that these are also included.

  In the above-described embodiment, the case where the sensor device 100 includes the acceleration sensor 101, the position sensor 102, the vital sensor 103, the camera 104, the microphone 105, and the like has been described as an example. You may have the sensor of. For example, the sensor device 100 may include a sensor that detects environmental information (temperature, humidity, etc.) where the user is.

DESCRIPTION OF SYMBOLS 100 Sensor apparatus 101 Acceleration sensor 102 Position sensor 103 Vital sensor 104 Camera 105 Microphone 110 Storage part 120 Data transmission part 200 Server 210 Data reception part 220 Data storage part 230 Data request reception part 240 Data transmission part 300 Sensor data analysis apparatus 310 Data request | requirement Unit 320 data reception unit 330 data analysis unit 340 learning data storage unit 350 utterance content configuration unit 360 robot action transmission unit 370 response result reception unit 380 response result analysis unit 390 label information transmission unit 400 robot apparatus 410 robot action reception unit 420 robot action Unit 430 response input unit 440 response result transmission unit

Claims (8)

A life log recording system comprising a sensor device, a server, a sensor data analysis device and a robot device,
The sensor device has means for obtaining sensor data relating to a user of the sensor device,
The server
Means for receiving the sensor data from the sensor device;
Means for storing the sensor data;
The sensor data analysis device
Means for receiving the sensor data from the server;
Learning storage means for storing learned user behavior as learning data;
When there is data in which the user's behavior cannot be specified even if the sensor data is referred to in the sensor data, an analysis unit that constitutes a conversation for extracting information specifying the user's behavior from the user;
Means for determining from the sensor data whether a user is in a state of being able to talk to the robot apparatus;
Means for transmitting the configured conversation to the robotic device when a user is in a state of being able to converse with the robotic device,
The robot apparatus is:
Means for conducting the conversation with the user;
Means for transmitting a user response to the conversation to the sensor data analysis device;
The sensor data analysis apparatus further includes means for transmitting label information that gives meaning to the sensor data to the server based on a response from the user.   2. The life log recording system according to claim 1, wherein the analyzing means constitutes a changed conversation when a desired response cannot be obtained from the user.   The life log recording system according to claim 1 or 2, wherein the analysis means constitutes a conversation that draws out the reason from the user when the user performs a normal action in a different situation. Life log recording system.   The life log recording system according to any one of claims 1 to 3, wherein the analysis unit constitutes a conversation that draws the reason from the user when the user has not performed daily actions. Life log recording system.   5. The life log recording system according to claim 1, wherein, when the user performs a daily action in an amount different from the usual amount, a conversation that draws the reason from the user is performed. A life log recording system characterized by comprising.   The life log recording system according to any one of claims 1 to 5, wherein the analysis unit refers to video and / or audio data that is not normally used for analysis in accordance with the sensor data. A life log recording system comprising a conversation for extracting information for identifying a user's action from the user.   The life log recording system according to any one of claims 1 to 6, wherein the analysis unit refers to an SNS and / or an email that is not normally used for analysis in accordance with the sensor data. A life log recording system comprising a conversation for extracting information specifying an action from a user. A program for operating a life log recording system including a sensor device, a server, a sensor data analysis device, and a robot device,
Causing the sensor device to perform a step of obtaining sensor data relating to a user of the sensor device;
To the server,
Receiving the sensor data from the sensor device;
Storing the sensor data; and
In the sensor data analysis device,
Receiving the sensor data from the server;
If there is data that cannot identify the user's behavior even if the received sensor data refers to the learned user's behavior, configuring a conversation for extracting information identifying the user's behavior from the user;
Determining from the sensor data whether a user is in a state of being able to talk to the robotic device;
Sending the configured conversation to the robotic device when a user is in a state of being able to talk to the robotic device,
In the robot apparatus,
Conducting the conversation with a user;
Sending a user response to the conversation to the sensor data analyzer,
A program that causes the sensor data analysis device to further execute a step of transmitting label information that gives meaning to the sensor data to the server based on a response from the user.

Images