JP6087624B2 - User interface device, program and method capable of presenting action correspondence information in a timely manner - Google Patents

Description

  The present invention relates to a technique for presenting information corresponding to user behavior, for example, health management information.

  2. Description of the Related Art Conventionally, a technique for presenting information that meets a user's needs and interests, for example, information related to health, to a user has been widespread. For example, many user interface devices, such as smartphones and tablet computers, enable a service that measures the number of steps per day of the user and displays advice according to the user's walking situation.

  In addition, as a technique for diagnosing the health condition of a vehicle user (driver), for example, Patent Document 1 collects the driver's body data using a health condition sensor and asks the driver a question. A technique for acquiring data on the health condition of a driver from the obtained answer is disclosed. Here, the timing at which the question is asked is determined as a timing with a low driving load such as when the vehicle is stopped such as waiting for a signal or when traveling on a straight road without an intersection such as an expressway.

  Furthermore, Patent Literature 2 discloses a content viewing system that talks to a user according to the user's state and content status. As a form of this talk, for example, in a sport content, prediction of a battle result is performed before a game or during a game, and support and dialogue are performed. In addition, a technique is disclosed in which even if the user views the same content twice or more, the same conversation as in the first viewing is not performed.

JP 2012-24481 A JP 2006-245941 A

  However, it is difficult to control the contents of the conversation according to the actions that can be normally performed by the user and the timing of the conversation with the conventional techniques as disclosed in Patent Documents 1 and 2.

  For example, the technique of Patent Document 1 only adjusts the timing of a question in a specific situation such as during driving of the vehicle, and does not control the content and timing of the conversation in the user's own normal behavior. Furthermore, the technique of Patent Document 2 does not control the timing of talking.

  Therefore, an object of the present invention is to provide a user interface device, a program, and a method capable of presenting information corresponding to a user's behavior in a timely manner.

According to the present invention, a user interface device that presents information corresponding to a user's behavior,
An acceleration measuring unit capable of measuring acceleration;
For each type of action, a presentation information management means for storing an acceleration pattern during action and dialogue information to be presented in association with each other,
Action estimation means for estimating the type of action by the user based on the measured acceleration and the stored acceleration pattern at the time of action;
End point estimation means for estimating the end point of the action by the user based on the measured acceleration;
Information presenting means for presenting dialogue information associated with the estimated action type at a time based on the estimated action end time ;
When the presented dialogue information includes a question for confirming the type of the estimated action, the answer information for acquiring the user's answer to this question and extracting the information about the user's action from this answer extracting means and <br/> and have a,
When the presentation information management means determines that the acceleration pattern at the time of the action associated with the type of action related to the extracted information about the user's action does not correspond to the measured acceleration, Update the acceleration pattern of the associated action
A user interface device is provided.

One embodiment of the user interface device, behavior estimation means, between the measured acceleration, the highest similarity or types of actions associated with the acceleration pattern having a predetermined threshold value or more similarity, It is also preferable that the type of action to be estimated.

Further, as another embodiment of a user interface device according to the present invention, Hisage shows information management means with the information about the behavior of the extracted user,
(A) Update the type of action and the dialogue information to be presented associated with the type of action, or (b) Update the acceleration pattern during action associated with the type of action, or (c) Extraction It is also preferable to register a new type of action related to the information on the user's action, and to store an acceleration pattern based on the acceleration data obtained at the time of the action in association with each other.

Furthermore, as one embodiment of the user interface device according to the present invention, it further comprises profile management means for recording a value of at least one profile parameter indicating the profile of the user,
The presentation information management means stores the interaction information to be presented in association with each type of action and at least one profile parameter,
It is also preferable that the information presenting means presents the dialogue information associated with the estimated action type and the recorded profile parameter value at a time based on the estimated action end time.

  Here, in the above-described embodiment using the profile management means, the at least one profile parameter value is a value indicating the user's health condition / mood, a value indicating the user's hobby preference, and a value indicating the user's habit behavior It is also preferred that there is at least one.

Further, in the above-described embodiment using the profile management means, a history of action execution time is recorded in association with each type of action, and a history of values indicating the health state / mood of the user is recorded together with this history. Action history management means,
Action correlation determining means for determining the correlation between the action execution time and the value indicating the state of health and mood,
The presentation information management means stores a dialogue information group including dialogue information that prompts an action related to the determination and dialogue information that restricts the action in association with the type of action,
It is also preferable that the information presenting unit selects the dialogue information to be presented from the dialogue information group based on the determination result by the behavior correlation determining unit.

Furthermore, as one embodiment of the user interface device according to the present invention, it further includes a microphone that detects surrounding sounds, and a sound determination unit that determines whether or not the detected sound is a user's voice,
The information presenting means corresponds to the type of the estimated action even though at least the predetermined time from the time when the true judgment is made by the voice judging means is a time based on the estimated end time of the action It is also preferable not to present the attached dialogue information.

  In addition, as an embodiment relating to the type of action, the type of action includes running, walking, climbing stairs, descending stairs, climbing on elevators, descending on elevators, climbing on escalators, descending on escalators, cycling It is also preferable to include at least one of a movement of a car, a movement by a car, and a movement by train.

Furthermore, as one embodiment of the user interface device according to the present invention, it further includes an atmospheric pressure sensor that measures atmospheric pressure, and a vertical movement speed determination means that determines a vertical movement speed from the detected atmospheric pressure value,
The presenting information management means stores the ascending or descending on the escalator as the type of action in association with the moving speed in the vertical direction corresponding to the up or down on this escalator and the dialogue information to be presented,
The behavior estimating means has a predetermined range in which a difference between a measured acceleration fluctuation is equal to or less than a predetermined threshold value and a determined vertical movement speed and an upward or downward movement speed corresponding to an escalator. It is also preferable to set the escalator ascending or descending as the type of action to be estimated.

As one embodiment of the user interface device according to the present invention, there is provided a microphone for detecting environmental sound, and environmental sound determination means for determining whether the detected environmental sound is a sound in a closed space or a sound in an open space. In addition,
The presentation information management means stores, in association with movement on a bicycle as a type of action, an acceleration pattern corresponding to the movement on the bicycle, information on the presence of sound in an open space, and dialogue information to be presented. Let
It is also preferable that the behavior estimation means sets the determination that the sound is an open space by the environmental sound determination means as a necessary condition for estimating movement by bicycle.

Furthermore, as one embodiment of the user interface device according to the present invention, there is provided a microphone for detecting environmental sound, and environmental sound determination means for determining whether the detected environmental sound is a sound in a closed space or a sound in an open space. In addition,
The presentation information management means stores, in association with movement in a car as a type of action, an acceleration pattern corresponding to the movement in the car, information on the presence of sound in a closed space, and dialogue information to be presented. Let
It is also preferable that the behavior estimation means sets the determination that the sound is in a closed space by the environmental sound determination means as a necessary condition for estimating movement in an automobile.

Furthermore, according to the user interface device of the present invention, the end point estimation means is
(A) Estimating a point in time when the measured acceleration fluctuation is equal to or less than a predetermined threshold, or (b) a point in time when the type of the behavior estimated by the behavior estimation means is changed, as the end point of the action by the user. Is also preferable.

According to the present invention, there is also provided a user interface device that presents information corresponding to user behavior,
An acceleration measuring unit capable of measuring acceleration;
A microphone that detects ambient sound,
Voice determination means for determining whether or not the detected voice is a voice of a user;
A presentation information management means for storing dialogue information to be presented in association with each type of action;
Action estimating means for estimating the type of action by the user based on the measured acceleration;
End point estimation means for estimating the end point of the action by the user based on the measured acceleration;
Information presentation means for presenting dialogue information associated with the estimated action type at a time based on the estimated action end time;
Have
The information presenting means corresponds to the type of the estimated action even though at least the predetermined time from the time when the true judgment is made by the voice judging means is a time based on the estimated end time of the action Do not present the attached dialogue information
A user interface device is provided.
Moreover, according to the present invention, there is further provided a user interface device for presenting information corresponding to a user's action including ascending and descending on an escalator as its type,
An acceleration measuring unit capable of measuring acceleration;
An atmospheric pressure sensor for measuring atmospheric pressure;
A vertical movement speed determining means for determining a vertical movement speed from the detected atmospheric pressure value;
A presentation information management means for storing dialogue information to be presented in association with each type of action;
Action estimating means for estimating the type of action by the user based on the measured acceleration;
End point estimation means for estimating the end point of the action by the user based on the measured acceleration;
Information presentation means for presenting dialogue information associated with the estimated action type at a time based on the estimated action end time;
Have
The presentation information management means stores the up and down movement on the escalator as the type of action in association with the moving speed in the vertical direction corresponding to the up or down movement on the escalator and the interactive information to be presented,
The behavior estimating means has a predetermined range in which a difference between a measured acceleration fluctuation is equal to or less than a predetermined threshold value and a determined vertical movement speed and an upward or downward movement speed corresponding to an escalator. Escalator ascends or descends as an estimated action type
A user interface device is provided.
Furthermore, according to the present invention, there is also provided a user interface device for presenting information corresponding to a user's action including movement on a bicycle as its type,
An acceleration measuring unit capable of measuring acceleration;
A microphone that detects environmental sounds;
Environmental sound determination means for determining whether the detected environmental sound is a sound in a closed space or a sound in an open space;
A presentation information management means for storing dialogue information to be presented in association with each type of action;
Action estimating means for estimating the type of action by the user based on the measured acceleration;
End point estimation means for estimating the end point of the action by the user based on the measured acceleration;
Information presentation means for presenting dialogue information associated with the estimated action type at a time based on the estimated action end time;
Have
The presentation information management means stores the acceleration pattern corresponding to the movement on the bicycle, the information on the presence of sound in the open space, and the interactive information to be presented in association with the movement on the bicycle as the type of action. ,
The behavior estimation means uses the determination that the sound is an open space by the environmental sound determination means as a necessary condition for estimating movement by bicycle.
A user interface device is provided.
Moreover, according to the present invention, there is further provided a user interface device that presents information corresponding to a user's action including movement in an automobile as its type,
An acceleration measuring unit capable of measuring acceleration;
A microphone that detects environmental sounds;
Environmental sound determination means for determining whether the detected environmental sound is a sound in a closed space or a sound in an open space;
A presentation information management means for storing dialogue information to be presented in association with each type of action;
Action estimating means for estimating the type of action by the user based on the measured acceleration;
End point estimation means for estimating the end point of the action by the user based on the measured acceleration;
Information presentation means for presenting dialogue information associated with the estimated action type at a time based on the estimated action end time;
Have
The presentation information management means stores the acceleration pattern corresponding to the movement in the automobile, the information that there is sound in the closed space, and the dialogue information to be presented in association with the movement in the automobile as the type of action. ,
The behavior estimation means uses the determination that the sound is in a closed space by the environmental sound determination means as a necessary condition for estimating movement in an automobile.
A user interface device is provided.
Furthermore, according to the present invention, there is also provided an information presentation program for causing a computer mounted on a user interface device that presents information corresponding to a user's behavior to function.
The apparatus includes an acceleration measuring unit capable of measuring acceleration,
The above information presentation program
For each type of action, a presentation information management means for storing an acceleration pattern during action and dialogue information to be presented in association with each other,
Action estimation means for estimating the type of action by the user based on the measured acceleration and the stored acceleration pattern at the time of action;
End point estimation means for estimating the end point of the action by the user based on the measured acceleration;
Information presenting means for presenting dialogue information associated with the estimated action type at a time based on the estimated action end time ;
When the presented dialogue information includes a question for confirming the type of the estimated action, the answer information for acquiring the user's answer to this question and extracting the information about the user's action from this answer Let the computer function as an extraction means ,
When the presentation information management means determines that the acceleration pattern at the time of the action associated with the type of action related to the extracted information about the user's action does not correspond to the measured acceleration, Update the acceleration pattern of the associated action
Information presentation program shall be the feature is provided that.

According to the present invention, there is further provided an information presentation method for presenting information corresponding to user behavior,
For each type of action, a first step of storing an acceleration pattern at the time of the action and dialogue information to be presented in association with each other;
A second step of measuring acceleration due to user behavior;
A third step of estimating the type of action by the user based on the measured acceleration and the stored acceleration pattern of action;
A fourth step of estimating an end point of the action by the user based on the measured acceleration;
A fifth step of presenting dialogue information associated with the estimated action type at a time based on the estimated action end time ;
When the presented dialogue information includes a question for confirming the type of the estimated action, a user's answer to this question is acquired, and information on the user's action is extracted from this answer And the steps
When the action acceleration pattern associated with the type of action related to the extracted user action information is determined not to correspond to the measured acceleration, the action time associated with this action type An information presentation method is provided that includes a seventh step of updating the acceleration pattern .

  According to the user interface device, the information presentation program, and the method of the present invention, it is possible to present information corresponding to the user's behavior in a timely manner.

It is the schematic for demonstrating the various form in presentation of the dialog information using the user interface apparatus by this invention. It is a functional block diagram in one Embodiment of the portable terminal 1 by this invention. It is the schematic of one Embodiment in the presentation information table which concerns on this invention. It is a flowchart which shows one Embodiment of the information presentation method by this invention. It is a flowchart which shows one Embodiment of selection of the dialog information based on the action history which concerns on this invention.

The user interface device according to the present invention can present “conversation information” associated with the “behavior type” of the user at a time based on this “behavior end time”. Here, “type of action” includes, for example,
(I) Run, (b) Walk, (c) Up the stairs, (d) Down the stairs,
(E) Ascending at the elevator, (f) Getting off at the elevator,
(G) Ascending on the escalator, (h) descending on the escalator,
It is preferable that at least one of (li) movement by bicycle, (nu) movement by automobile, and (le) movement by train be included.

In order to realize the presentation of such dialogue information, this user interface device
(A) an acceleration measuring unit capable of measuring “acceleration”;
(B) Each “action type” includes a presentation information management unit that stores “conversation information” to be presented in association with each other. Of these, the presentation information management department
(B ′) It is also preferable to store the “acceleration pattern” at the time of the action and the “dialog information” to be presented in association with each “type of action”.

The user interface device according to the present invention has such a configuration,
(C) Estimating the “type of action” by the user based on the measured “acceleration”;
(D) Based on the measured “acceleration”, estimate the “action end time” by the user,
(E) The “conversation information” associated with the estimated “type of action” is presented at a time based on the estimated “end time of action”.

Of these, when estimating the "type of action"
(C ′) The “action type” associated with the “acceleration pattern” having the highest similarity or a similarity equal to or higher than a predetermined threshold with the measured “acceleration” is estimated “Type” is also preferable.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  FIG. 1 is a schematic diagram for explaining various forms of presentation of dialogue information using the user interface device according to the present invention.

  A portable terminal 1 shown in FIGS. 1A to 1D is a user interface device having portability such that a user can carry and perform actions such as movement. For example, a smartphone, a tablet computer, a mobile phone, or the like can be used.

  The mobile terminal 1 shown in FIG. 1A is carried by a user who is “walking” (walking) and moves, and when the user ends “walking”, the end point of “walking” Is estimated. Next, at the time based on the estimated end time, a conversation as dialogue information, for example, output of a voice “Tone!” Of the content that encourages “walking”, or display on a display or the like is performed.

  Here, the conversation “That!” (Encouragement) is set and stored in advance as dialogue information associated with “walking”, for example, with the recognition that “walking” is good for the user's health. Yes.

Here, the time of talking based on the estimated end time of the action is, for example,
(1) the estimated end time itself, or (2) the time when a predetermined time (for example, 5 seconds) has elapsed from the estimated action end time, or (3) after the estimated action end time (predetermined ) It can be the time when the next action is detected. It is also preferred that the point in time of this talk is negotiated according to the corresponding estimated action type. In the case of “walking”, the point in time of this conversation may be, for example, (1) the estimated end point itself.

  1B is carried by a user who is “running” (running) and moves, and when the user finishes “running”, the “running” ends. Estimate the time. Next, at the time based on the estimated end time, the dialogue information is spoken, for example, the content of “running” is cautioned, “sounding is forbidden!” Or output on a display or the like. To do.

  In the case of “running”, the point of time for the talk can be, for example, (2) a point of time that has passed a predetermined time from the estimated point of end of the action.

  Here, the mobile terminal 1 holds profile parameter values relating to the current state of health and mood of the user, and recognizes that, for example, “running” may not be good for the user's health. On top of this, the phrase “do not overdo it!” (Encouragement) is preliminarily set and stored as dialogue information associated with “running” and the profile parameter value (range).

  Further, the mobile terminal 1 shown in FIG. 1C is carried by a user who is performing “uphill on an escalator” (on an uphill escalator), and the user moves “uphill on an escalator”. ”Is terminated, the end point of“ up on the escalator ”is estimated. Next, at a time point based on the estimated end time point, talking as dialogue information, for example, outputting a voice saying “Sometimes you can try using stairs” or a display or the like that prompts other actions. Display.

  In the case of “up on the escalator”, the point of time for the talk can be, for example, (1) the estimated end point itself.

  Here, “Sometimes you can try using stairs” (prompt) is, for example, the recognition that “up the stairs” is better for the user ’s health than “up the escalator”. Below, it is preset and stored as dialogue information associated with “up on the escalator”.

  Furthermore, the mobile terminal 1 shown in FIG. 1D moves with the user who is “moving in a car” (riding or riding and driving a car), and the user When “movement by car” is completed, the end point of “movement by car” is estimated. Next, at the time based on the estimated end time, a question as dialogue information, for example, a voice output of “Today is a car? Display.

  In the case of this “movement by car”, the time of this question may be, for example, (3) the time when “walking” is detected (the time when the type of action changes) after the estimated action end time. it can. Or (2) It is good also as the time which only predetermined time passed from the end time of the estimated action.

Here, the question “Are you going out in a car today?” Is preset and stored as dialog information associated with “movement in a car”, for example. In selecting whether to ask this question or to talk to others, for example,
(A) When the measured “acceleration” has the highest similarity but less than a predetermined threshold with the “acceleration pattern” associated with “movement by car” (ie, , When a clear decision cannot be made). Or
(B) With respect to the estimated number of times of “movement by car”, for example, it is possible to make an arrangement such that a question is asked for confirmation only once in 10 times.

  As can be understood from the above-described example of interactive information presentation on the mobile terminal 1, the mobile terminal 1 can present the interactive information corresponding to the user's behavior in a timely manner.

[Mobile terminal 1]
FIG. 2 is a functional configuration diagram of an embodiment of the mobile terminal 1 according to the present invention.

  According to FIG. 2, the portable terminal 1 has an acceleration sensor 100 and a processor memory. Here, the processor memory realizes its function by executing a program. The portable terminal 1 further includes an atmospheric pressure sensor 101, a microphone 102, a positioning unit 103, a floor map storage unit 104, a speaker 105, a display unit 106a, an input unit 106b, and a health / mood information storage unit 107. It is also preferable to have a hobby preference information storage unit 108 and a habit behavior information storage unit 109.

  Further, the processor memory includes an acceleration data generation unit 111, a presentation information management unit 112, a behavior estimation unit 113, an end time estimation unit 114, and an information presentation unit 115 as functional configuration units. The processor memory further includes a position determination unit 110, a vertical movement speed determination unit 116, an environmental sound determination unit 117, a sound determination unit 118, a profile management unit 119, an action history management unit 120, and an action correlation determination. It is also preferable to have a unit 121, an answer information extraction unit 122, and a speech synthesis unit 123.

  The acceleration sensor 100 is an acceleration measuring unit capable of measuring acceleration. As the acceleration sensor 100, for example, a three-axis type acceleration measuring instrument formed by using MEMS (Micro Electro Mechanical Systems) technology, for example, by a capacitance method or a piezoresistive method can be employed. A measurement result obtained by the acceleration sensor 100 is output to the acceleration data generation unit 111.

  The atmospheric pressure sensor 101 can be a barometer capable of detecting a minute atmospheric pressure change corresponding to an altitude difference of several tens of centimeters. As a result, it is possible to measure the amount of vertical movement by the elevator, escalator, or stairs. The measurement result obtained by the atmospheric pressure sensor 101 is output to the vertical movement speed determination unit 116.

  The positioning unit 103 captures a positioning radio wave from a GPS satellite, and further measures the location (latitude, longitude) of the mobile terminal 1 by a GPS positioning method. Further, it is also possible to calculate the acceleration in movement of the mobile terminal 1 from the change in position measured by the positioning unit 103. In this case, the positioning unit 103 functions as an acceleration measuring unit.

  The floor map accumulating unit 104 accumulates a floor map that is map information of an indoor / outdoor area in which information on positions and ranges of passages / corridors, rooms / areas, stairs, escalators, elevators, and the like in a predetermined indoor / outdoor area is described. By using the floor map storage unit 104 together with information from the indoor position sensor installed in the same area or the positioning result by the positioning unit 103, for example, based on the current position of the mobile terminal 1, the user can Information such as moving on an up escalator is acquired.

  The microphone 102 detects ambient sound or environmental sound (converts it into an electrical signal). In addition, the speaker 105 converts dialogue information (a voice signal synthesized corresponding to the dialogue information) output from the information presentation unit 115 described later into a voice and outputs the voice. The microphone 102 further detects a response by the voice of the user who has received a question as dialogue information issued from the speaker 105.

  In the display unit 106a, the dialogue information output from the information presenting unit 115 is displayed as characters, for example. In the input unit 106b, a response to the question is input by an operation by a user who visually recognizes the question displayed on the display unit 106a. The display unit 106a and the input unit 106b can be, for example, a touch panel display.

  The acceleration data generation unit 111 receives a signal including acceleration information from the acceleration sensor 100, generates acceleration data representing a temporal change in the acceleration component or acceleration absolute value in each axis direction, and outputs the acceleration data to the behavior estimation unit 113.

  The presentation information management unit 112 stores the dialogue information to be presented in association with each type of user action. Specifically, the presentation information table 112t described later with reference to FIG. 3 is generated and managed. Here, it is also preferable to store the acceleration pattern at the time of the action and the dialogue information to be presented in association with each type of action of the user.

  In addition, the presentation information management unit 112 preferably stores the dialogue information to be presented in association with each type of user action and at least one profile parameter. Here, the profile parameter value may be at least one of a value indicating the user's health condition / mood, a value indicating the user's hobby preference, and a value indicating the user's habit behavior.

  Further, the presentation information management unit 112 associates the up / down movement on the escalator as the type of user's behavior with the moving speed in the vertical direction corresponding to the up / down movement on the escalator and the dialogue information to be presented. It is also preferable to memorize. Here, the moving speed in the vertical direction is an item for comparing a value calculated from, for example, a change in an atmospheric pressure measurement value by an atmospheric pressure sensor 100 described later.

  Furthermore, the presentation information management unit 112 provides information on an acceleration pattern corresponding to movement on a bicycle, information on presence of sound in an open space, and a dialogue to be presented for “movement on a bicycle” as the type of user's action. It is also preferable to store information in association with each other. In addition, “movement by car” as the type of user behavior is stored by associating and storing the acceleration pattern corresponding to the movement by the car, information on the presence of sound in a closed space, and the dialogue information to be presented Is also preferable.

  The behavior estimation unit 113 estimates the type of behavior by the user based on the acceleration data input from the acceleration data generation unit 111. At this time, in the presentation information table 112t, the type of action associated with the acceleration pattern having the highest “similarity” or “similarity” equal to or higher than a predetermined threshold with the measured acceleration (acceleration data). It is also preferable to set the type of action to be estimated. Here, the degree of correlation can be adopted as the “similarity”. In this case, the predetermined threshold value is a threshold value of a predetermined correlation coefficient.

In addition, the behavior estimation unit 113
(A) Up and down direction corresponding to the measured acceleration fluctuation is equal to or less than a predetermined threshold value, and (b) the determined vertical movement speed and the up (down) time of the escalator in the presentation information table 112t When the difference from the moving speed is within the predetermined range,
It is also preferable to set “uphill (downhill) on the escalator” as the type of action to be estimated.

The end point estimation unit 114 estimates the end point of the action by the user based on the measured acceleration (acceleration data input from the acceleration data generation unit 111). here,
(A) Estimating a point in time when the variation in measured acceleration is equal to or less than a predetermined threshold, or (b) a point in time when the type of behavior estimated by the behavior estimation unit 113 is changed, as the end point of the user's behavior. Can do.

  The information presenting unit 115 presents the dialogue information associated with the type of behavior estimated by the behavior estimating unit 113 at a time based on the end time of the behavior estimated by the end time estimating unit 114. At this time, it is also preferable to output the dialogue information to the voice synthesis unit 123, synthesize a voice signal corresponding to the dialogue information, and output the voice of the dialogue information via the speaker 105. In addition, the information presentation unit 115 can output dialogue information to the display unit 106a to display characters or images corresponding to the dialogue information.

  The position determination unit 110 inputs the measurement result of the location (latitude, longitude) in the positioning unit 103, and combines the position information from the floor map to determine whether the mobile terminal 1 is at the location such as the escalator position. Further, it is determined whether or not the vehicle is moving at the escalator position. Further, the position determination unit 110 can also calculate an acceleration in movement of the mobile terminal 1 from a change in the location (latitude, longitude) of the mobile terminal 1. In this case, the positioning unit 103 functions as an acceleration measuring unit.

  The vertical movement speed determination unit 116 determines the vertical movement speed of the mobile terminal 1 from the change in atmospheric pressure detected by the atmospheric pressure sensor 101. The determined movement speed in the vertical direction is output to the behavior estimation unit 113.

  The environmental sound determination unit 117 determines whether the environmental sound detected by the microphone 102 is a sound in a closed space or a sound in an open space, and outputs the determination result to the behavior estimation unit 113. In general, in a sound field in a closed space, there are a plurality of resonance frequencies, and a characteristic pattern (mode) appears in the sound pressure distribution in the sound field. The environmental sound determination unit 117 determines that the environmental sound is a sound in a closed space when the presence of the resonance frequency or the mode is confirmed by performing spectrum analysis or mode analysis of the input environmental sound. If the environmental sound is not confirmed, it can be determined that the environmental sound is a sound in an open space.

  Furthermore, when a device for generating a sound wave of a constant frequency is installed in the car in advance, and the environmental sound detected by the microphone 102 includes this sound wave, the environmental sound is a sound in a closed space (in the car). You may determine that there exists.

  Here, the behavior estimation unit 113 determines that the environmental sound determination unit 117 determines that the sound is an open space, and that the user's behavior type is “bicycle movement”. It is also preferable to do. Moreover, it is also preferable to make the determination that the sound is a closed space by the environmental sound determination unit 117 as a necessary condition for estimating that the user's action type is “movement by car”.

  The sound determination unit 118 determines whether or not the detected sound is a user's voice, and outputs the determination result to the behavior estimation unit 113. Here, the voice determination unit 118 can store in advance the characteristics of the user's voice and determine whether or not the voice detected using the voiceprint authentication technique is the user's voice.

  Here, the information presentation unit 115 is based on the estimated action end time at least within a predetermined time from the time when the sound determination unit 118 performs the true determination (determination that the voice is the user's voice). However, it is also preferable not to present the dialogue information associated with the estimated type of action. That is, in this case, it is highly likely that the user is having a conversation, and it is determined that there is no room for responding / operating on the mobile terminal 1, and no talking is performed.

The mobile terminal 1 is equipped with a proximity sensor (such as an electric field, a magnetic field, or an infrared system) that detects an approach to a terminal having the same interactive information presentation function as the mobile terminal 1.
(A) the two terminals are close together (users are acting together)
(B) When the true determination is performed by the voice determination unit 118 (users are having a conversation), the interaction information associated with the estimated action type is presented within the predetermined time. You can also choose not to let it.

  The health / mood information accumulating unit 107 is not shown in the figure, for example, output from a biosensor such as a pulse sensor mounted on the mobile terminal 1 (pulse value, etc.), or an externally input user (hypertension, obesity) Etc.) is stored as health information data. In addition, the hobby preference information storage unit 108 extracts words such as “I like carbonated juice” from data related to daily actions created by the user, such as an electronic diary and a blog, and stores them as hobby preference data. Furthermore, the habit behavior information storage unit 109 also extracts words such as “a lot of late-night meals” and “a lot of late night” from data related to daily actions created by users such as electronic diaries and blogs. , Accumulate as habit behavior data.

  The profile management unit 119 records the value of at least one profile parameter indicating the user profile, and outputs the recorded value to the information presentation unit 115. Here, the profile management unit 119 inputs the health information, the hobby preference information, and the habit behavior information from the health information accumulation unit 107, the hobby preference information accumulation unit 108, and the habit behavior information accumulation unit 109, and sets profile parameters of the user. Record as a value.

  When this profile management unit 119 is provided, the presentation information table 112t stores the interaction information to be presented in association with each type of behavior and at least one profile parameter depending on the type of behavior of the user. Has been. In this case, in the presentation information table 112t, the information presentation unit 115 sets the interaction information associated with the estimated action type and the recorded profile parameter value based on the estimated action end time. It is also preferable to make it present.

  The action history management unit 120 records a history of the execution time of the action in association with each type of user's action, and records a history of values indicating the user's health state and mood together with this history. Specifically, an action history table 120t described later with reference to FIG. 5 is generated.

  The behavior correlation determination unit 121 uses the generated behavior history table 120t to determine the correlation between the behavior time (for example, “run”) and the value indicating the health state / mood.

  In this case, the presentation information management unit 112 stores a dialogue information group including dialogue information that prompts an action related to the above-described determination (for example, “run”) and dialogue information that restricts the action in association with the type of action. It is also preferable. Furthermore, it is also preferable that the information presentation unit 115 selects the dialogue information to be presented from the dialogue information group based on the determination result by the behavior correlation determination unit 121.

  The answer information extraction unit 122 acquires a user's answer to a question that is a question presented as dialogue information and that is used to check the estimated type of action. Specifically, for example, the voice of the user's answer input via the microphone 102 is acquired via the voice determination unit 118. At this time, it is preferable that the voice determination unit 118 outputs the answer voice information to the answer information extraction unit 122 only when it is determined that the detected voice is the voice of the user. The answer information extraction unit 122 can also input user input answer information input via the input unit 106a by a user operation.

  Next, the answer information extraction unit 122 extracts information about the user's behavior from the input answer voice information or the input answer information. For example, as an answer to the question “I was running up the stairs today!”, When I got the answer “No, I went up normally.” Action related information (speech or text) is extracted. Here, the answer information extraction unit 122 stores action-related information assumed to be included in the answer for each set question in advance, and the stored action-related information is included in the acquired answers. It is also preferable to determine whether or not it is included.

The answer information extraction unit 122 outputs information about the extracted user behavior to the presentation information management unit 112. In this case, the presentation information management unit 112 has information about the input user behavior,
(A) Updating the action type and the interaction information associated with this action type in the presentation information table 112t, or (b) the action time associated with the action type in the presentation information table 112t. Or (c) a new action type related to the extracted user action information is registered in the presentation information table 112t, and an acceleration pattern based on the acceleration data obtained at the time of the action Can be stored in association with each other. That is, by obtaining an answer to a question for confirming the type of user's action, the presentation information table 112t can be updated to a content that is more current.

Here, the above (a) is performed when the type of action associated with the acceleration pattern having the highest degree of similarity with the measured acceleration or a predetermined threshold or more is different from the extracted user action. May be. In the case (b), when the measured acceleration has the highest similarity but less than a predetermined threshold with the acceleration pattern associated with one type of action (ie, , When a clear determination cannot be made). Furthermore, the above (c) may be performed when the extracted user action is a new one not registered. In addition, the number of questions is usually gradually reduced by updating the content in the presentation information table 112t according to the current state.

[Presentation information table]
FIG. 3 is a schematic diagram of an embodiment of the presentation information table according to the present invention.

The presentation information table 112t shown in FIG. 3 is provided for each preset “type of action”.
(A) Acceleration pattern, (b) Profile parameters (A, B, ...)
(C) Vertical movement speed, (d) Ambient sound determination, (e) Dialog information A, and (f) Dialog information B (question)
Is a table recorded in association with each other.

Among these, as described above, “type of action” includes (a) running, (b) walking, (c) going up the stairs, (d) going down the stairs, (e) going up at the elevator, (f) Going down the elevator,
It may include at least one of (g) going up on an escalator, (h) going down on an escalator, (li) traveling on a bicycle, (nu) traveling on a car, and (le) traveling on a train. preferable.

  Of course, the type of action is not limited to the above. For example, “up climbing on stairs”, “down climbing on stairs”, and the like can also be adopted as action types.

  The acceleration pattern is a typical acceleration pattern corresponding to the type of action. Specifically, for example, a time change in acceleration when the user is actually running is measured using the acceleration sensor 100 and generated by the acceleration data generation unit 111 (for example, an acceleration component or acceleration absolute in each axis direction). Acceleration data (which represents a change in value over time) can be (a) an acceleration pattern corresponding to running. At this time, acceleration data obtained by running a plurality of times for a predetermined time may be averaged to obtain an acceleration pattern.

  The profile parameters (A, B,...) Are indexes representing profiles of the user's health condition / mood, hobbies, habits, etc. For example, profile parameter values include obesity tendency presence / absence, pulse value ( Classification), blood pressure (or classification within that range), morning / night lifestyle patterns, likes / dislikes certain preference products (eg carbonated juice), many late night meals / less / none , Etc.

  The vertical movement speed is a movement speed in a typical vertical direction (vertical direction) when the user is climbing up or down on the escalator. Specifically, for example, the change in atmospheric pressure when the user actually goes up or down on the escalator is measured using the atmospheric pressure sensor 101, and the vertical movement speed determined by the vertical movement speed determination unit 116 is measured. Can be set to a typical vertical movement speed corresponding to (g) up on the escalator or (h) down on the escalator. At this time, the acceleration data when the escalator is raised / lowered a plurality of times may be averaged to obtain the vertical movement speed.

  The environmental sound determination is a parameter indicating whether the environmental sound is a sound in a closed space or a sound in an open space. For example, (li) the value of sound in an open space is recorded in association with movement on a bicycle, and (nu) the value of sound in a closed space is associated with movement in a car. To be recorded.

  The conversation information A is the content of the conversation to the user associated with each type of action and profile parameter value. For example, dialog information “(Tone!) (Encouragement)” is recorded in association with (i) running and “with obesity tendency” (as the value of profile parameter A). On the other hand, (a) running information and dialogue information “normal behavior is important!” (Praise) is recorded in association with “no obesity tendency” (as the value of profile parameter A).

  The conversation information B is the content of a question for confirming the estimated action type associated with each action type. For example, dialogue information “Today is a bicycle?” Is recorded in association with the movement on a bicycle.

  The dialogue information A and B are not limited to the contents described above. For example, as the dialogue information A, it is possible to adopt a content that recommends purchase / use of a predetermined product (for example, sports drink) according to the type of action (for example, (i) running) performed by the user.

[Information presentation method]
FIG. 4 is a flowchart showing an embodiment of the information presentation method according to the present invention. As a situation where the method of FIG. 6 is applied, the user starts a certain type of action and then ends this action. After the action by the user is finished, the mobile terminal 1 talks or asks the user.

(S300) The mobile terminal 1 measures acceleration in order to estimate the type of action of the user and the end point of the action.
(S301) Next, it is determined which of the acceleration patterns stored in the presentation information table 112t is similar to the acceleration data acquired by the measurement.

  In this determination, it is also preferable to obtain the degree of correlation between the acceleration data acquired by measurement at a predetermined time and one predetermined time in the acceleration pattern recorded in the presentation information table 112t. Here, both the acceleration data and the acceleration pattern are data strings of acceleration values (each axis component value or absolute value) using time as a parameter. In each of these data strings, the maximum value is 1 and It is also preferable that the average value is adjusted to be zero. By calculating the correlation coefficient between the two data strings, the similarity (correlation coefficient) between the acquired acceleration data and the recorded acceleration pattern is obtained.

  Further, for example, when the type of action is (i) running, the frequency of the acceleration pattern changes if the running speed (steps per second) is different. In order to cope with this, a recorded acceleration pattern that is stretched or compressed by a predetermined ratio (a frequency that is reduced or increased) is generated, and between the deformed acceleration pattern and the acceleration measurement value. It is also preferable to calculate the correlation coefficient. Specifically, the correlation coefficient is calculated while changing the recorded acceleration pattern (for example, for 10 seconds) between, for example, plus and minus 10%, and the maximum value of the obtained correlation coefficient is recorded. It is determined as the degree of similarity between the measured acceleration pattern.

  Here, changing between plus and minus 10% generates a 10-second acceleration pattern geometrically extended to 11 seconds, and similarly 10.9 seconds and 10.8 seconds. Generate a pattern stretched to. Similarly, in the case of compression, a 10-second acceleration pattern is geometrically compressed to 9 seconds, and the pattern is similarly compressed to 9.1 seconds, 9.2 seconds, and so on. Generate a pattern. In this case, 21 patterns are generated in total. Naturally, the ratio / interval for stretching or contracting is not limited to the above-described one, and for example, an acceleration pattern that is stretched or contracted more finely may be generated.

  Next, based on the calculated correlation coefficient, an acceleration pattern having the highest similarity (correlation coefficient) with the acquired acceleration data or having a similarity (correlation coefficient) equal to or higher than a predetermined threshold value Select.

  (S302) The type of action by the user is estimated using the selected acceleration pattern. At this time, the type of action associated with the acceleration pattern having the highest similarity (correlation coefficient) with the acquired acceleration data can be determined as the type of action to be estimated.

  Or you may select the acceleration pattern which has a similarity (correlation coefficient) more than a predetermined threshold with the acquired acceleration data. In this case, for each type of action associated with the selected acceleration pattern, a determination is made based on other items (profile parameter, vertical movement speed, environmental sound determination, etc.) in the presentation information table 112t. One action type with higher relevance may be determined from the action types of the acceleration pattern.

(S303) Based on the measured acceleration (acquired acceleration data), the end point of the action by the user is estimated. Specifically, the time point at which the measured change in acceleration is equal to or less than a predetermined threshold value may be estimated as the end point of the action by the user. In this case, for example, variances V _x , V _y, and V _z of the axial components a _x , a _y, and a _z of acceleration in a predetermined acceleration measurement period are
(1) V _x <V _th , V _y <V _th , and V _z <V _th (V _th is a predetermined threshold)
It is possible to set the time when the action is satisfied as the end time of the action by the user. Alternatively, the variance V _a of the absolute value of acceleration during a predetermined acceleration measurement period is
(2) V _a <V _th '(V _th ' is a predetermined threshold value)
It is good also considering the time of satisfy | filling as the end time of the action by a user.

  Furthermore, as a change mode, steps S300 to S302 are repeatedly executed to periodically estimate the type of action, and the time point when the estimated action type is changed is the end point of the action by the user. It is also possible to estimate.

Hereinafter, an example of execution in steps S302 and S303 for each “action type” that can be set will be described.
(S302 and S303: (I) running)
(1) When the acquired acceleration data has the highest correlation coefficient with (b) the acceleration pattern associated with the running stored in the presentation information table 112t, the type of action of the user is (I) Estimated to run.
(2) A point in time when the measured acceleration fluctuation is equal to or less than a predetermined threshold, or a point in time when the type of estimated action changes (for example, (b) changes to walking) To do.

(S302 and S303: (B) walking)
(1) When the acquired acceleration data has the highest correlation coefficient with the acceleration pattern associated with (b) walking stored in the presentation information table 112t, the type of user's action is (B) Estimate walking.
(2) A time point at which the measured change in acceleration is equal to or less than a predetermined threshold is defined as the end time point of (b) walking by the user.

(S302 and S303: (C) Up the stairs, (D) Down the stairs)
(1) The acquired acceleration data has the highest correlation coefficient with the acceleration pattern associated with (c) ascending stairs / (d) descending stairs stored in the presentation information table 112t. At this time, the type of the user's action is estimated as (c) going up the stairs / (d) going down the stairs.
(2) A point in time when the variation of the measured acceleration is equal to or less than a predetermined threshold value or a point in time when the type of estimated action changes (for example, (b) changes to (b) running or (b) walking). The end of stairs up / (d) stairs down.

(S302 and S303: (e) ascending at the elevator, (f) descending at the elevator)
(1) The highest correlation coefficient between the acquired acceleration data and the acceleration pattern associated with (e) ascending at the elevator / (f) descending at the elevator, stored in the presentation information table 112t. , It is estimated that the type of user's action is (e) ascending at the elevator / (f) descending at the elevator.
(2) A time point at which the measured acceleration fluctuation is equal to or less than a predetermined threshold value is defined as an end point of (e) elevator up / (e) elevator down by the user.

(S302 and S303: (G) Uphill on the escalator, (H) Downhill on the escalator)
(1-1) The fluctuation of acceleration in the acquired acceleration data is equal to or less than a predetermined threshold, and the moving speed in the up / down direction is the up / (h) escalator in the (g) escalator in the presentation information table 112t. If the speed value is within the speed range stored in association with the downhill at (2), the user's action type is estimated to be (g) going up on the escalator / (h) going down on the escalator.
(1-2) Alternatively, in the position determination unit 110 in which the acceleration variation in the acquired acceleration data is equal to or less than a predetermined threshold value and the output (floor map information) from the positioning unit 103 and the floor map storage unit 104 is considered. If the determination result is that the user is moving at the up / down escalator position, the type of the user's action is estimated as (g) going up on the escalator / going down on the (h) escalator.
(2) The point in time when the type of the estimated action has changed (for example, (i) running or (b) changing to walking) is the end point of the user going up on (g) escalator / (h) going down on escalator And

(S302 and S303: (L) Bicycle movement)
(1) When the acquired acceleration data has the highest correlation coefficient with the acceleration pattern associated with the movement on the bicycle stored in the presentation information table 112t, The type is estimated to be (li) bicycle movement. Furthermore, it is also preferable to make the environmental sound a sound in an open space as a necessary condition for estimating the movement on a bicycle in combination with the acceleration pattern condition.
(2) A point in time when the measured acceleration fluctuation is below a predetermined threshold for a predetermined period or more, or a point in time when the type of estimated action has changed (for example, changed to (i) running or (b) walking). Is the end point of (i) bicycle movement by the user.

(S302 and S303: (nu) Travel by car)
(1) When the acquired acceleration data has the highest correlation coefficient with the acceleration pattern associated with the movement in the car stored in the presentation information table 112t, The type is estimated to be (nu) traveling by car. Furthermore, it is also preferable to make the environmental sound a sound in a closed space as a necessary condition for estimating (m) movement in an automobile in combination with the conditions of the acceleration pattern.
(2) A point in time when the measured acceleration fluctuation is below a predetermined threshold for a predetermined period or more, or a point in time when the type of estimated action has changed (for example, changed to (i) running or (b) walking). Is the end point of the user's (nu) car movement.

(S302 and S303: (Le) Moving by train)
(1) When the acquired acceleration data has the highest correlation coefficient with the acceleration pattern associated with (le) train movement stored in the presentation information table 112t, The type is estimated as (le) travel by train.
(2) A point in time when the measured acceleration fluctuation is below a predetermined threshold for a predetermined period or more, or a point in time when the type of estimated action has changed (for example, changed to (i) running or (b) walking). Is the end point of the user's (le) train movement.

(S304) Whether the dialogue information to be presented is a conversation (dialog information A) associated with one type of action estimated in the presentation information table 112t (for example, (nu) movement in a car) Information B) is determined. In this decision, for example,
(A) The measured “acceleration” has the highest similarity with the “acceleration pattern” associated with one type of action (for example, (A) running) stored in the presentation information table 112t. A question may be made when there is a similarity that is less than a predetermined threshold (that is, when a clear determination cannot be made). Or
(B) It is also possible to follow an arrangement in which a question is asked only once in 10 times for the estimated number of times of one type of action (for example, (nu) movement in a car).

Furthermore, in the case of (a) above, although there is a type of action that has not been registered in the presentation information table 112t so far (for example, “travel by car”), the type of action is determined from the answer to the question. (For example, if you get information that you have purchased a car)
(A ′) When a standard acceleration pattern separately managed by the presentation information management unit 112 (for example, in “movement by car”) is compared with an acceleration measurement value, and the similarity is equal to or higher than a predetermined threshold It is also possible to ask a question to confirm that this type of action has been performed.
Here, when it is confirmed from the answer to this question that “movement by car” has been performed, “movement by car” and the acceleration measurement value corresponding to this are displayed in the presentation information table 112t. be registered.

Furthermore, for example, the following criteria can be set as to whether or not to speak when estimating the action type / end point.
(A) (I always use good stairs, such as using the stairs, and when I do good, I usually say, “That!” Do about once.
(B) When you do something different from usual (for example, you climbed up the stairs), for example, talk to praise “You did your best!” Every time.
(C) If you have a normal action (such as walking as usual), for example, make a suggestion of "How about running once in a while?"

(S305) If it is determined in step S304 that the conversation information is to be spoken, profile information (profile parameter value) indicating the current state of the user's profile is acquired from the profile management unit 119.
(S306) In the presentation information table 112t, the conversation (dialog information A) associated with the estimated action type and the acquired profile parameter value (including the range) is determined as the dialog information to be presented.
(S307) The determined talk (dialog information A) is presented at a time based on the end time of the action estimated in step S303. Specifically, it is presented to the user by voice or as a character display.

(S308) On the other hand, if it is determined in step S304 that the dialogue information is to be a question, the question (dialog information B) associated with the estimated action type in the presentation information table 112t is determined as the dialogue information to be presented. It is presented at the time based on the end time of the action estimated in step S303. Specifically, a question is presented to the user by voice or as a character display.
(S309) An answer is acquired from the user. Specifically, the voice information of the user's answer may be acquired via the microphone 102, or the input answer information which is answer information by the user's input operation may be acquired via the input unit 106b.

(S310) Information about the user's behavior is extracted from the acquired answer voice information or input answer information.
(S311) The presentation information table 112t is updated based on the information about the user behavior extracted in step S310. Thereafter, the process proceeds to step S300, where the acceleration is monitored again, and preparation is made for presenting dialogue information corresponding to the type of action in a timely manner.

  According to the information presentation method described above, it is possible to present talks / questions corresponding to the user's behavior in a timely manner. In particular, since the end point of the action is often a scene where the user takes a break or is satisfied, the user's response to automatic dialogue information presentation is based on the conversation based on the end point. Negative impression or disgust can be reduced. Furthermore, it becomes easy to obtain a positive or favorable response from the user by the question at the time based on the end time.

  Further, since the user talks after grasping a little daily behavior of the user, it is possible to enhance the user's familiarity with the mobile terminal by setting an appropriate content of the conversation.

[Select dialogue information based on action history]
FIG. 5 is a flowchart showing an embodiment of selection of dialogue information based on an action history according to the present invention.

  As shown in FIG. 5, the action history management unit 120 generates an action history table 120t. In the action history table 120t, a history of action execution time is recorded in association with each type of action by the user, and values indicating the user's health state together with this history (blood pressure value and blood glucose level in FIG. 5). The history of is recorded.

(S500, S501) Using the generated action history table 120t, a correlation coefficient between the action execution time and the value indicating the health state is calculated, and the correlation is determined. For example, a correlation coefficient between a cumulative execution time until a certain time point of a certain action (for example, “run”) and a value (blood pressure value) indicating a health state at each time point is, for example,
When (a) −0.9 or less, (b) a value greater than −0.9 and −0.3 or less, (c) a value greater than −0.3 and 0.3. If it is less than (d) 0.3 or more and less than 0.9, (e) it is discriminated whether it is 0.9 or more.

  Here, in this case, the presentation information table 112t includes a dialogue information group including dialogue information for prompting an action related to the above-described determination (for example, “run”) and dialogue information for restricting the action (for example, “run”). It is recorded in association with the type of action.

(S502) Based on the determination result in step S501, the dialogue information to be presented is selected from the dialogue information group recorded in the presentation information table 112t. For example, in the case of (a) above, when the correlation coefficient between the cumulative execution time until each time point of “running” and the blood pressure value that is a value indicating the health state at each time point is −0.92 The dialogue information that prompts the action related to the above-described determination ("running") is selected and presented. Here, assuming that the user has a high blood pressure value, the lower the blood pressure value is, the better the health condition is handled.

  In addition, since a certain amount of time is required for the action execution time to appear as an effect of the health condition, the history recording period for calculating the correlation coefficient as described above is equivalent to, for example, several weeks or more. It is preferable to set the time.

  Furthermore, it is also preferable to calculate a correlation coefficient between “action execution time” and “health state” of a plurality of users possessing the mobile terminal 1. In this case, for example, in the user behavior / health state management server provided with a communication function in the portable terminal 1 and installed on the network, the “action execution time” and the “health state” of each user are set to the plurality of portable terminals 1 May be aggregated via communication. Next, the mobile terminal 1 acquires data indicating the relationship between “action execution time” and “health state” of a plurality of users from the user behavior / health state management server, and the correlation coefficient between the two is obtained. “Action types” and “health states” that are 0.9 or more (positive correlation) or −0.9 or less (negative correlation) are extracted. Here, the “health state” is defined by an amount indicating a better health state as the value is larger.

  Further, by using the extracted result, a specific “action execution time” having a high positive correlation coefficient (having a correlation coefficient of 0.9 or more) with a certain “health state” is a predetermined time. If it is less than the threshold value, dialogue information that prompts the action is selected and presented, and has a high negative correlation coefficient with another “health state” (correlation coefficient of −0.9 or less). If a specific “action execution time” is equal to or greater than a predetermined time threshold, dialogue information that prompts the user to refrain from the action may be selected and presented.

  As described above, by generating and using the action history table 120t, even when the same action type is estimated, it is possible to present a more useful conversation about health based on the past action history. It becomes.

  Various changes, modifications, and omissions of the above-described various embodiments of the present invention can be easily made by those skilled in the art. The above description is merely an example, and is not intended to be restrictive. The invention is limited only as defined in the following claims and the equivalents thereto.

1 Mobile terminal (user interface device)
DESCRIPTION OF SYMBOLS 100 Acceleration sensor 101 Barometric pressure sensor 102 Microphone 103 Positioning part 103
DESCRIPTION OF SYMBOLS 104 Floor map storage part 105 Speaker 106a Display part 106b Input part 107 Health / mood information storage part 108 Hobby preference information storage part 109 Habit action information storage part 110 Position determination part 111 Acceleration data generation part 112 Presentation information management part 112t Presentation information table 113 Action Estimation Unit 114 End Time Estimation Unit 115 Information Presentation Unit 116 Vertical Movement Speed Determination Unit 117 Environmental Sound Determination Unit 117
118 voice determination unit 119 profile management unit 120 behavior history management unit 120t behavior history table 121 behavior correlation determination unit 122 answer information extraction unit 123 speech synthesis unit

Claims (18)

A user interface device that presents information corresponding to user behavior,
An acceleration measuring unit capable of measuring acceleration;
For each type of action, a presentation information management means for storing an acceleration pattern at the time of the action and dialogue information to be presented in association with each other,
Action estimating means for estimating the type of action by the user based on the measured acceleration and the stored acceleration pattern of the action;
End point estimation means for estimating the end point of the action by the user based on the measured acceleration;
Information presenting means for presenting dialogue information associated with the estimated type of the action at a time based on the estimated end time of the action ;
When the presented dialogue information includes a question for confirming the estimated type of the action, the user's answer to the question is acquired, and information about the user's action is extracted from the answer Answer information extracting means to have have a <br/>,
When the presentation information management means determines that the acceleration pattern at the time of the action associated with the type of action related to the extracted information about the action of the user does not correspond to the measured acceleration, A user interface device , wherein an acceleration pattern at the time of the action associated with the type is updated . Before SL activity estimation means, between the measured acceleration, a type of the action associated with the acceleration pattern having the highest similarity or predetermined threshold value or more similarity, to the type of action to be estimated the user interface device of claim 1, wherein the Turkey. Before Symbol presenting information management means with the information about the behavior of the extracted the user, and updates the interactive information to present associated with the type of the type and the behavior of the action associated with the type of the action Update the acceleration pattern at the time of the action, or register the new action type related to the extracted action information of the user, and associate the acceleration pattern based on the acceleration data obtained at the time of the action the user interface device of claim 2, wherein the benzalkonium is stored Te. Profile management means for recording a value of at least one profile parameter indicating the profile of the user;
The presentation information management means stores the interactive information to be presented in association with each type of action and for each at least one profile parameter,
The information presenting means causes the dialogue information associated with the estimated type of the action and the recorded profile parameter value to be presented at a time based on the estimated end time of the action, The user interface device according to any one of claims 1 to 3.   The at least one profile parameter value is at least one of a value indicating the user's health condition / mood, a value indicating the user's hobbies, and a value indicating the user's habit behavior. The user interface device according to claim 4. An action history management unit that records a history of the execution time of the action in association with each type of the action, and records a history of values indicating the health state and mood of the user together with the history;
Action correlation determining means for determining the correlation between the time for performing the action and a value indicating the health state / mood,
The presentation information management means stores a dialogue information group including dialogue information for prompting the action related to the discrimination and dialogue information for restricting the action in association with the type of the action,
6. The user interface device according to claim 5, wherein the information presentation unit selects the dialogue information to be presented from the dialogue information group based on the determination result by the behavior correlation determination unit. A microphone that detects ambient sound,
Voice determination means for determining whether or not the detected voice is a voice of the user,
The information presenting means includes the estimated action in spite of being at a time based on the estimated end time of the action at least within a predetermined time from the time when the true determination is performed by the voice determining means. The user interface device according to any one of claims 1 to 6, wherein dialogue information associated with the type of the user is not presented. The types of behavior are: running, walking, climbing stairs, descending stairs, climbing by elevator, descending by elevator, climbing by escalator, descending by escalator, traveling by bicycle, traveling by car, and train The user interface device according to claim 1, wherein the user interface device includes at least one of movements. An atmospheric pressure sensor for measuring atmospheric pressure;
A vertical movement speed determining means for determining a vertical movement speed from the detected atmospheric pressure value;
The presenting information management means stores the ascending or descending on the escalator as the type of action in association with the moving speed in the up and down direction corresponding to the ascending or descending on the escalator and the dialogue information to be presented. Let
The behavior estimating means has a difference between a measured acceleration fluctuation that is equal to or less than a predetermined threshold and a determined vertical movement speed and a vertical movement speed corresponding to an up or down movement on the escalator. The user interface device according to claim 8, wherein when it is within a predetermined range, the ascending or descending on the escalator is set as the type of action to be estimated. A microphone that detects environmental sounds;
And environmental sound determination means for determining whether the detected environmental sound is a sound in a closed space or a sound in an open space,
The presentation information management means corresponds to the movement of the bicycle as the type of action, the acceleration pattern corresponding to the movement of the bicycle, the information that there is a sound in the open space, and the dialogue information to be presented And remember
The said action estimation means makes the determination that it is the sound in the open space by the said environmental sound determination means as a necessary condition for estimating the movement with a bicycle, The Claim 8 or 9 characterized by the above-mentioned. User interface device. A microphone that detects environmental sounds;
And environmental sound determination means for determining whether the detected environmental sound is a sound in a closed space or a sound in an open space,
The presentation information management means corresponds to the movement in the automobile as the type of action, the acceleration pattern corresponding to the movement in the automobile, the information that there is sound in a closed space, and the dialogue information to be presented To remember,
11. The behavior estimation means uses the determination that the sound is a closed space by the environmental sound determination means as a necessary condition for estimating a movement in an automobile. The user interface device according to item 1.   The end time estimation means is a time when the measured acceleration fluctuation is equal to or less than a predetermined threshold, or a time when the type of the action estimated by the action estimation means changes is an end time of the action by the user. The user interface device according to claim 1, wherein the user interface device is estimated.   A user interface device that presents information corresponding to user behavior,
  An acceleration measuring unit capable of measuring acceleration;
  A microphone that detects ambient sound,
  Voice determination means for determining whether or not the detected voice is a voice of a user;
  Presentation information management means for storing the interaction information to be presented in association with each type of action,
  Action estimating means for estimating the type of action by the user based on the measured acceleration;
  End point estimation means for estimating the end point of the action by the user based on the measured acceleration;
  Information presenting means for presenting dialogue information associated with the estimated type of the action at a time based on the estimated end time of the action;
Have
  The information presenting means includes the estimated action in spite of being at a time based on the estimated end time of the action at least within a predetermined time from the time when the true determination is performed by the voice determining means. Don't present conversation information associated with different types
A user interface device.   A user interface device presenting information corresponding to a user's behavior including ascending or descending on an escalator,
  An acceleration measuring unit capable of measuring acceleration;
  An atmospheric pressure sensor for measuring atmospheric pressure;
  A vertical movement speed determining means for determining a vertical movement speed from the detected atmospheric pressure value;
  Presentation information management means for storing the interaction information to be presented in association with each type of action,
  Action estimating means for estimating the type of action by the user based on the measured acceleration;
  End point estimation means for estimating the end point of the action by the user based on the measured acceleration;
  Information presenting means for presenting dialogue information associated with the estimated type of the action at a time based on the estimated end time of the action;
Have
  The presenting information management means stores the ascending or descending on the escalator as the type of action in association with the moving speed in the vertical direction corresponding to the ascending or descending on the escalator and the interactive information to be presented. Let
  The behavior estimating means has a difference between a measured acceleration fluctuation that is equal to or less than a predetermined threshold and a determined vertical movement speed and a vertical movement speed when the escalator is up or down. When it is within the specified range, going up or down on the escalator is the type of action to be estimated
A user interface device.   A user interface device that presents information corresponding to a user's behavior including movement on a bicycle as its type,
  An acceleration measuring unit capable of measuring acceleration;
  A microphone that detects environmental sounds;
  Environmental sound determination means for determining whether the detected environmental sound is a sound in a closed space or a sound in an open space;
  Presentation information management means for storing the interaction information to be presented in association with each type of action,
  Action estimating means for estimating the type of action by the user based on the measured acceleration;
  End point estimation means for estimating the end point of the action by the user based on the measured acceleration;
  Information presenting means for presenting dialogue information associated with the estimated type of the action at a time based on the estimated end time of the action;
Have
  The presentation information management means corresponds to the movement of the bicycle as the type of action, the acceleration pattern corresponding to the movement of the bicycle, the information that there is sound in the open space, and the interactive information to be presented. And remember
  The behavior estimation means uses the determination that the sound is an open space by the environmental sound determination means as a necessary condition for estimating a movement on a bicycle.
A user interface device. A user interface device that presents information corresponding to a user's action including movement in a car as its type,
  An acceleration measuring unit capable of measuring acceleration;
  A microphone that detects environmental sounds;
  Environmental sound determination means for determining whether the detected environmental sound is a sound in a closed space or a sound in an open space;
  Presentation information management means for storing the interaction information to be presented in association with each type of action,
  Action estimating means for estimating the type of action by the user based on the measured acceleration;
  End point estimation means for estimating the end point of the action by the user based on the measured acceleration;
  Information presenting means for presenting dialogue information associated with the estimated type of the action at a time based on the estimated end time of the action;
Have
  The presentation information management means corresponds to the movement of the car as the type of action, the acceleration pattern corresponding to the movement of the car, the information that there is sound in a closed space, and the dialogue information to be presented. And remember
  The behavior estimation means uses the determination that the sound is a closed space by the environmental sound determination means as a necessary condition for estimating movement in an automobile.
A user interface device. An information presentation program for causing a computer mounted on a user interface device to present information corresponding to a user's behavior,
The apparatus includes an acceleration measuring unit capable of measuring acceleration,
The information presentation program includes:
For each type of action, a presentation information management means for storing an acceleration pattern at the time of the action and dialogue information to be presented in association with each other,
Action estimating means for estimating the type of action by the user based on the measured acceleration and the stored acceleration pattern of the action;
End point estimation means for estimating the end point of the action by the user based on the measured acceleration;
Information presenting means for presenting dialogue information associated with the estimated type of the action at a time based on the estimated end time of the action ;
When the presented dialogue information includes a question for confirming the estimated type of the action, the user's answer to the question is acquired, and information about the user's action is extracted from the answer to be <br/> the answer information extraction unit to function the computer,
When the presentation information management means determines that the acceleration pattern at the time of the action associated with the type of action related to the extracted information about the action of the user does not correspond to the measured acceleration, Update the acceleration pattern during the action associated with the type
Information presentation program, wherein a call. An information presentation method for presenting information corresponding to user behavior,
A first step of storing, in association with each action type, an acceleration pattern at the time of the action and dialogue information to be presented;
A second step of measuring acceleration due to the user's behavior;
A third step of estimating the type of action by the user based on the measured acceleration and the stored acceleration pattern at the time of the action ;
A fourth step of estimating an end point of the action by the user based on the measured acceleration;
A fifth step of presenting dialogue information associated with the estimated type of the action at a time based on the estimated end time of the action ;
When the presented dialogue information includes a question for confirming the estimated type of the action, the user's answer to the question is acquired, and information about the user's action is extracted from the answer A sixth step,
When it is determined that the acceleration pattern at the time of the action associated with the type of action related to the extracted information about the action of the user does not correspond to the measured acceleration, the action associated with the type of action An information presentation method comprising: a seventh step of updating an acceleration pattern during action .

Images