JP2008003655A - Information processor, information processing method, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To generate, for example, an entertaining sentence automatically, in accordance with a user's behavior, without depending on the user's input. <P>SOLUTION: In the information processor, measurement by an acceleration sensor 11 and an inertial sensor 12 is repeated and, on the basis of a daily measurement result, for example, the number of steps, an average walking pace, the average number of turns, the average number of walks or stops of a user of a day, are obtained. A dictionary stored in an experience expression dictionary storage part 23 is referred to and sentences indicating the user's day's behaviors are generated according to the obtained number of steps, average walking pace, average number of turns, average number of average walks or stops, and the like, of the user. The experience expression dictionary storage part 23 stores a dictionary showing correlation between the sentences and the behaviors, the sentences having entertainment. Thus, the user can apply them to portable mobile equipment or the like. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to an information processing device, an information processing method, and a program, and in particular, for example, it is possible to automatically generate, for example, an entertainment sentence corresponding to a user's action without depending on the user's input. The present invention relates to an information processing device, an information processing method, and a program.

  Today, Internet-based Blog (Weblog), SNS (Social Network Service), etc. are tools for recording your actions and disclosing your actions to specific or unspecified people. Generalized.

  FIG. 1 is a diagram showing an example of a blog. As shown in FIG. 1, the user may leave his / her own action as a sentence in a diary or leave a photographed image. Can do.

  In Patent Document 1, the user's activity status such as walking and running obtained from the output of the acceleration sensor, the position of the user acquired from GPS (Global Positioning System), etc., and a combination of a plurality of sensor output results are combined with the user. A technique for recording as an action history is disclosed.

Japanese Patent Laid-Open No. 2004-228688 determines a story according to a static condition that is a temporal condition and a dynamic condition that is a user's action history, sequentially selects the corresponding text, and expresses the user's action history. A technique for automatically generating the above is disclosed.
JP 2003-296882 JP-A-2002-342764

  However, it is very troublesome to update the above-mentioned Blog etc. every day, and there are not a few users who give up updating continuously.

  Therefore, in order to reduce such a burden, various technologies such as those disclosed in Patent Document 1 and Patent Document 2 have been proposed, but sensors such as GPS that receive and detect radio waves are provided. In many cases, it is not practical because the apparatus is large-scale and some users are burdened when used.

  Also, the action history is expressed in the form shown in Fig. 2, such as a list of times, actions, and places. Many are not suitable.

  For example, according to the technique described in Patent Document 1, although the user's behavior history can be logged based on the sensor output, since the recognition result is recorded and output as it is, the output information is entertainment. It will be lacking in nature.

  Moreover, in the technique described in Patent Document 2, sentences such as words are determined mainly based on the impressions input by the user, so that the input of the impressions is complicated.

  The present invention has been made in view of such circumstances, and for example, it is possible to automatically generate, for example, an entertainment sentence corresponding to a user's action without depending on the user's input. is there.

  An information processing apparatus according to an aspect of the present invention includes a storage unit that recognizes a behavior parameter that is information related to a user's behavior and text information that is recognized based on an output of a sensor, and stores the text information in association with each other. Selection means for selecting text information corresponding to the behavior parameter extracted by the extraction means from the text information stored by the storage means when the behavior parameters that have been set match a predetermined condition; Prepare.

  Generation means for generating the behavior parameter by combining a plurality of parameters of the sensor output may be further provided.

  Normalization means for normalizing the behavior parameter may be further provided.

  Generation means for generating an experience parameter expressing the degree of user's action by combining a plurality of the action parameters can be further provided.

  The selection means can preferentially select unselected text information from the text information.

  The storage means can further store information relating to the date and the text data in association with each other.

  An information processing method or program according to an aspect of the present invention stores an action parameter that is recognized based on an output of a sensor and that is information about a user action and text information in association with each other, and that is extracted If the predetermined condition is satisfied, the step includes selecting text information corresponding to the extracted behavior parameter from the stored text information.

  In one aspect of the present invention, a behavior parameter that is information related to a user's behavior recognized based on the output of the sensor and text information are stored in association with each other, and the extracted behavior parameter matches a predetermined condition. In this case, text information corresponding to the extracted behavior parameter is selected from the stored text information.

  According to one aspect of the present invention, for example, an entertainment sentence corresponding to a user's action can be automatically generated without depending on the user's input.

  Embodiments of the present invention will be described below. Correspondences between the constituent elements of the present invention and the embodiments described in the specification or the drawings are exemplified as follows. This description is intended to confirm that the embodiments supporting the present invention are described in the specification or the drawings. Therefore, even if there is an embodiment which is described in the specification or the drawings but is not described here as an embodiment corresponding to the constituent elements of the present invention, that is not the case. It does not mean that the form does not correspond to the constituent requirements. On the contrary, even if an embodiment is described herein as corresponding to the invention, this does not mean that the embodiment does not correspond to other than the configuration requirements. .

  The information processing apparatus according to one aspect of the present invention (for example, the mobile phone 1 in FIG. 3) stores a behavior parameter, which is information related to a user's behavior, recognized based on the output of the sensor, and text information in association with each other. When the storage unit (for example, the experience expression dictionary storage unit 23 in FIG. 5) and the behavior parameter extracted by the extraction unit match predetermined conditions, the text information stored in the storage unit And selection means (for example, the experience expression selection unit 72 in FIG. 21) for selecting text information corresponding to the behavior parameters extracted by the extraction means.

  Generation means (for example, the behavior parameter extraction / behavior recognition unit 21 in FIG. 5) that generates the behavior parameter by combining a plurality of parameters of the sensor output may be further provided.

  Normalization means for normalizing the behavior parameters (for example, behavior parameter normalization unit 61 in FIG. 18) may be further provided.

  Generation means (for example, the experience parameter generation unit 63 in FIG. 18) for generating an experience parameter that expresses the degree of user's action by combining the plurality of action parameters may be further provided.

  An information processing method or program according to an aspect of the present invention stores an action parameter that is recognized based on an output of a sensor and that is information about a user action and text information in association with each other, and that is extracted Includes a step of selecting text information corresponding to the extracted behavior parameter from the stored text information (for example, step S24 in FIG. 27).

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  FIG. 3 is a diagram showing a configuration example of an information processing system to which the present invention is applied.

  As shown in FIG. 3, the information processing system is configured by connecting a mobile phone 1, a server 2, and a personal computer 3 via the Internet.

  The mobile phone 1 recognizes the user's behavior based on the output from the built-in sensor, and automatically generates a sentence representing the user's behavior according to the recognition result. The mobile phone 1 incorporates sensors such as an acceleration sensor and a gyro sensor. For example, how long the user walks in a day, whether he / she walks straight, or walks while walking. It is recognized from the output of the sensor.

  The mobile phone 1 stores text representing the user's behavior in association with the behavior. For example, when it is recognized that the user has walked more than usual, the text “I walked a lot” is selected. When it is recognized that he was walking while walking, the text “Shopping today” is selected. Whether or not the person has walked more than usual is determined based on the average walking distance recorded daily as a history. Whether or not he / she was walking while walking is determined based on how many times the number of turns to the left or the right is detected at a predetermined time.

  The cellular phone 1 transmits the sentence generated as described above to the server 2 via the Internet, and registers it in the server 2 as a blog or SNS sentence. Action recognition is performed using, for example, an acceleration sensor and a gyro sensor output history during a day, and a sentence representing a daily action created based on the recognized action is once a day. It is created at a predetermined cycle.

  The user of the personal computer 3 can access the server 2 by operating the personal computer 3 and browse the text generated by the mobile phone 1 that is published as a blog or SNS text of the user of the mobile phone 1. it can.

  As described above, in the mobile phone 1, since the action is recognized based on the outputs of the acceleration sensor and the gyro sensor, the action is recognized based on the output of the GPS or the like that receives and detects the radio wave. Compared to the above, the system can be miniaturized.

  In addition, as shown in FIG. 2, the user can easily enter entertainment sentences suitable for use in Blog and SNS, not just a list of actions such as “walking”, “running”,... Can be created.

  FIG. 4 is a block diagram illustrating a hardware configuration example of the mobile phone 1.

  As shown in FIG. 4, the mobile phone 1 includes an acceleration sensor 11, a gyro sensor 12, and a processor 13.

  The acceleration sensor 11 is a sensor capable of measuring accelerations in three axial directions orthogonal to each other, performs measurement at a predetermined cycle, and outputs acceleration data as a measurement result to the processor 13.

  The gyro sensor 12 is a sensor capable of measuring angular velocities around three axes orthogonal to each other, performs measurement at a predetermined cycle, and outputs gyro data as a measurement result to the processor 13.

  The processor 13 recognizes the action of the user carrying the mobile phone 1 based on the acceleration data supplied from the acceleration sensor 11 and the gyro data supplied from the gyro sensor 12, and creates a sentence according to the recognition result. To do.

  In addition, the processor 13 transmits the created text to the server 2 via the Internet, and registers it in Blog or SNS created by the user of the mobile phone 1 in advance. The processor 13 also displays the created text on a display provided in the mobile phone 1 and allows the user to confirm.

  FIG. 5 is a block diagram illustrating a functional configuration example of the mobile phone 1. At least a part of the functional units shown in FIG. 5 is realized by executing a predetermined program by the processor 13 of FIG.

  As shown in FIG. 5, in the mobile phone 1, a behavior parameter extraction / behavior recognition unit 21, an experience expression generation unit 22, and an experience expression dictionary storage unit 23 are realized.

  The behavior parameter extraction / behavior recognition unit 21 measures, for example, the walking pitch, the walking power, and the rotation angle around the gravity axis based on the supplied sensor data, measured by the acceleration sensor 11 and the gyro sensor 12, From the information, the number of steps, the average walking pitch, the average number of turns, and the average number of walks / stops are obtained.

  In addition, the behavior parameter extraction / recognition unit 21 performs “stationary”, “walking”, “running”, “right” based on the walking pitch, walking power, and rotation angle around the gravity axis obtained based on the sensor data. Either “turn” or “left turn” is recognized as a user action, and the time, ratio, number of times, duration, etc. of each action are obtained.

  As will be described later, the behavior parameter extraction / recognition unit 21 walks through an HMM (Hidden Markov Model) generated based on the sensor data output from the acceleration sensor and the gyro sensor in a stationary state. HMM generated based on the sensor data output from the acceleration sensor and gyro sensor when in the running state, HMM generated based on the sensor data output from the acceleration sensor and gyro sensor in the running state, HMM generated based on sensor data output from acceleration sensor and gyro sensor when making a right turn, generated based on sensor data output from acceleration sensor and gyro sensor when making a left turn HMMs are prepared in advance, and by referring to those HMMs, it is possible to select “stationary”, “walking”, “running”, “ Turn ", any of the behavior of the" left turn "is recognized.

  As shown in FIG. 6, the behavior parameter extraction / behavior recognition unit 21 describes various information obtained based on sensor data such as acceleration data and gyro data, and information representing the history of recognized behavior. A behavior parameter file is generated, and the generated behavior parameter file is output to the experience expression generation unit 22. The behavior parameter file is, for example, an XML (eXtensible Markup Language) format file.

  FIG. 7 is a diagram illustrating an example of information described in the behavior parameter file.

  As shown in FIG. 7, the behavior parameter file describes the start time of the log, the end time of the log, and the total time of the log. The log start time represents the time when the user started using the mobile phone 1, and the log end time represents the time when the user finished using the mobile phone 1. The total log time represents the time that the user was using the mobile phone 1 during the day.

  Also, the behavior parameter file contains the number of steps, average walking pitch, average number of turns, average number of walks / stops, total time, percentage, number of times, duration, maximum time, average of each recognized action Time is described.

  The maximum time of action is the longest time that is continuously recognized as being stationary for “stationary”, and the time that is continuously recognized as walking for “walking”. For the longest time, “running”, the longest time continuously recognized as being run, and for “right turn”, the time continuously recognized as having been making a right turn The longest time among them, the “left turn”, is the longest time continuously recognized as having made a left turn.

  The average action time is the average time (seconds) that is continuously recognized as being stationary for “still”, and the average time that is continuously recognized as walking for “walking”. For time (seconds) and “running”, the average time (seconds) continuously recognized as being run, and for “right turn”, the time continuously recognized as having been making a right turn The average time of “left turn” is the average time continuously recognized as a left turn.

  The action parameter file describes a time series of recognized actions. The information of one action described includes, for example, the start time, end time, total time, and action content of one action.

  A behavior parameter file in which such parameters are described is generated by the behavior parameter extraction / behavior recognition unit 21 and output to the experience expression generation unit 22. For example, one behavior parameter file as shown in FIG. 7 is generated every day based on the sensor data detected by the acceleration sensor 11 and the gyro sensor 12 during the day.

  Returning to the description of FIG. 5, the experience expression generation unit 22 refers to the dictionary stored in the experience expression dictionary storage unit 23 and is described in the behavior parameter file supplied from the behavior parameter extraction / behavior recognition unit 21. Based on the information, a sentence representing the daily action of the user is created. The experience expression generation unit 22 transmits the created text to the server 2 via the Internet or displays the text on the display device 24 provided in the mobile phone 1.

  The experience expression dictionary storage unit 23 stores a dictionary representing the correspondence between sentences and actions. The dictionary stored in the experience expression dictionary storage unit 23 may be prepared in advance by the manufacturer of the mobile phone 1 or the like. It may be generated and prepared by learning from the recognition result of the action recognized by the action parameter extraction / behavior recognition unit 21 and the text input by the user.

  Here, behavior parameter extraction and behavior recognition performed by the behavior parameter extraction / behavior recognition unit 21 will be described.

  FIG. 8 is a block diagram illustrating a detailed configuration example of the behavior parameter extraction / behavior recognition unit 21.

  As shown in FIG. 8, the behavior parameter extraction / behavior recognition unit 21 includes a calibration execution unit 31, a walking pitch detection unit 32, a walking power detection unit 33, a gravity direction detection unit 34, a heading detection unit 35, and a behavior recognition unit. 36 and a file generation unit 37.

  The calibration execution unit 31 calibrates the sensor data supplied from the acceleration sensor 11 and the gyro sensor 12, and uses the sensor data after calibration as the walking pitch detection unit 32, the walking power detection unit 33, and the gravity direction detection unit 34. , Output to the heading detector 35. For example, the sensitivity of the acceleration sensor 11 and the gyro sensor 12 are different depending on the temperature or the like, and the output values are also different. Therefore, the calibration execution unit 31 performs a process for correcting the sensitivity.

  The walking pitch detection unit 32 detects the walking pitch as shown in FIG. 9 based on the acceleration data in the sensor data supplied from the calibration execution unit 31. As a method for detecting and measuring the period of a periodic signal, there is a method of finding a peak in signal autocorrelation. For example, in the detection of a walking pitch using data measured by the acceleration sensor 11, for example, The method is used. The horizontal axis in FIG. 9 represents time (the number of samples of the acceleration sensor 11), and the vertical axis represents frequency (Hz). Since the frequency measured by the acceleration sensor when a person walks is generally 2 Hz (2 steps per second), from the time series of measurement results of frequencies around 2 Hz, the “walk” as the recognition result is Obtained by referring to the HMM. The walking pitch detection unit 32 outputs the detected walking pitch to the action recognition unit 36.

  The walking power detection unit 33 detects the walking power as shown in FIG. 10 based on the acceleration data among the sensor data supplied from the calibration execution unit 31. The horizontal axis in FIG. 10 represents time, and the vertical axis represents strength. The walking power detection unit 33 outputs the detected walking power to the action recognition unit 36.

  The gravity direction detection unit 34 detects the direction of the gravity axis based on acceleration data among the sensor data supplied from the calibration execution unit 31. When the acceleration sensor 11 is in a state of detecting acceleration in the horizontal direction with respect to the gravity axis, the output is 0. On the other hand, when the acceleration sensor 11 is in a state of detecting acceleration in a direction other than the horizontal direction with respect to the gravity axis. Since a predetermined value is measured, as described above, when it is possible to measure the acceleration in the three-axis direction, the gravity axis can be obtained from the measurement result of the acceleration in each direction. Become. FIG. 11 shows the change in the gravitational direction of acceleration. The horizontal axis in FIG. 11 represents time, and the vertical axis represents acceleration. The gravity direction detector 34 outputs information indicating the detected direction of the gravity axis to the heading detector 35.

  The heading detection unit 35 is shown in FIG. 12 based on gyro data in the sensor data supplied from the calibration execution unit 31 and information indicating the direction of the gravity axis supplied from the gravity direction detection unit 34. The rotation angle (Heading) around the gravity axis is detected. When the gyro data is projected in the direction of the gravity axis, the rotation angle around the gravity axis is obtained. The horizontal axis of FIG. 12 represents time, and the vertical axis represents the rotation angle. The heading detection unit 35 outputs the detected rotation angle around the gravity axis to the action recognition unit 36.

  Based on the information supplied from the walking pitch detection unit 32, the walking power detection unit 33, the gravity direction detection unit 34, and the heading detection unit 35, the behavior recognition unit 36 determines the number of steps, average walking pitch, average number of turns, and average. The number of times of walking / stopping is obtained, and the obtained information is extracted as an action parameter.

If the walking pitch obtained by the walking pitch detection unit 32 is represented by P _walk (i), the number of steps N _walk (steps) is obtained by the following equation (1). The obtained number of steps is used as a guideline for determining how much the user has walked in one day, for example. In equation (1), i is the order of samples and N is the total number of samples.

Further, _assuming that the number of samples whose walking pitch is 0 or more is represented by C _walk , the average walking pitch A _walk (Hz) is obtained by the following equation (2). The obtained average walking pitch is used as a standard for judging the speed of walking (whether walking was busy or leisurely walking).

  The average number of turns can be obtained by counting the number of turns from the rotation angle around the gravity axis supplied from the heading detector 35 and setting it as the number of turns per unit time. For example, in FIG. 12, a position where a rotation angle value of 0.1 or more is detected is counted as a position where the user has made a left turn, and a position where a rotation angle value of −0.1 or less is detected is the user. Is counted as a position where a right turn is made.

Assuming that the number of turns per minute (the number of left turns and right turns combined) is represented by C _turn and the number of samples per _minute is represented by N _min , the average number of turns A _turn (Times / min) is obtained by the following equation (3). The obtained average number of turns is used as a guideline for judging whether the player has been acting straight or swaying.

The average number of times of walking / stopping is obtained by detecting the position where the user is walking from the walking pitch obtained by the walking pitch detection unit 32 and counting the number of detected positions. For example, since a person's walking pitch is about 2.0 Hz on average, a position where a frequency of 1.0 Hz or higher is detected is detected as a position where the user is walking. _Assuming that the number of walks / stops per minute is represented by C _{stop_go} and the number of samples per _minute is represented by N _min , the average number of walks / stops A _{stop_go} (times / minute) is Is required. The obtained average number of walks / stops is used as a guideline for determining whether the user has taken a steady action or has taken an action.

  The number of steps extracted as behavior parameters as described above, the average walking pitch, the average number of turns, and the average number of walks / stops are output to the file generation unit 37.

  FIG. 13 is a diagram illustrating an example of some functional units provided in the action recognition unit 36.

  As shown in FIG. 13, the action recognition unit 36 displays the user's action in addition to the function unit for obtaining the number of steps, the average walking pitch, the average number of turns, and the average number of times of walking / stopping as described above. A functional unit recognized by the HMM is also provided.

  The recognizing unit 51 accumulates a predetermined amount of information supplied from the walking pitch detecting unit 32, the walking power detecting unit 33, the gravity direction detecting unit 34, and the heading detecting unit 35, and based on the time series of the accumulated feature data. The action of the user of the mobile phone 1 is recognized with reference to the HMM prepared in the HMM database 52.

  For example, when the recognizing unit 51 recognizes the user's action for one day, the total time of each action of “stationary”, “walking”, “running”, “right turn”, and “left turn”, The ratio of actions, the number of times of each action, the duration (second) of each action, the maximum duration (second) of each action, and the average duration (second) of each action are obtained. The total time of each action, the ratio of each action, and the number of times of each action are used as a standard for judging the tendency of the user's action on that day.

  Further, the recognition unit 51 may generate action information per arbitrary unit time including action start time, end time, and action taken in the unit time. This behavior information per unit time can be used, for example, as a guideline for determining behavioral transitions in time series.

  The HMM database 52 includes an HMM generated based on the sensor data output from the acceleration sensor 11 and the gyro sensor 12 when in a stationary state, and the acceleration sensor 11 and the gyro sensor 12 when in a walking state. HMM generated based on output sensor data, acceleration sensor 11 when running, HMM generated based on sensor data output from gyro sensor 12, acceleration sensor when making a right turn HMM generated based on sensor data output from gyro sensor, acceleration sensor when left turn is performed, HMM generated based on sensor data output from gyro sensor are prepared in advance By referring to the HMM, "stationary", "walking", "running", "right turn", "left turn" "Is recognized by the recognition unit 51.

  FIG. 14 is a diagram schematically illustrating action recognition by the recognition unit 51.

  The acceleration data and the gyro data as raw data are calibrated, and from the data obtained by the calibration, the walking pitch, walking power, and gravity axis as described above The direction and the rotation angle around the gravity axis are acquired as a low level context. Also, the HMM is referred to, and the user's behavior (high level context) is statistically recognized based on the acquired feature amount.

  In addition, action recognition may be performed not only as shown in FIG. 14 but also by various algorithms. For example, based on the rotation angle information about the gravity axis obtained by the heading detection unit 35, the position where the rotation angle value of 0.1 or more is detected is recognized as the position where the user has made a left turn, − An action can be recognized simply by threshold processing of sensor data, such as recognizing a position where a rotation angle value of 0.1 or less is detected as a position where the user has made a right turn.

  FIG. 15 is a diagram showing examples of sensor data, feature amounts, and recognition results in time series. In the example of FIG. 15, “walk”, “run”, “walk”, “run”, “still” The action recognition results are acquired in the order of “walking” and “still”.

  The result of the action recognition recognized as described above is also output from the action recognition unit 36 to the file generation unit 37 together with information such as the time and time when the user performed the action. In addition, information indicating the ratio of each action, the number of times of each action, the duration of each action, the maximum duration of each action, and the average duration of each action is also output to the file generation unit 37. .

  Returning to the description of FIG. 8, the file generation unit 37 generates a behavior parameter file in which the information supplied from the behavior recognition unit 36 is described, and outputs the generated behavior parameter file to the experience expression generation unit 22 (FIG. 5). To do.

  16 and 17 are diagrams illustrating examples of specific descriptions of behavior parameter files generated by the file generation unit 37. FIG. The numbers shown on the left side of FIGS. 16 and 17 are given for convenience of explanation and do not constitute information described in the behavior parameter file.

  “<Start> ˜ </ start>” described in the second line in FIG. 16 corresponds to the log start time in FIG. 7, and “<stop> ˜ </ stop>” described in the third line. "Corresponds to the log end time in FIG. “<Duration> ˜ </ duration>” described in the fourth line corresponds to the total time shown in FIG.

  “<Pedometer> ˜ </ pedometer>” described in the seventh line in FIG. 16 corresponds to the number of steps in FIG. 7, and “<pitch> ˜ </ pitch>” described in the eighth line This corresponds to the average walking pitch in FIG. “<Turn_freq> ˜ </ turn_freq>” described in the ninth line corresponds to the average number of turns shown in FIG. 7, and “<walk_stop_freq> ˜ </ walk_stop_freq>” described in the tenth line This corresponds to the average number of walks / stops referred to in FIG.

  The description from the 12th line to the 16th line in FIG. 16 corresponds to the total time of each action in FIG. “<Still> to </ still>” described in the 12th line represents the total time recognized as “still”, and “<walk> to </ walk>” described in the 13th line Represents the total time recognized as “walking”. "<Run> ~ </ run>" described in the 14th line represents the total time recognized as "running", and "<right> ~ </ right>" described in the 15th line is Represents the total time recognized as a “right turn”. “<Left> ˜ </ left>” described in the 16th line represents the total time recognized as “left turn”.

  The description from the 19th line to the 23rd line in FIG. 16 corresponds to the ratio of each action in FIG. “<Still> to </ still>” described in the 19th line represents the ratio of the time recognized as “still” to the total time in which the action is recognized, and “<still” described in the 20th line. “walk> ˜ </ walk>” represents the ratio of the time recognized as “walk” to the total time when the action was recognized. “<Run> ˜ </ run>” described in the 21st line represents the ratio of the time recognized as “running” to the total time in which the action is recognized, and “<run” described in the 22nd line. “right> ˜ </ right>” represents the ratio of the time recognized as the “right turn” to the total time when the action is recognized. “<Left> ˜ </ left>” described in the 23rd line represents the ratio of the time recognized as the “left turn” to the total time when the action is recognized.

  The description from the 26th line to the 30th line in FIG. 16 corresponds to the number of times of each action in FIG. “<Still> ˜ </ still>” described in the 26th line represents the number of times recognized as “still”, and “<walk> ˜ </ walk>” described in the 27th line “ Indicates the number of times recognized as "walking". “<Run> ˜ </ run>” described in the 28th line represents the number of times recognized as “running”, and “<right> ˜ </ right>” described in the 29th line is “ Indicates the number of times recognized as “right turn”. “<Left> ˜ </ left>” described in the 30th line represents the number of times recognized as “left turn”.

  The description from the 34th line to the 38th line in FIG. 16 corresponds to the maximum time of each action shown in FIG. “<Still> ˜ </ still>” described in the 34th line represents the maximum time recognized as “still”, and “<walk> ˜ </ walk>” described in the 35th line Represents the maximum time recognized as “walking”. "<Run> ~ </ run>" described in the 36th line represents the maximum time recognized as "running", and "<right> ~ </ right>" described in the 37th line is Represents the maximum time recognized as a “right turn”. “<Left> ˜ </ left>” described in the 38th line represents the maximum time recognized as “left turn”.

  The description from the 41st line to the 45th line in FIG. 16 corresponds to the average time of each action in FIG. “<Still> ˜ </ still>” described in the 41st line represents an average time recognized as “still”, and “<walk> ˜ </ walk>” described in the 42nd line It represents the average time recognized as “walking”. "<Run> ~ </ run>" described in the 43rd line represents the average time recognized as "running", and "<right> ~ </ right>" described in the 44th line is Represents the average time recognized as “right turn”. “<Left> ˜ </ left>” described in the 45th line represents an average time recognized as “left turn”.

  FIG. 17 is a diagram showing a description of a time-series portion of recognized actions.

  In FIG. 17, “<slot>” on the third line to “</ slot>” on the tenth line is information regarding the first recognized action, and “<slot>” on the eleventh line is the 18th line. Up to “</ slot>” is information on the second recognized action.

  For example, “<start> to </ start>” described in the fifth line of the information related to the action recognized first corresponds to the start time of the time series information shown in FIG. “<Stop> to </ stop>” described in the above corresponds to the end time of the time series information shown in FIG. “<Duration> ˜ </ duration>” described in the seventh line corresponds to the total time of the time series information shown in FIG. “<Activity> turn_left </ activity>” described in the ninth line indicates that the action recognized first is “left turn”.

  For other recognized actions, the same information is described in the recognized order. An action parameter file having such a description is output to the experience expression generating unit 22.

  Next, sentence generation performed by the experience expression generation unit 22 will be described.

  FIG. 18 is a block diagram illustrating a detailed configuration example of the experience expression generation unit 22 of FIG.

  As shown in FIG. 18, the experience expression generation unit 22 includes a behavior parameter normalization unit 61, a personal information database 62, an experience parameter generation unit 63, an experience classification unit 64, and a score table storage unit 65.

  The behavior parameter normalization unit 61 normalizes the behavior parameters described in the behavior parameter file supplied from the behavior parameter extraction / behavior recognition unit 21, and uses the behavior parameters obtained by normalization as the experience parameter generation unit. 63 and the experience classification unit 64. For normalization, for example, personal information stored in the personal information database 62 is referred to.

  FIG. 19 is a diagram illustrating an example of personal information stored in the personal information database 62.

  In the personal information database 62, an average of behavior parameters obtained based on behavior parameters described in a behavior parameter file generated in the past by the behavior parameter extraction / behavior recognition unit 21 is stored as personal information. For example, when the mobile phone 1 is used by a plurality of users, the average of the behavior parameters is managed by a different file for each user as shown in FIG.

  The top file among the files shown in FIG. 19 is the personal information of the user identified by the user name “hoge” and the user ID “0001”, and the various files described with reference to FIG. A past average value of information is described. When starting to use the mobile phone 1, the user inputs his / her user name and user ID registered in advance to the mobile phone 1. Note that not only the average value but also the variance value and integrated value of each behavior parameter may be managed as personal information.

  For example, when the behavior parameter file is supplied from the behavior parameter extraction / behavior recognition unit 21, the behavior parameter normalization unit 61 performs a behavior represented by the behavior parameter described in the supplied behavior parameter file. The personal information of the user is read from the personal information database 62, and the behavior parameters described in the behavior parameter file supplied from the behavior parameter extraction / behavior recognition unit 21 are normalized using the read personal information.

  Specifically, when considering whether the number of steps of the user is large or small, since the number of steps per day varies considerably from user to user due to differences in commuting routes, physical strength, etc. It is not possible to make an appropriate determination simply by comparing the number of steps with a threshold value. Therefore, the number of steps normally taken by checking the distribution of the number of steps of the user (assuming a normal distribution) from the personal information and normalizing the number of steps of the user today described in the behavior parameter file by using the distribution. You can see how different it is.

If the normalized number of steps is a normal distribution with an average of 0 and a variance of 1, for example,
If the normalized number of steps is between 0 and 1, "about the same number of steps as usual"
If the normalized number of steps is between 1 and 2, "I walked well"
If the normalized number of steps is between 2 and 3, "I walked a lot"
If the normalized number of steps is 3 or more, “I walked a considerable number of steps as if there were once a year!”
It becomes possible to judge.

  When such normalization is performed to determine the number of steps and a sentence is selected, it is possible to select a sentence such as “I walked a lot today” in a form that absorbed individual differences. Similarly, by combining a plurality of behavior parameters obtained by performing normalization, it is possible to make a more appropriate determination.

  For example, average by day of week, behavior parameter obtained by normalization using variance, average by weather, behavior parameter obtained by normalization by variance, average by user age, obtained by normalization by variance The given behavior parameter can be used for various determinations. In addition, classifying the behavior of the whole day, classifying the behavior parameters obtained by normalization using average and variance for each class, and short-term behavior, and using the average and variance for each class Action parameters obtained by normalization can be used.

  Returning to the description of FIG. 18, the experience parameter generation unit 63 generates an experience parameter based on the behavior parameter normalized by the behavior parameter normalization unit 61, and based on the generated experience parameter and the normalized behavior parameter. The display device 24 displays the screen.

  In addition, the experience parameter generation unit 63 transmits the generated experience parameter and the normalized behavior parameter to the server 2. Not only the text generated based on the user's action, but also information related to the experience parameter and the action parameter is published on the server 2 or displayed on the display device 24. Furthermore, the text, the experience parameters, and the information regarding the behavior parameters generated based on the user's behavior are not only disclosed on the server 2 but also stored on the server 2 and used as an individual behavior profile, for example. Is possible.

  For example, in the experience parameter generation unit 63, a plurality of behavior parameters normalized by the behavior parameter normalization unit 61 are combined, so that “the degree of staggering”, “the degree of dangling”, “the degree of perturbing”, “the degree of leisure” New parameters are generated. By newly generating parameters based on a plurality of behavior parameters and displaying information based on them, the behavior parameters such as the normalized number of steps and the number of left turns are used as they are. As compared with the case where “step count”, “left turn count” or the like is displayed, the entertainment property can be improved.

  As shown in FIG. 20, the “degree of progress” is obtained, for example, by multiplying a value obtained by adding the average walking pitch (Hz) and the average number of turns (times / minute) with the number of steps (steps). The “hanging degree” includes, for example, the number of steps (steps), the average walking pitch (Hz), the average number of turns (times / minute), the rate at which the user's behavior is recognized as “walking”, and the number of times as “walking” ( Times), and adding the time continuously recognized as “walking”.

  In addition, “Hatsuatsu degree” is calculated by adding calories burned, steps taken (steps), and average walking pitch (Hz), and “relaxation degree” is calculated based on the ratio that the user's action is recognized as “still”. It is obtained by subtracting the sum of the number of steps (steps) and the average walking pitch (Hz). The calorie consumption in “Hatsutsutsu” is obtained from, for example, the weight and age input in advance by the user, the total time of “walking” and “running” described in the behavior parameter file, and the like.

  Other parameters may be obtained by the experience parameter generating unit 63 as experience parameters.

  The experience classification unit 64 refers to the experience expression dictionary stored in the experience expression dictionary storage unit 23 and the score table stored in the score table storage unit 65, and the behavior normalized by the behavior parameter normalization unit 61. A sentence according to the experience parameter generated by the parameter and experience classification unit 64 is selected, and the selected sentence is displayed on the display device 24 or the selected sentence is transmitted to the server 2. As described above, sentences and actions are associated with the experience expression dictionary stored in the experience expression dictionary storage unit 23.

  FIG. 21 is a block diagram illustrating a detailed configuration example of the experience classification unit 64.

  As shown in FIG. 21, the experience classification unit 64 includes an experience classification selection unit 71, an experience expression selection unit 72, an update unit 73, and a display data generation unit 74.

  The experience classification selection unit 71 refers to the score table stored in the score table storage unit 65, and thereby the behavior parameters supplied from the behavior parameter normalization unit 61 and the behavior parameters supplied from the experience parameter generation unit 63. The user's action is classified based on the result, and the classification result is output to the experience expression selecting unit 72. The classification result consists of a combination of a large classification and a small classification, and a sentence is selected by the experience expression selection unit 72 according to this combination.

  The process of classifying a user's action by the experience classification selection unit 71 mainly includes three processes: a large classification selection process, a small classification condition check process, and a small classification selection process.

  The major classification selection process is a process of paying attention to the major classifications registered in the score table one by one in the registered order. Each major category has a probability set in advance. For a major category for which a probability of 100% is set, the condition of the minor category set for it is checked with a probability of 100%, 50% For the large classification for which the probability is set, the condition of the small classification set for it is checked with a probability of 50%.

  Each time the minor classification condition is checked according to the set probability, the major classification to be noticed is switched. In addition, when the check of the condition of the minor classification is not performed according to the set probability, the major classification to be noticed is switched to the next major classification.

  The minor classification condition check process is a process of checking the minor classification condition set for one major classification. When a plurality of minor classification conditions are set for one major classification, the conditions are checked one by one in order. Probabilities are also set in advance for each sub-classification condition. For sub-classification conditions for which a probability of 100% is set, the check is performed with a probability of 100% and a probability of 50% is set. The check is performed with a probability of 50% for the condition of the small classification.

  When checking the sub-category conditions, variables in the text selected by the experience expression selection unit 72 are set as appropriate. For example, when checking the number of days since the start of adding a diary (such as a blog) as a check for the condition of the small classification, if there is a variable that fits the number of days in the sentence selected by the experience expression selection unit 72, The same elapsed days as those used for the check are notified from the experience classification selection unit 71 to the experience expression selection unit 72, and the same elapsed days as those used for the check are set in the variable in the sentence.

  The small classification selection process is a process of selecting a small classification determined to match the condition by the small classification condition check process. Probabilities are also set in advance for the sub-classes. For sub-classes for which a probability of 100% is set, if it is determined that the conditions match, the sub-classes are selected with a probability of 100%, and 50% For a small category for which a probability is set, if it is determined that the conditions match, the small category is selected with a probability of 50%.

  The minor classifications that match the conditions but are not selected according to the probability are skipped, and the condition of the next minor classification belonging to the same major classification is checked. Further, when all the small categories belonging to one major category selected by the major category selection process are not selected, it is determined by the major category selection process whether or not the next major category is selected.

  The small classification selected by the small classification selection process is notified to the experience expression selection unit 72 together with the large classification to which the small classification belongs.

  FIG. 22 is a diagram illustrating an example of a score table stored in the score table storage unit 65.

  As shown in FIG. 22, a plurality of small classifications belong to one large classification, and a probability is set for each large classification and each small classification. The small category selection probability shown at the right end of FIG. 22 is a probability serving as a reference for determining whether or not to select a small category. A small category selection condition is also set for each small category.

  The maximum minor classification probability set for each minor classification is the maximum probability of the minor classification selection probability, and is updated by the updating unit 73 according to the subtraction and increment probabilities shown next to it.

  In the example of FIG. 22, “anniversary news item”, “day of the week”, “behavior parameter”, and “experience parameter” are shown as major classifications, and the respective probabilities are 100%, 100%, 50%, 50 %.

  The major classification of “anniversary news item” and “day of the week” for which the probability of 100% is set is selected with a probability of 100% in the major classification selection process, and the process of checking the conditions of the minor classification belonging to it is performed. In addition, the major categories of “behavior parameter” and “experience parameter” for which a probability of 50% is set are selected with a probability of 50% in the major category selection process, and only when selected according to the probability, Processing for checking the classification condition is performed.

  In the example of FIG. 22, the small classification “diary start date” and “every 100 days” belong to the large classification “anniversary news item”, and a small classification selection probability of 100% is set for each. In other words, when the conditions match, “diary start date” and “every 100 days” are sub-categories selected with a probability of 100%. The selection condition for “diary start date” is “diary start date”, and the selection condition for “every 100 days” is “every 100 days have elapsed”.

  Therefore, when the behavior parameter is supplied from the behavior parameter normalization unit 61 and the experience parameter is supplied from the experience parameter generation unit 63 is the “diary start date”, the “diary” belonging to the major classification “anniversary news item” The sub-category “start date” is determined to satisfy the condition, and is selected by the sub-class selection process with a probability of 100%. At this time, the combination of the major classification “anniversary news item” and the minor classification “diary start date” is output to the experience expression selection unit 72.

  Similarly, when the behavior parameter is supplied from the behavior parameter normalization unit 61 and the experience parameter is supplied from the experience parameter generation unit 63 today is “every 100 days” from the start date of the diary, The subcategory “every 100 days” belonging to the “net” is determined to satisfy the condition, and is selected by the subclass selection process with a probability of 100%. At this time, the combination of the major classification “anniversary news item” and the minor classification “every 100 days” is output to the experience expression selection unit 72. In the experience expression selection unit 72, dates are also managed.

  In the example of FIG. 22, the major category “day of the week” belongs to the minor categories “weekday” and “holiday”, and the minor category selection probability of 50% is set for each. The selection condition for “weekdays” is “weekdays”, and the selection condition for “holidays” is “holiday”.

  Accordingly, when the behavior parameter is supplied from the behavior parameter normalization unit 61 and the experience parameter is supplied from the experience parameter generation unit 63 today is “weekday”, the “weekdays” belonging to the major classification “anniversary news item” The minor classification is determined to have the same condition, and is selected by the minor classification selection process with a probability of 50%. When the subcategory “weekday” is selected in consideration of the probability, the combination of the major category “anniversary news item” and the subcategory “weekday” is output to the experience expression selection unit 72.

  Similarly, when the behavior parameter is supplied from the behavior parameter normalization unit 61 and the experience parameter is supplied from the experience parameter generation unit 63 today is “holiday”, “holiday” belonging to the major class “anniversary news item” It is determined that the conditions match, and are selected by the small classification selection process with a probability of 50%. When the minor category “holiday” is selected in consideration of the probability, the combination of the major category “anniversary news item” and the minor category “holiday” is output to the experience expression selection unit 72.

  Further, in the example of FIG. 22, a small classification of “steps” and “exercise calories” belongs to the large classification “behavior parameter”, and a small classification selection probability of 35% is set to the small classification of “steps”. The subcategory selection probability of 50% is set in the subcategory of “exercise calories”. The selection condition for “number of steps” is “when the number of steps exceeds a threshold”, and the selection condition for “exercise calories” is “when exercise calories exceed a threshold”.

  Therefore, when the large classification of “behavior parameter” is selected in consideration of the probability of 50%, a process for checking the condition of the small classification belonging to it is performed, and the behavior parameter supplied from the behavior parameter normalization unit 61 is performed. When the “step count” represented by the above is over the threshold, it is determined that the conditions of the small number of “steps” belonging to the large classification “behavior parameter” match, and the probability of 35% is determined by the small classification selection process. Selected. When the small category “step count” is selected in consideration of the probability, the combination of the large category “action parameter” and the small category “step count” is output to the experience expression selection unit 72.

  Similarly, when the large classification of “behavior parameter” is selected in consideration of the probability of 50%, a process for checking the condition of the small classification belonging to it is performed, and the action supplied from the action parameter normalization unit 61 When the “exercise calorie” represented by the parameter exceeds the threshold, it is determined that the conditions of the “exercise calories” belonging to the large classification “behavioral parameters” are the same, and the subcategory has a probability of 50%. Selected by the selection process. When the small category of “exercise calorie” is selected in consideration of the probability, the combination of the large category “behavior parameter” and the small category “exercise calorie” is output to the experience expression selection unit 72.

  In the example of FIG. 22, the large classification “experience parameter” belongs to the small classifications “Studying degree 1”, “Studying degree 2”, and “Blurring degree 1”. A small classification selection probability of 35% is set for the small classification of “Studying degree 2”, and a small classification selection probability of 50% is set for the small classification of “Standing degree 1”. The selection condition of “degree of promotion 1” is “when the degree of advancement exceeds the threshold value *”, and the selection condition of “degree of advancement 2” is “when the degree of advancement is greater than or equal to * and within *”. The selection condition of “hanging degree 1” is “when the hanging degree exceeds a threshold value *”.

  Therefore, when the large classification of “experience parameter” is selected in consideration of the probability of 50%, a process for checking the condition of the small classification belonging to the large classification is performed, and the experience parameter supplied from the experience parameter generation unit 63 is used. If the expressed “degree of support” exceeds the threshold value, it is determined that the condition of the subclass of “degree of support 1” belonging to the large classification “behavior parameter” matches the condition, and the small classification is selected with a probability of 35%. Selected by processing. When the subclass “Studder degree 1” is selected in consideration of the probability, the combination of the major class “Behavior parameter” and the subclass “Studder degree 1” is output to the experience expression selection unit 72.

  In addition, when a large classification of “experience parameter” is selected in consideration of the probability of 50%, a process for checking a condition of a small classification belonging to the large classification is performed, and the experience parameter supplied from the experience parameter generation unit 63 is used. If the “degree of support” is between * and *, it is judged that the conditions of the sub-class of “degree of support 2” belonging to the large classification “behavior parameters” match, and the sub-class is selected with a probability of 35%. Selected by processing. When the subclass “Studder degree 2” is selected in consideration of the probability, the combination of the major class “Behavior parameter” and the subclass “Studder degree 2” is output to the experience expression selection unit 72.

  Similarly, when the large classification of “experience parameter” is selected in consideration of the probability of 50%, a process for checking the condition of the small classification belonging to the large classification is performed, and the experiential parameter supplied from the experiential parameter generation unit 63 is performed. When the “hanging degree” expressed by the above expression exceeds the threshold, it is determined that the conditions of the small classification of “hanging degree 1” belonging to the large classification “behavior parameter” match, and the small classification selection is performed with a probability of 50%. Selected by processing. When a small classification of “hanging degree 1” is selected in consideration of the probability, a combination of the large classification “behavior parameter” and the small classification “hanging degree 1” is output to the experience expression selection unit 72.

  As described above, based on the score table stored in the score table storage unit 65, the combination of the large classification and the small classification from which the sentence is selected is selected by the experience classification selection unit 71.

  Returning to the description of FIG. 21, the experience expression selection unit 72 selects a sentence based on the result of classification by the experience classification selection unit 71 by referring to the experience expression dictionary stored in the experience expression dictionary storage unit 23. The selected text is output to the update unit 73 and the display data generation unit 74.

  FIG. 23 is a diagram illustrating an example of the experience expression dictionary stored in the experience expression dictionary storage unit 23.

  As shown in FIG. 23, the experience display dictionary is a table that associates experience classifications with sentences (text data). Here, the description is made on the assumption that the experience classification and the text are associated with each other. However, the present invention is not limited to this, and image information (image data) such as icons and animations and audio information (audio data) can also be associated.

  In the example of FIG. 23, the major classification “time story” and the minor classification “diary start date” are associated with the sentence “I will add a diary from today” and the sentence “I will write a diary”. Yes. “Random” is set as the selection policy for the sentences associated with the major classification “time news” and the minor classification “diary start date”.

  That is, when the major classification is selected as “time news” and the minor classification is selected as “diary start date” by the experience classification selection unit 71, a sentence “I will add a diary from today” and “write a diary” Any sentence among the sentences “Kazeze.” Is selected at random.

  In the example of FIG. 23, “1” is set as the number of selected times in the sentence “I will add a diary from today.”, And the sentence “I will add a diary from today.” It is assumed that it has been selected only once. In selecting a sentence, such a selected number of times is also considered as appropriate, and a sentence that has not been selected in the past is preferentially selected. Many sentences are associated with one combination of the large classification and the small classification, and further, the number of selected times is taken into consideration, thereby preventing the selected sentences from being similar. .

  In the example of FIG. 23, the major classification “time story” and the minor classification “every 100 days” include the sentence “It has started ** days since we started adding a diary today” and “It has already passed ** days.” Is associated with the sentence. Also, “random” is set as the selection policy for the sentences associated with the major classification “time news” and minor classification “every 100 days”.

  When the major category is selected as “time news” and the minor category is selected as “every 100 days” by the experience category selection unit 71, the text “Beginning diary today and ** Nikkei has passed.” Any sentence from "** It's been a day" is supposed to be selected randomly. In the sentence “It's already ** days,” “1” is set as the selected number of times.

  Similarly, in the example of FIG. 23, the major classification “experience parameter” and the minor classification “Studying degree 1” include the text “Did you work today?” And “Did you come to the usual place today? "And the sentence" I was busy today? " In addition, “random” is set as the selection policy for the sentences associated with the large classification “experience parameter” and the small classification “degree of staring 1”.

  When the major category is selected as the “experience parameter” by the experience category selection unit 71 and the minor category is selected as “Studying 1”, the text “Did you work today?” Any sentence of the sentence “Was you come to?” Or “Are you busy today?” Is selected at random. In the sentence “Did you work today?”, “1” is set as the selected number of times.

  In the experience expression dictionary, a plurality of sentences are associated with each combination of the major classification and the minor classification in this way, and sentences according to the classification result by the experience classification selecting section 71 are the experience expression selecting section 72. Is selected.

  The experience expression selection unit 72 outputs the sentence selected as described above to the experience classification selection unit 71, the update unit 73, and the display data generation unit 74.

  The update unit 73 updates the score table stored in the score table storage unit 65 and the experience expression dictionary stored in the experience expression dictionary storage unit 23 in accordance with the sentence selected by the experience expression selection unit 72.

  For example, the update unit 73 subtracts the small classification selection probability that is a criterion for selecting the small classification as described with reference to FIG. 22 within a range not exceeding the maximum small classification probability set in the small classification. , Update according to the probability set as increment. In addition, the updating unit 73 updates the selected number of times as described with reference to FIG. 23 according to the text selected by the experience expression selecting unit 72 so that the same text is not repeatedly selected.

  The display data generation unit 74 generates display data for displaying the text selected by the experience expression selection unit 72, and transmits the generated display data to the server 2 or outputs it to the display device 24. When the text is transmitted to the server 2, identification information of the user of the mobile phone 1 is also transmitted along with the text, and is registered in Blog or SNS established by the user of the mobile phone 1. The transmission of the text and identification information by the display data generation unit 74 and the transmission of the experience parameter by the experience parameter generation unit 63 described above are performed simultaneously, for example, and the experience parameter is also registered in Blog or SNS in accordance with the text.

  Similarly, when the display data is output from the display data generation unit 74 to the display device 24, the output is performed simultaneously with the output of the experience parameters by the experience parameter generation unit 63. In the display device 24, screen display is performed based on the text supplied from the display data generation unit 74 and the experience parameters supplied from the experience parameter generation unit 63.

  FIG. 24 is a diagram illustrating an example of a screen displayed on the display device 24.

  In the example of FIG. 24, “hoge diary November 12, 2005 (Sat)” is displayed at the top of the screen, and the screen shown in FIG. 24 is displayed by the user “hoge” “November 12, 2005”. The screen is displayed based on the “day” action.

  The pose of the character displayed as the image P under the date or the like is, for example, a pose selected by combining the experience parameters generated by the experience parameter generation unit 63 and represents the state of the user “hoge”.

  The number of stars of “Hatsutsu degree”, “Leased degree”, “Studying degree”, and “Hanging degree” shown on the right side of the image P is “Hatsu degree”, “Leased degree” generated by the experience parameter generation unit 63. , “Degree of stagnation” and “Degree of dangling” are selected and displayed based on the respective experience parameter values.

  “Steps: 2110 steps”, “Calories consumed: 35.0 calories”, and “Walking pitch: 2.05 steps / second” displayed under “Stillness” are supplied from the experience parameter generation unit 63 together with the experience parameters. Displayed based on the normalized behavior parameters.

  The text “Today's shopping or cleaning? I did not walk too much ...” displayed under “Walking Pitch” etc. is a list of sentences selected by the experience expression selection unit 72 It is.

  In this manner, the user of the mobile phone 1 can record a sentence related to his / her action such as “You didn't walk too much today” without inputting a sentence representing the action of the day. Further, since the displayed text is a text that is more entertaining than when a list such as the number of steps and the place where the user walks is simply displayed, a more interesting text can be recorded.

  Note that the same screen as that shown in FIG. 24 is also managed in the server 2 as a Blog or SNS of the user of the mobile phone 1. The user of the personal computer 3 can browse the daily action of the user of the mobile phone 1 from the screen shown in FIG. 24 by accessing the server 2.

  Here, a series of processes of the mobile phone 1 having the above configuration will be described with reference to a flowchart.

  First, with reference to the flowchart of FIG. 25, the process of the mobile telephone 1 which produces | generates an action parameter is demonstrated.

  In step S1, the behavior parameter extraction / recognition unit 21 of the mobile phone 1 obtains, for example, the walking pitch, the walking power, and the rotation angle around the gravity axis based on the sensor data supplied from the acceleration sensor 11 and the gyro sensor 12. From the obtained information, the number of steps, the average walking pitch, the average number of turns, and the average number of walks / stops are extracted as action parameters. The process of extracting the behavior parameter is started when, for example, a user name or a user ID is input and the user instructs the mobile phone 1 to start behavior recognition.

  In addition, the behavior parameter extraction / recognition unit 21 performs “stationary”, “walking”, “running”, “right” based on the walking pitch, walking power, and rotation angle around the gravity axis obtained based on the sensor data. Either “turn” or “left turn” is recognized as a user's action by using an HMM or the like, and information such as the time, ratio, number of times, and duration of each action is also extracted as an action parameter.

  The process of extracting the behavior parameter is repeatedly performed until, for example, the user finishes using the mobile phone 1 on a day or until a predetermined time (for example, an average time to go home or bedtime).

  In step S <b> 2, the behavior parameter extraction / recognition unit 21 generates a behavior parameter file in, for example, XML format in which the extracted behavior parameters are described, and outputs the generated behavior parameter file to the experience expression generation unit 22. The behavior parameter file generated by the behavior parameter extraction / behavior recognition unit 21 represents the user's daily behavior.

  Next, with reference to the flowchart of FIG. 26, the process of the mobile phone 1 for generating a sentence as a user's experience expression based on the behavior parameters described in the behavior parameter file will be described.

  This process is performed, for example, when the process of FIG. 25 is performed and the behavior parameter file is supplied from the behavior parameter extraction / behavior recognition unit 21 to the experience expression generation unit 22.

  In step S <b> 11, the experience expression generation unit 22 reads the behavior parameter file supplied from the behavior parameter extraction / behavior recognition unit 21. When the behavior parameter file is read, the behavior parameter normalization unit 61 of the experience expression generation unit 22 normalizes various behavior parameters described in the behavior parameter file, and the behavior parameters obtained by normalization are obtained. Is output to the experience parameter generation unit 63 and the experience classification unit 64.

  In step S <b> 12, the experience parameter generation unit 63 of the experience expression generation unit 22 generates experience parameters by combining some of the behavior parameters supplied from the behavior parameter normalization unit 61, and the generated experience parameters are classified into experience classifications. To the unit 64. In addition, the experience parameter generation unit 63 outputs the generated experience parameters to the display device 24 together with the behavior parameters supplied from the behavior parameter normalization unit 61.

  In step S13, the experience classification unit 64 of the experience expression generation unit 22 generates experience classification / experience expression based on the behavior parameters supplied from the behavior parameter normalization unit 61, the experience parameters supplied from the experience parameter generation unit 63, and the like. Process. Here, as described with reference to FIGS. 21 to 23, a combination of the major classification and the minor classification is selected, and a sentence is selected according to the selected combination. Details of the experience classification / experience expression generation processing will be described later with reference to the flowchart of FIG.

  In step S14, for example, the display device 24 uses the behavior parameters and experience parameters supplied from the experience parameter generation unit 63 of the experience expression generation unit 22 and the sentence selected by the experience classification unit 64 as user experience expressions. The screen as shown in FIG. 24 is displayed, and the process is terminated.

  Next, the experience classification / experience expression generation process performed in step S13 of FIG. 26 will be described with reference to the flowchart of FIG.

  In step S21, the experience category selection unit 71 selects one major category registered in the score table shown in FIG. 22, for example, in the registered order, and checks the probability. For the major classification for which the probability of 100% is set, the condition of the minor classification set for it is checked with the probability of 100%, and the process proceeds to step S22. For the large classification for which the probability of 50% is set, the condition of the small classification set for it is checked with a probability of 50%. That is, in step S21, if not selected by probability, the next major classification is selected.

  In step S <b> 22, the experience classification selection unit 71 sequentially checks the minor classification conditions set for the major classification selected in step S <b> 21. If the condition does not match as a result of the check, the condition of the next minor classification is checked. If there is no minor category that matches the condition, the process returns to step S21, and the next major category is selected.

  In step S22, when there is a small classification that matches the condition, the process proceeds to step S23, and the experience classification selection unit 71 determines whether or not to select the small classification determined to match the condition with the current selection probability. . If the conditions match but are not selected with the current selection probability, the process returns to step S22 to check the next sub-classification condition. If no minor category is selected, the process returns to step S21, and the next major category is selected.

  In step S23, when it is determined that the experience classification selection unit 71 matches the conditions and is selected with the current selection probability, the combination of the large classification and the small classification is output to the experience expression selection unit 72, and the process proceeds to step S24.

  In step S24, the experience expression selection unit 72 refers to the experience expression dictionary stored in the experience expression dictionary storage unit 23, so that the major classification and the minor classification notified from the experience classification selection unit 71 in the process of step S23. The experience expression (text data) is selected randomly or sequentially based on the combination of. If the experience expression is not selected, the process returns to step S22, and the condition for the next subclass is checked.

  After the experience expression is selected in the process of step S24, in step S25, the update unit 73 determines the score table and the experience stored in the score table storage unit 65 according to the sentence selected by the experience expression selection unit 72. The experience expression dictionary stored in the expression dictionary storage unit 23 is updated.

  In step S <b> 25, the display data generation unit 74 generates display data for displaying the text selected by the experience expression selection unit 72. Thereafter, the process returns to step S14 in FIG. Accordingly, a screen as shown in FIG. 24 as a user's experience expression is displayed based on the action parameter and experience parameter generated by the experience parameter generation unit 63 and the display data generated by the display data generation unit 74. The

  In the above, everything from recognizing the user's behavior based on the output of the sensor and generating a behavior parameter file to generating display data for selecting a sentence representing the user's experience and performing screen display However, a part of the processing is performed by the mobile phone 1, and the processing subsequent to the processing performed by the mobile phone 1 is performed by another device such as the server 2. It is also possible to do so.

  FIG. 28 is a block diagram illustrating another configuration example of the mobile phone 1.

  In the mobile phone 1 shown in FIG. 28, for example, a process up to generation of a behavior parameter file is performed, a user experience is classified based on the behavior parameter file generated by the mobile phone 1, and a sentence is selected. Is performed by the server 2. That is, in the mobile phone 1, the behavior parameter extraction / behavior recognition unit 21 of FIG. 5 is realized, and the experience expression generation unit 22 is realized by the server 2.

  The processor 13 in FIG. 28 generates a behavior parameter file on the basis of the outputs from the acceleration sensor 11 and the gyro sensor 12, similarly to the mobile phone 1 having the configuration in FIG. 4 described above, and incorporates the generated behavior parameter file. Store in the storage 91. The behavior parameter file stored in the storage 91 is read by the server 2 at a predetermined timing, and the server 2 performs classification of user experiences, selection of sentences, and the like. A screen as shown in FIG. 24 is managed by the server 2 and is disclosed to other users.

  FIG. 29 is a block diagram showing still another configuration example of the mobile phone 1.

  In the mobile phone 1 shown in FIG. 28, for example, processing up to acquiring the outputs of the acceleration sensor 11 and the gyro sensor 12 is performed, and processing for generating a behavior parameter file based on the sensor data acquired by the mobile phone 1 is performed. The server 2 classifies the user's experience and selects a sentence. That is, the behavior parameter extraction / behavior recognition unit 21 and the experience expression generation unit 22 of FIG. 5 are realized by the server 2.

  The storage 91 in FIG. 29 stores sensor data that is output from the acceleration sensor 11 and the gyro sensor 12, for example, in chronological order. The sensor data stored in the storage 91 is read by the server 2 at a predetermined timing, and the server 2 generates behavior parameter files, classifies user experiences, selects sentences, and the like. A screen as shown in FIG. 24 is managed by the server 2 and is disclosed to other users.

  In this way, only a part of the functions may be mounted on the device carried by the user, and the behavior parameter file may be generated based on the information collected by the device carried by the user in the other device. .

  In the above, user behavior parameters recognized based on sensor data and information on dates (anniversaries, days of the week, etc.) are stored in association with text information. However, temperature, humidity, or weather information is also stored. May be stored in association with the text information.

  In addition, although the HMM is used for the action recognition, the determination may be performed using another algorithm for pattern recognition for time series data, for example. Examples of such an algorithm include a Viterbi algorithm and a neural network.

  Further, although the acceleration sensor 11 and the gyro sensor 12 are used, both are not necessarily required, and only one of them may be used. When only the acceleration sensor 11 is used, the walking pitch and the number of steps are detected, and “walking”, “running”, and “stillness” are recognized based on the detected sensor data. When only the gyro sensor 12 is used, the walking pitch and the number of steps are detected, and “walking”, “running”, and “still” are recognized based on the detected sensor data.

  The series of processes described above can be executed by hardware or can be executed by software. When a series of processing is executed by software, a program constituting the software executes various functions by installing a computer incorporated in dedicated hardware or various programs. For example, it is installed from a program recording medium in a general-purpose personal computer or the like.

  FIG. 30 is a block diagram showing an example of the configuration of a personal computer that executes the above-described series of processing by a program.

  A CPU (Central Processing Unit) 201 executes various processes according to a program stored in a ROM (Read Only Memory) 202 or a storage unit 208. A RAM (Random Access Memory) 203 appropriately stores programs executed by the CPU 201 and data. The CPU 201, ROM 202, and RAM 203 are connected to each other via a bus 204.

  An input / output interface 205 is also connected to the CPU 201 via the bus 204. Connected to the input / output interface 205 are an input unit 206 composed of a keyboard, mouse, microphone, and the like, and an output unit 207 composed of a display, a speaker, and the like. The CPU 201 executes various processes in response to commands input from the input unit 206. Then, the CPU 201 outputs the processing result to the output unit 207.

  The storage unit 208 connected to the input / output interface 205 includes, for example, a hard disk, and stores programs executed by the CPU 201 and various data. The communication unit 209 communicates with an external device via a network such as the Internet or a local area network.

  The drive 210 connected to the input / output interface 205 drives a removable medium 211 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, and drives programs and data recorded there. Get etc. The acquired program and data are transferred to and stored in the storage unit 208 as necessary.

  As shown in FIG. 30, a program recording medium for storing a program that is installed in a computer and is ready to be executed by the computer includes a magnetic disk (including a flexible disk), an optical disk (CD-ROM (Compact Disc-Read Only). Memory, DVD (Digital Versatile Disc), a magneto-optical disk, a removable medium 211 that is a package medium made of a semiconductor memory, a ROM 202 in which a program is temporarily or permanently stored, or a storage unit 208 It is comprised by the hard disk etc. which comprise. The program is stored in the program recording medium using a wired or wireless communication medium such as a local area network, the Internet, or digital satellite broadcasting via a communication unit 209 that is an interface such as a router or a modem as necessary. Done.

  In the present specification, the steps for describing a program are not only processes performed in time series in the order described, but also processes that are executed in parallel or individually even if they are not necessarily processed in time series. Is also included.

  Further, in this specification, the system represents the entire apparatus constituted by a plurality of apparatuses.

It is a figure which shows the example of Blog. It is a figure which shows the example of expression of action history. It is a figure which shows the structural example of the information processing system to which this invention is applied. It is a block diagram which shows the hardware structural example of the mobile telephone of FIG. It is a block diagram which shows the function structural example of the mobile telephone of FIG. It is a figure which shows the example of the input with respect to an action parameter extraction and action recognition part, and the output from an action parameter extraction and action recognition part. It is a figure which shows the example of the information described in a behavior parameter file. It is a block diagram which shows the detailed structural example of an action parameter extraction and action recognition part. It is a figure which shows the example of a walking pitch. It is a figure which shows the example of walking power. It is a figure which shows the example of the change of the acceleration of a gravity axis direction. It is a figure which shows the example of the rotation angle around a gravity axis. It is a figure which shows the example of the one part function part provided in the action recognition part of FIG. It is a figure which shows typically the action recognition by the recognition part of FIG. It is a figure which shows the example of sensor data, a feature-value, and a recognition result in time series. It is a figure which shows the example of the specific description of an action parameter file. It is a figure following FIG. 16 which shows the example of the specific description of a behavior parameter file. It is a block diagram which shows the detailed structural example of the experience expression production | generation part of FIG. It is a figure which shows the example of personal information. It is a figure which shows the example of how to obtain | require an experience parameter. It is a block diagram which shows the detailed structural example of the experience classification | category part of FIG. It is a figure which shows the example of a score table. It is a figure which shows the example of an experience expression dictionary. It is a figure which shows the example of a display of a screen. It is a flowchart explaining the action parameter generation process of a mobile telephone. It is a flowchart explaining the experience expression production | generation process of a mobile telephone. It is a flowchart explaining the experience classification / experience display process performed in step S13 of FIG. FIG. 25 is a block diagram illustrating another hardware configuration example of the mobile phone. FIG. 25 is a block diagram illustrating still another hardware configuration example of the mobile phone. And FIG. 16 is a block diagram illustrating a configuration example of a personal computer.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 Mobile phone, 2 Server, 21 Action parameter extraction and action recognition part, 22 Experience expression production | generation part, 23 Experience expression dictionary memory | storage part, 24 Display apparatus

Claims (8)

Storage means for recognizing an action parameter, which is information related to a user's action, recognized based on the output of the sensor, and text information;
When the behavior parameter extracted by the extraction unit matches a predetermined condition, the text information corresponding to the behavior parameter extracted by the extraction unit is selected from the text information stored by the storage unit And an information processing apparatus. The information processing apparatus according to claim 1, further comprising a generation unit configured to generate the behavior parameter by combining a plurality of parameters of the sensor output. The information processing apparatus according to claim 1, further comprising a normalizing unit that normalizes the behavior parameter. The information processing apparatus according to claim 1, further comprising a generation unit configured to generate an experience parameter that represents a degree of user behavior by combining a plurality of the behavior parameters. The information processing apparatus according to claim 1, wherein the selection unit preferentially selects unselected text information from text information. The information processing apparatus according to claim 1, wherein the storage unit further stores information related to a date and the text data in association with each other. A behavior parameter, which is information related to the user's behavior, recognized based on the output of the sensor, and text information are stored in association with each other,
An information processing method including a step of selecting text information corresponding to the extracted behavior parameter from the stored text information when the extracted behavior parameter meets a predetermined condition. A behavior parameter, which is information related to the user's behavior, recognized based on the output of the sensor, and text information are stored in association with each other,
A program for causing a computer to execute a process including a step of selecting text information corresponding to an extracted behavior parameter from stored text information when the extracted behavior parameter matches a predetermined condition.

Images