JP2016149136A - Information processing system, information processing method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To combine a result of recognizing a behavior pattern on the basis of information acquired from a position sensor or a motion sensor, with information other than the information acquired from the position sensor or the motion sensor, to provide more sophisticated information.SOLUTION: An information processing system includes: an experience extraction section which extracts experience information indicating an experience of a user, from text information acquired from a social networking service or an application; a behavior extraction section which extracts a behavior pattern from position sensor information and time information; an experience collation section which extracts experience information from the position sensor information, in reference to information indicating a correspondence between the behavior pattern and the experience information; and a display control section which displays information on the experience information extracted from the text information and information on the experience information acquired from the position sensor information, together.SELECTED DRAWING: Figure 24

Description

  The present technology relates to an information processing system, an information processing method, and a program.

  Attention has been focused on a technique for mounting a motion sensor on a mobile terminal such as a mobile phone so that a user's behavior history can be automatically detected and recorded. For example, Patent Document 1 below discloses a technique for detecting a walking motion, a running motion, a left-right motion, and a stationary state using a motion sensor such as an acceleration sensor or a gyro sensor. This document calculates the walking pitch, walking power, rotation angle around the gravity axis from the output data of the motion sensor, and uses the calculated results to walk, run, turn left and right, and stand still. The detection method is described.

  Furthermore, the same document detects the user's behavior pattern by statistical processing that inputs the pattern of the operation and the state, such as the type of the operation and the state, the time during which the operation and the state have continued, and the number of operations. A method is described. By using the above method, it is possible to obtain behavioral patterns such as “slow walking” and “slow motion” as time series data. However, the behavior pattern obtained by this method mainly represents the user's actions and states performed in a relatively short time. For this reason, it is difficult to infer specific action contents from the history of action patterns, such as “shopping at a department store today” or “dining at a hotel restaurant yesterday”.

  The behavior pattern obtained by using the method described in Patent Document 1 below is a stack of behaviors performed in a relatively short time. The individual behaviors that form the behavior patterns are not intended to be performed by the user. On the other hand, the specific action content is often performed by the user with a purpose, and is highly entertaining for a relatively long time. Therefore, it is difficult to know the specific action contents as described above from the accumulation of actions performed in a short time. However, recently, a technique has been developed for detecting a behavior pattern with high entertainment performance that is performed for a relatively long time from a behavior pattern of a relatively short time obtained by using a motion sensor (see Patent Document 2 below). .

JP 2008-003655 A JP 2011-081431 A

  On the other hand, in recent years, the network environment surrounding users has been advanced and diversified, and social network services for uploading comments input by users themselves to servers on the network have become widespread. Such comments may include information on user behavior and intentions.

  This technology has been devised in view of such circumstances, and the action pattern recognition result based on the information acquired from the position sensor or motion sensor and the information other than the information acquired from the position sensor or motion sensor. It is intended to provide a new and improved information processing system, information processing method, and program capable of providing a combination and more advanced information.

  According to an aspect of the present technology, an experience extraction unit that extracts experience information indicating a user's experience from text information acquired from a social networking service or application, and an action that extracts an action pattern from position sensor information and time information An extraction unit, an experience collation unit that extracts experience information from the position sensor information with reference to information indicating a correspondence relationship between the behavior pattern and the experience information, information about the experience information extracted from the text information, and There is provided an information processing system including a display control unit that displays information related to the experience information obtained from the position sensor information.

  According to another aspect of the present technology, a step of extracting experience information indicating a user's experience from text information acquired from a social networking service or application, and a behavior pattern from position sensor information and time information are extracted. A step of extracting experience information from the position sensor information with reference to information indicating a correspondence relationship between the step, the behavior pattern, and the experience information, and the experience information extracted from the text information by the arithmetic processing device An information processing method is provided that includes displaying information and information related to experience information obtained from the position sensor information together.

  Further, according to another aspect of the present technology, an experience extraction unit that extracts experience information indicating a user's experience from text information acquired from a social networking service or application, an action pattern from position sensor information and time information An action extracting unit to extract, an experience matching unit for extracting experience information from the position sensor information with reference to information indicating a correspondence relationship between the behavior pattern and the experience information, and experience information extracted from the text information And a display control unit for displaying information related to the experience information obtained from the position sensor information together with the display control unit.

  According to another aspect of the present technology, a computer-readable recording medium on which the program is recorded is provided.

  As described above, according to the present technology, the action pattern recognition result based on the information acquired from the position sensor or the motion sensor and the information other than the information acquired from the position sensor or the motion sensor are combined to provide more advanced information. It becomes possible to provide.

It is explanatory drawing for demonstrating the structural example of a behaviour / situation analysis system. It is explanatory drawing for demonstrating the function of a movement and state recognition part. It is explanatory drawing for demonstrating the function of a movement and state recognition part. It is explanatory drawing for demonstrating the function of a GIS information acquisition part. It is explanatory drawing for demonstrating the function of a GIS information acquisition part. It is explanatory drawing for demonstrating the function of a GIS information acquisition part. It is explanatory drawing for demonstrating the function of a GIS information acquisition part. It is explanatory drawing for demonstrating the function of a behaviour / situation recognition part. It is explanatory drawing for demonstrating the function of a behaviour / situation recognition part. It is explanatory drawing for demonstrating the determination method of an operation | movement / state pattern. It is explanatory drawing for demonstrating the calculation method of the score distribution using a geohistogram. It is explanatory drawing for demonstrating the calculation method of the score distribution using machine learning. It is explanatory drawing which showed an example of the detected action / situation pattern. It is explanatory drawing for demonstrating an example of the system configuration | structure which concerns on one Embodiment of this invention. It is explanatory drawing for demonstrating the structure of the information provision system which concerns on structural example # 1. It is explanatory drawing for demonstrating in detail the function of the information provision system which concerns on structural example # 1. It is explanatory drawing for demonstrating in detail the function of the information provision system which concerns on structural example # 1. It is explanatory drawing for demonstrating in detail the function of the information provision system which concerns on structural example # 1. It is explanatory drawing for demonstrating in detail the function of the information provision system which concerns on structural example # 1. It is explanatory drawing for demonstrating in detail the function of the information provision system which concerns on structural example # 1. It is explanatory drawing for demonstrating in detail the function of the information provision system which concerns on structural example # 1. It is explanatory drawing for demonstrating in detail the function of the information provision system which concerns on structural example # 1. It is explanatory drawing for demonstrating operation | movement of the information provision system which concerns on structural example # 1. It is explanatory drawing for demonstrating the structure of the information provision system which concerns on structural example # 2. It is explanatory drawing for demonstrating in detail the function of the information provision system which concerns on structural example # 2. It is explanatory drawing for demonstrating in detail the function of the information provision system which concerns on structural example # 2. It is explanatory drawing for demonstrating in detail the function of the information provision system which concerns on structural example # 2. It is explanatory drawing for demonstrating operation | movement of the information provision system which concerns on structural example # 2. It is explanatory drawing for demonstrating the structure of the information provision system which concerns on structural example # 3. It is explanatory drawing for demonstrating in detail the function and discrimination | determination operation | movement of an information provision system which concern on structural example # 3. It is an explanatory view for explaining an example of a hardware configuration capable of realizing the functions of the system and each device according to the embodiment.

  Hereinafter, preferred embodiments according to the present technology will be described in detail with reference to the accompanying drawings. In addition, in this specification and drawing, about the component which has the substantially same function structure, duplication description is abbreviate | omitted by attaching | subjecting the same code | symbol.

[About the flow of explanation]
Here, the flow of explanation described below will be briefly described.

  First, a behavior pattern recognition technique related to the technique of the present embodiment will be described with reference to FIGS. Next, an example of a system configuration according to an embodiment of the present invention will be described with reference to FIG. Next, functions and operations of the information providing system 13 according to the configuration example # 1 will be described with reference to FIGS.

  Next, functions and operations of the information providing system 17 according to the configuration example # 2 will be described with reference to FIGS. Next, functions and operations of the information providing system 19 according to Configuration Example # 3 will be described with reference to FIGS. 29 and 30. Next, a hardware configuration example capable of realizing the functions of the system and each device according to the embodiment will be described with reference to FIG.

  Finally, the technical idea of the embodiment will be summarized and the effects obtained from the technical idea will be briefly described.

(Description item)
1: Introduction 1-1: Action pattern recognition technology 1-2: Outline of embodiment 2: Details of embodiment 2-1: Example of system configuration 2-2: Configuration example # 1 (suggestion of target achievement)
2-2-1: Details of system configuration
2-2-2: Flow of processing
2-2-3: Example of screen display
2-2-4: Modified example (application to animals)
2-3: Configuration example # 2 (display of detailed behavior)
2-3-1: System configuration details
2-3-2: Flow of processing
2-3-3: Example of screen display 2-4: Configuration example # 3 (Determination of daily / unusual behavior)
2-4-1: Details of system configuration
2-4-2: Application examples 2-5: Combinations of configuration examples 3: Hardware configuration examples 4: Summary

<1: Introduction>
First, a behavior pattern recognition technique related to the technique of this embodiment will be described.

[1-1: Behavior pattern recognition technology]
The behavior pattern recognition technology described here uses the information on the user's movement and state detected using a motion sensor and the position information detected using a position sensor to determine the user's behavior and situation. It relates to the technology to detect.

  In addition, as a motion sensor, for example, a 3-axis acceleration sensor (including an acceleration sensor, a gravity detection sensor, a fall detection sensor, etc.), a 3-axis gyro sensor (including an angular velocity sensor, a camera shake correction sensor, a geomagnetic sensor, etc.), etc. Is used. As the position sensor, for example, information such as a GPS (Global Positioning System), an RFID (Radio Frequency Identification), a Wi-Fi access point, and a wireless base station can be used. Using these pieces of information makes it possible to detect the latitude and longitude of the current location, for example.

(System configuration of action / situation analysis system 10)
First, the system configuration of the behaviour / situation analysis system 10 capable of realizing the above-described action pattern recognition technology will be described with reference to FIG. FIG. 1 is an explanatory diagram for explaining the overall system configuration of the behaviour / situation analysis system 10.

  In this paper, the expressions “movement and state” and “behavior and situation” are used in the following ways. “Movement, state” means an action performed by the user in a relatively short time of several seconds to several minutes, and refers to actions such as “walking”, “running”, “jumping”, and “still”. Further, this action is collectively expressed as “movement / state pattern” or “LC (Low-Context) action”. On the other hand, “behavior / situation” is a life activity performed by the user over a longer time than “movement / state”, and refers to actions such as “meal”, “shopping”, and “work”. Further, this action is collectively expressed as “behavior / situation pattern” or “HC (High-Context) action”.

  As shown in FIG. 1, the behaviour / situation analysis system 10 mainly includes a motion sensor 101, a movement / state recognition unit 102, a time information acquisition unit 103, a position sensor 104, a GIS information acquisition unit 105, The action / situation recognition unit 106 is configured.

  The behaviour / situation analysis system 10 may be provided with an application AP or service SV that uses the behaviour / situation pattern detected by the behaviour / situation recognition unit 106. Further, the use result of the action / situation pattern by the application AP and the user profile information may be input to the action / situation recognition unit 106.

  First, when the user acts, the motion sensor 101 detects a change in acceleration, rotation around the gravity axis, and the like (hereinafter referred to as sensor data). The sensor data detected by the motion sensor 101 is input to the movement / state recognition unit 102 as shown in FIG.

  When sensor data is input, the movement / state recognition unit 102 detects a movement / state pattern using the input sensor data, as shown in FIG. As the movement / state pattern detectable by the movement / state recognition unit 102, for example, as shown in FIG. 3, “walking”, “running”, “still”, “jumping”, “train (boarding / non-boarding)”, “elevator ( Boarding / non-boarding / up / down). The movement / state pattern detected by the movement / state recognition unit 102 is input to the behaviour / situation recognition unit 106.

  The position sensor 104 continuously or intermittently acquires position information indicating the user's whereabouts (hereinafter, current position). For example, the position information of the current location is expressed by latitude and longitude. The current position information acquired by the position sensor 104 is input to the GIS information acquisition unit 105.

  When the current location information is input, the GIS information acquisition unit 105 acquires GIS (Geographic Information System) information. And the GIS information acquisition part 105 detects the attribute of the present location using the acquired GIS information, as shown in FIG. GIS information includes, for example, map information, various additional information obtained by artificial satellites and field surveys, etc., and is also used for scientific surveys, management of land, facilities, roads, etc. and city planning. is there. If the GIS information is used, it is possible to determine the attribute of the current location. The GIS information acquisition unit 105 expresses the attribute of the current location using identification information called a geo category code (for example, see FIG. 5).

  As shown in FIG. 5, the geo category code is a classification code for classifying the type of information related to a place. This geo category code is set according to, for example, the type of building, the shape of the terrain, geographical characteristics, regionality, and the like. Therefore, by specifying the geo category code of the current location, it is possible to recognize to some extent the environment where the user is.

  The GIS information acquisition unit 105 refers to the acquired GIS information, specifies the current location and buildings around the current location, and extracts a geo category code corresponding to the building. The geo category code selected by the GIS information acquisition unit 105 is input to the behaviour / situation recognition unit 106. If there are many buildings around the current location, the GIS information acquisition unit 105 extracts the geo category code of each building, etc., and uses the geo category code shown in FIGS. 6 and 7 as information on the extracted geo category code. Information such as a category histogram may be input to the behaviour / situation recognition unit 106.

  As shown in FIG. 8, the behavior / situation recognition unit 106 receives a movement / state pattern from the movement / state recognition unit 102 and a geo category code from the GIS information acquisition unit 105. Further, time information is input from the time information acquisition unit 103 to the behaviour / situation recognition unit 106. This time information includes information indicating the time when the sensor data is acquired by the motion sensor 101. Further, the time information may include information indicating the time when the position information is acquired by the position sensor 104. Further, as the time information, for example, in addition to the information indicating the time, information on the day of the week, holiday information, date information, and the like may be included.

  When the above information is input, the behaviour / situation recognition unit 106 detects the behaviour / situation pattern based on the input movement / state pattern, geo category code (or geo category histogram, etc.), and time information. . At this time, the behaviour / situation recognition unit 106 detects a behaviour / situation pattern using a rule-based determination process (hereinafter, rule-based determination) or a learning model-based determination process (hereinafter, learning model determination). Here, the rule base determination and the learning model determination will be briefly described.

(About rule-based judgment)
First, rule base determination will be described. Rule-based determination is a method in which a score is assigned to each combination of a geo category code and an action / situation pattern, and an appropriate action / situation pattern corresponding to input data is determined based on the score.

  The score assignment rule is expressed by a score map SM as shown in FIG. The score map SM is prepared for each time information such as date, time, day of the week, etc. For example, a score map SM corresponding to Monday of the first week of March is prepared. Furthermore, the score map SM is prepared for each movement / state pattern such as walking, running, and train. For example, a score map SM during walking is prepared. Therefore, the score map SM is prepared for each combination of time information and movement / state pattern.

  As shown in FIG. 10, the behaviour / situation recognition unit 106 selects a score map SM that matches the input time information and movement / state pattern from among a plurality of score maps SM prepared in advance. Further, the behaviour / situation recognition unit 106 extracts a score corresponding to the geo category code from the selected score map SM as shown in FIG. With these processes, the behaviour / situation recognition unit 106 can extract the score of each behaviour / situation pattern existing in the score map SM in consideration of the current position at the time of sensor data acquisition.

  Next, the behaviour / situation recognition unit 106 identifies the maximum score from the extracted scores, and extracts the behaviour / situation pattern corresponding to the maximum score. A method for detecting a behaviour / situation pattern in this way is rule-based determination. The score of the score map SM indicates the probability that the user is assumed to take the behaviour / situation pattern corresponding to the score. That is, the score map SM represents the score distribution of the behaviour / situation pattern that the user is supposed to take in the current situation represented by the geo category code.

  For example, it is estimated that a user at a department store has a high probability of “shopping” around 3 o'clock on Sunday. However, even at the same department store, it is estimated that the user at the department store is likely to be “mealing” at around 19:00. As described above, the score map SM (more precisely, the score map SM group) shows the score distribution of the behaviour / situation pattern performed by the user at a certain place at a certain time.

  For example, the score map SM may be input in advance by the user himself / herself or another person, or may be obtained using machine learning or the like. The score map SM may be optimized based on the personal profile information PR and the action / situation feedback FB obtained from the user (whether the output action / situation pattern is correct or not). As the profile information PR, for example, age, sex, occupation, home information, workplace information, and the like are used. The above is the specific processing content of the rule base determination.

(About learning model judgment)
Next, learning model determination will be described. Learning model determination is a method of generating a determination model for determining an action / situation pattern using a machine learning algorithm and determining an action / situation pattern corresponding to input data using the generated determination model.

  As the machine learning algorithm, for example, k-means method, Nearest Neighbor method, SVM, HMM, Boosting, etc. can be used. However, SVM is an abbreviation for Support vector machine. HMM is an abbreviation for Hidden Markov Model. In addition to these methods, there is also a method of generating a determination model using an algorithm construction method based on a genetic search described in Japanese Patent Application Laid-Open No. 2009-48266.

  As the feature vector input to the machine learning algorithm, for example, as shown in FIG. 12, time information, motion / state pattern, geo category code (or geo category histogram, etc.), sensor data, current location information, etc. Used. However, when an algorithm construction method based on genetic search is used, the genetic search algorithm is used in the feature vector selection stage in the learning process. The behaviour / situation recognition unit 106 first inputs a feature vector in which the correct behaviour / situation pattern is known to the machine learning algorithm as learning data, and determines the accuracy of each behaviour / situation pattern or the optimum behaviour / situation pattern. A determination model for determination is generated.

  Next, the behaviour / situation recognition unit 106 inputs input data to the generated determination model, and determines a behaviour / situation pattern that is presumed to be suitable for the input data. However, when correct / incorrect feedback is obtained for the determination result made using the generated determination model, the determination model is reconstructed using the feedback. In this case, the behaviour / situation recognition unit 106 determines a behaviour / situation pattern presumed to be suitable for the input data using the reconstructed determination model. The above is the specific processing content of the learning model determination.

  By the method described above, the behaviour / situation recognition unit 106 detects a behaviour / situation pattern as shown in FIG. The behaviour / situation pattern detected by the behaviour / situation recognition unit 106 is used to provide a recommended service SV corresponding to the behaviour / situation pattern or used by an application AP that performs processing according to the behaviour / situation pattern. The

  The system configuration of the behaviour / situation analysis system 10 has been described above. A technique according to an embodiment to be described later relates to the function of the behaviour / situation analysis system 10 described above. For detailed functions of the behaviour / situation analysis system 10, for example, the description in Japanese Patent Application Laid-Open No. 2011-081431 is helpful.

[1-2: Outline of Embodiment]
Here, an outline of the present embodiment will be described. The technology according to the present embodiment relates to a mechanism for providing higher value-added information by combining action pattern information obtained by using the above-described action / situation analysis system 10 and input information such as text information. .

  For example, Configuration Example # 1 introduced in the latter part relates to a mechanism for providing “state information” representing a state related to a matter based on one or a plurality of action patterns corresponding to “predetermined matter”. For example, the above-mentioned “predetermined matter” is a user's goal / declaration obtained from input information, and the “state information” is an achievement level for the goal / declaration.

  The “predetermined matter” is not limited to the user's goal / declaration obtained from the input information, and the “state information” is not limited to the achievement level with respect to the goal / declaration. Now, the description will proceed mainly by taking as an example a mechanism for providing the user with the degree of achievement of the user's goal / declaration obtained from the input information.

  The “state information” represents, for example, the current state for a predetermined matter, in addition to the achievement level that is an example of comparison information between the current state for the predetermined matter and the state when the goal / declaration is achieved. Information or comparison information between a current state and a past state with respect to a predetermined matter may be used, and even in this case, the technology according to Configuration Example # 1 described later can be applied.

  In addition, the configuration example # 2 described later adds information related to the user's experience obtained from input information such as text information to the behavior pattern information obtained by using the behavior / situation analysis system 10 described above, and more. The present invention relates to a mechanism for providing detailed information to a user. Furthermore, the configuration example # 3 described later is an extraordinary part of the user's experience obtained from input information such as action pattern information and text information obtained by using the action / situation analysis system 10 described above. The present invention relates to a mechanism for discriminating actions and experiences and providing them to users.

  Note that the techniques according to these configuration examples # 1 to # 3 can be arbitrarily combined. In the following description, text information is mainly assumed as input information used for experience extraction. However, for example, voice information acquired using a microphone or the like can be used. In this case, it is possible to obtain information on the surrounding environment and behavior using the waveform of the voice signal as it is, and it is also possible to obtain text information from the voice signal using voice recognition technology. Since text information is obtained when using the speech recognition technology, the technology according to configuration examples # 1 to # 3 described later can be applied as it is.

<2: Details of the embodiment>
Hereinafter, details of the technology according to the present embodiment will be described.

[2-1: Example of system configuration]
First, an example of a system configuration according to the present embodiment will be introduced with reference to FIG. FIG. 14 is an explanatory diagram for describing an example of a system configuration according to the present embodiment. The system configuration introduced here is merely an example, and the technology according to the present embodiment can be applied to various system configurations that can be used at present and in the future.

  As shown in FIG. 14, the information providing systems 13, 17, and 19 to be described later are mainly configured by a plurality of information terminals CL and a server device SV. The information terminal CL is an example of a device used by a user. For example, as the information terminal CL, a mobile phone, a smartphone, a digital still camera, a digital video camera, a personal computer, a tablet terminal, a car navigation system, a portable game machine, a health device (including a pedometer (registered trademark), etc.), Medical equipment is assumed. On the other hand, as the server device SV, for example, a home server or a cloud computing system is assumed.

  Of course, the system configuration to which the technology according to the present embodiment can be applied is not limited to the example of FIG. 14, but for convenience of explanation, a plurality of information terminals CL and server devices SV connected by a wired and / or wireless network are included. I will proceed with the explanation in mind. Therefore, a configuration is assumed in which information can be exchanged between the information terminal CL and the server device SV. However, of the various functions of the information providing systems 13, 17, and 19, it is free to configure which function is performed by the information terminal CL and which function is performed by the server apparatus SV. For example, it is desirable to design in consideration of the computing capability and communication speed of the information terminal CL.

[2-2: Configuration example # 1 (implication of target achievement)]
First, configuration example # 1 will be described. Configuration example # 1 relates to a mechanism for providing a user with an achievement level for a user's goal / declaration obtained from input information.

(2-2-1: System configuration details)
A system (information providing system 13) according to configuration example # 1 is, for example, as shown in FIG. As shown in FIG. 15, the information providing system 13 includes a text information acquisition unit 131, an experience extraction unit 132, a target / declaration extraction unit 133, a target / declaration collation unit 134, a correspondence relationship storage unit 135, a target / Declaration registration unit 136. Furthermore, the information providing system 13 includes an achievement level storage unit 137, a sensor information acquisition unit 138, an action pattern extraction unit 139, an achievement level update unit 140, and an achievement level display unit 141.

  The functions of the sensor information acquisition unit 138 and the behavior pattern extraction unit 139 can be realized by using the functions of the behavior / situation analysis system 10 described above. In addition, among the above-described components included in the information providing system 13, it is free to design which component element functions as the information terminal CL and which component component functions as the server device SV. For example, it is desirable to design in consideration of the computing capability and communication speed of the information terminal CL.

  The text information acquisition unit 131 acquires text information input by the user. For example, the text information acquisition unit 131 may be an input device for the user to input text, or may be an information collection device that acquires text information from a social network service or application. Here, for convenience of explanation, the description will be made assuming that the text information acquisition unit 131 is an input means such as a software keyboard.

  The text information acquired by the text information acquisition unit 131 is input to the experience extraction unit 132. At this time, time information when the text information is input may be input to the experience extracting unit 132 together with the text information. When text information is input, the experience extraction unit 132 analyzes the input text information and extracts information related to the user's experience from the text information. The information related to the experience is, for example, information including the experienced event (type of experience, etc.), the experienced location, the experienced time, and the like.

  Here, the functional configuration of the experience extraction unit 132 will be described in more detail with reference to FIG. As illustrated in FIG. 16, the experience extraction unit 132 mainly includes a type feature amount extraction unit 151, an experience type determination unit 152, and an experience type model storage unit 153. Furthermore, the experience extraction unit 132 includes a location feature amount extraction unit 154, an experience location extraction unit 155, and an experience location model storage unit 156. The experience extraction unit 132 includes a time feature amount extraction unit 157, an experience time extraction unit 158, and an experience time model storage unit 159.

  When the text information is input to the experience extraction unit 132, the text information is input to the type feature amount extraction unit 151, the location feature amount extraction unit 154, and the time feature amount extraction unit 157.

  The type feature amount extraction unit 151 extracts a feature amount related to the type of experience (hereinafter referred to as type feature amount) from the input text information. The type feature amount extracted by the type feature amount extraction unit 151 is input to the experience type determination unit 152. The experience type determination unit 152 uses the learning model stored in the experience type model storage unit 153 to determine the type of experience from the input type feature amount. Then, the discrimination result by the experience type discrimination unit 152 is input to the goal / declaration extraction unit 133.

  Further, the location feature amount extraction unit 154 extracts a feature amount related to the experienced location (hereinafter referred to as a location feature amount) from the input text information. The location feature amount extracted by the location feature amount extraction unit 154 is input to the experience location determination unit 155. The experience place determination unit 155 uses the learning model stored in the experience place model storage unit 156 to determine the place experienced from the input place feature amount. Then, the determination result by the experience location determination unit 155 is input to the target / declaration extraction unit 133.

  Further, the time feature amount extraction unit 157 extracts a feature amount related to the experienced time (hereinafter, time feature amount) from the input text information. The time feature amount extracted by the time feature amount extraction unit 157 is input to the experience time determination unit 158. The experience time determination unit 158 uses the learning model stored in the experience time model storage unit 159 to determine the experience time from the input time feature amount. Then, the determination result by the experience time determination unit 158 is input to the target / declaration extraction unit 133.

  Here, with reference to FIG. 17, the contents of the processing executed by the experience extraction unit 132 will be supplemented by taking a music experience as an example. FIG. 17 is an explanatory diagram for explaining the contents of specific processing executed by the experience extracting unit 132. Here, for convenience of explanation, the description will be given by taking a music experience as an example, but the technical scope of the present embodiment is not limited to this.

  As shown in FIG. 17, in the case of a music experience, examples of the type of experience include “listening to music”, “watching music video on TV / movie / DVD” (watch), “buying a song / CD (Buy) ”“ participate in live concerts (live) ”,“ sing a song / perform / compose (play) ”, and the like. The experience extraction unit 132 uses the functions of the type feature amount extraction unit 151 and the experience type determination unit 152 to determine these types of experiences.

  For example, when the type of experience “listen” is determined, first, the type feature amount extraction unit 151 extracts the type feature amount related to the type of experience “listen” using a technique such as morpheme, n-gram, and maximum partial character string. . Next, the experience type discriminating unit 152 discriminates whether or not the experience type corresponds to the type of experience “listen” from the type feature amount by a method such as SVM or Logistic Regression. The discrimination result by the experience type discrimination unit 152 is output as information indicating the type of experience. Similarly, a discrimination result is obtained for the types of experience “watch”, “buy”, “live”, “play”, and the like.

  The extraction of the experience place is realized by the functions of the place feature amount extraction unit 154 and the experience place extraction unit 155. First, the place feature amount extraction unit 154 performs morphological analysis on the input text information, and inputs the result to the experience place extraction unit 155. Next, the experience place extraction unit 155 extracts the experience place using a technique such as CRF (Conditional Random Field) based on the result of the morphological analysis. For example, the experience place extraction unit 155 extracts an experience place (extracts “Kyoto Station” in the example of FIG. 19) using a feature template as shown in FIG. 18, as shown in FIG.

  The extraction of the experience time is realized by the functions of the time feature amount extraction unit 157 and the experience time extraction unit 158. The extraction of the experience time is realized by a sequence labeling method using morphological analysis, CRF, and the like, similarly to the extraction of the experience place described above. As the expression of the experience time, various units of expression such as “present”, “past”, “future”, “morning”, “daytime”, “evening”, and “midnight” can be used. The information on the experience place and the experience time obtained in this way is input to the target / declaration extraction unit 133 together with the determination result indicating the type of experience. In some cases, part or all of the experience type, the experience location, and the experience time cannot be obtained.

  Refer to FIG. 15 again. When the information on the type of experience, the experience location, and the experience time is obtained, the goal / declaration extraction unit 133 uses the information on the type of experience and the experience time to determine whether the text information includes information on the goal / declaration. Determine whether. For example, as shown in FIG. 20, when the type of experience is “diet” and the experience time is “future”, the goal / declaration extraction unit 133 determines that the text information corresponding to the result includes the goal / declaration. To do. On the other hand, even if the type of experience is “diet”, if the experience time is “past”, the goal / declaration extraction unit 133 determines that the text information corresponding to this result does not include the goal / declaration.

  That is, when the experience type corresponds to the goal / declaration and the experience time is in the future, the goal / declaration extraction unit 133 determines that the text information corresponding to the result includes the goal / declaration. Then, the goal / declaration extraction unit 133 extracts the experience type obtained from the text information determined to include the goal / declaration as the goal / declaration. The target / declaration information extracted by the target / declaration extraction unit 133 is input to the target / declaration collation unit 134. When the goal / declaration information is input, the goal / declaration collation unit 134 refers to the correspondence relationship storage unit 135 to identify one or more action patterns related to the input goal / declaration, and Extract behavior patterns.

  Here, in the above description, the goal / declaration collation unit 134 specifies one or more action patterns corresponding to the goal / declaration after the goal / declaration information is input. The application range of the technology according to the embodiment is not limited to this.

  For example, it is conceivable to recognize all action patterns that can be acquired in advance and store the recognition results in a database regardless of whether they are related to the goal / declaration. In this case, when a goal / declaration is input, data on the behavior pattern associated with the input goal / declaration may be referred to from the database storing the recognition results of all the behavior patterns.

  For example, as shown in FIG. 21, the correspondence relationship storage unit 135 stores a database indicating the correspondence relationship between the target / declaration and the action pattern. Further, in the example of FIG. 21, contributions are associated with each combination of a target / declaration and an action pattern. For example, when a diet is a target / declaration, actions such as “walking” and “running” have an effect on the diet, but actions such as “riding a car” and “getting on a train” have no effect on the diet. From such a viewpoint, in the example of FIG. 21, contributions are associated with each combination of a target / declaration and an action pattern.

  The goal / declaration collation unit 134 inputs the goal / declaration information and the behavior pattern information corresponding to the goal / declaration to the goal / declaration registration unit 136. When the goal / declaration information and the behavior pattern information corresponding to the goal / declaration are input, the goal / declaration registration unit 136 registers the input goal / declaration and behavior pattern in the reach storage unit 137. When the goal / declaration is registered in this manner, the achievement level calculation and the provision of information regarding the achievement level are started with respect to the registered goal / declaration. Further, the achievement level is calculated according to the behavior pattern detected every day, and information on the achievement level for the target / declaration is provided to the user in real time.

  Refer to FIG. 15 again. The detection of the behavior pattern is realized by the functions of the sensor information acquisition unit 138 and the behavior pattern extraction unit 139. First, the sensor information acquisition unit 138 acquires sensor information from a motion sensor, a position sensor, or the like. The sensor information acquired by the sensor information acquisition unit 138 is input to the behavior pattern extraction unit 139. The behavior pattern extraction unit 139 extracts a behavior pattern from the input sensor information. Information on the behavior pattern extracted by the behavior pattern extraction unit 139 is input to the achievement level update unit 140. The behavior pattern extraction method can be the same as the behavior pattern extraction method by the behavior / situation analysis system 10 described above.

  When the behavior pattern information is input, the achievement level update unit 140 refers to the information on the goal / declaration registered in the achievement level storage unit 137, and the behavior pattern indicated by the input information corresponds to the goal / declaration. It is judged whether it corresponds to the action pattern to do. When the action pattern corresponding to the goal / declaration is met, the achievement degree storage unit 137 recognizes an achievement effect (for example, see FIG. 21) corresponding to the combination of the goal / declaration and the input action pattern. . Next, the achievement degree storage unit 137 calculates the current achievement degree based on the updated value of the achievement degree associated with the achievement effect, and records it in the achievement degree storage unit 137.

  For example, when the goal / declaration is “Diet”, Achievement effect “None” = − 5 points, Achievement effect “Small” = + 5 points, Achievement effect “Medium” = + 15 points, Achievement effect “Large” = + 30 points I want to think concretely with reference to FIG. First, when the user gets on the train for one hour, the behavior pattern “get on the train (achievement effect“ none ”)” is detected, so the achievement level update unit 140 sets the current achievement level to −5 points. . Next, when the user walks for 10 minutes, the behavior pattern “walking (achievement effect“ medium ”)” is detected, so the achievement level update unit 140 adds 15 points to the previous achievement level to obtain the current achievement level. Update to 10 points. In this way, the achievement level is updated according to the behavior pattern.

  Refer to FIG. 15 again. As described above, the achievement level for each target / declaration stored in the achievement level storage unit 137 is updated in real time. The achievement level for each goal / declaration stored in the achievement level storage unit 137 is read by the achievement level display unit 141 and presented to the user. For example, as shown in display examples # 1 and # 2 in FIG. 22, the achievement level display unit 141 displays the value of the achievement level updated according to the behavior together with the object indicating the behavior pattern taken by the user. . Display example # 1 indicates that the achievement level has increased because the user has performed a “run” action, and the updated achievement level has become 35.

  The configuration of the information providing system 13 according to configuration example # 1 has been described above. In the above description, the text information is analyzed first and the sensor information is analyzed later. However, the order of the analysis processing may be reversed. The achievement level notification method may be a voice guidance instead of the screen display or a method of expressing by vibration or light blinking. For example, a configuration in which the intensity of vibration changes according to the degree of achievement or the blinking speed or brightness of light changes can be considered. Such modifications naturally belong to the technical scope of the present embodiment.

(2-2-2: Process flow)
Next, the flow of processing executed by the information providing system 13 will be described with reference to FIG. FIG. 23 is an explanatory diagram for describing a flow of processing executed by the information providing system 13. Note that the order of some of the processing steps shown in FIG. 23 may be changed. For example, the order of the processing steps related to the analysis of the text information may be interchanged with the order of the processing steps related to the analysis of the sensor information.

  As shown in FIG. 23, first, the information providing system 13 acquires text information by the function of the text information acquisition unit 131 (S101). Next, the information providing system 13 extracts information about the experience from the text information by the function of the experience extracting unit 132 (S102). Information related to the experience includes, for example, the type of experience, the experience location, and the experience time. Next, the information providing system 13 uses the function of the goal / declaration extraction unit 133 to extract information on the goal / declaration from information on the experience (S103).

  Next, the information providing system 13 extracts an action pattern corresponding to the target / declaration extracted in step S103 by the function of the target / declaration collating unit 134 (S104). Next, the information providing system 13 registers the goal / declaration and the behavior pattern corresponding to the goal / declaration in the achievement level storage unit 137 by the function of the goal / declaration registration unit 136 (S105). ).

  On the other hand, the information providing system 13 acquires sensor information by the function of the sensor information acquisition unit 138 (S106). Next, the information providing system 13 extracts a behavior pattern from the sensor information by the function of the behavior pattern extraction unit 139 (S107). Next, the information providing system 13 recognizes the achievement effect of the goal / declaration corresponding to the behavior pattern extracted in step S107 by the function of the achievement degree update unit 140, and calculates the current achievement degree according to the recognized achievement effect. (S108). Next, the information providing system 13 displays the achievement level of the goal / declaration by the function of the achievement level display unit 141 (S109), and ends the series of processes.

  The flow of processing executed by the information providing system 13 has been described above.

(2-2-3: Example of screen display)
Here, a supplementary explanation will be given for the method of displaying the achievement level.

  As a method of displaying the achievement level, there are methods such as display examples # 1 and # 2 in FIG. That is, a method of displaying an object corresponding to the current behavior pattern and displaying the achievement level obtained as a result of taking the current behavior pattern is an example. In particular, in the example of FIG. 22, an arrow indicates whether the achievement level has increased or decreased according to the current action pattern, and the effect on the target / declaration can be seen at a glance. Such a display makes it possible to prompt the user to take an action pattern that has a good effect. In addition, since the effect is reflected in the numerical value of the achievement level in real time, the effect of sustaining the motivation of the user working on the target / declaration can be expected.

(2-2-4: Modified example (application to animals))
By the way, so far we have been explaining the technology that targets human behavior patterns. However, the technique according to Configuration Example # 1 can be applied to animals other than humans. For example, it is also possible to obtain a degree of achievement of a pet's goal by attaching a sensor to a collar of a pet such as a dog or a cat and inputting the pet's goal / declaration as text information. For example, it is possible to obtain data such as how much a pet is exercising in a place or time zone where the owner cannot reach the eyes. By analyzing such data and managing the health of the pet, it is thought that it will be effective in preventing the disease of the pet.

  Heretofore, the technology according to Configuration Example # 1 has been described. According to the technology according to the configuration example # 1, it is possible to present the achievement status related to the user declaration from the matching status between the user declaration obtained from the input information and the estimated behavior pattern.

(Application example)
As described above, when the technology according to Configuration Example # 1 is applied, an achievement level based on the user's goal / declaration and behavior pattern is obtained. Therefore, this achievement is displayed in a graph and displayed to the user, or the degree of effort toward the goal / declaration is determined depending on whether the achievement is increasing or decreasing toward the achievement of the goal / declaration. A display method for displaying can be realized. In addition, when it is difficult to achieve the goal / declaration (when the achievement level is extremely low (such as below the threshold)), representative examples of behavior patterns that caused the achievement level to decrease (high frequency) It is also possible to realize a display method that presents the cause of a difficult situation to the user by presenting a user) or a histogram. Furthermore, by presenting the user with an action pattern that has the opposite tendency to the representative example of the action pattern that caused the achievement level to be reduced, a display method that gives the user advice to achieve the goal / declaration can be realized. It is. By applying such a display method, it is possible to directly or indirectly support the achievement of the user's goal / declaration.

[2-3: Configuration example # 2 (display detailed action)]
Next, configuration example # 2 will be described. The configuration example # 2 relates to a mechanism for providing information related to a user experience obtained from input information by adding it to behavior pattern information.

(2-3-1: System configuration details)
The system (information providing system 17) according to Configuration Example # 2 is, for example, as shown in FIG. As shown in FIG. 24, the information providing system 17 includes a text information acquisition unit 171, an experience extraction unit 172, an extraction result storage unit 173, a sensor information acquisition unit 174, an action pattern extraction unit 175, and an action / experience. A collation unit 176, a correspondence relationship storage unit 177, an additional experience search unit 178, and an action / additional experience display unit 179 are provided.

  The functions of the sensor information acquisition unit 174 and the behavior pattern extraction unit 175 can be realized by using the functions of the behavior / situation analysis system 10 described above. In addition, among the above-described components included in the information providing system 17, it is free to design which component element functions as the information terminal CL and which component component functions as the server device SV. For example, it is desirable to design in consideration of the computing capability and communication speed of the information terminal CL.

  The text information acquisition unit 171 acquires text information input by the user. For example, the text information acquisition unit 171 may be an input device for a user to input text, or may be an information collection device that acquires text information from a social network service or application. Here, for convenience of explanation, the description will be made assuming that the text information acquisition unit 171 is an input means such as a software keyboard.

  The text information acquired by the text information acquisition unit 171 is input to the experience extraction unit 172. At this time, the time information when the text information is input may be input to the experience extraction unit 172 together with the text information. When text information is input, the experience extraction unit 172 analyzes the input text information and extracts information related to the user's experience from the text information. The information related to the experience is, for example, information including the experienced event (type of experience, etc.), the experienced location, the experienced time, and the like. The function of the experience extraction unit 172 is substantially the same as the function of the experience extraction unit 132 according to Configuration Example # 1. Information about the experience extracted by the experience extraction unit 172 is stored in the extraction result storage unit 173.

  On the other hand, the sensor information acquisition unit 174 acquires sensor information from a motion sensor, a position sensor, or the like. The sensor information acquired by the sensor information acquisition unit 174 is input to the behavior pattern extraction unit 175. The behavior pattern extraction unit 175 extracts a behavior pattern from the input sensor information. Information on the behavior pattern extracted by the behavior pattern extraction unit 175 is input to the behavior / experience collation unit 176. The behavior pattern extraction method can be the same as the behavior pattern extraction method by the behavior / situation analysis system 10 described above.

  When the behavior pattern information is input, the behavior / experience collation unit 176 refers to the correspondence relationship between the behavior pattern and the experience stored in the correspondence relationship storage unit 177, and corresponds to the behavior pattern indicated by the input information. Extract experiences. For example, as illustrated in FIG. 25, the correspondence relationship storage unit 177 stores experiences associated with each behavior pattern. As described above, the behavior pattern is obtained from the sensor information. On the other hand, experience information is obtained from text information. The method of obtaining behavior pattern and experience information is substantially the same as the above-described configuration example # 1.

  The experience information extracted by the behavior / experience collation unit 176 and the behavior pattern information corresponding to the experience are input to the additional experience search unit 178. When experience information is input, the additional experience search unit 178 refers to the extraction result storage unit 173 and searches for the same experience as the experience indicated by the input information. When the same experience as the experience indicated by the input information is detected as a result of the search, the additional experience search unit 178 displays text information corresponding to the detected experience, information related to the experience (type of experience, experience location, experience time). , Target of experience, etc.). For example, the additional experience search unit 178 obtains information related to the experience as shown in FIG.

  The search result by the additional experience search unit 178 is input to the action / additional experience display unit 179. When the search result is input, the action / additional experience display unit 179 displays information related to the experience based on the input search result. For example, as shown in FIG. 27, the action / additional experience display unit 179 displays information related to the action pattern and the experience. FIG. 27 shows a case of displaying behavior pattern information obtained when only sensor information is used, and a case of displaying behavior pattern information and experience information obtained when sensor information and text information are used. Are shown side by side. When text information is used in addition to sensor information, detailed information about the experience can be obtained as shown in FIG. 27, so that detailed information can be displayed.

  In case # 1, the sensor information alone can only display an object corresponding to the behavior pattern “walking”, but if the text information is additionally used, an object related to “dog” that is the target of the behavior is displayed. Is possible. In case # 2, the sensor information alone can only display objects corresponding to the behavior pattern “run”, but if text information is additionally used, objects related to the “working” type of experience are displayed. Is possible.

  Furthermore, in case # 3, the sensor information alone can only display an object corresponding to the behavior pattern “get in the car”, but if the text information is additionally used, the experience type “conversation” and the experience place “ It is possible to display objects related to “car”. In addition, although the method of using text information additionally has been illustrated here, it is possible to specify the type of experience “conversation” or the experience location “car” by using voice information. It is also possible to realize the same detailed display by using the above. In addition, when a voice recognition technique is used, a voice signal can be converted into text information. Therefore, a detailed display as shown in FIG. 27 can be realized by a similar method.

  The configuration of the information providing system 17 according to Configuration Example # 2 has been described above. In the above description, the text information is analyzed first and the sensor information is analyzed later. However, the order of the analysis processing may be reversed. The detailed information notification method may be voice guidance instead of screen display. Such modifications naturally belong to the technical scope of the present embodiment.

(2-3-2: Process flow)
Next, the flow of processing executed by the information providing system 17 will be described with reference to FIG. FIG. 28 is an explanatory diagram for describing a flow of processing executed by the information providing system 17. Note that the order of some of the processing steps shown in FIG. 28 may be changed. For example, the order of the processing steps related to the analysis of the text information may be interchanged with the order of the processing steps related to the analysis of the sensor information.

  As shown in FIG. 28, first, the information providing system 17 acquires text information by the function of the text information acquisition unit 171 (S111). Next, the information providing system 17 extracts information about the experience from the text information by the function of the experience extracting unit 172 (S112). Next, the information providing system 17 acquires sensor information by the function of the sensor information acquisition unit 174 (S113). Next, the information providing system 17 extracts a behavior pattern from the sensor information by the function of the behavior pattern extraction unit 175 (S114).

  Next, the information providing system 17 collates the behavior pattern extracted in step S114 with the experience by the function of the behavior / experience collation unit 176, and extracts experience information corresponding to the behavior pattern (S115). Next, the information providing system 17 extracts information related to the experience corresponding to the experience extracted in step S115 from the information related to the experience extracted in step S112 by the function of the additional experience search unit 178 (S116). Next, the information providing system 17 displays information corresponding to the information related to the experience extracted in step S116 together with information corresponding to the behavior pattern extracted from the sensor information by the function of the behavior / additional experience display unit 179 ( S117), a series of processing ends.

  The flow of processing executed by the information providing system 17 has been described above.

(2-3-3: Example of screen display)
Here, a supplementary explanation will be given regarding the display method of detailed information.

  As a detailed information display method, there are methods as shown in cases # 1 to # 3 in FIG. In other words, information on the behavior pattern detected from the sensor information is displayed, and information about the experience is additionally displayed when the information about the experience corresponding to the behavior pattern is obtained from the text information. One example is the method. In addition, time information (“5 minutes”) is additionally displayed as a detailed display as in case # 2, and a passenger and conversation contents are additionally displayed as a detailed display as in case # 3. Is also possible. When such detailed display becomes possible, the situation can be conveyed more accurately.

  Heretofore, the technology according to Configuration Example # 2 has been described.

[2-4: Configuration example # 3 (Determination of daily / unusual behavior)]
Next, configuration example # 3 will be described. Configuration example # 3 relates to a mechanism for discriminating and providing an extraordinary behavior and experience among user experiences obtained from input information such as behavior pattern information and text information.

(2-4-1: Details of system configuration)
A system (information providing system 19) according to Configuration Example # 3 is, for example, as shown in FIG. As shown in FIG. 29, the information providing system 19 includes a text information acquisition unit 191, an experience extraction unit 192, an extraction result storage unit 193, a sensor information acquisition unit 194, an action pattern extraction unit 195, and an action / experience. The collation unit 196, the correspondence relationship storage unit 197, the extraordinary behavior determination unit 198, and the extraordinary behavior display unit 199 are included.

  The functions of the sensor information acquisition unit 194 and the behavior pattern extraction unit 195 can be realized by using the functions of the behavior / situation analysis system 10 described above. In addition, among the above-described components included in the information providing system 19, it is free to design which component component functions as the information terminal CL and which component component functions as the server device SV. For example, it is desirable to design in consideration of the computing capability and communication speed of the information terminal CL.

  The text information acquisition unit 191 acquires text information input by the user. For example, the text information acquisition unit 191 may be an input device for a user to input text, or may be an information collection device that acquires text information from a social network service or application. Here, for convenience of explanation, the description will be made assuming that the text information acquisition unit 191 is an input means such as a software keyboard.

  The text information acquired by the text information acquisition unit 191 is input to the experience extraction unit 192. At this time, the time information when the text information is input may be input to the experience extraction unit 192 together with the text information. When text information is input, the experience extraction unit 192 analyzes the input text information and extracts information related to the user's experience from the text information. The information related to the experience is, for example, information including the experienced event (type of experience, etc.), the experienced location, the experienced time, and the like. The function of the experience extraction unit 192 is substantially the same as the function of the experience extraction unit 132 according to Configuration Example # 1. Information about the experience extracted by the experience extraction unit 192 is stored in the extraction result storage unit 193.

  On the other hand, the sensor information acquisition unit 194 acquires sensor information from a motion sensor, a position sensor, or the like. The sensor information acquired by the sensor information acquisition unit 194 is input to the behavior pattern extraction unit 195. The behavior pattern extraction unit 195 extracts a behavior pattern from the input sensor information. The behavior pattern information extracted by the behavior pattern extraction unit 195 is input to the behavior / experience collation unit 196. The behavior pattern extraction method can be the same as the behavior pattern extraction method by the behavior / situation analysis system 10 described above.

  When the behavior pattern information is input, the behavior / experience collation unit 196 refers to the correspondence relationship between the behavior pattern and the experience stored in the correspondence storage unit 197 and corresponds to the behavior pattern indicated by the input information. Extract experiences. For example, as shown in FIG. 25, the correspondence relationship storage unit 197 stores experiences associated with each behavior pattern. As described above, the behavior pattern is obtained from the sensor information. On the other hand, experience information is obtained from text information. The method of obtaining behavior pattern and experience information is substantially the same as the above-described configuration example # 1.

  Information on the experience extracted by the behavior / experience collation unit 196 and information on the behavior pattern corresponding to the experience are input to the extraordinary behavior determination unit 198. When the behavior pattern information is input, the extraordinary behavior discriminating unit 198 discriminates whether or not the input behavior pattern information indicates an extraordinary behavior. When experience information is input, the extraordinary behavior determination unit 198 determines whether the input experience information indicates an extraordinary experience.

  For example, the extraordinary behavior discriminating unit 198 discriminates an extraordinary behavior and an extraordinary experience based on an extraordinary condition as shown in FIG.

  In the example of FIG. 30, in the case of (unusual # 1), the extraordinary behavior discriminating unit 198 determines whether or not a time zone abnormality has occurred in the behavior pattern extracted from the sensor information. In other words, the extraordinary behavior discriminating unit 198 determines that the behavior is non-existent when a certain type of behavior is extracted in a time zone different from the time zone that is normally extracted or when it is not extracted in any time zone. It is determined that it is a daily action. Specifically, for a user who normally extracts “walking” behavior in the morning and evening, if “walking” behavior is extracted at midnight, it is determined that the “walking” behavior at midnight is an extraordinary behavior. .

  In the case of (unusual # 2), the extraordinary behavior determination unit 198 determines whether or not a type abnormality has occurred in the behavior pattern extracted from the sensor information. That is, the extraordinary behavior discriminating unit 198 discriminates that the behavior is an extraordinary behavior when a type of behavior different from the behavior usually extracted is extracted for a certain time period. Specifically, for users who normally extract “walking” behavior or “train” behavior in the morning, if “running” behavior or “bicycle” behavior is extracted, It is determined that it is an extraordinary action.

  In the case of (unusual # 3), the extraordinary behavior determination unit 198 determines whether or not a time zone abnormality has occurred in the experience extracted from the text information. In other words, the extraordinary behavior discriminating unit 198 determines that the experience is non-existent when a certain type of experience is extracted in a time zone different from the time zone that is normally extracted or when it is not extracted in any time zone. It is determined that it is a daily action. Specifically, for users whose “meal” experience is extracted in the morning, noon, and evening, this applies when the “meal” experience is extracted late at night, or when the “meal” experience is not extracted during the day The experience is determined to be an extraordinary action.

  In the case of (unusual # 4), the extraordinary behavior determination unit 198 determines whether or not a type abnormality has occurred in the experience extracted from the text information. In other words, the extraordinary behavior discriminating unit 198 discriminates that the experience is an extraordinary behavior when an experience of a different type from the experience that is normally extracted is extracted for a certain time period. Specifically, for a user whose “meal” experience is extracted at noon, when the “run” experience at noon is detected, the “run” experience at noon is determined to be an extraordinary action.

  Refer to FIG. 29 again. The extraordinary behavior discrimination result by the extraordinary behavior discriminating unit 198 is input to the extraordinary behavior display unit 199 as described above. The extraordinary behavior display unit 199 highlights an object or text corresponding to the extraordinary behavior, or displays a new object indicating the extraordinary behavior.

  Heretofore, the configuration of the information providing system 19 according to Configuration Example # 3 has been described. In the above description, the text information is analyzed first and the sensor information is analyzed later. However, the order of the analysis processing may be reversed. In addition, the processing for discriminating daily / irregular is described as an example, but a mechanism for discriminating positive behavior / negative behavior can be similarly constructed. Such modifications naturally belong to the technical scope of the present embodiment.

(2-4-2: Application examples)
FIG. 30 specifically shows the extraordinary behavior discrimination logic, but by making it possible to discriminate the daily behavior / the extraordinary behavior in this way, it is possible to estimate the user's health condition and detailed situation. become. For example, in the case of (non-daily # 1) shown in FIG. 30, since late-night behavior that is not normally extracted is extracted, it is estimated that it is insomnia. In the example of FIG. 30, the conditions are simplified for convenience of explanation. For example, if the “sleeping” behavior is extracted during the day, it is estimated that the reason is staying up late or disturbing the life rhythm. Furthermore, by adding conditions, it can be used for diagnosis of health conditions such as insomnia.

  Similarly, in the example of (unusual # 2), it is presumed that dieting is attempted during commuting time as a reason for the extraordinary behavior. In the example of (unusual # 3), it is presumed that the reason for the extraordinary behavior is simply busy with work. Furthermore, in the example of (unusual # 4), it is presumed that the reason for the extraordinary behavior is that they are on a diet that skips lunch. It may be possible to improve the accuracy of reason estimation by combining extraordinary actions. It would also be useful to improve extraordinary conditions and construct reason estimation algorithms from extraordinary behavior histories and statistical results based on sensor information and text information. Thus, the technology according to the present embodiment can be applied in various ways.

  Heretofore, the technology according to Configuration Example # 3 has been described.

[2-5: Combination of configuration examples]
The techniques according to the above configuration examples # 1 to # 3 can be arbitrarily combined. Since the technology for extracting information about the experience from the text information and the technology for extracting the behavior pattern from the sensor information are common, by connecting other functional blocks in series or in parallel, any configuration example # 1 It is possible to combine some or all of # 3. Further, when combining a plurality of configuration examples, it is possible to expect a reduction in processing load, a reduction in memory usage, and the like by designing to share functional blocks having common functions. Such a combined configuration naturally belongs to the technical scope of the present embodiment.

<3: Hardware configuration example>
The functions of the constituent elements of the behaviour / situation recognition system 10, the information providing systems 13, 17, 19, the information terminal CL, and the server device SV are realized by using, for example, a hardware configuration as illustrated in FIG. 31. It is possible. That is, the function of each component is realized by controlling the hardware shown in FIG. 31 using a computer program. The form of the hardware is arbitrary, and includes, for example, a personal computer, a mobile phone, a portable information terminal such as a PHS, a PDA, a game machine, or various information appliances. However, the above PHS is an abbreviation of Personal Handy-phone System. The PDA is an abbreviation for Personal Digital Assistant.

  As shown in FIG. 31, this hardware mainly includes a CPU 902, a ROM 904, a RAM 906, a host bus 908, and a bridge 910. Further, this hardware includes an external bus 912, an interface 914, an input unit 916, an output unit 918, a storage unit 920, a drive 922, a connection port 924, and a communication unit 926. However, the CPU is an abbreviation for Central Processing Unit. The ROM is an abbreviation for Read Only Memory. The RAM is an abbreviation for Random Access Memory.

  The CPU 902 functions as, for example, an arithmetic processing unit or a control unit, and controls the overall operation of each component or a part thereof based on various programs recorded in the ROM 904, the RAM 906, the storage unit 920, or the removable recording medium 928. . The ROM 904 is a means for storing a program read by the CPU 902, data used for calculation, and the like. In the RAM 906, for example, a program read by the CPU 902, various parameters that change as appropriate when the program is executed, and the like are temporarily or permanently stored.

  These components are connected to each other via, for example, a host bus 908 capable of high-speed data transmission. On the other hand, the host bus 908 is connected to an external bus 912 having a relatively low data transmission speed via a bridge 910, for example. As the input unit 916, for example, a mouse, a keyboard, a touch panel, a button, a switch, a lever, or the like is used. Further, as the input unit 916, a remote controller (hereinafter referred to as a remote controller) capable of transmitting a control signal using infrared rays or other radio waves may be used.

  As the output unit 918, for example, a display device such as a CRT, LCD, PDP, or ELD, an audio output device such as a speaker or a headphone, a printer, a mobile phone, or a facsimile, etc. Or it is an apparatus which can notify audibly. However, the above CRT is an abbreviation for Cathode Ray Tube. The LCD is an abbreviation for Liquid Crystal Display. The PDP is an abbreviation for Plasma Display Panel. Furthermore, the ELD is an abbreviation for Electro-Luminescence Display.

  The storage unit 920 is a device for storing various data. As the storage unit 920, for example, a magnetic storage device such as a hard disk drive (HDD), a semiconductor storage device, an optical storage device, a magneto-optical storage device, or the like is used. However, the HDD is an abbreviation for Hard Disk Drive.

  The drive 922 is a device that reads information recorded on a removable recording medium 928 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, or writes information to the removable recording medium 928. The removable recording medium 928 is, for example, a DVD medium, a Blu-ray medium, an HD DVD medium, or various semiconductor storage media. Of course, the removable recording medium 928 may be, for example, an IC card on which a non-contact type IC chip is mounted, an electronic device, or the like. However, the above IC is an abbreviation for Integrated Circuit.

  The connection port 924 is a port for connecting an external connection device 930 such as a USB port, an IEEE 1394 port, a SCSI, an RS-232C port, or an optical audio terminal. The external connection device 930 is, for example, a printer, a portable music player, a digital camera, a digital video camera, or an IC recorder. However, the above USB is an abbreviation for Universal Serial Bus. The SCSI is an abbreviation for Small Computer System Interface.

  The communication unit 926 is a communication device for connecting to the network 932. For example, a wired or wireless LAN, Bluetooth (registered trademark), or a WUSB communication card, an optical communication router, an ADSL router, or various types It is a modem for communication. The network 932 connected to the communication unit 926 is configured by a wired or wireless network, such as the Internet, home LAN, infrared communication, visible light communication, broadcast, or satellite communication. However, the above LAN is an abbreviation for Local Area Network. The WUSB is an abbreviation for Wireless USB. The above ADSL is an abbreviation for Asymmetric Digital Subscriber Line.

<4: Summary>
Finally, the technical idea of this embodiment will be briefly summarized. The technical idea described below can be applied to various information processing apparatuses such as a PC, a mobile phone, a portable game machine, a portable information terminal, an information home appliance, and a car navigation system.

  The functional configuration of the information processing apparatus described above can be expressed as follows.

(1)
An experience extraction unit that extracts experience information indicating a user's experience from text information acquired from a social networking service or application;
An action extraction unit that extracts an action pattern from position sensor information and time information;
An experience verification unit that extracts experience information from the position sensor information with reference to information indicating a correspondence relationship between the behavior pattern and the experience information;
A display controller that displays information related to the experience information extracted from the text information and information related to the experience information obtained from the position sensor information;
An information processing system comprising:

(2)
The information processing system according to (1), wherein the text information is text information input by the user.

(3)
The information processing system according to (1) or (2), wherein the text information is information obtained based on voice information issued by the user.

(4)
The information processing system according to any one of (1) to (3), wherein the experience information is image information.

(5)
The information processing system according to any one of (1) to (4), wherein the experience extraction unit extracts the experience information based on a feature amount extracted from the text information.

(6)
The information processing system according to any one of (1) to (5), wherein the experience information extracted from the text information includes information related to an experience place.

(7)
The information processing system according to any one of (1) to (6), wherein the behavior extraction unit further extracts a behavior pattern by further using the movement / state pattern of the user.

(8)
The information processing system according to any one of (1) to (7), wherein the behavior pattern is determined based on a learning model.

(9)
The information processing system according to any one of (1) to (7), wherein the behavior pattern is determined based on a rule base.

(10)
The information on the experience information extracted from the displayed text information is any one of the above (1) to (9), which is information extracted corresponding to the information on the experience information obtained from the position sensor information. Information processing system according to item.

(11)
The information processing system according to any one of (1) to (10), wherein the display control unit further displays additional information related to at least one of the experience information.

(12)
The information processing system according to (11), wherein the additional information includes at least either text information or image information.

(13)
The information processing system according to any one of (1) to (12), wherein information processing of each unit of the information processing system is executed in a distributed manner by two or more information processing devices.

(14)
The information processing system according to any one of (1) to (12), wherein information processing of each unit of the information processing system is executed in a distributed manner between a server and an information terminal.

(15)
Extracting experience information indicating a user experience from text information obtained from a social networking service or application;
Extracting a behavior pattern from position sensor information and time information;
Extracting the experience information from the position sensor information with reference to information indicating a correspondence relationship between the behavior pattern and the experience information;
Displaying together information related to experience information extracted from the text information and information related to experience information obtained from the position sensor information by the arithmetic processing unit;
Including an information processing method.

(16)
An experience extraction unit that extracts experience information indicating a user's experience from text information acquired from a social networking service or application;
An action extraction unit that extracts an action pattern from position sensor information and time information;
An experience verification unit that extracts experience information from the position sensor information with reference to information indicating a correspondence relationship between the behavior pattern and the experience information;
A display controller that displays information related to the experience information extracted from the text information and information related to the experience information obtained from the position sensor information;
A program to make a computer realize.

  The preferred embodiments according to the present technology have been described above with reference to the accompanying drawings, but it is needless to say that the present technology is not limited to the configuration examples disclosed herein. It will be apparent to those skilled in the art that various changes and modifications can be made within the scope of the claims, and these are naturally within the technical scope of the present technology. Understood.

DESCRIPTION OF SYMBOLS 10 Action / situation analysis system 101 Motion sensor 102 Motion / state recognition part 103 Time information acquisition part 104 Position sensor 105 GIS information acquisition part 106 Action / situation recognition part 13 Achievement provision system 131 Text information acquisition part 132 Experience extraction part 133 Objective / Declaration Extraction Unit 134 Target / Declaration Verification Unit 135 Correspondence Relationship Storage Unit 136 Target / Declaration Registration Unit 137 Achievement Level Storage Unit 138 Sensor Information Acquisition Unit 139 Behavior Pattern Extraction Unit 140 Achievement Level Update Unit 141 Achievement Level Display Unit 151 Type Features Extraction unit 152 Experience type discrimination unit 153 Experience type model storage unit 154 Location feature amount extraction unit 155 Experience location extraction unit 156 Experience location model storage unit 157 Time feature amount extraction unit 158 Experience time extraction unit 159 Experience time model storage unit 171 Text information Acquisition part 1 72 Experience extraction unit 173 Extraction result storage unit 174 Sensor information acquisition unit 175 Action pattern extraction unit 176 Action / experience collation unit 177 Correspondence relationship storage unit 178 Additional experience search unit 179 Action / additional experience display unit 191 Text information acquisition unit 192 Experience extraction Section 193 Extraction result storage section 194 Sensor information acquisition section 195 Action pattern extraction section 196 Action / experience collation section 197 Correspondence relation storage section 198 Extraordinary action discrimination section 199 Extraordinary action display section CL Information terminal SV server device

Claims (16)

An experience extraction unit that extracts experience information indicating a user's experience from text information acquired from a social networking service or application;
An action extraction unit that extracts an action pattern from position sensor information and time information;
An experience verification unit that extracts experience information from the position sensor information with reference to information indicating a correspondence relationship between the behavior pattern and the experience information;
A display controller that displays information related to the experience information extracted from the text information and information related to the experience information obtained from the position sensor information;
An information processing system comprising:   The information processing system according to claim 1, wherein the text information is text information input by the user.   The information processing system according to claim 1, wherein the text information is information obtained based on voice information issued by the user.   The information processing system according to claim 1, wherein the experience information is image information.   The information processing system according to claim 1, wherein the experience extraction unit extracts the experience information based on a feature amount extracted from the text information.   The information processing system according to claim 1, wherein the experience information extracted from the text information includes information related to an experience place.   The information processing system according to claim 1, wherein the behavior extraction unit further extracts a behavior pattern by further using the movement / state pattern of the user.   The information processing system according to claim 1, wherein the behavior pattern is determined based on a learning model.   The information processing system according to claim 1, wherein the behavior pattern is determined based on a rule base.   The information processing system according to claim 1, wherein the information related to the experience information extracted from the displayed text information is information extracted corresponding to the information related to the experience information obtained from the position sensor information.   The information processing system according to claim 1, wherein the display control unit further displays additional information related to at least one of the experience information.   The information processing system according to claim 11, wherein the additional information includes at least either text information or image information.   The information processing system according to claim 1, wherein information processing of each unit of the information processing system is executed in a distributed manner by two or more information processing apparatuses.   The information processing system according to claim 1, wherein information processing of each unit of the information processing system is executed in a distributed manner between a server and an information terminal. Extracting experience information indicating a user experience from text information obtained from a social networking service or application;
Extracting a behavior pattern from position sensor information and time information;
Extracting the experience information from the position sensor information with reference to information indicating a correspondence relationship between the behavior pattern and the experience information;
Displaying together information related to experience information extracted from the text information and information related to experience information obtained from the position sensor information by the arithmetic processing unit;
Including an information processing method. An experience extraction unit that extracts experience information indicating a user's experience from text information acquired from a social networking service or application;
An action extraction unit that extracts an action pattern from position sensor information and time information;
An experience verification unit that extracts experience information from the position sensor information with reference to information indicating a correspondence relationship between the behavior pattern and the experience information;
A display controller that displays information related to the experience information extracted from the text information and information related to the experience information obtained from the position sensor information;
A program to make a computer realize.

Images