KR101256907B1 - Apparatus and method for recognizing user state in real-time - Google Patents

Abstract

In accordance with an aspect of the present invention, an apparatus for recognizing a real-time user situation includes: an acceleration data collector configured to collect acceleration data through an acceleration sensor; A GPS coordinate collecting unit which collects GPS coordinate data through a GPS sensor; A user motion inference unit for inferring a user's motion from the collected acceleration data; A user behavior inference unit for inferring the user's behavior from the user's motion; A place name converting unit converting the GPS coordinate data into a place name; A user location determiner configured to determine a location of the user from the place name; And a user context determiner configured to determine a user's situation using the inferred user's behavior and the determined user's location.

Description

Apparatus and method for recognizing user state in real-time}

The present invention relates to an apparatus and method for recognizing a user's situation, and more particularly, to an apparatus and method for recognizing a user's situation in real time using acceleration information and GPS information collected in a mobile environment.

Recently, various smartphones such as Apple's iPhone and Google's Android phone have been released, which are equipped with hardware such as CPU, acceleration sensor, light sensor, magnetic sensor, gyro sensor, GPS receiver, WiFi, etc. have. Therefore, it is possible to provide various services through applications based on such hardware.

Since smartphones have characteristics of a mobile phone, it is a platform suitable for providing a user-specific service by recognizing a user's behavior because the user's portability is high. The large amount of data collected from smartphones is constantly changing over time. In order to analyze the changing data to recognize the user's situation in real time, a technique for analyzing the data, converting it into semantic information, and synthesizing the semantic information to derive the overall upper level semantic information is required.

An object of the present invention is to provide an apparatus and method for recognizing a user situation for effectively recognizing a user's situation in real time using acceleration information and GPS information collected in a mobile environment.

In order to solve the above technical problem, the user situation real-time recognition apparatus according to the present invention, an acceleration data collection unit for collecting the acceleration data through the acceleration sensor; A GPS coordinate collecting unit which collects GPS coordinate data through a GPS sensor; A user motion inference unit for inferring a user's motion from the collected acceleration data; A user behavior inference unit for inferring the user's behavior from the user's motion; A place name converting unit converting the GPS coordinate data into a place name; A user location determiner configured to determine a location of the user from the place name; And a user context determiner configured to determine a user's situation using the inferred user's behavior and the determined user's location.

In order to solve the above technical problem, the user situation real-time recognition method according to the present invention, collecting the acceleration data through the acceleration sensor; Collecting GPS coordinate data via a GPS sensor; Inferring a user's motion from the collected acceleration data; Inferring the user's action from the user's action; Converting the GPS coordinate data into a place name; Determining a location of the user from the place name; And determining the user's situation using the inferred user's behavior and the determined user's location.

In order to solve the above technical problem, there is provided a computer-readable recording medium having recorded thereon a program for executing the user situation real-time recognition method according to the present invention.

According to the present invention described above, the user's situation is probabilistically inferred from the acceleration data and the GPS coordinate data collected in the mobile environment, and the stochastic inferred user's situation is provided to the mobile terminal in real time, which is suitable for the user's situation. You can recommend services or applications. For example, when the user's situation is 'meal', it may be used to recommend a restaurant that suits the user's taste, and when the user's situation is 'boarding and moving', the bus route map may be provided. . In addition, according to the present invention, the user's motion (sitting, standing, walking, running, etc.) information can be recognized in real time using acceleration data as well as the upper situation of the user's behavior (meal, bedtime, shopping, etc.). Detailed information such as calculating calories for health, recommending the right music according to the current operation, or adjusting the volume of the mobile phone according to the current state is also available using the information.

1 is a block diagram of a user context real time recognition apparatus according to an embodiment of the present invention.
2 illustrates a hierarchical probabilistic model for 'user behavior inference' and 'user behavior inference'.
3 shows a hierarchical probabilistic model for 'place name conversion' and 'user location determination'.
4 shows an example of a hierarchical Bayesian network according to an embodiment of the present invention.
5 shows an example of how the user context information is visualized and displayed according to an embodiment of the present invention.
6 is a flowchart of a user context real-time recognition method according to an embodiment of the present invention.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the drawings. In the following description and the accompanying drawings, substantially the same components are denoted by the same reference numerals, and redundant description will be omitted. In the following description of the present invention, a detailed description of known functions and configurations incorporated herein will be omitted when it may make the subject matter of the present invention rather unclear.

1 is a block diagram of a user context real time recognition apparatus according to an embodiment of the present invention. The user situation recognition apparatus according to the present embodiment may be provided in a smart phone. However, in addition to a smartphone, any device having an acceleration sensor and a GPS sensor as a portable mobile terminal (eg, a mobile phone, a tablet PC, etc.) may be provided anywhere.

As shown in the user context recognition apparatus according to the present embodiment, the acceleration data collection unit 110, the user motion inference unit 120, the user behavior inference unit 130, the GPS coordinate collection unit 140, the place name conversion unit 150, a user location determiner 160, a user context determiner 170, and a user interface 180.

The acceleration data collection unit 110 collects acceleration data through, for example, an acceleration sensor (not shown) provided in the smartphone. Since the user carries the smartphone, the acceleration data collected from the acceleration sensor becomes acceleration data corresponding to the user's movement. The acceleration sensor is preferably an acceleration sensor of two or more axes, and preferably has a sensitivity of -2g to 2g for each axis. The acceleration data is preferably collected at a collection cycle of 10 Hz or more, that is, at least 10 cycles per second.

The GPS coordinate collecting unit 140 collects GPS coordinate data through, for example, a GPS sensor (not shown) provided in the smartphone. Since the user carries the smartphone, the GPS coordinate data collected from the GPS sensor becomes GPS coordinate data corresponding to the user's location. GPS coordinate data may be latitude, longitude, speed, direction of travel, altitude, and the like. The GPS coordinate data is preferably collected at a collection cycle of 1 second or less, that is, at least once per second.

The user motion inference unit 120 infers the user's motion from the acceleration data collected by the acceleration data collector 110. The acceleration data collected by the acceleration data collector 110 is time series data. The user motion inference unit 120 infers a user's motion from the acceleration data collected for a certain period of time using a time series probability model, and repeats the user motion inference at a predetermined interval. For example, the user's motion is inferred from the acceleration data collected for 5 seconds using a time series probability model, and the user's motion inference is repeated every 5 seconds. Here, the 'movement' of the user refers to a user's behavior that is recognized for a relatively short time, and may be, for example, sitting, standing, walking, running, climbing, or falling. As a time series probabilistic model, a Hidden Markov Model (HMM), a Dynamic Bayesian Network (DBN), a Markov Random Field (MRF), a Conditional Random Field (CRF), and the like may be used. Referring to FIG. 2, a user's motion of 'walking', 'walking', 'standing', 'walking', 'walking', and 'walking' is derived from user's motion inference using a time series probability model from acceleration data collected for a certain period of time. It is shown how the sequence is obtained.

The user behavior inference unit 130 infers the user's behavior from the user's motion obtained in the user's motion inference unit 120. Here, the 'action' of the user refers to the user's behavior recognized for a relatively long time, and the user's behavior inference unit 130 uses the time series probability model from the sequence of the user's movements obtained from the user's motion inference unit 120. Infers the behavior of The user's behavior may be, for example, 'walking movement', 'bus movement', 'shopping', 'exercise', 'sleep', and the like. Similarly, the time series probabilistic model may include a Hidden Markov Model (HMM), a Dynamic Bayesian Network (DBN), a Markov Random Field (MRF), a Conditional Random Field (CRF), and the like. Referring to FIG. 2, a user of a 'walking move' through user behavior inference using a time series probability model from a user's action sequence of 'walk', 'walk', 'stand', 'walk', 'walk', and 'walk' The action is obtained.

As described above, in the embodiment of the present invention, 'user behavior inference' and 'user behavior inference' use a so-called hierarchical time series probability model. 2 illustrates such a hierarchical probabilistic model.

Referring again to FIG. 1, the place name converting unit 150 converts the GPS coordinate data collected by the GPS coordinate collecting unit 140 into a place name. GPS coordinate data collected by the GPS coordinate collecting unit 140 is also time series data. The place name converting unit 150 converts the GPS coordinate data collected for a certain period into a place name using a time series probability model. Similarly, the time series probabilistic model may include a Hidden Markov Model (HMM), a Dynamic Bayesian Network (DBN), a Markov Random Field (MRF), a Conditional Random Field (CRF), and the like. Referring to FIG. 3, a sequence of place names of 'road', 'road', 'house', 'house', 'unknown', and 'house' is obtained using a time series probability model from GPS coordinate data collected for a certain period of time. Is shown.

The user location determiner 160 determines the location of the user from a sequence of place names obtained by the place name converter 150 using a time series probability model. Referring to FIG. 3, a user's position is finally determined as a 'home' using a time series probability model from a sequence of place names of 'road', 'road', 'house', 'house', 'unknown' and 'house'. It shows the appearance.

As described above, in the embodiment of the present invention, 'place name conversion' and 'user location determination' use a so-called hierarchical time series probability model. 3 illustrates such a hierarchical probabilistic model.

Referring back to FIG. 1, the user context determination unit 170 may determine the user's behavior by using the user's behavior inferred by the user's behavior inference unit 130, the user's location and time information determined by the user's location determiner 160. Judge the situation. Here, the 'state' of the user refers to the current state of the user including the 'action' of the user. That is, the user's behavior inferred by the user behavior inference unit 130 is finally determined as the user's situation based on the location and time information of the user determined by the user location determiner 160, or the inferred user's behavior and the determined user The situation of the user is determined as new semantic information based on the location and time information of the. In this case, the user context determination unit 170 may infer a user's situation by integrating the user's behavior, the user's location, and time information using a Bayesian network. The user context determination unit 170 obtains a probability value of the current situation of the user by using the user's behavior, the user's location, and time information as evidence.

The Bayesian network preferably uses a hierarchical Bayesian network having a hierarchical structure. A hierarchical Bayesian network is a modeling technique that expresses the relationship between variables inherent in data with nodes and arcs in time. Each node represents a random variable and an arc represents an association between nodes. . 4 shows an example of such a hierarchical Bayesian network. 4 is an example of a hierarchical Bayesian network having a situation item of 'class', 'sleep' and 'exercise', wherein nodes related to user's actions, nodes related to user's actions, nodes related to time, places The nodes associated with are hierarchical.

Referring back to FIG. 1, the user interface 180 visualizes and displays the context information of the user determined by the user context determiner 170 to the user. Of course, the user interface 180 may be a typical user interface of a smart phone or a mobile terminal equipped with a user situation real-time recognition device. The user situation determination unit 170 may display the highest probability situation as the current situation of the user, or may display probability information about each situation. FIG. 5 illustrates an example in which user context information is visualized and displayed through the user interface 180.

6 is a flowchart of a user context real-time recognition method according to an embodiment of the present invention. The user context real time recognition method according to the present embodiment includes steps processed by the apparatus for real time user context recognition described above. Therefore, even if omitted below, the above description of the apparatus for real-time user situation recognition is applied to the user context real-time recognition method according to the present embodiment.

When the acceleration data is collected through the acceleration sensor (step 610), and the acceleration data is collected for a predetermined period (step 615), the user's motion is inferred from the collected acceleration data (step 620). If a certain amount of data is accumulated through user's motion inference (step 625), the user's action is inferred from the sequence of user's actions (step 630).

On the other hand, the GPS coordinate data is collected (step 640), and if the GPS coordinate data is collected for a predetermined period (step 645), the collected GPS coordinate data is converted into a place name (step 650). When a certain amount of data is accumulated (step 655), the user's position is determined from the sequence of place names (step 660).

In operation 670, the user's current situation is determined using the user's behavior inferred in operation 630 and the user's position determined in operation 660.

The above-described embodiments of the present invention can be embodied in a general-purpose digital computer that can be embodied as a program that can be executed by a computer and operates the program using a computer-readable recording medium. The computer-readable recording medium includes a storage medium such as a magnetic storage medium (e.g., ROM, floppy disk, hard disk, etc.), optical reading medium (e.g., CD ROM,

So far I looked at the center of the preferred embodiment for the present invention. It will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims. Therefore, the disclosed embodiments should be considered in an illustrative rather than a restrictive sense. The scope of the present invention is defined by the appended claims rather than by the foregoing description, and all differences within the scope of equivalents thereof should be construed as being included in the present invention.

Claims (15)

An acceleration data collector configured to collect acceleration data at predetermined cycles through the acceleration sensor;
A GPS coordinate collecting unit which collects GPS coordinate data at predetermined intervals through a GPS sensor;
A user motion inference unit that infers a user's motion using a time series probability model from acceleration data which is time series data collected for a predetermined period by the acceleration data collector;
A user behavior inference unit that infers a user's behavior using a time series probability model from the sequence of user operations obtained by the user's motion inference unit;
A place name converting unit for converting GPS coordinate data, which is time series data collected for a predetermined period, from the GPS coordinate collecting unit into place names using a time series probability model;
A user location determiner configured to determine a location of a user using a time series probability model from a sequence of place names obtained by the place name converter;
A user situation determination unit obtaining a probability value of the user's situation using a hierarchical Bayesian network from the inferred user's behavior and the determined user's location; And
And a user interface for visualizing and displaying the obtained probability value of the situation of the user. delete delete delete The method of claim 1,
And the user context determination unit further calculates a probability value of the situation of the user by further using time information. The method of claim 1,
The user context real-time recognition device is a user context real-time recognition device, characterized in that provided in the portable mobile terminal. delete (a) collecting acceleration data at predetermined intervals through the acceleration sensor;
(b) collecting GPS coordinate data at predetermined intervals through a GPS sensor;
(c) inferring a user's motion using a time series probability model from acceleration data which is time series data collected for a period of time in step (a);
(d) inferring a user's behavior using a time series probability model from the sequence of user actions obtained in step (c);
(e) converting GPS coordinate data, which is time series data collected for a period of time, to a place name using a time series probability model;
(f) determining a location of a user using a time series probability model from the sequence of place names obtained in step (e);
(g) obtaining a probability value of the situation of the user using a hierarchical Bayesian network from the inferred user's behavior and the determined user's location; And
and (h) visualizing and displaying the obtained probability value of the situation of the user. delete delete delete 9. The method of claim 8,
In the step (g), the user situation real-time recognition method, characterized by obtaining the probability value of the user's situation further using the time information. 9. The method of claim 8,
Each step of the user context real time recognition method is performed in a portable mobile terminal. delete A computer-readable recording medium having recorded thereon a program for executing the user situation real-time recognition method according to any one of claims 8, 12, and 13.

Images