KR20130071755A - Service server and terminal for providing service based on prediction of user's behavior - Google Patents

Abstract

PURPOSE: A service server for providing an adaptive service by predicting a user behavior and a terminal thereof are provided to construct a database by learning the common behavior patterns of users, thereby supplying a current behavior of a user and a surrounding environment. CONSTITUTION: A common pattern database (220) stores common behavior patterns of users. A receiving unit (230) receives sensory data from terminals. A prediction unit (240) predicts a user behavior by using the common pattern database and the sensory data. A terminal collects information related to a behavior pattern of a user by using a sensor. A service server generates the sensory data by using the information. A common pattern calculation unit (210) calculates the common behavior patterns by using the sensory data. [Reference numerals] (210) Common pattern calculation unit; (220) Common pattern DB; (230) Receiving unit; (240) Prediction unit; (250) Transmitting unit; (AA) Service information

Description

SERVICE SERVER AND TERMINAL FOR PROVIDING SERVICE BASED ON PREDICTION OF USER'S BEHAVIOR}

The following embodiments are related to a service server and a terminal for predicting a user's behavior using sensory data measured by the terminal and providing a service based on the predicted user's behavior.

Ubiquitous technology allows users to connect to the network freely regardless of time and place. Such ubiquitous technology can automatically detect the purpose of the user in daily life, thereby providing the necessary services for the user.

In order to provide such a service, research has been conducted on a method of recognizing a user's current situation and predicting the most appropriate service for the situation.

However, the existing service prediction method operates by setting a target for a current situation using a preset rule and executing an application program corresponding to the set target.

For example, the existing service prediction method presets a rule according to context information collected from a sensor and fact information prepared by domain experts, and predicts a service to be provided to a user based on the preset rule.

However, in the conventional service prediction method, service flexibility is inferior by predicting a service according to rules that are based on service prediction or rules that match a specific environment. In addition, when the user's environment is other than the preset rules, it is impossible to predict the corresponding service.

Embodiments of the present invention provide a technique for adaptively providing a service to a user's current behavior and surrounding environment by building a database by learning common behavior patterns from a plurality of users.

In addition, embodiments of the present invention provides a technology for providing a service that matches a unique individual pattern of each user by having a separate pattern database based on the unique behavior pattern of each user.

Furthermore, embodiments of the present invention retain not only an individual pattern database but also a common pattern database based on a common behavioral pattern of a plurality of users, thereby using the common pattern database for a period of time necessary for learning a unique individual pattern of each user. To provide technology to provide alternative services.

Service server according to an embodiment of the present invention comprises a common pattern database for storing a common behavior pattern of a plurality of users; A receiver configured to receive sensory data from at least one terminal; And a prediction unit for predicting the behavior of the user by using the common pattern database and the sensory data to generate service information to be provided to the user of the terminal, wherein the terminal uses the at least one sensor. Collects information related to a user's behavior pattern, and generates the sensory data using the information.

The service server further includes a common pattern calculator configured to calculate the common behavior pattern, the receiver obtains a plurality of sensory data from the plurality of users, and the common pattern calculator uses the plurality of sensory data. The common behavior pattern can be calculated.

The sensory data includes GPS data measured at the location of the user, and the prediction unit estimates a current location of the user based on at least the GPS data; A candidate selecting unit configured to select at least one location candidate recommended as the location to be moved next by the user by using the current location, the sensory data, and the common pattern database; And a probability calculator configured to calculate a probability associated with an expected value of each of the at least one position candidate based on the common pattern database.

The location estimator selects one location category from a plurality of predetermined location categories to estimate the location of the user, and the candidate selector selects the at least one location category to select the at least one location candidate. At least one of the location categories may be selected.

The sensory data includes a 360 degree panoramic image generated at the location of the user, and the prediction unit accumulates the panoramic image; And an image analyzer configured to analyze the accumulated panoramic image to predict the behavior of the user.

The image analyzer may further include: a destination predictor configured to predict a destination to which the user is currently moving based on the accumulated panorama image and the common pattern database; And an action determination unit that determines at least one action recommended to the user based on the destination based on the common pattern database.

The sensory data may further include information about a front face included in the panoramic image and information about a motion of a moving object included in the panoramic image, and the image analyzer may include the accumulated panoramic image and information regarding the front face. And a behavior recognizing unit recognizing a behavior being performed by the moving object based on the information about the motion of the moving object. And an action determiner configured to determine at least one action recommended to the user based on the recognized behavior based on the recognized behavior and the common pattern database.

The sensory data includes information about the state of the user, and the predictor comprises: a state predictor which predicts the physical state of the user and the mental state of the user using the sensory data and the common pattern database; And an action determiner configured to determine at least one action recommended to the user with respect to the physical state and the mental state based on the physical state, the mental state, and the common pattern database.

The service server includes an individual pattern database that stores unique behavior patterns of individual users; And an individual pattern calculator configured to calculate the unique behavior pattern using the sensory data, and the predictor may further use the individual pattern database to predict the user's behavior.

According to an embodiment of the present invention, a terminal may include a collection unit configured to collect information related to a behavior pattern of a user using a plurality of sensors; A generator configured to generate sensory data using the information; A transmitter for transmitting the sensory data to a server; A receiving unit for receiving service information for the user from the server; And an information providing unit for automatically providing the service information in response to the reception, wherein the server predicts the user's behavior based on common behavior patterns of the plurality of users and the sensory data. Create

The plurality of sensors may include a GPS sensor measuring GPS data at a location of the user, and the sensory data may include at least the GPS data.

The service information includes a plurality of location candidates and a probability of each of the plurality of location candidates recommended as the next position to be moved by the user, and the information provider includes information about the plurality of location candidates or each of the plurality of location candidates. It may include an information display unit for displaying at least one of the information about the probability of.

The information providing unit may include: an application determiner configured to determine a plurality of applications corresponding to the plurality of location candidates; And an application execution unit that automatically executes at least one of the plurality of applications based on the probabilities of each of the plurality of location candidates.

The plurality of sensors may include a plurality of cameras generating a 360 degree panoramic image at the location of the user, and the sensory data may include at least the panoramic image.

The service information includes a destination to which the user is currently moving and an action recommended to the user in relation to the destination, wherein the information providing unit displays at least one of information about the destination or information about the action. It may include.

The plurality of sensors may include a face detector for detecting a frontal face included in the panoramic image; And a motion detector configured to detect a motion of the moving object included in the panoramic image, wherein the sensory data may further include information about the front of the face and information about the motion of the moving object.

The service information includes an action recommended by the user in relation to the action being performed by the moving object and the action of the moving object, and the information providing unit includes at least one of information about the action of the moving object or information about the action. It may include an information display unit for displaying one.

The plurality of sensors may include an input unit configured to receive information regarding a state of a user, and the sensory data may include at least information about a state of the user.

The service information includes a physical state of the user, a mental state of the user, and an action recommended to the user in relation to the physical state and the mental state, wherein the information providing unit includes information about the physical state, the mental state It may include an information display unit for displaying at least one of information about the state, or information about the action.

The terminal includes a separate pattern database for storing a unique behavior pattern of each user; An individual pattern calculator configured to calculate the unique behavior pattern using the sensory data; A predicting unit predicting the user's behavior using the sensory data and the individual pattern database; And a selection unit for selecting any one of the service information generated based on the received service information and the behavior predicted by the prediction unit according to the reliability of the individual pattern database.

Embodiments of the present invention can provide a technology for adaptively providing a service to a user's current behavior and surrounding environment by building a database by learning common behavior patterns from a plurality of users.

In addition, embodiments of the present invention may provide a technology for providing a service corresponding to a unique individual pattern of each user by having a separate pattern database based on a unique behavior pattern of each user.

Furthermore, embodiments of the present invention retain not only an individual pattern database but also a common pattern database based on a common behavioral pattern of a plurality of users, thereby using the common pattern database for a period of time necessary for learning a unique individual pattern of each user. It is possible to provide a technology for providing an alternative service.

1 is a diagram illustrating a service system for predicting a user's behavior based on a user's behavior pattern according to an embodiment of the present invention, and providing a service to the user using the predicted behavior.
2 is a block diagram illustrating a service server for generating service information using sensory data according to an embodiment of the present invention.
3 is a block diagram illustrating a service server for predicting a user's behavior using at least GPS data according to an embodiment of the present invention.
4 is a block diagram illustrating a service server for predicting a user's behavior using at least a 360 degree panoramic image according to an embodiment of the present invention.
5 is a block diagram illustrating a service server for predicting a user's behavior using at least user's state information according to an embodiment of the present invention.
6 is a block diagram illustrating a service server predicting a user's behavior using a user's individual pattern database according to an embodiment of the present invention.
7 illustrates a terminal for generating sensory data using a GPS sensor and providing a service to a user based on a user's behavior predicted by a server using the sensory data according to an embodiment of the present invention. It is a block diagram.
8 is a diagram illustrating an information display unit that provides a service to a user based on a user's behavior predicted by a server using GPS data according to an embodiment of the present invention.
9 is a terminal for generating sensory data using at least a plurality of cameras and providing a service to a user based on a user's behavior predicted by a server using the sensory data according to an embodiment of the present invention. A block diagram is shown.
10 is a diagram illustrating an arrangement of a plurality of cameras for generating a 360 degree panoramic image according to an embodiment of the present invention.
FIG. 11 is a diagram illustrating an information display unit providing a service to a user based on a user's behavior predicted by a server using a 360-degree panoramic image according to an embodiment of the present invention.
12 is a view illustrating an input unit for receiving input of status information of a user according to an embodiment of the present invention.
FIG. 13 is a block diagram illustrating a terminal for predicting a user's behavior using an individual pattern database of a user according to an embodiment of the present invention.

1. A system including a service server and a plurality of terminals

1 is a diagram illustrating a service system for predicting a user's behavior based on a user's behavior pattern according to an embodiment of the present invention, and providing a service to the user using the predicted behavior.

1, a plurality of terminals 120 according to an embodiment of the present invention generates sensory data using at least one sensor and transmits the generated sensory data to the service server 110. .

Here, the sensory data generated by the terminal 120 includes a result value collected using various sensors. That is, the sensory data includes latitude and longitude information measured using a GPS sensor, temperature information measured using a thermometer, noise information measured using a sound level meter, illuminance information measured using an illuminometer, and an acceleration sensor. It may include acceleration information measured using the.

In addition, the sensory data includes all data capable of predicting a user's behavior pattern. For example, the sensory data may include all information that helps to predict a user's behavior pattern, such as a user's e-mail transmission and reception history, a user's call history, and a user's schedule (schedule) history.

In addition, the service server 110 may generate service information to be provided to the user by predicting the user's behavior in response to the received sensory data using a database that stores the user's behavior pattern.

More specifically, the service server 110 may calculate common behavior patterns of the plurality of users using the plurality of sensory data collected from the plurality of users. The service server 110 has a database (common pattern database) that stores the calculated common behavior pattern, and uses the common pattern database to store user's data corresponding to sensory data transmitted from at least one terminal 120. Predict behavior.

In this case, assume that there is a terminal 120 (user) first connected to the service system network by the service server 110. The service predicting user's behavior according to an embodiment of the present invention can learn a unique behavior pattern of each user and provide a customized service for each user using the individual pattern. However, in the case of the initial entry user, the service cannot be provided during the learning period of the user's unique behavior pattern.

The service server 110 according to an embodiment of the present invention maintains the aforementioned common pattern database to provide a service based on common behavior patterns to a plurality of users during a period of learning a unique behavior pattern of an initial entry user. This can solve the above problem.

Embodiments of the present invention may use a Hierarchal Temporal Memory (HTM) model when calculating common behavior patterns of a plurality of users or calculating unique behavior patterns of individual users. The HTM model is based on machine learning techniques. The service server 110 according to an embodiment of the present invention uses a HTM model to select common patterns frequently observed from a plurality of sensory data in a training step (a), and based on the plurality of spatial patterns (spatial patterns) can be generated. In addition, the service server according to an embodiment of the present invention can determine whether the sensory data received from the terminal is the same as any one of the generated plurality of spatial patterns, if not the same but very similar, The transmitted sensory data can be regarded as the very similar spatial pattern. (b) Next, the service server according to an embodiment of the present invention may generate a temporal sequence of spatial patters of the plurality of spatial patterns according to the order in which the generated plurality of spatial patterns are transmitted. . A plurality of such temporal sequences may be gathered to form a set of temporal sequences, which may be stored in a database.

According to an embodiment of the present invention, the service server may predict the user's behavior from the sensory data input using the temporal sequence set (ie, common pattern database) of the plurality of spatial patterns in the inferring step. Can be. At this time, the service server according to an embodiment of the present invention may select the actions of one or more users as candidates. In this case, the service server may calculate a probability for each of the actions of the plurality of users selected as candidates using the HTM model.

In the above, the service server according to an embodiment of the present invention has been described a method of calculating the user's behavior pattern, building a database, and predicting the user's behavior using the HTM model, the scope of the present invention is It is not limited only to the configuration using the HTM model.

2. Service server according to embodiments of the present invention

2 is a block diagram illustrating a service server for generating service information using sensory data according to an embodiment of the present invention.

Referring to FIG. 2, the service server 200 according to an embodiment of the present invention includes a receiver 230, a common pattern calculator 210, a common pattern database 220, a predictor 240, and a transmitter 250. ).

In a training step according to an embodiment of the present invention, the receiver 230 obtains a plurality of sensory data from a plurality of users, and the common pattern calculator 210 uses the plurality of sensory data. Calculate common behavior patterns of the plurality of users. As described above with reference to FIG. 1, the common pattern calculator 210 may calculate common behavior patterns of the plurality of users using the HTM model. In this case, the common pattern database 210 stores common behavior patterns of a plurality of users calculated by the common pattern calculator 210.

On the other hand, in the infer (infer) step according to an embodiment of the present invention, the receiver 230 receives sensory data from at least one terminal, the prediction unit 240 is the common pattern database and the sensor Using the data, the user's behavior is predicted. Similarly, the prediction unit 240 according to an embodiment of the present invention may predict the behavior of the user by using the HTM model.

Furthermore, the prediction unit 240 may generate service information to be provided to the user of the terminal based on the predicted behavior. The transmitter 250 transmits the generated service information to the user of the terminal.

In this case, at least one terminal transmitting sensory data collects information related to the behavior pattern of the user by using at least one sensor and generates sensory data using the information.

Hereinafter, the configuration and operation method of a service server according to embodiments of the present invention will be described in detail with reference to FIGS. 3 to 6.

3 is a block diagram illustrating a service server for predicting a user's behavior using at least GPS data according to an embodiment of the present invention.

Referring to FIG. 3, the service server 300 according to an exemplary embodiment of the present invention includes a receiver 320, a common pattern database 310, and a predictor 330.

The receiver 320 receives sensory data including GPS data measured at the user's location. At this time, the service server 300 according to an embodiment of the present invention by performing the above-described training step, the common patterns sufficient to predict the user's behavior in response to the sensory data received by the receiver 320 I can hold it. Therefore, in the following description of the embodiment of FIG. 3, it is assumed that the common pattern database 310 is already established.

The predictor 330 may include a position estimator 331, a candidate selector 332, and a probability calculator 333.

Here, the location estimator 331 estimates the current location of the user based on at least the GPS data. Since the GPS data includes information about longitude and information about latitude, the location estimator 331 may estimate the location of the user based on the information. In addition, the GPS data may further include information about the altitude. The location estimator 331 may more accurately estimate a user's location using the information on the altitude.

In this case, as described above with reference to FIG. 1, the sensory data may include not only GPS data but also all kinds of information to help predict a user's behavior pattern. Therefore, the position estimator 331 according to an embodiment of the present invention may further utilize not only the GPS data but also acceleration information of the user, illuminance information around the user, and noise information around the user.

For example, as a result of estimating the location using the GPS data, the location of the user may be estimated in an area where the overpass and the general road overlap on the GPS map. At this time, the position estimator 331 according to an embodiment of the present invention may further determine whether the user is passing over the elevated road or the general road using the acceleration information of the user.

Meanwhile, the location estimator 331 may be implemented according to a method of selecting any one of a plurality of location categories in order to estimate the location of the user. Here, the predetermined plurality of location categories may include a school, a hospital, a veterinary hospital, a library, a gas station, a house, and the like. That is, according to an embodiment of the present invention, the location estimator 331 may determine whether the user is currently located at a place corresponding to any one of the location categories.

In addition, the candidate selecting unit 332 selects at least one position candidate recommended by the user to the next position using the current position, the sensory data, and the common pattern database estimated by the position estimating unit 331. can do.

That is, the candidate selector 332 may recommend a place where the user of the terminal will move next according to a common pattern of a plurality of users. In this case, as described above, the candidate selector 332 may predict location candidates recommended as a place to which the user of the terminal moves. In this case, the candidate selector 332 may predict the location candidates by applying the estimated location of the user and various information included in the sensory data as input of the HTM model.

Meanwhile, the candidate selector 332 may be implemented according to a method of selecting at least one location category from among a plurality of predetermined location categories in order to select the location candidates. Here, the predetermined plurality of location categories may include a school, a hospital, a veterinary hospital, a library, a gas station, a house, and the like. That is, according to an embodiment of the present invention, the candidate selector 332 may select location candidates recommended as a place to be moved by the user by selecting one or more location categories from the location categories.

In addition, the probability calculator 333 may calculate a probability associated with an expected value of each of the at least one location candidate based on the common pattern database. In this case, as described above, the probability calculator 333 may calculate the probability of each of the position candidates recommended as a place to which the user of the terminal moves. Here, the probability of each of the location candidates may be associated with an expectation recommended as a place to which the user of the terminal moves.

The candidate selector 332 and the probability calculator 333 according to an embodiment of the present invention may be configured as one independent component according to the implementation method of the predictor 330. That is, the prediction unit 330 according to an embodiment of the present invention uses the location of the user estimated by the location estimator 331, the GPS data received by the receiver 320, and the common pattern database 310. In addition, location candidates recommended as a next place to be moved by the user of the terminal that has transmitted the GPS data may be simultaneously calculated.

The service server according to the embodiment of FIG. 3 may provide the service according to the embodiment of FIG. 8 in association with the terminal according to the embodiment of FIG. 7 to be described later. More specific matters will be described later.

4 is a block diagram illustrating a service server for predicting a user's behavior using at least a 360 degree panoramic image according to an embodiment of the present invention.

Referring to FIG. 4, the service server 400 according to an embodiment of the present invention includes a receiver 420, a common pattern database 410, and a predictor 430.

The receiver 420 receives sensory data including a 360 degree panoramic image measured at the user's location. Here, the 360-degree panoramic image is an image in which a 360-degree image is successively pasted around the user using a plurality of cameras. An arrangement of a plurality of cameras for generating the 360 degree panoramic image will be described later with reference to FIG. 10.

At this time, the service server 400 according to an embodiment of the present invention by performing the above-described training step, the common patterns sufficient to predict the user's behavior in response to the sensory data received by the receiving unit 420 I can hold it. Therefore, in the following description of the embodiment of FIG. 4, it is assumed that the common pattern database 410 is already established.

The predictor 430 may include an accumulator 431 and an image analyzer 432. The accumulator 431 may accumulate the received panoramic image, and the image analyzer 432 may predict the user's behavior by analyzing the accumulated panoramic image.

The image analyzer 432 according to an embodiment of the present invention may include a destination predictor 433 and an action determiner 434.

The destination predictor 433 may predict a destination to which the user is currently moving based on the accumulated panorama image and the common pattern database 410. That is, the destination predicting unit 433 according to an embodiment of the present invention analyzes the 360-degree panoramic image received from the terminal (user), and whether the image is frequently observed on the way to the user's commute or moves to the library. It can be determined whether the image is frequently observed in the middle.

In addition, the action determiner 434 may determine at least one action recommended to the user in relation to the user's destination predicted by the destination predictor 433. For example, when the destination predicting unit 433 determines that the user is moving to the library, the action determination unit 434 provides the user with information regarding the remaining rental period of the book loan or the late book, and the like. With regard to the information, it is possible to determine an action recommended to the user. Here, the destination predictor 433 and the action determiner 434 according to an embodiment of the present invention may perform the above-described operation using the HTM model.

Meanwhile, the image analyzer 432 according to another embodiment of the present invention may include a behavior recognizer 433 instead of a destination predictor.

In this case, the receiver 420 may receive not only a 360-degree panoramic image, but also sensory data further including information about the front face included in the panoramic image and information about the motion of the moving object included in the panoramic image. have.

The behavior recognizing unit 433 may recognize the behavior being performed by the moving object based on the accumulated panoramic image, information about the front face of the face, and information about the motion of the moving object. That is, the behavior recognizing unit 433 according to an embodiment of the present invention may detect the movement of a person around the user based on information on the front face of the accumulated panorama image and information on the motion of the moving object. have.

Furthermore, the behavior recognition unit 433 according to an embodiment of the present invention may recognize the action being performed by the neighboring person by applying the detected movement of the neighboring person as an input of the HTM model.

In this case, the action determiner 434 may determine at least one action recommended to the user based on the recognized behavior based on the recognized behavior and the common pattern database 410.

The service server according to the embodiment of FIG. 4 may provide a service according to the embodiment of FIG. 11 in association with a terminal according to the embodiment of FIGS. 9 and 10 to be described later. More specific matters will be described later.

5 is a block diagram illustrating a service server for predicting a user's behavior using at least user's state information according to an embodiment of the present invention.

Referring to FIG. 5, the service server 500 according to an exemplary embodiment of the present invention includes a receiver 520, a common pattern database 510, and a predictor 530.

The receiver 520 receives sensory data including state information of the user. Here, the state information of the user includes all information related to the state of the user, such as a feeling that the user feels at any point in time and symptoms that the user experiences at any point in time. The terminal used by the user may provide an input means to input information related to the state of the user. The input means provided by the terminal will be described later with reference to FIG. 12.

At this time, the service server 500 according to an embodiment of the present invention by performing the above-described training step, a common pattern sufficient to predict the user's behavior in response to the user's state information received by the receiving unit 520 Can hold them. Accordingly, in the following description of the embodiment of FIG. 5, it is assumed that the common pattern database 510 is already established.

The predictor 530 may include a state predictor 531 and an action determiner 532.

Here, the state predictor 531 may predict the physical state of the user and the mental state of the user by using the sensory data and the common pattern database. The state predicting unit 531 according to an embodiment of the present invention may predict the physical state of the user and the mental state of the user by applying state information of the user received by the receiving unit 520 as an input of the HTM model. have.

In addition, the action determiner 532 may determine at least one action recommended to the user in relation to the physical state and the mental state based on the physical state, the mental state, and the common pattern database. For example, when the state predictor 531 according to an embodiment of the present invention determines that the physical state of the user is a cold symptom, the action determination unit 532 asks the user to go to a hospital or stop at a pharmacy. The recommended action can be determined. Similarly, the action determiner 532 according to an embodiment of the present invention may determine the recommended action by using the HTM model.

6 is a block diagram illustrating a service server predicting a user's behavior using a user's individual pattern database according to an embodiment of the present invention.

Referring to FIG. 6, the service server 600 according to an embodiment of the present invention includes a receiver 630, a common pattern calculator 610, a common pattern database 620, an individual pattern calculator 640, and an individual pattern. A database 650, and a prediction unit 660.

Here, since the details described with reference to FIG. 2 may be applied to the receiver 630, the common pattern calculator 610, and the common pattern database 620, a detailed description thereof will be omitted.

In a training step of an individual pattern according to an embodiment of the present invention, the receiver 630 obtains a plurality of sensory data from individual users who are training targets of the individual patterns, and the individual pattern calculator 640 receives the plurality of sensory data. Calculate unique behavioral patterns of the individual users using sensory data. Like the common pattern calculator 610, the individual pattern calculator 640 according to an embodiment of the present invention may calculate a unique behavior pattern of the individual user by using the HTM model. In this case, the individual pattern database 650 stores unique behavior patterns of individual users calculated by the individual pattern calculator 640.

Meanwhile, in the inferring of individual patterns according to an embodiment of the present invention, the receiver 630 receives sensory data from at least one terminal, and the predictor 660 is the individual pattern database. 650 and the sensory data are used to predict the behavior of the individual user. Similarly, the prediction unit 660 according to an embodiment of the present invention may predict the behavior of the user by using the HTM model.

In addition, the prediction unit 660 according to an embodiment of the present invention predicts the user's behavior using the common pattern database 620 according to the reliability of the individual pattern database 650 or the individual pattern database. Using 650, it may be determined whether to predict the user's behavior.

For example, suppose that the service server 600 according to an embodiment of the present invention does not receive enough sensory data to establish an individual pattern database from a user of a terminal that has sent sensory data. In this case, the prediction unit 660 may determine that the reliability of the individual pattern database is low. If the prediction unit 660 determines that the reliability of the individual pattern database is low, the prediction unit 660 provides a service to the user by predicting the user's behavior using the common pattern database.

On the other hand, when the service server 600 according to an embodiment of the present invention receives the sensory data enough to build a separate pattern database from the user of the terminal transmitting the sensory data, the prediction unit 660 is the individual pattern It can be judged that the reliability of the database is high. In this case, the prediction unit 660 may provide a service related to the user's behavior corresponding to the unique behavior pattern of the individual user by predicting the user's behavior using the individual pattern database.

3. Terminal according to embodiments of the present invention

7 illustrates a terminal for generating sensory data using a GPS sensor and providing a service to a user based on a user's behavior predicted by a server using the sensory data according to an embodiment of the present invention. It is a block diagram.

Referring to FIG. 7, the terminal 700 according to an embodiment of the present invention includes a collector 710, a generator 720, a transmitter 730, a receiver 740, and an information provider 750. do.

The collector 710 collects information related to the behavior pattern of the user by using the plurality of sensors. Collecting unit 710 according to an embodiment of the present invention may include a GPS sensor for measuring GPS data at the location of the user.

In addition, the generation unit 720 generates sensory data using the information collected by the collection unit 710. As described above, sensory data encompasses all data capable of predicting a user's behavior pattern. The transmitter 730 transmits the sensory data to the server.

In addition, the receiving unit 740 receives service information for the user from the server. At this time, the server generates the service information by predicting the behavior of the user based on the common behavior pattern of the plurality of users and the sensory data. 1 to 6 may be applied to the server as it is, so a detailed description thereof will be omitted.

In addition, the information providing unit 750 automatically provides the service information to the user in response to the reception. In this case, the service information may include a plurality of location candidates recommended as the location to be moved by the user and a probability of each of the plurality of location candidates.

In this case, the information providing unit 750 may include an information display unit 753 displaying at least one of information about the plurality of position candidates or information about the probability of each of the plurality of position candidates. More specific details of the content displayed on the information display unit 753 will be described later with reference to FIG. 8.

In addition, the information providing unit 750 may further include an application determining unit 751 and an application executing unit 752.

The application determiner 751 may determine a plurality of applications corresponding to the plurality of location candidates. Referring to Table 1, in an embodiment of the present invention, a list of applications corresponding to locations of each of the plurality of location candidates may be constructed in a table form.

Location candidate's place Corresponding application library Auto manner mode APP Theater Movie Review APP, Movie Booking APP Mart Account book APP restaurant Menu Board APP, Restaurant Rating APP parking lot Empty parking space APP

In addition, the application execution unit 752 may automatically execute at least one of the plurality of applications based on the probability of each of the plurality of position candidates. For example, the application executor 752 according to an embodiment of the present invention may automatically execute an application corresponding to a location candidate most likely to be recommended as a place to which the user moves next.

8 is a diagram illustrating an information display unit that provides a service to a user based on a user's behavior predicted by a server using GPS data according to an embodiment of the present invention.

Referring to FIG. 8, the information display unit 753 may display information about the plurality of position candidates and information about the probability of each of the plurality of position candidates on the lower right side.

More specifically, the information display unit 753 may display a place 810 estimated as the current location of the user on a map. In this case, as described above, the terminal may receive information from the server that the current location of the user corresponds to any one of a plurality of predetermined location categories. For example, the place where the user is currently located in FIG. 8 may belong to a location category of high school.

The information display unit 753 according to an embodiment of the present invention may display the locations of each of the plurality of location candidates received from the server in a circle on the map (820, 830, and 840). At this time, the information display unit 753 according to an embodiment of the present invention may control the size of the circle displayed on the map according to the size of the probability of each of the plurality of location candidates.

For example, if the user's location is estimated to be high school, the service server according to an embodiment of the present invention is a location candidate recommended for the user to move to the next place, animal hospital (75% probability), bank ( 11% probability), and Oriental medicine doctor (8% probability). At this time, the information display unit 753 according to an embodiment of the present invention uses the largest circle when displaying the animal hospital 820 on the map, and the next size when displaying the bank 830 on the map. The circle of the circle is used, and the circle of the smallest size may be used when displaying the oriental medicine circle 840 on the map.

In addition, the information display unit 753 according to an embodiment of the present invention may directly recommend a next place to the user based on the probability of each of the plurality of location candidates. For example, the information display unit 753 may display a text message 850 on the upper left of the screen recommending that the user go to the animal hospital having the greatest probability.

The information providing unit 750 according to an embodiment of the present invention may provide new information to the user by reflecting the location of the user when the location of the user is estimated to be a new location different from the previously estimated location. That is, when the user's location is newly estimated to the supermarket, the information display unit 753 may update the user's current location 860 on the map.

Furthermore, when the current location of the user is changed, the list of location candidates recommended as the place to be moved next and the probability of each of the location candidates are changed, so that the information display unit 753 can provide the changed information to the user. have. For example, if it is estimated that the user's location has been moved to the supermarket, the service server according to an embodiment of the present invention selects the bank 870 as the location candidate most likely to recommend the user to the next location. This can be displayed in a circle on the map.

9 is a terminal for generating sensory data using at least a plurality of cameras and providing a service to a user based on a user's behavior predicted by a server using the sensory data according to an embodiment of the present invention. A block diagram is shown.

Referring to FIG. 9, the terminal 900 according to an embodiment of the present invention includes a collector 910, a generator 920, a transmitter 930, a receiver 940, and an information provider 950. do.

The collector 910 collects information related to a behavior pattern of a user by using a plurality of sensors.

Collection unit 910 according to an embodiment of the present invention may include a plurality of cameras for generating a 360-degree panoramic image at the location of the user.

10 is a diagram illustrating an arrangement of a plurality of cameras for generating a 360 degree panoramic image according to an embodiment of the present invention. Referring to FIG. 10, the collection unit 910 according to the exemplary embodiment of the present invention may include four cameras each having a viewing angle of 150 degrees. More specifically, the collection unit 910 according to an embodiment of the present invention is the first camera 1010 facing north, the second camera 1020 facing west, the third camera 1030 facing south centered around the user ), And a fourth camera 1040 facing east. In this case, the collection unit 910 simultaneously photographs the first camera 1010, the second camera 1020, the third camera 1030, and the fourth camera 1040, the 360-degree panoramic image around the user Can be generated.

In addition, the generator 920 generates sensory data using the information collected by the collector 910. As described above, sensory data encompasses all data capable of predicting a user's behavior pattern. The transmitter 930 transmits the sensory data to the server.

In addition, the receiving unit 940 receives service information for the user from the server. At this time, the server generates the service information by predicting the behavior of the user based on the common behavior pattern of the plurality of users and the sensory data. 1 to 6 may be applied to the server as it is, so a detailed description thereof will be omitted.

In addition, the information providing unit 950 automatically provides the service information to the user in response to the reception. Here, the service information may include a destination to which the user is currently moving and an action recommended to the user in relation to the destination.

In this case, the information providing unit 950 may include an information display unit 951 that displays at least one of information about the destination or information about the action.

FIG. 11 is a diagram illustrating an information display unit providing a service to a user based on a user's behavior predicted by a server using a 360-degree panoramic image according to an embodiment of the present invention.

Referring to FIG. 11, the terminal 900 according to an embodiment of the present invention may view a 360-degree panoramic image generated by using a plurality of cameras 911 while the user moves from the first place to the second place. You can send it to the server.

For example, a user is moving from the house 1130 to the gas station 1140, moving from the coffee shop 1120 to the house 1130, or moving from the gas station 1140 to the coffee shop 1120. You can send a 360-degree panoramic image to the server in the middle of a back movement.

As described above, the server may analyze the received panoramic image to predict the place where the user is currently moving. In the embodiment of FIG. 11, assume that user 1110 is moving from home 1130 to coffee shop 1120.

In this case, the information providing unit 950 may provide the user with information related to an action (purchasing coffee powder, etc.) recommended to the user in relation to the coffee shop and information indicating that the user is moving to the coffee shop from the server. .

Meanwhile, the collector 910 according to another embodiment of the present invention may further include a face detector 912 and a motion detector 913.

The face detector 912 may detect a frontal face included in the panoramic image. The face detector 912 according to an embodiment of the present invention may use an OpenCV library and a Region of Interest (ROI) technique.

In addition, the motion detector 913 may detect the motion of the moving object included in the panoramic image. The motion detector 913 according to an embodiment of the present invention may use a hardware capture technique to detect the motion of the moving object.

In this case, the service information received by the receiver 940 may include an action that is performed by the moving object and an action that is recommended as an action to be taken by the user in relation to the behavior of the moving object. In this case, the information providing unit 950 may include an information display unit 951 that displays at least one of information about the behavior of the moving object or information about the action.

12 is a view illustrating an input unit for receiving input of status information of a user according to an embodiment of the present invention.

Referring to FIG. 12, a terminal according to an embodiment of the present invention includes an input unit for receiving information regarding a state of a user.

The user may input information regarding the user's condition, such as a headache, a good skin condition, or an insomnia when the user feels a headache at any time.

For example, according to an embodiment of the present invention, the user may click the headache button 1210 included in the input unit at the time when the headache is felt. Thus, the terminal used by the user can transmit the corresponding information to the server.

Furthermore, the server may use the information to provide information about the physical state of the user, the mental state of the user, and provide an action recommended to the user in relation to the physical state and the mental state.

In addition, according to an embodiment of the present invention, a user may individually create a new button by describing the state of the user. Therefore, the input unit according to the embodiment of the present invention may include a button 1220 for generating the new button.

FIG. 13 is a block diagram illustrating a terminal for predicting a user's behavior using an individual pattern database of a user according to an embodiment of the present invention.

Referring to FIG. 13, the terminal 1300 according to an embodiment of the present invention includes a collector 1310, a generator 1320, a transmitter 1330, a receiver 1340, an individual pattern calculator 1350, and an individual unit. The pattern database 1360, the predictor 1370, the selector 1380, and the information provider 1390 are included.

Here, the details described with reference to FIGS. 7 and 9 may be applied to the collecting unit 1310, the generating unit 1320, the transmitting unit 1330, the receiving unit 1340, and the information providing unit 1390. Description is omitted.

In the training step of the individual pattern according to an embodiment of the present invention, the collection unit 1310 obtains information related to the unique behavior pattern of the individual user for training the individual pattern, the generation unit 1320 is A plurality of sensory data is generated using the information.

The individual pattern calculator 1350 calculates a unique behavior pattern of the individual user using the plurality of sensory data. In this case, the individual pattern calculator 1350 according to an embodiment of the present invention may calculate a unique behavior pattern of the individual user by using the HTM model. In this case, the individual pattern database 1360 stores unique behavior patterns of individual users calculated by the individual pattern calculator 1350.

Meanwhile, in the inferring of individual patterns according to an embodiment of the present invention, the collection unit 1310 obtains at least one information for predicting a user's behavior, and the generation unit 1320 obtains the information. At least one sensory data is generated.

The prediction unit 1370 predicts the behavior of the individual user by using the individual pattern database 1360 and the sensory data. Similarly, the prediction unit 1370 according to an embodiment of the present invention may predict the behavior of the user by using the HTM model.

In addition, the selector 1380 according to an embodiment of the present invention may generate a service based on the received service information and the behavior predicted by the predictor 1370 according to the reliability of the individual pattern database 1360. Any one of the information can be selected.

For example, suppose that the terminal 1300 according to an embodiment of the present invention does not secure enough sensory data to construct an individual pattern database. In this case, the selector 1380 may determine that the reliability of the individual pattern database 1360 is low. If the selector 1380 determines that the reliability of the individual pattern database 1360 is low, the selector 1380 selects the service information received from the server, and the information provider 1390 provides the received service information to the user.

On the other hand, when the terminal 1300 according to an embodiment of the present invention secures enough sensory data to build an individual pattern database, the selector 1380 may determine that the individual pattern database 1360 has high reliability. have. In this case, the selector 1380 may provide a service related to the user's behavior corresponding to the unique behavior pattern of the individual user by predicting the user's behavior using the individual pattern database 1360.

The above-described operating methods of the service server and the terminal may be implemented in the form of program instructions that may be executed by various computer means and may be recorded in a computer readable medium. The computer readable medium may include program instructions, data files, data structures, etc. alone or in combination. The program instructions recorded on the medium may be those specially designed and constructed for the present invention or may be available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tape, optical media such as CD-ROMs, DVDs, and magnetic disks, such as floppy disks. Magneto-optical media, and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine code generated by a compiler, but also high-level language code that can be executed by a computer using an interpreter or the like. The hardware device described above may be configured to operate as one or more software modules to perform the operations of the present invention, and vice versa.

As described above, the present invention has been described by way of limited embodiments and drawings, but the present invention is not limited to the above embodiments, and those skilled in the art to which the present invention pertains various modifications and variations from such descriptions. This is possible.

Therefore, the scope of the present invention should not be limited to the described embodiments, but should be determined by the equivalents of the claims, as well as the claims.

210: common pattern calculation unit
220: common pattern database
230: receiver
240:

Claims (20)

A common pattern database for storing common behavior patterns of a plurality of users;
A receiver configured to receive sensory data from at least one terminal; And
A prediction unit for predicting the behavior of the user using the common pattern database and the sensory data to generate service information to be provided to the user of the terminal.
Lt; / RTI >
The terminal
The service server collects information related to the behavior pattern of the user by using at least one sensor, and generates the sensory data using the information.
The method of claim 1,
A common pattern calculator for calculating the common behavior pattern
Further comprising:
The receiving unit
Obtaining a plurality of sensory data from the plurality of users,
The common pattern calculation unit
The service server calculates the common behavior pattern using the plurality of sensory data.
The method of claim 1,
The sensory data is
Including GPS data measured at the location of the user,
The prediction unit
A location estimator for estimating a current location of the user based on at least the GPS data;
A candidate selecting unit configured to select at least one location candidate recommended as the location to be moved next by the user by using the current location, the sensory data, and the common pattern database; And
A probability calculator configured to calculate a probability associated with an expected value of each of the at least one position candidate based on the common pattern database
/ RTI >
The method of claim 3,
The position estimating unit
Select a location category of any one of a plurality of predetermined location categories to estimate the location of the user;
The candidate selection unit
Selecting at least one location category of the plurality of predetermined location categories to select the at least one location candidate.
The method of claim 1,
The sensory data is
A 360-degree panorama image generated at the location of the user,
The prediction unit
An accumulator for accumulating the panoramic image; And
An image analyzer for analyzing the accumulated panoramic image to predict the behavior of the user
/ RTI >
The method of claim 5,
The image analysis unit
A destination predicting unit predicting a destination to which the user is currently moving based on the accumulated panorama image and the common pattern database; And
An action determination unit that determines at least one action recommended to the user with respect to the destination based on the common pattern database
/ RTI >
The method of claim 5,
The sensory data is
Further comprising information on the front of the face included in the panoramic image and information about the motion of the moving object included in the panoramic image,
The image analysis unit
A behavior recognition unit recognizing a behavior being performed by the moving object based on the accumulated panoramic image, information on the front face of the face, and information on the motion of the moving object; And
An action determination unit that determines at least one action recommended to the user based on the recognized behavior based on the recognized behavior and the common pattern database
/ RTI >
The method of claim 1,
The sensory data is
Includes information about the status of the user,
The prediction unit
A state predictor for predicting the physical state of the user and the mental state of the user by using the sensory data and the common pattern database; And
An action determination unit that determines at least one action recommended to the user in relation to the physical state and the mental state based on the physical state, the mental state, and the common pattern database
/ RTI >
The method according to any one of claims 1 to 8,
An individual pattern database that stores unique behavioral patterns of individual users; And
Individual pattern calculation unit for calculating the unique behavior pattern using the sensory data
Further comprising:
The prediction unit
A service server further utilizing the individual pattern database to predict the behavior of the user.
A collector configured to collect information related to a behavior pattern of a user using a plurality of sensors;
A generator configured to generate sensory data using the information;
A transmitter for transmitting the sensory data to a server;
A receiving unit for receiving service information for the user from the server; And
An information provider for automatically providing the service information in response to the reception
Lt; / RTI >
The server
The terminal generates the service information by predicting the behavior of the user based on the common behavior pattern of the plurality of users and the sensory data.
The method of claim 10,
The plurality of sensors
A GPS sensor measuring GPS data at the location of the user,
The sensory data is
At least the terminal including the GPS data.
The method of claim 11,
The service information is
A plurality of location candidates recommended as a location to be moved by the user and a probability of each of the plurality of location candidates,
The information providing unit
An information display unit displaying at least one of information about the plurality of position candidates or information about a probability of each of the plurality of position candidates
.
The method of claim 12,
The information providing unit
An application determiner configured to determine a plurality of applications corresponding to the plurality of location candidates; And
An application execution unit that automatically executes at least one of the plurality of applications based on the probability of each of the plurality of location candidates
Further comprising:
The method of claim 10,
The plurality of sensors
A plurality of cameras for generating a 360 degree panoramic image at the location of the user,
The sensory data is
Terminal including at least the panoramic image.
15. The method of claim 14,
The service information is
A destination to which the user is currently moving and an action recommended for the user in relation to the destination,
The information providing unit
Information display unit for displaying at least one of the information about the destination or the information about the action
.
15. The method of claim 14,
The plurality of sensors
A face detector detecting a frontal face included in the panoramic image; And
Motion detection unit for detecting the motion of the moving object included in the panoramic image
Further comprising:
The sensory data is
The terminal further includes information about the front of the face and information about the motion of the moving object.
17. The method of claim 16,
The service information is
An action recommended as an action to be taken by the user in relation to the action being performed by the mobile object and the behavior of the mobile object,
The information providing unit
Information display unit for displaying at least one of the information about the action of the moving object or the information about the action
.
The method of claim 10,
The plurality of sensors
Including an input unit for receiving information about the state of the user,
The sensory data is
At least a terminal comprising information about the status of the user.
19. The method of claim 18,
The service information is
A physical state of the user, a mental state of the user, and an action recommended to the user with respect to the physical state and the mental state,
The information providing unit
An information display unit displaying at least one of information on the physical state, information on the mental state, or information on the action
.
20. The method according to any one of claims 10 to 19,
An individual pattern database that stores unique behavioral patterns of individual users;
An individual pattern calculator configured to calculate the unique behavior pattern using the sensory data;
A predicting unit predicting the user's behavior using the sensory data and the individual pattern database; And
A selection unit for selecting one of the service information generated based on the received service information and the behavior predicted by the prediction unit according to the reliability of the individual pattern database;
Further comprising:

Images