WO2010004485A1

WO2010004485A1 - A system for managing measured sensor data of a user in accordance to predefined policy rules

Info

Publication number: WO2010004485A1
Application number: PCT/IB2009/052878
Authority: WO
Inventors: Robert P. Koster; Franciscus L. A. J. Kamperman
Original assignee: Koninklijke Philips Electronics N.V.
Priority date: 2008-07-09
Filing date: 2009-07-02
Publication date: 2010-01-14

Abstract

This invention relates to a system and a method for managing measured sensor data of a user in accordance to pre-defined policy rules. A policy rule definer is operates by a user for receiving policy related input data defining policy rules. These policy rules are associated to user identity data. A first sensor collects first sensor data of one or more users, and identity means provides user identity data identifying the one or more users being subject to the at least one first sensor. A processor determines whether the first sensor data match with the identity data. A policy engine enforces policy rules in accordance to the policy rules being associated to the matched user identity data.

Description

A system for managing measured sensor data of a user in accordance to predefined policy rules

FIELD OF THE INVENTION

The present invention relates to a system and a method for managing measured sensor data of a user in accordance to pre-defined policy rules.

BACKGROUND OF THE INVENTION

WO2006/031988 discloses a security system which is nonintrusive of personal privacy in a space. The system comprises a first localization sensor subsystem, in the possession of the person; a video surveillance subsystem arranged and configured to collect visual data related to the person in the space; and a computer subsystem coupled to the localization sensor subsystem. The system further comprises a video surveillance subsystem to associate a predetermined privacy level with the localization sensor subsystem, and to provide an access control privilege with the localization sensor subsystem. The computer subsystem determines how to present, store and/or retrieve the visual data while meeting the predetermined privacy level associated with the person. In this reference the localization is combined with policy rules to determine violations within a space. Thus, it is only when an unauthorized access is made to a particular space that the system reacts. As an example, a first set of rules is associated to an employ for one particular space (e.g. canteen, this may be an unlimited access) but another set of much more restricted rules is associated to the same employee for another space (e.g. the office of the CEO where the access is forbidden).

Although WO2006/031988 is a considerable improvement over prior art surveillance systems, this system it is limited to surveillance systems where the policy rules are defined at management interface (centralized surveillance manager) and the sensed person has no influence.

BRIEF DESCRIPTION OF THE INVENTION

The object of the present invention is to overcome the above mentioned drawbacks by providing a system that is focused on the consumer, e.g. at home, public spaces where users come and go. According to one aspect the present invention relates to a system for managing measured sensor data of a user in accordance to pre-defined policy rules, comprising: a policy rule definer adapted to be operated by a user for receiving policy related input data defining policy rules and associating the policy rules to user identity data, - at least one first sensor adapted to collect first sensor data of one or more users, identity means for providing user identity data identifying the one or more users being subject to the at least one first sensor, a processor adapted to determine whether the first sensor data match with the identity data, and a policy engine adapted to enforce policy rules on the first sensor data in accordance to the policy rules being associated to the matched user identity data.

Thus, a very user friendly system is provided allowing users to set their own policy rules, and these are then applied or looked-up as soon as a user becomes known in an environment. Also, this system has the capability of identifying a user among plurality of users thus allowing enforcing different policy rules for the different users. An example of implementation is at somebody's home where there are a number of devices and sensors, where the user of the system wants to give a bit of his identity information but at the same time control his privacy. The first case ("guest usage at home") is when the user uses a device for the first time in the home, e.g. a guest user like a neighbor or a friend. For such guests it may be desired to bring their own policies to control their privacy in the environment that is full with devices and sensors that observe the people present. Such policy could for example be that their presence or emotional feedback may (not) be shared with services. In a second case ("register to new device") the user buys a new device that must learn the identities and policies of its users (and for some reason cannot learn it directly from other devices in the home). For the rest, this case is similar to the guest user case.

Another example of implementation is at public spaces ("introduce at public space"), e.g. a town hall, the office, hotel lobby, etc. A very simple application of sensors in such environment could be to report presence in some form or to do some kind of personalization e.g. in advertising. The rest of the application would work quite similar to the case above.

The cases above have in common that the environment typically cannot or should not be the party that defines the privacy policy. In many cases people would refuse to release their identity data at all. In one embodiment, the identity means comprises: at least a second sensor adapted to be carried by the one or more users for collecting at least a second set of sensor data relating to the user carrying the at least second sensor, - a user identity module for providing identity data identifying the user carrying the at least second sensor, wherein determining whether the first sensor data match with the identity data comprises correlating the first sensor data with the at least second sensor data, wherein in case the correlation fulfils a pre-defined criteria the first sensor data is associated to the user identity data.

In one embodiment, the user identity module is a token and wherein the at least second sensor is embedded therein.

Thus, using such a token the user voluntarily makes his identity available along with the associated policy rules. Using the token in a combination with the second sensor provides a reliable way of identifying a user among plurality of users is provided. Thus, the identity means becomes very compact and easy to carry. In one embodiment, correlating the first sensor data with the at least second sensor data includes determining a correlation coefficient, the fulfillment of the pre-defined criteria being based on whether the determined correlation coefficient is above a pre-defined threshold value.

Thus, the correlation criteria can easily be adjusted by re-defining the threshold value.

In one embodiment, the at least second sensor is a movement sensor and the at least second set of sensor data is a second movement vector derived from the second set of sensor data, the first sensor including a movement detection means for detecting movement of the one or more users resulting in a first movement vector.

In one embodiment, the system further comprises a biometric means for collecting biometric data related to the user for identifying the user carrying the at least second sensor. Thus, a link is provided between the identity data which identifies the user and some biometric which are characteristic for the user. Thus, for all subsequent identifications it is sufficient to rely on the biometrics. The biometric data may be obtained from the first set of sensor data, or via an additional device, or via the second set of sensor data. In one embodiment, the biometric means is a face recognition means which determines a biometric profile of the face of the user carrying the at least second sensor.

According to another aspect, the present invention relates to a method of managing measured sensor data of a user in accordance to pre-defined policy rules, comprising: receiving a user input indicating policy related input data defining policy rules and associating the policy rules to user identity data, collecting first sensor data of one or more users using at least one first sensor, providing user identity data identifying the one or more users being subject to the at least one first sensor, determining whether the first sensor data match with the user identity data, and enforcing policy rules on the first sensor data in accordance to the policy rules being associated to the matched user identity data.

In one embodiment, the step of providing user identity data identifying the one or more users comprises: collecting at least a second set of sensor data relating to the user by using at least a second sensor carried by the one or more users, providing identity data identifying the user carrying the at least second sensor, wherein determining whether the first sensor data matches with the identity data comprises correlating the first sensor data with the at least second sensor data, wherein in case the correlation fulfils a pre-defined criteria the first sensor data is associated to the user identity data.

In one embodiment, the method further comprises collecting biometric data related to the user for providing further identification identifying the user carrying the at least second sensor.

Thus, the biometric data can be linked directly to the user identity meaning that e.g. user having ID 124 has biometric data X.

In one embodiment, the biometric data related to the user is used for subsequent identification of the user. Thus, after a user has been reliably identified for a first time and since the biometrics are linked to the user or the user identity data all subsequent identifications of this user may be done via the biometric data. Therefore, instead of e.g. using a token with second sensor it is now possible to use biometrics to identify a user in a group of users and use the identity data associated to it during the first time identification, including one or more user identifiers and associated user-defined policies. Accordingly, recognizing a user via biometrics may be done using e.g. sensor data from the first sensor ("user X with the particular facial expression Y"). Based on this recognized user the device (with e.g. the first sensor) can look up the associated data it learned earlier (identity data, policy) from the token. Thus, no there's no correlation required anymore for the subsequent identifications. The biometric data may be a raw measurement, e.g. picture of a face, a processed biometric in the form of specific features, or even a unique digital representation as known from template protection.

According to another aspect, the present invention relates to a computer program product for instructing a processing unit to execute the above mentioned method steps when the product is run on a computer.

The aspects of the present invention may each be combined with any of the other aspects. These and other aspects of the invention will be apparent from and elucidated with reference to the embodiments described hereinafter.

BRIEF DESCRIPTION OF THE DRAWINGS

Embodiments of the invention will be described, by way of example only, with reference to the drawings, in which

Fig. 1 shows a system according to the present invention, Fig. 2 shows a flowchart of a method according to the present invention.

DESCRIPTION OF EMBODIMENTS

Figure 1 shows a system 100 according to the present invention for managing measured sensor data of a user in accordance to pre-defined policy rules. The system 100 comprises a policy rule definer (P R D) 101, at least one first sensor (Se I) 103, identity means (I M) 105 and a processor (P) 104.

The policy rule definer (P R D) 101 is adapted to be operated by a user 107 for receiving policy related input data defining policy rules and associating the policy rules to user identity data. An example of policy rules is following: "data may be shared with host X", "data may be shared maximally Y times", "data may be stored for TIME", "data fields are filtered according to a FILTER", "data fields+values are filtered according to FILTER: detailed level; features; values, e.g. not report of "stress"", "use of data must be reported to USER on URL", "a (carbon) copy of acquired sensor data must also be added to PROFILE of USER at URL/ADDRESS". Thus, the user 107 manages the policy rules by defining them and associating them one or more users.

The policy rule definer may also be used to remove existing policy rules or update existing policy rules. The at least one first sensor (Se I) 103 is adapted to collect first sensor data of one or more users, but the first sensor (Se I) 103 may as an example be a web camera, a digital camera, an infra-red sensor and the like.

The identity means (I M) 105 provides user identity data identifying the one or more users being subject to the at least one first sensor. The user identity data can e.g. comprise user identity number, any type of an identifier or any other information attribute belonging the user. The processor (P) 104 determines whether the first sensor data matches with the identity data, and the policy engine (P E) 102 enforces policy rules in accordance to the policy rules being associated to the matched user identity data.

In one embodiment, the identity means comprises a second sensor and a user identity module. The second sensor is carried by the one or more users and collects a second set of sensor data relating to the user carrying the at least second sensor. In one embodiment, this second sensor is an accelerometer and the second set of sensor data is a second acceleration vector. In this embodiment, the user identity module is a token which transmits a user ID identifying the user carrying the token along with the second set of sensor data. In this embodiment, the first sensor (e.g. a camera) is provided with an acceleration tracking module or similar means which also determines a first acceleration vector for a user carrying the token and the second sensor. The determining of whether the first sensor data match with the identity data comprises then correlating the first acceleration vector with the second acceleration vector. Thus, if the first and the second acceleration vectors match with each other, or fulfils a pre-defined criteria the first sensor data is associated to the user identity data, i.e. the user ID identifying the user. Such pre-defined criteria may be based on determining a correlation coefficient, where the fulfillment of the pre-defined criteria is based on whether the determined correlation coefficient is above a pre-defined threshold value.

In one embodiment, the above mentioned policy rules are stored on the token and transferred to the first sensor. Data acquired by the first sensor it is associated to the user and thereby also to the policy. This policy is then enforced. In addition the policy may be attached to the sensor data. Such "sticky policies" may be either a verbatim copy of the policies supplied by the identity means (I M) 105 (which will be discussed in more details later) to the at least one first sensor (Se I) 103, or be specified separately (possibly as part of the general policy).

In another embodiment, the user-defined policy is stored in a database. Thus, the policy rules are retrieved using the user identifier when the policy must be enforced on the sensor data.

In one embodiment, the system 100 further comprises a biometric means (B M) 106 for collecting biometric data related to the user for identifying the user carrying the at least second sensor. This biometric means is in one embodiment a face recognition means which determines a facial expression profile of the user carrying the at least second sensor or biometric profile. These additional data are linked to the user identity data and are adapted to be used for subsequent identification of the user. This will be discussed in more details later.

Figure 2 shows a flowchart of a according to the present invention of managing measured sensor data of a user in accordance to pre-defined policy rules. In step (Sl) 201, a user input is received indicating policy related input data defining policy rules and associating the policy rules to user identity data. Thus, the user or operator of the system 100 may manually enter policy rules via e.g. keyboard commands and associated the various policy rules to different user identities. As discussed previously,

In step (S2) 203, a first sensor data of one or more users using at least one first sensor is collected.

In step (S3) 205, a user identity data is provided identifying the one or more users being subject to the at least one first sensor.

In step (S4) 207, it is determined whether the first sensor data match with the identity data. In step (S5) 209, policy rules are enforced in accordance to the policy rules being associated to the matched user identity data.

In one embodiment, the step of providing user identity data (S3) 205 comprises collecting at least a second set of sensor data relating to the user by using at least a second sensor carried by the one or more users, and providing identity data identifying the user carrying the at least second sensor. The determining of whether the first sensor data match with the identity data comprises correlating the first sensor data with the at least second sensor data, wherein in case the correlation fulfils a pre-defined criteria the first sensor data is associated to the user identity data. In one embodiment, the method further comprises (S6) 211 collecting biometric data related to the user for providing further identification identifying the user carrying the at least second sensor. The biometric data may be based on some characteristic features obtained from the second set of sensor data, or this may be e.g. based on face recognition, or other features that characterize a user. Thus, these features are then associated to the user identity data.

The above mentioned steps are characteristic for a first time recognition (registration). After having linked the biometric data to the user identity, all subsequent identifications of the users may be done via the biometric data (S7) 213. This means that the identification process via e.g. the correlation is only required once, namely when identifying a user for a first time. After that, the identification is based on the biometric data.

The following two embodiments show in further details the embodiments of using a token and the biometric means.

Embodiment 1 :

This embodiment realizes the invention with the following specifics: Linking sensor data to user via user identity device and policy transfer via user identity device. It starts with registering a user with a sensor followed by actually using the sensors. This is reflected in two protocols. It is assumed that hosts and sensors already have a relationship, e.g. through an appropriate registration or subscription protocol.

Protocols:

Registering a user with a sensor: 0. first sensor: start first sensor data acquisition 1. first sensor->user ID device (token): HELLO = { sensor id, capabilities, ... }

2. user->user ID device: press button to initiate registration with sensor

3. user ID device (token)->first sensor: I AM = { transaction id, user id device id, user id, policy, second sensor data }

4. first sensor: match the first and second sensor data; in case of a positive match continue with the next protocol step

5. first sensor->user ID device (token): CONFIRM = { transaction id }

Taking measurements and forwarding these to a host:

6. user->first sensor: acquire first sensor data relating to user 7. user ID device (token)->fϊrst sensor: PRESENT = { user id device id [, second sensor data ]}

8. first sensor: optionally match first and second sensor data; associate first sensor data to user id via user id device id 9. enforce policy, i.e. determine if acquisition is allowed, which filters must be applied, etc.

10. process and cache sensor data

11. sensor->host: DATA = { sensor id, user id, data [, policy] } if allowed by policy

This method is advantageous for both: the host because he gets a real user id that a user voluntarily makes available, and for the user because while disclosing his identity he can also set a policy. Note that the user id may be a pseudonym.

Associate sensor data to a user:

Note that steps 6-8 are preferably performed close in time (exact time window depends on application). Note further that depending on the possibilities that associating sensor data to a user in step 8 may be best effort or even include multiple possibilities, e.g. because multiple user_id_devices reported their presence using message 3. The message in step 11 then includes an array of user ids instead of a single one, with optionally chances. The sensor may use additional information to make the best association. A technical enhancement to make this association is, as discussed previously, to embed also a sensor in the token and correlate the sensed data with the sensed data from the sensor in the environment. This is reflected by the aspects marked optional (through "[...]" or "optionally") in steps 7 and 8, which reflects a similar functionality as steps 0, 3 and 4. A high correlation enables identification of the proper user in case of multiple candidates, e.g. an accelerometer based movement sensor embedded in the token and a webcam with movement detection algorithm in the environment sensor. Both methods create an array of movement vectors, which can be matched.

Step 10 optionally includes (part of) a policy with the sensor data. This represents a sticky policies concept. The host will enforce these policies while accessing, using and otherwise handling the sensor data.

Embodiment 2:

This embodiment realizes the invention with the following specifics: linking sensor data to user via biometrics, and policy transfer via network discovery or optionally a combination with manual entry. It starts with registering a user with a sensor followed by actually using the sensors. This is reflected in two protocols. It is assumes that hosts and sensors already have a relationship, e.g. through an appropriate registration or subscription protocol.

Protocols:

Registering a user with a sensor:

1. user->fϊrst sensor: initiate registration with sensor, e.g. through button, gesture, etc. 2. user ->fϊrst sensor: have bio metric taken

3. first sensor-> networked hosts (broadcast): DISCOVER = { transaction id, biometric }

4. policy/identity server -> sensor: POLICY = { transaction id, user id, policy }

It should be noted that it might be preferred that the user actually consents to the registration and that it is not done by somebody else while the user is in the neighborhood. One way to do this is by having the user to respond to a challenge, e.g. where he must respond with a gesture.

Alternative to, or in addition to, step 2 the user could enter his user id manually. This can then be used in step 3-4 to obtain the biometric and policy. Step 3 represents a discovery, e.g. in a home network (compare e.g. UPnP,

DHCP, etc.). Alternatively, a lookup is done at certain servers at the Internet where people may register their privacy policies.

Alternatively, note that the above protocol to register a user with a first sensor may be replaced by the protocol of embodiment 1 augmented by the first sensor taking the biometrics of the user reflected by step 2 above. This biometric measurement is then associated to the user id obtained in step 3 of the first embodiment. The remainder, i.e. the protocol below, would remain unchanged in this alternative.

Taking measurements and forwarding these to a host: 5. user->first sensor: acquire sensor data relating to user

6. user->first sensor: acquire biometric or determine biometric features from sensor data

7. first sensor: associate sensor data to user id via biometric

8. enforce policy, i.e. determine if acquisition is allowed, which filters must be applied, etc.

9. process and cache sensor data 10. first sensor->host: DATA = { sensor id, user id, data [, policy] } if allowed by policy

Note that steps 5-10 do not involve a token, but just biometrics to determine the identity of the user. As mentioned previously, the biometric may be a raw measurement, e.g. picture of the face, a processed biometric in the form of specific features, or even a unique digital representation as known from template protection.

Biometric templates: Biometric templates may be beneficial, because they protect a users privacy

(for its biometrics) and because it allows fast lookups using the biometric template as an index.

In order for biometric templates to function it is required to have so-called helper data: raw biometric measurement + helper data -> biometric template. Suppose a sensor determines the raw biometric measurements. To determine the biometric template the sensor needs to acquire the helper data, which may be obtained from a token.

Alternatively, there is no token with helper data, but the helper data is stored in an identity/policy server. In this case the user inputs an identifier, which is used to retrieve the helper data (and possibly the policy when combined with the next step) belonging to this user at a database with the identifier as index. As a result the sensor has knowledge of the helper data.

Subsequently, the biometric template can be used in the policy lookup process.

The biometric template here serves as an index or identifier resulting in an efficient lookup in a policy database with biometric index. Now that the biometrics and policies are known the sensor goes to normal operation sensing data. It uses the obtained biometric helper data to do efficient biometric matches (without further interaction with a token or user) when measuring data to associate the data to a user. This can be done efficiently, because a sensor at most only knows a few users. The previous embodiments have illustrated that a sensor can do sensing/measuring of data which in some cases can be used for biometrics and sometimes not. Similarly, identification can be done using biometrics, using the measurement data or an independent biometric measurement, or a token. Certain specific details of the disclosed embodiment are set forth for purposes of explanation rather than limitation, so as to provide a clear and thorough understanding of the present invention. However, it should be understood by those skilled in this art, that the present invention might be practiced in other embodiments that do not conform exactly to the details set forth herein, without departing significantly from the spirit and scope of this disclosure. Further, in this context, and for the purposes of brevity and clarity, detailed descriptions of well-known apparatuses, circuits and methodologies have been omitted so as to avoid unnecessary detail and possible confusion.

Reference signs are included in the claims, however the inclusion of the reference signs is only for clarity reasons and should not be construed as limiting the scope of the claims.

Claims

CLAIMS:

1. A system (100) for managing measured sensor data of a user in accordance to pre-defined policy rules, comprising: a policy rule definer (101) adapted to be operated by a user ( 107) for receiving policy related input data defining policy rules and associating the policy rules to user identity data, at least one first sensor (103) adapted to collect first sensor data of one or more users, identity means (105) for providing user identity data identifying the one or more users being subject to the at least one first sensor, - a processor (104) adapted to determine whether the first sensor data match with the identity data, and a policy engine (102) adapted to enforce policy rules on the first sensor data in accordance to the policy rules being associated to the matched user identity data.

2. A system according to claim 1, wherein the identity means (105) comprises: at least a second sensor adapted to be carried by the one or more users for collecting at least a second set of sensor data relating to the user carrying the at least second sensor, a user identity module for providing identity data identifying the user carrying the at least second sensor, wherein determining whether the first sensor data match with the identity data comprises correlating the first sensor data with the at least second sensor data, wherein in case the correlation fulfils a pre-defined criteria the first sensor data is associated to the user identity data.

3. A system according to claim 2, wherein the user identity module is a token and wherein the at least second sensor is embedded therein.

4. A system according to claim 2, wherein correlating the first sensor data with the at least second sensor data includes determining a correlation coefficient, the fulfillment of the pre-defined criteria being based on whether the determined correlation coefficient is above a pre-defined threshold value.

5. A system according to claim 2, wherein the at least second sensor is a movement sensor and the at least second set of sensor data is a second movement vector derived from the second set of sensor data, the first sensor including a movement detection means for detecting movement of the one or more users resulting in a first movement vector.

6. A system according to claim 2, wherein the movement sensor is an accelerometer.

7. A system according to claim 1 or 3, further comprising a bio metric means (106) for collecting bio metric data related to the user for identifying the user carrying the at least second sensor.

8. A system according to claim 1 or 3, wherein the biometric means (106) is a face recognition means which determines a biometric profile of the face of the user carrying the at least second sensor.

9. A method of managing measured sensor data of a user in accordance to predefined policy rules, comprising: receiving a user input (201) indicating policy related input data defining policy rules and associating the policy rules to user identity data, collecting first sensor data (203) of one or more users using at least one first sensor, providing user identity data (205) identifying the one or more users being subject to the at least one first sensor, - determining whether the first sensor data match with the identity data (207), and enforcing policy rules (209) on the first sensor data in accordance to the policy rules being associated to the matched user identity data.

10. A method according to claim 1, wherein providing user identity data identifying the one or more users comprises: collecting at least a second set of sensor data relating to the user by using at least a second sensor carried by the one or more users, - providing identity data identifying the user carrying the at least second sensor, wherein determining whether the first sensor data match with the identity data comprises correlating the first sensor data with the at least second sensor data, wherein in case the correlation fulfils a pre-defined criteria the first sensor data is associated to the user identity data.

11. A method according to claim 10, further comprising collecting biometric data related to the user (211) for providing further identification identifying the user carrying the at least second sensor.

12. A method according to claim 11, wherein the data related to the user is used for subsequent identification of the user (213).

13. A computer program product for instructing a processing unit to execute the method step of claim 9 when the product is run on a computer.