FR3116927A1 - Method for biometric classification of a user of a terminal - Google Patents

Abstract

Method for the biometric classification of a user of a terminal connected to a first server The present invention relates to a method for the biometric classification of a user of a terminal (1) connected to a first server (2a), the method being characterized in that it comprises the implementation by a data processing module (21a) of the first server (2a), of steps of: Obtaining a candidate vector of temporal sequences of weight values measured when the user sits on a seat (3) associated with said terminal (1) and equipped with a plurality of weight sensors (4a, 4b, 4c, 4d); biometric classification of said user according to said candidate vector of time sequences of weight values, by means of a classification model trained on a basis of learning of reference vectors of time sequence of weight values. Figure for abstract: Fig. 1a

Description

Method for biometric classification of a user of a terminal

GENERAL TECHNICAL AREA

The present invention relates to a method for biometric classification (such as authentication/identification) of a user of a terminal, in particular for personalized use of the terminal.

STATE OF THE ART

The "smart home" or intelligent habitat designates all the techniques allowing automated management of a domestic environment, in particular the functionalities linked to comfort (energy management, optimization of lighting and heating) and security ( alarm).

Smart home applications are scenarios in which domestic devices with a certain autonomy, called "smart devices", interact and communicate with each other to achieve a specific objective that suits the inhabitants of the house.

Some smart devices are configured to operate automatically depending on the circumstances. For example, we are familiar with programmable heating designed to maintain a predetermined thermal profile, or smart lighting which is configured to increase or decrease its brightness depending on the amount of sunlight outside.

On the other hand, some smart devices still need to be manually commanded to start, operate or change their state. An example of this type of device is the smart TV, which must be turned on before manually selecting the channel of your choice. These smart TVs are now more “connected” objects than “smart” objects.

A difficulty, if one wishes to improve the automation of such equipment, lies in the identification of their user. For example, a smart TV could work more autonomously if it were able to identify the person who wants to watch it, for example by offering a favorite channel, an appropriate automatic volume, etc.

A first solution for this is to use biometric techniques, for example by facial or voice recognition. These methods are complex and above all intrusive from a privacy point of view.

Alternatively, non-biometric approaches have been proposed, recognizing for example a person's gait, see the document Francisco Manuel Castro, Manuel J. Marın-Jimenez, Nicolas Guil, and Nicolas Perez de la Blanca. Automatic learning of gait signatures for people identification. In Ignacio Rojas, Gonzalo Joya, and Andreu Catala, editors, Advances in Computational Intelligence, pages 257–270, Cham, 2017. Springer International Publishing. ISBN 978-3-319-59147-6 .

In addition to the fact that these approaches still require expensive equipment (such as a 3D camera), they are less robust in practice. For example, the gait recognition algorithm can be affected by some loose clothing.

It would therefore be desirable to have a mechanism for authenticating a user of a terminal which is simple, reliable, robust, not very intrusive, and does not require any expensive hardware.

PRESENTATION OF THE INVENTION

The present invention thus relates according to a first aspect to a method of biometric classification of a user of a terminal connected to a first server, the method being characterized in that it comprises the implementation by a processing module of data from the first server, steps from:

1. Obtaining a candidate vector of time sequences of weight values measured when the user sits on a seat associated with said terminal and equipped with a plurality of weight sensors;
2. Biometric classification of said user according to said candidate vector of temporal sequences of weight values, by means of a classification model trained on a basis of learning of reference vectors of temporal sequence of weight values.

According to advantageous and non-limiting characteristics:

The method further comprises a step (c) of controlling said terminal according to the result of the biometric classification of said user.

Said classification model is a neural network.

Said neural network is of the recurrent network type with short and long term memory, LSTM.

Each time sequence reference vector of said learning base is associated with a user among a plurality of possible users of said terminal.

The seat having several places, said learning base comprises for each user of said plurality of possible users of said terminal at least one reference vector of time sequences for each place of said seat.

The method comprises a prior step (a0) of learning by a data processing module of a second server (possibly connected to the first server) of said classification model from said learning base of time sequence reference vectors of weight values.

Said seat is equipped with at least four weight sensors arranged at the four corners.

Step (a) comprises the prior detection that a person is sitting on the seat, each time sequence of said candidate vector comprising a predetermined number of successive weight values measured by one of said weight sensors from said detection.

Step (a) comprises the continuous obtaining of successive weight values measured at regular intervals by each weight sensor.

The prior detection that a person is sitting on the seat comprises the calculation of a Z dimension in a sliding window on said time sequences of weight values, and the comparison of the Z dimension with a predefined threshold.

According to a second aspect, the invention relates to a server for the biometric classification of a user of a terminal connected to said server, characterized in that it comprises a data processing module configured to implement steps of:

1. Obtaining a candidate vector of time sequences of weight values measured when the user sits on a seat associated with said terminal and equipped with a plurality of weight sensors;
2. Biometric classification of said user according to said candidate vector of temporal sequences of weight values, by means of a classification model trained on a basis of learning of reference vectors of temporal sequence of weight values.

According to a third aspect, the invention relates to a system comprising a first server according to the second aspect, at least one terminal and a seat associated with said terminal and equipped with a plurality of weight sensors.

According to advantageous and non-limiting characteristics, the system further comprises a second server for learning said classification model from said learning base of reference vectors of time sequence of weight values.

According to a fourth and a fifth aspect, the invention relates to a computer program product comprising code instructions for the execution of a method according to the first aspect of biometric classification of a user of a terminal connected to a first server ; and a storage means readable by computer equipment on which a computer program product comprises code instructions for the execution of a method according to the first aspect of biometric classification of a user of a terminal connected to a first server.

PRESENTATION OF FIGURES

Other characteristics and advantages of the present invention will appear on reading the following description of a preferred embodiment. This description will be given with reference to the appended drawings in which:

• [Fig.1a] FIG. 1a is a diagram of an example of architecture for implementing the method according to the invention;
• [Fig.1b] Figure 1b is a diagram of an example of a seat and an associated terminal used during the implementation of the method according to the invention;
• [Fig.2] Figure 2 illustrates the steps of a preferred embodiment of the method according to the invention;
• [Fig.3] FIG. 3 represents an example of a vector of time sequences of weight values measured when the user is sitting down;
• [Fig.4a] . [Fig.4b] Figures 4a and 4b show an example of segmentation of a time sequence of weight values to detect that a person sits on the seat.

DETAILED DESCRIPTION

Architecture

The present method is a method of biometric classification of a user of a terminal 1 as represented on the .

The latter can be any equipment, in particular a connected television, a digital decoder, a mobile terminal, such as a smartphone, touch pad, etc. having a data processing module 11 (typically a processor), a data storage module 12 (a memory, for example Flash).

The terminal 1 is connected to a first server 2a, in particular via a local network 10. Typically the server 2a is a gateway of a local home network (advantageously a Wi-Fi network), itself connected to an extended network 20 such as the internet. Alternatively, the terminal 1 and the first server 2a can both be in the extended network 20, or even be confused. Advantageously, there is a second server 2b (which is a learning device as we will see), typically remote (i.e. in the network 20), but which can be confused with the first server 2a.

Each server 2a, 2b also has a data processing module 21a, 21b (typically a processor) and a data storage module 22a, 22b (a memory, for example a hard disk). As will be seen, the data processing module 22b of the second server 2b can store a learning database. For the sake of simplification, the learning database is referred to as the “learning base” in the remainder of this description.

Originally, there is also a seat 3 associated with said terminal 1 and equipped with a plurality of weight sensors 4a, 4b, 4c, 4d. By seat is meant any piece of furniture or any other object arranged so that one can sit on it, in the remainder of this description the example of a sofa or sofa will be taken, but alternatively the seat 3 may be an armchair, a chair, a bench, a stool, etc.

Seat 3 is said to be "associated" with said terminal 1, which just means that there is a link between the two, in particular terminal 1 can be arranged in such a way that you sit on seat 3 to use it , for example if the terminal 1 is a television or a game console opposite the seat 3 as represented on the , or else a lighting, heating or sound system for the room in which the seat 3 is located. The idea is to be able to assume with a high probability that a command from the terminal 1 emanates from a person seated on the seat 3. It should be noted that the terminal 1 can even be integrated into the seat 3 because “smart sofas” with control interfaces, for example lights or ambient music, are known.

Furthermore, the seat 3 is equipped with a plurality of weight sensors 4a, 4b, 4c, 4d, at least two and advantageously at least four, preferably arranged at the four corners (typically in each of the four legs), such as the we see on the : a front left sensor 4a, a front right sensor 4b, a rear left sensor 4c and a rear right sensor 4d. There can be more, for example six if in addition to the four angles there is a foot in the middle in front and a foot in the middle behind. By “weight sensor”, is meant in practice a balance, capable of measuring over time a weight value (ie a force) applied to the sensor, which can be expressed as a mass (by dividing by g). With a plurality of sensors 4a, 4b, 4c, 4d it is therefore possible to have a plurality of temporal sequences of weight values.

The present method aims at the biometric classification of a user of the terminal 1 (in particular his identification or his authentication) thanks to the measurements of the sensors 4a, 4b, 4c, 4d, alternatively for example to known biometric techniques such as the recognition of the face or posture. Said biometric classification is advantageously chosen from an identification or authentication of the user.

The method can be implemented at any time when the identity of the person handling the terminal 1 may need to be determined (identification – to determine and apply usage preferences such as the selection of a favorite TV channel, usual brightness, comfort temperature, etc.) or verified (authentication - attempt to access personal data, parental control, etc.). In the rest of this description, we will take the example in which the terminal is a connected television opposite the seat 3, and thus we can apply a favorite channel, a default sound volume, etc.

In other words, we understand that we use the way of sitting of a user as a biometric signature. Indeed the weight values vary when a person stands up or sits down, and in a way that varies from one user to another. We all have different ways of sitting.

Process

With reference to the , the method is implemented by the data processing module 21a of the first server 2a, and begins with a step (a) of obtaining a candidate vector of temporal sequences of weight values measured when the user sits on the seat 3 associated with the terminal 1 (which, it is recalled, is equipped with a plurality of weight sensors 4a, 4b, 4c, 4d). In practice, the vector can be generated by the first server 2a, or directly received if, for example, the sensors 4a, 4b, 4c, 4d have intelligence (the seat 3 can carry an electronic card).

In particular, each sensor 4a, 4b, 4c, 4d is advantageously an independent connected object (for example connected to the home network if the server 2a is a gateway), which can publish a message at regular intervals, for example using the MQTT protocol (Message Queuing Telemetry Transport), containing a batch of consecutive measurements. The message can directly include other data such as a timestamp (of the first value in the batch), an average, a standard deviation, etc. Alternatively, each sensor 4a, 4b, 4c, 4d can send raw data back. In all cases, it is possible, for example, to have ten weight measurements per second, this frequency possibly being higher, in particular if there is a large number of possible users.

The first server 1a then reconstitutes said temporal sequences of weight values, and forms the vector by synchronizing them. There are indeed a plurality of sensors 4a, 4b, 4c, 4d, and therefore a plurality of synchronized sequences in the vector (typically four).

We see on the an example of four time sequences of weight values. In order we have: front left sensor, front right sensor, rear left sensor and rear right sensor. We see that temporally the left front sensor 4a is the first mobilized, then the left rear sensor 4c and finally the two right sensors 4b, 4d. In intensity, it is the front right sensor 4b which records the strongest peak.

Preferably, step (a) comprises the prior detection that a person is sitting on the seat 3, each time sequence of said candidate vector comprising a predetermined number of successive weight values measured by one of said weight sensors 4a, 4b, 4c, 4d from said detection (and then the continuous obtaining of successive weight values measured at regular intervals by each weight sensor 4a, 4b, 4c, 4d). Indeed over the course of a day, users only sit down or stand up a few times, which means that there are only a few seconds of really useful measurements.

Thus, in the preferred embodiment, we always continue to measure the weights "permanently" (ie to loop back weight values), but the first server 2a only generates a vector when it has been detected that a person sits down. This can be implemented in many different ways, in particular by a segmentation algorithm applied to the plurality of sequences adapted to detect strong variations in values, see for example the And .

The algorithm can be implemented on a sliding window, by calculating a Z rating or Z-score in Anglo-Saxon terminology (or reduced centered variable), i.e. the difference of a value with the mean on the window, divided by the standard deviation. If this Z-score is above a predetermined threshold (for example 3), the value is considered an "anomaly" (outlier), i.e. an event has just occurred on sofa 3 (a person has just sat down or stood up). One can easily determine if the person sat down or stood up by looking at the direction of variation of the temporal sequence: if the weight increases it is that a person has sat down, and if the weight has dropped c is that a person stood up. We are interested in the first case.

It is then possible to generate a candidate vector comprising for each sequence a predetermined number of successive weight values measured by one of said weight sensors 4a, 4b, 4c, 4d from said detection, said number being a few tens, for example twenty values . If we have four sequences, we then have a vector of 4 times 20 consecutive values, i.e. 80 values in total (i.e. object of dimension 80).

In a next step (b), the processing module 21a classifies said user as a function of said candidate vector of temporal sequences of weight values, by means of a classification model trained on a learning basis of sequence reference vectors of weight values. The model takes said candidate vector as input and returns a relative identifier of the user, which can be either an identifier of a user identity (among several possible identities) or directly a boolean indicating whether the user is a user authorized (i.e. if it has an expected identity).

Indeed, the determination of the identity of the person who sits can be seen as a classification of said candidate vector which represents it among a plurality of possible classes, each identity being a possible class among a plurality of classes (N classes) possible corresponding to various possible identities. Alternatively, said classification model can be binary and simply return a boolean indicating whether said candidate vector is of the expected class or classes (i.e. corresponds to the authorized user) for authentication. Here there is not strictly speaking a determination of the identity of the user, just a verification that it corresponds to that expected.

Thus it is possible, as explained, to have said learning base comprising a set of reference vectors of time sequences, each associated with a user among a plurality of possible users of said terminal, said base being for example stored by a memory 22b of the second server 2a. Note that there may be several reference vectors of temporal sequences for the same user, which is common to improve the robustness of a machine learning algorithm, but in the case in point one can in particular have for a same user several reference vectors corresponding in particular to the user seated in different places of the seat 3. In other words, if the seat 3 has several places, said learning base comprises for each user of said plurality of users at the at least one reference vector of time sequences for each seat of said seat 3 (i.e. there is at least one reference vector for each possible seat/user pair). For example, for a three-seater sofa, one can have for each user three reference vectors corresponding respectively to the left seat, the middle seat and the right seat.

It is thus possible to have a prior step (a0) of learning (or training) of said classification model on the learning base, in particular implemented by the processing module 21b of the second server 2b, the model then being loaded on the first server 2a. It is recalled that the second server 2b can be confused with the first server 2a. We can use 70% of the data for learning, 30% of the data for testing (and fine-tuning the model).

The classification model may conform to any known machine learning model, and in particular learned using any suitable algorithm.

Particularly preferably, said classification model is a neural network, in particular of the recurrent network type with short and long-term memory, LSTM (Long-Short Term Memory).

This network is very effective for the analysis of temporal sequences as is the case here, because of their "recurrent" character, i.e. we have a "hidden" internal state maintained from temporal iteration to temporal iteration.

Each candidate/reference vector can be seen as a 2D object, and the LDTM can even take as input 3D objects as multiple stacked vectors (which can be classified simultaneously).

In the case of a neural network, learning concerns in particular the parameters of the neural network.

Finally, the method advantageously comprises a step (c) of controlling said terminal 1 (by the first server 2a) according to the result of the biometric classification (in particular authentication/identification) of said user.

As explained, authentication means verifying that the user is who they say they are. The terminal 1 can for example be locked, and have an assigned user: if it is indeed the one determined in step (b) then step (c) includes the control of the terminal 1 so as to unlock it (passage from locked state to unlocked state). Otherwise, the authentication is rejected, and the terminal 1 remains in the locked state.

Identification means determining who the user is among a plurality of possible users, each potentially associated with usage preferences. Step (c) then comprises the control of the terminal 1 in accordance with the preferences of the identified user. For example, we can have a user sitting on the sofa (seat 3) who turns on the television (terminal 1). The control of step (c) notably comprises the automatic selection of a television channel preferred by the identified user.

Server

According to a second aspect, the invention relates to a first server 2a for implementing the method according to the first aspect.

Thus, this first server 2a comprises, as explained, at least one data processing module 21a and a memory 22a. It is typically a gateway (for internet access) or a home automation system.

The data processing module 21a is configured to implement steps of:

1. Obtaining a candidate vector of time sequences of weight values measured when the user sits on a seat 3 associated with said terminal 1 and equipped with a plurality of weight sensors 4a, 4b, 4c, 4d;
2. biometric classification of said user as a function of said candidate vector of temporal sequences of weight values, by means of a classification model trained on a basis of learning of reference vectors of temporal sequence of weight values; and advantageously
3. control of said terminal 1 according to the result of the biometric classification of said user.

According to a third aspect, the invention proposes a system comprising said first server 2a, as well as at least one terminal 1 connected (via the local network 10). Advantageously, said system also comprises the seat 3 associated with the said terminal 1 and equipped with a plurality of weight sensors 4a, 4b, 4c, 4d and/or the second server 2b, connected to the first server 2a via the extended network 20.

The second server 2b comprises a data processing module 21b configured to implement the learning of said classification model from said learning base of time sequence reference vectors of weight values.

computer program product

According to a fourth and a fifth aspect, the invention relates to a computer program product comprising code instructions for the execution (on the data processing module 11 of the terminal 1) of a method according to the first classification aspect biometric of a user of a terminal 1, as well as storage means readable by computer equipment (for example the data storage module 12 of the terminal 1) on which this computer program product is found.

Claims (15)

Method for the biometric classification of a user of a terminal (1) connected to a first server (2a), the method being characterized in that it comprises the implementation by a data processing module (21a) of the first server (2a), steps of:

1. Obtaining a candidate vector of time sequences of weight values measured when the user sits on a seat (3) associated with said terminal (1) and equipped with a plurality of weight sensors (4a, 4b, 4c, 4d);
2. biometrically classifying said user as a function of said candidate vector of time sequences of weight values, by means of a classification model trained on a basis of learning of reference vectors of time sequence of weight values.

Method according to claim 1, further comprising a step (c) of controlling said terminal (1) according to the result of the biometric classification of said user. Method according to one of Claims 1 and 2, in which the said classification model is a neural network. A method according to claim 3, wherein said neural network is of the recurrent short and long term memory, LSTM, type. Method according to one of Claims 1 to 4, in which each reference vector of time sequences of the said learning base is associated with a user among a plurality of possible users of the said terminal (1). Method according to claim 5, in which the seat (3) has several places, said learning base comprising for each user of said plurality of possible users of said terminal (1) at least one reference vector of time sequences for each place said seat (3). Method according to one of Claims 1 to 5, comprising a preliminary step (a0) of learning by a data processing module (21b) of a second server (2b) of said classification model from said database learning reference vectors of temporal sequences of weight values. Method according to one of Claims 1 to 7, in which step (a) comprises the prior detection that a person is sitting on the seat (3), each time sequence of said candidate vector comprising a predetermined number of successive values of weights measured by one of said weight sensors (4a, 4b, 4c, 4d) from said detection. Method according to claim 8, in which step (a) comprises continuously obtaining successive weight values measured at regular intervals by each weight sensor (4a, 4b, 4c, 4d), Method according to one of Claims 8 and 9, in which the prior detection that a person is sitting on the seat (3) comprises the calculation of a dimension Z in a sliding window on the said time sequences of weight values, and the comparison of the Z dimension with a predefined threshold. Method according to one of Claims 1 to 10, in which the said biometric classification is an identification or authentication of the user. Server (2a) for the biometric classification of a user of a terminal (1) connected to said server (2a), characterized in that it comprises a data processing module (21a) configured to implement steps of:

1. Obtaining a candidate vector of time sequences of weight values measured when the user sits on a seat (3) associated with said terminal (1) and equipped with a plurality of weight sensors (4a, 4b, 4c, 4d);
2. biometrically classifying said user as a function of said candidate vector of time sequences of weight values, by means of a classification model trained on a basis of learning of reference vectors of time sequence of weight values.

System comprising a first server (2a) according to claim 11, at least one terminal (1) and at least one seat (3) associated with said terminal (1) and equipped with a plurality of weight sensors (4a, 4b, 4c , 4d). Computer program product comprising code instructions for the execution of a method according to one of Claims 1 to 11 for the biometric classification of a user of a terminal (1) connected to a first server (2a), when said program is executed by a computer. Storage medium readable by computer equipment on which a computer program product comprises code instructions for the execution of a method according to one of Claims 1 to 11 for the biometric classification of a user of a terminal ( 1) connected to a first server (2a).